puppeteer: a node.js package to simulate browsers

It can be used for web scrapping as well.

Setting up one:

npm init --yes
npm install puppeteer

Create an index.js page, with the following:

const puppeteer = require('puppeteer');

(async () => {
  const browser = await puppeteer.launch();
  const page = await browser.newPage();
  await page.goto('');
  await page.screenshot({ path: 'takingasnapshotofyourpagehere.png' });

  await browser.close();
express node.js

Express: the basics

On returning content

# automagically detects content and set the right headers:


# explicitly return json objects:


# you are more interested in sending http codes than actual return objects:


Angular.js express Heroku mongodb node.js

boostrapping a MEAN app (angular.js, express, node.js, mongodb)

Complete code example here

– in the folder you are creating your app, place a manifest file called package.json, example:

“name”: “yourappnamehere”

– install express and other dependencies:

$ sudo apt-get update

$ sudo apt-get install npm

sudo apt-get install nodejs

Note: copying and pasting these commands sometimes will give you the following error message:

TypeError: Cannot read property ‘latest’ of undefined

make sure to retype the “–” part manually, and you’ll be all right

$ sudo npm install –save express

$ sudo npm install –save body-parser

– if you want node to automatically restart when there are changes in the files, you can also install the following package:

$ npm install –global nodemon

– and then, when you start your server, you need to start it as:

$ npm install –global nodemon

– install mongodb, create the following file on your home directory: mongo_install.bash, with the following content:

apt-key adv --keyserver --recv 7F0CEB10
echo "deb dist 10gen" | tee -a /etc/apt/sources.list.d/10gen.list
apt-get -y update
apt-get -y install mongodb-10gen

– run sudo bash ./mongo_install.bash

– the installation starts mongod by default, but that is the daemon you need to start if you don’t see it running

– install mongoose:

$ npm install –save mongoose

– to enter the console mode (and verify the installation), type mongo, if you want to connect to an specific db, you do:

mongo nameofyourdbhere

> db.posts.find()

that will give you all the records saved under the Post model

– create a server.js file, that will host your app (see code in heroku instance for content details)

– run your server:

nodejs server.js

( :forwarded_port, guest: 3000, host: 3000 on Vagrantfile if you are running inside vagrant)

– server.js is kind of your single point of entry for your app. It is always a good idea to keep it lean, and move as much code as possible away from it into other files. Some things that are worth having at this file are:

— the server listening loop

— global configuration and other middleware packages

— logging and error handling

— controllers spawning and mounting

– on static files: it is a good idea not to serve them via nodejs. Try to keep your node instance as an API, and let apache and other cache services to do the static servers job. But if you must, it is always a good idea to spin them into:


and inside that file:

var express = require(‘express’)
var router = express.Router()

router.use(express.static(__dirname + ‘/../assets’))

– so now any file you put on your /assets folder will be served by node

– on services: things like $http are better constructed via a service, and then injected to wherever they are needed. Below is an example of doing just that:

app.service(‘PostsSvc’, function ($http) {

  this.fetch = function () {

    return $http.get(‘/api/posts’)


  this.create = function (post) {

    return $‘/api/posts’, post)



– and then, the controllers that consume it would looks something like this:

    // create the PostsCtrl module

    // dependency inject $scope

    app.controller(‘PostsCtrl’, function ($scope, PostsSvc) {

      // the function runs when the “Add Post” button is clicked

$scope.addPost = function () {

    if ($scope.postBody) {


        username: ‘ramiro’,

        body: $scope.postBody

      }).success(function (post) {


        $scope.postBody = null




 Deploying to Heroku

– create a .gitignore file with the following lines:


– if you are in a new vagrant instance, install heroku tools first:

$ wget -qO- | sh

$ heroku login

heroku create your-app-name-here

heroku addons:create mongolab

– check what is the address of your mongolabs instance:

$ heroku config

– modify your db.js file according to what you see printed by that command, it would look something like this:

var mongoose = require(‘mongoose’);

var url = process.env.MONGOLAB_URI || ‘mongodb://localhost/social’;


module.exports = mongoose;

– you also need to do a similar move for the listen command in the server.js file:

// process.env.PORT for the benefit of Heroku

app.listen(process.env.PORT || 3000, function () {

  console.log(‘Server listening on’, 3000)



node.js: boilerplating a site with express.js

Install express globally in your system, so you can use it from anywhere:
$npm install -g express

Initialize your installation:
$express myapp; cd myapp;

Modify your package.json file inside of there, and run it. Here’s a boilerplate example of it:

  "name": "nameofyourapphere",
  "version": "0.0.1",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo "Error: no test specified" && exit 1"
  "dependencies": {
    "express": "~3.3.5"
  "author": "",
  "license": "BSD"

$npm install // only if you modify the boilerplate generated package.json

You can also save dependencies to your installation file. For example, if you need extra packages, specify the –save option and you will have them

$npm install less-middleware –save

Create a boilerplate using some of the express flags:
$express -s -e -c less -f cli-app
-s or –sessions adds session support
-e or –ejs adds EJS1 engine support, by default jade2 is used
-J or –jshtml adds JSHTML3 engine support, by default jade is used
-H or –hogan adds hogan.js engine support
-c or –css adds stylesheet support (e.g., LESS4 or Stylus5), by
default plain CSS is used
-f or –force forces app generation on non-empty directory
The directories created with this option are:
1. public for static assets
2. views for templates
3. routes for request handlers

Go to the directory you created with the boilerplate options, and start your app:
$ cd cli-app; npm install; # you have to install again because you have new options after boilerplating
$ node app # your server should start at this point, and you should see the Welcome to Express page

Deploy to heroku
Your root directory needs a Procfile file (no extension), with the following line:
web: node app.js

That will tell Heroku what to run (app.js)

Also, if you are using mongodb, you need to have the following line in your app.js, so Heroku switch to the proper connection:
var host = process.env.MONGOHQ_URL || “mongodb://@”;

You also need to set MONGOHQ plugin on your heroku account. It is free for low traffic.

Also, you have to make your port Heroku friendly, and be able to switch to a known port when you are in your local env, by doing the following (in app.js as well):

var port = process.env.PORT || 3000;

server.listen(port, function() {

console.log(“Server listening on port ” + port + “.”);


$git init; git add .
$git commit -am “Initial commit”
$heroku create [your app name]
$git push heroku master
$heroku open

Things to keep in mind

– Variables set with “set” are global to the app now. For example:
and later, in your template, you can read it like:
title= appName // Inside a jade file

– To make jsonp work, you need to set the following:
app.set(‘jsonpcallbackname’,’cb’); // So now cb=[your cb function] will be the function used for jsonp

– etags are enabled by default:


Node.js: an app example using Yahoo geolocation service, and request library

This is a bad copy for personal use. The original series / tutorial here:

1) Make sure node and npm are installed:

$node –version

$npm –version

2) Setup a local space for your project

$mkdir myapp; cd myapp;

$npm install express

Since we are not using -g to install this globally, this express installation is local to your project only. It will be installed under node_modules


$npm install

This last command install any additional resouces listed in package.json

$npm start

If everything went fine, you should see your server running now at http://localhost:3000/

3) Install handlebars, and save any dependencies to your package.json file:

$npm install express3-handlebars

And, since we are using handlebars instead of jade, remove anything we don’t need, and bring what we do need:

$rm views/*.jade

$cp -R node_modules/express3-handlebars/examples/basic/views/* views/

4) The main application resides in app.js. Do the following modifications:

// Add this include
var exphbs = require(‘express3-handlebars’);

// app.set(‘view engine’, ‘jade’); Comment out jade

// Add in express3-handlebars.
app.engine(‘handlebars’, exphbs({defaultLayout: ‘main’}));
app.set(‘view engine’, ‘handlebars’);

//app.get(‘/’, routes.index); Get rid of routes
//app.get(‘/users’, user.list);

And start the app again, to see if everything still working:

$node app.js

To modify the home view, the file to edit is:


5) Install bootstrap (optional)

$curl -C – -O


$cp -R bootstrap/css/* public/stylesheets/

$rm -rf bootstrap

Modify your main layout file, views/layouts/main.handlebars, and add the following references:

<link rel=’stylesheet’ href=’/stylesheets/bootstrap.min.css’/>
<link rel=’stylesheet’ href=’/stylesheets/bootstrap-responsive.min.css’/>
<link rel=’stylesheet’ href=’/stylesheets/style.css’/>

5) Install library so you are able to make external requests

$npm install request –save

Looks like that is the equivalent of curl in PHP. Here’s an example of usage:

request(url, function(error, response, contentBody) {
        // Attempt to build the interpoloated address, or fail.
        var address;
        try {
            address = JSON.parse(contentBody).query.results.Result;
            address = Array.isArray(address) ? address[0] : address;
            address = address.line1 + " " + address.line2;
        catch(e) {
            callback("Could not retrieve the location at "+lat+", "+lon);

        if (error || response.statusCode != 200) {
            callback("Error contacting the reverse geocoding service.");
        else {
            callback(null, address);



node.js: API

global — The equivalent of window in the browser.

process.nextTick(cbfunction); // Schedule a function callback for the next time the loop goes around

console.log and console.error // Same as in firebug




node.js: how it is different from other web platforms

Node has a single thread, looping on itself to handle requests. Unlike PHP and other platforms, where each request spawn its own thread.

This means: if a request modify a variable with global scope, the modified variable will be served in the next request.

Node process in a nutshell:

loop taking request (running in infinite recursion) > request inn > asynchronous event spawn to handle the request > when the process the event spawn is done, it calls back the loop > request is returned

Because of that architecture, the things you need to avoid are:

1) Blocking I/O processes. Try to trigger events instead.

2) Unhandled error exceptions. Since you are sharing the thread with all the other requests, it can really make your app unstable.

In general, you are going to choose node.js because your app needs to handle lots of I/O concurrent events. If each (or some of those events) take a long time (more than 500ms) to handle, what we call CPU intensive or blocking I/O operations, you may be better off with other more traditional languages.


Node.js: the basics

Install node.js (no need to be verbose, see website for more details)

$ node -v // Verify installation

Install the NPM (node package manager, this will allow you to install and manage node packages):
$ curl | sh

$ npm –version // Verify version and installation of package manager

Creating your application and installing packages:

$ mkdir my-project/

$ cd my-project/

$ npm init --yes

$ npm install colors

Now you will see inside your directory the node_modules folder, and inside of it, the package “color” you have recently installed.

To use the newly installed package:

$ vi index.js

And then, inside index.js, type:

require(‘colors’); // Will include the package 'colors', and now you can use it as in the line below

console.log(‘smashing node’.rainbow);

To manage / import modules into your own project, you will need a package.json file in the root directory of it, something like:


    “name”: “my-colors-project”,

    “version”: “0.0.1”, // Notice how first versions in node start at 0.0.1,

    “dependencies”: {

      “colors”: “0.5.0” // The modules you need to export to make your module



Modules installed this way end up inside your ./node_modules/ file, but you can also have relative modules, not installed inside node_modules, you just have to specify the local path to them.

Once that is setup, run:

$ npm install // Will fetch dependencies for you

$ npm publish // To make your package available for others to install.

Installing binary utilities is a bit different. For example, to install the express framework:

$ npm install -g express

And then, to create a site using express:

$ mkdir my-site

$ cd mysite

$ express

To search for available packages:

$npm search realtime // Search for the realtime package

To get more information about the package you just discover:

$ npm view realtime


An alternative way to do this:

$sudo npm install -g express@2.5.4 (or whatever version you need)

$expression your_app_name  // Expression is the framework

$cd your_app_name && npm install // That will install dependencies specified in the package.json manager. You can modify that file if you need more dependencies

$node app // Start your app at port 3000

After this, everything happens inside app.js, that is the file that will serve your app