Developing OpenShift Node apps with local Grunt

The last episode didn't really have an ending. I decided that perhaps it wasn't the right approach. While I enjoyed what Yeoman angular-fullstack offered, it may be a bit too opinionated/confined. I still think that OpenShift is a platform worth playing with, and my new aim is to find a way to make local development (perhaps with livereload) play nicely with OpenShift. So my new approach is to start with OpenShift, and gradually load other parts in. Ideally, I'd like this to work from my Windows machine without having to run a local OpenShift Origin.

If you want to follow along, you'll need to create an OpenShift account.

I started by creating a new node.js app on OpenShift Online, which I called nodule. OpenShift gives me the command line to clone the app locally:

> git clone ssh://5350ba6be0b8cd0c52000024@nodule-yesberg.rhcloud.com/~/git/nodule.git/
> cd nodule/

The app consists of  5 files and an empty directory.

nodule> ls -l
total 47
-rw-rw-rw-   1 user     group         178 Apr 18 15:51 README.md
-rw-rw-rw-   1 user     group         457 Apr 18 15:51 deplist.txt
-rw-rw-rw-   1 user     group       39855 Apr 18 15:51 index.html
drwxrwxrwx   1 user     group           0 Apr 18 15:51 node_modules
-rw-rw-rw-   1 user     group         701 Apr 18 15:51 package.json
-rw-rw-rw-   1 user     group        4790 Apr 18 15:51 server.js

I want to be able to develop this code locally, so I need to be able to run it. I tried node server.js, but it gave an error at (ironically) "throw err;". Time to delve a little more.

The README.md file points to the OpenShift documentation for the nodejs cartridge. The deplist.txt file contains a message noting that it's deprecated and that dependencies should be described in package.json. The package.json file shows that the app has a single dependency, express.js 3.4.4. To install that, I used npm.

nodule> npm install

A screen full of npm http GET commands rolled past, and there is now an express directory inside node_modules, and another dozen inside express\node_modules. Now I can successfully run the app:

nodule> node server.js
No OPENSHIFT_NODEJS_IP var, using 127.0.0.1
Warning: express.createServer() is deprecated, express applications no longer inherit from http.Server, please use:
  var express = require("express");
  var app = express();
Fri Apr 18 2014 16:01:40 GMT+1000 (E. Australia Standard Time): Node server started on 127.0.0.1:8080 ...

I pointed Chrome to http://127.0.0.1:8080 and saw the familiar OpenShift app boilerplate index.html. That's a good start.

Before I make a change to index.html and commit and push, I want to check out what the local file system is like on the server. I would like to be able to avoid committing the node_modules, so I want to understand how that works. After using ssh to connect, and tree to see the file system hierarchy, it seems that the server has a bunch of node_modules available (async, connect, express, formidable, generic_pool, mime, mkdirp, mongodb, mysql, node-static, pg, and qs) at /dependencies/nodejs/node_modules.

So the next step is to make a small change to index.html, and to see if I can see that in the browser. Refresh. Refresh. Change isn't appearing. Stop the node server and restart - works. Well it's good to see, but it's not satisfactory for a development environment. Can't wait to get the livereload going! But ideally it will be part of the development environment only, and not the OpenShift one. It would be nice to have a staging/testing server in the cloud, and it might not matter if such a server had dev-dependencies loaded. But I want to make sure that I can configure a production-shaped system there.

Well perhaps it's best to exercise the commit/push process once before playing with connect-reload. I'm pretty new at git...

nodule>git status
# On branch master
# Changed but not updated:
#   (use "git add ..." to update what will be committed)
#   (use "git checkout -- ..." to discard changes in working directory)
#
#       modified:   index.html
#
# Untracked files:
#   (use "git add ..." to include in what will be committed)
#
#       node_modules/.bin/
#       node_modules/express/
no changes added to commit (use "git add" and/or "git commit -a")
nodule>git add index.html
nodule>git commit -m "Modify index"
[master c25d1a4] 
Modify index 1 files changed, 
263 insertions(+), 270 deletions(-) 
rewrite index.html (83%)
nodule>git push
Counting objects: 5, done.
Delta compression using up to 8 threads.
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 313 bytes, done.
Total 3 (delta 2), reused 0 (delta 0)
remote: Stopping NodeJS cartridge
remote: Fri Apr 18 2014 03:55:55 GMT-0400 (EDT): Stopping application 'nodule' ...
remote: Fri Apr 18 2014 03:55:55 GMT-0400 (EDT): Stopped Node application 'nodule'
remote: Saving away previously installed Node modules
remote: Building git ref 'master', commit c25d1a4
remote: Building NodeJS cartridge
remote: npm info it worked if it ends with ok
remote: npm info using npm@1.2.17
remote: npm info using node@v0.10.5
remote: npm info preinstall OpenShift-Sample-App@1.0.0
remote: npm info trying registry request attempt 1 at 03:56:01
remote: npm http GET https://registry.npmjs.org/express
remote: npm http 200 https://registry.npmjs.org/express
remote: npm info retry fetch attempt 1 at 03:56:01
remote: npm http GET https://registry.npmjs.org/express/-/express-3.4.8.tgz
remote: npm http 200 https://registry.npmjs.org/express/-/express-3.4.8.tgz
remote: npm info shasum aa7a8986de07053337f4bc5ed9a6453d9cc8e2e1
remote: npm info shasum /tmp/npm-452368-ggNewZcR/1397807761356-0.1033226354047656/tmp.tgz
remote: npm info shasum b9556fdb117f47bb5a97bc61ab5af7fc2dad8928
remote: npm info shasum /var/lib/openshift/5350ba6be0b8cd0c52000024/.npm/express/3.4.8/package.tgz
remote: npm info install express@3.4.8 into /var/lib/openshift/5350ba6be0b8cd0c52000024/app-root/runtime/repo
remote: npm info installOne express@3.4.8
remote: npm info /var/lib/openshift/5350ba6be0b8cd0c52000024/app-root/runtime/repo/node_modules/express unbuild
remote: npm info preinstall express@3.4.8
remote: npm info trying registry request attempt 1 at 03:56:02
remote: npm http GET https://registry.npmjs.org/connect/2.12.0

And about 10 screenfuls later,

remote: npm info ok
remote: Preparing build for deployment
remote: Deployment id is d0a8dd36
remote: Activating deployment
remote: Starting NodeJS cartridge
remote: Fri Apr 18 2014 03:56:18 GMT-0400 (EDT): Starting application 'nodule' ...
remote: -------------------------
remote: Git Post-Receive Result: success
remote: Activation status: success
remote: Deployment completed with status: success
To ssh://5350ba6be0b8cd0c52000024@nodule-yesberg.rhcloud.com/~/git/nodule.git/   
105dafe..c25d1a4  master -> master

Yes, it seems that the OpenShift Online page now shows my update --- good. When I login via ssh, it seems that the /app-root/runtime/repo/node_modules directory now has express and below that all its dependencies. I don't understand. Why did it work before with express elsewhere? Was it something I did that made node want to install express in the application itself? That's a mystery for another time, I suppose.

Now to start with the development environment. It seems that for now the essential tool is Grunt. That will run karma tests, jshint, and the live reloading that I find so neat. It took me quite a while to get the live reloading working. There are so many plugins that all seem to do similar things. I found it hard to understand how they should work together - each one seems to publish only a fraction of a gruntfile on its own readme.

I found that Romaric Pascal's tutorial at Rhumaric was the best way to get started with live reloading.

To start with, I need to install Grunt and some plugins.

> npm install grunt grunt-contrib-watch grunt-express grunt-open load-grunt-tasks --save-dev

Then I created a Gruntfile.js to get things started:

'use strict';

var path = require('path');

module.exports = function (grunt) {

  // Load grunt tasks automatically
  require('load-grunt-tasks')(grunt);

  // Define the configuration for all the tasks
  grunt.initConfig({

    express: {
      options: {
        port: 8080
      },
      devServer: {
        options: {
	  bases: path.resolve('.'),
	  livereload: true,
        }
      },
    },
    open: {
      server: {
        url: 'http://localhost:<%= express.options.port %>'
      }
    },
    watch: {
      all: {
	files: 'index.html',
	options: {
          livereload: true
        }
      }
    },

  });

  grunt.registerTask('default', [
      'express:devServer', 'open', 'watch'
  ]);
};

This file sets up three tasks (express, open, and watch), and then runs them all as the default target. It's very basic for the moment, with no karma or jshint, and only serving the index.html as a static file (rather than through the express app). I saved that file at the top level of my project and started it all up from the command line.

nodule>grunt
Running "express:devServer" (express) task

Running "express-server:devServer" (express-server) task
Web server started on port:8080, no hostname specified [pid: 10184]

Running "open:server" (open) task

Running "watch" task
Waiting...

It was nice to see that a new tab opened on my browser (Chrome) and showed the index.html page. I edited and saved the file, and magically the page in my browser updated! The console showed

>> File "index.html" changed.
Completed in 0.001s at Sat Apr 19 2014 15:25:25 GMT+1000 (E. Australia Standard Time) - Waiting...

Now, I need to git add the package.json and Gruntfile.js, git commit, and git push, and see what OpenShift makes of it all. There is heaps of line noise as OpenShift tries to install all the devDependencies, but it finally all breaks because grunt-cli isn't installed. There are other 3rd-party cartridges you could use if you wanted grunt on OpenShift, but I don't. What I want is to make OpenShift only install the production dependencies. And it seems this is now possible.

nodule>rhc env set NODE_ENV=production -a nodule
Password: *********
Setting environment variable(s) ... done

I touched the package file and added, committed, and pushed, and OpenShift deployed the system successfully.

That's all I have time for at the moment. Next steps will be to improve the Gruntfile to add some karma and jshint, and to make sure that it's running the express app, rather than just serving static html.