Debugging Node.js Applications in Cloud Foundry

The other day some colleagues of mine and I were tasked with testing out the process of debugging a Node.js application running in Cloud Foundry.

Before I go any further into the details of how this was done, I feel compelled to share why and when it should be done. First and foremost, this is never, ever to be done in production. Dropping a breakpoint into a single-threaded application running in production with tens of thousands of active consumers is not a good idea. By not good I really mean catastrophic. Many kittens will die. Unicorns will lose their horns. The sky will fall.

Even if you don’t use a breakpoint, interacting with the debugger (especially with Node) can have adverse effects on the app — everything from degraded performance to outright crashes or hangs. Did I mention you should never do this in production?

What I’m describing in this post is how to attach a debugger to an application running in a testing environment. In production, you’ll need to rely on application monitoring tools, metrics, and log analysis for diagnosis. Cloud native applications deployed in production are like satellites in orbit — you can’t treat them as though they are reachable with a tether.

The application we intend to debug has to have the node-inspector module in it, which, when you run npm install, will add the node-inspector module in the node-modules/.bin directory. The dependency in package.json looks like this:

node-inspector”: “0.12.6”

We’re going to need this dependency later.

Now that we have an app to debug, we need to push it to Cloud Foundry with an altered start command. This start command runs the Node application with the debugger active. This exposes a debugger port inside the app’s container on port 5858.

cf push myapp -c “node --debug server.js”

​At this point, if your app started, you should see that your app is responding as it would normally, and you also have a trace message (you can see this with the cf logs CLI command) that looks like the following:

debugger listening on port 5858

Note that the port indicated in that trace message only applies inside your container. We’re going to have to do a bit of (somewhat dangerous) poking and prodding in the container to expose the debugger port outside.

Now that we’ve got our app running in debug mode, we’ll need to ssh into our container:

$ cf ssh myapp
$ export PATH=$PATH:/app/.heroku/node/bin

Here we’re adding the path to the node executable to our shell’s path so that we can invoke node scripts like node inspector. Next, run the following command from your /app/node_modules/.bin directory:

$ ./node-inspector --web-port=9090

This starts node inspector. Node inspector is essentially a web proxy. It exposes a set of JavaScript scripts via an HTML response on the designated port. These scripts are then used to fire up Chrome’s JavaScript debugger. Rumor has it there may be a simplified way of starting node inspector and the node debugger simultaneously, but I’ve always run into port conflict problems attempting it.

If node inspector starts properly, you’ll see the following message in your ssh output:

Visit http://127.0.0.1:9090/?port=5858 to start debugging.

At this point, we’ve got our node application running internally on some port (usually 8080). We’ve got a debugger responding internally on port 5858. And we also have a web proxy for the debugger (node inspector) running internally on port 9090.

To make this at all useful, we need to expose the node inspector outside the container, usually on our local workstation via localhost. This allows us to use the URL we saw from the earlier trace message in a local Chrome tab.

To do that, we’re going to use cf ssh to create a tunnel for port 9090.

cf ssh -N -T -L 9090:localhost:9090 myapp

This will sit in your terminal and keep the ssh tunnel active until you Ctrl+C out of it. So you might want to do this in a separate terminal tab just to keep things flexible.

Note that we don’t need to tunnel any of the other ports. Ports 8080 and 5858 are fine running internally in the app’s container. The Cloud Foundry router is taking care of getting regular HTTP traffic to the app (port 8080 inside), and the node inspector proxy is going to talk internally to the port 5858 debugger.

Now we’re ready to debug! (finally!)

Open http://localhost:9090/?port=5858 in Chrome to run the JavaScript scripts that talk to the Node Inspector, which is talking to the debugger inside your container. This will likely take a few minutes, and because of the nature of the scripts, Chrome might not even display a little “waiting for…” message in the bottom. Give it a few seconds, grab some coffee, and hopefully your debugger comes up.

Next hit your application using its Cloud Foundry host and domain, e.g. http://myapp.my.domain.com. Now you’re remotely debugging a Node app in Cloud Foundry.

Many thanks to Rohit Kelapure for helping me walk through these steps and overcome my base level of disdain for all things Node.js.