Node.js: managing child processes

Running shell command

There are two methods for running child processes under Node.js - exec and spawn. Here is an example of using the first one:

var exec = require('child_process').exec;
exec('node -v', function(error, stdout, stderr) {
    console.log('stdout: ' + stdout);
    console.log('stderr: ' + stderr);
    if (error !== null) {
        console.log('exec error: ' + error);
    }
});

We are passing our command node -v as a first argument and expect to get response in the provided callback. stdout is what we normally need. That's the output of the command. error should be null if everything is ok. Let's see what will happen if we run the following failing script:

// failing.js
console.log('Hello!');
throw new Error('Ops!')
console.log('End!');

// script.js
var exec = require('child_process').exec;
exec('node ./failing.js', function(error, stdout, stderr) {
    console.log('stdout: ', stdout);
    console.log('stderr: ', stderr);
    if (error !== null) {
        console.log('exec error: ', error);
    }
});

The result is:

exec error:  { [Error: Command failed: failing.js:2
throw new Error('Ops!')
      ^
Error: Ops!
    at Object. (failing.js:2:7)
    at Module._compile (module.js:456:26)
    at Object.Module._extensions..js (module.js:474:10)
    at Module.load (module.js:356:32)
    at Function.Module._load (module.js:312:12)
    at Function.Module.runMain (module.js:497:10)
    at startup (node.js:119:16)
    at node.js:902:3
] killed: false, code: 8, signal: null }

stderr contains the text of the error and error is an instance of Error class. So we may use error.code or error.signal to get more information about the problem.

The above exec snippet works for short-time commands. Processes whose job is to return a response immediately. However, I'm planning to use Yez! for much complex tasks like starting Gulp or Grunt tasks and monitoring their output. Or running a Node.js server and watching its logs. For example:

// server.js
var http = require('http');
var visits = 0;
http.createServer(function (req, res) {
    res.writeHead(200, {'Content-Type': 'text/plain'});
    visits += 1;
    var msg = 'Visits: ' + visits;
    res.end(msg + '\\n'); console.log(msg);
    if(visits == 5) {
        process.exit();
    }
}).listen(1337, '127.0.0.1');
console.log('Server running at http://127.0.0.1:1337/');

// script.js
var exec = require('child_process').exec;
exec('node ./server.js', function(error, stdout, stderr) {
    console.log('stdout: ', stdout);
    console.log('stderr: ', stderr);
    if (error !== null) {
        console.log('exec error: ', error);
    }
});

We run node ./script.js which internally starts our server. It listens on localhost, port 1337. In order to stop the server we use a counter visits. We should request http://127.0.0.1:1337 several times and see the following response in the terminal:

stdout:  Server running at http://127.0.0.1:1337/
Visits: 1
Visits: 2
Visits: 3
Visits: 4
stderr:

The problem is that we get the result when the process exits. The sentence saying that the server is running is sent to stdout stream, but because we have only one callback we get the result at the end. It's the same with the console.log(msg) messages. It won't work for my use case.

So, the better approach here will be to attach listeners to the stdout and stderr. There is also a close event available. Let's change our script.js file to the following code, that illustrates the idea:

var exec = require('child_process').exec;
var child = exec('node ./commands/server.js');
child.stdout.on('data', function(data) {
    console.log('stdout: ' + data);
});
child.stderr.on('data', function(data) {
    console.log('stdout: ' + data);
});
child.on('close', function(code) {
    console.log('closing code: ' + code);
});

Now, it is much better because we get instant updates on what is going on. The output is:

stdout: Server running at http://127.0.0.1:1337/
stdout: Visits: 1
stdout: Visits: 2
stdout: Visits: 3
stdout: Visits: 4
stdout: Visits: 5
closing code: 0

There is an alternative of exec called spawn. Unlike exec it launches a new process and should be used if we expect long-time communication with the running command. The API is a little bit different, but both methods work similar.

child_process.exec(command, [options], callback)
child_process.spawn(command, [args], [options])

spawn doesn't accept a callback and if we need to pass arguments along with the command name we have to do this in an additional [args] array.

Killing/Stopping the command

Okey, I didn't say anything new. There are tons of article about how to run a shell command from a Node.js script. However, there are just few resources explaining how to stop the running process. In the beginning I used child.kill and it worked with the simple scripts which I tested. The problem occurred when I launched grunt command. The Chrome extension nicely showed the results of the tasks but at some point I wanted to restart the process. And it looks like (even after the usage of child.kill) I didn't get the close event firing. So I investigated a bit and saw that the child fired the exit event, which for me means that the job is somehow 50% done. What was happening was that Grunt launches a bunch of other processes that were still active. In such situations, I normally start searching for a ready-to-use solution, and I found one. It was in the forever-monitor module:

var psTree = require('ps-tree');

var kill = function (pid, signal, callback) {
    signal   = signal || 'SIGKILL';
    callback = callback || function () {};
    var killTree = true;
    if(killTree) {
        psTree(pid, function (err, children) {
            [pid].concat(
                children.map(function (p) {
                    return p.PID;
                })
            ).forEach(function (tpid) {
                try { process.kill(tpid, signal) }
                catch (ex) { }
            });
            callback();
        });
    } else {
        try { process.kill(pid, signal) }
        catch (ex) { }
        callback();
    }
};

// ... somewhere in the code of Yez!
kill(child.pid);

We are using the PID of the created child to find out its sub processes (via the ps-tree dependency). So far so good. Everything works, but ... guess what ... only under Linux. Once I tried the solution on my Windows7 I got no results. The tasks that I run were active no matter what.

And as it normally happens, few hours later I found a workaround. It's dummy, but it works. There is a Windows command line instrument called taskkill. So, what I have to do is to detect that my Node.js module is operating on a Windows machine and send the PID to that small utility.

var isWin = /^win/.test(process.platform);
if(!isWin) {
    kill(processing.pid);
} else {
    var cp = require('child_process');
    cp.exec('taskkill /PID ' + processing.pid + ' /T /F', function (error, stdout, stderr) {
        // console.log('stdout: ' + stdout);
        // console.log('stderr: ' + stderr);
        // if(error !== null) {
        //      console.log('exec error: ' + error);
        // }
    });             
}

The /T (terminates all the sub processes) and /F (forcefully terminating) arguments are also critical and without them we can't achieve the desired results.

Summary

I was trying to do the same things like six or seven months back. The spawn process wasn't act as it acts now. Especially under Windows. Now, everything works smooth and as we saw with only one dirty hack we are able to execute shell commands. So, my dream for terminal in the Chrome's DevTools seems attainable.