JavaScript heap out of memory.

This is the kind of error you will be getting when you parsed a large file (sized around 80mb and above) in Node.js, and this happens only when you are trying to load the entire file into memory at one go.

The solution to this problem is to parse the file part by part. In this article, we will be using a library called line-by-line to illustrate the concept. Processing the file line by line will not load the entire file into memory, hence you will not get any memory exception error.

With the line-by-line npm library, you can either process a file synchronously or asynchronously, depending on your requirements.

The next few code snippets will give you a better idea on how it works.

var LineByLineReader = require('line-by-line'),
    lr = new LineByLineReader('big_file.txt');

lr.on('error', function (err) {
	// 'err' contains error object
});

lr.on('line', function (line) {
	// pause emitting of lines...
	lr.pause();

	// ...do your asynchronous line processing..
	setTimeout(function () {

		// ...and continue emitting lines.
		lr.resume();
	}, 100);
});

lr.on('end', function () {
	// All lines are read, file is closed now.
});