Node.js memory usage issue

[viddeshshinde]

I have created https server using https module. When I hit the server with the requests and run the 'top' command, I can see the memory usage goes on increasing with the subsequent requests. After the server becomes idle the memory usage does't go down, it remains constant as maximum used. If I hit another bunch of transactions again it goes on increasing and stays at same size.
Is this a normal behaviour of Node.js or there is a memory leak issue in my code?

[ChALkeR]

I would recommend trying io.js 2.x:
nodejs/node#1522
nodejs/node#1529

Aside from that tls bug that was fixed in io.js, the memory (RSS, not heapUsed) not going down is pretty normal for all apps, it is kept by the process and should be re-used instead.

[viddeshshinde]

Thanks. I have tried a simple http server example as -

var http = require("http");

http.createServer(function(req,res){
res.end('hell world');
}).listen(9999);

Even this code if pumped with transactions the memory usage goes on increasing. I understand that app
uses memory and keep it for further use, so don't release it. If we hit requests again then it should be reused, but it keeps on increasing and increasing.

Is there any way to know whether the application is reusing the memory?
What is highest memory limit node.js uses after which memory is released?

[ChALkeR]

What are your exact rss/heapUsed values?

I just tested your code sample (using recent io.js, but it should be alike in node 0.12 for such a simple testcase) and performed 100000 requests (10 parallel at each moment).
RSS stabilized just below 70 MiB (was 21 MiB at start), heapUsed goes between 6 MiB and 30 MiB back and forth (well, that's how GC works).
Memory usage did not go above 70 MiB, it stopped growing at that point.

There are at least two moments that you should keep in mind here:

1. GC does not start collecting garbage instantly after each single line, it watches your memory usage, or else it would have been utterly slow. Once it assumes that it's worth performing a gc run, it collects your garbage. For me, that results in the numbers I mentioned above.
2. RSS in a general case does not go back to the system. While your heapUsage could be (and should be) dropped by the garbage collector, your total used memory (aka rss until you swap things) is held by the process and is reserved for the future allocations. It's not the gc, it's not v8, it's not even node. The same it true for plain c++ apps. That's how glibc works if the memory was allocated in small enough chunks. It could decrease a bit, but since the

[viddeshshinde]

Thanks. I am much clear about it.

I have tested my code with hitting 1000 requests and concurrency of 10 it works fine but if I increase concurrency to 1000 I see there is a memory leak. I have used memwatch module for tracing it.

Output:
start: Fri Jun 12 2015 19:59:08 GMT+0530 (IST),
end: Fri Jun 12 2015 19:59:43 GMT+0530 (IST),
growth: 4127384,
reason: 'heap growth over 5 consecutive GCs (35s) - 404.86 mb/hr' }

1. Why does memory leak occurs only when concurrency of requests is increased?
2. How could I know exact parts of the code which causes the leak?
3. Can you explain in short the output pasted above of the memwatch when found a leak?

[ChALkeR]

What's your absolute rss/heapTotal/heapUsed values after those requests?

That thing that you are using does not report memory leaks, it reports memory usage growth. It's quite obvious that 1000 parallel requests will increase your memory usage a bit.

[viddeshshinde]

Hello,

Following are the values before hitting the 1000 concurrent requests and after all the requests are served:

start
mem Usage : { rss: 77012992, heapTotal: 32171264, heapUsed: 11939824 }

end
mem Usage : { rss: 252735488, heapTotal: 72670976, heapUsed: 31140224 }

Can you then suggest me how to make sure the code is not leaking out anywhere?

[ChALkeR]

Is that https or http?

[viddeshshinde]

Its https.

[ChALkeR]

@viddeshshinde The above testcase was http-based.
Please share your https-based testcase and node/io.js version.
If you are running anything lower than iojs 2.0.0, your code is probably suffering from nodejs/node#1522
The work-around would be to enable keep-alive (if the requests are from the same host, of course).

[viddeshshinde]

The values pasted above are https-based testcase. I am using node v0.10.33.

[jasnell]

Closing as there does not appear to be anything to do here.