New Relic causing memory leak/bloat?

Question

I've been tearing my hair out trying to pinpoint the cause of the memory my app is using continually growing, to the point where in production on heroku it fails and goes down.

After some pretty extensive research, using Top to observe RSIZE growing locally and using Oink to try and pinpoint I just couldn't find what was wrong.

I then found a post here that refers in one of the comments to the New Relic gem causing the same problem. As a test in development I commented out the gem and now without it my memory usage seems to be pretty stable at around 100M whereas with the gem installed it grows to the point where I can push it up to almost 1GB!! Clearly this behaviour is the root of the app going down at Heroku.

Has anyone else had this with New Relic? I'd like to continue using it, I think it's a great service but I can't if it's the root of this memory leak. Any suggestions on how I can fix it?

I'm running Rails 3.2.0 & Ruby 1.9.2

When I run Gem List it shows I have newrelic_rpm (3.6.0.83, 3.5.6.46, 3.5.5.38, 3.4.2.1, 3.4.2)

My Gemfile just references gem 'newrelic_rpm' so should I be specifying a version?

Zakarias

development mode newrelic is a very different beast to production mode new relic - if you are trying to reproduce locally, it would be worth doing it with settings that are close to production – Hagiology

Good point, i'll give that a go. – Zakarias

Is the newrelic_rpm gem just for development do you know? – Zakarias

Not at all, but it does have a developer_mode that is only for development – Hagiology

That's what I thought, however, I've removed it from my production Gemfile and Gemfile.lock files and New Relic seems to still be tracking my app and things appear more stable. – Zakarias

developer_mode is intended to write all New Relic performance information to a local log and is extremely resource-heavy, unlike the default mode for the agent. More info here: newrelic.com/docs/ruby/developer-mode – Tableland

I have exactly same issue and culprit is newrelic_rpm gem. – Fauna

Answer 1

It seems like this may be resolved, but if you continue to see huge resource use for New Relic's Ruby agent you should definitely get in touch with Support right away. I haven't seen anything like this for the Ruby agent so it's definitely something we'd be curious to look at. https://support.newrelic.com/home

Zing

Hi Toby, no it's not 100% resolved. It seems like it was actually the exception_notifier gem that was causing this after a few tests with & without. – Zakarias

Ryan- our Ruby guys would definitely like to do some log examination and try to work it out. Please do open a ticket and if we come up with any general solutions I can post them back here. – Tableland

Ok I will do. The Gem is no longer installed but I hope they will be able to view the logs from when he problem occurred? – Zakarias

Hi Toby, i've just pushed an update to the app that now includes the exception notification gem. I'll monitor memory usage and open a ticke if it starts behaving badly again. – Zakarias