sidekiq does not generate sidekiq.pid file on heroku, using with Redistogo

Question

I've been struggling for two days with getting sidekiq work on heroku production environment. I've read all the available documentation about similar issues, and still haven't been able to produce a working solution, I'd really like some help!

After deploying on heroku, my app crashes and I get the following error stack trace:

2014-09-23T23:38:40.905093+00:00 app[worker.1]: No such file or directory @ rb_sysopen - /app/tmp/pids/sidekiq.pid
2014-09-23T23:38:40.905122+00:00 app[worker.1]: /app/vendor/bundle/ruby/2.1.0/bin/sidekiq:23:in `<main>'
2014-09-23T23:38:40.905119+00:00 app[worker.1]: /app/vendor/bundle/ruby/2.1.0/gems/sidekiq-3.2.5/bin/sidekiq:7:in `<top (required)>'
2014-09-23T23:38:40.905117+00:00 app[worker.1]: /app/vendor/bundle/ruby/2.1.0/gems/sidekiq-3.2.5/lib/sidekiq/cli.rb:347:in `write_pid'
2014-09-23T23:38:40.905115+00:00 app[worker.1]: /app/vendor/bundle/ruby/2.1.0/gems/sidekiq-3.2.5/lib/sidekiq/cli.rb:347:in `open'
2014-09-23T23:38:40.905121+00:00 app[worker.1]: /app/vendor/bundle/ruby/2.1.0/bin/sidekiq:23:in `load'
2014-09-23T23:38:40.905118+00:00 app[worker.1]: /app/vendor/bundle/ruby/2.1.0/gems/sidekiq-3.2.5/lib/sidekiq/cli.rb:41:in `parse'
2014-09-23T23:38:40.905114+00:00 app[worker.1]: /app/vendor/bundle/ruby/2.1.0/gems/sidekiq-3.2.5/lib/sidekiq/cli.rb:347:in `initialize'
2014-09-23T23:38:39.588001+00:00 heroku[worker.1]: State changed from starting to up

First, sidekiq is working correctly on my local machine. I'm using heroku's REDISTOGO for redis, on local production, sidekiq have been pointed to the REDISTOGO correctly and running fine.

Second, according to the stack trace, especially this line No such file or directory @ rb_sysopen - /app/tmp/pids/sidekiq.pid; it leads me to think that for some reason sidekiq.pid file is not generated correctly when running on heroku. On local environment, the sidekiq.pid file is generated everytime I start the app in the app/tmp/pids/ directory, and assigns a different pid number every time. I'm guessing that when running on heroku, sidekiq tried to read from this file yet could not find it.

Here is contents in my Procfile:

web: bundle exec rails server
worker: bundle exec sidekiq -C config/sidekiq.yml

Here is contents in my config/sidekiq.yml

---
:verbose: true
:pidfile: ./tmp/pids/sidekiq.pid
:concurrency: 25
# Set timeout to 8 on Heroku, longer if you manage your own systems.
:timeout: 8
:queues:
  - carrierwave

Here is contents in my sidekiq.rb

Sidekiq.configure_server do |config|
  config.redis = { :url => ENV['REDISTOGO_URL'], :namespace => "mynamespece"}
end

Sidekiq.configure_client do |config|
    config.redis = { :url => ENV['REDISTOGO_URL'], :namespace => "mynamespece"}
end

Update 1:

I'm using carrierwave and carrierwave-backgrounder in sync with sidekiq.

Subtractive

Why are you creating a pidfile at all? – Cathey

@MikePerham Hey Mike, I know you are the author of sidekiq, I appreciate your attention to this matter! I wasn't using pid intentionally. I'm new to sidekiq and I thought that's the default behavior, because during my implementation I didn't run into any configuration that references pid anywhere. What would you suggest, am I doing anything wrong? Maybe not using pid is the optimal way to implement sidekiq? I didn't find too much documentation on the pid part. – Subtractive

@MikePerham on my local environment, everytime I use foreman start, I see the following lines: 17:28:57 web.1 | started with pid 20788 17:28:57 worker.1 | started with pid 20789 The pid number would be different every-time. – Subtractive

@MikePerham Sorry I forgot to mention this in my original post; I was using carrierwave and carrierwave-backgrounder with sidekiq; so maybe using pid was a decision made by implementation of carrierwave – Subtractive

Just remove the pidfile line in your Sidekiq.yml, that should fix it. – Cathey

@MikePerham Thanks, removing the pidfile line in sidekiq.yml does fix the problem of No such file or directory @ rb_sysopen - /app/tmp/pids/sidekiq.pid error message; however, now when I run foreman create with bundle exec sidekiq -C config/sidekiq.yml in Procfile, in my local environment, the worker no longer pulls job from the queue. It was working previously, with the pidfile line in sidekiq.yml – Subtractive

Answer 1

This issue has been resolved by the following actions:

1) Thanks to @MikePerham for pointing me to the right direction, first I removed this line in my sidekiq.yml file:

:pidfile: ./tmp/pids/sidekiq.pid

2) Then, in my Procfile, I had to use the following line to replace the origin:

web: bundle exec rails server -p $PORT
worker: bundle exec sidekiq -C config/sidekiq.yml

Now sidekiq is working correctly with redistogo on heroku for me.

Subtractive

This worked like a freaking charm. Thank you very much! – Citriculture

Answer 2

I found it useful to add the directory to the repository and re-deploy the application.

mkdir -p tmp/pids
touch tmp/pids/.gitkeep
git add -f tmp/pids/.gitkeep
git commit -m 'Keep tmp/pids directory in repo'

Hope this helps.

Dugout

Answer 3

It was fixed by making pids directory in tmp directory in root of project

Nursemaid