How the Sidekiq server process pulls jobs from the queue in Redis?

Question

I've two Rails application running on two different instance(lets say Server1 and Server2) but they have similar codes and shares the same Postgresql DB.

I installed Sidekiq and pushing the jobs in Queue from both the servers, but I'm running the Sidekiq process only in Server1.

I've single Redis server and its running on Server1 which shares the Redis with Server2.

If a job pushed from Server2 it getting processed in Server1's Sidekiq process and its what I actually wanted.

My question is

• How the Sidekiq process on Server1 knows that a job is pushed in Redis?

• Whether the Sidekiq process continuously checks on the Redis server for any new jobs or the Redis server is intimating to the Sidekiq process about the new job?

I got confused and amazed about this!!!

Could anyone please clarify the Sidekiq's process to get the job from Redis server?

It will be helpful for newbies like me.

Quarto

Answer 1

Sidekiq uses redis command named BRPOP.

This command gets an element from a list (which is your job queue). And if the list is empty, it waits for element to appear and then pops/returns it. This also works with multiple queues at the same time.

So no, sidekiq does not poll redis and redis does not push notifications to sidekiq.

Rotary

is there any command in rails console for sidekiq-server that can be used to get a job from the queue? – Daltondaltonism

Answer 2

Sidekiq uses a polling mechanism to check for new jobs in Redis. The default polling interval is set at 5 seconds and can be adjusted in the configuration file located at lib/sidekiq/config.rb [link]

# lib/sidekiq/config.rb
average_scheduled_poll_interval: 5

By the way, jobs are stored in Redis as a list and Sidekiq retrieves them by using the BRPOP (Blocking Right Pop) command to avoid any race conditions. This ensures that multiple Sidekiq processes running on different instances are able to retrieve the jobs in a coordinated manner.

Rosana