What is a good Sidekiq-like job system for node.js? [closed]

Question

(Most of the questions asked here about this subject are a bit old, and I was wondering what had change in the Node ecosystem it all those years.)

I'm basically looking to implement a job queue in an app of mine in node.js. I've heard about and seen Sidekiq in action in the Ruby world and how great of a job it does, and was wondering if something similar existed in node.

Workers will be written in Javascript so it doesn't have to be polyglot (it's great if it is, but definitely not a requirement).

A big plus would be an easy way to visualize the jobs currently running, those that failed etc.

What do you guys use? What lib/service do you know about that fits these requirements?

Alfaro

Kue - automattic.github.io/kue – Geraint

Bull - github.com/OptimalBits/bull – Disconcert

There's a website that list out various libraries for background jobs in various languages: taskqueues.com – Camelot

This question should have been closed back in 2016 when it was asked; it was off-topic then, just as it is now. See what's on topic and what's not... – Langouste

Answer 1

Try https://github.com/taskrabbit/node-resque.

Resque and Sidekiq share the same data structures. In node, getting multiple jobs to process at once is very simple.

Tribasic

Answer 2

Disclaimer: I develop and maintain the node.js module for Faktory.

The author of Sidekiq has created a language-agnostic job queue server backed by redis. It's called Faktory and it implements the best parts of Sidekiq (and more) on the job server side so you can get sidekiq-like job processing in any language (where a client library is available).

The node.js (and TypeScript) client can be found here.

From the README:

At a high level, Faktory is a work server. It is the repository for background jobs within your application. Jobs have a type and a set of arguments and are placed into queues for workers to fetch and execute.

You can use this server to distribute jobs to one or hundreds of machines. Jobs can be executed with any language by clients using the Faktory API to fetch a job from a queue.

Basic Features:

• Jobs are represented as JSON hashes.
• Jobs are pushed to and fetched from queues.
• Jobs are reserved with a timeout, 30 min by default.
• Jobs FAIL'd or not ACK'd within the reservation timeout are requeued.
• FAIL'd jobs trigger a retry workflow with exponential backoff.
• Contains a comprehensive Web UI for management and monitoring.

Enterprise and Pro licensing is available for the extra features that you may be familiar with in Sidekiq Pro and Sidekiq Enterprise.

I found myself on this thread because I searched "node.js sidekiq" in search of the same thing and suddenly (embarrassingly) remembered Faktory.

Totality

Answer 3

In order of decreasing Github popularity as of 2024-02:

1. Bull (though the active branch is rewritten in Typescript as BullMQ)
2. Kue
3. Agenda
4. Faktory (from the author of Sidekiq)
5. bee-queue
6. node-resque

Jibe