What is a good alternative to Kue that works with MongoDB instead of Redis?

Question

I am building a web app with node.js and mongodb. I need to add delayed jobs. For example, sending users an email one month after signing up. I haven't found an existing solution for delayed jobs besides Kue, but it uses Redis, and I would prefer to use my existing Mongodb before adding another resource to my web app.

Is there any existing solution for that?

Foreword

Answer 1

The short answer

The short answer is No. There is no port of Kue to MongoDB, nor there are any plans for such. There are no other open-source popular projects offering similar functionality at the moment.Moreover Redis seems like a better fit to this sort of project any-way.

The long answer

While Kue is very interesting and offers much more than just delayed tasks, it seems your requirements are much simple.

If all you need is to send users an email one month after signing up, or that sort of thing, it is usually built into the OS level.

What I suggest you do, assuming you have a sendEmail method (in general, that you figured out how the tasks would be done) is the following:

1. Depending on the operating system, schedule sendEmailHandler a task to run once a day. In Windows this is done with "Scheduled Tasks", in OS X, BSD and Linux this is done using cron. (cron tutorial). Most PaaS options also have this sort of option (Like nodejitsu, Azure...).
2. That tasks should be a node.js script that iterates through a list of long running tasks (that is, stuff that runs once a month, a week, or longer).
3. In MongoDB which you choose to use, hold a collection of tasks,each with a 'start time'. An example for such tasks would be a specific user getting an email after 1 month.
4. In your sendEmailHandler script, check which tasks should run, and execute them, after which you should remove them from the MongoDB collection.

While this sounds like some work, it shouldn't take too long. All the code described here is very straightforward.

Kue lets you do stuff like priority, attempts, progress, etc which you do not need if I understand correctly. Working with a library that does many things to do something simple might end up biting you since degugging and maintenance are usually harder.

Good luck! Feel free to let me know if you'd like me to elaborate more on a specific part.

Metzgar

Thanks for the explanation, but I was looking for something more scalable. Sending an email a month after sign-up was just an example. I would like to create workers who work on the job queue, send emails, process data etc. So, I am looking for something that can be distributed, and that doesn't only run on a single server – Foreword

What I suggested isn't limited to a single server, one server will triggle the job daily, whoever executes it might be as distributed as you'd like (either on the node.js level or on the MongoDB level). If you'd like a more straightforward answer then No. Kue doesn't work on MongoDB yet and there are no plans to port it. There are no projects who offer similar functionality at the moment with MongoDB and to be honest Redis seems like a much better fit than MongoDB for this sort of task. If you'd like I can edit this into my answer. – Metzgar

That's much more of the answer I was looking for. I'm wondering why there's now MongoDB solution, though. I can understand how Redis is superior when it comes to queues of tasks that should be processed as soon as possible (image resize etc), but the queue of delayed tasks can grow pretty quickly. – Foreword

@Foreword Very well. I have updated the answer to accommodate what you meant in your question. Let me know what you think – Metzgar

Answer 2

Another option is Agenda

Agenda is a light-weight job scheduling library for Node.js.

Mollymollycoddle

Does agenda have queue (LiFo)? – Northeastward

Answer 3

I was just researching the same thing and found this npm package:

monq - MongoDB-backed job queue for Node.js - https://www.npmjs.org/package/monq

Parricide

Answer 4

I was also looking for such queue library that works with MongoDB. I just found mongodb-queue.

When it is required to process high volume messages and persistence then libraries like Bull, Bee or Kue are essential. For all these libraries persistence is achieved with Redis.

Sewellel