How to manage stuck jobs in Kue?
Asked Answered
R

0

6

I'm using Kue to create jobs for my MEAN.js app. If the application is idle for some time the Redis connection is closed, apparently Kue is trying to process jobs while the connection is closed, and I get some errors.

I'm watching for stuck jobs every 6 seconds, but this doesn't seem to help avoid the errors.

app.jobs.watchStuckJobs(1000 * 6);

These are the errors I'm getting, for each job I'm processing, after the connection is closed and before the connection is restored:

ERROR: { [Error: Redis connection to XXX failed - read ETIMEDOUT] code: 'ETIMEDOUT', errno: 'ETIMEDOUT', syscall: 'read' }

ERROR: { [AbortError: Redis connection lost and command aborted. It might have been processed.]
      code: 'UNCERTAIN_STATE',
      command: 'BLPOP',
      args: [ 'q:send-email-invitations:jobs', 0 ],
      origin: { [Error: Redis connection to XXX failed - read ETIMEDOUT] code: 'ETIMEDOUT', errno: 'ETIMEDOUT', syscall: 'read' } }

I've been reading the Kue documentation for stuck jobs, but the solution they recommend is using Domains, which for the Node version I'm using is deprecated; using promises or binding the error to uncaughtException, which will lose the error context.

What's the best approach in this case, so I don't lose the error context, and I can trace what's happening with the jobs?

If I have to choose one of these options, which is the best I can choose and why?

Is there any Redis configuration or anything outside Kue that I need to be aware of?

Recombination answered 30/8, 2016 at 16:28 Comment(1)
I am facing the same problem, at the peak time when application get load, KUE jobs stuck in queued and we have to move it via script to active state and then restart jobs again to make system smooth. I am thinking to switch some other lib or so. DO you overcome this problem??Trussell

© 2022 - 2024 — McMap. All rights reserved.