I am using http://python-rq.org/ to queue and execute tasks on Heroku worker dynos. These are long-running tasks and occasionally I need to cancel them in mid-execution. How do I do that from Pytho...

I'll start using django-rq in my project. Django integration with RQ, a Redis based Python queuing library. What is the best practice of testing django apps which is using RQ? For example, if...

I'm using RQ, and I have a failed queue with thousands of items, and another test queue I created a while back for testing which is now empty and unused. I'm wondering how to remove all jobs from t...

I'm trying to enqueue a basic job in redis using python-rq, But it throws this error "ValueError: Functions from the main module cannot be processed by workers" Here is my program: import re...

We have a distributed architecture based on rabbitMQ and Celery. We can launch in parallel multiple tasks without any issue. The scalability is good. Now we need to control the task remotely: PAUSE...

I followed the excellent Flask Mega Tutorial by Miguel Grinberg and have successfully setup a Flask web app with a Redis task queue and RQ workers, all in Docker containers. To improve task queue ...

I have a nested job structure in my python redis queue. First the rncopy job is executed. Once this is finished the 3 dependant registration jobs follow. When the computation of all these 3 jobs is...

I am trying to change the timeout on an rq job but nothing seems to work. I've got something to the effect of: my_queue = Queue('my_task', connection=Redis()) job_args = (1, 2, 4) my_queue.enque...

While reading through the rq docs, I notice that there are some arguments that you can pass to rq worker when starting the worker Example: rq worker --worker-class 'foo.bar.MyWorker' Argument l...

We have recently forced to replace celery with RQ as it is simpler and celery was giving us too many problems. Now, we are not able to find a way to create multiple queues dynamically because we ne...

How do I pass the result of a job to a job that depends on it? What I currently do is passing id of the first job to the second, first = queue.enqueue(firstJob) second = queue.enqueue(secondJob, ...

I want to queue my ml predictions using rq. Example code (pesudo-ish): predict.py: import tensorflow as tf def predict_stuff(foo): model = tf.load_model() result = model.predict(foo) return r...

I'm trying to run the rq info command, but I want to get info on a remote redis machine. how do I specify the url of the redis machine?

In a similar vein to this question, is there any way to submit a function defined in the same file to python-rq? @GG_Python who asked me to create a new question for this. Usage example: # somem...

I'm using python-rq to manage Redis-based jobs and I want to determine which jobs are currently being processed by my workers. python-rq offers a get_current_job function to find 'the current job'...

Due to having trouble with the reliability and scheduling in celery we decided to evaluate alternatives. I have been struggling setup a benchmark between the two message queue solutions with regard...

I have now a big number of documents to process and am using Python RQ to parallelize the task. I would like a pipeline of work to be done as different operations is performed on each document. Fo...

I am writing a flask app that asks user to upload excel spreadsheet and then calculate and populate the database.I am trying to do the processing part in the background via Redis RQ but I keep gett...

Currently I'm working on python project that requires implement some background jobs (mostly for email sending and heavily database updates). I use Redis for task broker. So in this point I h...

I am trying python-rq and I do not see how to explicitely give priorities to the queues? Does the priority come from the order they are defined when the worker is launched? rqworker queueA queueB...

Trying to find a good way to catch a timeout of an RQ job, so it can be requeued after the timeout. Basically, the correct solution would provide a way (for example, an exception handler in the wo...

I'm using the django-redis and django_rq frameworks to support both redis caching and redis background task processing for my Django application on Heroku. It's worked smoothly in the past, however...

Trying to use python-rq to support the back end to our Web application, but pushing new jobs takes very long - up to 12 seconds. The performance hit happens when executing the enqueue_call functio...