Setting a long timeout for RabbitMQ ack message
Asked Answered
S

1

14

I was wondering if this is possible. I want to pull a task from a queue and have some work that could potentially take anywhere from 3 seconds or longer (possibly) minutes before an ack is sent back to RabbitMQ notifying that the work has been completed. The work is done by a user, hence this is why the time it takes to process the job varies.

I don't want to ack the message immediately after I pop off the queue because I want the message to be requeued if no ack is received. Can anyone give me any insights into how to solve my problem?

Sepia answered 24/6, 2014 at 0:30 Comment(0)
P
11

Having a long timeout should be fine, and certainly as you say you want redelivery if something goes wrong, so you want to only ack after you finish.

The best way to achieve that, IMO, would be to have multiple consumers on the queue (i.e. multiple threads/processes consuming from the same queue). That should be fine as long as there's no particular ordering constraint on your queue contents (i.e. the way there might be if the queue were to contain contents representing Postgres data that involves FK constraints).

This tutorial on the RabbitMQ website provides more info (Python linked, but there should be similar tutorials for other languages): https://www.rabbitmq.com/tutorials/tutorial-two-python.html

Edit in response to comment from OP:

What's your heartbeat set to? If your worker doesn't acknowledge the heartbeat within the set period of time, the server will consider the connection to be dead.

Not sure which language you're using, but for Java you would use the setRequestedHeartbeat method to specify the heartbeat.

The way you implement your workers, it's vital that the heartbeat can still be sent back to the RabbitMQ server. If something blocks the client from sending the heartbeat, the server will kill the connection after the time interval expires.

Peculiarize answered 24/6, 2014 at 2:18 Comment(1)
Thanks for the tip. I can't seem to find where to set that timeout configuration on the message with RabbitMQ. Looks like if I have not ack the message after 10 seconds or so RabbitMQ places the message back into the queue to be consumed by another worker, which is not what I want. Any suggestions?Sepia

© 2022 - 2024 — McMap. All rights reserved.