RabbitMQ PRECONDITION_FAILED - unknown delivery tag

Question

We have a PHP app that forwards messages from RabbitMQ to connected devices down a WebSocket connection (PHP AMQP pecl extension v1.7.1 & RabbitMQ 3.6.6).

Messages are consumed from an array of queues (1 per websocket connection), and are acknowledged by the consumer when we receive confirmation over the websocket that the message has been received (so we can requeue messages that are not delivered in an acceptable timeframe). This is done in a non-blocking fashion.

99% of the time, this works perfectly, but very occasionally we receive an error "RabbitMQ PRECONDITION_FAILED - unknown delivery tag ". This closes the channel. In my understanding, this exception is a result of one of the following conditions:

1. The message has already been acked or rejected.
2. An ack is attempted over a channel the message was not delivered on.
3. An ack is attempted after the message timeout (ttl) has expired.

We have implemented protections for each of the above cases but yet the problem continues.

I realise there are number of implementation details that could impact this, but at a conceptual level, are there any other failure cases that we have not considered and should be handling? or is there a better way of achieving the functionality described above?

Tyrannicide

Answer 1

"PRECONDITION_FAILED - unknown delivery tag" usually happens because of double ack-ing, ack-ing on wrong channels or ack-ing messages that should not be ack-ed.

So in same case you are tying to execute basic.ack two times or basic.ack using another channel

Conrado

As answered by @DenisKolodin, I think first we should check if the auto_ack is set to False. If it is true, we get the same error on acknowledging manually. – Shoon

Thanks, my case is ack-ing messages that should not be ack-ed – Unformed

Answer 2

(Solution below)

Quoting Jan Grzegorowski from his blog:

If you are struggling with the 406 error message which is included in title of this post you may be interested in reading the whole story.

Problem

I was using amqplib for conneting NodeJS based messages processor with RabbitMQ broker. Everything seems to be working fine, but from time to time 406 (PRECONDINTION-FAILED) message shows up in the log:

"Error: Channel closed by server: 406 (PRECONDITION-FAILED) with message "PRECONDITION_FAILED - unknown delivery tag 1"

Solution <--

Keeping things simple:

• You have to ACK messages in same order as they arrive to your system
• You can't ACK messages on a different channel than that they arrive on If you break any of these rules you will face 406 (PRECONDITION-FAILED) error message.

Original answer

Carty

Answer 3

It can happen if you set no-ack option of a Consumer to true that means you souldn't call ack function manually:

https://www.rabbitmq.com/amqp-0-9-1-reference.html#basic.consume.no-ack

The solution: set no-ack flag to false.

Squeeze

Answer 4

If you aknowledge twice the same message you can have this error.

Sevenfold

Answer 5

A variation of what they said above about acking it twice:
there is an "obscure" situation where you are acking a message more than once, which is when you ack a message with multiple parameter set to true, which means all previous messages to the one you are trying to ack, will be acked too.
And so if you try to ack one of the messages that were "auto acked" by setting multiple to true then you would be trying to "ack" it multiple times and so the error, confusing but hope you understand it after a few reads.

Staggard

Tried to edit to make your answer clearer. hit the "edit queue is full" problem. try dviding it into paragraphs with the lines in following comments – Clinkstone

A variation of what they said above about ack'ing it twice: – Clinkstone

there is an "obscure" situation where you are ack'ing a message more than once, which is when you ack a message with multiple parameters set to true, which means all previous messages to the one you are trying to ack, will be ack'ed too. – Clinkstone

and so if you try to ack one of the messages that were "auto ack'ed" by setting multiple to true, then you would be trying to "ack" it multiple times and so the error – Clinkstone

ok, improved it a little. Thanks – Staggard

Answer 6

Make sure you have the correct application.properties:

If you use the RabbitTemplate without any channel configuration, use "simple":

spring.rabbitmq.listener.simple.acknowledge-mode=manual

In this case, if you use "direct" instead of "simple", you will get the same error message. Another one looks like this:

spring.rabbitmq.listener.direct.acknowledge-mode=manual

Derisible

Answer 7

I simply forgot to exit my callback method after NACKing:

if (!hasMatchingSubscriptions(context)) {
    channel.nack(mqMsg, false, false);
    // return; <-- forgot to exit
}
channel.ack(mqMsg);

Which resulted in both a NACK and ACK.

Limey