Tibco RV: Message Lifetime + No Copy for fanouts

Question

1, How long does a message live in the listener queue? Until the dispatcher reads the message out of the queue in a "1 publisher 1 consumer" scenario?

Listener listener = new Listener(Queue.Default, transport, subject, new object());
listener.MessageReceived += OnMessageReceived;
Dispatcher dispatcher = new Dispatcher(listener.Queue);

2, Tibco RV is typically used in a large fanout system, with relatively loose requirements on delivery reliability, say, market data published to 20 applications in an enterprise. I've heard Tibco RV implements a "no copy" solution for fanouts - how is that even possible? I'd assume we need to at least go through all registered listeners for that queue and notify each of them, in which process the message is copied 20 times. Please enlighten me.

3, Combine question 1 and 2, it doesn't make sense for a message to live in the listener queue until ALL registered listeners have consumed the message - What happens if 1 of the 20 applications goes offline? It's going to bring down the rv daemon process due to ever increasing messages. Does Tibco RV have a lifetime limit(ttl) for each message? How do I check it and set it to new values?

There isn't much related info on Google, so please help.

Thanks.

Ardel

Answer 1

Great questions.

1. Keep in mind that unless you are using RV certified messaging there will be no persistence to disk. Messages sent will remain in the memory of the sending Rendezvous daemon until they are delivered to all consumers.

   That said, another thing to understand is that RV is an optimistic protocol as opposed to say, TCP which is a pessimistic protocol. Every packet is sent using TCP must be acknowledged. This round-trip protocol slows things down. Rendezvous on the other hand uses a protocol that sits on top of UDP which is session-less and does not require acks. Therefore, when a Rendezvous daemon sends a message, it is assumed to have been delivered successfully unless a retransmission request is received. So to completely answer your first question the default behavior of a Rendezvous daemon is to keep messages that it has sent in memory for 60 seconds after it has sent the message. This allows it to honor retransmission requests.

2. Fan out is achieved using broadcast or multicast protocols on top of UDP. The use of broadcast is discouraged and multicast is encouraged. Using multicast groups uses considerably less network resources. At the network interface level only those hosts that have joined the multicast group will receive packets associated with the Rendezvous traffic. Similarly at the network switch level multicast is a lot less resource-intensive.

   Bottom line is that the sending Rendezvous daemon only sends the message out once, and the network delivers a copy of the associated packets to either each host on the subnet if broadcast is used, or the hosts that have registred interest if multicast is used.

3. In pub-sub, typically consumers receive messages that are sent while they are alive and consuming. Thus with pure Rendezvous, if a consumer was to go down, the subscription would be cancelled for that consumer. If we think about your example of market data, this is exactly the behavior we want. IBM trades at thousands of times a second, so if I miss a price quote it's no big deal. I'll get the next one. Furthermore, I don't want stale prices.

   That said, sometimes consumers do want messages sent while they were down. This can be achieved using certified messaging and setting up persistent correspondents. For more on this see the Rendezvous Concepts Guide. Lastly, the 60-second behavior that I mentioned in point #1 can be changed using the -reliability parameter when starting the Rendezvous daemon. There are some cases where this may make sense (although the default is best for most common cases). For more details on this take a look at the Rendezvous Admin Guide.

Barroom

Brilliant answer, you're the man! I did some reading on udp multicast, and as you said, network nodes copying the messages is the key! Also, thanks for the RV Guide for the default 60 seconds - exactly what I was looking for. – Ardel