When does a single JMS connection with multiple producing sessions start becoming a bottleneck?

Question

I've recently read a lot about best practices with JMS, Spring (and TIBCO EMS) around connections, sessions, consumers & producers

When working within the Spring world, the prevailing wisdom seems to be

• for consuming/incoming flows - to use an AbstractMessageListenerContainer with a number of consumers/threads.
• for producing/publishing flows - to use a CachingConnectionFactory underneath a JmsTemplate to maintain a single connection to the broker and then cache sessions and producers.

For producing/publishing, this is what my (largeish) server application is now doing, where previously it was creating a new connection/session/producer for every single message it was publishing (bad!) due to use of the raw connection factory under JmsTemplate. The old behaviour would sometimes lead to 1,000s of connections being created and closed on the broker in a short period of time in high peak periods and even hitting socket/file handle limits as a result.

However, when switching to this model I am having trouble understanding what the performance limitations/considerations are with the use of a single TCP connection to the broker. I understand that the JMS provider is expected to ensure it can be used in the multi-threaded way etc - but from a practical perspective

• it's just a single TCP connection
• the JMS provider to some degree needs to co-ordinate writes down the pipe so they don't end up an interleaved jumble, even if it has some chunking in its internal protocol
• surely this involves some contention between threads/sessions using the single connection
• with certain network semantics (high latency to broker? unstable throughput?) surely a single connection will not be ideal?

On the assumption that I'm somewhat on the right track

• Am I off base here and misunderstanding how the underlying connections work and are shared by a JMS provider?
• is any contention a problem mitigated by having more connections or does it just move the contention to the broker?
• Does anyone have any practical experience of hitting such a limit they could share? Either with particular message or network throughput, or even caused by # of threads/sessions sharing a connection in parallel
• Should one be concerned in a single-connection scenario about sessions that write very large messages blocking other sessions that write small messages?

Would appreciate any thoughts or pointers to more reading on the subject or experience even with other brokers.

Apply

Answer 1

When thinking about the bottleneck, keep in mind two facts:

1. TCP is a streaming protocol, almost all JMS providers use a TCP based protocol

2. lots of the actions from TIBCO EMS client to EMS server are in the form of request/reply. For example, when you publish a message / acknowledge a receive message / commit a transactional session, what's happening under the hood is that some TCP packets are sent out from client and the server will respond with some packets as well. Because of the nature of TCP streaming, those actions have to be serialised if they are initiated from the same connection -- otherwise say if from one thread you publish a message and in the exact same time from another thread you commit a session, the packets will be mixed on the wire and there is no way server can interpret the right message from the packets. [ Note: the synchronisation is done from the EMS client library level, hence user can feel free to share one connection with multiple threads/sessions/consumers/producers ]

My own experience is multiple connections always output perform single connection. In a lossy network situation, it is definitely a must to use multiple connections. Under best network condition, with multiple connections, a single client can nearly saturate the network bandwidth between client and server.

That said, it really depends on what is your clients' performance requirement, a single connection under good network can already provides good enough performance.

Yukyukaghir

Thanks for your response. In my experience, multiple connections will usually only significantly outperform single connections in a lossy or high latency network situation or with an application layer protocol that internally involves request/wait/request/wait to process a single "message"; rather than always being the case. I guess what I'm getting it is that I've found no recommendation on Tibco side; and all the stuff I've read on general JMS (and indeed ActiveMQ) seems to say to just use a single connection; without much discussion of its limitations. – Apply

Answer 2

Even if you use one connection and 100 sessions it means finally you are using 100threads, it is same as using 10connections* 10 sessions = 100threads.

You are good until you reach your system resource limits

Guimpe