Azure Service Bus - subscribers can independently subscribe to a subscription and share the same message?

Question

I'm new to Azure Service Bus and have created successful pocs for creating a topic and a separate subscriber application to receive its message.

Based on this article, I quote: "A topic can have up to 2,000 subscriptions associated with it, each of which gets independent copies of all messages sent to the topic. One or more subscribers can independently subscribe to a subscription and compete for messages from it."

http://convective.wordpress.com/2011/06/08/windows-azure-appfabric-service-bus-queues-and-topics/

What I'm interested to do is to expand this for multiple applications to independently subscribe to the same topic, but not compete for them.

My current poc has one sender and two separate applications subscribed to the same topic and subscription. The behavior that I'm seeing is that if I post one message from the sender, either one of the two running subscribing applications will receive it; but not the other.

My question is if multiple, independent applications can receive the same topic message? Any advice would be greatly appreciated!

Emmalineemmalyn

Answer 1

When you create a subscription to a topic it has a specific subscription name. Any consumer that then starts requesting messages for that subscription name will compete for messages on that subscription. If you want independent applications to each receive a copy of the message sent to a topic they will each have to create their own subscription to the topic. You can almost view each subscription as a queue unto itself being fed by the topic.

The example I'll give is of a college. The topic is "New Student" and each department within the college wants to receive a copy of a message for a new student. Thus each department would have it's own subscription. There would be "Music", "Billings", "Science", "Math" subscriptions, etc. Each of them subscribing to the New Student topic. In this way each department receives a copy of the new student message, or can even filter on things they care about if they want. If the department gets behind in processing these they can spin up more instances of the processor using the their subscription name, thus increasing their throughput in theory since more consumers are then competing for the messages in their subscription.

So, in your example, each application would need to either create it's own subscription or be assigned an unique subscription to start pulling from when it starts up. Note that if you let the application lifetime decide the lifetime of a subscription (meaning you create a subscription when the app starts and destroy it when the app closes) you need to be aware that if there are no active subscriptions then messages sent to the topic will simply be lost. You can however create a catch all subscription that only receives messages that are not delivered to any other subscription. It really depends on what you are trying to accomplish.

Luna

@mikewo, when you say "if there are no active subscriptions then messages sent to the topic will simply be lost" doesn't that depend on the time-to-live of the message? Even if there are no active subscriptions and a message is added to the topic, what if an application subscribes to the topic, before the message expires? Would it not receive it? Thanks for the post. It's very helpful. – Transport

@NoelAbrahams No. If there are no subscriptions and a message is submitted to the topic, the system assumes no one cares and doesn't keep the message. The expiration on the message is for when it is delivered to a queue/subscription that cares. The default time to live on a message for a queue or topic is Timespan.MaxValue, so in theory if it worked the way you are asking about then they would always be around until someone subscribed to them. – Luna

Answer 2

What you are looking for is not Service Bus but Event Hub. Event Hub is working exactly you describe and can also be accessed by means of AMQP.

Rephrase

This does not provide an answer to the question. Once you have sufficient reputation you will be able to comment on any post; instead, provide answers that don't require clarification from the asker. - From Review – Gauffer

I don't understand the reviewer's comment. I had exactly the same problem as outlined in the question and I solved it as I described in my response. I have code to prove if you are interested. Are you an expert of Service Bus or Event Hub so that you dismiss a properly researched answer so out of hand? – Stooge

The format is wrong imho, and this is likely why it was marked as a low quality post. It is either a (good) comment or a link only answer but without a link, and none of them are a fit as an answer on SO. Review is not about expertise, but upvotes and downvotes are. – Gauffer

I am sorry but you lost me completely. The format of my post? What link only answer? What upvote or downvote? I did not do any of this. I got to this question from Google and I had the same problem as the user who asked the question. It took me some time to realize that this very frequent problem is hard to solve with Service Bus but is much easier to solve with Event Hub and I shared my opinion that I actually implemented in our product. Then I kept receiving these boilerplate comments (it is not the first time) which, honestly, really puzzles me. – Stooge

The format is wrong, your answer does not demonstrate how Event Hub solves the problem, and you did not even provide a link. You simply cannot expect SO readers take your word for gospel, and that is likely why your post was not upvoted. Long story short, when you have an opinion, click on comment, and when you have facts, click on answer. These is basically how SO works, but do not take my words for gospel, and Take the tour instead, – Gauffer