Updated May 1st 2015
There are two kinds of queues in Microsoft Azure. The first is Azure Storage Queues and the second is Azure Service Bus Queues. Both are quite different.
Meet the Azure Queues
The Azure Storage Queue is capable of handling 2000 message transactions per second and each message has a maximum time to live of 7 days. Billing is calculated by storage transaction and size of the storage used by the queue.
The Azure Service Bus Queue is capable of handling 2000 message transaction per second and messages can live forever! Billing is calculated by the number of 64kb message transactions. This Queue flavor is feature rich and is especially interesting in Hybrid scenarios where part of the application is Cloud based and the other part is on-premise in your data center.
Azure Storage Queues
Saying that they both have a dramatically different API isn’t an overstatement. The Azure Storage Queue is simple and the developer experience is quite good. It uses the local Azure Storage Emulator and debugging is made quite easy. The tooling for Azure Storage Queues allows you to easily peek at the top 32 messages and if the messages are in XML or Json, you’re able to visualize their contents directly from Visual Studio Furthermore, these queues can be purged of their contents, which is especially useful during development and QA efforts.
The REST API for the Azure Storage Queue allows you to create a polling system where you manage the frequency at which the queues are checked for messages. In ideal circumstances, it has an average latency of 10 ms.
Access to Azure Storage Queues is secured by Shared Access Signatures.
Consider Using Azure Storage Queues if
- Your application must store over 80 GB of messages in a queue, where the messages have a lifetime shorter than 7 days.
- Your application wants to track progress for processing a message inside of the queue. This is useful if the worker processing a message crashes. A subsequent worker can then use that information to continue from where the prior worker left off.
- You require server side logs of all of the transactions executed against your queues.
Azure Service Bus Queues
The Azure Service Bus Queues are evolved and surrounded by many useful mechanisms that make it enterprise worthy! They are built into the Service Bus and are able to forward messages to other Queues and Topics. They have a built-in dead-letter queue and messages have a time to live that you control, hence messages don’t automatically disappear after 7 days.
Furthermore, Azure Service Bus Queues have the ability of deleting themselves after a configurable amount of idle time. This feature is very practical when you create Queues for each user, because if a user hasn’t interacted with a Queue for the past month, it automatically gets clean it up. Its also a great way to drive costs down. You shouldn’t have to pay for storage that you don’t need. These Queues are limited to a maximum of 80gb. Once you’ve reached this limit your application will start receiving exceptions.
Azure Service Bus Queues also have two modes of operation, the Queue can be set to deliver messages at-least-once or at-most-once. Both of the models have their own advantages and disadvantages. The delivery of a message of at most once isn’t great at dealing with failure of processing messages.
Access to the Service Bus is secured by Shared Access Signatures and AMQP. Local development is possible but it requires the installation of the Windows Azure Pack Service Bus. It’s consistent with the Azure Service Bus, but quite frankly this is only really viable if all your dev machines are setup using an image. To dev & test more effectively without all the hassle of maintaining identical configurations across all environments. I suggest creating a service bus namespace for each developer on your team using your dev & test MSDN Azure Subscription.
Consider Using Queues in the Azure Service Bus if
- Your solution must be able to receive messages without having to poll the queue. With Service Bus, this can be achieved through the use of the long-polling receive operation using the TCP-based protocols that Service Bus supports.
- Your solution requires the queue to provide a guaranteed first-in-first-out (FIFO) ordered delivery.
- You want a symmetric experience in Azure and on Windows Server (private cloud). For more information, see Service Bus for Windows Server.
- Your solution must be able to support automatic duplicate detection.
- You want your application to process messages as parallel long-running streams (messages are associated with a stream using the SessionId property on the message). In this model, each node in the consuming application competes for streams, as opposed to messages. When a stream is given to a consuming node, the node can examine the state of the application stream state using transactions.
- Your solution requires transactional behavior and atomicity when sending or receiving multiple messages from a queue.
- The time-to-live (TTL) characteristic of the application-specific workload can exceed the 7-day period.
- Your application handles messages that can exceed 64 KB but will not likely approach the 256 KB limit.
- You deal with a requirement to provide a role-based access model to the queues, and different rights/permissions for senders and receivers.
- Your queue size will not grow larger than 80 GB.
- You want to use the AMQP 1.0 standards-based messaging broker. For more information about AMQP, see Service Bus AMQP Overview.
- You can envision an eventual migration from queue-based point-to-point communication to a message exchange pattern that enables seamless integration of additional receivers (subscribers), each of which receives independent copies of either some or all messages sent to the queue. The latter refers to the publish/subscribe capability natively provided by Service Bus.
- Your messaging solution must be able to support the “At-Most-Once” delivery guarantee without the need for you to build the additional infrastructure components.
- You would like to be able to publish and consume batches of messages.
- You require full integration with the Windows Communication Foundation (WCF) communication stack in the .NET Framework.
Making the Final Decision
By gaining a deeper understanding of the two technologies, you will be able to make a more informed decision on which queue technology to use, and when. The decision on when to use Azure Queues or Service Bus queues clearly depends on a number of factors. These factors may depend heavily on the individual needs of your application and its architecture. If your application already uses the core capabilities of Microsoft Azure, you may prefer to choose Azure Queues, especially if you require basic communication and messaging between services or need queues that can be larger than 80 GB in size.
Because Service Bus queues provide a number of advanced features, such as sessions, transactions, duplicate detection, automatic dead-lettering, and durable publish/subscribe capabilities, they may be a preferred choice if you are building a hybrid application or if your application otherwise requires these features.
Source: Azure Queues and Service Bus queues – compared and contrasted
Find Out More
Alexandre Brisebois ☁ 