Windows Azure Storage Queues VS Windows Azure Service Bus Queues
October 20, 2013 9 Comments
Meet the Windows Azure Queues
The Windows Azure Storage Queue is capable of handling 2000 message transactions per second and each message has a maximum time to live of 7 days. Billing is calculated by storage transaction and size of the storage used by the queue.
The Windows Azure Service Bus Queue is capable of handling 2000 message transaction per second and messages can live forever! Billing is calculated by the number of 64kb message transactions. This Queue flavor is feature rich and is especially interesting in Hybrid scenarios where part of the application is Cloud based and the other part is on-premise in your data center.
Windows Azure Storage Queues
Saying that they both have a dramatically different API isn’t an overstatement. The Windows Azure Storage Queue is simple and the developer experience is quite good. It uses the local Windows Azure Storage Emulator and debugging is made quite easy. The tooling for Windows Azure Storage Queues allows you to easily peek at the top 32 messages and if the messages are in XML or Json, you’re able to visualize their contents directly from Visual Studio Furthermore, these queues can be purged of their contents, which is especially useful during development and QA efforts.
The REST API for the Windows Azure Storage Queue allows you to create a polling system where you manage the frequency at which the queues are checked for messages. In ideal circumstances, it has an average latency of 10 ms.
Consider Using Windows Azure Storage Queues if
- Your application needs to store over 5 GB worth of messages in a queue, where the messages have a lifetime shorter than 7 days.
- Your application requires flexible leasing to process its messages. This allows messages to have a very short lease time, so that if a worker crashes, the message can be processed again quickly. It also allows a worker to extend the lease on a message if it needs more time to process it, which helps deal with non-deterministic processing time of messages.
- Your application wants to track progress for processing a message inside of the message. This is useful if the worker processing a message crashes. A subsequent worker can then use that information to continue where the prior worker left off.
- You require server side logs of all the transactions executed against your queues.
Windows Azure Service Bus Queues
The Windows Azure Service Bus Queues are evolved and surrounded by many useful mechanisms that make it enterprise worthy! They are built into the Service Bus and are able to forward messages to other Queues and Topics. They have a built-in dead-letter queue and messages have a time to live that you control, hence messages don’t automatically disappear after 7 days.
Furthermore, Windows Azure Service Bus Queues have the ability of deleting themselves after a configurable amount of idle time. This feature is very practical when you create Queues for each user, because if a user hasn’t interacted with a Queue for the past month, it automatically gets clean it up. Its also a great way to drive costs down. You shouldn’t have to pay for storage that you don’t need. These Queues are limited to a maximum of 5gb. Once you’ve reached this limit your application will start receiving exceptions.
Windows Azure Service Bus Queues also have two modes of operation, the Queue can be set to deliver messages at-least-once or at-most-once. Both of the models have their own advantages and disadvantages. The delivery of a message of at most once isn’t great at dealing with failure of processing messages.
Access to the Service Bus is secured by Windows Azure ACS and local development is possible but it requires the installation of the Windows Azure Pack Service Bus. It’s consistent with Windows Azure Service Bus, but quite frankly this is only really viable if all your dev machines are setup using an image. To dev & test more effectively without all the hassle of maintaining identical configurations across all environments. I suggest creating a service bus namespace for each developer on your team using your dev & test MSDN Windows Azure Subscription.
Consider Using Queues in the Windows Azure Service Bus if
- You require full integration with the Windows Communication Foundation (WCF) communication stack in the .NET Framework.
- Your solution needs to be able to support automatic duplicate detection.
- You need to be able to process related messages as a single logical group.
- Your solution requires transactional behavior and atomicity when sending or receiving multiple messages from a queue.
- The time-to-live (TTL) characteristic of the application-specific workload can exceed the 7-day period.
- Your application handles messages that can exceed 64 KB but will not likely approach the 256 KB limit.
- Your solution requires the queue to provide a guaranteed first-in-first-out (FIFO) ordered delivery.
- Your solution must be able to receive messages without having to poll the queue. With the Service Bus, this can be achieved through the use of the long-polling receive operation.
- You deal with a requirement to provide a role-based access model to the queues, and different rights/permissions for senders and receivers.
- Your queue size will not grow larger than 5 GB.
- You can envision an eventual migration from queue-based point-to-point communication to a message exchange pattern that allows seamless integration of additional receivers (subscribers), each of which receives independent copies of either some or all messages sent to the queue. The latter refers to the publish/subscribe capability natively provided by the Service Bus.
- Your messaging solution needs to be able to support the “At-Most-Once” delivery guarantee without the need for you to build the additional infrastructure components.
- You would like to be able to publish and consume message batches.
Making the Final Decision
The decision on when to use Windows Azure Queues or Service Bus Queues clearly depends on a number of factors. These factors may depend heavily on the individual needs of your application and its architecture. If your application already uses the core capabilities of Windows Azure, you may prefer to choose Windows Azure Queues, especially if you require basic communication and messaging between services or need queues that can be larger than 5 GB in size.
Because Service Bus Queues provide a number of advanced features, such as sessions, transactions, duplicate detection, automatic dead-lettering, and durable publish/subscribe capabilities, they may be a preferred choice if you are building a hybrid application or if your application otherwise requires these features.
Find Out More
- Windows Azure Queues and Windows Azure Service Bus Queues – Compared and Contrasted
- How to Use Service Bus Queues
- How to Use the Queue Storage Service
- Configuring and Using the Storage Emulator with Visual Studio
- Service Bus Pricing Details
- Windows Azure Queue Storage Service Polling Task
- Securing and Authenticating a Service Bus Connection
- Service Bus Authentication and Authorization with the Access Control Service
- Looking at Windows Azure Service Bus Message Pump Programming Model
- Windows Azure Pack
- Introducing Table SAS (Shared Access Signature), Queue SAS and update to Blob SAS
- Understanding Windows Azure Storage Billing – Bandwidth, Transactions, and Capacity
- Best Practices for Leveraging Windows Azure Service Bus Brokered Messaging API
- Best Practices for Performance Improvements Using Service Bus Brokered Messaging