Azure Storage Queues VS Azure Service Bus Queues

October 20, 2013 — 12 Comments

Updated May 1st 2015

Azure-Storage-Queue-vs-Azure-ServiceBus-Queue There are two kinds of queues in Microsoft Azure. The first is Azure Storage Queues and the second is Azure Service Bus Queues. Both are quite different.

Meet the Azure Queues

The Azure Storage Queue is capable of handling 2000 message transactions per second and each message has a maximum time to live of 7 days. Billing is calculated by storage transaction and size of the storage used by the queue.

The Azure Service Bus Queue is capable of handling 2000 message transaction per second and messages can live forever! Billing is calculated by the number of 64kb message transactions. This Queue flavor is feature rich and is especially interesting in Hybrid scenarios where part of the application is Cloud based and the other part is on-premise in your data center.

Azure Storage Queues

Saying that they both have a dramatically different API isn’t an overstatement. The Azure Storage Queue is simple and the developer experience is quite good. It uses the local Azure Storage Emulator and debugging is made quite easy. The tooling for Azure Storage Queues allows you to easily peek at the top 32 messages and if the messages are in XML or Json, you’re able to visualize their contents directly from Visual Studio Furthermore, these queues can be purged of their contents, which is especially useful during development and QA efforts.

The REST API for the Azure Storage Queue allows you to create a polling system where you manage the frequency at which the queues are checked for messages. In ideal circumstances, it has an average latency of 10 ms.

Access to Azure Storage Queues is secured by Shared Access Signatures.

Consider Using Azure Storage Queues if

  • Your application must store over 80 GB of messages in a queue, where the messages have a lifetime shorter than 7 days.
  • Your application wants to track progress for processing a message inside of the queue. This is useful if the worker processing a message crashes. A subsequent worker can then use that information to continue from where the prior worker left off.
  • You require server side logs of all of the transactions executed against your queues.

Azure Service Bus Queues

The Azure Service Bus Queues are evolved and surrounded by many useful mechanisms that make it enterprise worthy! They are built into the Service Bus and are able to forward messages to other Queues and Topics. They have a built-in dead-letter queue and messages have a time to live that you control, hence messages don’t automatically disappear after 7 days.

Furthermore, Azure Service Bus Queues have the ability of deleting themselves after a configurable amount of idle time. This feature is very practical when you create Queues for each user, because if a user hasn’t interacted with a Queue for the past month, it automatically gets clean it up. Its also a great way to drive costs down. You shouldn’t have to pay for storage that you don’t need. These Queues are limited to a maximum of 80gb. Once you’ve reached this limit your application will start receiving exceptions.

Azure Service Bus Queues also have two modes of operation, the Queue can be set to deliver messages at-least-once or at-most-once. Both of the models have their own advantages and disadvantages. The delivery of a message of at most once isn’t great at dealing with failure of processing messages.

Access to the Service Bus is secured by Shared Access Signatures and AMQP. Local development is possible but it requires the installation of the Windows Azure Pack Service Bus. It’s consistent with the Azure Service Bus, but quite frankly this is only really viable if all your dev machines are setup using an image. To dev & test more effectively without all the hassle of maintaining identical configurations across all environments. I suggest creating a service bus namespace for each developer on your team using your dev & test MSDN Azure Subscription.

Consider Using Queues in the Azure Service Bus if

  • Your solution must be able to receive messages without having to poll the queue. With Service Bus, this can be achieved through the use of the long-polling receive operation using the TCP-based protocols that Service Bus supports.
  • Your solution requires the queue to provide a guaranteed first-in-first-out (FIFO) ordered delivery.
  • You want a symmetric experience in Azure and on Windows Server (private cloud). For more information, see Service Bus for Windows Server.
  • Your solution must be able to support automatic duplicate detection.
  • You want your application to process messages as parallel long-running streams (messages are associated with a stream using the SessionId property on the message). In this model, each node in the consuming application competes for streams, as opposed to messages. When a stream is given to a consuming node, the node can examine the state of the application stream state using transactions.
  • Your solution requires transactional behavior and atomicity when sending or receiving multiple messages from a queue.
  • The time-to-live (TTL) characteristic of the application-specific workload can exceed the 7-day period.
  • Your application handles messages that can exceed 64 KB but will not likely approach the 256 KB limit.
  • You deal with a requirement to provide a role-based access model to the queues, and different rights/permissions for senders and receivers.
  • Your queue size will not grow larger than 80 GB.
  • You want to use the AMQP 1.0 standards-based messaging broker. For more information about AMQP, see Service Bus AMQP Overview.
  • You can envision an eventual migration from queue-based point-to-point communication to a message exchange pattern that enables seamless integration of additional receivers (subscribers), each of which receives independent copies of either some or all messages sent to the queue. The latter refers to the publish/subscribe capability natively provided by Service Bus.
  • Your messaging solution must be able to support the “At-Most-Once” delivery guarantee without the need for you to build the additional infrastructure components.
  • You would like to be able to publish and consume batches of messages.
  • You require full integration with the Windows Communication Foundation (WCF) communication stack in the .NET Framework.

Making the Final Decision

By gaining a deeper understanding of the two technologies, you will be able to make a more informed decision on which queue technology to use, and when. The decision on when to use Azure Queues or Service Bus queues clearly depends on a number of factors. These factors may depend heavily on the individual needs of your application and its architecture. If your application already uses the core capabilities of Microsoft Azure, you may prefer to choose Azure Queues, especially if you require basic communication and messaging between services or need queues that can be larger than 80 GB in size.

Because Service Bus queues provide a number of advanced features, such as sessions, transactions, duplicate detection, automatic dead-lettering, and durable publish/subscribe capabilities, they may be a preferred choice if you are building a hybrid application or if your application otherwise requires these features.

Source: Azure Queues and Service Bus queues – compared and contrasted

Find Out More

Trackbacks and Pingbacks:

  1. Dew Drop – October 21, 2013 (#1,649) | Morning Dew - October 21, 2013

    […] Windows Azure Storage Queues VS Windows Azure Service Bus Queues (Alexandre Brisebois) […]

    Like

  2. From my reading list #3 – October 24, 2013 | Pascal Laurin - October 24, 2013

    […] Windows Azure Storage Queues VS Windows Azure Service Bus Queues by Alexandre Brisebois A good comparison between the two options in Windows Azure regarding queuing technologies […]

    Like

  3. Windows Azure - October 25, 2013

    Windows Azure Community News Roundup #74

    Welcome to the latest edition of our weekly roundup of the latest community-driven news, content and

    Like

  4. Windows Azure Community News Roundup #74 | The Right Tool Kit For Your Home Based Business - October 25, 2013

    […] Windows Azure Storage Queues VS Windows Azure Service Bus Queues (posted October 20th) […]

    Like

  5. Windows Azure Community News Roundup #74 - Windows Azure Blog - October 25, 2013

    […] Windows Azure Storage Queues VS Windows Azure Service Bus Queues (posted October 20th) […]

    Like

  6. Reading Notes 2013-10-28 | Matricis - October 29, 2013

    […] Windows Azure Storage Queues VS Windows Azure Service Bus Queues – Nice and useful post that compares Azure Storage Queues and Azure Service Bus Queues. […]

    Like

  7. 微软云计算: Windows Azure 中文博客 - October 29, 2013

    Windows Azure 社区新闻综述(#74 版)

    欢迎查看最新版本的每周综述,其中包含有关云计算和 Windows Azure 的社区推动新闻、内容和对话。以下是本周的亮点。 文章、视频和博客文章 Azure CDN:吸取的宝贵经验

    Like

  8. Reading Notes 2013-11-04 | Matricis - November 4, 2013

    […] Windows Azure Storage Queues VS Windows Azure Service Bus Queues – Nice and useful post that compares Azure storage queues and Azure Service Bus queues. […]

    Like

  9. End of Week Research Roundup | endjin blog - November 10, 2013

    […] Windows Azure Storage Queues VS Windows Azure Service Bus Queues (Alexandre Brisebois) […]

    Like

  10. The Top 10 Most-Read #WindowsAzure Posts of 2013 | Alexandre Brisebois - December 29, 2013

    […] Windows Azure Storage Queues VS Windows Azure Service Bus Queues – 1,103 reads […]

    Like

  11. Focus sur le stockage Azure | Nouvelles Chroniques d'Amethyste - May 21, 2015

    […] https://alexandrebrisebois.wordpress.com/2013/10/20/windows-azure-storage-queues-vs-windows-azure-se… […]

    Like

  12. Scheduling WebJobs, Batch, and Worker Roles? - Falafel Software Blog - February 28, 2016

    […] Scheduler runs jobs at a specific time and can kick off HTTP/HTTPS endpoints, queue messages or service bus options. You can create scheduled jobs in the Azure Portal or using Visual […]

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s