Azure Integration Part I: Reasons for Azure Storage Queue, Service Bus Queue, Topics/Subscription and Relays

Microsoft Azure can provide messaging in the cloud through a variety of different options. Sometimes it can be confusing as to which cloud technology you want to leverage with your company’s integration solution. Through a series of blogs, I hope to help summarize the options on these Azure technologies. In Part 1 of this series, I summarize and give insight on Azure Storage Queues, Azure Service Bus Queues, Topics/Subscription and Relays.

Azure Storage Queue

One common question when dealing with queues in Azure is trying to figure out the differences between Azure storage queues and the Azure service bus queues. Azure storage queues offer a REST based interface and provide reliable messaging between services. It is part of the set of Azure Storage service components which also include blobs, files and table storage.   Let us start with some reasons on using Azure storage queues. Listed below are some things to think about when deciding on using the storage queues.

• You want to store messages in any format
• Your messages are 64KB or less in size
• You only need message storage for less than 7 days
• Order delivery is not important
• Duplicate messaging is not important (At-Least-Once delivery)
• You do not need to integrate with WCF services
• You can have a decoupled architecture that can scale independently
• It provides asynchronous messaging between application components that are in the cloud, on-premises or mobile
• You need to be able to store millions of messages with no limit to maximum queue size
• You are okay with the limit of receiving of up to 32 messages on batch receive
• You need a maximum queue size greater than 80GB but less than 200TB
• The cost is $0.0036 per 100,000 transactions

Azure Service Bus

The Azure Service Bus is part of the messaging infrastructure that includes Queues, publish/subscribe through Topic/Subscriptions, Web service remoting through Relays, and other integration patterns. It provides a brokered and reliable message delivery services. Listed below are reasons to think about when deciding whether to use Azure Service Bus technologies.

• Require brokered messaging capabilities
  • Support for sessions
  • Dead lettering
  • Message deferral
  • Duplicate detection (Exactly-once delivery)
  • Message expiration
• Message size limit of 256Kb for Standard tier and 1MB for Premium tier
• No limit on the number of messages in the queue but the max on total size of all messages is 5GB
• Maximum queue size is 80GB. Each service bus queue can be configured to be 1 to 5GB in size, each GB having 16 partitions, thus max size is 5GB x 16 partition equaling 80GB in total size
• Cost is $0.05 per million operations (Basic tier)
• Brokered messages have a header and body

Now let us summarize further the different service bus options available to you:  Queues, Topics/Subscriptions and Relays.

Queues

Queues provide one way messaging infrastructure where a publisher sends a message and the message is received by a consumer. There can be many different consumers but only one consumer can receive a particular message.  Listed below are reasons for using a service bus queue.

• When you only need First In, First Out (FIFO) message delivery capabilities. This means that a single consumer will receive and process the messages in the order they were placed into the queue
• Using a point-to-point pattern of sending brokered messages from one application and receiving messages into another application, where either of these can be cloud based or on-premises
• Some capabilities of an Azure storage queue:
  • A decoupled or asynchronous and durable brokered message process
  • Reliable storage of messages until they can be processed
  • Guaranteed delivery of messages
  • Scale up or down rates at which messages are added to queues
  • Help manage workload during peak and non-peak times
  • Decoupled architecture allows recovery as messages can be added or processed if either publisher or consumer of messages has down time
  • Send or receive a batch of messages

Topic/Subscription

Topics/Subscription also provide one-way messaging infrastructure where a publisher sends a message and the message is received by a consumer, but that same message can be received by multiple receivers (subscribers). Topics will enqueue the messages whereas subscriptions will dequeue the messages. Each topic you can have one or more subscriptions and each subscription can only belong to one topic. A filter can be created to pick up messages matching some criteria.  Listed below are reasons for using a service bus topic/subscription.

• When you need subscription based processing using a publish-subscribe model
• An application that will send brokered messages and need to be received by one or more applications (subscribers)
• Need to have the same Queue capabilities listed above

Relays

Relays differ from Queues and Topics/Subscriptions as they provides bi-directional flow and do not store messages. With Relays, disparate applications can exchange messages. It is a hybrid solution where applications can send and receive the messages from source to destination. This is all accomplished as the relay service is connected to an on-premises application through a persistent two-way communication channel. The relay service exposes an endpoint where clients can communicate to the on-premises application. It also requires both client and server to be online.

Here is a list on reasons you might consider using an Azure relay service.

• Host an on-premises application and securely expose an end point in the cloud without opening a firewall or make changes to network
• Provide a hybrid solution with load balancing
• Need support for SOAP, WS-* and REST
• Have a requirement for a one-way, request/response or peer-to-peer messaging pattern
• Don’t want client direct access to on-premises service
• Want to expose a service to external clients without opening any inbound ports
• Want to keep your data on-premises but give external clients access.

That does it for Part 1 of the series. Hopefully, you have some basic information when deciding on solutions using Azure Storage Queues and Azure Service Bus Queues, Topics/Subscription and Relays.

In Part 2 of the series, I will write about the reasons for using Notification Hubs, Event Hubs, IoT Hubs.

In Part 3, I will discuss Azure storage offerings such as Azure Storage Tables,  Azure SQL Database, SQL Server on VM, Azure SQL Data Warehouse, Data Lake Store and DocumentDB

In Part 4, I will go in describe Azure Stream Analytics, Worker Roles and Web Roles.