Azure Integration Part II: What’s the Difference between Event Hubs, IoT Hubs and Notification Hubs?
In part 2 of our Azure Integration series, let’s look at the differences between Azure Event Hub, IoT Hub, and Notification Hub. You can go back and read part 1, Reasons for Azure Storage Queue, Service Bus, Topics/Subscriptions and Relays.
As the world of cloud-based integrations become more popular, especially in the IoT space, our solutions must change. Now, cloud integrations require a way to process messages or events between publishers and consumers. These solutions can eventually push millions of event messages. With Event Hubs, IoT Hubs and Notification Hubs, Microsoft has given any solution an easy way to do this that has high throughput and is very scalable.
Azure Event Hubs
Microsoft first came out with Event Hubs, which provide publish-subscribe capabilities at low latency. Before IoT Hubs, Event Hubs was Microsoft’s product to integrate high volume solutions with the cloud. It can ingest millions of events per second using partitions that can scale horizontally.
Architecturally, Event Hubs are made up of partitions. Picture a path with many different lanes and the events being sent to one of these lanes. It is what makes Event Hubs different than, say, Azure Service Bus Queues or Topics. These are more like single lanes. The partitions allow Event Hubs to have massive scaling.
From a security standpoint, publishers will use a Shared Access Signature token to identify themselves to the Event Hub. This simplifies the process of connecting to your hub.
For receivers, the Event Hub can consume the events via individual consumer groups. A consumer group will give each application their own view of the event stream. These consumer groups are also what provides the publish-subscribe functionality of Event Hubs and allows different consumers to be able to read this data whenever they want. When events are placed in the Event Hub, it will stay in the hub until the retention period expires. This provides the capability to be able to re-read the messages if needed, even after other consumers have read the event/message.
The protocols the Event Hub supports is AMQP and HTTP. When publishing to the Event Hub, you can publish using HTTP or AMQP. If you can use AMQP, it will provide lower latency and overall better performance than using HTTP. On the flip side, when you are consuming from the Event Hub, it will push the events using only AMQP, which means consumer client applications do not poll.
Event Hub Quick Summary Capabilities
- Provides ingress of telemetry data, device-to-cloud
- Supports AMQP, AMQP over WebSockets, and HTTP/1
- Security is Event hub wide using the Shared Access Signature (SAS) tokens
- Supports up to 5,000 connected devices
- Supports .NET and C
- Events/Messages have a 256K size limit (single or batch)
- Throughput is 1MB/second for publishing, 2MB/second for consuming
Azure IoT Hubs
As solutions started using Event Hubs, users realized that although it provided many features an event messaging based solution could want, there were some things missing. Azure IoT Hubs was the next hub service that Microsoft created and was built to receive telemetry data from IoT devices at a grander scale. IoT Hubs are like Event Hubs, but with additional features. One major difference it has with Event Hubs in that it is bi-directional. Devices can send data to the IoT hub, as well being able to receive data from the IoT Hub. If your solution requires device-to-cloud (D2C) and cloud-to-device functionality (C2D), then IoT Hubs is what you want to use.
One thing missing from Event Hubs was security features at the device level. Therefore, unlike Event Hubs, IoT Hubs offer security per device credentials and supports more options of popular languages and platforms. In order to accomplish this, each device needs to be registered in the IoT hub and thus it has management functionality for the device identities. You can control the list of devices that are registered in the IoT Hub, as well as control whether these devices can connect and send messages.
Like Event Hubs, IoT Hubs support protocols for AMQP and HTTP. In addition to these, it also supports MQTT. You can therefore integrate IP-capable devices to the IoT hub. If your devices are non-IP capable, you can communicate with the IoT Hub using a field gateway. For most devices, AMQP and HTTP protocols and field gateways will provide the capabilities to connect with the IoT Hub. However, if devices do not use one of these standard protocols and require protocol adaptation, Microsoft provides a custom gateway using the Azure IoT protocol gateway for these scenarios.
With Event Hubs being able to ingest millions of messages, it is limited to only have 5,000 simultaneous connections. With IoT Hubs, it can support up to a million devices connected at a given time. With this, there is a maximum daily quota of the number of messages you can send per day. The SKU and units will determine this for your pricing level. The SKU also determines your throttling level.
Quotas and throttling
Each Azure subscription is allowed up to 10 IoT Hubs.
S1: IoT Hub S1 edition is designed for IoT solutions that generate relatively small amounts of data. Each unit of the S1 edition allows up to 400,000 messages per day across all connected devices.
S2: IoT Hub S2 edition is designed for IoT solutions that generate large amounts of data. Each unit of the S2 edition will allow up to 6 million messages per day across all connected devices.
S3: IoT Hub S3 edition is designed for IoT solutions that generate large amounts of data. Each unit of the S3 edition will allow up to 300 million messages per day across all connected devices.
IoT Hub Quick Summary Capabilities
- Provides both ingress and egress of data, device-to-cloud (D2C) and cloud-to-device (C2D)
- Supports MQTT, AMQP, AMQP over WebSockets, and HTTP/1. Integrates with IoT Protocol Gateway for custom protocols
- Security is per device
- Supports millions of connected devices
- Supports for C, Node.js, Java, .NET, Python
- Ability to upload files from devices to cloud
Azure Notification Hubs
Being able to send out or broadcast notifications to millions of devices can be complicated. Push notifications work just in terms of managing and developing your backend push notification solution. You have to think about the different platforms devices use and different platform push notification systems (PNS) each use. For example Microsoft Push Notification Services (MPNS) is used to send notifications to Windows Phones, Apple Push Notification Service (APNS) and Firebase Cloud Messaging for Android devices (formally known as Google Cloud Messaging). With Azure Notification Hubs, it simplifies and removes the complexities of building this process on your own.
Normally, a device will call a push notification system and get a handle depending on the platform or system, and then stored on the backend. The backend application then calls the PNS, which forwards to the device. Challenges include the logic to handle sending to the different platforms, refreshing of tokens/handles, and PNS don’t support sending to multiple devices and need to have a way to store the devices and what they are subscribed to.
With Azure Notification Hubs, you can make one call and be able to send to millions of devices at once. Being an Azure service, it can scale and it supports multiple platforms. Devices need only have to register the PNS handles, which ultimately simplifies the code for your backend application, as the Notification Hub handles the task for storing and managing this. It also can be integrated with backend apps using the .NET or Node.js SDKs, on-premises or cloud applications.
As you can see, there are some differences between these hubs, in particular, with IoT Hubs and Event Hubs.
If you need bi-directional processing or security per device, use IoT Hubs. Or if you need more than 5,000 simultaneous connections, consider it as well. However, if you do not need those capabilities but still want to ingest millions of messages at a lower cost, Event Hubs is more than capable.
Notification Hubs is useful if you want to send mass notifications to client applications and devices at a massive scale, without incurring the cost of having to develop one.
Your solution could very well leverage all three technologies. Ingest all data with IoT Hub from millions of devices, use Event Hubs in the middle, and Notification Hubs to send mass alerts, for example. With these three Azure technologies and Services Buses from my previous blog post, Microsoft has given us integration capabilities in the cloud to handle almost all kinds of scenarios.
To continue on with Part 3, I will talk about the Azure storage offerings which will include Azure Storage Tables, Azure Sql Database, SQL Server on VM, Azure Sql Data Warehouse, Data Lake Store and DocumentDb.
