The mobility offers from Tier, Bolt and other providers show how quickly IoT has arrived in everyday life. Approximately 500,000 e-scooters and bicycles should be available on the streets of European cities, which interested parties can find and borrow via an app.
According to the provider Bolt, the number of trips in 2021 increased by 400 percent compared to the previous year (Bolt Safety Report 2022). Such scooters or bicycles are networked in terms of information technology and report their battery charge level and location to central points, among other things. These functions use the IoT standard protocol MQTT.
Other areas of application for MQTT are smart buildings, intelligent street lighting or parking lot status information. The number of IoT applications for basic services is growing steadily. This requires almost 100% availability, analogous to digital services such as online banking, messenger applications or contactless payment transactions.
High availability requirement
The term High Availability (HA) has become the standard requirement for running critical systems. The often used term four nines availability stands for an availability of 99.99 percent. Availability specified in this way is part of the contract documents for cloud infrastructure providers such as Amazon Web Services or Microsoft Azure.
On April 26th and 27th the eighth edition of the building IoT 2023 in Munich. The conference is aimed at those who create applications and products for the Internet of Things.
This year, the topic of IIoT is even more in focus than in previous years. The program offers including lectures on data analysis for the IIoT, time series databases, secure edge computing with Kubernetes and EU standards for cybersecurity.
To illustrate four nines availability, one can roughly assume an acceptable outage of about fifty minutes per year of operation. From the user’s point of view, the HA approach thus enables uninterrupted and consistent operation of services. In the IoT case, this means above all the exchange of information via the MQTT protocol.
Challenges for MQTT brokers
The provision of data or websites is state-of-the-art in terms of availability through zone replication, load balancing and automatic scaling. However, if you want to make MQTT brokers highly available, protocol-specific properties must be taken into account. It is an essential basis of the protocol that the connection between broker and client is maintained permanently, unlike with website requests. Furthermore, the brokers store the information that arises when the connection is established and during the connection as session information, or sessions for short. These contain, among other things, information on the topics that clients have subscribed to (subscriptions), messages that are in the queue (queued messages) and information on the last will (last will) in the event that a system can no longer be reached is.
The protocol also allows retained messages to be stored in the broker. Such messages must be persistently available along with the client-specific sessions in the event of a disconnect, broker failure, or network route change.
Consequently, in the event of a disruption, a HA-MQTT service must ensure the following functions consistently and with at least 99.99 percent availability, for example:
- Resuming the interrupted client-broker connection: The MQTT broker knows the authorization (access control lists) and authentication (credentials and, if applicable, TLS session) of the clients.
- The broker must provide or send topic subscriptions and messages that have not yet been delivered in the queue (queued messages with QoS > 0) of the client.
- He must have retained messages (retained messages including Last Will) ready or send them.
In summary, it is client- and connection-specific persistent data.
Scalability is not the same as high availability
In order to implement HA services, companies typically replicate virtual machines or containers (nodes) with identical applications in clusters that they operate in different availability zones. Infrastructure providers generally offer three availability zones in the Central Europe region. A zone is a physically separate data center within a region. A zone-redundant load balancer ensures that the individual nodes can be reached and evenly loaded across zone boundaries. From the client’s point of view, an identical public endpoint can always be reached. Replication rules and a scaling rule (auto scaling) adjust the number of nodes. Usually the operators set a minimum and a maximum number.
A high-availability cluster provides load balancing and replication rules to operate multiple nodes redundantly (Fig. 1).
Control variables for scaling are essentially the number of connections or the load on individual nodes. Status information (health probes) and cyclically sent signs of life (heartbeats) ensure that the individual nodes can be monitored and replaced by the replications in the event of a failure or overload. All nodes are zone-independent part of a common virtual network.
This setup can guarantee consistently high performance, which is largely independent of whether there are tens, thousands or a million client requests. Such architectures theoretically ensure the availability of Five Nines. However, the assumption applies here that the individual nodes act as loners and can be replaced by newly started replicas in the cluster in the event of an error. No data needs to be exchanged between nodes.
In contrast to this, it is not sufficient for a highly available MQTT broker to provide one or more replicas of the broker, since the protocol-specific functions must always be fulfilled. In particular, this means that when replacing a failed node, the client and connection-specific persistent data must be available to resume the connection. First, to ensure this, one can use a fixed number of replicas instead of auto scaling. Secondly, additional communication can be set up between the nodes, which always synchronizes the persistent data in order to keep all brokers up to date with the MQTT active-active cluster concept.
