MQTT’s benefits for digital transformation

Industry 4.0, the smart factory and cyber-physical systems are distinct initiatives, but have one thing in common – they all depend on massively parallel machine-to-machine communications. Analysts across the board project the global market for the Industrial Internet of Things (IIoT) to reach hundreds of billions of dollars by the end of the decade. And whether those things are sensors, motors, controllers or human-machine interfaces (HMIs), they all need to connect with an increasing number of users, from supervisory control and data acquisition (SCADA) systems and manufacturing execution systems (MES) to programmable logic controllers (PLCs) or even original equipment manufacturers (OEMs) and third-party reliability services. Having data isn’t enough – it needs to be made available and consumable in a timely fashion. That’s where message queueing telemetry transport (MQTT) comes in.

An open source messaging protocol developed for mission-critical SCADA pipeline systems in oil and gas, MQTT addresses many of the challenges involved in today’s increasingly complex industrial networks. Current industrial communications protocols are based on direct connections between data source and data consumer. They’re effective for machine-level operations, but when it comes to connecting large numbers of devices with outside applications, they can be complex and time-consuming to deploy and maintain. They lack the flexibility and scalability to serve the new data-driven environment.

MQTT was designed with machine-to-machine communications in mind. Its streamlined network architecture and lightweight packet specification make it flexible enough for reconfigurable factory lines and scalable enough to support the addition of hundreds or even thousands of connected devices. “It’s the right paradigm for manufacturing,” said Mike Bowers, chief architect at FairCom (Sandy, Utah). “I strongly believe that MQTT will do for manufacturing what the Internet did for everything else.”

MQTT publish/subscribe

MQTT is an application layer protocol (layer 7 in the OSI networking model) that uses TCP/IP as the transport layer. In this way, it’s like the HTTP application used for the Internet. But HTTP requires long and complex headers, while the MQTT header is lightweight, at 2 B. MQTT messages have a maximum payload size of 256 MB. The content is format agnostic—payloads can contain binary data, text data, images files such as JPEGs, XML, JavaScript object notation (JSON), and more. MQTT also supports encrypted files.

The message specification is important but some of the biggest benefits of MQTT result from its architecture. Current industrial networking protocols are based on poll/response communications over direct connections among devices.* The linked devices respond when polled or send updates at regular clock intervals. These protocols are highly effective for the type of real-time data exchange needed at the machine level, such as between encoders and motors or motors and controllers. As the number of applications and devices that need to exchange data increases beyond the direct machine level, however, the network and even the devices themselves become bogged down. And don’t forget, these models are still constrained by the need for direct connection. Even for protocols that follow a producer-consumer model, the consumer still needs to know the identity of the producing device.

To avoid these issues, MQTT is based on a publish/subscribe (pub/sub) model. Devices and applications, called clients, interface through middleware known as a broker. Clients don’t connect to one another or poll a specific address for data. Instead, all clients connect to the broker, which functions something like a post office, accepting information from publisher clients and relaying it to subscriber clients (see Figure 1).

MQTT involves a single TCP/IP connection from client to broker. Each client is assigned a unique client ID, but this is only used by the broker to establish the connection. The actual data gathering is governed by topics. An MQTT topic is a text string that identifies the data being published. Consider a motor with onboard temperature and current sensors connected to the broker as publishing clients (either directly or through another connected device). Their topics might be defined as follows:

cellB/motorA/sensor/temperature

or

cellB/motorA/sensor/current

A predictive maintenance (PM) application could subscribe to these two topics to receive updates from the sensors. To further simplify the process, topics can be used with wildcards, so the PM application would subscribe to cellB/+/sensor/current to access changes to current draw for all motors in the cell (a single-level wildcard) or cellB/motorA/sensor/# to receive all sensor data for that motor (a multi-level wildcard; see Figure 2). Similarly, the SCADA system, MES, or data historian, as well as the PM application, could subscribe to the appropriate topic path to get status from a safety PLC, for staying on top of machine state.

Subscribing by topic means that the process of receiving the published data is independent of the type or IP address of the publishing client. Think of it as Twitter for machines, compared to traditional poll/response systems, which are more akin to email. If a sensor is swapped out but assigned the same topic, any changes to its details are transparent to the subscribing clients.

“I’m not tying my topics to devices,” Bowers said. “I’m tying them to what they do, as opposed to what they are. That decoupling frees the factory to be agile.” Customers using conventional poll/response networking haven’t been able to easily redesign or reconfigure production lines, he notes. MQTT provides an alternative. “Now, they can dynamically change which equipment talks to each other piece of equipment by just changing topics and subscriptions. It’s very quick and easy to just add new devices and reshape and reorganize the line.”

The discussion of publishing brings up another key MQTT advantage, which is that it reports by exception. In a conventional industrial network, applications poll devices for updates. If four different applications need to capture sensor data aggregated by a PLC and also check its status, each of the four applications polls the PLC. The problem is that industrial equipment is built to last, so the vast majority of the readings are the machine equivalent of “I’m fine.” As the number of devices proliferates, these status quo readings clog the network while adding little value.

With MQTT, a publishing client only sends data to the broker if it represents a change above a specified threshold. The approach solves the “I’m fine” problem, significantly reducing network traffic and letting applications concentrate on meaningful data. Clients do send out a 16-bit “keep alive” message to the broker at specified intervals (typically 60 seconds), but this doesn’t have a significant impact on network bandwidth.

A closer look at the broker

The MQTT protocol was designed to move massive numbers of message packets while minimizing latency. To help reach these goals, the broker is middleware. As such, it can reside on any server in any location – in the cloud, on premises, or both. Its function is to filter and relay messages, making it highly scalable. “A single broker can scale to millions of topics,” said Benson Hougland, vice president of product strategy at Opto 22 (Temecula, California). “If you need more capacity, you just add another broker, and you can load balance between them.” Multiple brokers can be installed for automatic failover to ensure uninterrupted connectivity. “There are a lot of capabilities, even at the broker level, to build a resilient network.”

A basic MQTT broker stores only the most recent values published by active clients. To maintain a record, an implementation needs a data historian or data logger subscribed to the appropriate topic paths. Here, once again, the pub/sub model brings value, as the historian can also publish its data to other subscribers such as the MES or a third-party reliability provider. Alternatively, an advanced MQTT broker stores its messages in an embedded database or integrates with a database. The

Four MQTT connectivity provisions

MQTT includes several provisions intended to provide the security and reliability that is essential to industrial operations.

1. Security

One of the big advantages to the client-broker model is in the area of security. MQTT needed to ensure secure connections but adding it to the code would be counter to the goal of a lightweight messaging transport. Instead, its creators took a different tack. “There’s no security model defined in MQTT. And that, indeed, was on purpose,” said Arlen Nipper, president and CTO at Cirrus Link (Stillwater, Oklahoma). While working for Arcom Controls, Nipper teamed with IBM’s Andrew Stanford-Clark to create MQTT. “MQTT is built on top of TCP, so we decided let’s let TCP do the heavy lifting. Let’s apply best-in-class TCP/IP security. Today, that happens to be TLS, but as technology advances in TCP, MQTT will be able to ride along with that natively.”

One of the concerns about industrial networking has always been the prospect of connecting assets to the Internet. Here, the MQTT architecture comes into play. With MQTT, a device only makes one connection, which is to the broker. The only open port in the entire system is at the broker, which is protected by a firewall. Because MQTT uses TLS, any connections to that port need to be authenticated and encrypted. And because each MQTT client only connects to the broker through a single secure, authenticated connection, the client is protected. “The beauty of MQTT is that it doesn’t require any ports open on the client,” Hougland said. The data publisher, say, an MQTT-enabled PLC, sends its data to the broker over an outbound connection. “There’s no incoming connection to the client, so right out of the gate, there is no cybersecurity vulnerability.”

That’s not to say that MQTT doesn’t support bidirectional communications, however, he hastens to add. “When we first connect to an MQTT broker, we establish a TLS encrypted authenticated connection,” Hougland said. “If a SCADA application needs to issue a command, it can do so securely through the broker and back down to a PLC with no open ports. We can verify that that command came from a trusted source, and we initiate the action. And all of that takes place in milliseconds.”

Bowers said, “To ensure trusted sources publish and subscribe to topics, a broker may apply security access controls to topics.”

2. Connection configurations

Client-broker connections can be defined as clean or persistent:

• Clean sessions: The client exchanges messages with the broker for the length of that session. When the connection is interrupted, the session is closed, and no data is retained.

• Persistent sessions: The broker maintains the authenticated session even if the connection is broken. It keeps the subscription information and storing any QoS 1 or 2 messages that are published in a buffer. When the device reconnects, it receives those messages in chronological order.

The type of session is defined at the time of connection. Clean sessions are useful for troubleshooting and monitoring. Persistent sessions are particularly beneficial for mission critical and/or bidirectional communications. Once the connection is encrypted and authenticated, the client can securely receive input without delay.

3. Quality of service

MQTT was designed with special features to ensure message delivery and monitor the connectivity of devices on the network. The protocol defines three quality of service (QoS) levels:

• QoS 0: publish once, with no required acknowledgment (at least once)

• QoS 1: publish repeatedly until acknowledgment sent (at most, once)

• QoS 2: publish repeatedly and guarantee single receipt (exactly once).

Assigning a QoS for a topic path is part of client configuration. Users also have the option to configure a topic as a retained message. In this case, the MQTT broker will store the most recent update it has received for that topic path. Any new subscriber receives the last retained message immediately upon connection, which is particularly beneficial for communicating device status.

MQTT clients can also be configured with a “last will and testament” (LWT) message that is sent to the broker upon connection. All LWT settings include topic path, QoS, and keep-alive timer value. When a client loses connection to the broker for a duration exceeding the keep alive timer value, the broker distributes the LWT message to subscribers. This is useful for notifying clients that a device has gone offline.

4. Sparkplug B

Some of the same features that give MQTT so much flexibility and configurability can be problematic in the industrial environment. Lack of topic naming conventions, for example, can make building out a network and adding large numbers of new devices from different vendors very labor-intensive. The minimally defined nature of the MQTT packet adds to flexibility but also presents a barrier to interoperability. To address these issues and several others, the same team that developed MQTT created Sparkplug, and released the specification to the open source community, managed by the Eclipse Foundation. Sparkplug B isn’t a variation of MQTT but an add-on. Sparkplug B is to MQTT as HTML is to HTTP – a means for serving up data.

Nipper describes the motivation behind the development of Sparkplug B. “What we saw was a lot of people using MQTT, but they had different topic namespaces and they had different payload definitions,” he said. “The Sparkplug specification does not change MQTT at all. All it says is hey, if you’re going to do an operational real-time mission-critical system, then this is probably a good place to start.”

Sparkplug B is a detailed specification but here, we highlight several key aspects:

• An OT-centric topic namespace definition: Sparkplug B establishes a standard format for MQTT topics, increasing interoperability.

• An OT-centric payload definition using the Google protocol buffer: The specification defines a structure for the payload, including metadata, timestamps, and other user-defined information units. The goal of the modification is to make the data more easily consumable and to increase interoperability and minimizing payload size. “Sparkplug B is meant to organize the data that’s going to be moved over to the MQTT transport in a method that everybody who’s involved in the system will know what’s going on,” Hougland said.

• State management capabilities – Each client publishes a birth message when first connected that announces it is online and provides a list of tags and topics. Any subscribers will first receive the birth message. If anything changes on that connected device, the birth message will be updated.

• If the device connection to the broker is disrupted beyond some specified time, the device client will begin storing messages. At the same time, the broker will send a death message—essentially an enhanced LWT message—to all subscribers, along with a time and date stamp. When the device connection is restored, the client will first send the current data, and then forward the stored data in reverse chronological order. Commonly referred to as “store & forward,” this ensures that subscribers receive valuable data that may have been generated during a network or publishing client outage.

This is just a quick summary of key aspects of Sparkplug B. They are designed to address issues and improve MQTT’s usefulness for the industrial environment. As with anything in engineering, there are trade-offs involved and not everyone agrees on the cost/benefit proposition.

As an architect, Bowers appreciates the issues identified by the Sparkplug B group but prefers to address the challenges differently. “They’ve exposed several problems,” he notes. “I just don’t agree with their solutions. For example, one of the problems is that a topic is a string and there’s little structure to what’s in that string. That’s true, but [the Sparkplug B solution] too narrowly defines what goes into a topic and it’s not realistic enough for the real world. Another problem with MQTT is that there’s no structure in the payload. The Sparkplug B solution is to define a structure. Their structure is okay, but again, it’s limiting.”

By using Google protocol buffers, Sparkplug B expresses the payload in binary code, which is highly compact but can’t be read and debugged without a software translator. Bowers prefers an alternative like JSON, which is readable at a glance and which he considers to be the de facto standard for data interchange, Bowers said.

Ultimately, it’s up to the end user and the rest of their team to determine implementation. Hougland focuses on the benefits of Sparkplug B in terms of enabling devices from different vendors to communicate without the need for custom code. “In industrial automation, we really need something that’s open and interoperable,” he said. “So, if vendor A, vendor B, and vendor C all choose to use MQTT and Sparkplug B—both open-source technologies—now those systems can start communicating with each other out of the box.”

MQTT use in the real world

MQTT is increasingly gaining a foothold in the industrial space. It’s being used to implement reconfigurable factory lines. “I think the main driver is the need to replace the nightmare of point-to-point integration, which is expensive and prevents agility,” Bowers said. “Decoupled integration is the solution and MQTT allows you to decouple systems better than anything else.” It can be used to send recipes to equipment to dynamically change operations very quickly, and also, of course, any system can dynamically subscribe to any telemetry for various types of analytics.

The ecosystem is thriving, with MQTT-enabled sensors, open source and commercial brokers, and an array of edge devices designed to streamline the integration of conventional industrial networks and legacy equipment with MQTT.

The technology is rapidly gaining acceptance and has passed the early adopter phase. “It’s secure, it’s open source, and it’s absolutely high performance,” Hougland said. “I would definitely say that we’re in the early majority now.”

“When I present on this at conferences, we have big audiences and they’re very interested,” Bowers said. “I don’t see this as a marketing hype cycle. I believe MQTT will truly transform manufacturing and make it so much cheaper and faster and more agile. I really think MQTT’s the future.”

– The Association for Advancing Automation (A3) is a CFE Media and Technology content partner.

Original content can be found at A3.

Do you have experience and expertise with the topics mentioned in this content? You should consider contributing to our CFE Media editorial team and getting the recognition you and your company deserve. Click here to start this process.