Communication Protocols of an Industrial Internet of Things Environment: a Comparative Study

Home , Communication protocol, Messaging pattern, Modbus, MQTT, ZeroMQ

Article Communication Protocols of an Industrial Internet of Things Environment: A Comparative Study

Samer Jaloudi Department of Information and Communication Technology, Al Quds Open University, Nablus 00407, West Bank, Palestine; [email protected] or [email protected]; Tel.: +970-0599-376-492

Received: 4 February 2019; Accepted: 4 March 2019; Published: 7 March 2019

Abstract: Most industrial and SCADA-like (supervisory control and data acquisition) systems use proprietary communication protocols, and hence interoperability is not fulfilled. However, the MODBUS TCP is an open de facto standard, and is used for some automation and telecontrol systems. It is based on a polling mechanism and follows the synchronous request–response pattern, as opposed to the asynchronous publish–subscribe pattern. In this study, polling-based and event-based protocols are investigated to realize an open and interoperable Industrial Internet of Things (IIoT) environment. Many Internet of Things (IoT) protocols are introduced and compared, and the message queuing telemetry transport (MQTT) is chosen as the event-based, publish–subscribe protocol. The study shows that MODBUS defines an optimized message structure in the application layer, which is dedicated to industrial applications. In addition, it shows that an event-oriented IoT protocol complements the MODBUS TCP but cannot replace it. Therefore, two scenarios are proposed to build the IIoT environment. The first scenario is to consider the MODBUS TCP as an IoT protocol, and build the environment using the MODBUS TCP on a standalone basis. The second scenario is to use MQTT in conjunction with the MODBUS TCP. The first scenario is efficient and complies with most industrial applications where the request–response pattern is needed only. If the publish–subscribe pattern is needed, the MQTT in the second scenario complements the MODBUS TCP and eliminates the need for a gateway; however, MQTT lacks interoperability. To maintain a homogeneous message structure for the entire environment, industrial data are organized using the structure of MODBUS messages, formatted in the UTF-8, and then transferred in the payload of an MQTT publish message. The open and interoperable environment can be used for Internet SCADA, Internet-based monitoring, and industrial control systems.

Keywords: automation; IIoT; MQTT; MODBUS; publish–subscribe; request–response; SCADA

1. Introduction The Internet of Things (IoT) is an emerging technology that represents a cost-effective, scalable, and reliable ecosystem, proposed for many applications, including smart city sectors [1,2], consumer devices [3,4], industrial environments [5,6], Internet of vehicles [7,8], multimedia [9,10], and 5G systems [11,12]. The IoT platform, from the communications perspective, consists of a TCP/IP network and standard protocols [13]. Standard protocols primarily include advanced message queuing protocol version 1.0 (AMQP 1.0) [14–16], message queuing and telemetry transport (MQTT) [17], constrained application protocol (CoAP) [18], extensible messaging and presence protocol (XMPP) [19], and JavaScript object notation (JSON) [20]. The TCP/IP networks include Wi-Fi [21], Internet, Intranet, and modern mobile networks. The employment of a communication infrastructure and protocol depends on the ﬁeld of application, the timing requirements, and the data transmission rates [13]. Therefore, the requirements of consumer electronics are different from those of smart city sectors and industrial applications.

Future Internet 2019, 11, 66; doi:10.3390/ﬁ11030066 www.mdpi.com/journal/futureinternet Future Internet 2019, 11, 66 2 of 18

In the industry context, Industrial Internet of Things (IIoT) [22,23], Industry 4.0 [24,25], Smart Manufacturing [26,27], and Smart Factory [28,29] are all terminologies for the emerging industrial environments that employ information and communication technologies (ICTs), including IoT platforms, while maintaining industry requirements. The term IIoT refers to the use of IoT technologies in many fields of application such as manufacturing, factories, transportation, gas, oil, and electric grids. However, most industrial and SCADA-like (supervisory control and data acquisition) systems of these fields employ proprietary communication protocols and ICTs, which lead to closed industrial systems. Hence, customers are stuck to a single vendor, costs are high, and interoperability is not fulfilled. In this paper, standard IoT protocols are introduced and compared. Then, MQTT is chosen for machine-to-machine (M2M) communications to complement the MODBUS TCP [30–32] operations in an IIoT environment. This environment integrates an event-based message-oriented protocol, i.e., MQTT, with a polling-based request–response protocol, intended for industrial applications, i.e., MODBUS TCP. The study shows that the MODBUS TCP and MQTT can coexist together, and in parallel, within the same IIoT environment. While industrial requirements of control and monitoring can be met via the MODBUS TCP using the request–response model, the MQTT protocol complements its operation by fulfilling the IoT requirements, using the publish–subscribe pattern for M2M communications. The IoT protocol works in parallel with the MODBUS TCP and relays industrial data to an Internet-based server for remote monitoring, analysis, and archiving. To solve the interoperability problem, industrial data are formatted using the MODBUS message format and transferred in the payload of the MQTT publish messages. In fact, the IoT protocols cannot replace the MODBUS protocol in most industrial applications, especially when an optimized message structure is required in the application layer. A second scenario is proposed here, which is to consider the MODBUS TCP as an IoT protocol, and hence build the IIoT environment with the MODBUS TCP only. This solution is efficient and complies with most industrial applications; however, it only depends on the request–response model. The choice of solution is totally dependent on the requirements of the industrial application. The developed environment can be employed in many industrial applications, including Internet SCADA, Internet-based monitoring, and industrial control systems. Hence, customers are not stuck to a single vendor, costs are reduced, and interoperability is maintained. The remainder of the paper is organized as follows. Section2 presents a related literature survey, Section3 presents a theoretical background of the MODBUS protocol, Section4 compares IoT protocols, and Section5 compares MQTT and the MODBUS TCP. Section6 compares the latencies and resource usage of both protocols. Section7 presents a discussion and Section8 concludes the paper.

2. Related Work In the context of industry, researchers are proposing and examining protocols, networks, and middleware architectures for industrial ICT infrastructure and integration. For integration purposes, the authors of [33] proposed a data-oriented machine-to-machine (M2M) communication middleware based on the ZeroMQ platform for IIoT applications. The researchers of [34] presented a case study for controlling industrial robots and monitoring energy consumption remotely based on Ebbits middleware, which transforms data into web services. Another service-oriented IIoT middleware is proposed in [35] for balancing task allocation between a mobile terminal and a utility cloud service. In [36], a platform based on SystemJ for IIoT is proposed using an FPGA (ﬁeld programmable gate array), and then tested in an automation system. Reference [37] describes a collaboration-oriented M2M (CoM2M) messaging mechanism for IIoT, based on the platform PicknPack food packaging line. In [38], legacy ﬂexible manufacturing systems are integrated with the SCADA system of an Industry 4.0 framework via Ethernet. However, these research papers investigated middleware architectures for integration purposes. The infrastructure of an IIoT was examined in [39], where the IoT vision in industrial wireless sensor networks was implemented using the IPv6 over low-power wireless personal area network Future Internet 2019, 11, 66 3 of 18

(6LoWPAN) and CoAP. Furthermore, the author of [40] proposed a software-defined network for an IIoT based on new networking technologies. However, these research papers investigated network solutions and architectures. Infrastructure protocols were examined in many studies. For example, the authors of [41] developed an edge IoT gateway to extend the connectivity of MODBUS devices to IoT by storing the scanned data from MODBUS devices locally, and then transferring the changes via an MQTT publisher to MQTT clients via a broker. The researchers of [42] designed and implemented a web-based real-time data monitoring system that uses MODBUS TCP communications, following which all data are displayed in a real-time chart in an Internet browser, which is refreshed at regular intervals using hypertext transfer protocol (HTTP) polling communications. In [43], a MODBUS serial protocol was reported that collects data serially via a RS-232 protocol and transfers the collected data over the ZigBee protocol. In [44], measurements of field devices collected by the MODBUS serial protocol were transferred over HTTP using a Wi-Fi network. However, these papers proposed a gateway as a bridge between MODBUS and the Internet, Intranet, or wireless network. In this study, the MODBUS TCP is proposed to be implemented in two scenarios—either alone, or in parallel with an MQTT publisher without a gateway. The first solution proposes the MODBUS TCP as an IoT protocol to build the industrial environment on a standalone basis. Using the second arrangement, the MODBUS TCP executes its operations while maintaining the requirements of industrial applications and MQTT achieves the M2M communications for IoT functions.

3. MODBUS Theory The MODBUS TCP is a byte-oriented, industrial communication protocol, open de facto standard, used for data exchange between embedded systems, devices, and industrial applications. Devices, reacting as clients, may benefit from the inexpensive implementation of such a lightweight protocol for polling industrial devices that react as servers. Polling communications follow the request–response mechanism, where a client queries the server for specific data or executes commands in the server using a frame of bytes arranged in a specific way, called a frame format. The server replies to the client queries via a frame of bytes either holding measurement data from sensors or confirming the execution of commands. Sixteen-bit data registers store measurement values, and coils hold the status of ON and OFF switches. Therefore, MODBUS TCP uses the polling mechanism, as opposed to the event-based mechanism, explained in the next section. As listed in Table1, the protocol specifications [ 30] define three categories of function codes for the access of data in remote devices. These data are stored in coils or registers as status values for measurements or transferred as setpoints for control. Coils perform one-bit read and write operations for switching the attached devices ON and OFF or reading and writing one-bit internal configuration values. Discrete inputs perform one-bit read operations for reading the status of the attached devices, whether they are switched ON or OFF. The 16-bit input registers are responsible for measurements from physical devices, and the 16-bit holding registers perform read and write operations related to internal reconfigurable values.

Table 1. Function codes for data access in MODBUS.

Data Access Type Function Code Meaning 1 bit physical discrete input 0x02 read discrete inputs 1 bit internal bits, physical coils 0x01 read coils 1 bit internal bits, physical coils 0x05 write single coil 1 bit internal bits, physical coils 0x0F write multiple coils 16 bit physical input registers 0x04 read input registers 16 bit internal and physical output registers 0x03 read holding registers 16 bit internal and physical output registers 0x06 write single register 16 bit internal and physical output registers 0x10 write multiple registers 16 bit internal and physical output registers 0x17 read/write registers 16 bit internal and physical output registers 0x16 mask write register 16 bit internal and physical output registers 0x18 read ﬁrst in ﬁrst out (FIFO) queue Future Internet 2019, 11, x FOR PEER REVIEW 4 of 18

internal and physical output 16 bit 0x06 write single register registers internal and physical output 16 bit 0x10 write multiple registers registers internal and physical output 16 bit 0x17 read/write registers registers internal and physical output 16 bit 0x16 mask write register registers Future Internet 2019, 11internal, 66 and physical output read first in first out (FIFO)4 of 18 16 bit 0x18 registers queue

A message structure of a MODBUS TCP client query for reading input registers is shown in Figure 11.. TheThe slaveslave repliesreplies toto thethe mastermaster queryquery inin thethe samesame formatformat withwith thethe readread registersregisters usingusing thethe function code (FC) “read input registers” (FC(FC = 0x04 0x04),), or as a confirmation confirmation to executing commands in case of other function codes such as “write single coil”coil” (FC(FC == 0x05). The header of the MODBUS frame consists of four fields: fields: a two-byte transaction identifier identifier (ID); (ID); a two-byte protocol type (MODBUS over TCP); a two-bytetwo-byte length, which counts the number of bytes for thethe restrest fields; fields; and and a one-bytea one-byte unit unit identifier identifier (Unit). (Unit). However, However, the protocol the pr dataotocol unit data (PDU) unit consists (PDU) ofconsists a one-byte of a one-byte function function code (FC), code which (FC), is, which here, ais, code here, to a read code the to registersread the andregisters a data and field a data that field may containthat may other contain fields other depending fields ondepending the FC itself. on the Both FC the itself. header Both and the the header PDU form and anthe application PDU form data an unitapplication (ADU), data which unit is the(ADU), complete which frame is the ofcomplete the query. frame of the query. The followingfollowing illustrativeillustrative example example explains explains the th principlee principle of theof the MODBUS MODBUS frame frame format format that usesthat auses function a function code code (0x04) (0x04) to read to read three three continuous continuo inputus input registers registers in ain remote a remote device. device. The The function function is ableis able to to read read from from 1 1 to to 125125 contiguouscontiguous input registers. registers. Here, Here, a aclient client query query asks asks a server a server to read to read the thevalues values of three of three continuous continuous input input registers—regi registers—registerster address address “14” “14”(0x000E), (0x000E), register register address address “15” “15”(0x000F), (0x000F), and andregister register address address “16” “16” (0x0010). (0x0010). Therefore, Therefore, the the client client sends sends a asingle single message “0001000000060104000E0003”“0001000000060104000E0003” andand the serverthe repliesserver by sendingreplies one frameby “000100000009010406FE20sending one frame 6666A63F”“000100000009010406FE206666A63F” that contains three values of that continuous contains registers. three values The first of register continuous contains registers. the hexadecimal The first valueregister “0xFE20,” contains whichthe hexadecimal corresponds value to the “0xFE20,” sixteen-bit which signed corresponds short integer to the value sixteen-bit “11111110 signed 00100000” short orinteger the decimalvalue “11111110 value “–480”. 00100000” The or last thetwo decimal registers value hold “–480”. the The IEEE last 754 two short registers floating-point hold the IEEE [45] representation754 short floating-point “0x3FA66666” [45] representation or the decimal “0x3FA66666” value “1.3”. or the decimal value “1.3”.

Figure 1. MODBUS query and response, an illustrative example.

Using the MODBUSMODBUS speciﬁcationsspecifications [30[30],], the the information information from from Table Table1, and1, and the the MODBUS MODBUS frame frame of Figureof Figure1, the1, the total total consumed consumed bytes bytes (size) (size) can can be be calculated calculated via via Equation Equation (1). (1). The The formula formula is is derived derived for the function code of “read input registers” (0x04) and is plotted in Figure 2 2,, wherewhere NN refersrefers toto thethe number of registers, which is doubled becausebecause eacheach registerregister containscontains twotwo bytes:bytes:

Size = request_bytes + response_bytes = (7 + 5) + (9 + 2 × N) . (1)

= 2 × N + 21

For the request case, the header occupies 7 bytes, the function code occupies 1 byte, the start register-address occupies 2 bytes, and the quantity occupies 2 bytes. For the response case, the header occupies 7 bytes, the function code 1 byte, and the length 1 byte. Future Internet 2019, 11, x FOR PEER REVIEW 5 of 18

Sizerequest _ bytes response _ bytes ()(*N)75 92 . (1) 221*N For the request case, the header occupies 7 bytes, the function code occupies 1 byte, the start register-address occupies 2 bytes, and the quantity occupies 2 bytes. For the response case, the

Futureheader Internet occupies2019, 11 ,7 66 bytes, the function code 1 byte, and the length 1 byte. 5 of 18

FigureFigure 2. 2.Relationship Relationship between between the the number number ofbytes of bytes and an thed numberthe number of registers of registers of function of function codes (FC)codes “read(FC)input “read registers” input registers” (0x04), “write (0x04), multiple “write coils”multiple (0x0F), coils” and (0x0F), “write multipleand “write registers” multiple (0x10). registers” (0x10). The same principle is applied to other function codes, such as “read holding registers” (0x03), whichThe has thesame same principle linear equation;is applied “write to other multiple function registers” codes, (0x10),such as which “read can holding be represented registers” by (0x03), the linearwhich equation has the (2same∗N + linear 25); and equation; “write “write multiple multiple coils” (0x0F),registers” which (0x10), can bewhich represented can be represented by the linear by equationthe linear (N equation+ 25). As (2* anN example, + 25); and if the “write number multiple of registers coils” ((0x0F),N) is 4, which the size can is 29be for represented the “read inputby the registers”linear equation (0x04) and(N +33 25). for As the an “write example, multiple if the registers”number of (0x10) registers function (N) is codes. 4, the size is 29 for the “read inputIf theregisters” client application (0x04) and 33 requires for the the “write execution multiple of bothregisters” operations—read (0x10) function and codes. write—within the same messageIf the client for application a remote device, requir thees the function execution code of “read/write both operations—read multiple registers” and write—within (0x17) shows the highersame performancemessage for a than remote both device, the “read the function input registers” code “read/write (0x04) and multiple the “write registers” multiple (0x17) registers” shows (0x10)higher function performance codes ifthan used both separately. the “read Equation input regist (2) illustratesers” (0x04) two and linear the formulae“write multiple for the functionregisters” code(0x10) “read/write function codes multiple if used registers” separately. (0x17), Equation whichsends (2) illustrates both commands two linear within formulae the same for the message: function code “read/write multiple registers” (0x17), which sends both commands within the same message: Bytes_request = 17 + 2 × NRead Bytes_*N request 17 2 Read = + × N . (2) Bytes_response 9 2 Write . Bytes_*N response 9 2 Write (2) ∴ Total_size = 26 + 2 × (NRead + NWrite) Total_*NN size 26 2 Read Write Both equations of the function code “read/write multiple registers” (0x17) request and response Both equations of the function code “read/write multiple registers” (0x17) request and response are plotted in Figure3. Both equations are linear and depend on the number of registers to be written are plotted in Figure 3. Both equations are linear and depend on the number of registers to be written (NWrite) and read (NRead). (NWrite) and read (NRead). As a result, the MODBUS TCP has an optimized frame structure suitable for SCADA-like systems As a result, the MODBUS TCP has an optimized frame structure suitable for SCADA-like and has a communication mechanism that fulﬁlls the industrial requirements. In addition, it has a systems and has a communication mechanism that fulfills the industrial requirements. In addition, it communication model and pattern that are compatible with industrial applications. As shown in has a communication model and pattern that are compatible with industrial applications. As shown Figures1–3, the protocol is lightweight. Moreover, it has an open speciﬁcation, and uses TCP/IP networks. Accordingly, it can be considered as an IoT protocol. Future Internet 2019, 11, x FOR PEER REVIEW 6 of 18

in Figures 1–3, the protocol is lightweight. Moreover, it has an open specification, and uses TCP/IP

Futurenetworks. Internet Accordingly,2019, 11, 66 it can be considered as an IoT protocol. 6 of 18

FigureFigure 3. 3.Relationship Relationshipbetween betweenthe thenumber numberof of bytes bytes and and the the number number of of registers registers of of the the MODBUS MODBUS functionfunction code code (FC) (FC) “read/write “read/write multiple multiple registers” registers” (0x17). (0x17). 4. Comparison between IoT Protocols 4. Comparison between IoT Protocols A performance comparison between HTTP and MQTT is conducted in [46] on required network A performance comparison between HTTP and MQTT is conducted in [46] on required resources for IoT, while the payload was fixed to zero bytes and the topic names were varied. network resources for IoT, while the payload was fixed to zero bytes and the topic names were In addition, a performance analysis of MQTT, HTTP, and CoAP was performed in [47] for IoT-based varied. In addition, a performance analysis of MQTT, HTTP, and CoAP was performed in [47] for monitoring of a smart home. The authors of [48] discussed and analyzed the efficiency, usage, and IoT-based monitoring of a smart home. The authors of [48] discussed and analyzed the efficiency, requirements of MQTT and CoAP. Moreover, the authors of [49] compared AMQP and MQTT over usage, and requirements of MQTT and CoAP. Moreover, the authors of [49] compared AMQP and mobile networks, and the authors of [50] emulated a quantitative performance assessment of CoAP in MQTT over mobile networks, and the authors of [50] emulated a quantitative performance comparison with HTTP. assessment of CoAP in comparison with HTTP. In this section, the main differences between HTTP, CoAP, MQTT, AMQP, XMPP, and MODBUS In this section, the main differences between HTTP, CoAP, MQTT, AMQP, XMPP, and TCP protocols are discussed from various telecommunication aspects. Then, a protocol is chosen that MODBUS TCP protocols are discussed from various telecommunication aspects. Then, a protocol is fulfills the requirements of the IIoT environment. chosen that fulfills the requirements of the IIoT environment. Accordingly, Table2 summarizes these differences from different communication aspects Accordingly, Table 2 summarizes these differences from different communication aspects including infrastructure, architecture, mechanism, model, messaging pattern, methodology, and including infrastructure, architecture, mechanism, model, messaging pattern, methodology, and transmission paradigm. These protocols use the client–server communication architecture. HTTP transmission paradigm. These protocols use the client–server communication architecture. HTTP uses the request–response model and is a document-oriented protocol, whereas MQTT uses the uses the request–response model and is a document-oriented protocol, whereas MQTT uses the publish–subscribe model and is message-oriented. Thus, MQTT is one-to-many, and HTTP publish–subscribe model and is message-oriented. Thus, MQTT is one-to-many, and HTTP is is one-to-one (peer-to-peer). CoAP uses a specific infrastructure—namely, 6LoWPAN (IEEE one-to-one (peer-to-peer). CoAP uses a specific infrastructure—namely, 6LoWPAN (IEEE 802.15.4)—which employs IPv6 in the network layer. Both MQTT and HTTP use an inexpensive and 802.15.4)—which employs IPv6 in the network layer. Both MQTT and HTTP use an inexpensive and available communication infrastructure, which is Internet or Intranet in wire mode (Ethernet—IEEE available communication infrastructure, which is Internet or Intranet in wire mode (Ethernet—IEEE 802.3) or wireless mode (Wi-Fi—IEEE 802.11)—which may employ either IPv4 or IPv6 in the network 802.3) or wireless mode (Wi-Fi—IEEE 802.11)—which may employ either IPv4 or IPv6 in the layer. In the transport layer, MQTT and HTTP protocols use TCP port numbers 1883 and 80, respectively. network layer. In the transport layer, MQTT and HTTP protocols use TCP port numbers 1883 and 80, However, CoAP uses UDP port number 5683. Given that MQTT is event-based, it is a message-oriented respectively. However, CoAP uses UDP port number 5683. Given that MQTT is event-based, it is a protocol. Thus, CoAP mimics HTTP in using polling-based messaging, but in a shorter time and message-oriented protocol. Thus, CoAP mimics HTTP in using polling-based messaging, but in a smaller frame-size. shorter time and smaller frame-size.

Future Internet 2019, 11, x FOR PEER REVIEW 7 of 18 Future Internet 2019, 11, 66 7 of 18

Table 2. Comparison of Internet of Things (IoT) protocols. Table 2. Comparison of Internet of Things (IoT) protocols. Feature HTTP CoAP MQTT MODBUS TCP infrastructureFeature Ethernet, HTTP Wi-Fi 6LoW CoAPPAN Ethernet, MQTT Wi-Fi Ethernet, MODBUS Wi-Fi TCP infrastructurenetwork layer IPv4 Ethernet, or IPv6 Wi-Fi 6LoWPAN IPv6 IPv4 Ethernet, or IPv6 Wi-Fi IPv4 Ethernet, or IPv6 Wi-Fi networktransport layer layer IPv4 TCP or IPv6 UDP IPv6 IPv4 TCP or IPv6 IPv4 TCP or IPv6 transporttransport layer port 80, TCP443 5683 UDP 1883, TCP8883 502, 802 TCP transportmodel port synchronous 80, 443 asynchronous 5683 asynchronous 1883, 8883 synchronous 502, 802 modelpattern request—response synchronous asynchronous both publish—subscribe asynchronous request—response synchronous pattern request—response both publish—subscribe request—response mechanism one-to-one one-to-one one-to-many one-to-one mechanism one-to-one one-to-one one-to-many one-to-one methodology document-oriented document-oriented message-oriented byte-oriented methodology document-oriented document-oriented message-oriented byte-oriented paradigmparadigm long long polling-based polling-based polling-based polling-based event-based event-based polling-based polling-based qualityquality levellevel one one level level two: two: CON CON or orNON NON three: three: QoS QoS 0, 0,1, 1,2 2 one one level level standardstandard IETF IETF (RFC7230) (RFC7230) IETF IETF (RFC (RFC7252)7252) ISO/IEC, ISO/IEC, OASIS OASIS modbus.org modbus.org encodingencoding ASCII ASCII text text RESTful RESTful (Binary) (Binary) UTF-8 UTF-8 (Binary) (Binary) Binary Binary securitysecurity SSL, SSL, TLS TLS DTLS DTLS SSL, SSL, TLS TLS TLS TLS

CoAP is an application layer protocol, dedicated to communication with constrained devices in IPv6-basedCoAP isIoT an applicationinfrastructures. layer protocol,Two communi dedicatedcation to communication patterns are with used constrained by CoAP, devices i.e., in publish–subscribeIPv6-based IoT infrastructures. and request–response Two communication [51]. The Co patternsAP messaging are used pattern by CoAP, is based i.e., publish–subscribe on the exchange ofand messages request–response between endpoints [51]. The CoAP and uses messaging a short patternfixed-length is based binary on the header exchange that ofmay messages be followed between by aendpoints compact andbinary uses option a short and ﬁxed-length a payload. binary Compared header to that HTTP, may as be shown followed in byFigure a compact 4, CoAP binary runs option over theand connectionless a payload. Compared UDP in tothe HTTP, transport as shown layer, in wherea Figures4 ,in CoAP the network runs over layer, the connectionlessCoAP uses either UDP IPv6 in orthe 6LoWPAN. transport layer, When whereas CoAP uses in the IPv6, network it is necessary layer, CoAP for usesit to eitheruse Ethernet IPv6 or or 6LoWPAN. Wi-Fi for the When data CoAP link anduses physical IPv6, it is layers, necessary respectively. for it to use When Ethernet CoAP or Wi-Fi uses for6LoWPAN, the data link it employs and physical IEEE layers, 802.15.4e respectively. for the Whendata link CoAP and uses physical 6LoWPAN, layers. it employs IEEE 802.15.4e for the data link and physical layers. The content (payload) of HTTP may vary according to the type of transferred data, called content-type, which which could could be be plain plain text, text, HTML, HTML, XML, XML, GIF image, GIF image, PDF application, PDF application, or audio. or For audio. the exchangeFor the exchange of data ofusing data usingHTTP, HTTP, XML XMLis used, is used, which which handles handles verbose verbose plain plain text text for for solving interoperability issues. issues. However, However, for for CoAP, CoAP, the effi efﬁcientcient XML interchange (EXI) [52] [52] is used, which encodes verbose XML documents in binary format, if interoperability is considered. This is normally used for constrained devices to increase the perf performanceormance and and decrease decrease the the consumed consumed power. power. Hence, Hence, CoAP isis suitable suitable for for constrained constrained devices devices in IoT-based in IoT-based wireless wireless sensor networks sensor networks that employ that IPv6-based employ IPv6-basedinfrastructure. infrastructure. However, it However, needs a gateway it needs to a exchangegateway to data exchange over the data Internet. over the Internet.

(a) (b) (c)

Figure 4. Communication protocols in the IEEE model (a); the HTTP (b); the CoAP (c). Figure 4. Communication protocols in the IEEE model (a); the HTTP (b); the CoAP (c).

Future Internet 2019, 11, 66 8 of 18

There are two common features and two main differences between MQTT and CoAP. Both target constrained devices and networks, and both have low data overhead. Importantly, MQTT is one-to-many and TCP-based message-oriented, whereas CoAP is one-to-one and UDP-based. Since TCP is connection-oriented, and UDP is connectionless, MQTT is more reliable. Moreover, CoAP needs a dedicated communication infrastructure based on IPv6 networks in addition to a dedicated gateway to pass content over the Internet. Hence, CoAP was not used for this IIoT environment. In the same manner, HTTP, which is polling-based, was not considered for this study because of its philosophy that uses the synchronous communication model for peer-to-peer and request–response exchange of data. XMPP is an XML-based messaging protocol that is able to transfer verbose messages, audio and video signals in chat conversations. In addition, it supports request–response as well as an event-based communication pattern. However, the XML-based verbose messages of XMPP increase the message size of SCADA-like applications, which have byte-oriented messages, and hence, cannot be used for IIoT efficiently. As a comparison, MODBUS has small-sized data units, with a maximum of 255 bytes, which are suitable for automation, telecontrol, and monitoring, whereas XMPP messages need more than 400 bytes of overhead. To summarize, many IoT protocols exist, and event-based protocols are of considerable interest for transferring data as notifications to complement the MODBUS TCP. This MODBUS protocol is polling-based, synchronous, request–response, and optimized for control and monitoring in industrial applications. It can establish an IIoT environment, either on a standalone basis or in conjunction with an event-based protocol to cover the publish–subscribe mechanism if needed. MQTT is able to complement the MODBUS TCP via its asynchronous model, event-based paradigm, and publish–subscribe pattern. Alternatively, HTTP uses a request–response mechanism and, hence, was not considered for this study. CoAP was also found to be not suitable for this scenario because it needs a specific infrastructure, and hence, a gateway to pass data over the Internet, which adds more costs and causes complications to the environment. In addition, XMPP was not considered here, because it is XML-based verbose protocol that requires a large overhead for small-sized industrial packets. Moreover, AMQP was eliminated because it is dedicated to the exchange of business messages between two entities. This protocol is used normally for application-to-application integration at the enterprise level, which is higher than the level of both MODBUS TCP and MQTT. Transport Layer Security (TLS) and its predecessor, Secure Sockets Layer (SSL), have been designed to provide security over a TCP/IP network. If HTTP uses SSL or TLS, it employs port 443. This is also applicable to MODBUS TCP and MQTT, which employ ports 802 and 8883, respectively.

5. Comparison between MODBUS TCP and MQTT In this section, a comparison between the MODBUS TCP and the MQTT protocol is conducted from three aspects. These aspects are the communication model of the protocol in the original IEEE model, the message exchange philosophy, and the required number of bytes for some message types that demonstrate the overhead of each type. However, the other protocols are not compared for the reasons listed above. As shown in Figure5, the MQTT and the MODBUS TCP protocols are both in the same level in the IEEE model. While the MQTT protocol encodes the user data in UTF-8, the MODBUS TCP uses a byte-encoded frame format for the user data, which is intended for industrial applications. Figure6 illustrates a comparison between MQTT philosophy and MODBUS philosophy, from the perspective of the exchange of messages. The request of the MODBUS query uses a TCP-based connection and employs a frame-format based on an application-layer message structure, which is optimized and dedicated for telecontrol and monitoring. The case is different with MQTT; while the ﬁrst client (publisher) produces an event using four messages, the second client (subscriber) consumes that event in six messages. Future Internet 2019, 11, 66 9 of 18 Future Internet 2019, 11, x FOR PEER REVIEW 9 of 18 Future Internet 2019, 11, x FOR PEER REVIEW 9 of 18

(a()a ) (b) (c)( c)

Figure 5. The IEEE model (a); compared to the MODBUS TCP (b); and the MQTT (c). FigureFigure 5. 5. The The IEEE IEEE model model ( (aa);); comparedcompared to thethe MODBUSMODBUS TCP TCP ( b(b);); and and the the MQTT MQTT (c ).(c ). The publisher sends a connect packet (CONNECT), with username (un) and password (pwd) if The publisher sends a connect packet (CONNECT), with username (un) and password (pwd) if required,The publisher to the server sends (broker) a connect trying packet to establish (CONNECT), a TCP with connection. username The (un) server and password acknowledges (pwd) the if required, to the server (broker) trying to establish a TCP connection. The server acknowledges the attemptrequired, with to the a CONNACK server (broker) packet, trying telling to theestabl clientish (publisher)a TCP connection. whether The the connectionserver acknowledges is successfully the attempt with a CONNACK packet, telling the client (publisher) whether the connection is established.attempt with Then, a CONNACK the client publishes packet, thetelling temperature the client value, (publisher) for example, whether via athe PUBLISH connection packet, is successfully established. Then, the client publishes the temperature value, for example, via a withsuccessfully temp topic established. and a value Then, of 22.7. the Theclient client publishes ends the the publish temperature event with value, the serverfor example, by sending via a PUBLISHPUBLISH packet, packet, with with temp temp topic topic andand aa valuevalue of 22.7. TheThe clientclient ends ends the the publish publish event event with with the the DISCONNECTserver by sending packet. a DISCONNECT packet. server by sending a DISCONNECT packet. Meanwhile,Meanwhile, and and inin additionaddition to to the the aforementioned aforementioned steps, steps, the thesubscriber subscriber must must SUBSCRIBE SUBSCRIBE to the to Meanwhile, and in addition to the aforementioned steps, the subscriber must SUBSCRIBE to the thesame same topic topic (temp) (temp) to to receive receive the the published published mess messagesages of of interest. interest. The The subscription subscription packet packet is is same topic (temp) to receive the published messages of interest. The subscription packet is acknowledgedacknowledged with with the the SUBACK SUBACK packet.packet. When the the broker broker receives receives the the message message that that handles handles the the acknowledged with the SUBACK packet. When the broker receives the message that handles the temperaturetemperature value, value, via via the the same same PUBLISHPUBLISH packet, it it forwards forwards it it to to the the subscriber, subscriber, which which may may then then terminatetemperatureterminate the the value, connection connection via the using using same the the PUBLISH DISCONNECT DISCONNECT packet, message.it forwards it to the subscriber, which may then terminate the connection using the DISCONNECT message.

(a) (b) Figure 6. Comparison of protocols for the exchange of messages: (a) MQTT; (b) MODBUS TCP. Figure 6. Comparison(a) of protocols for the exchange of messages: (a) MQTT; (b) MODBUS (b) TCP. As a comparison between MODBUS TCP and MQTT messages, Figure7 shows the required FigureAs a 6. comparison Comparison between of protocols MODBUS for the exchangeTCP and of MQTT messages: messages, (a) MQTT; Figure (b) MODBUS7 shows theTCP. required number of bytes of the MODBUS messages, compared to those of MQTT. The message of the MODBUS number of bytes of the MODBUS messages, compared to those of MQTT. The message of the TCP requires 27 bytes to read the values of three registers using the function code “read input registers” MODBUSAs a comparison TCP requires between 27 bytes MODBUS to read the TCP values and ofMQTT three registersmessages, using Figure the function7 shows codethe required “read (FCnumberinput = 0x04), registers”of bytes as shown (FCof the in= 0x04), Figures MODBUS as1 andshown messages,2. However, in Figures compared the 1 and message 2. toHowever, ofthose the MODBUSof the MQTT. message TCP The of requires themessage MODBUS 29 bytesof the MODBUS TCP requires 27 bytes to read the values of three registers using the function code “read input registers” (FC = 0x04), as shown in Figures 1 and 2. However, the message of the MODBUS

Future Internet 2019, 11, 66 10 of 18

Future Internet 2019, 11, x FOR PEER REVIEW 10 of 18 Future Internet 2019, 11, x FOR PEER REVIEW 10 of 18 to write the values of two registers using the function code “write multiple registers” (FC = 0x10) in a TCP requires 29 bytes to write the values of two registers using the function code “write multiple remoteTCP requires device. In29 bothbytes cases,to write the the MODBUS values of TCP two issuesregisters a queryusing fromthe function the client code side “write and multiple a response registers” (FC = 0x10) in a remote device. In both cases, the MODBUS TCP issues a query from the fromregisters” the server (FC side.= 0x10) Meanwhile, in a remote if device. the function In both code cases, “read/write the MODBUS multiple TCP issues registers” a query (0x17) from is the used client side and a response from the server side. Meanwhile, if the function code “read/write multiple for both operations, the total consumed bytes for reading three registers and writing two registers registers” (0x17) is used for both operations, the total consumed bytes for reading three registers and within the same connection is 36 bytes. For MQTT, in this example, publishing a 10-byte payload by writing two registers within the same connection is 36 bytes. For MQTT, in this example, publishing the producer and consuming it by the subscriber requires 115 bytes. It is important to mention that the a 10-byte payload by the producer and consuming it by the subscriber requires 115 bytes. It is total consumed bytes of MQTT publish and subscribe messages depends on the topic length and the important to mention that the total consumed bytes of MQTT publish and subscribe messages length of the user data. depends on the topic length and the length of the user data.

FigureFigure 7. 7.Required Required bytesbytes ofof MQTTMQTT messages compared compared to to those those of of MODBUS. MODBUS.

FigureFigure8 shows8 shows the the overhead overhead portion portion ofof thethe threethree messages for for a a constant constant payload payload of of10 10 bytes. bytes. TheThe function function code code “read/write “read/write multiplemultiple registers” (0x1 (0x17)7) has has an an overhead overhead of of26 26 bytes bytes out out of 36 of 36bytes. bytes. TheThe function function codes codes “read “read input input registers”registers” (0x04)(0x04) an andd “write “write multiple multiple registers” registers” (0x10) (0x10) have have a total a total overheadoverhead of of 46 46 bytes bytes out out of of 56-bytes 56-bytes (the(the totaltotal size).size). In comparison, the the MQTT MQTT publish–subscribe publish–subscribe messagemessage has has an an overhead overhead of of 105 105 bytes bytes outout ofof 115115 bytes.

Figure 8. MQTT and MODBUS overheads for a payload of 10 bytes. FigureFigure 8. 8.MQTT MQTT andand MODBUSMODBUS overheads for for a a payload payload of of 10 10 bytes. bytes.

Future Internet 2019, 11, x FOR PEER REVIEW 11 of 18 Future Internet 2019, 11, 66 11 of 18 The result of Figure 8 is illustrated in Figure 9 for values between 2 bytes and 106 bytes. The figureThe demonstrates result of Figure the8 is overhead illustrated percentage in Figure9 forwith values respect between to the 2 bytestotal andmessage 106 bytes. size according The figure to demonstratesEquation (3): the overhead percentage with respect to the total message size according to Equation (3): Overhead Overhead(%)Overhead 100% . (3) Overhead(%) = Overhead  Payload× 100%. (3) Overhead + Payload The percentage overheads of the aforementioned function codes are calculated and plotted in FigureThe 9. percentage The figure overheads shows values of the for aforementioned fewer than 106 function bytes because codes arethe calculatedmessage size and in plotted industrial in Figureapplications9. The figureand SCADA-like shows values systems for fewer is small. than MQTT 106 bytes has becausethe highest the messageoverhead; size the infunction industrial code applications“read input andregisters” SCADA-like (0x04) has systems the least is small. overhead MQTT (21 has bytes), the highestand the overhead;function code the “write function multiple code “readregisters” input (0x10), registers” which (0x04) is covered has the leastby the overhead curve of (21 the bytes), function and code the function “read/write code multiple “write multiple registers registers”(0x17), has (0x10), 25 bytes which of overhead. is covered However, by the curve if the of theapplication function client code reads “read/write and writes multiple registers registers in the (0x17),same hasmessage, 25 bytes the offunction overhead. code However, “read/write if the multip applicationle registers” client (0x17) reads shows and writes lower registers overhead in thethan samethe function message, codes the function “read input code registers” “read/write (0x04) multiple and “w registers”rite multiple (0x17) registers” shows lower(0x10) overheadif both are than used theseparately function for codes the “readsame inputpurpose. registers” (0x04) and “write multiple registers” (0x10) if both are used separatelyAccordingly, for the same the MQTT purpose. protocol is suitable for IoT-based publish–subscribe applications, but it is unableAccordingly, to replace the MQTT the MODBUS protocolis TCP, suitable which for IoT-basedfulfills the publish–subscribe industrial requirements applications, and butuses it isan unableoptimized to replace frame-format the MODBUS for TCP,request–response which fulfills thecommunications industrial requirements between andindustrial uses an clients optimized and frame-formatservers. These for properties request–response make it communicationssuitable for SCADA-like between systems, industrial automation, clients and monitoring, servers. These and propertiescontrol. In make Section it suitable 6, performance for SCADA-like of MQTT systems, and the automation, MODBUS monitoring, TCP are tested and control.and compared In Section based6, performanceon the Round-Trip of MQTT time and (RTT) the MODBUS measurements TCP are an testedd the central and compared processing based unit on (CPU) the Round-Trip usage. time (RTT) measurements and the central processing unit (CPU) usage.

FigureFigure 9. 9.The The overhead overhead of of an an MQTT MQTT message message compared compared to to that that of of MODBUS. MODBUS. 6. Latency and CPU Usage 6. Latency and CPU Usage In order to support the results of Section5, the performance of MQTT and the MODBUS TCP are In order to support the results of Section 5, the performance of MQTT and the MODBUS TCP compared in this section. This performance is based on the Round-Trip Time (RTT) measurements and are compared in this section. This performance is based on the Round-Trip Time (RTT) the CPU usage. A Java-based MQTT publisher was developed and installed on a laptop that runs the measurements and the CPU usage. A Java-based MQTT publisher was developed and installed on a operating system Linux Ubuntu 16.04. The speciﬁcations of the laptop were: Dell-Inspiron 3537, Intel® laptop that runs the operating system Linux Ubuntu 16.04. The specifications of the laptop were: CoreTM i5-4200U CPU at 1.6 GHz × 4, 5.85 GiB RAM. The server (broker) was the Apache ActiveMQ Dell-Inspiron 3537, Intel® CoreTM i5-4200U CPU at 1.6 GHz × 4, 5.85 GiB RAM. The server (broker) software, which was installed on a desktop PC that runs the operating system Microsoft Windows XP. was the Apache ActiveMQ software, which was installed on a desktop PC that runs the operating The desktop PC featured Pentium Dual Core CPU at 3.2 GHz and 2 GB RAM. The consumer was the system Microsoft Windows XP. The desktop PC featured Pentium Dual Core CPU at 3.2 GHz and 2 MQTT.FX software, which was installed on the desktop PC as well. The RTT values measured from the GB RAM. The consumer was the MQTT.FX software, which was installed on the desktop PC as well.

Future Internet 2019, 11, x 66 FOR PEER REVIEW 1212 of of 18 18

The RTT values measured from the publisher and the CPU usage measured from the desktop PC are shownpublisher in Figure and the 10. CPU The usage publisher measured that fromconnected the desktop to the server PC are (broker) shown in started Figure counting 10. The publisher the RTT fromthat connected the time of to creating the server the (broker) socket to started closing counting it, which the includes RTT from the the time time needed of creating for formatting the socket the to messageclosing it, in which a publish includes message, the time connecting needed for to formatting the server the without message a in logon a publish process, message, publishing connecting the message,to the server and withoutdisconnecting a logon from process, the server, publishing in ad thedition message, to the andnetwork disconnecting latency. The from CPU the server,usage valuesin addition are the to themaximum network registered latency. The values, CPU usageand the values tests arewere the conducted maximum for registered payload values, sizes of and 5, the10, 25,tests 50, were and conducted100 bytes. for payload sizes of 5, 10, 25, 50, and 100 bytes.

(a) (b)

Figure 10. MQTT-related measurements: (a) Round-Trip Time (RTT); (b) CPU usage. Figure 10. MQTT-related measurements: (a) Round-Trip Time (RTT); (b) CPU usage. The RTT measurements of the MODBUS TCP were also conducted using the same network. The RTT measurements of the MODBUS TCP were also conducted using the same network. A A MODBUS TCP client was developed and installed on the laptop, and a MODBUS TCP server was MODBUS TCP client was developed and installed on the laptop, and a MODBUS TCP server was developed and installed on the desktop PC. Many tests of function codes were carried out for “read developed and installed on the desktop PC. Many tests of function codes were carried out for “read holding registers” (FC = 0x03), “read input registers” (FC = 0x04), and “write multiple registers” holding registers” (FC = 0x03), “read input registers” (FC = 0x04), and “write multiple registers” (FC (FC = 0x10). The total transmitted bytes in each transaction were 27 bytes, 33 bytes, and 35 bytes = 0x10). The total transmitted bytes in each transaction were 27 bytes, 33 bytes, and 35 bytes respectively; the total was 95 bytes when all messages were transmitted within one transaction. respectively; the total was 95 bytes when all messages were transmitted within one transaction. The The RTT values, which were measured on the client side, were between 6 ms and 8 ms, and the CPU RTT values, which were measured on the client side, were between 6 ms and 8 ms, and the CPU usage, which was measured on the server side, had a maximum value of 3%. usage, which was measured on the server side, had a maximum value of 3%. These results confirm the theoretical outcomes of Section5, which show that the MODBUS TCP is These results confirm the theoretical outcomes of Section 5, which show that the MODBUS TCP a lightweight protocol suitable for SCADA-like systems and industrial applications. The MODBUS is a lightweight protocol suitable for SCADA-like systems and industrial applications. The MODBUS TCP client connects to the MODBUS TCP server using polling communications (direct one-to-one TCP client connects to the MODBUS TCP server using polling communications (direct one-to-one communications). However, the MQTT client connects to another client via a server, called a broker. communications). However, the MQTT client connects to another client via a server, called a broker. This fact explains the high CPU consumption while the MQTT server was busy in receiving and This fact explains the high CPU consumption while the MQTT server was busy in receiving and redirecting the publish messages. redirecting the publish messages. 7. Discussion 7. Discussion Based on the comparisons of Sections4–6 as well as the theoretical study of the MODBUS TCP in SectionBased3, two on scenariosthe comparisons are discussed of Sections here to4–6 build as well the as industrial the theoretical IoT environment. study of the MODBUS TCP in SectionThe first 3, two scenario scenarios is to are employ discussed only thehere MODBUS to build the TCP industrial to build theIoT IIoTenvironment. environment. The study of SectionThe first3 proved scenario that is MODBUSto employ TCPonly isthe able MODBUS to react TCP as an to IoT build protocol. the IIoT Security environment. is a fundamental The study ofissue Section in IoT, 3 proved which wasthat solvedMODBUS in [32 TCP] for is MODBUS able to react TCP as using an IoT TLS. protocol. As mentioned Security above is a fundamental in Section5, issuethe MODBUS in IoT, which TCP was is able solved to build in [32] the for IIoT MODBUS environment TCP using using TLS. the As synchronous mentioned request–response above in Section 5,communication the MODBUS patternTCP is able on a to standalone build the basis,IIoT environment which is a scenario using the that synchronous complies with request–response most industrial communicationapplications. However, pattern thison a solutionstandalone totally basis, relies which on theis a polling-basedscenario that complies mechanism. with most industrial applications.In the secondHowever, scenario, this solution the event-basedtotally relies on mechanism the polling-based is fulfilled mechanism. by MQTT, where M2M communicationsIn the second are required,scenario, usingthe event-based the asynchronous mechanism publish–subscribe is fulfilled communication by MQTT, where pattern. M2M For communicationsindustrial functions, are therequired, MODBUS using TCP the is employed,asynchronous which publish–subscribe fulfills the synchronous communication request–response pattern. Forcommunication industrial functions, pattern. In the this MODBUS scenario, MQTTTCP worksis employed, in parallel which with thefulfills MODBUS the synchronous TCP within request–responsethe same platform, communication as shown in Figure pattern. 11. ThisIn this figure scenario, illustrates MQTT the simulation works in environmentparallel with of the the MODBUSsecond scenario. TCP within The environment the same consistedplatform, of as a desktopshown PCin runningFigure 11. Microsoft This figure Windows illustrates XP, a laptop the simulationrunning Linux environment Ubuntu 16.04,of the andsecond an scenario. LAN/Internet The environment network. The consisted desktop of PC a desktop is equipped PC running with a Microsoft Windows XP, a laptop running Linux Ubuntu 16.04, and an LAN/Internet network. The

FutureFuture InternetInternet 20192019,,, 111111,,, 66xx FORFOR PEERPEER REVIEWREVIEW 13 1313 of of 18 18

desktop PC is equipped with a java-based MODBUS TCP client, the ActiveMQ MQTT server, and java-based MODBUS TCP client, the ActiveMQ MQTT server, and the MQTT.FX consumer. The laptop thethe MQTT.FXMQTT.FX consumer.consumer. TheThe laptoplaptop waswas equippedequipped wiwithth aa multithreadedmultithreaded JavaJava program;program; oneone threadthread was equipped with a multithreaded Java program; one thread ran the MQTT publisher, and another ranran thethe MQTTMQTT publisher,publisher, andand anotheranother threadthread ranran thethe MODBUSMODBUS TCPTCP server.server. InIn addition,addition, anan onlineonline thread ran the MODBUS TCP server. In addition, an online MQTT broker and consumer, provided by MQTT broker and consumer, provided by HiveMQ for testing, were employed here as an HiveMQ for testing, were employed here as an Internet-based server. Internet-basedInternet-based server.server.

Figure 11.11. SimulationSimulation environment environment ofof thethe secondsecond scenario.scenario.

TheThe RTTRTT measurements measurements and and the the CPU CPU usage usage of this of environment this environment were measured were measured for the MODBUS for the TCPMODBUS and then TCP compared and then to compared the previous to the results previous of Section resultsresults6. The ofof SectionSection RTT values 6.6. TheThe of theRTTRTT MODBUS valuesvalues ofof TCP, thethe whichMODBUS were TCP, measured which on were the desktopmeasured PC, on had the a maximumdesktopsktop PC,PC, value hadhad of aa 9 maximummaximum ms. This represents value of an9 ms. increase This ofrepresentsrepresents maximum anan 12.5% increaseincrease from ofof themaximummaximum previous 12.5%12.5% RTT fromfrom values thethe presented previous in RTT Section values6. As presented illustrated in Section in Figure 6. 12As, theillustratedillustrated CPU history inin FigureFigure of the 12,12, laptop, thethe CPUCPU which historyhistory contains ofof anthethe MQTT laptop,laptop, publisher whichwhich containscontains and a MODBUS anan MQTTMQTT TCP publisherpublisher server, showed andand aa thatMODBUS the CPU TCP usage server, is alwaysshowed less that than the CPU 20%. usage These is results always indicate less than that 20%. the These concurrent results execution indicate ofthatthat MQTT thethe concurrentconcurrent in parallel executionexecution with the of MODBUSof MQTTMQTT inin TCP parallelparallel within with the the same MODBUS platform TCP does within not severelythe same inﬂuence platform thedoes performance not severely of influence the MODBUS the TCP.performance Nevertheless, of the while MODBUS MQTT TCP. fulﬁlls Nevertheless, the event-based while paradigm, MQTT itfulfillsfulfills lacks thethe interoperability. event-basedevent-based paradigm,paradigm, itit lackslacks interoperability.interoperability.

Figure 12. CPU history of the laptop that ran an MQTT publisher andand MODBUSMODBUS TCPTCP server.server.

TheThe MQTTMQTT protocolprotocol formatsformats thethe datadata usingusing thethe UTF-8UTF-8 standard,standard, andand hencehence interoperabilityinteroperability isis notnot fulﬁlled.fulfilled. To To maintainmaintain a homogeneoushomogeneous message for for thethe entireentire environment,environment, industrialindustrial datadata areare organizedorganized usingusing thethe structurestructure ofof MODBUSMODBUS messages,messages, formattedformatted inin thethe UTF-8UTF-8 andand thenthen transferred inin thethe payloadpayload ofof anan MQTTMQTT publishpublish message,memessage,ssage, asas shownshown inin FigureFigure 1313a.13a.a. The The hexadecimal hexadecimal representations representations ofof FigureFigure 1313aa werewere obtainedobtained basedbased onon thethe informationinformationion ofof TableTable3 3..3. DigitDigitDigit numbers numbersnumbers zerozerozero tototo nine ninenine are areare formattedformatted in in UTF-8UTF-8 using using the the hexadecimalhexadecimal represen representation.representation.tation. DigitDigit numbernumber “0”“0” “0” isis representedrepresented byby by “30”“30” andand digitdigit numbernumber “9”“9” isis representedrepresented byby “39”.“39”. Addi Additionally,tionally,tionally, letters letters “A” “A” to to “F” “F” are are representedrepresented by “41”“41” toto “46”,“46”, respectively.respectively. EachEach bytebyte ofof thethe MODBUSMODBUS messagemessage isis representedrepresented byby twotwo bytes,bytes, asas shownshown inin Figure Figure 13.1313.. ForFor For example,example, example, thethe the IDID ID partpart part ofof of thethe the MOMO MODBUSDBUS message message is is“00 “00 01”, 01”, which which leads leads to “30 to “30 30 3030 3031” 31” representation, representation, and and doubles doubles the the payload payload of of thethe the MQTTMQTT MQTT publishpublish publish message.message. TheThe MQTT MQTT publishpublish message was subsequently transmitted to the online hivemq.com server, and the result is illustrated inin FigureFigure 13b.13b.

Future Internet 2019, 11, 66 14 of 18

Futuremessage Internet was 2019 subsequently, 11, x FOR PEER transmitted REVIEW to the online hivemq.com server, and the result is illustrated 14 of in18 Figure 13b. Using the arrangement of Figure 13a,13a, the mess messageage structure, which which is is the MODBUS TCP, TCP, was maintained throughoutthroughout thethe entireentire IIoT IIoT environment environmen fort for both both the the control control part part and and the monitoringthe monitoring part. part.This ledThis to led a uniﬁed to a unified message message structure, structure, uniﬁed unified data processing, data processing, and hence and the hence interoperability the interoperability problem problemwas solved. was solved.

(a) (b)

Figure 13. (a) MODBUS message formatted via UTF-8 in the MQTT payload to solve the interoperability Figure 13. (a) MODBUS message formatted via UTF-8 in the MQTT payload to solve the problem in the second scenario; (b) message as it appeared on hivemq.com consumer. interoperability problem in the second scenario; (b) message as it appeared on hivemq.com consumer. Table 3. UTF-8 encoding characters.

In conclusion,Character the choice UTF-8of solution (Decimal) is totally dependent UTF-8 (Hex) on the requirements Meaning of the industrial application. The 0MODBUS TCP builds 48 the IIoT environment 0x30 if publish–subscribe Digit Zero communications are not required,1 representing a secured 49 solution that 0x31 fulfills the requirements Digit One of most IIoT 2 50 0x32 Digit Two applications. If M2M3 communications 51 are required, MQTT 0x33 can be used, Digitalthough Three it doubles the payload of the publish4 message in order 52 to solve the interoperability 0x34 problem. Digit Four 5 53 0x35 Digit Five 6Table 54 3. UTF-8 encoding 0x36 characters. Digit Six 7 55 0x37 Digit Seven 8Character UTF-8 56 (Decimal) UTF-8 0x38 (Hex) Meaning Digit Eight 90 5748 0x30 0x39 Digit Zero Digit Nine A1 6549 0x31 0x41 Digit Capital One Letter A B 66 0x42 Capital Letter B C2 6750 0x32 0x43 Digit Capital Two Letter C D3 6851 0x33 0x44 Digit CapitalThree Letter D E4 6952 0x34 0x45 Digit CapitalFour Letter E F5 7053 0x35 0x46 Digit Capital Five Letter F 6 54 0x36 Digit Six In conclusion, the7 choice of solution55 is totally dependent0x37 on theDigit requirements Seven of the industrial application. The MODBUS8 TCP builds56 the IIoT environment0x38 if publish–subscribeDigit Eight communications are not required, representing9 a secured solution57 that fulfills0x39 the requirementsDigit Nine of most IIoT applications. If M2M communicationsA are required, 65 MQTT can be 0x41used, although Capital it doublesLetter A the payload of the publish message in orderB to solve the interoperability 66 problem. 0x42 Capital Letter B C 67 0x43 Capital Letter C 8. Conclusions D 68 0x44 Capital Letter D Two scenarios areE introduced to build 69 the industrial 0x45 IoT environment. Capital Letter The first E scenario employs the MODBUS TCP onlyF for synchronous 70 polling communications; 0x46 Capital this solution Letter F complies with most industrial control systems and SCADA-like applications. However, if asynchronous event-based 8.communications Conclusions are required, MQTT complements MODBUS TCP operations. In this particular case, an industrialTwo scenarios IoT environment are introduced requires to build the employment the industrial of at IoT least environment. two protocols—one The first for scenario the IoT employsfunctions, the mainly MODBUS for M2M TCP communications, only for synchronous and another polling for thecommunications; industrial functions. this solution MODBUS complies fulfills withthe industrial most industrial requirements, control mainly systems the and telecontrol, SCADA-like monitoring, applications. and automation However, functions.if asynchronous MQTT event-basedworks in parallel communications with the MODBUS are required, TCP and MQTT complements complements its functions, MODBUS but cannotTCP operations. replace MODBUS. In this particular case, an industrial IoT environment requires the employment of at least two protocols—one for the IoT functions, mainly for M2M communications, and another for the industrial functions. MODBUS fulfills the industrial requirements, mainly the telecontrol,

Future Internet 2019, 11, 66 15 of 18

Since MQTT lacks interoperability, the industrial data are formatted using the structure of MODBUS messages, and then transferred in the payload of the MQTT publish message, which uses the format of the UTF-8. Thus, the message structure of the MODBUS TCP is maintained throughout the entire environment. This solution provides interoperability, but doubles the payload of the industrial data. The simulation results and measurements, presented in Sections6 and7, show that the concurrent execution of MQTT in parallel with the MODBUS TCP within the same platform does not severely inﬂuence the performance of the MODBUS TCP. Security is a fundamental issue in IoT, which was solved in [32] for MODBUS TCP using TLS via TCP port 802. MQTT may also employ TLS for encryption via port 8883. More information on the security of industrial IoT systems can be found in [53–55], on intrusion detection in [56–58], on encryption in [59], and on black-hole detection in [60].

Funding: This research received no external funding. Conﬂicts of Interest: The author declares no conﬂict of interest.

Abbreviations

6LoWPAN IPv6 over low-power wireless personal area network AMQP Advanced Message Queuing Protocol CoAP Constrained Application Protocol EXI Efﬁcient XML Interchange HTTP Hypertext Transfer Protocol IETF Internet Engineering Taskforce IoT Internet of Things IIoT Industrial Internet of Things JSON JavaScript Object Notation M2M Machine-to-Machine MQTT Message Queuing Telemetry Transport OASIS Organization for the Advancement of Structured Information Standards SCADA Supervisory Control and Data Acquisition SSL Secure Sockets Layer TLS Transport Layer Security UTF-8 Unicode Transformation Format—8-bit XMPP eXtensible Message and Presence Protocol

References

1. Zanella, A.; Bui, N.; Castellani, A.; Vangelista, L.; Zorzi, M. Internet of Things for Smart Cities. IEEE Internet Things J. 2014, 1, 22–32. [CrossRef] 2. Ahlgren, B.; Hidell, M.; Ngai, E.C.-H. Internet of Things for Smart Cities: Interoperability and Open Data. IEEE Internet Comput. 2016, 20, 2–56. [CrossRef] 3. Jaloudi, S. Software-Defined Radio for Modular Audio Mixers: Making Use of Market-Available Audio Consoles and Software-Defined Radio to Build Multiparty Audio-Mixing Systems. IEEE Consum. Electron. Mag. 2017, 6, 97–104. [CrossRef] 4. Watthanawisuth, N.; Maturos, T.; Sappat, A.; Tuantranont, A. The IoT wearable stretch sensor using 3D-Graphene foam. In Proceedings of the IEEE Conference on SENSORS, Busan, Korea, 1–4 November 2015. [CrossRef] 5. Chi, Q.; Yan, H.; Zhang, C.; Pang, Z.; Da Xu, L. A Reconfigurable Smart Sensor Interface for Industrial WSN in IoT Environment. IEEE Trans. Ind. Inform. 2014, 10, 1417–1425. [CrossRef] 6. El Kaed, C.; Khan, I.; Berg, A.V.D.; Hossayni, H.; Saint-Marcel, C. SRE: Semantic Rules Engine for the Industrial Internet-Of-Things Gateways. IEEE Trans. Ind. Inform. 2018, 14, 715–724. [CrossRef] 7. Iqbal, R.; Butt, T.; Shafiq, O.; Talib, M.; Umer, T. Context-Aware Data-Driven Intelligent Framework for Internet of Vehicles. IEEE Access 2018, 6, 58182–58194. [CrossRef] Future Internet 2019, 11, 66 16 of 18

8. Silva, R.; Iqbal, R. Ethical Implications of Social Internet of Vehicle Systems. IEEE Internet Things J. 2018. [CrossRef] 9. Long, C.; Cao, Y.; Jiang, T.; Zhang, Q. Edge Computing Framework for Cooperative Video Processing in Multimedia IoT Systems. IEEE Trans. Multimed. 2018, 20, 1126–1139. [CrossRef] 10. Ja’afreh, M.A.; Aloqaily, M.; Ridhawi, I.A.; Mostafa, N. A hybrid-based 3D streaming framework for mobile devices over IoT environments. In Proceedings of the 3rd International Conference on Fog and Mobile Edge Computing (FMEC), Barcelona, Spain, 23–26 April 2018. [CrossRef] 11. Al Ridhawi, I.; Aloqaily, M.; Kotb, Y.; Al Ridhawi, Y.; Jararweh, Y. A collaborative mobile edge computing and user solution for service composition in 5G systems. Wiley Trans. Emerg. Telecommun. Technol. 2018, 29, e3446. [CrossRef] 12. Balasubramanian, V.; Aloqaily, M.; Zaman, F.; Jararweh, Y. Exploring Computing at the Edge: A Multi-Interface System Architecture Enabled Mobile Device Cloud. In Proceedings of the 7th International Conference on Cloud Networking (CloudNet), Tokyo, Japan, 22–24 October 2018. [CrossRef] 13. Jaloudi, S. Open source software of smart city protocols current status and challenges. In Proceedings of the International Conference on Open Source Software Computing (OSSCOM), Amman, Jordan, 10–13 September 2015. [CrossRef] 14. Standard 19464. Advanced Message Queuing Protocol 1.0 (AMQP 1.0); ISO/IEC: Geneva, Switzerland, 2016. 15. O’Hara, J. ISO 19464 Connecting Business for Value. 2014. Available online: http://www.amqp.org/sites/ amqp.org/ﬁles/2014.05.01%20ISO%2019464%20AMQP-ORG_0.pdf (accessed on 4 February 2019). 16. Godfrey, R.; Ingham, D.; Schloming, R. OASIS Standard Advanced Message Queuing Protocol (AMQP) Version 1.0. 2012. Available online: http://docs.oasis-open.org/amqp/core/v1.0/os/amqp-core-complete- v1.0-os.pdf (accessed on 4 February 2019). 17. Standard PRF 20922. Message Queuing and Telemetry Transport (MQTT) Version 3.1.1; ISO/IEC: Geneva, Switzerland, 2016. 18. Standard RFC 7252. Constrained Application Protocol (CoAP); IETF: Fremont, CA, USA, 2014. 19. Standard RFC 6120. Extensible Message and Presence Protocol (XMPP); IETF: Fremont, CA, USA, 2011. 20. Standard RFC 7159. The JavaScript Object Notation (JSON) Data Interchange Format; IETF: Fremont, CA, USA, 2014. 21. Standard IEEE 802.11. Wireless LAN Medium Access Control (MAC) and Physical Layer (PHY); IEEE: New York, NY, USA, 2012. 22. Tao, F.; Cheng, J.; Qi, Q. IIHub: An Industrial Internet-of-Things Hub Toward Smart Manufacturing Based on Cyber-Physical System. IEEE Trans. Ind. Inform. 2018, 14, 2271–2280. [CrossRef] 23. Ferrari, P.; Flammini, A.; Rinaldi, S.; Sisinni, E.; Maffei, D.; Malara, M. Impact of Quality of Service on Cloud Based Industrial IoT Applications with OPC UA. Electronics 2018, 7, 109. [CrossRef] 24. Angrisani, L.; Cesaro, U.; D’Arco, M.; Grillo, D.; Tocchi, A. IOT Enabling Measurement Applications in Industry 4.0: Platform for Remote Programming ATES. In Proceedings of the IEEE Workshop on Metrology for Industry 4.0 and IoT, Brescia, Italy, 16–18 April 2018. [CrossRef] 25. Müller, J.M.; Kiel, D.; Voigt, K.-I. What Drives the Implementation of Industry 4.0? The Role of Opportunities and Challenges in the Context of Sustainability. Sustainability 2018, 10, 247. [CrossRef] 26. Sangkeun, Y.; Kim, Y.W.; Choi, H. An assessment framework for smart manufacturing. In Proceedings of the IEEE 20th International Conference on Advanced Communication Technology, Chuncheon-si Gangwon-do, Korea, 11–14 February 2018. [CrossRef] 27. Moyne, J.; Iskandar, J. Big Data Analytics for Smart Manufacturing: Case Studies in Semiconductor Manufacturing. Processes 2017, 5, 39. [CrossRef] 28. Chekired, D.A.; Khoukhi, L.; Mouftah, H.T. Industrial IoT Data Scheduling Based on Hierarchical Fog Computing: A Key for Enabling Smart Factory. IEEE Trans. Ind. Inform. 2018, 14, 4590–4602. [CrossRef] 29. Mabkhot, M.M.; Al-Ahmari, A.M.; Salah, B.; Alkhalefah, H. Requirements of the Smart Factory System: A Survey and Perspective. Machines 2018, 6, 23. [CrossRef] 30. Modbus Application Protocol Speciﬁcation V1.1b3. 2012. Available online: http://www.modbus.org/docs/ Modbus_Application_Protocol_V1_1b3.pdf (accessed on 3 February 2019). 31. Modbus Messaging on TCP/IP Implementation Guide V1.0b. 2006. Available online: http://www.modbus. org/docs/Modbus_Messaging_Implementation_Guide_V1_0b.pdf (accessed on 3 February 2019). Future Internet 2019, 11, 66 17 of 18

32. MODBUS/TCP Security. 2018. Available online: http://www.modbus.org/docs/MB-TCP-Security-v21_ 2018-07-24.pdf (accessed on 4 February 2019). 33. Meng, Z.; Wu, Z.; Muvianto, C.; Gray, J. A Data-Oriented M2M Messaging Mechanism for Industrial IoT Applications. IEEE Internet Things J. 2017, 4, 236–246. [CrossRef] 34. Brizzi, P.; Conzon, D.; Khaleel, H.; Tomasi, R.; Pastrone, C.; Spirito, A.M.; Knechtel, M.; Pramudianto, F.; Cultrona, P. Bringing the Internet of Things along the manufacturing line: A case study in controlling industrial robot and monitoring energy consumption remotely. In Proceedings of the IEEE 18th Conference on Emerging Technologies & Factory Automation (ETFA), Cagliari, Italy, 10–13 September 2013. [CrossRef] 35. Chang, C.; Srirama, S.N.; Mass, J. A middleware for discovering proximity-based service-oriented Industrial Internet of Things. In Proceedings of the IEEE International Conference on Services Computing, New Your, NY, USA, 27 June–2 July 2015. [CrossRef] 36. Packwood, D.; Sharma, M.; Ding, D.; Park, H.; Salcic, Z.; Malik, A.; Kevin, I.; Wang, K. FPGA-based Mixed-Criticality Execution Platform for SystemJ and the Internet of Industrial Things. In Proceedings of the IEEE 18th International Symposium on Real-Time Distributed Computing, Auckland, New Zealand, 13–17 April 2015. [CrossRef] 37. Meng, Z.; Wu, Z.; Gray, J. A Collaboration-Oriented M2M Messaging Mechanism for the Collaborative Automation between Machines in Future Industrial Networks. Sensors 2017, 17, 2694. [CrossRef][PubMed] 38. Calderón Godoy, A.J.; González Pérez, I. Integration of Sensor and Actuator Networks and the SCADA System to Promote the Migration of the Legacy Flexible Manufacturing System towards the Industry 4.0 Concept. J. Sens. Actuator Netw. 2018, 7, 23. [CrossRef] 39. Kruger, C.P.; Hancke, G.P. Implementing the Internet of Things vision in industrial wireless sensor networks. In Proceedings of the 12th IEEE International Conference on Industrial Informatics, Porto Alegre, Brazil, 27–30 July 2014. [CrossRef] 40. Hu, P. A System Architecture for Software-Deﬁned Industrial Internet of Things. In Proceedings of the IEEE International Conference on Ubiquitous Wireless Broadband, Montreal, QC, Canada, 4–7 October 2015. [CrossRef] 41. Corotinschi, G.; Gitan, V.G. Enabling IoT connectivity for Modbus networks by using IoT edge gateways. In Proceedings of the IEEE International Conference on Development and Application Systems, Suceava, Romania, 24–26 May 2018. [CrossRef] 42. Joshi, R.; Jadav, H.M.; Mali, A.; Kulkarni, S.V. IOT application for real-time monitor of PLC data using EPICS. In Proceedings of the IEEE International Conference on Internet of Things and Applications, Pune, India, 22–24 January 2016. [CrossRef] 43. Trancă, D.-C.; Pălăcean, A.V.; Mihu, A.C.; Rosner, D. ZigBee based wireless modbus aggregator for intelligent industrial facilities. In Proceedings of the IEEE 25th Telecommunication Forum, Belgrade, Serbia, 21–22 November 2017. [CrossRef] 44. Shinde, K.S.; Bhagat, P.H. Industrial process monitoring using loT. In Proceedings of the IEEE International conference on IoT in Social, Mobile, Analytics and Cloud, Palladam, India, 10–11 February 2017. [CrossRef] 45. Standard IEEE 754. Binary Floating-Point Arithmetic; IEEE: New York, NY, USA, 2008. 46. Yokotani, T.; Sasaki, Y. Comparison with HTTP and MQTT on required network resources for IoT. In Proceedings of the IEEE International Conference on Control, Electronics, Renewable Energy and Communications, Bandung, Indonesia, 13–15 September 2016. [CrossRef] 47. Joshi, J.; Rajapriya, V.; Rahul, S.R.; Kumar, P.; Polepally, S.; Samineni, R.; Tej, D.K. Performance enhancement and IoT based monitoring for smart home. In Proceedings of the IEEE International Conference on Information Networking, Da Nang, Vietnam, 11–13 January 2017. [CrossRef] 48. Thota, P.; Kim, Y. Implementation and Comparison of M2M Protocols for Internet of Things. In Proceedings of the IEEE International Conference on ACIT-CSII-BCD, Las Vegas, NV, USA, 12–14 December 2016. [CrossRef] 49. Luzuriaga, J.E.; Perezy, M.; Boronaty, P.; Cano, J.C.; Calafate, C.; Manzoni, P. A comparative evaluation of AMQP and MQTT protocols over unstable and mobile networks. In Proceedings of the 12th Annual IEEE Consumer Communications and Networking Conference (CCNC), Las Vegas, NV, USA, 9–12 January 2015. [CrossRef] 50. Gao, W.; Nguyeny, J.; Yu, W.; Lu, C.; Kuy, D.; Hatcher, W.G. Towards Emulation-Based Performance Assessment of Constrained Application Protocol (CoAP) in Dynamic Networks. IEEE Internet Things J. 2017, 4, 1597–1610. [CrossRef] Future Internet 2019, 11, 66 18 of 18

51. Koster, M.; Keranen, A.; Jimene, J. IETF Draft Standard Publish-Subscribe Broker for the Constrained Application Protocol (CoAP). 2019. Available online: https://tools.ietf.org/html/draft-ietf-core-coap- pubsub-06 (accessed on 4 February 2019). 52. Käbisch, S.; Peintner, D. W3C Recommendation Canonical EXI. 2018. Available online: https://www.w3. org/TR/exi-c14n/ (accessed on 4 February 2019). 53. Carías, J.F.; Labaka, L.; Sarriegi, J.M.; Hernantes, J. Deﬁning a Cyber Resilience Investment Strategy in an Industrial Internet of Things Context. Sensors 2019, 19, 138. [CrossRef][PubMed] 54. Kwon, S.; Jeong, J.; Shon, T. Toward Security Enhanced Provisioning in Industrial IoT Systems. Sensors 2018, 18, 4372. [CrossRef][PubMed] 55. Xun, P.; Zhu, P.-D.; Hu, Y.-F.; Cui, P.-S.; Zhang, Y. Command Disaggregation Attack and Mitigation in Industrial Internet of Things. Sensors 2017, 17, 2408. [CrossRef][PubMed] 56. Aloqaily, M.; Otoum, S.; Ridhawi, I.A.; Jararweh, Y. An Intrusion Detection System for Connected Vehicles in Smart Cities. J. Ad Hoc Netw. 2019, in press. [CrossRef] 57. Otoum, S.; Kantarci, B.; Mouftah, H. Adaptively Supervised and Intrusion-Aware Data Aggregation for Wireless Sensor Clusters in Critical Infrastructures. In Proceedings of the IEEE International Conference on Communications (ICC), Kansas City, MO, USA, 20–24 May 2018. [CrossRef] 58. Otoum, S.; Kantarci, B.; Mouftah, H.T. Detection of Known and Unknown Intrusive Sensor Behavior in Critical Applications. IEEE Sens. Lett. 2017, 1, 1–4. [CrossRef] 59. Wang, C.; Shen, J.; Liu, Q.; Ren, Y.; Li, T. A Novel Security Scheme Based on Instant Encrypted Transmission for Internet of Things. Secur. Commun. Netw. 2018, 2018, 3680851. [CrossRef] 60. Otoum, S.; Kantarci, B.; Mouftah, H.T. Hierarchical trust-based black-hole detection in WSN-based smart grid monitoring. In Proceedings of the IEEE International Conference on Communications (ICC), Paris, France, 21–25 May 2017. [CrossRef]

© 2019 by the author. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).