The Water/Wastewater System Manager's Guide to Remote

JULY 2017

1 Figure 1: The first direct connector in the world with a push-in spring connection allows application-specific device designs with just a few PCB layouts The Water/Wastewater System Manager’s Guide to Remote Telemetry and Remote Maintenance Dave Eifert and Eike Wedekind, Industry Solutions [email protected] [email protected] INSIDE:

Executive summary: development of remote Executive summary: development of remote control technology...... 1 control technology Difference between remote control Just a few years ago, it was extremely difficult to monitor and maintain and remote maintenance ...... 2 distributed stations. The reason for this was that the remote systems were Communication protocols ...... 3 either unable to communicate with a control center, or communication involved a great deal of effort and expense. Basic requirements...... 3 Protocols for remote control ...... 4 Today, all installations in remote distribution and collection systems, such as water, gas or electricity supply, pipelines or even transportation Protocols (not for remote control) ...... 5 technology applications, must be connected to a central control system. Transmission paths ...... 6 This allows smaller installations such as pumping stations, transformer Continuous connections in the cellular stations, or transformer substations to be monitored and controlled communication network ...... 7 remotely. This connection presents new challenges for the technology. Selection matrix ...... 8

© 2017 PHOENIX CONTACT PHOENIX CONTACT • P.O. BOX 4100 • HARRISBURG, PA 17111-0100 Phone: 800-888-1300 • 717-944-1300 • Technical Service: 800-322-3225 • Fax: 717-944-1625 E-mail: [email protected] • Website: www.phoenixcontact.com JULY 2017

Remote control and remote maintenance have since become the established standard in communication. Remote systems or external stations can be easily connected to the control center using a wide range of telemetry methods. Standardized transmission is made easier thanks to defined, standardized protocols. The predictive maintenance, made possible as a result of this, enables huge cost savings.

But which communication method is best suited to the relevant application?

This white paper will provide the reader with a practical selection guide to the communication mechanisms and media Remote control technology describes the remote monitoring for remote maintenance and remote control technology that and control of physically separate system parts by means of are relevant to them. data transmission. Measured values and control commands are transmitted over long distances and visualized, processed, and stored in a control center. In contrast to remote maintenance, Difference between remote control and remote control requires a permanent connection to a remote remote maintenance station in the vast majority of cases.

2 In industrial communication, there is a vast difference Remote control technology between remote control and remote maintenance, Continuous monitoring of important process data can be even when using identical technology. This often leads realized by using suitable remote control technology. System to confusion when it comes to selecting the right status is continuously displayed in the control center. The communication media. The particular features of the required logging of flow rates or liquid levels, for example, different applications are described below. can be stored centrally for many years, in compliance with regulatory requirements. Thanks to special protocols, process Even predating all the talk about the Industrial Internet data can be transmitted securely over wide area networks, of Things (IIoT), central control systems have been used even with low bandwidth and with poor transmission quality. to operate systems and machines in ever-more distant locations. In addition, the installed systems are constantly Communication used to be performed via dedicated wiring growing in complexity. This often leads to problems when with discrete digital and analog signals. As processes became needing to diagnose and correct errors in the event of a more complex, more information was needed from the fault. In the event of a fault, there is much more to it than process, and it was possible to process this information just remote maintenance. Thanks to developments in recent in control systems, the discrete lines were, in many cases, years, complete support and monitoring of the production replaced by serial connections via permanent line modems. process are now possible. Stations that did not have dedicated permanent line connections were connected via analog dial-up connections, leased permanent lines, or wireless systems. More recently, Remote maintenance describes remote access to cellular communication connections have been used. One a station for fault diagnostics or for maintenance of the disadvantages of dial-up connections is that there is purposes. This saves costs, as a technician does no permanent connection to the control system and, as with voice telephony, connection establishment takes some not have to be on site or travel long distances in time. Recently there has been a clear move away from serial order to return the system to operation. It also transmission paths toward IP-based communication. Modern reduces downtime. communication options such as Internet, VPN, cellular 3G, and 4G-LTE are also integrated in remote control technology

Communication protocols on a transmission protocol. It must provide internal security mechanisms to prevent data loss in the network. There are specific protocols for every type of industrial communication. In factory automation, fieldbuses with defined transmission paths utilize one of several specific Basic requirements protocols optimized for the applications found in discrete A remote control protocol must feature the following in order manufacturing. Even the cables are often specified, and to satisfy current requirements: process information is usually exchanged cyclically, with very Bidirectional, event-oriented transmission high determinism between the controller and I/O station, It must be possible to send data to the external station HMIs, and other nodes. and receive data from the external station simultaneously. Another group of industrial protocols originates from the For example, an external station must not be hindered by time of serial communication via permanent lines. These control system polling if it is transmitting an alarm message. are mostly what are referred to as polling protocols. One Storage of process information, such as in the event example is the Modbus protocol, which is used worldwide. of connection interruptions A typical feature of this form of data exchange is that the Some of the data in the processes is relevant for billing. Other control system polls each external station in a round-robin data is just as critical for regulatory compliance. The data must fashion. These communications are not deterministic – that not be lost as a result of a connection interruption and must 3 is, it is not possible to predict how long it will take to poll be stored by the external station as historical data until it has any particular remote node, nor the entire system. Polling been received by the control system. either only identifies changes or transmits all data points Time stamping of process information to the control system. However, in the event of a lost For the control system to be able to reconstruct the data connection, it is impossible to later determine what happened trend, the historical data must at least have an exact at the external station during this time, or whether specific time stamp. threshold values were exceeded. The advantage of these protocols is their easy parameterization. The disadvantage is Time synchronization the setup of time dependencies, which can be labor-intensive. In order to reconstruct data trends using the time stamp, all of the system’s external stations must use the same time as With the introduction of remote control technology and the control system. A remote control protocol must therefore the associated expansion of systems, a new type of industrial offer the option of synchronizing all devices in the system. protocol was needed. Previously, a physical connection was required between the control system and the external Serial communication via dedicated permanent cabling station, which limited the amount of data that could be sent. In some systems, distributed system nodes are connected Over time, cellular communication networks have become via the supplier’s dedicated permanent lines. One example is more commonly used to connect external stations to the the connection of the remote pumping station to the central control system. As a result, it was no longer possible to use process control system. Serial communication is possible here time-critical polling protocols. because copper cables are traditionally installed along with the supply pipelines. With protocols according to IEC 60870-5-101 Cellular communication networks are storage networks that or IEEE 1815 (DNP3), for example, the system is equipped can retain the individual data packets, which are then only with communication standards that are established in remote reassembled again at the receiver. Once a connection has control technology. This satisfies all requirements for secure been established, the user no longer has to deal with data remote control technology. In the event that the permanent transmission. It is controlled by using TCP/IP, for example. line fails, the selected remote control protocol ensures that the data for a pumping station is stored in the remote In certain geographic areas, cellular networks can experience control device. intermittent outages. This behavior places new demands

JULY 2017

Communication via TCP/IP (or UDP/IP) or cables If the external stations are connected via the ODP server If no dedicated cables are present, communication via a TCP by means of GPRS, they can communicate with virtually any protocol for networking distributed systems to the control control system. In order to do this, the AX ODP server system is a reliable method for data transmission. Protocols must be installed in the control center. The ODP server according to IEC 60870-5-104 or IEEE 1815 (DNP3) support developed specifically for GPRS communication makes data standardized connections for remote communications, for connection to the control system available via OPC. including storm water overflow tanks, pumping stations, or They are manufacturer-neutral and can communicate with well shafts – via the every OPC-based control system. cellular communication network or wired communication, In “An Abbreviated History of Automation and depending on the Industrial Controls Systems and Cybersecurity” application. If such (January 22, 2016), Ernie Hayden wrote: interfaces already exist at the control system, “In the late 1980s, the IEEE PES Substation Commit- the remote control tee formed a working group to investigate the prob- devices used can reliably communicate lem if the expanding number of proprietary, closed- 4 with the control source protocols and identify some practical solutions. system. Depending The Working Group (WG) collected information on on the protocol used, 120 potential protocols which were then screened communication is Mobile phone tower against industry requirements. Upon review and event-oriented and ballot, two protocols were selected for standardized prepared for the use of cellular-based hardware. In the use: Distributed Network Protocol version 3 (DNP3) event the communication infrastructure fails, the remote control protocol ensures that data is stored locally and and International Electro-technical Commission (IEC) automatically uploaded to the central control system once 60870-5-101. communications have been re-established. “DNP3 is now the most widely deployed protocol in Online data North America – not only for substation use but also The process image data is read cyclically from the PLC for substation-to-master station communications. and displayed in the control system via the OPC interface. DNP3 ownership and maintenance has been under Switching operations can also be performed from the control center. control of the DNP Users Group (www.dnp.org) since 1996.” Historical data Historical data is retrieved cyclically or in a user-controlled way. The time-stamped historical data is sorted in chronological order into *.csv files by the AX ODP server. Protocols for remote control The memory capacity in the PLC depends on the number Various remote control protocols are listed and described of variables and the corresponding memory cycle. below: Malfunctions, alarms, and messages IEEE 1815 – DNP3 Malfunctions, alarms, and messages are transmitted to the DNP3 (Distributed Network Protocol) enables control system by the substation in an event-oriented way. communication between a substation and control room via a These can then be stored in chronological order and serial connection as well as via an Ethernet connection. evaluated in the control system.

DNP3 was developed between 1992 and 1994 by Westronic Protocol IEC 60870-5-104 is the standard protocol for data and is based on IEC 60870-5. However, the protocol is more transmission with TCP/IP in Europe and the U.S. and is also restrictive and offers less room for interpretation than its used in some Asian countries. European counterparts. Protocols (not for remote control) Over time, DNP3 has been extended to include authentication. Authentication means that external stations In addition to the remote control protocols listed, a number check the authenticity of the control system before executing of other protocols have become established. These protocols critical operations. They do this by exchanging encrypted data. are frequently used, however they do not save any data in the event of a connection interruption. Data that does not reach Protocol IEEE 1815 is the standard protocol for serial and the control system can no longer be reproduced. data transmission via TCP/IP. It is used in North, Central, and South America, South Africa, the UK, Australia, and Asia. Modbus As noted earlier, DNP3 is one of the two remote control Modbus is frequently used to allow devices from different protocols that IEEE recommended. manufacturers to communicate with one another based on a standard protocol. The Modbus protocol is a IEC 60870-5-101 communication protocol based on a master/slave or client/ IEC 60870-5-101 is an official communication standard for server architecture. It was developed in 1979 by Modicon for remote control technology in infrastructure sectors. This communication with programmable logic controllers. Modbus 5 protocol is used as a general transmission protocol between has become a de-facto standard in the industry, as it is an (network) control systems and substations. Messages are open protocol. Modbus can be used to connect a client (e.g., transmitted via serial connections. This standard enables a PC) and several servers (e.g., measurement and control devices and systems for remote control and station control systems). There are three versions of Modbus: Modbus technology from different manufacturers to communicate ASCII, Modbus RTU, and Modbus TCP. The last two are by with one another without the need for fundamental far the most popular. adaptations. Modbus is probably the most widely used communication Protocol IEC 60870-5-101 is the standard protocol for protocol in the world. serial data transmission in Europe and is also used in Asia. SMS IEC 60870-5-104 SMS stands for Short Message Service and is also used IEC 60870-5-104 enables communication between the for communication between an external station and the control center and substation via a standard TCP/IP network. control system – usually as an alarm message. The SMS The TCP protocol is specifically used for connection-oriented, can be used by any receiver: mobile phone, landline, fax secure data transmission. As the name of the standard or e-mail account. In addition, several programmable logic suggests (“Network access for IEC 60870-5-101 using controllers can be networked together. The provision of an standard transport profiles”), the protocol follows standard acknowledgment function ensures that the SMS reaches the IEC 60870-5-101 with respect to the application layer. receiver. Otherwise, the SMS is sent to another address. The biggest advantage of IEC 60870-5-104 is communication via a TCP/IP network, which enables simultaneous data Use of the SMS function in remote control transmission with multiple devices and services. technology In addition to its function as a means of communication, the Because the Internet can be used for communication, security SMS service is also used to control and monitor technical by means of data encryption is an important topic. In addition, devices and equipment. In applications where only alarm this is an event-oriented protocol where the substation can messages are to be transmitted or where control commands send data to the control system automatically. are sent relatively infrequently, the SMS service of the GSM network is a viable option for reasons of cost.

For example, industrial modems offer configurable warning used to transmit this serial data. A digital signal is modulated and alarm inputs and are therefore ideal for monitoring to a carrier frequency in the high-frequency range by the small applications. An alarm SMS can be sent directly to a transmitting modem; the original information is then retrieved service technician’s mobile phone, informing them directly by the receiving modem by means of demodulation. This of the type of error and its location. When dealing with an technology has a maximum transmission speed of 33.6 kbps. increased number of messages or control commands, GSM SHDSL modem (Ethernet extender) modems can also be connected to programmable logic It is not usually possible to remotely program external controllers. This allows the I/O area of the PLC to be linked stations that are connected in series. An Ethernet-capable to text messages. It is also frequently the case that an SMS network is required for this. An SHDSL modem can be alarm message is not only intended to be sent to a mobile used to make a serial communication network Ethernet- phone, but also to the alarm management of a control capable. Modems based on the newer DSL standard operate system. When a PLC is combined with a GSM modem, with higher carrier frequencies and larger bandwidths SMS messages can, for example, be received, decoded, and for the phone line than the models based on the older forwarded to a control system in a variety of ways. standard (narrowband). With industrial SHDSL permanent Transmission paths line modems, existing cables can also be used for modern Ethernet applications. Transmission speeds of up to 30 Mbps Wired are possible via in-house cables. Automatic detection of 6 transmission the DSL data rate, as well as automatic adjustment to the When a Ethernet cable (1:1, crossed) and protocol transparency dedicated simplify startup considerably. In addition to point-to-point data line is connection, a line structure (daisy chain) is also possible. used for the With respect to transmission, there are modems for remote control Ethernet, fieldbus or serial interfaces technology connection, Fiber optics e.g., in the Today, when new trunk lines are laid, they are usually power supply fiber optic (FO) lines. FO communication is immune network of to electromagnetic interference and, depending on the municipal technology, allows several kilometers to be covered with a works, bandwidth in excess of 100 Mbps. Media converters can be it should used to forward Ethernet data as well as all common remote be noted control or fieldbus data. Even in power distribution, an Sever room that if the external station (such as a substation) is usually connected to copper cable is longer than 1.5 kilometers, the signals to the central node point via fiber optics. be transmitted must be modulated in order to increase the Non-wired connections (wireless) range. Whether based on copper cables or fiber optics, this If no cable is present, other means of communication must means that process information can be exchanged with a be found. Depending on the distance and the required central control system over long distances. bandwidth, it is also possible to use wireless connections. Permanent line modem Depending on the requirements, one of the following three Serial interfaces (RS-232, RS-485) are often available at the connections can be used: remote control substations and in the control system; these Trusted Wireless interfaces should be used to transmit standard protocols The application of Trusted Wireless was developed such as IEC 60870-5-101. Analog permanent line modems, specifically for industrial use. As a result, it can cover which are specifically tailored to industrial requirements, are

distances of up to 20 miles from one serial interface slip rings available for them. By using Bluetooth devices, to another. For example, Trusted Wireless is used for maintenance costs can be saved and any number of signals communication between the control center and a pumping can be transmitted over a distance of up to 1200 feet with station. There should be a line of sight between the systems. line of sight. Communication over distances of more than a couple of WLAN (WiFi) miles will typically require directional antennas and towers. WLAN is recommended in applications where, in addition With visual contact, it is possible to transmit over a distance to process-specific data, data from webcams or IP phones is of up to 20 miles, for example, when networking well fields. also transmitted. Wireless Local Area Network describes a In the event of failure, a new communication path is selected local wireless network with a data rate of up to 300 Mbps in and data transfer is still ensured. Two-hundred fifty-five the wireless application, which can be used by a large number devices can be connected per network. Multiple networks of devices. Wireless is particularly suitable for mobile can be operated in parallel without any restrictions. monitoring, operation, and data acquisition as it can be easily integrated into IT networks.

Continuous connections in the cellular communication network 7 The cellular communication network and the Internet provide a solution for transmission over long distances and on a global scale. An almost constant connection can be achieved in this way. Globally, the GSM network has been the standard in most countries. In the U.S., AT&T and T-Mobile are the main providers who have built out networks over the years based on the GSM standards. Verizon and Sprint have built out networks based on the CDMA standards. While popular in the U.S., CDMA has not been implemented in other parts of the world. Since the early 1990s, both GSM and CDMA have evolved through various generations, more or less in lockstep with each other. The popular “1G, 2G, 3G, 4G, 5G” terminology is shorthand for first generation Wireless in practice through fifth generation. Bluetooth 2G Bluetooth is a standard for wireless transmission over short • GSM (used by AT&T and T-Mobile in the US) paths involving up to seven devices. Bluetooth operates over • CDMA (used by Verizon and Sprint, only in the US) the 2.4 GHz ISM frequency band, which does not require a Sever room license and is free of charge worldwide. It allows serial data These networks have been valid for use in remote industrial to be transmitted even if another network is already active. communications, but are at or approaching end of life. AT&T has shut down its 2G families of GSM-based of Bluetooth in use networks. T-Mobile is still supporting some of its 2G families Slip rings are frequently used to transmit measured values of networks. Verizon and Sprint are supporting their 2G or process data from the scraper bridge of a wastewater networks, but likely not beyond 2019. treatment plant to the PLC. This technology is very high maintenance, resulting in high costs for the operator. 3G Additional signals are often required, but there are no • GSM (UMTS) (used by AT&T and T-Mobile in the US • CDMA (CDMA2000 EV-DO) (used by Verizon and Sprint)

3G UMTS (Universal Mobile Telecommunications System) as the fourth generation (4G) of mobile communication is a third-generation mobile communication standard with technology. The technology is the new standard in mobile transmission speeds of up to 7.2 Mbps in the GSM network. communication and is the successor to GSM UMTS AT&T has shut down, or will shortly shut down its 3G (Universal Mobile Telecommunications System, 3G) and networks. T-Mobile may support its 3G networks through CDMA 3G. This technology enables a high-speed connection 2019 or 2020. On the CDMA side, Verizon and Sprint are of up to 300 Mbps and, unlike conventional standards, still supporting 3G, but the future is unclear for how much it supports different bandwidths. A peak data rate of up longer. If your carrier can commit to supporting 3G, whether to 300 Mbps for download and 75 Mbps for uplink with it be CDMA-based (Verizon or Sprint), or GSM-based 5-millisecond latency is theoretically possible. 4G LTE will (AT&T or T-Mobile), these networks are acceptable for use one day be superseded by 5G. However, 4G is regarded to in remote telemetry, provided your system uses a suitable have acceptable longevity for the deployments of remote remote telemetry protocol as discussed earlier in this paper. telemetry systems being installed today.

LTE – 4G Selection matrix Due to the ever-increasing use of smartphones and tablets, as well as the networking of stand- alone machines and Various communication options are available for data devices with the control center, increasing strain is being transmission in remote or wide-ranging networks and for placed on the network. To ensure accommodation of this monitoring machines all over the world. This overview 8 high level of demand in future, a new technology like LTE helps users to select the optimum form of communication is required. LTE (Long Term Evolution) is also referred to for the application (See the following tables).

Technical Transmission Ideal field of Range Protocols Operating costs parameters speed application

Basic monthly Mobile Up to 12 Mbps IEC 60870-5-104 charge and billing communication downloads, • Remote data 4G/LTE World-wide DNP3 based on the network Up to 5Mbps acquisition Modbus TCP transmitted data coverage uploads volume Basic monthly Mobile GSM/ IEC 60870-5-104 charge and billing communication Up to 7.2 • Remote data CDMA World-wide DNP3 based on the network Mbps acquisition 3G Modbus TCP transmitted data coverage volume • Connection of Ethernet networks Mobile Basic monthly communication • Connection of SMS World-wide – charge and billing – network substations per SMS message coverage • Worldwide alarm generation • Transmission of Mobile communication network systems network Mobile communication Mobile Basic monthly GSM/ IEC 60870-5-101 I/O information communication charge and usage- Serial data up CDMA World-wide DNP3 network dependent billing to 9600 bps • Worldwide remote 2G Modbus RTU coverage based on time programming connection

Technical Operating Transmission Ideal field of Range Protocols parameters costs speed application

Basic monthly Analog IEC 60870-5-101 Analog • Worldwide remote chargeand usage- Serial data up to dial-up World-wide DNP3 telephone programming dependent billing 33,600 bps connection Modbus RTU connection connection based on time Annex A: Up to 25 Mbps downstream (from the Internet) • Remote data acquisition Analog Basic monthly Up to 3.5 Mbps ADSL IEC 60870-5-104 telephone charge upstream (to the • Connection of broad-band World-wide DNP3 connection (usage/time- Internet) Annex B: substations connection Modbus TCP and DSL independent Up to 24 Mbps • Connection of access billing) downstream (from Ethernet networks the Internet) Up to 1 Mbps upstream (to the 9 Internet) • Remote data Public telephone network telephone Public acquisition • Connection of Installation and substations Existing two/ maintenance IEC 60870-5-104 four-wire • Connection of costs for two/ Ethernet data of SHDSL Up to 20 km DNP3 cable for remote PROFIBUS four-wire cable up to 30 Mbps Modbus TCP optimum devices range No monthly costs • Connection of Ethernet networks • Connection of substations

Technical Transmission Ideal field of Range Protocols Operating costs parameters speed application

Free of charge • Serial data up to • Wireless Line of sight and no license 115,200 bps Trusted Up to networking of Modbus RTU for optimum required in the Wireless 2.0 20 mi. • I/O data sensors and range 900MHz ISM or bidirectional and actuators 2.4 GHz ISM band unidirectional • Protocol- transparent • WLAN data Ethernet Free of charge rates of up to communication IEC 60870-5-104 Line of sight and no license Up to 2 300 Mbps with PLCs, I/O WLAN DNP3 for optimum required in the 2.4 km stations, PCs, etc. Modbus TCP range GHz and 5 GHz • Ethernet net ISM band data rates of up • Wireless network to 95 Mbps integration of remote installations • Wireless programming

10 network telephone Public connection

Free of charge • Serial data up to • Cable replacement IEC 60870-5-104 Line of sight Up to and no license 187,500 bps for PROFIBUS, Bluetooth DNP3 for optimum PROFINET, 300 m required in the 2.4 • Ethernet up to Modbus TCP range Modbus RTU/TCP, GHz ISM band 1 Mbps net and serial TCP/IP data in the case of busy and mobile system parts

ABOUT PHOENIX CONTACT

Phoenix Contact develops and manufactures industrial electrical and electronic technology products that power, protect, connect, and automate systems and equipment for a wide range of industries. Phoenix Contact GmbH & Co. KG, Blomberg, Germany, operates 50 international subsidiaries, including Phoenix Contact USA in Middletown, Pa.

For more information about Phoenix Contact or its products, visit www.phoenixcontact.com, call technical service at 800-322-3225, or e-mail [email protected].