VOL. 1, NO. 3, SEPTEMBER 2010 Time Synchronization in Packet Networks and Influence of Network Devices on Synchronization Accuracy
Michal Pravda 1, Pavel Lafata 1, Jiří Vodrážka 1
1Department of Telecommunication Engineering, Faculty of Electrical Engineering, Czech Technical University in Prague Email: {pravdmic,lafatpav,vodrazka}@fel.cvut.cz
Abstract – Time synchronization and its distribution still one of the most popular protocols used for computer time represent a serious problem in modern data networks, synchronization across Internet. SNTP [2] is a simplified especially in packet networks such as the Ethernet. This NTP with worse accuracy. Today, the standard IEEE 1588 article deals with a relatively new synchronization known as PTP (Precision Time Protocol) is increasingly protocol - the IEEE 1588 Precision Clock Synchronization wide spreading. The first standard, which describes PTP, Protocol (PTP). This protocol reaches the typical accuracy was issued in 2002, but in 2008 a new revision of this better than 100 nanoseconds and uses hierarchical document was published [3]. The main difference between synchronization network infrastructure, which enables to NTP and PTP protocol is the method of their synchronize all devices connected into this network. The implementation. While NTP allows only software first part of the article describes the Precise Time implementation, PTP protocol allows also hardware Protocol, while the second part deals with synchronization implementation. Hardware implementation enables more accuracy measurements for various network topologies accurate determination of arrival and departure time of with different network devices. This paper contains the PTP packet, which is a significant improvement of the presentation of a laboratory synchronization network as synchronization accuracy [4]. well as it includes practical results and comments The NTP is very popular and widely used protocol concerning its functionality and performance. Several mostly in computer networks. The precision of NTP is in measurements were performed and the comparisons of the several milliseconds. It is possible to reach better accuracy accuracy of precision time distribution for different by the statistical methods and the filtering algorithms, but network topologies are presented in this article for various it is suitable only for LANs (Local Area Network) [5]. This network conditions and devices. The conclusion contains a method uses the filtering algorithm with IIR filter and summary of measurements, and it shows the resulting reaches the accuracy approximately dozens of accuracy of PTP synchronization. microseconds. The determination of precise time is provided by exchanging timestamps between the client and a synchronization server. The PTP uses the same principle 1 Introduction with several specific improvements. The PTP is the best accurate protocol today for packet Time synchronization and time distribution are very based networks. Its accuracy is usually better than hundred important parts of the present telecommunication of nanoseconds. The main reason of the better accuracy is networks. The precise time synchronization is necessary in a hardware implementation and precise determination of for correct transmissions of high speed digital signals, the timestamps on network interface. The main reducing their jitter, wander, slips and the bit error rate disadvantage comparing to the NTP is the necessity of during the transmission. More and more complex devices special network card, which determines the precise time of need to know the very precise time for their good work. packet arrival and which process timestamps. The PTP and Special synchronization networks and protocols are used to its accuracy is the main topic of this article. We decided to synchronize in backbone networks, e.g. SDH and OTH, build the laboratory workplace with demonstrational PTP but today it is also necessary to deliver the synchronization synchronization network in order to research the synchronization possibilities and accuracy of PTP protocol. into other type of networks, such as the Ethernet. The We also performed several measurements of the precision synchronization is very important for real time synchronization accuracy and its dependency on the services, such as Voice over IP [9]. network topology and conditions. The results are presented In general, packet networks such as Ethernet are the in this article with several conclusions. fastest growing networks today. We encounter them everywhere and therefore should be used for time 2 IEEE 1588 - Precision Time Protocol synchronization. This paper focuses on clock synchronization and time distribution across packet network. Network Time Protocol (NTP), Simple Network The Precision Time Protocol (PTP) is a name of the Time Protocol (SNTP) and relatively new IEEE 1588 synchronization protocol which is described in the standard Precision Time Protocol (PTP) belong to the most popular IEEE 1588. This standard was published in 2002, but today synchronization protocols. Network Time Protocol [1] is there is the second version with small changes and improvements.
8
VOL. 1, NO. 3, SEPTEMBER 2010
The PTP protocol is designed to synchronize the clock o = t − t + d (1) in all devices, which are connected in the same network. 2 1 MS The PTP protocol is based on the system of transmission of PTP messages between the synchronized clocks. For the From the Delay_Req and the Delay_Resp messages are determined the values of t and t . We can calculate the proper operation of the system, it is important to determine 3 4 clock offset from t and t . the boundary and ordinary clock and to establish a master 3 4 slave synchronization hierarchy. PTP version 2 defines o = t4 − t3 + d SM (2) transparent clock in addition [6]. An example of a simple network system is shown in the Fig. 1 and it contains the Considering the assumption that the delay of packets two boundary and two ordinary clocks. One master clock will be the same in both directions ( dMS = d SM ), we can is always chosen for a single network segment, depending calculate them using the equation (3) [7]. on the accuracy of each device on each network segment. (t − t )+ (t − t ) This algorithm is called the Best Master Clock (BMC). d = 2 1 4 3 (3) Detailed description of the topology design is given in the 2 chapter 3.3 [7]. The clock offset is then equal: (t − t )− (t − t ) o = 2 1 4 3 (4) 2
It is clear that the accuracy of timestamps has fundamental impact on the synchronization accuracy. Therefore, network devices allow the accurate determination of the packet arrival time. The specific network topology is presented in the chapter 3.
Fig. 1. The network topology of a simple system. 3 Laboratory Network with IEEE 1588 protocol The accurate time is determined by calculating the delay of the transmission through the packet network. In
this schematic process, the time server starts the 3.1 The Description of Equipments communication by sending the multicast packets (Sync and The IEEE 1588 synchronization protocol usually Follow up). These packets are received by the network operates above a hierarchically organized network. This card and are used for its synchronization. The means that there is typically one grand master clock synchronization process is described in the Fig. 2. synchronization server in the network and all other
network devices work in the synchronous locked mode as Master Clock Time Slave Clock Time the slave clocks. As it was described in the previous section, the mechanism for the selection of the master t1 Sync message clock device is already implemented in the IEEE 1588
t2m t2 protocol. For that reason, our solution of the Follow_Up message demonstrational laboratory network contains one synchronization server with the GPS receiver, one Ethernet switch and one computer network card with the IEEE 1588 protocol’s support and several standard Ethernet devices
t3m t3 (routers, switches, hubs and network cards) that do not Delay_Req message support this synchronization protocol.
t4 The LANTIME (Local Area Network Timeserver) Delay_Resp message M600/GPS server from the Meinberg Radio Clocks co. was selected as the primary synchronization server for our laboratory network. This server provides a high precision timing signal via Ethernet (TCP/IP) network. The server is of a modular conception and thus can be equipped with the Master time Slave time various modules and interfaces. Typical configuration, which we also used in our network, contains the internal Fig. 2: The PTP synchronization process [7]. oscillator of OCXO LQ type, the 6 channels GPS C/A code receiver, a single board computer SBC LX800 500 The Sync message is received at the time t2. The MHz with integrated network card, and a power supply Follow_Up message contains the value of t1. When the unit. The server in our configuration also consists of four d packet delay will be known, it is possible to calculate the standard 100Base T Ethernet interfaces, which support the clock offset o (1) of the Slave clock.
9
VOL. 1, NO. 3, SEPTEMBER 2010
NTP and SNTP protocols, one 100Base T Ethernet 1.10 11 . All these characteristics are temperature dependent interface with the IEEE 1588 protocol’s support, several and their more detailed descriptions can be found in the BNC connectors with various frequency outputs, RS232 documentation [8]. console input/output, GPS antenna input and power input. The laboratory network further consists of a modular The server can be configured by the console commands switch from the Hirschmann co., which supports the IEEE using the RS232 port, by the Telnet protocol and by the 1588 protocol. The switch is of a modular industrial simple WWW environment using standard web browser Ethernet type and the configuration used in our network and the HTTP protocol. The front panel also contains consists of one module with the 4 Fast Ethernet ports and several buttons and a small graphic display (vacuum the USB and V.24 management interfaces. The switch can be configured by the HTTP protocol and a web browser or fluorescent graphic display VFD) for the diagnostic by the SNMP protocol and it supports the synchronization purpose. The server has standard proportions and can be by the SNTP protocol and also by the IEEE 1588 PTP easily mounted into an ordinary server rack. The server protocol. For the purpose of the IEEE 1588 PTP protocol, provides many network services via Ethernet network, the switch can operate in the port master and/or the port such as the NTP v2, v3 and v4 (in broadcast and multicast slave boundary clock mode. mode), the SNTP, the SNMP (Simple Network The last component of our laboratory demonstrational Management Protocol) v1, v2 and v3, the DHCP Client network is the network card PTP270PEX from the (Dynamic Host Configuration Protocol), the Telnet, the Meinberg Radio Clocks co. The card uses standard PCI E FTP, the HTTP, the IP Protocol v4, v6 and others. (Express) bus interface and is mounted in a computer. This The main role for provisioning the IEEE 1588 protocol card supports the IEEE 1588 PTP protocol and typically in the LANTIME server plays the Time stamp Unit, which acts as a slave device in a synchronization network. The is implemented in a FPGA (Field Programmable Gate network card has an on board Time stamp Unit module, Array) and which checks the data traffic on the MII (Media which is integrated in a FPGA and which operates Independent Interface) between the physical (PHY) layer similarly as the module in a synchronization server and the medium access layer (MAC). If a valid Ethernet described in Fig. 3. After decoding valid time information frame containing the IEEE 1588 structure’s pattern is from the Ethernet frame received from the grand master detected, the Time stamp Unit reads its time stamp and clock device, it calculates the time offset and adjusts the performs the next processing (calculation of transmission local oscillator of the network card. The card has also delay by using equation (3) and (4)) by a computational several outputs for 10 MHz clock, the PPS (Pulse per unit. In the second direction, the Time stamp Unit inserts Second) and the IRIG time codes (Inter range Instrumentation Group Time Codes). These outputs will be specific frames with the time stamps to be sent to the used in our network for obtaining the precision time and to slave clock network devices. The precious clock signal is investigate the influence of network devices, which do not derived from the communication with the GPS module by support the IEEE 1588 PTP protocol, on the accuracy of using a PLL (Phase Locked Loop). The whole process is network synchronization and the distribution of the time described in the following schematic (Fig. 3). stamps.
3.2 The Network Topologies
The accuracy of the synchronization was measured using several different network topologies. These topologies were created from the common network devices and also devices, which support PTP protocol. Each topology contains at least the LANTIME PTP server and the PC with PTP network card (Fig. 4). The first case is Fig. 3. A schematic model of the Time stamp Unit. only the simple network with the server and the network
card directly connected. The reverse process (when the server works in a slave
clock mode) is also implemented in the Time stamp Unit, but in our laboratory network, the server always operates as a grand master clock device. The most important characteristics of the synchronization server are derived from the type of the internal oscillator and its output clock stability. The configuration in our laboratory network uses the OCXO LQ type of internal oscillator, which has the typical short term stability (for 1 sec) approx. 1.10 9 and the long term stability (for 1 year) approx. 4.10 7 in the free running mode. Thanks to the GPS module, the typical accuracy in the GPS synchronous mode (averaged for 24 hours) is
10
VOL. 1, NO. 3, SEPTEMBER 2010
Fig. 4. The network topology with the server and the PC with PTP network card.
The rest of the topologies contain other devices such as the switches and a hub. As an example, here is the topology (Fig. 5) with the PTP switch from the Hirschmann co.
Fig. 5. The network topology with the server, the PTP switch and the PC card.
The next measurements were performed for the Fig. 6. The measurement of the synchronization accuracy of a hub in network with high speed data traffic between the a network with high speed data traffic. terminals. The main reason was to investigate the influence of the level of the network load on the distribution delays of synchronization. In this case, we measured a hub with 5 ports (Fig. 6), Hirschman switch with 4 ports and a simple switch with 8 ports (Fig. 7). The Hirschmann switch and an ordinary switch from the Planet co. both support the maximum speed 100 Mbps but the Hub supports only 10 Mbps. The synchronization accuracy across the packet network was measured for all network topologies. The PTP server and PC network card are able to generate different clock signals, which can be displayed on an oscilloscope. We used a method of measuring the time difference between the rising edge of the clock signals per second generated by the server and the network card. This way we obtained one value of the time deviation between the clock signal generated by the server and by the client network Fig. 7. The measurement of the synchronization accuracy of a card per second. These values were saved in the computer standard switch in a network with high speed data traffic. and used for the next evaluations and processing. The measurements were performed for the long time intervals 4 The Results of the measurements to ensure the objectiveness of the measurement for each topology. PTP protocol synchronization accuracy is typically under one hundred nanoseconds. This accuracy is dependent on the method of algorithm implementation. The first measurements were performed for the network topologies described in Fig. 4 and 5 with devices supporting PTP protocol. According to the assumptions, the direct connection between the server and the network card and the topology using PTP switch achieve best results (Fig. 8). The measurement was carried out during 1000 seconds but the figure displays only the first 300 seconds. The maximum deviation between the clock signals generated by the LANTIME server and the network card was 58 ns and the mean deviation was around 15 ns (Tab. 1).
11
VOL. 1, NO. 3, SEPTEMBER 2010
Fig. 8. The comparison of time behaviour of the deviation for the direct Fig.10. The comparison of the time behaviour of the deviation connection and for the network using the PTP switch. between clock signals of using a standard switch and a hub in the network with and without the network overload. The next measurements were focused on the investigation of the PTP synchronization protocol behavior The PTP switch shows approximately the same results in a network containing standard network devices. The with or without the network load, therefore there is no measurements were performed according to the Fig. 5 but influence of the network load level on the distribution of the PTP Hirschmann switch was replaced by a standard the synchronization packets. The PTP principle, which was switch or a hub. The time behaviour of the deviation described in the previous chapter, guarantees the between the clock signals is displayed in the Fig. 9. synchronization accuracy. The deviation between the clock signals is under one hundred nanoseconds in the both cases (with or without the traffic). The standard switch has worse synchronization accuracy in the case of high speed network traffic but it is not so significant (Fig. 11, Tab. 1).
Fig. 9. The time behaviour of the deviation between clock signals of using a standard switch and a hub in the network.
The maximum deviation between the clock signal
generated by the network card and the clock signal from Fig. 11. The time behaviour of the deviation between the clock the server in a network containing a standard switch was signals for a standard switch in the network with and without high speed 820 ns and the mean deviation was around 106 ns. While data traffic. the maximum deviation for the situation of using a hub was 787 ns and the mean deviation was approx. 33 ns. For The synchronization accuracy difference between the the distribution of synchronization the behavior of a hub is standard switch and the PTP switch is shown in the Fig. slightly better than the behavior of a standard switch, 12. The accuracy of the synchronization with the standard because the switch uses the transmission scheme store and switch is about 10 times worse than with the PTP switch forward, which adds some additional delay, while the hub (Fig. 12, Tab. 1). just forwards the received packets in all directions. This behaving has a negative effect during network load when packets' collisions lead to the packets’ losses. The measurement of a hub with high speed network traffic according to the Fig. 6 is displayed in the graph in the Fig. 10. The graph shows a loss of the synchronization packets that occurs when the hub is overloaded.
12
VOL. 1, NO. 3, SEPTEMBER 2010
The table shows the mean value, negative and positive maximal values of time deviation between the clock signals generated by the PTP server and the PTP network card. The variance and standard deviation are also shown in the table to demonstrate the stability of generated clock signals. Devices, which do not support the PTP protocol, have negative impact on the synchronization accuracy in the whole network. The synchronization accuracy is typically about 10 times worse. If we want to maintain high accuracy of the synchronization, we have to use devices, which support PTP protocol, in the whole network. Standard network devices (hub, switch) also demonstrate the dependence of the output clock stability on the level of the network load. Fig. 12: The time behaviour of the deviation for a standard and the PTP switch. 6 Acknowledgement We also measured several more complex topologies. For example the synchronization path led from the PTP This work was supported by the Grant Project FRVS server via the PTP switch and a standard switch to the PTP G1 830/2010 and MSM6840770014. network card in the PC. The results were influenced by the weakest network element and in our example the results 7 References were very similar to the characteristics of a standard switch (Fig. 9). [1] RFC 1305, Network Time Protocol (Version 3), 1992. [2] RFC 2030, Simple Network Time Protocol (SNTP) Version 4, 1996. [3] IEEE 1588, Standard for a Precision Clock Synchronization Protocol 5 Conclusion for Networked Measurement and Control Systems, 2008 [4] T. Neagoe, V Cristea, L. Banica „NTP versus PTP in Computer Ne The PTP protocol is currently the most accurate time tworks Clock Synchronization“, IEEE International Symposium, 2006. protocol for synchronization in local packet based [5] M. Pravda, Z. Kocur, “Time Synchronization through Network Time networks. That is why we decided to build the Protocol and Improvement of Its Accuracy” 32ND International Con experimental and demonstrational laboratory network with ference on Telecommunication and Signal Processing, pp. 182–186, 2009. the synchronization server, PTP switch and the network [6] Burch, J.; Green, K.; Nakulski, J.; Vook, D.; , "Verifying the perfor card. We also performed several measurements and we mance of transparent clocks in PTP systems," Precision Clock Syn described the results in this article. For that purpose, we chronization for Measurement, Control and Communication, 2009. ISPCS 2009. International Symposium on , vol., no., pp.1 6, 12 16 used various network devices and network conditions. Oct. 2009. These results show that if only PTP elements were used, [7] J. C. Eidson, Measurement, Control and Communication Using IE the synchronization accuracy was better than 100 ns. The EE1588, Springer, 2006 . [8] LANTIME M600/GPS/PTP, [online], 2009 [cit. 2010–04–20], synchronization accuracy in case of a standard switch and Available: http://www.meinberg.de/english/products/lantime m600 a hub was much worse the level of network traffic also gps ptpv2.htm. plays an important role. The statistical summary of [9] M. Voznak, F. Rezac, M. Halas, „Speech Quality Evaluation in IPsec measured results is shown in Tab. 1. Environment,“ Recent Advances in Networking, VLSI and Signal Processing, p. 49 53 ,2010 Univ Cambridge, Cambridge, England, ISBN 978 960 474 162 5 Topology Mean[s] Var. Std. [s] Min [s] Max [s] Server – PC 1.5 10 8 1.7 10 16 1.29 10 8 5.4 10 8 3.0 10 8 Server – PTP switch – PC 1.9 10 8 2.2 10 16 1.49 10 8 5.8 10 8 3.75 10 8 . Server – Hub – PC 3.4 10 8 5.5 10 14 23.5 10 8 67 10 8 78.7 10 8 Server – Switch – PC 10.6 10 8 5.7 10 14 23.9 10 8 72 10 8 82 10 8 Server – PTP switch – PC 2.0 10 8 2.7 10 16 1.66 10 8 7.1 10 8 3.2 10 8 (with load) Server – Hub – PC 24.9 10 8 4.9 10 12 222 10 8 338 10 8 499 10 8 (with load) Server – Switch – PC 23.8 10 8 5.5 10 14 23.6 10 8 79 10 8 121 10 8 (with load) Server – PTP switch – 12.6 10 8 3.6 10 14 18 10 8 46 10 8 86 10 8 switch –PC (with load)
Tab. 1. The statistical results of all measurement for different topologies and under different conditions.
13