Clock Synchronization Pedro F. Souto ([email protected]) April 28, 2021 1/41 Roadmap Synchronization Models Clock Synchronization Centralized Clock Synchronization NTP Applications Further Reading 2/41 Roadmap Synchronization Models Clock Synchronization Centralized Clock Synchronization NTP Applications Further Reading 3/41 Distributed System Model I A set of sequential processes that execute the steps of a distributed algorithm I DS are inherently concurrent, with real parallelism p p I Processes communicate and 1 2 synchronize by exchanging mn messages m I The communication is not instantaneous, but suffers delays H(t) I Processes may have access to a local clock I But local clocks may drift wrt real time t I DS may have partial failure modes I Some components may fail while others may continue to operate correctly 4/41 Fundamental Models Synchronism characterizes the system according to the temporal behavior of its components: I processes I local clocks I communication channels Failure characterizes the system according to the types of failures its components may exhibit 5/41 Models of Synchronism Synchronous iff: 1. there are known bounds on the time a process takes to execute a step 2. there are known bounds on the time drift of the local clocks 3. there are known bounds on message delays Asynchronous No assumptions are made regarding the temporal behavior of a distributed system I These 2 models are the extremes of a range of models of synchronism Dilemma I It is relatively simple to solve problems in the synchronous model, but these systems are very hard to build 6/41 Roadmap Synchronization Models Clock Synchronization Centralized Clock Synchronization NTP Applications Further Reading 7/41 Clock Synchronization Observation I In our every-day lives we use time to coordinate our activities I Distributed applications can do the same, as long as each process has access to a "sufficiently" synchronized clock Synchronized Clocks I Are essential for implementing: some basic services: Synchronous distributed algorithms i.e. rounds Communication protocols e.g. TDMA some distributed applications: Data collection and event correlation Control systems GPS I Enable to reduce communication and therefore improve performance (Liskov 93) 8/41 Applications (according to David Mills ) I Distributed database transaction journaling and logging I Stock market buy and sell orders I Secure document timestamps I Aviation traffic control and position reporting I Radio and TV programming launch I Real-time teleconferencing I Network monitoring, measurement and control I Distributed network gaming and training I Take it with a grain of salt src: David Mills, Network Time Protocol (NTP) General Overview 9/41 Local Clocks I Computers have hardware clocks based on quartz crystal oscillators, which provide a local measure of time, H(t). Often: I These clocks generate periodic interrupts I The OS uses these interrupts to keep the local time dH(t) I These clocks have a drift dt − 1 wrt a perfect clock I Quartz-based oscillators have drifts in theH(t) order of 10−6, i.e. they may run faster/slower a few µs per second I The clock drift of a quartz-based oscilator depends on the environmental conditions, especially temperature I Even if this drift is bounded, unless clocks synchronize, their values may diverge forever, i.e. their offset/skew (Hi (t) t− Hj (t)), may grow unbounded 10/41 What is Clock Synchronization? External Clock Synchronization (CS) I.e. synchronization wrt an external time reference (R) jCi (t) − R(t)j < α; 8i where α is known as the CS accuracy Internal Clock Synchronization I.e. synchronization among the local clocks in the system – needs not be related to real time jCi (t) − Cj (t)j < π; 8i; j where π is known as the CS precision Note that external clock synchronization implies internal clock synchronization. I What’s the precision? 11/41 Time Scales/Standards International Atomic Time (TAI) This is a weighted average of the time kept by around 300 atomic clocks in over 50 national labs I More stable clocks are given larger weights I It uses GPS satellites for clock comparisons that provide the data for the calculation of TAI Coordinated Universal Time (UTC) It is derived from the TAI, by adding leap seconds to compensate for variations in Earth’s rotation speed 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 TAI Solar 0 1 2 3 4 5 6 7 8 9 11 12 1314 15 16 17 18 19 21 22 23 24 25 seconds Leap seconds introduced into UTC to get it in synch with TAI GPS provides time with an accuracy better than 100 nanoseconds I Each satellite has its own atomic clocks (3 or 4), which are kept synchronized with the Master Clock at US Naval Observatory I The MC@USNO is a set of more than 40 atomic clocks that produces UTC(USNO), which is used for computing the UTC 12/41 Roadmap Synchronization Models Clock Synchronization Centralized Clock Synchronization NTP Applications Further Reading 13/41 Clock Synchronization: Centralized Algorithm Assumptions Each process has a local clock Master/server clock provides the time reference Slave/client clock synchronize with the master/server clock Local clock drifts are bounded −1 (1 + ρ) ≤ dCi (t)=dt ≤ 1 + ρ, for all correct processes i System is synchronous I There are known bounds (both lower and upper) for communication delays I The time a process requires to take a step is negligible with respect to the communication delays (and therefore bounded) 14/41 Clock Synchronization: Specification Accuracy jCi (t) − Cm(t)j ≤ α, for all correct processes i where Cm(t) is the master’s clock 15/41 Centralized Clock Synchronization: Idea Master I Periodically I read the local time I broadcast it in a SYNC message Slave I Upon reception of a SYNC message, I update the local clock 16/41 Master Time Estimation Issue There is a delay between the reading of the time @ the master, and its processing @ the slaves. By the time the slave gets the message, the time @ the master will be: Cm(t) Cs(t) Cs(t) = Cm(t) + delay Cm(t1) = T1 SYNC(T1) dey Problem What is the value of delay? Response Do not know, but can be estimated: Cm(t) Cs(t) mn min < delay < max m max is known because the system is synchronous I To minimize the error, should use [Cm(t) + min; Cm(t) + max] midpoint: min + max Cs(t) = Cm(t) + 2 17/41 SYNC Message Period To ensure accuracy, it must be; jCm(t) − Cs(t)j < α Because the clock drifts are bounded: dCs(t) dCm(t) − < 2ρ dt dt Hence, max − min + 2ρT < α 2 Observation The clock skew lower bound is determined by the delay jitter ((max − min)) I If T is large, the second term may dominate I NTP sometimes uses a T of 15 days, and ρ ∼ 10−6 18/41 Message Delay Estimation in Practical Systems (1/2) Issue Often, there is no known upper bound for the communications delay Approach (Christian) Estimate it based on the round-trip-delay, round Let min be the lower delay bound Then, when the slave receives the master’s clock reading, time at the master will be in the interval: mn [t + min; t + round − min] rond t midpoint By choosing the in this interval, i.e. t round mn by assuming that the delay is 2 , we bound the error to: round − min 2 19/41 Message Delay Estimation In Practical Systems (2/2) Insight By measuring the time the server takes to reply, we can reduce the error I This can be implemented by timestamping with the local clocks the sending/receiving of messages m T2 T3 Let Cs(t) = Cm(t) + O(t) Then, (T2,T3) T = T − O(t ) + d s 2 1 1 1 T1 round T4 T4 = T3 + O(t3) + d2 Note: Upper-case T’s are clock times, lower case are real-times Assume Synchronized rates O = O(t1) = O(t3) Symmetry del = (d1 + d2)=2 = ((T4 − T 1) − (T 3 − T 2))=2 20/41 Clock Offset Estimation Using Timestamps Let Cs(t) = Cm(t) + O(t) Then, T2 = T1 − O(t1) + d1 T4 = T3 + O(t3) + d2 Note: Upper-case T’s are clock times, lower case are real-times m T2 T3 Assume O = O(t1) = O(t3) (~T2,T3) Then O = O + (d1 − d2)=2 s ~ = (( − ) + ( − )) T1 Whereround O T4 T4 T3 T1 T2 =2 Remember: del = (d1 + d2)=2 = ((T4 − T 1) − (T 3 − T 2))=2 Because d1; d2 ≥ 0, then d1 − d2 ≤ d1 + d2 Therefore: O~ − del ≤ O ≤ O~ + del O~ can be seen as an estimate of the offset del can be seen as an estimate of O~ ’s error bound 21/41 Clock Offset and Delay Time Estimation Precision Precision depends on: Rate synchronization i.e. O(t1) = O(t3) Symmetry assumption i.e. d1 = d2, only in the case of delay estimation Timestamp accuracy I The lower in the protocol stack they are generated the better I In some cases, the timestamps are sent in later messages, PTP calls them FOLLOW-UP messages. 22/41 Precision Time Protocol (PTP - IEEE 1588) Description Protocol for clock synchronization with high precision on packet-based networks Goals I Precision of at least one microsecond I Minimum hardware resource requirements I Minimum administration on single subnet systems I Applicable, but not limited, to Ethernet Application domains I Test and measurement I Industrial automation I Power industry I Telecoms I Aerospace, navigation and positioning I Aujdio and video networks 23/41 Precision Time Protocol: Syntonization Goal To ensure that master and slaves have the same clock rate k I Two alternative ways to send the Ti timestamp Sync message: requires the ability to insert the timestamp into the message on the fly I This is an issue if the timestamps are generated in the lower layers (see below) Follow_up message: requires an additional message I Slave clock rate is adjusted so that: k+1 k k+1 k T1 − T1 = T2 − T2 k+1 k+1 k k T2 − T1 = T2 − T1 I Sync messages are sent periodically with a period of a few seconds I To correct variations due to changing environmental conditions, 24/41 e.g.