What’s it for? • Temporal ordering of events produced by concurrent processes Distributed Systems • Synchronization between senders and receivers of messages • Coordination of joint activity Clock Synchronization: Physical Clocks • Serialization of concurrent access for shared objects Paul Krzyzanowski [email protected] Except as otherwise noted, the content of this presentation is licensed under the Creative Commons Attribution 2.5 License. Page 1 Page 2 Logical vs. physical clocks Logical clock keeps track of event ordering – among related (causal) events Physical clocks Physical clocks keep time of day – Consistent across systems Page 3 Page 4 Quartz clocks Atomic clocks • 1880: Piezoelectric effect • Second is defined as 9,192,631,770 periods of – Curie brothers radiation corresponding to the transition – Squeeze a quartz crystal & it generates an electric field – Apply an electric field and it bends between two hyperfine levels of cesium-133 • 1929: Quartz crystal clock – Resonator shaped like tuning fork • Accuracy: – Laser-trimmed to vibrate at 32,768 Hz better than 1 second in six million years – Standard resonators accurate to 6 parts per million at 31° C – Watch will gain/lose < ½ sec/day – Stability > accuracy: stable to 2 sec/month • NIST standard since 1960 – Good resonator can have accuracy of 1 second in 10 years • Frequency changes with age, temperature, and acceleration Page 5 Page 6 1 UTC UTC • UT0 • Coordinated Universal Time – Mean solar time on Greenwich meridian • Temps Universel Coordonné – Obtained from astronomical observation • UT1 – Kept within 0.9 seconds of UT1 – UT0 corrected for polar motion – Atomic clocks cannot keep mean time • UT2 • Mean time is a measure of Earth’s rotation – UT1 corrected for seasonal variations in Earth’s rotation • UTC – Civil time measured on an atomic time scale Page 7 Page 8 Physical clocks in computers Problem Real-time Clock: CMOS clock (counter) circuit Getting two systems to agree on time driven by a quartz oscillator – Two clocks hardly ever agree – battery backup to continue measuring time when – Quartz oscillators oscillate at slightly different power is off frequencies Clocks tick at different rates OS generally programs a timer circuit to – Create ever-widening gap in perceived time generate an interrupt periodically – Clock Drift – e.g., 60, 100, 250, 1000 interrupts per second (Linux 2.6+ adjustable up to 1000 Hz) Difference between two clocks at one point in time – Programmable Interval Timer (PIT) – Intel 8253, 8254 – Clock Skew – Interrupt service procedure adds 1 to a counter in memory Page 9 Page 10 8:00:00 8:00:00 8:01:24 8:01:48 Sept 18, 2006 Skew = +84 seconds Oct 23, 2006 Skew = +108 seconds +84 seconds/35 days +108 seconds/35 days 8:00:00 Drift = +2.4 sec/day 8:00:00 Drift = +3.1 sec/day Page 11 Page 12 2 Perfect clock Drift with slow clock C C skew Computer’stime, Computer’stime, UTC time, t UTC time, t Page 13 Page 14 Drift with fast clock Dealing with drift Assume we set computer to true time C skew Not good idea to set clock back – Illusion of time moving backwards can confuse message ordering and software development environments Computer’stime, UTC time, t Page 15 Page 16 Dealing with drift Dealing with drift Go for gradual clock correction OS can do this: Change rate at which it requests interrupts If fast: e.g.: Make clock run slower until it synchronizes if system requests interrupts every 17 msec but clock is too slow: request interrupts at (e.g.) 15 msec If slow: Or software correction: redefine the interval Make clock run faster until it synchronizes Adjustment changes slope of system time: Linear compensating function Page 17 Page 18 3 Compensating for a fast clock Compensating for a fast clock C C Clock synchronized skew Linear compensating function applied Computer’stime, Computer’s time, time, Computer’s UTC time, t UTC time, t Page 19 Page 20 Resynchronizing Getting accurate time After synchronization period is reached • Attach GPS receiver to each computer – Resynchronize periodically ± 1 msec of UTC – Successive application of a second linear • Attach WWV radio receiver compensating function can bring us closer to true Obtain time broadcasts from Boulder or DC slope ± 3 msec of UTC (depending on distance) • Attach GOES receiver Keep track of adjustments and apply ± 0.1 msec of UTC continuously – e.g., UNIX adjtime system call Not practical solution for every machine – Cost, size, convenience, environment Page 21 Page 22 Getting accurate time RPC Synchronize from another machine Simplest synchronization technique – One with a more accurate clock – Issue RPC to obtain time – Set time Machine/service that provides time information: client what’s the time? server Time server 3:42:19 Does not account for network or processing latency Page 23 Page 24 4 Cristian’s algorithm Cristian’s algorithm Compensate for delays Client sets time to: T – Note times: server server • request sent: T0 request reply • reply received: T1 – Assume network delays are symmetric client T0 T1 time Tserver = estimated overhead server in each direction request reply client T0 T1 time Page 25 Page 26 Error bounds Error bounds Tserver If minimum message transit time (Tmin) is known: server request reply Place bounds on accuracy of result client T0 T1 time Tmin Tmin Earliest time Latest time message arrives message leaves range = T1-T0-2Tmin accuracy of result = Page 27 Page 28 Cristian’s algorithm: example Cristian’s algorithm: example T = 5:08:15.100 • Send request at 5:08:15.100 (T0) If best-case message time=200 msec 0 T1 = 5:08:15.900 Ts = 5:09:25:300 • Receive response at 5:08:15.900 (T1) Tserver Tmin = 200msec – Response contains 5:09:25.300 (Tserver) server request reply • Elapsed time is T1 -T0 client time 5:08:15.900 - 5:08:15.100 = 800 msec T0 T1 200 200 • Best guess: timestamp was generated 400 msec ago 800 • Set time to Tserver+ elapsed time 5:09:25.300 + 400 = 5:09.25.700 Error = Page 29 Page 30 5 Berkeley Algorithm Berkeley Algorithm • Gusella & Zatti, 1989 • Machines run time dæmon – Process that implements protocol • Assumes no machine has an accurate time • One machine is elected (or designated) as the source server (master) • Obtains average from participating computers – Others are slaves • Synchronizes all clocks to average Page 31 Page 32 Berkeley Algorithm Berkeley Algorithm • Master polls each machine periodically Algorithm has provisions for ignoring readings – Ask each machine for time from clocks whose skew is too great • Can use Cristian’s algorithm to compensate for network latency – Compute a fault-tolerant average • When results are in, compute average – Including master’s time If master fails • Hope: average cancels out individual clock’s – Any slave can take over tendencies to run fast or slow • Send offset by which each clock needs adjustment to each slave – Avoids problems with network delays if we send a time stamp Page 33 Page 34 Berkeley Algorithm: example Berkeley Algorithm: example 3:00 3:00 2:50 2:50 3:25 2:50 9:10 3:25 2:50 9:10 1. Request timestamps from all slaves 2. Compute fault-tolerant average: Page 35 Page 36 6 Berkeley Algorithm: example Network Time Protocol, NTP +0.15 1991, 1992 3:00 Internet Standard, version 3: RFC 1305 +0:15 3:25 2:50 9:10 3. Send offset to each client Page 37 Page 38 NTP Goals NTP servers • Enable clients across Internet to be accurately Arranged in strata synchronized to UTC despite message delays – 1st stratum: machines 1 – Use statistical techniques to filter data and gauge quality of connected directly to results accurate time source 2 • Provide reliable service – 2nd stratum: machines – Survive lengthy losses of connectivity 3 synchronized from 1st – Redundant paths stratum machines 4 – Redundant servers – … • Enable clients to synchronize frequently – offset effects of clock drift • Provide protection against interference – Authenticate source of data SYNCHRONIZATION SUBNET Page 39 Page 40 NTP Synchronization Modes NTP messages Multicast mode • Procedure call and symmetric mode – for high speed LANS – Messages exchanged in pairs • NTP calculates: – Lower accuracy but efficient – Offset for each pair of messages Procedure call mode • Estimate of offset between two clocks – Similar to Cristian’s algorithm – Delay Symmetric mode • Transmit time between two messages – Filter Dispersion – Intended for master servers • Estimate of error – quality of results – Pair of servers exchange messages and retain data • Based on accuracy of server’s clock and consistency of to improve synchronization over time network transit time • Use this data to find preferred server: All messages delivered unreliably with UDP – lower stratum & lowest total dispersion Page 41 Page 42 7 NTP message structure NTP message structure • Leap second indicator • Root delay – Last minute has 59, 60, 61 seconds – Total roundtrip delay to primary source • Version number – (16 bits seconds, 16 bits decimal) • Root dispersion • Mode (symmetric, unicast, broadcast) – Nominal error relative to primary source • Stratum (1=primary reference, 2-15) • Reference clock ID • Poll interval – Atomic, NIST dial-up, radio, LORAN-C navigation – Maximum interval between 2 successive messages, system, GOES, GPS, … nearest power of 2 • Reference timestamp • Precision of local clock – Time at which clock was last set (64 bit) – Nearest power of 2 • Authenticator (key ID, digest) – Signature (ignored in SNTP) Page 43 Page 44 NTP message structure NTP’s validation tests • T1: originate timestamp • Timestamp provided ≠ last timestamp received