Handling Time and Reactivity for Synchronization and Clock Drift Calculation in Wireless Sensor/Actuator Networks

Marcel Baunach Graz University of Technology, Institute for Technical Informatics, Graz, Austria

Keywords: Event Timestamping, Reaction Scheduling, Time Synchronization, Clock Drift Calculation, Information Tagging.

Abstract: The precise temporal attribution of environmental events and measurements as well as the precise scheduling and execution of corresponding reactions is of utmost importance for networked sensor/actuator systems. Apart, achieving a well synchronized cooperation and interaction of these wirelessly communicating distributed systems is yet another challenge. This paper summarizes various related problems which mainly result from the discretization of time in digital systems. As an improvement, we’ll present a novel technique for the automatic creation of highly precise event timestamps, as well as for the scheduling of related (re- )actions and processes. Integrated into an operating system kernel at the lowest possible software level, we achieve a symmetric error interval around an average temporal error close to 0 for both the timestamps and the scheduled reaction times. Based on this symmetry, we’ll also introduce a dynamic self-calibration technique to achieve the temporally exact execution of the corresponding actions. An application example will show that our approach allows to determine the clock drift between two (or more) independently running embedded systems without exchanging any explicit information, except for the mutual triggering of periodic interrupts.

1 INTRODUCTION nally obtained values in order to properly correlate the contained information, and to trigger adequate re- Wireless sensor/actuator networks (WSAN) are com- actions. In fact, measured or otherwise obtained en- monly deployed to observe and interact with their environmental information is often useless unless it is vironment. In this respect, temporal and spatial in- associated with temporal and spatial information. formation are the two most fundamental measures for This paper starts with a discussion of various the “attribution” or “tagging” of states and events (i.e. problems regarding time in digital systems. Next, we state transitions) within any observed environment. In present a novel approach for taking timestamps, for this context, the states describe a set of physical and measuring and specifying temporal delays, and for logical conditions at a given position and at a certain scheduling and ensuring reaction times with a sym- time, and they are specified by one or more continu- metric temporal error around 0. Finally, a real-world ous or discrete state values. test bed shows how periodically communicating embedded systems can determine their relative clock Recorded over a certain period of time, variations drifts without any additional information exchange. in the state values allow the detection and analysis of events and event patterns within the environment (Wittenburg et al., 2010) (Romer,¨ 2008). These variations do not only indicate the events’ spatial extend, 2 TIME IN DIGITAL SYSTEMS propagation speed, and influence on the environment, but most commonly they also allow the prediction of In contrast to specific position vectors and state values future states for both the observing system and its sur- of e.g. sensor nodes (which can change sporadically, rounding. In this regard, the interaction with the envi- arbitrarily and independently from each other), time ronment, which we already proclaimed to be the most is a common property. It is system independent, and central objective in sensor/actuator systems, typically advances continuously with a globally constant rate requires the precise knowledge of time and space to of change. If the sensor nodes manage to establish a be associated with a node’s self-captured and exter- network-wide and consistent notion of time, this in-

Baunach M.. 63 Handling Time and Reactivity for Synchronization and Clock Drift Calculation in Wireless Sensor/Actuator Networks. DOI: 10.5220/0004680400630072 In Proceedings of the 3rd International Conference on Sensor Networks (SENSORNETS-2014), pages 63-72 ISBN: 978-989-758-001-7 Copyright c 2014 SCITEPRESS (Science and Technology Publications, Lda.) SENSORNETS2014-InternationalConferenceonSensorNetworks

formation provides a natural base for their joint inter- applied “delta time” is constant and won’t adapt to action and with the environment. Since processors in changing system loads as well. synchronous digital systems like sensor nodes are al- In the following we’ll indicate and discuss the ways driven by a clock generator C with frequency fC causes and effects of the mentioned problems in de- 1 and period λC = , time and time intervals can easily tail, and present an approach at the kernel level to re- fC and individually be measured – at least in theory: If it liably compensate the related imprecision in the aver- is possible to count the number of elapsed clock peri- age case. ods since a well defined point in time, e.g. the system start, each captured event e, e.g. indicated by an inter- Problem P1: Discretization of time. The differ- rupt, can be attributed with its current counter value ence between the true global time t0 and the individual ce. Consequently, the event’s absolute local system system time t˜ has already become visible in Eq. (1) time t˜e can easily be recovered by and Eq. (4). While the first advances continuously, the use of a digital counter leads to a discretization of t˜e := ce ·λC, (1) the latter, and imposes a resolution which directly de- ˜ and the time difference (i.e. the delay) ∆e1,e2 between pends on the counter’s clock frequency fC. This may two events e1,e2 computes as lead to serious systematic errors for the time measurement and the subsequent scheduling of reactions: ∆˜ e ,e := t˜e −t˜e = (ce − ce )·λC. (2) 1 2 2 1 2 1 The simple capturing of a timestamp t for an event ˜ Obviously, both the time t˜e and the delay ∆e1,e2 al- – the so called timestamping – is immediately affected ready involve a concept-inherent imprecision caused by some inevitable rounding, and suffers from a mea- by the discretized counter values ce ∈ N. In addition, surement error Et ∈ I1 with |I1| = λC. For the na¨ıve we silently assumed for Eq. (1) and Eq. (2) that λC is and adverse reading of the timer counter, rounding perfectly known and constant. Neither is true under down results in I1 := [0,λC), and induces a symmetry 1 real-world conditions, and leads to well-known prob- around the average measurement error Et,av = 2 λC. lems, we’ll address and counteract in this paper. Depending on the use of such timestamps, the emerg- Furthermore, as often requested for interactive ing errors might accumulate during the system run- systems, a reaction r can be scheduled for a captured time. Similarly, the explicit specification of delays 0 R 0 event e. Its intended execution time tr ∈ is com- ∆t in software is also subject to rounding errors E∆. 0 R monly related to any event timestamp te ∈ by the However, we can round half up (e.g. according to 0 R specification of a corresponding delay ∆e,r ∈ : DIN 1333) manually when selecting a delay, and thus 1 1 the corresponding error is E∆ ∈ I3 := [− λC,+ λC). t0 = t0 + ∆0 (3) 2 2 r e e,r Though not avoidable entirely, I3 is at least symmetric However, in the best case, i.e. if the (operating around 0, and the average error is 0. system) scheduler permits the timely switch to the re- Based on these two fundamental error intervals sponding task’s context, the reaction will be triggered I1 and I3, other intervals can be derived, and con- upon reaching the corresponding counter value cr ∈ N sequently exhibit an imprecision, too: For the mea- and the corresponding system time t˜r. In any case, the surement of delays ∆E , we see the implicit compen- finally observable reaction delay depends on the res- sation of the asymmetry in I1: E∆ ∈ I2 := I1 − I1 = olution λC of the system timer: (−λC,+λC). In contrast, the scheduling of reaction 0 0 times t on external events inherits the asymmetry in I1: ∆ ∆ 1 3 e,r e,r Et ∈ I4 := I1 + I3 = [− λC,+ λC). System reactions cr = ce + t˜r = t˜e + ·λC (4) 2 2 λC λC will consequently suffer from an average systematic 1 Although the inherent rounding imprecision is lateness of 2 λC. quite intuitive and introduces various hidden imple- Table 1 summarizes the error types and their cor- mentation problems in real systems, it is commonly responding intervals which must be expected for the simply ignored. Moreover, for concurrent task sys- na¨ıve capturing of timestamps by simply reading the tems with dynamic execution flows, there is an addi- timer register after the corresponding event occurrence (e.g. within an IRQ handler). The resulting ef- tional error in t˜r which is neither constant nor pre- dictable. Since most embedded operating systems fects, and our proposed solution to compensate this silently accept even this problem, application devel- asymmetry, will be discussed later. opers are urged to compensate the imprecision with little control at the task level. SensorOS (Kuorilehto Problem P2: Capturing of timestamps. The cre- et al., 2007) at least tries to execute reactions in time ation of reactive systems demands for the precise by scheduling the responsible task earlier. Yet, the assignment of timestamps for internal and external

64 HandlingTimeandReactivityforSynchronizationandClockDriftCalculationinWirelessSensor/ActuatorNetworks

Table 1: Error intervals for different discretization techniques (system time resolution: λC). Na¨ıve discretization Our discretization approach problem / error type derived from error interval symmetry error interval symmetry 1 1 1 P1: capturing of fundamental I1 = [0,λC) E 2 λC E I3 = [− 2 λC,+ 2 λC) 0 timestamps P2: measurement of I1 − I1 I2 = (−λC,+λC) 0 I2 = (−λC,+λC) 0 delays 1 1 1 1 P3: specification of fundamental I3 = [− 2 λC,+ 2 λC) 0 I3 = [− 2 λC,+ 2 λC) 0 delays 1 3 1 P4: scheduling of I1 + I3 I4 = [− 2 λC,+ 2 λC) E 2 λC E I4 = [−λC,+λC) 0 reaction times events. Reaching a voltage threshold at an analog- IRQ controller, this is already true for the generation digital-converter (ADC) or detecting a signal edge at of timestamps. In fact, the maximum degree of par- an I/O pin are just two simple examples. However, allelism is always limited by the number of available most observable changes within the environment have functional units2. A reliable scheduling (e.g. for com- one thing in common: They are indicated to the CPU plying to hard real-time demands) must be achieved at runtime by so called interrupt requests (IRQs), and through either static techniques at development time should be handled as soon and as fast as possible by or dynamic methods at runtime. A corresponding the corresponding interrupt service routines (ISRs). technique for dynamic resource management under Since ISRs are commonly higher privileged than reg- real-time conditions is presented in (Baunach, 2012). ular application code, they will preempt the latter for their own execution. Thus, they seem to be perfectly Problem P4: Imprecision in the timer frequency. suitable for capturing the timestamp for any emerging Time measurement in digital systems is usually ac- event. However, even the first instruction within each complished by using a pulse generator with a spec- ISR is not executed before some additional delay, ified frequency f0. Internally, this component uses which is also known as interrupt latency ∆IRQ: If the an oscillator (most commonly a quartz crystal) to timer value cTS for the timestamp itself is copied af- generate a periodic clock signal. The characteristics ter another implementation-specific delay ∆ISR within and stability of such oscillators depend significantly the ISR, then we can compute the discrete timestamp on their manufacturing parameters, age, and various ˜ te for the captured event e as follows: environmental conditions like e.g. voltage variations (Hewlett Packard, 1997): A varying frequency drift t˜e = cTS ·λC − (∆IRQ + ∆ISR) = t˜TS − ∆TS (5) ∆ f must always be expected. Its relative error ∆ f f0 Hence, a prerequisite for reliable time tracking via is commonly expressed in units of parts per million Eq. (5) is, that the correction value ∆TS is constant (ppm). For simple low-cost quartzes, and within the and free from rounding errors with respect to the dis- typical temperature ranges of WSAN applications, crete system time period. As we will see, both can be the temperature sensitivity of a typical HC49 quartz achieved through careful code preparation. can already result in deviations of ± 20 ppm. Variations in the clock precision are especially Problem P3: Simultaneity and scheduling reliabil- critical in distributed applications. Since time mea- ity. Although the perfectly simultaneous transition surement is initially individual for each involved sys- of two states can never occur in real systems1, the sur- tem and therefore can drift apart, this may quickly jective discretization of time can easily lead to the as- generate inconsistent data, and must be compensated signment of exactly the same system time for multiple by adequate and repeated synchronization measures. events or scheduled actions. Since resource conflicts often prevent the truly parallel processing of events Problem P5: Global time base and synchroniza- as well as the simultaneous execution of (re)actions, tion with other systems. When does time measure- they usually lead to an implicit serialization. The or- ment actually start, i.e. when is or was time t0 = 0? If der depends on the task scheduler and the internal task we consider a completely independent system which priorities. Since there is most commonly just a single uses the notion of time only for its internal operation,

1The resolution of the time measurement must simply 2Functional units refer to e.g. processors and their cores, be chosen ﬁne enough! or to autonomously operating peripheral components.

65 SENSORNETS2014-InternationalConferenceonSensorNetworks

e.g. to capture events and to schedule actions by a par- standardized and architecture dependent interrupt ser- tial order3, the use of a pure local time with arbitrary vice routines for introducing a constant and carefully begin is absolutely sufficient – e.g. time t0 = 0 may dimensioned delay ∆TS = ∆IRQ +∆ISR before actually simply indicate the system start. However, as soon capturing the timer’s counter value after the IRQ oc- as time is of global relevance, e.g. if actions have to currence. According to Eq. (5) we then have to reduce take place synchronized on different systems, a com- the captured counter value by an adequate correction mon time base is often indispensable. This immedi- value ∆corr: Selected properly, this correction finally 1 1 ately raises the question about which time or which results in the symmetry about 0 for I1 := [− 2 λ, 2 λ). system is used as reference. In any case, its provider While the timestamp measurement error Ete will still should be highly available and exhibit a high clock be equally distributed over I1, this interval is shifted, stability and precision. Several methods exist for the and the average timestamp error is reduced from ini- 1 actual synchronization: Some are based on (regular) tially 2 λ down to 0. At the same time, the propaga- time checks or on the measurement of the pairwise tion and amplification of systematic errors for time- drift between the involved systems. Others rely on dependent reactions will also be kept low and sym- dedicated reference systems and allow the synchro- metric about 0, i.e. I4 = I1 + I3 = [−λC,+λC). Table nization based on centrally triggered events like e.g. 1 compares the error intervals of our compensation radio broadcasts (cf. GPS and the DCF77 protocol). approach with the na¨ıve technique. Finally, distributed methods are available for multi- How can this symmetry be guaranteed? In or- hop systems to successively achieve a common time der to deal with the related problems P1 and P2, we base, e.g. via Desynchronization (Muhlberger,¨ 2013). propose a concept based on two synchronized clocks with interdependent frequency. Thereby, we assume the CPU frequency to be higher than the timer fre- 3 AN ADVANCED TIME quency, while conversely, the system time is derived from the quartz-stabilized CPU clock by an even inte- DISCRETIZATION APPROACH ger divider. Commonly, both requests do not impose an unreasonable restriction on the hardware/software Considering the aforementioned problems, which design: In fact, they are already satisfied in many originated from the integration of time-awareness into systems, since usually only a single central oscilla- digital systems, P1-P3 directly affect the environmen- tor is used as base for all other system clocks. While tal interaction and can be addressed by each system the CPU is commonly directly driven by this main individually. In contrast, P4 and P5 require some clock, other components apply power-of-two dividers information exchange with other systems. These to derive their individual frequencies. Finally, and “peers”, however, are not necessarily available dur- for computationally constrained embedded systems in ing the entire system runtime. For this reason, P1-P3 particular, driving a local time with the maximum res- are treated locally at the embedded systems level (e.g. olution would cause unnecessary CPU load4. in the operating system kernel), while P4 and P5 must Besides the following formal description of our be addressed more globally (e.g. in the network layer approach, we also refer to the example in Figure 1 or at application level). for a comprehensive understanding. Initially, we de- At the embedded systems level, our approach re- note the CPU clock frequency as f and its period as lies on a hardware timer component to provide a lo- λ. The system time frequency is denoted as fC and its cal system timeline with a fixed temporal resolution. period as λC. In addition, we demand for The timeline management is integrated directly into f the OS kernel and accessible for all software layers f := and λ := α·λ with α ∈ N≥2,α even. C α C through the OS API. This unifies the usage by appli- (6) cation tasks and avoids execution time imponderabil- 0 If an interrupt e occurs at time te, the corre- ities through unpredictable code interleaving at run- sponding timer counter c will not be copied before time. Based on our approach, the kernel automat- e some system inherent delay ∆ has passed. For our ically captures a timestamp t˜ for each interrupt e, TS e approach, we request this delay to take exactly ∆ and compensates the error’s asymmetry about 1 λ c 2 C CPU cycles as follows: which would result from using the na¨ıve approach with I1 = [0,λC) as explained in Section 2. Therefore, 4The system time must be accumulated in software at the kernel as a hardware abstraction layer provides every timer overflow. Especially for timers with small word widths, this can quickly lead to a huge performance penalty. 3 “Partial”, since the discretization of time may lead to For instance, a 16 Bit counter counting at fC = 1 MHz will simultaneity (→ P3). already overflow after every 65.536 ms.

66 HandlingTimeandReactivityforSynchronizationandClockDriftCalculationinWirelessSensor/ActuatorNetworks

corr

0 Figure 1: IRQ timestamp acquisition with our approach for various potential event occurrence times te.

proofed by some interval arithmetic. With Eq. (8), α E t˜ ∆c := n·α + with n ∈ N (7) (10) the expected error te of the timestamp e com- 2 0 putes as 0 Thus, the delayed acquisition of the timestamp takes Ete = te −t˜e place at time 0 0 1 1 tTS = tTS − n·α + ·α · − ·λC − n·λC 0 0 0 1 0 α 1 2 f λC tTS = te + ∆TS = te + ∆c · = te + n·α + · . 0 f 2 f 0 tTS 1 1 1 = tTS − ·λC − ·λC ∈ − λC,+ λC (8) λC 2 2 2 To compensate for this delay, and to force the | {z } ∈[0,λC) timestamp error interval I1 to become symmetric around the true event occurrence time while also ex- 1 1 ⇒ I = − λ ,+ λ hibiting an average error close to 0, we can select the 1 2 C 2 C correction value as an integer multiple of λC: 1 1 1 1 ⇒ I2 = I1 − I1 = − λC,+ λC − − λC,+ λC 1 2 2 2 2 ∆corr := (n·α) · = n·λC (9) f = (−λC,+λC) 1 1 1 1 Thus, we simply have to subtract n from the ⇒ I = I + I = − λC,+ λC + − λC,+ λC 4 1 3 2 2 2 2 copied timer value ce to compute the timestamp t˜e for the interrupt e: = [−λC,+λC) 0 tTS t˜e = ·λC −∆corr = ce ·λC −n·λC = (ce − n) ·λC λC 3.1 An Implementation Example (10) Since ce (timer driven) and n (constant) are inte- As an example, we’ll take a look at the integration gers of architecture word width, their subtraction is of our novel approach into the reference implemen- easily accomplished. Besides, the result’s resolution tation of SmartOS (Baunach et al., 2007) for the implicitly equals the resolution of the system time. MSP430F1611 (Texas Instruments Inc., 2006) MCU However, we still have to prove the symmetry about 0 and the SNoW5 sensor nodes (→ Figure 1) . While for the error intervals in Table 1: the main clock drives the CPU at f = 8 MHz, the di- Lemma 1. For our discretization approach the error vider α = 8 derives the frequency fC = 1 MHz for the system time with a resolution of 1 µs. According to intervals I1,I2,I3, and I4 for taking timestamps, for measuring and specifying delays, as well as for com- Eq. (7), an adequate delay ∆c between each interrupt puting reaction times are symmetric about 0. occurrence and the acquisition of its timestamp can be adjusted through n: Proof. While I3 is not affected by our novel approach, α I ,I I ∆c := n·α + = n·8 + 4 with n ∈ N0 (11) the demanded symmetry of 1 2, and 4 can easily be 2

67 SENSORNETS2014-InternationalConferenceonSensorNetworks

1 __irq_e: 0 2 ; hardwareIRQ latency; +6CPU cycles from te ∆IRQ = 6λ 3 nop ; +1CPU cycle\

4 nop ; +1CPU cycle> ∆ISR = 6λ 5 mov &TIMER_COUNTER,&TS; +4CPU cycles for latch in/

6 ; Total delay of captured timestamp: ∆TS = ∆IRQ + ∆ISR = 12λ = 1.5µs 7 8 mov #e, &__irq_number; saveIRQ number for subsequent processing in the associated event handler 9 jmp __kernel_entry; jump to kernel code

Listing 1: Timestamping within a specially prepared kernel ISR for IRQ number e.

Listing 1 shows the standardized kernel ISR for any is received and timestamped (t˜0) by the other node interrupt with number e: Since the CPU inherently de- A through the just presented timestamping approach. lays the acceptance of an interrupt by ∆IRQ = 6 CPU After some fixed delay ∆delay the signal will be re- cycles, we already have to select n ≥ 1. In fact, we turned by node A, and in turn the other node B catches, did select n = 1 and thus have to wait for an additional timestamps, and returns the signal after the same de- number of ∆ISR := ∆c − ∆IRQ = 6 CPU cycles within lay ∆delay. Having received the last trigger en with lo- the ISR (1·8 + 4 = 6 + 6). According to the specifi- cal timestamp tñ in a perfect system, the observed de- cation of the mov instruction, which is used for saving lay ∆˜ total,n between each node’s captured first and last the timer value TS in Line 5, it takes 4 CPU cycles signal timestamp should obviously equal the mathe- 0 until the value is read from the special function reg- matically expected delay ∆total,n: ister TIMER COUNTER. The remaining two cycles are ˜ ! 0 filled up by nop instructions. After the acquisition of ∆total,n := tñ −t˜0 = 2n·∆delay =: ∆total,n (13) the counter value, the specific IRQ number is saved However, this equality will commonly not be ob- and the kernel mode is entered for the actual event servable in real systems. In fact each involved device handling. will suffer from its own and the other device’s impre- To save CPU time the ISR will only save the cur- cision: rent 16 Bit timer value which indicates the delay since First, the nodes apply independent clocks, drift the last timeline update. The computation of the final apart, and, though globally fixed, they will finally not absolute timestamp t˜e is initially avoided, and delayed defer their responses by exactly the same delay ∆delay. until the application’s event handler requests this in- Though our nodes’ CPUs are driven by quartzes from formation from the OS. Then, according to Eq. (9), n the same lot, the clock drifts vary depending on the is simply subtracted from the event’s absolute counter selected node pair and the environmental influences value ce, which in turn is the sum of the timeline and described before. In fact, Figure 3 shows significantly the just captured timer value TS. With Eq. (10) the different drifts dA,B(t) for three node pairs measured result can directly be interpreted as absolute system over some time t.5 time given in the timeline resolution of 1 µs: Second, the responses must be scheduled and ini- tiated by the responsible task on each node. There- t˜ = (c − n)·λ = (timeline + TS − n)·λ (12) e e C C fore, these tasks compute their next intended local re- ∗ Note that the applied computation is correct, as long sponse time tr from each previously captured signal as the IRQ handlers are always executed in kernel timestamp, and then sleep to release the CPU for other mode where further interrupts are disabled and nei- tasks. However, waking up sufficiently early to emit ther the timeline nor TS will change concurrently. the signal in time is not that easy since some load- dependent and variable system overhead must always be taken into account. 4 EVALUATION AND Third, the base for each delay computation is never perfect since each captured timestamp t˜c ex-

APPLICATION EXAMPLE hibits an inherent error Et˜c ∈ I1. While this cannot be 5 The test bed for demonstrating the beneﬁt of our The nodes with IDs 10, 11, and 72 were arbitrarily se- A,B lected from our pool. The drift was measured via an oscillo- timestamping approach consists of pairs of nodes scope tracking the delay between two periodically triggered playing some sort of Ping Pong game as depicted in I/O pins at both nodes of each pair. As a plausibility test, Figure 2: By a wired or wireless remote connection, the measured drift of each pair corresponds perfectly to the µs µs µs one node, WLOG B, triggers an IRQ signal e0 which other pairs’ drifts: 918 100s + 1900 100s = 2818 100s .

68 HandlingTimeandReactivityforSynchronizationandClockDriftCalculationinWirelessSensor/ActuatorNetworks

35000

30000 28580 25760 25000 22940 20120 19165 20000 17300 17264 15364 14480 15000 13463 11660 11563 9662 9431 8839 8515 10000 7763 7599 6682 6021 5767

trigger 5862 4851 trigger 3932 5000 3181 3960 2060 400 160 3013 0 1175 2094 256 0 100 200 300 400 500 600 700 800 900 1000

Figure 2: The Ping Pong test Figure 3: Clock drift for three node pairs. bed setup (remote connections can be wired or wireless). avoided entirely as discussed before, the error should nodes’ x ∈ {A,B} local timing error ex which was au- at least be about 0 µs in the average case according to tonomously calculated by each node after n iterations: Lemma 1. Eq. (13) ˜ 0 ex := ∆total,n −∆total,n = (tñ −t˜0)−2n·∆delay (16) 4.1 Signal TX and Self-Calibration Obviously, both timing errors eA,eB have different For the precisely timed signal emission, we pro- sign unless the clocks are perfectly synchronous (then pose a dynamic self-calibration scheme based on self- eA = eB = 0 µs). Additionally, we define the symme- observation. Therefore, the trigger signal will not try error esymm as seen by an external observer as the only be captured by the other node where it is tagged average value over eA,eB. Since the average timestamp error Et,av ∈ I1 will accumulate over the two ac- with the timestamp t˜c, but also by the emitting node itself. We denote the corresponding local timestamp as quired trigger timestamps within each iteration, we expect t˜r. If the intended local response time for the current ∗ eA + eB iteration has been computed as tr , the lateness can be esymm := = 2n·Et,av. (17) computed afterwards and used as compensation value 2 ∆comp to adjust the delay for the next iteration at its If we indeed achieved the timestamp error in- ∗ emission time tr : terval I1 to be symmetric about 0, i.e. by selecting ∆ = n·α+ 1 ·α properly according to Eq. (7), we can ∆ := t˜ −t∗ (14) c 2 comp r r expect two observations for any pair of nodes A,B: ∗ tr := t˜c + ∆delay − ∆comp (15) 1. If both values eA and eB are made available to an ∗ In fact, the response time precision error (Etr ∈ external observer, their measured clock drift dA,B, I4) depends not only on the two timestamps and their as depicted in Figure 3, can be verified through particular precision error (Et˜r ,Et˜c ∈ I1), but also on the 0 ! error in the measured delay (E∆comp ∈ I2) and the hard dA,B := eA −eB = dA,B with dA,B = −dB,A. coded delay for the reply (E∆delay ∈ I3) itself. Since we (18) intentionally selected := m· with m ∈ N, at ! ∆delay λC 2. According to Eq. (17), esymm = 2n·0µs = 0µs, and I = least this value is free from rounding errors and 3 : thus both values eA and eB will show the same ab- [0;0) for this special application. solute values. In direct consequence, each node can autonomously estimate its own drift towards 4.2 Pairwise Drift Calculation the other node autonomously: ˜ For our tests we set up various node pairs A and B as dA,B = 2·eA (for node A) (19) depicted in Figure 2, and we were interested in each d˜B,A = 2·eB (for node B) (20)

69 SENSORNETS2014-InternationalConferenceonSensorNetworks

1483.3 1454.3 1461.4 1474.3 1484.8 1500 1397.3 1407.6 1430.0 1440.7 Expected: 2818 1000

s] d' = d' = d' = d' = d' = d' = d' = d' = d' =

µ 500 2823.0 2822.0 2816.6 2839.0 2835.0 2839.7 2823.8 2828.5 2825.8 esymm 71.8 -13.7 -0.7 10.5 23.2 34.5 49.5 60.0 71.9 0 e72 Expected: 0.0 Expected: 50.0 e10 -500 time errors [ -1000

-1500 -1339.7 -1424.7 -1409.0 -1409.0 -1394.3 -1385.3 -1362.4 -1354.3 -1341.0 10 11 12 13 14 15 16 17 18

1500 Expected: 1900

1024.7 950.3 965.0 997.7 995.7 1016.0 1016.0 1030.3 1000 924.3

s] d' = d' = d' = d' = d' = d' = d' = d' = d' =

µ 500 1909.3 1879.0 1901.3 1909.0 1947.0 1922.3 1934.0 1908.3 1920.0 esymm 70.0 -15.2 -0.4 10.5 24.2 34.5 49.0 61.8 70.3 0 e72 Expected: 0.0 Expected: 50.0 e11 -500 time errors [

-1000 -884.7 -954.7 -951.0 -944.0 -949.3 -926.7 -918.0 -892.3 -889.7 -1500

10 11 12 13 14 15 16 17 18

Expected: 918 1500 d' = d' = d' = d' = d' = d' = d' = d' = d' = 936.3 916.5 917.0 919.0 938.3 924.5 936.5 932.0 919.3 1000

s] 539.5 531.3 443.0 458.0 468.8 492.8 497.8 516.5 526.8 µ 500 71.6 esymm 71.4 -15.3 -0.5 9.3 23.6 35.5 48.3 60.8 0 e11 Expected: 0.0 Expected: 50.0 e10 -500 -396.8 -420.0 -405.3 -388.0 time errors [ -473.5 -459.0 -450.3 -445.5 -426.8 -1000

-1500

10 11 12 13 14 15 16 17 18 ∆ λ c [ ]

Figure 4: The node timing error for different values of ∆c after 100 s as autonomously measured by each node (see Figure 3 for the expected values).

70 HandlingTimeandReactivityforSynchronizationandClockDriftCalculationinWirelessSensor/ActuatorNetworks

Table 2: Drift calculation for ∆c = 12. Table 3: Drift calculation for ∆c = 16.

local information1,2 observer3 local information1,2 observer3 ˜ 0 ˜ 0 dA,B node 10 node 11 node 72 dA,B dA,B node 10 node 11 node 72 dA,B

d˜72,11 1900.0 1902.0 1900.0 1900 d˜72,11 1884.0 1836.0 2032.0 1900 d˜72,10 2818.0 2818.0 2816.0 2818 d˜72,10 2724.0 2870.0 2922.0 2818 d˜11,10 918.0 916.0 916.0 918 d˜11,10 840.0 1034.0 890.0 918 1 regular font: measured according to Eq. (19) 1 regular font: measured according to Eq. (19) 2 bold/italic: derived according to Eq. (21) 2 bold/italic: derived according to Eq. (21) 3 true drift as expected from Figure 3 3 true drift as expected from Figure 3

In particular, the exchange of any additional data, For any other values of ∆c violating Eq. (7), the such as timestamps, between the nodes is not nec- nodes can not gain reliable information about their essary to obtain this information (since ∆delay is relative drift autonomously. When using e.g. ∆c = constant). 2·8 = 16, Figure 4 shows values close to the expected symmetry error e = 2n· 1 λ = 50 µs (cf. The reason becomes clear when considering the symm 2 c Eq. (17)). As a result, Table 3 summarizes the au- involved error intervals over n iterations in Eq. (24): tonomously measured and computed drifts between Obviously, all error intervals remain symmetric about the node pairs, and reveals quite large and asymmet- 0 µs throughout the entire test. In particular, the aver- ric deviations towards the true drifts. age error for each variable is 0 µs, and consequently esymm = 0 µs, too. Besides the precision of the autonomous drift esti- In contrast, if we intentionally violate Eq. (7) by mation, another interesting metric is the resulting trig- using e.g. ∆c := n·α instead, the average times- ger frequency. The theoretical value tamp error interval would be symmetric around E = t −1 1 1 ftrig := 2·∆delay (22) 2 λC. Consequently, esymm = 2n· 2 λC, and neither the autonomous drift computation through Eq. (19) nor will not be visible in reality since neither node uses the external drift verification through Eq. (18) would a perfect clock. However, we would at least like to work any more. achieve 0 0 −1 4.3 Real-World Test Bed Analysis ftrig, av. = ∆delay,A + ∆delay,B , (23) which is definitely the best compromise two nodes Figure 4 shows the test bed results for the three al- A,B can find if their true drift compared to the perfect ready mentioned node pairs from Figure 3, and for global clock is unknown. Again, this is only possible various values of ∆c after n = 50 iterations with if esymm = 0. When looking at e.g. the graph for the 0 ∆delay = 1 s (∆total,n = 100 s). Note that the results nodes with IDs 11 and 72 in Figure 4, the extrapola- repeat in a cyclic manner with period α = 8, and thus tion of esymm leads to a symmetry error of −345.6 µs the values for ∆c = 10 are similar to those for ∆c = 18. for ∆c = 12 and 42336 µs for ∆c = 16 within one com- 8 When using ∆c = 1·8 + 2 = 12, we did indeed plete day. Thus, the larger |esymm| the larger the de- achieve the expected symmetry error esymm ≈ 0 µs viation from the intended frequency ftrig, av.. The ef- for all pairs. At least we received |esymm| < λC = fects are once more visible in Tables 2 and 3: For 1 µs, which is the timeline resolution and thus the best ∆c = 12 the values in each row are almost equal (i.e. timestamp precision a node can reach. Furthermore, consistent), while they exhibit significant variations 0 ˜ for ∆c = 12, dA,B ≈ dA,B verifies the measured values for ∆c = 16. from Figure 3. Most important, as shown in Table 2, the autonomously measured drifts between two nodes are almost perfect. Indeed, the maximum visible de- 5 CONCLUSIONS AND viation is in range ±2 µs. Another fact which we can verify from this table is, that since WLOG node A OUTLOOK knows its drifts dÃ,B and dÃ,C towards the two other nodes B and C respectively, it can also reliably derive In this paper we proposed an approach for obtaining the drift d˜B,C via precise timestamps t˜e for external events e, and for ensuring the precisely timed execution of reactions d˜B,C := dÃ,C − dÃ,B. (21) r at scheduled times t˜r. The error intervals for both

71 SENSORNETS2014-InternationalConferenceonSensorNetworks

∗ Eq. (15) tr := t˜c + ∆delay − ∆comp iteration I4 I1 I3 I2 1 1 1 1 1 : [−λC;λC)[− 2 λC; 2 λC)[0;0)(− 2 λC; 2 λC) ...... 1 1 1 1 n : [−nλC;nλC)[− 2 λC; 2 λC)[0;0)(−(n − 2 )λC;(n − 2 )λC) (24) Eq. (14) ∗ ∆comp := t˜r − tr iteration I2 I1 I1 3 3 1 1 1 : (− 2 λC; 2 λC)[− 2 λC; 2 λC)[−λC;λC) ...... 1 1 1 1 n : (−(n + 2 )λC;(n + 2 )λC)[− 2 λC; 2 λC)[−nλC;nλC)

t˜e and t˜r are symmetric about 0. While the first is Conference on Emerging Technology & Factory Au- achieved through the unified and carefully prepared tomation (ETFA 2012). IEEE Computer Society. preprocessing of interrupts by the kernel, the latter Baunach, M., Kolla, R., and Muhlberger,¨ C. (2007). Intro- becomes possible through a simple self-calibration duction to a Small Modular Adept Real-Time Operat- scheme at application layer. Both techniques proved ing System. In 6. GI/ITG KuVS Fachgesprach¨ Draht- lose Sensornetze. RWTH Aachen University. to be a great benefit for an inherent problem within distributed but interacting (embedded) systems: As Hewlett Packard (1997). Fundamentals Of Quartz Oscilla- tors. Electronic Counters Series. long as time is not properly manageable locally by Ito, K., Suzuki, N., Makido, S., and Hayashi, H. (2009). Pe- the individual nodes, network-wide synchronization riodic Broadcast Type Timing Reservation MAC Pro- and event or state tagging will hardly achieve the po- tocol for Inter-Vehicle Communications. In 28th IEEE tentially feasible precision. conference on Global telecommunications (GLOBE- Using our approach, a corresponding test bed ver- COM 2009). IEEE Press. ified, that it is possible to determine the drift be- Kuorilehto, M., Alho, T., Hannik¨ ainen,¨ M., and tween two nodes without the explicit exchange of any Ham¨ al¨ ainen,¨ T. D. (2007). SensorOS: A New Oper- quantitative information (like e.g. timestamps or pre- ating System for Time Critical WSN Applications. In 7th International Workshop on Embedded Computer viously measured delays). Instead, it is sufficient to Systems (SAMOS 2007), LNCS. Springer. periodically pass events (i.e. interrupts) between the Muhlberger,¨ C. (2013). On the Pitfalls of Desynchro- nodes. Since suitable periodic behavior can also be nization in Multi-hop Topologies. In 2nd Inter- found in several (wireless) communication protocols national Conference on Sensor Networks (SENSOR- like (Støa and Balasingham, 2011), (Ito et al., 2009), NETS 2013). SciTePress. (Mutazono et al., 2009), the proposed techniques can Mutazono, A., Sugano, M., and Murata, M. (2009). Frog also be applied to support time synchronization and Call-Inspired Self-Organizing Anti-Phase Synchro- self-organization among the involved systems. nization for Wireless Sensor Networks. In 2nd Inter- In fact, we already observed good time synchro- national Workshop on Nonlinear Dynamics and Syn- chronization (INDS 2009). nization results when integrating our approach into Romer,¨ K. (2008). Discovery of Frequent Distributed Event the Desync protocol from (Muhlberger,¨ 2013). An- Patterns in Sensor Networks. In 5th European Con- other objective for us is to support our timestamping ference on Wireless Sensor Networks (EWSN 2008). concept in hardware: Within a hardware/software co- Støa, S. and Balasingham, I. (2011). Periodic-MAC: Im- design project, a specifically prepared interrupt con- proving MAC Protocols for Wireless Biomedical Sen- troller of an experimental CPU architecture is already sor Networks through Implicit Synchronization. In able to pre-process and store timer values even for si- Biomedical Engineering, Trends in Electronics, Com- multaneously occurring events. munications and Software. InTech. Texas Instruments Inc. (2006). MSP430x161x Mixed Signal Microcontroller. Dallas (USA). Wittenburg, G., Dziengel, N., Wartenburger, C., and REFERENCES Schiller, J. (2010). A System for Distributed Event Detection in Wireless Sensor Networks. In 9th ACM/IEEE International Conference on Information Baunach, M. (2012). Towards Collaborative Resource Shar- Processing in Sensor Networks (ISPN2010). ing under Real-Time Conditions in Multitasking and Multicore Environments. In 17th IEEE International