Simulation of the IEEE 1588 Precision Time Protocol in Omnet++

Simulation of the IEEE 1588 Precision Time Protocol in OMNeT++

Wolfgang Wallner [email protected]

September 15, 2016 Basic concepts Clock Model Precision Time Protocol

Implementation Clock Model for OMNeT++ PTP in OMNeT++

Conclusion

Presentation Outline

Introduction Motivation Problem statement Implementation Clock Model for OMNeT++ PTP in OMNeT++

Conclusion

Presentation Outline

Introduction Motivation Problem statement

Basic concepts Clock Model Precision Time Protocol Conclusion

Presentation Outline

Introduction Motivation Problem statement

Basic concepts Clock Model Precision Time Protocol

Implementation Clock Model for OMNeT++ PTP in OMNeT++ Presentation Outline

Introduction Motivation Problem statement

Basic concepts Clock Model Precision Time Protocol

Implementation Clock Model for OMNeT++ PTP in OMNeT++

Conclusion Presentation Outline

Introduction Motivation Problem statement

Basic concepts Clock Model Precision Time Protocol

Implementation Clock Model for OMNeT++ PTP in OMNeT++

Conclusion I Requirements depend on the application

I Precision I Cost I Fault tolerance I ...

I IEEE 1588 speciﬁes the Precision Time Protocol (PTP)

Motivation

Motivation:

I Distributed real-time systems need a global time base I IEEE 1588 speciﬁes the Precision Time Protocol (PTP)

Motivation

Motivation:

I Distributed real-time systems need a global time base

I Requirements depend on the application

I Precision I Cost I Fault tolerance I ... Motivation

Motivation:

I Distributed real-time systems need a global time base

I Requirements depend on the application

I Precision I Cost I Fault tolerance I ...

I IEEE 1588 speciﬁes the Precision Time Protocol (PTP) I Experimenting with real hardware is expensive

I Prohibitive for experiments with larger networks

⇒ Use simulation to avoid costs and provide ﬂexibility

Motivation

I PTP provides a large feature set

I Design space exploration is challenging ⇒ Use simulation to avoid costs and provide ﬂexibility

Motivation

I PTP provides a large feature set

I Design space exploration is challenging

I Experimenting with real hardware is expensive

I Prohibitive for experiments with larger networks Motivation

I PTP provides a large feature set

I Design space exploration is challenging

I Experimenting with real hardware is expensive

I Prohibitive for experiments with larger networks

⇒ Use simulation to avoid costs and provide ﬂexibility Presentation Outline

Introduction Motivation Problem statement

Basic concepts Clock Model Precision Time Protocol

Implementation Clock Model for OMNeT++ PTP in OMNeT++

Conclusion I Requirements:

I simple I eﬃcient I realistic

Problem statement

I Simulation goal Provide a tool for PTP design space exploration Problem statement

I Simulation goal Provide a tool for PTP design space exploration

I Requirements:

I simple I eﬃcient I realistic Problem statement

Simulation components: Problem statement

Simulation components: Simulation Components: Network

Network components

I Prior work suggets usage of OMNeT++

I INET library provides common network components Simulation Components: Clocks

Clocks

Various noise sources:

I Random noise

I Deterministic inﬂuences

I Environment, Aging, Drift, . . . Simulation Components: Clocks

Clocks

Various noise sources:

I Random noise

I Deterministic inﬂuences

I Environment, Aging, Drift, ... Simulation Components: PTP

PTP Components

I PTP Hardware

I Timestamping NICs, . . .

I Software Components

I PTP Stack I Clock Servo Problem statement: Summary

Problem Statement (revised)

I Implement PTP in OMNeT++

I Provide realistic clock noise Presentation Outline

Introduction Motivation Problem statement

Basic concepts Clock Model Precision Time Protocol

Implementation Clock Model for OMNeT++ PTP in OMNeT++

Conclusion Clock modelI

The model of a digital clock consists of two parts:

I Oscillator

I Counter

I Both parts may suﬀer from noise Clock modelII

Modiﬁed clock model:

I Oscillator and counter are perfect

I Additional components:

I Noise generator I Correction stage Basic deﬁnitions

Frequency Stability Analysis Discipline of judging clock stability

Important attribute: Time Deviation (TD) Instantaneous time departure from a nominal time

→ How wrong is this clock (now)? Visualization of Time Deviation Combined Powerlaw Noise

Figure: Combination of diﬀerent PLNs Presentation Outline

Introduction Motivation Problem statement

Basic concepts Clock Model Precision Time Protocol

Implementation Clock Model for OMNeT++ PTP in OMNeT++

Conclusion PTP - Intro

Precision Time Protocol

I Network based clock synchronization

I Compromise between cost and precision I Clocks have ports

I Ports have states

I Ports communitate via messages I Nodes can

I timestamp messages I scale their local clock

PTP - Concepts

PTP Concepts

I Network nodes are called clocks I Ports have states

I Ports communitate via messages I Nodes can

I timestamp messages I scale their local clock

PTP - Concepts

PTP Concepts

I Network nodes are called clocks

I Clocks have ports I Ports communitate via messages I Nodes can

I timestamp messages I scale their local clock

PTP - Concepts

PTP Concepts

I Network nodes are called clocks

I Clocks have ports

I Ports have states I Nodes can

I timestamp messages I scale their local clock

PTP - Concepts

PTP Concepts

I Network nodes are called clocks

I Clocks have ports

I Ports have states

I Ports communitate via messages PTP - Concepts

PTP Concepts

I Network nodes are called clocks

I Clocks have ports

I Ports have states

I Ports communitate via messages I Nodes can

I timestamp messages I scale their local clock I Best Master Clock algorithm I Root is called grand master

I Timestamp broadcast I Path Delay measurement

PTP - Services

PTP Services

I Clock hierarchy

I Oﬀset estimation

I Conﬁguration I Timestamp broadcast I Path Delay measurement

PTP - Services

PTP Services

I Clock hierarchy

I Best Master Clock algorithm I Root is called grand master

I Oﬀset estimation

I Conﬁguration PTP - Services

PTP Services

I Clock hierarchy

I Best Master Clock algorithm I Root is called grand master

I Oﬀset estimation

I Timestamp broadcast I Path Delay measurement

I Conﬁguration PTP - Services

PTP Services

I Clock hierarchy

I Best Master Clock algorithm I Root is called grand master

I Oﬀset estimation

I Timestamp broadcast I Path Delay measurement

I Conﬁguration PTP - Services

PTP Services

I Clock hierarchy

I Best Master Clock algorithm I Root is called grand master

I Oﬀset estimation

I Timestamp broadcast I Path Delay measurement

I Conﬁguration Presentation Outline

Introduction Motivation Problem statement

Basic concepts Clock Model Precision Time Protocol

Implementation Clock Model for OMNeT++ PTP in OMNeT++

Conclusion Clock model for OMNeT++

Clock model hierarchy

I Diﬀerent modules for diﬀerent tasks Clock model for OMNeT++

HardwareClock

I Convert global real-time to locale estimate

I Real-time: perfect, continous I Estimate: non-perfect, discrete Clock model for OMNeT++

TdGen

I Provide the Time Deviation (TD) for a given point in time

I Future TD values may be estimates Clock model for OMNeT++

AdjustableClock

I Provide abstraction on top of HardwareClock

I Add linear correction Clock model for OMNeT++

ScheduleClock

I Locale alternative to scheduleAt()

I Internal event queue Presentation Outline

Introduction Motivation Problem statement

Basic concepts Clock Model Precision Time Protocol

Implementation Clock Model for OMNeT++ PTP in OMNeT++

Conclusion PTP Stack

Basic PTP node

I Architecture is based on StandardHost and EthernetSwitch from INET library PTP Stack

PTP stack

I Implements core of IEEE 1588

I Message types I BMC algorithm I Port states I Data sets I Clock types I Delay mechanisms I ... PTP Ethernet Mapping

PTP Ethernet Mapping

I Annex F of IEEE 1588

I PTP over Ethernet PTP Clock servo

Clock servo

I Generic interface

I 1 implementation:

I PI controller Logical Link Control

Logical Link Control

I Layer 2 access

I Move frames to correct application based on EtherType PTP NIC

PTP NIC

I Clock

I PHY

I MAC Presentation Outline

Introduction Motivation Problem statement

Basic concepts Clock Model Precision Time Protocol

Implementation Clock Model for OMNeT++ PTP in OMNeT++

Conclusion Project State

Current project state

I Code released as GPL

I github.com/w-wallner

I Thesis ﬁnished

I Simulation of time-synchronized networks using IEEE 1588-2008 [7]

I Papers

I ISPCS1 2016, Stockholm I OMNeT++ Community Summit 2016, Brno

1Conference with a focus on Precision Time Protocol (PTP) Outlook

Future work

I PTP features I Hardware properties

I Deterministic clock inﬂuences I Switch models (queues)

I More clock servos Conclusion

Conclusion

I Simulation approach is feasible

I Clocks can be implemented eﬃciently I Assembling PTP networks is easy with Graphical User Interface (GUI)

I Simulation has already been useful for teaching PTP

I There is strong interest for such a simulation Questions

I Do you have any questions? Thanks

Thanks for your attention! AcronymsI

AAS Austrian Academy of Sciences ADEV Allan Deviation AVAR Allan Variance BC Boundary Clock BMC Best Master Clock DES Discrete Event Simulation E2E End-to-End FFM Flicker Frequency Modulation FPM Flicker Phase Modulation FSA Frequency Stability Analysis GUI Graphical User Interface LLC Logical Link Control NIC Network Interface Card OC Ordinary Clock OMNeT++ Objective Modular Network Testbed in C++ P2P Peer-to-Peer AcronymsII

PI proportional-integral PLN Powerlaw Noise PSD Power Spectral Density PTP Precision Time Protocol RW Random Walk TC Transparent Clock TD Time Deviation WFM White Frequency Modulation WPM White Phase Modulation ReferencesI

John C. Eidson Measurement, Control, and Communication Using IEEE 1588 Springer, 2006 Georg Gaderer, et al An Oscillator Model for High-Precision Synchronization Protocol Discrete Event Simulation Proceedings of the 39th Annual Precise Time and Time Interval Meeting, 2007 IEEE Std 1139-2008 IEEE Standard Deﬁnitions of Physical Quantities for Fundamental Frequency and Time Metrology - Random Instabilities 2009 ReferencesII

N. Jeremy Kasdin and Todd Walter Discrete Simulation of Power Law noise Proceedings of the 1992 IEEE Frequency Control Symposium, 1992 William J. Riley Handbook of Frequency Stability Analysis NIST Special Publication 1065, 2008 Enrico Rubiola The Leeson eﬀect - Phase noise in quasilinear oscillators ArXiv Physics e-prints, 2005 W. Wallner Simulation of Time-synchronized Networks using IEEE 1588-2008 Master’s thesis, Faculty of Informatics, Vienna University of Technology, 2016 Dropped Slides

Dropped Slides Motivation

PTP: Compromise between cost and precision Motivation

PTP: Compromise between cost and precision Problem statement

Reminder: Any simulation is just as good as its models.

Possible risks of simulation include:

I Too naive clock model → false positives

I Clumsy control loop → false negatives Frequency Stability Analysis - Intro

I We need to justify the stability of clocks

I This discipline is called Frequency Stability Analysis (FSA)

I Literature:

I Handbook of Frequency Stability Analysis[5] I IEEE 1139 [3] (Standard deﬁnitions for random instabilities) 2 σy(τ) Allan Variance (AVAR) Special variance to measure stabilty of clocks Useful for time domain analysis

Clock Noise: Description

Two important measures for description of noise:

Sy(f) Power Spectral Density (PSD) One-sided PSD of y(t) Useful for frequency domain analysis Clock Noise: Description

Two important measures for description of noise:

Sy(f) Power Spectral Density (PSD) One-sided PSD of y(t) Useful for frequency domain analysis

2 σy(τ) Allan Variance (AVAR) Special variance to measure stabilty of clocks Useful for time domain analysis Powerlaw NoiseI

Random noise in oscillators has a special PSD shape:

α Sy(f) ∝ f

Deﬁnition This is called Powerlaw Noise (PLN) Powerlaw NoiseII

Special cases for α:

I 2 WPM White Phase Modulation

I 1 FPM Flicker Phase Modulation

I 0 WFM White Frequency Modulation

I -1 FFM Flicker Frequency Modulation

I -2 RW Random Walk Powerlaw Noise examples I Alternative: Allan Variance (AVAR)

I Equally widespread: Allan Deviation (ADEV)

I Example:

Allan Variance

I Problem: standard variance does not converge I Equally widespread: Allan Deviation (ADEV)

I Example:

Allan Variance

I Problem: standard variance does not converge

I Alternative: Allan Variance (AVAR) I Example:

Allan Variance

I Problem: standard variance does not converge

I Alternative: Allan Variance (AVAR)

I Equally widespread: Allan Deviation (ADEV) Allan Variance

Allan Variance

I Problem: standard variance does not converge

I Alternative: Allan Variance (AVAR)

I Equally widespread: Allan Deviation (ADEV)

I Example: Relationship AVAR/PSDI

PLNs have characteristic AVAR:

Image was taken from [6]. Relationship AVAR/PSDII

Table B.2 of IEEE 1139[3]:

2 PLN Sy(f) σy(τ)

−2 1 RW h−2 · f A · h−2 · τ −1 0 FFM h−1 · f B · h−1 · τ 0 −1 WFM h0 · f C · h0 · τ 1 −2 FPM h1 · f D · h1 · τ 2 −2 WPM h2 · f E · h2 · τ

I A, B and C are constants

I D and E depend on certain parameters I Capable of timestamping outgoing frames on-the-ﬂy I Needs explicit hardware support

I Not capable to timestamp on-the-ﬂy I Use FollowUp messages

PTP - Timestamp modes

Diﬀerent timestamp modes:

I 1-step clocks

I 2-step clocks I Not capable to timestamp on-the-ﬂy I Use FollowUp messages

PTP - Timestamp modes

Diﬀerent timestamp modes:

I 1-step clocks

I Capable of timestamping outgoing frames on-the-ﬂy I Needs explicit hardware support

I 2-step clocks PTP - Timestamp modes

Diﬀerent timestamp modes:

I 1-step clocks

I Capable of timestamping outgoing frames on-the-ﬂy I Needs explicit hardware support

I 2-step clocks

I Not capable to timestamp on-the-ﬂy I Use FollowUp messages PTP - State machine

3 non-transient states:

I MASTER

I SLAVE

I PASSIVE PTP - Best Master Clock algorithm

I Clocks decide periodically about port states I Next port state depends on

I received Announce messages I timeouts I synchronization errors I ...

I Best Master Clock (BMC) is eventually consistent

I BMC results in a forest PTP - Simple BMC exampleI

I At ﬁrst, all nodes start in LISTENING PTP - Simple BMC exampleII

I They see an idle PTP network, and try to become MASTER PTP - Simple BMC example III

I As the nodes start to see Announce messages, some ports change to SLAVE PTP - Simple BMC exampleIV

I Final hierarchy BMC example: rings

I Example network with 1 good clock

I Passive states break rings BMC example: 2 excellent clocks

I Example network 2 excellent clocks

I Passive states divide network PTP - Synchronization principle

Two tasks:

I Timestamp distribution

I Delay estimation PTP - Clock typesI

Ordinary Clock (OC)

I 1 port

I typical end node PTP - Clock typesII

Boundary Clock (BC)

I multiple ports

I otherwise similar to OC PTP - Clock types III

Transparent Clock (TC)

I multiple ports

I tries to not inﬂuence the PTP network

I residence time correction

I introduced in IEEE 1588-2008 PTP - Delay mechanismsI

I End-to-End (E2E)

I Peer-to-Peer (P2P) PTP - Delay mechanismsII

I E2E: Slave measures and corrects full distance

I P2P: Each nodes measures and corrects small part

I Advantages E2E:

I Expected precision

I Advantages P2P:

I Reduced overhead

I Fast reaction on path change PLN simulation - Combining PSDsI

I Combined PSD results in expected AVAR PLN simulation - Prior work

Austrian Academy of Sciences (AAS) Prior work:

I Was engaged in PTP and PLN simulation

I Several publications, e.g.

I Gaderer, et al An Oscillator Model for High-Precision Synchronization Protocol Discrete Event Simulation, 2007[2]

I Served as inspiration PLN simulation - Prior work

Prior work: Kasdin/Walter Method

I N. Jeremy Kasdin and Todd Walter, Discrete Simulation of Power Law noise, 1992[4]

I Generic method for PLN generation

I Basis for AAS papers

I Approach: Filtering of white noise PLN simulation - Prior work

Kasdin/Walter approach: Filtering of white noise PLN simulation - Challenges

I Problem solved theoretically by KW-approach

I Too complex for practical simulation purpose

I Maximum simulation time is limited I Ineﬃcient for Discrete Event Simulation (DES) PLN simulation - Approaches

Maximum simulation time

I Combining PSDs with diﬀerent fs

I IIR ﬁlters for even α

Eﬃciency

I Skip unneeded PSD contributions PLN simulation - Combining PSDs

Combining PSDs PLN simulation - BenchmarkI

I Time Deviation at diﬀerent sampling rates

I Overall clock wander determined by Random Walk (RW) I High frequency noise is there when needed PLN simulation - BenchmarkII

I Simulation speed ∝ 1/fs

I Results on my systems (Intel Core i7 2.00GHz): PLN simulation - Benchmark III

I Correct ADEV for any sampling rate PTP NIC - MAC

PTP MAC

I Timestamps Event messages need ingress and egress timestamps

I Residence time correction When acting as a TC, the MAC must correct the residence time of outgoing frames PTP NIC - Clock

Clock

I Timestamps Used to timestamp events

I Scalable Controlled by Clock Servo

I Event scheduling PTP stack relies on it for timeouts ⇒ Clock Noise Node nodes

Node Symbols Example network

Example network Debugging and Logging

Debugging and Logging

Figure: Port States and State Decisions Message symbols

Message Symbols Example network

Example network with PTP devices and standard oﬃce gear. Experiment A1: Best Master Clock Algorithm

Experiment A1: Best Master Clock Algorithm Best Master Clock Algorithm

Figure: Example network from Eidson’s book[1]. Best Master Clock Algorithm

Figure: Simulation of the example network from Eidson’s book. Example Oscillators

I LibPLN implements 2 example oscillators Experiment 1: Sync Interval

Experiment 1: Sync Interval Simple Network

Simple test network Parameter Study: Sync Interval

Parameter Study: Sync Interval

Figure: Mean value of the oﬀset Parameter Study: Sync Interval

Parameter Study: Sync Interval

Figure: Jitter2 of the oﬀset

2|Max − Min| Experiment: Path Asymmetry

Experiment: Path Asymmetry Asymmetry

Conﬁguration

I Network with 2 PTP nodes

I 3 Conﬁgurations

I No path asymmetry I Path asymmetry without correction I Path asymmetry with correction Asymmetry