Deterministic Ethernet for Real-Time and Critical Applications Wilfried Steiner [email protected]
Total Page:16
File Type:pdf, Size:1020Kb
Deterministic Ethernet for Real-Time and Critical Applications Wilfried Steiner [email protected] https://at.linkedin.com/in/wilfriedsteiner www.tttech.com © NASA © Boeing 787 NASA Orion Vestas Wind Turbines Reliable Networks and Safety Controls from TTTech Thales Railway Signalling Audi Piloted Driving Prinoth Leitwolf www.tttech.com 2 Virtualization of Control Fully Integrated Safety Control Functions Real-Time Machine-to-Machine Communication Unconstrained Access www.tttech.com to Data From Machine to Fog / Cloud 3 • 3 Content • Deterministic Networking • Synchronized Communication • Non-Synchronized Communication • IT/OT Convergence • Recap of Audio/Video Bridging (AVB) • IEEE 802.1 Time-Sensitive Networking (TSN) • Complementing Deterministic Functions • Conclusions www.tttech.com 4 Deterministic Networking 5 Switched Ethernet • Today we are mostly concerned with switched Ethernet • Switches (correct technical term according to the IEEE 802.1 standards is “bridge”) • End Systems (or end points / end stations) • Sometimes Switch and End System are integrated www.tttech.com 6 Ethernet Frame Format 7B 1B 6B 6B 4B 2B 42B – 1500B 4B 12B 802.1Q MAC MAC Ethertype/ Preamble “VLAN” Payload FCS IFG SOF Destination Source Length Tag 16 bits 3 bits 1 bit 12 bits Tag Protocol Priority Code Drop Eligible VLAN Identifier Point Indicator Identifier www.tttech.com 7 Example Network 1 EN 3 4 5 EN EN EN 1 2 3 4 5 SW SW SW SW SW 2 EN 8 7 6 SW SW SW Physical Topology (Ethernet Links) Logical Topology Port 8 7 6 www.tttech.com EN EN EN 8 Switch overview 4 Traffic Policing Switching Traffic Shaping o Deterministic Fabric Queue of Ethernet Switches Traffic Class 0 n o i t c e l e S Queue of n Traffic Class 1 o o Embedded i s s i . Switch Chips m . s . n a r Queue of T Traffic Class 7 o Deterministic Ethernet IP Solutions www.tttech.com 9 Deterministic Network Objectives • Known upper bounds on latency through the network • Can be achieved by • synchronized communication (TT) • constrained unsynchronized communication (e.g., RC) www.tttech.com 10 Synchronized Communication (TT) • Time-Triggered Communication is build on two principles • Synchronized global notion of time • Communication schedule • Traditionally this is calculated and distributed offline as part of a device configuration www.tttech.com 11 C NI C Synchronized Communication NI IC SW N IT CH SW IT CH C NI C NI C NI C NI C NXI SW IT CH IC Synchronous Communication N X C NI C Exactly one order of messages NI (in contrast to PERM( M i ) in M async i . comm ) Synchronized Global Time • A protocol has to be used to synchronize the local clocks in the switches and end systems. • There are various protocols defined in the academic literature as well as in industrial standards. • Three important synchronization protocols are: • SAE AS6802 • IEEE 1588 • IEEE 802.1AS www.tttech.com 13 Synchronization Protocols (cont.) • Synchronization protocols synchronize the local clocks, which means formally, they ensure that: • at any point in real time, when the system is synchronized, then the difference of clock readings of any two non-faulty clocks in the system does not deviate more than a defined value (which we call the precision) www.tttech.com Late Clock Perfect Clock Early Clock 14 Synchronization Protocols (cont.) • Synchronization protocols differentiate themselves with respect to how well they synchronize: • What is the precision (ms, us, ns, sub-ns)? • Is the precision probabilistic or deterministic? • What failures are tolerated (fail-silent, Byzantine)? • Are startup/recovery protocols defined? • How fast do they converge from an unsynchronized to a synchronized state? www.tttech.com 15 Communication Schedule • Message transmissions from different end systems and/or switches are separated in time to avoid congestions. • For examples, if two messages need to traverse the same switch-to-switch link (aka multi-hop link) then the senders of the two messages can agree to send their messages at different times. www.tttech.com 16 Communication Schedule (cont.) • This scheduling problem becomes quite complex because of: • number of messages to be scheduled • size of the network • efficient network utilization • dependencies between messages (e.g., “m1 has to be sent x usec before m2”) www.tttech.com 17 2 1 5 3 Dataflow Links are enumerated 4 6 on the x-axis 1 2 … X www.tttech.com 18 Communication Schedule (cont.) • Different strategies exist to solve the scheduling problem: • Genetic algorithms (like simulated annealing) • Usage of general purpose search tools: • Integer Linear Programming (e.g., CPLEX) • Constraint Programming • SMT Solving www.tttech.com 19 Communication Schedule (cont.) • The problem of finding a communication schedule is not necessarily an optimization problem (i.e., search for the best solution out of a set of valid solutions) • Indeed it is in most use-cases a satisfiability problem (i.e., search for one existing solution) • E.g., upcoming TTTech tools will move from genetic algorithms towards SMT-based solutions. • Other tools are under research. www.tttech.com 20 C NI Unsynchronized Communication C NI IC SW N X IT CH SW IT CH C X NI C NI C NI C NI C NXI SW I TC H C NI Asynchronous Communication C NI . Transmission Points in Time are not predictable C NI Transmission Latency and Jitter accumulate Number of Hops has a significant impact Deterministic Unsynch. Traffic Rate-Constrained Traffic (RC) Sw itc h/ Ro r ute ive r ce Re er nd Se min. duration min. duration min. duration Deterministic Unsynch. Traffic • Sophisticated tools are necessary (but available) to calculate the latency/jitter/buffer for unsynchronized traffic, e.g.: • Network Calculus • Trajectory Approach • Response-Time Analysis www.tttech.com Information Technology (IT) Operations Technology (OT) Convergence 24 IT-OT Convergence • Information Technology: (office) Ethernet, SDN, Data Centers, Internet, high throughput, performance, etc. • Operations Technology: embedded systems, cyber- physical systems, real-time, fault-tolerance, robustness, etc. • Information Technology and Operations Technology converge • Motor that drives smart* developments www.tttech.com OT – Ethernet Variants • Commercial products: PROFINET, EtherCAT, Ethernet Powerlink, Ethernet IP, AFDX, etc. • Academic results: Flexible Time-Triggered Ethernet, Time-Triggered Ethernet, switched Ethernet, ... www.tttech.com 26 Two Main Drivers for Convergence • Industrial Automation: • Industrial Internet of Things, Industrie 4.0, etc. • Automotive Industry: • ADAS, etc. Standard Ethernet becomes more and more ready for OT use. www.tttech.com 27 Organizations involved • IEEE 802.1: a working group within which the IEEE 802.1 TSN task group is operational – here switch functions are being defined • AVnu: industry consortium that defines inter- operability for AVB and TSN products; different profiles are defined depending on application area • Open Alliance: has distributed Broad-R-Reach and is now capturing Automotive switch requirements www.tttech.com 28 IEEE 802.1 AVB Summary • 802.1AS: clock synchronization protocol. • 802.1Qat: Stream Reservation Protocol (SRP). • 802.1Qav: Forwarding and Queuing Enhancements for Time-Sensitive Streams • 802.1BA: definition of profiles for AVB systems. The AVB projects have been published in 2011 ! www.tttech.com 29 IEEE 802.1 TSN Introduction • IEEE 802.1 working group maintains several task groups • IEEE 802.1 Time-Sensitive Networking (TSN) is one of these task groups (others are, e.g., security) • For some projects IEEE 802.1 closely interoperates with IEEE 802.3 which maintains and extends the Ethernet PHY and MAC standards. www.tttech.com 30 TSN projects (#10) overview • .1AS-rev: synchronization improvements • .1Qbv: time-triggered queues • .1Qbu: frame preemption and resumption • .1CB: stream identification and redundancy management • .1Qca: redundant route configuration • .1Qcc: configuration and SRP improvements • .1Qch: cyclic queuing and forwarding • .1Qci: per-flow policing and filtering • .1Qcr: asynchronous traffic shaping • .1Qcs: improved reservation/registration protocol (a.k.a. MRP++) www.tttech.com 31 TSN projects status (Mar/2016) www.tttech.com 32 .1AS-rev • Improvements to the .1AS synchronization standard. • Support for multiple grandmaster clocks • Support for multiple routes through the network • Merge of the time from the different grand masters is not standardized (for now) • One-step clock • Fine-Time Measurement for Wireless www.tttech.com 33 .1Qbv: time-aware shaping Time-Aware Shaping Synchronized Clock Schedule T00: Oc...O T01: cO...O … Queue of Gate Traffic Class 0 n o i t c e l e S Switching Queue of Gate n Traffic Class 1 o Fabric i s s i . m . s . n a r Queue of T Gate Traffic Class 7 www.tttech.com 34 .1Qbu: frame preemption and resumption • Ongoing frame transmission can be interrupted. • Differentiation between preemptable and preemptive frames by traffic class. • Preemption generates framelets. • Minimum Ethernet frame size is respected, as a consequence a frame (or remaining frame) of 127 bytes cannot be preempted. • Standard PHYs can be used. www.tttech.com 35 .1CB: stream identification and redundancy management • Defines which bits to use for the identification of a stream (flow) • Defines a frame redundancy management algorithm similar to HSR/PRP – i.e., how to merge multiple redundant copies of the same frame into a single frame. • Redundancy information is carried in a Redundancy Tag. www.tttech.com 36 .1Qca: redundant route configuration • Based