INDustrial EXploitation of the
genesYS cross-domain architecture
Developing deterministic networking technology for railway applications using TTEthernet software-based end systems
Project n° 100021 Astrit Ademaj, TTTech Computertechnik AG Outline INDustrial EXploitation of the
genesYS cross-domain architecture GENESYS requirements - railway Time-triggered communication TTEthernet SW based implementation of the TTEthernet Conclusion
ARTEMISIA Association Title Presentation - 2 GENESYS – GENeric Embedded SYStems INDustrial EXploitation of the
genesYS cross-domain architecture Instruction how to build your embedded systems architecture GENESYS: ¾ is a reference architecture template providing specifications and requirements to design a cross domain embedded systems architecture. ¾ architecture style supports a composable, robust and comprehensible, component based framework with strict separation of computation from message based communication ¾ distinguishes between 3 integration levels: • Chip Level (IP cores communicate via a deterministic Network-on-a-Chip) • Device Level (Chips communicate within a device) • System Level (Devices communicate in an open or closed environment)
ARTEMISIA Association Title Presentation - 3 GENESYS and the railway domain INDustrial EXploitation of the
genesYS cross-domain architecture
Safety-critical applications in the railway domain require ¾ deterministic communication networks ¾ robustness and ¾ composability are key issues.
GENESYS ¾ architecture style supports a composable, robust and comprehensible, component based framework with strict separation of computation from message based communication ¾ distinguishes between 3 integration levels: • …. • System Level (Devices communicate in an open or closed environment)
ARTEMISIA Association Title Presentation - 4 TTEthernet INDustrial EXploitation of the
genesYS cross-domain architecture
TTEthernet is a suitable candidate to implement the integration at the system level ¾ For the railway domain … and not only Composability/determinism and robustness are key issue Enables a cost effective implementation of design diversity.
ARTEMISIA Association Title Presentation - 5 Time-Triggered communication INDustrial EXploitation of the
genesYS cross-domain architecture
Predictability ¾ Time-Triggered Communication ¾ Scalable
Composability ¾ Properties at the component level remain unchanged after integration ¾ TT communication architectures suitable ¾ Reduce testing and certification efforts ¾ Easy implementation of fault-tolerance mechanisms
Robustness ¾ Fault containment ¾ Error containment ¾ Replication of components
ARTEMISIA Association Title Presentation - 6 What time-triggered systems need INDustrial EXploitation of the
genesYS cross-domain architecture Any Time-Triggered System must have two key properties: a notion of time ¾ in case of a distributed system: a GLOBAL notion of time, available to each node in the system
a schedule (when to do what) ¾ in case of a distributed system: a GLOBAL schedule or CONSISTENT parts of a GLOBAL schedule available to each node in the system
ARTEMISIA Association Title Presentation - 7 Example: Time-Triggered vs Event- Triggered INDustrial EXploitation of the
genesYS cross-domain architecture
Transportation ¾ cars and taxis are event-triggered: they go whenever they are needed ¾ buses and trains are time-triggered: they go according to a fixed schedule Advantage of the event-triggered approach: very flexible Advantage of the time-triggered approach: very predictable When would you prefer a time-triggered solution?
ARTEMISIA Association Title Presentation - 8 Nondeterminism under Peak Load INDustrial EXploitation of the
genesYS cross-domain architecture
“Peak load” can corrupt established system properties ¾ The communication network fails to provide the properties established in functional verification – timing, latencies, error rates increase massively ¾ The control functions fail to respond to external influences (e.g. steering commands) in time – unspecified or unacceptable behavior occurs
Î for complex electronic systems with high availability, reliability, and safety requirements peak load scenarios must be avoided by design!
ARTEMISIA Association Title Presentation - 9 Addressing Peak Load in Critical Systems INDustrial EXploitation of the
genesYS cross-domain architecture
How can you ensure that a system stays reliable under load? ¾ A time-triggered system uses the same amount of resources and provides the same amount of throughput all the time ¾ In cases of low load, this performance is “wasted” ¾ But in cases of high load, unexpected loads or faults, no peak occurs
no peak load throughput 100 % ideal system time-triggered system
load requirements
“wasted” area
ARTEMISIA Association Title Presentation - 10 Composability with a Time-Triggered communication INDustrial EXploitation of the
genesYS cross-domain architecture Properties established at the component level are maintained after the system integration ¾ the properties of the communication schedule are defined by the system integrator before implementation and integration ¾the schedule is distributed consistently to all nodes ¾all nodes can communicate only according to this schedule Integration does not “change” anything – it only “completes” the communication pattern which was incomplete for each subsystem.
ARTEMISIA Association Title Presentation - 11 Composability with a Time-Triggered communication INDustrial EXploitation of the
genesYS cross-domain architecture
a R cd hk sop Communication schedule as designed by the ab Rt fg hk so system integrator - contains one spare aRcfghksop (green) slot ab Rt fg hk so acdhk ab fg hk Communication schedule for Subsystem A only acfghk ab fg hk
Rsop Rt so Communication schedule for Subsystem B only Rsop Rt so
Communication schedule for Subsystem C can be added without affecting A and B
ARTEMISIA Association Title Presentation - 12 Fault Tolerance and design diversity INDustrial EXploitation of the
genesYS cross-domain architecture Fault-tolerance is implemented by replication of system components ¾Two or more components perform the same services in parallel and provide their output simultaneously to mask failures of one of them. Tolerance against design failures – design diversity ¾ Use different specification to implement the same service
ARTEMISIA Association Title Presentation - 13 What is TTEthernet? INDustrial EXploitation of the
genesYS cross-domain architecture
A TT communication system, which integrates real-time and non real-time traffic into a single communication infrastructure
¾integrate traffic with different characteristics (requirements) in a flexible way ¾switched topology ¾Support application with different criticality requirements • data acquisition, multimedia, • real-time control app., safety-critical applications.
ARTEMISIA Association Title Presentation - 14 What is TTEthernet (2)? INDustrial EXploitation of the
genesYS cross-domain architecture
In principle we can enable any carrier protocol with time-triggered technology, … … but there are some good reasons for using Ethernet. ¾ Ethernet is a well-established open-world standard ¾ Scalable. ¾ Bandwidth (10 Mbit/s, 100 Mbit/s, 1Gbit/s, 10Gbit/s) ¾ COTS Ethernet hardware is low cost. ¾ Existing tools can be leveraged • cost-efficient monitoring tool (e.g., Wire Shark) • for maintenance and configuration (ssh, web servers,…).
ARTEMISIA Association Title Presentation - 15 TTEthernet Topology INDustrial EXploitation of the
genesYS cross-domain architecture Consist mainly of ¾TTE-Switches, ¾TTE- End Systems (ES) and Standard Ethernet End Systems
h T Et TE s wi tch T TE s wi tch
E TT
E TT h Et TE T E TT E TT
ARTEMISIA Association Title Presentation - 16 TTEthernet features INDustrial EXploitation of the
genesYS cross-domain architecture
Time-Triggered comm systems are ¾ deterministic, composable and scalable ¾ real-time comm. network compatible with IEEE 802.3. ¾ Standard Ethernet traffic does not affect the properties of the real-time traffic. End Systems HW based (dedicated chip/component) or SW based The software-based TTEthernet software based implementation uses COTS Ethernet controllers showcasing that TTEthernet can be implemented on any Ethernet compliant hardware. ¾ thus providing a cost-efficient and flexible technology implementation, allowing the usage of design diversity
ARTEMISIA Association Title Presentation - 17 Ethernet Standard IEEE 802.3 INDustrial EXploitation of the
genesYS cross-domain architecture
IEEE 802.3 addresses the lowest layers of the ISO/OSI reference model, some higher layers are represented by other IEEE 802 parts. TTEthernet performs services transparently within the Data Link layer, using all IEEE 802.3 services without modification.
7 Application architecture, NM, layers above (TCP,UDP,IP) 6 Presentation 5 Session Logical Link Control (IEEE 802.3 LLC) 4 Transport 3 Network Media Access Control (IEEE 802.3 MAC) 2 Data Link 1 Physical Physical Layer (IEEE 802.3 PHY) 10BaseT 100BaseTx 1000BaseCX … ISO/OSI layer model
ARTEMISIA Association Title Presentation - 18 TTEthernet Traffic Classes INDustrial EXploitation of the
genesYS cross-domain architecture TTE-frames - compatible to the standard Ethernet frame format. Destination MAC address is use to identify the frames, ¾ where the first 4 bytes represent the critical traffic marker (cluster ID), ¾ the last 2 bytes the critical traffic identifier (denoted also as message ID).
Schedule ID VLID
TTEthernet traffic classes ¾ Time-Triggered - TT (hard real-time) – configuration required ¾ Rate-Constraint – RC ¾ Best Effort - BE (or Event-triggered – ET, or background - BG)
ARTEMISIA Association Title Presentation - 19 Virtual Links INDustrial EXploitation of the
genesYS cross-domain architecture
End-Systems exchange frames through Virtual Links (VLs) A Virtual Link defines a unidirectional path from one End-System to one or more destination End-Systems
VL1
ES
Network ES ES
ES
VL2
ARTEMISIA Association Title Presentation - 20 TTEthernet TT traffic class INDustrial EXploitation of the
genesYS cross-domain architecture Time-Triggered (TT) ¾used for periodic exchange of messages ¾sending instant is triggered by the time - statically configured schedule ¾constant transmission delay and small and bounded jitter ¾networks can be utilized fully (close to maximum) due to the possibility of strictly deterministic communication scheduling ¾each TTE frame is transmitted by the end system at a certain time ¾the switch expects the frame from the transmitter within a certain time interval (window) • this provides an implicit bus guardian functionality: TTE traffic received outside of the expected time interval is discarded ¾switch forwards the frame to the receivers (end systems or other switches) at certain times - these times can be different for each port! ¾receivers receive the frame with well-defined latency and minimal jitter Best Effort (BE) Rate-Constraint (RC)
ARTEMISIA Association Title Presentation - 21 TTEthernet scheduling configuration INDustrial EXploitation of the
genesYS cross-domain architecture Senders have a defined transmit schedule Switches have an “acceptance schedule” for incoming data Switches have a “forwarding schedule” per port
VL ID Sender Receiver(s) 1 a @ 07:30 b @ [07:40-07:50]; d @ [8:20-8:30] 2 a @ 09:00 c @ [10:30- 10:40]; 3 b @ 10:00 c @ [10:20- 10:30]; e @ [10:25- 10:35],f @ [10:30- 10:40] … … … 8 b @ 11:15 a @ [11:30- 11:40]; f @ [11:30- 11:40] … … …
a g
3 VL ID Time b f 3 @ 10:00 c e 8 @ 11:15 d
ARTEMISIA Association Title Presentation - 22 TTEthernet Traffic Classes INDustrial EXploitation of the
genesYS cross-domain architecture Time-triggered - TT Best Effort- BE ¾Best effort traffic - BE (also denoted as event-triggered or background traffic) ¾usually are used for sporadic exchange of event information ¾sending instant is driven by an event ¾transmission delay – unknown ¾BE messages are stored in the switch • messages in the queue are processed on the FIFO order ¾BE messages shall be transmitted when communication medium is free of TT traffic ¾fully compatible with Ethernet standard Rate-Constraint – RC
ARTEMISIA Association Title Presentation - 23 TTEthernet Traffic Classes INDustrial EXploitation of the
genesYS cross-domain architecture Time-Triggered (TT) Best-Effort (BE) Rate-Constraint (RC) - AFDX ¾RC traffic class is defined by its • End-to-end transmission latency, • BAG – bandwidth allocation gap • Jitter. ¾BAG defined the maximum amount of bytes (or frames) per time interval. ¾RC traffic – can be shaped within an End-System, in order to ensure BAG times. ¾Different priorities
ARTEMISIA Association Title Presentation - 24 Fault Isolation INDustrial EXploitation of the
genesYS cross-domain architecture
Restricted access for configured VL Host Traffic Filtering – Firewalling ES At Switch and ES VL not configured at Switch
Switch
VL not configured at ES ES ES Host Host
ARTEMISIA Association Title Presentation - 25 SW based TTEthernet solution INDustrial EXploitation of the
genesYS cross-domain architecture TTEthernet End System Protocol Stack can be implemented in any general purpose computer that has a standard Ethernet interface. ¾ supporting not only the features of TT communication systems for predictability, composability, robustness ¾ … but also the flexible way for design diversity as different ES can be implemented into different targets.
ARTEMISIA Association Title Presentation - 26 SW architecture – TTEthernet INDustrial EXploitation of the
genesYS cross-domain architecture
TTEthernet core protocol is HW and OS independent TTE_API - message handling - status and diagnosis - control and configuration HW_API (low-level API) - for the Ethernet controller - API_ETH_CTRL - API for the HW timer - API_HW_TIMER
ARTEMISIA Association Title Presentation - 27 SW based TTEthernet core INDustrial EXploitation of the
genesYS cross-domain architecture TTEthernet core is HW and OS independent. It contains: ¾ Initialization, ¾ Start-up ¾ Dispatching/scheduling of action points according to the configuration • Clock synchronization • TTE message transmission and reception. • BG message transmission and reception. • Task execution. ¾ Error handling ¾ It provides the API functions to the host application ¾ It uses the low-level API functions for the Ethernet (reading incoming messages and triggering the start of transmission) and timer unit. ¾ Implements the TTE state machine, which is triggered by the timer interrupts (timer driver).
ARTEMISIA Association Title Presentation - 28 Ethernet and Timer driver INDustrial EXploitation of the
genesYS cross-domain architecture Ethernet driver contains the functions for: ¾ Ethernet controller initialization and configuration ¾ Allocation of Ethernet buffers and buffer descriptor for transmission (TX) and reception (RX) ¾ Frame handling • Frame transmission • Managing RX buffer descriptors (frame reception is handled automatically by the Ethernet HW unit) ¾ Raising interrupts on frame reception Timer driver ¾ Configuring one programmable timer with timer interrupt. ¾ Timeout function implementation Ethernet and Timer drivers are HW and OS dependent Porting of SW based TTEthernet is equivalent with the development of these two drivers
ARTEMISIA Association Title Presentation - 29 Middleware layer – Linux example INDustrial EXploitation of the
genesYS cross-domain architecture
Host app. ET traffic Host app. TT traffic
TT-Ethernet core protocol
Fast Ethernet Controller driver HW timer driver
HW
System without OS support System with OS support
ARTEMISIA Association Title Presentation - 30 Middleware layer INDustrial EXploitation of the
genesYS cross-domain architecture
To support the usage of existing operating systems mechanism Linux ¾ET messages (background communication) • eth0 device driver ¾TT messages • eth1 device driver for all TT messages
• eth1, eth2, eth3, … ethn device driver for each TT messages • Char device file for each TT message •…
ARTEMISIA Association Title Presentation - 31 Performance INDustrial EXploitation of the
genesYS cross-domain architecture
1.6 GHz CPU Intel ATOM, 1 GB RAM, 0.5 MB cache, Standard Linux OS Cluster cycle: 3 ms, 1 Sync msg/cycle Dummy application sending dummy TTE data with length of 1,500 bytes Measurement with Linux command “top”
TTE Messages Bandwidth CPU Utilization Configuration 1 2 8 MBit/s 1 % (1,500 bytes each) Configuration 2 6 24 MBit/s 2 % (1,500 bytes each) Configuration 3 10 40 MBit/s 3 % (1,500 bytes each) Configuration 4 15 60 MBit/s 3 % (1,500 bytes each)
ARTEMISIA Association Title Presentation - 32 Block print INDustrial EXploitation of the
genesYS cross-domain architecture
Approx 12 KLOC TTE-Core and configuration requires 20 KB memory Minimum of 20 KB necessary for Ethernet buffers memory
Ported to different targets ¾ With no OS small memory 64 KB
¾ ENEA OSE OS
¾ Standard Linux
¾ Linux with RT extension • Industrial PC (100 Mb/s and 1Gb/s) • EeePC (100 Mb/s)
ARTEMISIA Association Title Presentation - 33 Summary INDustrial EXploitation of the
genesYS cross-domain architecture SW based TTEthernet
¾ cost effective way for implementing deterministic communications systems
¾ Design diversity Software based TTEthernet will be ported in the HW target used in the railway industry and it will be used for investigation of robustness services in the course of the INDEXUS project by using a railway app.
ARTEMISIA Association Title Presentation - 34 INDEXYS: http://www.indexys.eu
INDustrial EXploitation of the
genesYS cross-domain architecture
Thank you for your attention
Astrit ADEMAJ, Senior TTEthernet Project Engineer TTTech Computertechnik AG Tel: +43 1 585 34 34 986 Mail-to: [email protected]