Real-Time Data Distribution Service (DDS)

Home , OSI model

Introduction to DDS

OMG Real-Time Workshop Washington DC July 2008

Gerardo Pardo-Castellote, Ph.D. Co-chair OMG DDS SIG CTO, Real-Time Innovations [email protected] www.rti.com Agenda

z Background: Middleware Technologies z Introduction: DDS Model & Applicability z Details: DDS in depth

z The concept of network middleware z Communications Model z Object Model z Architecture Model z Protocol

End User Application

Network Stack Routing protocols File System

Kernel Device Drivers Operating System

End User Application

HTTP Service Mail Service FTP Service Data base

Network Stack Routing protocols File System

Kernel Device Drivers Operating System

End User Application

Application Server Event Processing Engine Physics Engine Message Service Data base Middleware Graphics Engine

HTTP Service Mail Service FTP Service

Network Stack Routing protocols File System

Kernel Device Drivers Operating System & Pre-Packaged Services

Middleware = API and service layer above operating system and below “application” code that abstracts common interaction patterns

Network Middleware = Most popular class of middleware Middleware used for developing distributed applications

Distributed Applications = Those requiring interaction/communication between multiple computers

© 2008 RTI - All rights Reserved Network Middleware Examples z DCE/RPC, DCOM, CORBA, ICE z TIBCO, 29 West z JMS, MQ Series, ActiveMQ, SoniqMQ z DDS, RTI DDS, OpenSplice, tao-dds, GigaSpaces z New trend: – Application Servers (WebSphere, WebLogic, JBOSS) – Include network middleware as a component z NOTE: Middleware “packages” are building blocks, not stand-alone applications like… – Skype, gtalk, … – BitTorrent, eMule, … z Why aren’t ‘popular’ consumer applications built on top of middleware?

© 2008 RTI - All rights Reserved Historical note: From Telephone to Blogs z Why so many flavors? z Parallels evolution of general communication patterns: – Started with point-to-point connections – Then request-reply services – Then Message Queue Services – Then Publish-Subscribe Services – Then Data-Caching services z Other examples: – FTP, email -> WEB -> Blogs, RSS -> Podcasts

z Service model composed

of: Communications – Communications model Model – Object Model

– Architecture model Object Model

z All these interact Architecture Model

z The service model and PROTOCOL protocol are also coupled Middleware

Service Model = Communications Model + Object Model + Architecture Model z Communications Model: – Abstract model of how applications interact: z Remote Method Invocation z Queue Based, Pub-Sub z Data-Distribution z Replicated Data z Distributed Transactions z Object Model – Middleware entities the application uses to interact with the service: z Queues, Publishers, Domains, Caches, Federations, Remote Objects… z Architecture Model – Centralized, Brokered, Peer-to-Peer

z The concept of network middleware z Communications Model z Object Model z Architecture Model z Protocol

– Remote Method Invocation – Message-Oriented, Queue Based – Pub-Sub Data-Distribution – Replicated Data – Distributed Transactions

Point-to-Point Client-Server Telephone, TCP File systems, Database, RPC, CORBA, DCOM Simple, high-bandwidth Good if information is naturally centralized Leads to stove-pipe systems Single point failure, performance bottlenecks

Publish/Subscribe Messaging Replicated Data Magazines, Newspaper, TV Libraries, Distributed databases Excels at many-to-many Excels at data-mining and analysis communication Excels at distributing time-critical information © 2008 RTI - All rights Reserved RMI vs Pub-Sub/Messaging/Data-Distribution

z RMI (WebServices, CORBA, DCOM) offer a remote method abstraction – Familiar OO programming model – Results in a tightly-coupled system y Forces synchronous invocations y Imposes global object model y Limited QoS (appearance of local method call) y Lack robustness: cascading points of failure – Typically built on top of TCP: y impacts scalability and time-determinism – Best-suited to smaller, closely-coupled systems Topic/Queue z Pub-Sub (Messaging Data-Distribution) offer a queue- based and/or replicated-data model – Subsystems are decoupled in time, space, and synchronization y Contracts established by verifying QoS compatibility – Supports a variety of transports including multicast UDP – Better suited for high-performance and real-time

z Queue – Message sent to Queue 1 2 3 1 – Multiple readers can read from the queue 2 Queue – Each message is delivered to ONLY one reader z Readers “affect each other” 3 – Apps: z Job Scheduling z Load Balancing z Collaboration

z Pub Sub – Message Sent to Topic 1 2 3 – Multiple readers can subscribe to Topic 1 2 3 with or without filters 1 2 3 – Each message delivered to ALL Topic/Queue subscribers that pass filter z Readers are decoupled 1 1 3 – Apps: z Notifications z Information Distribution © 2008 RTI - All rights Reserved Pub-Sub versus Data-Distribution z Pub-Sub – Only messages no concept of data 1 2 3 – Each message is interpreted without 1 2 3 context 1 2 3 – Messages must be delivered FIFO or Topic/Queue according to some “priority” attribute No Caching of data – 1 3 – Simple QoS: filters, durability, lifespan 1 z Data-Distribution – Messages represent update to data-objects – Data-Objects identify by a key – Middleware maintains state of each object 1 2 2 3 – Objects are cached. Applications can read at leisure – Smart QoS z Ownership z History (per key) z Deadline 1 2 3 1 2 3 1 2 3 – Subsumes messaging

© 2008 RTI - All rights Reserved Other (non DDS) Commercial Pub-Sub Models z Older, but widely deployed – TIBCO (RendezVous, EMS) – IBM MQSeries z Limited deployment: – CORBA Event Service – CORBA Notification Service z Emerging standards – not really used yet – WS-Eventing – WS-Notification z Emerging – 29West – IBM LLM

Provides a “Shared Queue Space” that is accessible to all interested applications. – Message are sent to an Exchange – Each message has an associated Routing Key – Brokers forward messages to one or more Queues based on the Routing Key – Subscriber get messages from named Queues – Only one subscriber can get a given message from each Queue

Client Client Pub Sub Queue1

Client E Queue2 Sub

Client Pub E Queue3 Client Shared Queue Space Sub

Provides a “Global Data Space” that is accessible to all interested applications. – Data objects addressed by Domain, Topic and Key – Subscriptions are decoupled from Publications – Contracts established by means of QoS – Automatic discovery and configuration

Participant Participant Pub Pub

Track,2 Participant Sub Sub Track,1 Track,3 Global Data Space Participant Participant Pub Alarm Sub

Routing key Queue AMPQ? AMPQ?

Queue Publisher Exchange Binding Client Subscriber Client Queue Content Broker Server

z Exchange – Receives messages and routes to a set of message queues z Queue – Stores messages until they can be processed by the application(s) z Binding – Routes messages between Exchange and Queue. Configured externally to the application – Default binding maps routing-key to Queue name z Routing Key – label used by the exchange to route Content to the queues z Content – Encapsulates application data and provides the methods to send receive, acknowledge, etc.

Topic Topic

Data Offered Data Requested Writer QoS Reader QoS

Publisher Domain Subscriber Domain Participant Participant

Global Data Space z DomainParticipant – Allows application to join a DDS Domain (Global Data Space) z Topic – A string that addresses a group of objects in the Global Data Space – Each Object is identified by a Key (some fields within the object data) z Publisher, Subscriber – Pools resources for DataWriters and DataReaders z DataWriter – Declares intent to publish a Topic and provides type-safe operations to write/send data z DataReader – Declares intent to subscribe to a Topic and provides type-safe

z The concept of network middleware z Communications Model z Object Model z Architecture Model z Protocol

z Brokered – Centralized – Segmented – Federated z Peer to Peer

One central server materializes all middleware entities All traffic flows via server E.g. “naïve” implementations of JMS, CORBA Notification, etc.

Client Application Server

Pub-Sub Service

Each Queue/Topic Can be placed on a different Server E.g. Better implementations of JMS, CORBA Notification, etc.

Client Application Server

Pub-Sub Service

App uses messaging or RMI to interact with Service Access points Pub-Sub Service distributes messages internally between servers Internally PS-Service can be peer-to-peer, hub-and-spoke, multicast, etc.

Client Application Server

Pub-Sub Service

App links (binds) directly with the Pub-Sub service Queuing occurs locally on each client Clients communicate peer-to-peer

Client Application

Server

Pub-Sub Service

Model Examples Centralized Typical JMS implementations Brokered

Segmented Better JMS implementations (Grid Based) CORBA event & notification service

Federated Brokered TIBCO RendezVous TIBCO SmartSockets IBM WebSphere MQ (MQSeries) using client connection

Peer-to-Peer Most DDS implementations: RTI DDS, OpenSplice, Tao-DDS Un-brokered IBM LLM, 29West

z The concept of network middleware z Communications Model z Object Model z Architecture Model z Protocol

Client Protocol Service Service Protocol Model

z There are interactions between the service model and the Protocol z A Brokered Service model requires 2 protocols: – Client Protocol (used by client applications) – Service Protocol (used between the Brokers) z A fully peer-to-peer middleware requires only one protocol

Layer 6 The existing middleware Application operations perspectives are inadequate to model middleware protocols:

The 7-layer OSI model: Layer 5 z Has layers with very limited Middleware operations functionality: (Session Presentation) z Considers TCP and UDP both “layer 4 concepts” Layer 4 Reliability, Flow Control, The 5-layer TCP/IP model: Packet Assembly z Combines all layers above Transport into single “Application Layer” z Considers TCP and UDP Layer 3 equivalent Packet delivery & routing Some protocols may expand multiple layers

Invoke Operations with known semantics Layer 6 Send messages to queues Application operations Pub/Sub specific topics

Multi-layer protocols HTTP DTLS Layer 5 GIOP STUN A middleware protocol at this Middleware operations SOAP Level must operate on top of a FTP Layer4 protocol like TCP TLS

SCTP DTLS Layer 4 TCP STUN A middleware protocol at this Reliability, Flow Control, RTP Level can operate on top of Packet Assembly PGM UDP or Multicast

UDP Layer 3 IP Packet delivery & routing IP Multicast

Client Service Protocol Service Protocol Model

JMS: Queues Not Specified! Exchanges RoutingKeys Not Specified!

DDS: Not Needed! Domain DDS-RTPS Topic Key

Invoke Operations with known semantics Layer 6 Send messages to queues Application operations Pub/Sub specific topics

HTTP DTLS Layer 5 GIOP STUN Middleware operations SOAP FTP TLS DDS-RTPS is the only standard middleware protocol that supports reliable multicast SCTP DTLS Layer 4 TCP STUN DDS-RTPS Reliability, Flow Control, RTP Packet Assembly PGM

UDP Layer 3 IP Packet delivery & routing IP Multicast

© 2008 RTI - All rights Reserved Take away points z What and why of middleware – Components of network middleware – Common models: z Request Reply z Pub Sub z Queue Centric z Data Centric z Architectures used by Middleware – Brokered, Peer to Peer z Kinds of Protocols used by Network middleware – Layer 3, 4, 5 – Client to Service. Service to Service

Service Model: • Communication Model • Object Model • Architecture Model (+ required brokers)

Protocol: • Layer • Features • Transport Assumptions • Overhead

Impact/Capabilities: • Features: Reliability, Persistence, QoS, Batching • Impact: • Deployment • Performance (protocol overhead, #intermediaries)

www.rti.com Agenda

z History & Applications z What is Data-Centricity? z Top Reasons to use DDS

z Data Distribution Service for Real-Time Systems – Joint submission (RTI, THALES, OIS) – DDS version 1.0 Adopted December 2004 – DDS version 1.1 Adopted December 2005 – DDS version 1.2 Adopted October 2006

z Interoperability wire protocol – Joint submission (RTI and PrismTech) – DDS-RTPS version 1.2 adopted in July 2006 – DDS-RTPS version 2.0 adopted in June 2007 – DDS-RTPS version 2.1 approved in June 2008

z Related Standards – Joint submission (Sparx, RTI, PrismTech) – UML Profile for DDS adopted June 2008

z Standards under Development – DDS for light weight CCM work in progress – Extensible and Dynamic Topic Types for DDS – Native Language C++ API for DDS

Boeing AWAKS E2C Hawkeye Northtrop? Qinetiq? SAAB?

Boeing Future Combat Systems Raytheon SSDS

Lockheed AEGIS Insitu Unmanned Air Vehicles

44 DDS Adoption – Transportation, Industrial

WiTronix Train and vehicle Schneider Electric Tracking Industrial Automation

Tokyo Japan Traffic Control Kuka Robotics

EU and US Air Traffic Management Varian Medical Instruments

45 Main differences of DDS vs other Pub-Sub

z Flexibility and Power of the data-centric model

z Performance & Scalability

z Rich set of built-in services

z Interoperability across platforms and Languages

z Natural integration with SOA building-blocks

“Global Data Space” generalizes Subject-Based Addressing – Data objects addressed by DomainId, Topic and Key – Domains provide a level of isolation – Topic groups homogeneous subjects (same data-type & meaning) – Key is a generalization of subject z Key can be any set of fields, not limited to a “x.y.z …” formatted string

Data Reader Data Writer TopicKey (subject)

Data Object Data Reader Data Writer

Data Reader Data Writer

Display Area: Shows state of objects z Topics – Square, Circle, Triangle – Attributes z Data types (schemas) – Shape (color, x, y, size) z Color is instance Key – Attributes z Shape & color used for key z QoS – Deadline, Liveliness – Reliability, Durability – History, Partition Control Area: – Ownership

Data Domain Data Domain New Writer Participant Reader Participant “Alarm” Got new “Alarm” subscriber! data Offered Offered Listener QoS Listener QoS

z Participants scope the global data space (domain) z Topics define the data-objects (collections of subjects) z Writers publish data on Topics z Readers subscribe to data on Topics z QoS Policies are used configure the system z Listeners are used to notify the application of events

QoS Policy QoS Policy DURABILITY USER DATA HISTORY (per subject) TOPIC DATA READER DATA LIFECYCLE GROUP DATA WRITER DATA LIFECYCLE PARTITION LIFESPAN PRESENTATION ENTITY FACTORY DESTINATION ORDER RESOURCE LIMITS OWNERSHIP RELIABILITY OWNERSHIP STRENGTH TIME BASED FILTER LIVELINESS DEADLINE LATENCY BUDGET CONTENT FILTERS TRANSPORT PRIORITY

Writers and readers state Start demo Their needs z Topics – Square, Circle, Triangle – Attributes z Data types (schemas) – Shape (color, x, y, size) z Color is instance Key – Attributes z Shape & color used for key z QoS – Deadline, Liveliness – Reliability, Durability – History, Partition – Ownership RTI DDS delivers © 2008 RTI - All rights Reserved #2 Performance & Scalability

z DDS was designed to support high performance

z RTI DDS was developed to maximize performance and minimize jitter

z Advanced techniques employed: – Pre-allocation of memory z Never allocate/free memory in the critical path – Use dedicated threads per receive port z Minimize thread switching z Avoid expensing operating system calls (e.g. select()) – Maximize concurrency z Carefully design critical sections z Patented concurrent mutex-free thread-safe data structures – Employ high-performance data-access APIs z Read data by array (no additional copies) z Scatter/gather APIs to access transport. z Buffer loaning for zero copy access

Web Services

Java RTSJ (soft RT) RTSJ (hard RT)

Java/JMSJava/RMI

CORBA RT CORBA

Data Distribution Service / DDS

MPI Messaging Technologies and Standards Messaging Technologies Messaging Technologies and Standards Messaging Technologies Non-real-time Soft real-time Hard real-time Extreme real-time

DDS/JMS/Notification Service Comparison - Latency

2500

2000

DDS JMS Notification Service 1500

1000

500

0 4 8 16 32 64 128 256 512 1024 2048 4096 8192 16384 Message Size (bytes)

Message Length (samples)

DDS/CORBA Notification Service Comparison - Jitter DDS/JMS/CORBA Notification Service Comparison - Jitter 100

2000 80 1800

1600 DDS JMS Notification service 60 1400 DDS JMS Notification service 1200 40 1000

800

Standard Deviation (usecs) Deviation Standard 20 600

Standard Deviation (usecs) Deviation Standard 400 0 200 4 8 16 32 64 128 256 512 1024 2048 4096 8192 16384 0 Message LengthMessage (samples) Size (bytes) 4 8 16 32 64 128 256 512 1024 2048 4096 8192 16384 Message LengthMessage (samples) Size (bytes)

– Redundancy & Failover – Persistent Data – Last value cache – Historical cache – Recording service

Producer / Writer I1 P strength=10 rimary Topic T1 Producer / Writer I1 Backup strength=5 I1 I2 I2 Primary Producer / Writer I2 Backup strength=1

z Owner determined per subject z Only extant writer with highest strength can publish a subject (or topic for non-keyed topics) z Automatic failover when highest strength writer: – Loses liveliness – Misses a deadline – Stops writing the subject

z Shared Ownership allows any writer to update the subject

A standalone service that persists data outside of the context of a DataWriter

Data Data Writer Data Writer Reader

Data Global Reader Data Space

Persistence Persistence Service Service

Permanent Permanent Storage Storage

© 2008 RTI - All rights Reserved Last value cache z A last-value cache is already built-in into every Writer in the system – Can used in combination with a Durable Writer z A late joiner will automatically initialize to the last value z Last value cache can be configure with history depth greater than 1 z The Persistence Service can be used to provide a last value cache for durable data

© 2008 RTI - All rights Reserved Historical cache z A partial historical cache is already built-in into every Writer in the system – Can used in combination with a Durable Writer z The Persistence Service can be used to provide a historical cache with larger/unlimited depth z A late joiner will automatically initialize to the desired history – Currently amount of history can only be specified as message count. – Next release will also allow a age/time based specification. z Request for historical cache is done by creating a Reader with the desired history depth specified as a QoS

z Protocol – Interoperability Wire protocol adopted in 2006 z Languages: C/C++, Java, ADA, .NET z Systems/Platforms/Models – Data distribution (publishers and subscribers): DDS – Data management (storage, retrieval, queries): SQL – ESB Integration, Business process integration: WSDL

Distributed Distributed SQL SQL Node Node DBMS Distributed DDS D T SQL Node Global Data Space

Distributed WSDL DBMS DBMS Distributed Node DDS Node

Real-Time Devices Fault Auditing & Tolerance Recording

WS-DDS

Real-Time Pub-Sub/Caching/Messaging SOA & Real-Time Web Services

Database

Event Processing

Tools & Visualization

Messaging Actions Relational Actions Write() UPDATE & INSERT Read() & Take() SELECT Dispose() DELETE Wait() & Listener

Topic T1 Table T1 I1 I1 I2 I3 I2 I3

Event driven – The fastest way to observe database changes! © 2008 RTI - All rights Reserved Complex Event Engine Integration z CEP: programmable engines used to transform “data” into “information” z CEP engines are programmed using a derivative of SQL z CEP engines save time: They can implement a lot of the application logic: – Classification, Correlation, Aggregation, Filter, Cleansing, Pattern Detection, etc. z DDS is the perfect ‘data’ and ‘information’ pipe for CEP engines – Use high-speed data streams (1,000-1,000,000 msg/sec) – Require latency measured in sub-milliseconds – Demand access to events from a heterogeneous systems

Dashboards

DDS Applications

Alerts

Web Hi-Performance Enabled Embedded Protocols Real-Time

HTTP

Generic Web Client (.NET / Java) SOAP DDS-RTPS

WS-DDS Global DDS Web Service Data Space RTI, DDS, Real-Time DDSParticipant

IONA IONA Artix Transports Web Client SOAP, JMS, IIOP DDSParticipant

www.rti.com Agenda

z Part I -- Concepts – Middleware Models: Messaging & Data-Distribution – Publish/Subscribe & Data-Centric Design z Part II -- Patters & Use-Cases – Sending Data – State Information – Alarms and Events – Discovery – Sample Application Requirements mapped to QoS

© 2008 RTI - All rights Reserved Do-yourself Message-Centric System z Model – 1-1, FIFO z Applications coupled in Lifespan & Content – Both must be present simultaneously – Everything sent is received z Excellent performance z Doesn’t scale – To large-scale systems – To loosely-coupled systems

Application Application

Queue co-located with each application

© 2008 RTI - All rights Reserved Middleware-based Message-Centric Systems (JMS) z Model – Broker-based, nÎn communication – Independent messages, No state z Coupling – Not coupled in Lifespan – Coupled in Order and Content (presentation) z Worse performance z Better scalability

Application Application

Application Application Global Queues Queue per message kind

© 2008 RTI - All rights Reserved Do-yourself Data-Centric System z Model – Shared/replicated structured data/state – Asynchronous, Selective sharing z Coupling: – Coupled in Lifespan, Decoupled in Presentation & Content z Excellent performance & scalability

Application Application Replication Mechanism

© 2008 RTI - All rights Reserved Middleware-based Data-Centric Systems (DDS) z Shared data-space, shared state z Asynchronous communication z Decoupled in Lifespan, Content, presentation z Excellent performance & scalability z Subsumes Message-Centric via QoS

Application Application

Provides a virtual “Global Data Space” that is accessible to all interested applications. – Data objects addressed by DomainId, Topic and Key – Subscriptions are decoupled from Publications – Contracts established by means of QoS – Automatic discovery and configuration

Data Reader Data Writer TopicKey

Data Object Data Reader Data Writer

Data Reader Data Writer

© 2008 RTI - All rights Reserved DDS Global Data z Address in Global Data Space = (DomainId, Topic, Key) – Each topic corresponds to a multiple data instances with a common schema – A DataWriter can write to any instances of a single topic – Multiple DataWriters may write to the same instance – A DataReader receives updates from all instances of a single topic – Multiple DataReaders may read from the same instances & values

Data Reader Data Writer

© 2008 RTI - All rights Reserved DDS Global Data: Domains z Address in Global Data Space = (DomainId, Topic, Key) – Each Domain is identified by the value of the domainId – Each Domain is a separate Global Data Space z The same Topic name can mean different things in different domains z The same Topic can have different Types on each Domain – An application may join multiple Domains – Domains can be used for isolation, scalability, modulariy

// Entities creation DomainParticipant participant = TheParticipantFactory->create_participant( domain_id, participant_qos, participantA_listener);

Publisher publisher = domain->create_publisher( publisher_qos, publisher_listener);

Topic topic = domain->create_topic( “MyTopic”, “Text”, topic_qos, topic_listener);

DataWriter writer = publisher->create_datawriter( topic, writer_qos, writer_listener);

TextDataWriter twriter = TextDataWriter::narrow(writer);

TextStruct my_text; twriter->write(&my_track);

// Entities creation Subscriber subscriber = domain->create_subscriber( subscriber_qos, subscriber_listener);

Topic topic = domain->create_topic( “Track”, “TrackStruct”, topic_qos, topic_listener);

DataReader reader = subscriber->create_datareader( topic, reader_qos, reader_listener);

// Use listener-based or wait-based access

// Listener creation and attachment Listener listener = new MyListener(); reader->set_listener(listener);

// Listener code MyListener::on_data_available( DataReader reader ) { TextSeq received_data; SampleInfoSeq sample_info; TextDataReader reader = TextDataReader::narrow(reader);

treader->take( &received_data, &sample_info, …) // Use received_data printf(“Got: %s\n”, received_data[0]->contents); }

// Creation of condition and attachement Condition foo_condition = treader->create_readcondition(…); waitset->add_condition(foo_condition); // Wait ConditionSeq active_conditions; waitset->wait(&active_conditions, timeout); // Wait returns when there is data (or timeout) FooSeq received_data; SampleInfoSeq sample_info; treader->take_w_condition (&received_data, &sample_info, foo_condition); // Use received_data printf(“Got: %s\n”,© 2008received_data[0]->contents); RTI - All rights Reserved Listeners, Conditions & WaitSets

Middleware must notify user application of relevant events: – Arrival of data – But also: z QoS violations z Discovery of relevant entities – These events may be detected asynchronously by the middleware … Same issue arises with POSIX signals

DDS allows the application to choice: – Either to get notified asynchronously using a Listener – Or to wait synchronously using a WaitSet

Both approaches are unified using STATUS changes

DDS defines z A set of enumerated STATUS z The statuses relevant to each kind of DDS Entity DDS entities maintain a value for each STATUS

STATUS Entity INCONSISTENT_TOPIC Topic DATA_ON_READERS Subscriber LIVELINESS_CHANGED DataReader REQUESTED_DEADLINE_MISSED DataReader struct LivelinessChangedStatus { RUQESTED_INCOMPATIBLE_QOS DataReader long active_count; DATA_AVAILABLE DataReader long inactive_count; SAMPLE_LOST DataReader long active_count_change; long inactive_count_change; SUBSCRIPTION_MATCH DataReader } LIVELINESS_LOST DataWriter OFFERED_INCOMPATIBLE_QOS DataWriter

PUBLICATION_MATCH © 2008 RTI - AllDataWriter rights Reserved Listeners, Conditions and Statuses z A DDS Entity is associated with: – A listener of the proper kind (if attached) – A StatusCondition (if activated) z The Listener for an Entity has a separate operation for each of the relevant statuses STATUS Entity Listener operation INCONSISTENT_TOPIC Topic on_inconsistent_topic DATA_ON_READERS Subscriber on_data_on_readers LIVELINESS_CHANGED DataReader on_liveliness_changed REQUESTED_DEADLINE_MISSED DataReader on_requested_deadline_missed RUQESTED_INCOMPATIBLE_QOS DataReader on_requested_incompatible_qos DATA_AVAILABLE DataReader on_data_available SAMPLE_LOST DataReader on_sample_lost SUBSCRIPTION_MATCH DataReader on_subscription_match LIVELINESS_LOST DataWriter on_liveliness_lost OFFERED_INCOMPATIBLE_QOS DataWriter on_offered_incompatible_qos PUBLICATION_MATCH DataWriter on_publication_match

© 2008 RTI - All rights Reserved Listeners & Condition duality z A StatusCondition can be selectively activated to respond to any subset of the statuses z An application can wait changes in sets of StatusConditions using a WaitSet z Each time the value of a STATUS changes DDS – Calls the corresponding Listener operation – Wakes up any threads waiting on a related status change

Asynchronous notification via Listener operation DDS Status Change Entity Synchronous notification via activation/wakeup of conditions/waitsets

Type Four easy steps: Definition 1. Define your data 2. Create your project MyType 3. Build 4. Run: publisher subscriber MyType.sln rtiddsgen MyType.h Aux: MyTypeSupport.c MyTypePublisher.cpp File Browser MyTypeSubscriber.cpp Console compiler

Publisher.exe Subscriber.exe

Application Application

GDS Definition

Discovery

Application Application

Listener Listener Protocol

Cache Management Cache

Application Application

Domain Modeling & Sys. Design

GDS Definition

Discovery

Config & Application Application Qos

Listener Listener Protocol

Cache Management Cache

© 2008 RTI - All rights Reserved Management Designing a Data-Centric System z Define/Model the Global Data Space z Configure the Cache Management z Configure Discovery z Configure the Protocol z Configure/Use hooks for – Fault detection – Controlled access

• Container for N1 App 1 N4 App 4 Pub/Sub Pub/Sub applications that (A,B/C,D) (D/C,E,F) want to communicate

• Applications can N2 App 2 N4 App 5 join or leave a Subscribe Domain Publish domain in any order (C) (C) • New Applications are “Auto-Discovered”

N3 App 3 N5 App 6 • An application that Pub/Sub Subscribe (E,F/A,C) (B,C) has joined a domain is also called a “Domain Participant”

Single ‘Domain’ System

Using Multiple domains for Scalability, Modularity & Isolation

Node 1 - App 1 Node 4 - App 1 Pub/Sub Domain A Pub/Sub

Node 2 - App 1 Node 4 - App 2 Subscribe Publish

Domain B

Node 3 - App 1 Node 5 - App 1 Pub/Sub Subscribe

Domain C Node 5 - App 2 Added Func. Node 6 - App 1 Pub/Sub Pub/Sub

Topic “MarketData” Data-type (name-type-value pairs) source type symbol Exchange volume bid ask OPRA IBM NYSE 200000 118.30 118.36 OPRA AAPL NASDAQ 171.20 171.28 RTFP EQ

Key fields Æ Subject Additional fields (payload)

Topic “OrderEntry”

Exchange type Symbol Order num number limit stop expiration NYSE BUY IBM 11956 500 120 - DAY NYSE BUY IBM 11957 1000 124.5 124 DAY NASDAQ SELL AAPL 11958 400 - 160 DAY

Specifies whether more than one DataWriter can update the same instance of a data-object

Ownership = EXCLUSIVE Ownership = SHARED “Only highest-strength “All data-writers can data writer can update each update data- each data-instance” instance”

Data Data Data Data Data Data Writer Writer Writer Writer Writer Writer

Specifies which DataWriter is allowed to update the values of data-objects OWNERSHIP_STRENGTH “Integer to specify the ORDER strength of an instance”

Strength = 1 Strength = 4 Domain Participant Data After DataQoS Expires Data “RIGHT” Writer Writer Reader “LEFT” - Deadline“RIGHT” “LEFT” - Liveliness Publisher Publisher Subscriber S S

Note: Only applies to Topics with Ownership = Exclusive © 2008 RTI - All rights Reserved Configure the Cache Management z Cache State Content – History Writer – Lifespan Application – Persistence – Resources z Reader Cache View – Partitions – Content-Based Filter – Time-Based Filter Reader – Order Application

KEEP_ALL: KEEP_LAST: “depth” integer for Publisher: keep all until delivered the number of samples to keep at Subscriber: keep each sample until the any one time application processes that instance

S7 S7 S7 Data S6 Data Data Writer Writer S6 Reader S6 S5 S5 S4 S4 S3 Publisher S2 Publisher Subscriber S1 Keep All Keep Last 2 Keep Last 4

User can set lifespan duration Manages samples in the history queues, attached to each Sample Topic

S1 Data Data Writer Reader S2 S3 S4 S5 Perm. Publisher Subscriber S6 Storage S7 S1 S2

S4 S3 S2 S1

Instance 1 Value = 249 Content Filtered Topic Instance 2 Value = 230

Instance 3 Value = 275 Topic

Instance 4 Value = 262

“Filter Expression ” Value = 258 Instance 5 Ex. Value > 260

Instance 6 Value = 261

Instance 7 Value = 259

The Filter Expression and Expression Params will determine which instances of the Topic will be received by the subscriber.

Topic: “Market Data” Payload Field Source Symbol Type Exchange Volume Bid Ask …

Value * * * * *

Topic: “Order Entry” Payload Field Symbol Type Exchange OrderNumber Symbol OrderKind Stop Limit …

Value * * NYSE *

Subject Filter (for a Reader)

Topic: “Market Data”

Field Source Symbol Type Exchange Payload

Value REUTERS * EQ NYSE Volume > x, Ask < y

Topic “minimum_separation”: Data Reader does not want to receive data Domain faster than the min_separation time Participant

Data Data Writer Reader

Publisher Discarded Subscriber samples

S S S S S S S

minimum separation Data Samples

© 2008 RTI - All rights Reserved Configure the Protocol z Discovery z Reliability z Liveliness Writer Application z Flow Control z Asynchronous write z Network Configuration – Enabled Transports + transport properties – Multicast addresses – Transport Priority z OS settings – Threads – Memory Reader Application

© 2008 RTI - All rights Reserved Tunable Reliability Protocol z Configurable AckNack reply z Performance can be tracked times to eliminate storms by senders and recipients – Configurable high/low z Fully configurable to bound watermark, Buffer full latency and overhead z Flexible handling of slow – Heartbeats, delays, buffer recipients sizes – Dynamically remove slow receivers

S7 S1 Data Data S6 Reliable S2 Writer Reader S5 •Guaranteed S3 S4 Ordered Delivery S4 S3 •“Best effort” also S5 Publisher Subscriber S6 S2 supported S1 S7

Writer Application z Listeners z Deadline Qos z Liveliness Qos z Built-in Readers z Notification of matching

Reader Application

Topic DEADLINE “deadline period”

Commits Failed to to provide get data Data data each Data Writer deadline Listener Reader period.

Publisher Expects data every Subscriber deadline period.

deadline S X S S S S S

Type: Controls who is responsible for issues of ‘liveliness packets’ AUTOMATIC = Infrastructure Managed MANUAL = Application Managed

Topic Topic

Failed to renew lease Data Data Writer Listener Reader

Publisher Subscriber

lease_duration

Scenario: Fire-control radar system, gets updates from multiple sensors up to 500x/sec

z Quality of Service Settings z Requirements – Radars – Radars z Key for each track z Multiple radars, may track same z Ownership= Exclusive; Assign objects, but some higher strength by accuracy performance/accuracy than – (each object tracked by best radar others that sees it) – Fire control z Publishers offer reliability z Needs every possible update – Fire Control quickly; last is best z Maximize determinism z Needs to know when a track is lost z Best efforts; get every sample z Use key state/deadline to know – Display console when objects lost z Can only display 10Hz updates – Display console – Logger z Don’t waste bandwidth z Must record all information in a z Best efforts database for later analysis z Time-based filter at 0.1sec – Logger z Reliable subscription

© 2008 RTI - All rights Reserved Application Example: Financial Trading z Requirements z Quality of Service Settings – Feed handlers – Feed handlers z Take direct market data z Offer multicast, reliable z Publish 750k msgs/sec – Trading displays – Trading displays z Subscribe at low bandwidth with time-based filters z Watch only “interesting” Automated trading symbols – z Partition market data to divide z Can handle only 2Hz load – Automated trading z Watch for transient price Exchange, ECNs, OPRA Feeds

differences in specific Orders symbols Feed Handlers z 300 machines watching various symbols/aspects of Normalization market Messaging Middleware – Audit database Order Book z Save everything Audit DB Mgmt

Trading Desks Automated Trading

© 2008 RTI - All rights Reserved Exchange, ECN Order Execution Summary z Designing a fault-tolerant distributed system is not a simple task z A powerful middleware framework can provide a lot of value and help you focus on the business logic z The middleware can save you a lot of time and effort. – It is worth learning how to use its power!

Gerardo Pardo-Castellote, Ph.D. [email protected] www.rti.com