Introduction to DDS OMG Real-Time Workshop Washington DC July 2008 Gerardo Pardo-Castellote, Ph.D. Co-chair OMG DDS SIG CTO, Real-Time Innovations [email protected] www.rti.com Agenda z Background: Middleware Technologies z Introduction: DDS Model & Applicability z Details: DDS in depth © 2008 RTI - All rights Reserved Introduction: Middleware Technologies z The concept of network middleware z Communications Model z Object Model z Architecture Model z Protocol © 2008 RTI - All rights Reserved With increased complexity… End User Application © 2008 RTI - All rights Reserved With increased complexity… End User Application Network Stack Routing protocols File System Kernel Device Drivers Operating System © 2008 RTI - All rights Reserved With increased complexity… End User Application HTTP Service Mail Service FTP Service Data base Network Stack Routing protocols File System Kernel Device Drivers Operating System © 2008 RTI - All rights Reserved … middleware becomes necessary End User Application Application Server Event Processing Engine Physics Engine Message Service Data base Middleware Graphics Engine HTTP Service Mail Service FTP Service Network Stack Routing protocols File System Kernel Device Drivers Operating System & Pre-Packaged Services © 2008 RTI - All rights Reserved What is network middleware? Middleware = API and service layer above operating system and below “application” code that abstracts common interaction patterns Network Middleware = Most popular class of middleware Middleware used for developing distributed applications Distributed Applications = Those requiring interaction/communication between multiple computers © 2008 RTI - All rights Reserved Network Middleware Examples z DCE/RPC, DCOM, CORBA, ICE z TIBCO, 29 West z JMS, MQ Series, ActiveMQ, SoniqMQ z DDS, RTI DDS, OpenSplice, tao-dds, GigaSpaces z New trend: – Application Servers (WebSphere, WebLogic, JBOSS) – Include network middleware as a component z NOTE: Middleware “packages” are building blocks, not stand-alone applications like… – Skype, gtalk, … – BitTorrent, eMule, … z Why aren’t ‘popular’ consumer applications built on top of middleware? © 2008 RTI - All rights Reserved Historical note: From Telephone to Blogs z Why so many flavors? z Parallels evolution of general communication patterns: – Started with point-to-point connections – Then request-reply services – Then Message Queue Services – Then Publish-Subscribe Services – Then Data-Caching services z Other examples: – FTP, email -> WEB -> Blogs, RSS -> Podcasts © 2008 RTI - All rights Reserved Middleware = Service Model + Protocol z Service model composed of: Communications – Communications model Model – Object Model – Architecture model Object Model z All these interact Architecture Model z The service model and PROTOCOL protocol are also coupled Middleware Protocols cannot be compared in isolation! They must be compared in the context of the service model © 2008 RTI - All rights Reserved Middleware Service Model Service Model = Communications Model + Object Model + Architecture Model z Communications Model: – Abstract model of how applications interact: z Remote Method Invocation z Queue Based, Pub-Sub z Data-Distribution z Replicated Data z Distributed Transactions z Object Model – Middleware entities the application uses to interact with the service: z Queues, Publishers, Domains, Caches, Federations, Remote Objects… z Architecture Model – Centralized, Brokered, Peer-to-Peer © 2008 RTI - All rights Reserved Agenda z The concept of network middleware z Communications Model z Object Model z Architecture Model z Protocol © 2008 RTI - All rights Reserved Communications Model z Abstract model of how applications interact – Remote Method Invocation – Message-Oriented, Queue Based – Pub-Sub Data-Distribution – Replicated Data – Distributed Transactions © 2008 RTI - All rights Reserved Middleware Communication Models Point-to-Point Client-Server Telephone, TCP File systems, Database, RPC, CORBA, DCOM Simple, high-bandwidth Good if information is naturally centralized Leads to stove-pipe systems Single point failure, performance bottlenecks Publish/Subscribe Messaging Replicated Data Magazines, Newspaper, TV Libraries, Distributed databases Excels at many-to-many Excels at data-mining and analysis communication Excels at distributing time-critical information © 2008 RTI - All rights Reserved RMI vs Pub-Sub/Messaging/Data-Distribution z RMI (WebServices, CORBA, DCOM) offer a remote method abstraction – Familiar OO programming model – Results in a tightly-coupled system y Forces synchronous invocations y Imposes global object model y Limited QoS (appearance of local method call) y Lack robustness: cascading points of failure – Typically built on top of TCP: y impacts scalability and time-determinism – Best-suited to smaller, closely-coupled systems Topic/Queue z Pub-Sub (Messaging Data-Distribution) offer a queue- based and/or replicated-data model – Subsystems are decoupled in time, space, and synchronization y Contracts established by verifying QoS compatibility – Supports a variety of transports including multicast UDP – Better suited for high-performance and real-time © 2008 RTI - All rights Reserved Queue versus Pub-Sub z Queue – Message sent to Queue 1 2 3 1 – Multiple readers can read from the queue 2 Queue – Each message is delivered to ONLY one reader z Readers “affect each other” 3 – Apps: z Job Scheduling z Load Balancing z Collaboration z Pub Sub – Message Sent to Topic 1 2 3 – Multiple readers can subscribe to Topic 1 2 3 with or without filters 1 2 3 – Each message delivered to ALL Topic/Queue subscribers that pass filter z Readers are decoupled 1 1 3 – Apps: z Notifications z Information Distribution © 2008 RTI - All rights Reserved Pub-Sub versus Data-Distribution z Pub-Sub – Only messages no concept of data 1 2 3 – Each message is interpreted without 1 2 3 context 1 2 3 – Messages must be delivered FIFO or Topic/Queue according to some “priority” attribute No Caching of data – 1 3 – Simple QoS: filters, durability, lifespan 1 z Data-Distribution – Messages represent update to data-objects – Data-Objects identify by a key – Middleware maintains state of each object 1 2 2 3 – Objects are cached. Applications can read at leisure – Smart QoS z Ownership z History (per key) z Deadline 1 2 3 1 2 3 1 2 3 – Subsumes messaging © 2008 RTI - All rights Reserved Other (non DDS) Commercial Pub-Sub Models z Older, but widely deployed – TIBCO (RendezVous, EMS) – IBM MQSeries z Limited deployment: – CORBA Event Service – CORBA Notification Service z Emerging standards – not really used yet – WS-Eventing – WS-Notification z Emerging – 29West – IBM LLM © 2008 RTI - All rights Reserved JMS/MessageQueue Service Model: Communication Model Provides a “Shared Queue Space” that is accessible to all interested applications. – Message are sent to an Exchange – Each message has an associated Routing Key – Brokers forward messages to one or more Queues based on the Routing Key – Subscriber get messages from named Queues – Only one subscriber can get a given message from each Queue Client Client Pub Sub Queue1 Client E Queue2 Sub Client Pub E Queue3 Client Shared Queue Space Sub © 2008 RTI - All rights Reserved DDS Service Model: Communication Model Provides a “Global Data Space” that is accessible to all interested applications. – Data objects addressed by Domain, Topic and Key – Subscriptions are decoupled from Publications – Contracts established by means of QoS – Automatic discovery and configuration Participant Participant Pub Pub Track,2 Participant Sub Sub Track,1 Track,3 Global Data Space Participant Participant Pub Alarm Sub © 2008 RTI - All rights Reserved MessageQueue Service Model : Object Model Routing key Queue AMPQ? AMPQ? Queue Publisher Exchange Binding Client Subscriber Client Queue Content Broker Server z Exchange – Receives messages and routes to a set of message queues z Queue – Stores messages until they can be processed by the application(s) z Binding – Routes messages between Exchange and Queue. Configured externally to the application – Default binding maps routing-key to Queue name z Routing Key – label used by the exchange to route Content to the queues z Content – Encapsulates application data and provides the methods to send receive, acknowledge, etc. © 2008 RTI - All rights Reserved DDS Service Model: Object Model Topic Topic Data Offered Data Requested Writer QoS Reader QoS Publisher Domain Subscriber Domain Participant Participant Global Data Space z DomainParticipant – Allows application to join a DDS Domain (Global Data Space) z Topic – A string that addresses a group of objects in the Global Data Space – Each Object is identified by a Key (some fields within the object data) z Publisher, Subscriber – Pools resources for DataWriters and DataReaders z DataWriter – Declares intent to publish a Topic and provides type-safe operations to write/send data z DataReader – Declares intent to subscribe to a Topic and provides type-safe operations to read/receive data© 2008 RTI - All rights Reserved Agenda z The concept of network middleware z Communications Model z Object Model z Architecture Model z Protocol © 2008 RTI - All rights Reserved Architecture Models z Brokered – Centralized – Segmented – Federated z Peer to Peer © 2008 RTI - All rights Reserved Centralized Broker Pub-Sub Service One central server materializes all middleware entities All traffic flows via server E.g. “naïve” implementations of JMS, CORBA Notification, etc. Client Application Server Pub-Sub Service