Hermes: a Scalable Event-Based Middleware

UCAM-CL-TR-590 Technical Report ISSN 1476-2986 Number 590 Computer Laboratory Hermes: A scalable event-based middleware Peter R. Pietzuch June 2004 15 JJ Thomson Avenue Cambridge CB3 0FD United Kingdom phone +44 1223 763500 http://www.cl.cam.ac.uk/ c 2004 Peter R. Pietzuch This technical report is based on a dissertation submitted February 2004 by the author for the degree of Doctor of Philosophy to the University of Cambridge, Queens’ College. Technical reports published by the University of Cambridge Computer Laboratory are freely available via the Internet: http://www.cl.cam.ac.uk/TechReports/ ISSN 1476-2986 Abstract Large-scale distributed systems require new middleware paradigms that do not suffer from the limitations of traditional request/reply middleware. These limitations include tight coupling between components, a lack of information filtering capabilities, and support for one-to-one communication semantics only. We argue that event-based middleware is a scalable and power- ful new type of middleware for building large-scale distributed systems. However, it is important that an event-based middleware platform includes all the standard functionality that an application programmer expects from middleware. In this thesis we describe the design and implementation of Hermes, a distributed, event- based middleware platform. The power and flexibility of Hermes is illustrated throughout for two application domains: Internet-wide news distribution and a sensor-rich, active building. Hermes follows a type- and attribute-based publish/subscribe model that places particular emphasis on programming language integration by supporting type-checking of event data and event type inheritance. To handle dynamic, large-scale environments, Hermes uses peer-to-peer techniques for autonomic management of its overlay network of event brokers and for scalable event dissemination. Its routing algorithms, implemented on top of a distributed hash table, use rendezvous nodes to reduce routing state in the system, and include fault-tolerance features for repairing event dissemination trees. All this is achieved without compromising scalability and efficiency, as is shown by a simulational evaluation of Hermes routing. The core functionality of an event-based middleware is extended with three higher-level middleware services that address different requirements in a distributed computing environment. We introduce a novel congestion control service that avoids congestion in the overlay broker network during normal operation and recovery after failure, and therefore enables a resource-efficient de- ployment of the middleware. The expressiveness of subscriptions in the event-based middleware is enhanced with a composite event service that performs the distributed detection of complex event patterns, thus taking the burden away from clients. Finally, a security service adds access control to Hermes according to a secure publish/subscribe model. This model supports fine-grained access control decisions so that separate trust domains can share the same overlay broker network. 3 4 To my parents 6 Declaration This dissertation is not substantially the same as any that I have submitted or am currently submitting for a degree, diploma or any other qualification at any other university. No part of this dissertation has already been or is being concurrently submitted for any such degree, diploma or any other qualification. The work for Chapter 6 was done with Sumeer Bhola while at the IBM TJ Watson Re- search Center [PB03b]. Material from Chapter 7 is based on a paper co-authored with Brian Shand [PSB03]. Chapter 8 is based on work with András Belokosztolszki, David Eyers, and Brian Shand [BEP+03]. This dissertation is the result of my own work and is not the outcome of work done in collabo- ration, except as specified in the text. This dissertation does not exceed 60, 000 words, including tables and footnotes, but excluding bibliography and diagrams. 7 8 Acknowledgements First of all, I would like to thank Jean Bacon, my supervisor, who has provided tremendous advice and support during the time of my PhD. Her insight helped my work go in the right direction, and she taught me the principles of research work. I am extremely grateful for this. I also appreciate the assistance and feedback given by Ken Moody. I have had the pleasure of being a member of the Opera Group. In particular, I enjoyed collab- orating with András Belokosztolszki, David Eyers, and Brian Shand on various topics. Many ideas stem from (often hour-long) tea-breaks with the regular tea crowd that included Alan, Alexis, András, Aras, Brian, Chaoying, Chris, Dan, David, Eiko, Lauri, Nathan, Niki, Salman, and Walt. I would also like to thank Alan Abrahams, David Eyers, and András Belokosztolszki for proof-reading my thesis and suggesting improvements. I am grateful to Sumeer Bhola, who was my mentor while I was doing an internship at the IBM TJ Watson Research Center. His guidance led to some of the work described in this thesis. My research was funded by the Engineering and Physical Sciences Research Council (EPSRC), the Cambridge European Trust, and QinetiQ, Malvern. Finally, I would like to thank my parents for their support and encouragement throughout the years, which made it all possible. 9 10 Publications [BEP+03] András Belokosztolszki, David M. Eyers, Peter R. Pietzuch, Jean Bacon, and Ken Moody. Role-Based Access Control for Publish/Subscribe Middleware Architectures. In H. Arno Jacobsen, editor, Proceedings of the 2nd International Workshop on Distributed Event-Based Systems (DEBS’03), ACM SIGMOD, San Diego, CA, USA, June 2003. ACM. [CBP+02] Jon Crowcroft, Jean Bacon, Peter Pietzuch, George Coulouris, and Hani Naguib. Channel Islands in a Reflective Ocean: Large-scale Event Distribution in Heterogeneous Networks. IEEE Communications Magazine, 40(9):112–115, September 2002. [KDHP03] Evangelos Kotsovinos, Boris Dragovic, Steven Hand, and Peter R. Pietzuch. Xeno- Trust: Event-Based Distributed Trust Management. In Proceedings of Trust and Privacy in Digital Business (TrustBus’03). In conjunction with the 14th International Confer- ence on Database and Expert Systems Applications (DEXA’03), Prague, Czech Republic, September 2003. [PB02] Peter R. Pietzuch and Jean M. Bacon. Hermes: A Distributed Event-Based Middle- ware Architecture. In Jean Bacon, Ludger Fiege, Rachid Guerraoui, H. Arno Jacobsen, and Gero Mühl, editors, Proceedings of the 1st International Workshop on Distributed Event-Based Systems (DEBS’02). In conjunction with the 22nd International Conference on Distributed Computing Systems (ICDCS’02), pages 611–618, Vienna, Austria, July 2002. IEEE. [PSB03] Peter R. Pietzuch, Brian Shand, and Jean Bacon. A Framework for Event Compo- sition in Distributed Systems. In Markus Endler and Douglas Schmidt, editors, Pro- ceedings of the 4th International Conference on Middleware (Middleware’03), volume 2672 of LNCS, pages 62–82, Rio de Janeiro, Brazil, June 2003. Best Paper Award. ACM/IFIP/USENIX, Springer Verlag. [PB03a] Peter R. Pietzuch and Jean Bacon. Peer-to-Peer Overlay Broker Networks in an Event-Based Middleware. In H. Arno Jacobsen, editor, Proceedings of the 2nd Interna- tional Workshop on Distributed Event-Based Systems (DEBS’03), ACM SIGMOD, San Diego, CA, USA, June 2003. ACM. [PB03b] Peter R. Pietzuch and Sumeer Bhola. Congestion Control in a Reliable Scalable Message-Oriented Middleware. In Markus Endler and Douglas Schmidt, editors, Proceed- ings of the 4th International Conference on Middleware (Middleware’03), volume 2672 of LNCS, pages 202–221, Rio de Janeiro, Brazil, June 2003. ACM/IFIP/USENIX, Springer Verlag. [PSB04] Peter R. Pietzuch, Brian Shand, and Jean Bacon. Composite Event Detection as a Generic Middleware Extension. IEEE Network Magazine, Special Issue on Middleware Technologies for Future Communication Networks, 18(1):44–55, January/February 2004. 11 12 Contents 1 Introduction 21 1.1 Large-ScaleDistributedSystems . ....... 21 1.2 ApplicationScenarios . ..... 22 1.2.1 Internet-WideDistributedSystems . ...... 22 1.2.2 Large-ScaleUbiquitousSystems . 22 1.3 WhyEvent-BasedMiddleware? . 23 1.4 ResearchStatement ............................... 24 1.5 DissertationOutline ............................. ..... 25 2 Background 27 2.1 Middleware ...................................... 27 2.1.1 Synchronous Request/Reply Middleware . ....... 28 2.1.2 Asynchronous Message-Oriented Middleware . ....... 32 2.1.3 OtherMiddleware ............................... 34 2.2 Publish/SubscribeSystems . ...... 36 2.2.1 Topic-BasedPublish/Subscribe . ..... 37 2.2.2 Content-Based Publish/Subscribe . ...... 37 2.3 Peer-to-PeerSystems............................... 46 2.3.1 DistributedHashTables . 47 2.3.2 Application-LevelMulticast . ..... 49 2.4 Summary ........................................ 51 3 Event-Based Middleware 53 3.1 Overview ........................................ 53 3.2 Requirements .................................... 54 3.2.1 ScalabilityandExpressiveness . ...... 55 3.2.2 Reliability................................... 55 3.2.3 Administrability .............................. 56 3.2.4 Usability .................................... 56 3.2.5 Extensibility ................................. 57 3.2.6 Interoperability. 58 3.3 Design.......................................... 58 3.3.1 EventModel .................................. 58 3.3.2 ComponentModel ............................... 62 3.3.3 RoutingModel ................................. 64 3.3.4 ReliabilityModel.............................

Load more