Monitor Object 1 Intent 2 Also Known As 3 Example

Total Page:16

File Type:pdf, Size:1020Kb

Monitor Object 1 Intent 2 Also Known As 3 Example Monitor Object An Object Behavioral Pattern for Concurrent Programming Douglas C. Schmidt [email protected] Department of Computer Science Washington University, St. Louis 1Intent the corresponding consumer handler, whose thread then de- livers the message to its remote consumer. The Monitor Object pattern synchronizes method execution When suppliers and consumers reside on separate hosts, to ensure only one method runs within an object at a time. the Gateway uses the connection-oriented TCP [1] proto- It also allows an object’s methods to cooperatively schedule col to provide reliable message delivery and end-to-end flow their execution sequences. control. TCP’s flow control algorithm blocks fast senders when they produce messages more rapidly than slower re- ceivers can process the messages. The entire Gateway should 2 Also Known As not block, however, while waiting for flow control to abate on outgoing TCP connections. To minimize blocking, there- Thread-safe Passive Object fore, each consumer handler can contain a thread-safe mes- sage queue that buffers new routing messages it receives from its supplier handler threads. 3 Example One way to implement a thread-safe Message Queue is to use the Active Object pattern [2], which decouples the Consider the design of the communication Gateway shown thread used to invoke a method from the thread used to exe- in Figure 1. The Gateway process contains multiple supplier cute the method. As shown in Figure 2, each message queue active object contains a bounded buffer and its own thread of control that maintains a queue of pending messages. Using Supplier Routing Consumer Handler Table Handler Consumer 2: find (msg) 3: put (msg) 1: put (msg) Handler Supplier 2: put (msg) Consumer Handler Message Queue Handler 4: get (msg) 5: send (msg) Supplier 1: recv (msg) Handler 3: get (msg) INCOMING OUTGOING 4: send (msg) INCOMING GATEWAY MESSAGES OUTGOING MESSAGES MESSAGES MESSAGES GATEWAY OUTGOING MESSAGES Figure 2: Implementing Message Queues as Active Objects SUPPLIER CONSUMER SUPPLIER CONSUMER Figure 1: Communication Gateway the Active Object pattern to implement a thread-safe mes- sage queue decouples supplier handler threads in the Gate- handler and consumer handler objects that run in separate way process from consumer handler threads so all threads threads and route messages from one or more remote sup- can run concurrently and block independently when flow pliers to one or more remote consumers, respectively. When control occurs on various TCP connections. a supplier handler thread receives a message from a remote Although the Active Object pattern can be used to imple- supplier, it uses an address field in the message to determine ment a functional Gateway, it has the following drawbacks: 1 Performance overhead: The Active Object pattern pro- 3. Objects should be able to schedule their methods coop- vides a powerful concurrency model. It not only synchro- eratively: If an object’s methods must block during their nizes concurrent method requests on an object, but also can execution, they should be able to voluntarily relinquish their perform sophisticated scheduling decisions to determine the thread of control so that methods called from other client order in which requests execute. These features incur non- threads can access the object. This property helps prevent trivial amounts of context switching, synchronization, dy- deadlock and makes it possible to leverage the concurrency namic memory management, and data movement overhead, available on hardware/software platforms. however, when scheduling and executing method requests. Programming overhead: The Active Object pattern re- quires programmers to implement up to six components: 6 Solution proxies, method requests,anactivation queue,ascheduler,a For each object accessed concurrently by client threads de- servant,andfutures for each proxy method. Although some fine it as a monitor object. Clients can access the services components, such as activation queues and method requests, defined by a monitor object only through its synchronized can be reused, programmers may have to reimplement or sig- methods. To prevent race conditions involving monitor ob- nificantly customize these components each time they apply ject state, only one synchronized method at a time can run the pattern. within a monitor object. Each monitored object contains a In general, the performance and programming overhead monitor lock that synchronized methods use to serialize their outlined above can be unnecessarily expensive if an applica- access to an object’s behavior and state. In addition, syn- tion does not require all the Active Object pattern features, chronized methods can determine the circumstances under particularly its sophisticated scheduling support. Yet, pro- which they suspend and resume their execution based on one grammers of concurrent applications must ensure that certain or more monitor conditions associated with a monitor object. method requests on objects are synchronized and/or sched- uled appropriately. 7 Structure 4 Context There are four participants in the Monitor Object pattern: Applications where multiple threads of control access ob- Monitor object jects simultaneously. A monitor object exports one or more methods to clients. To protect the internal state of the monitor 5Problem object from uncontrolled changes or race conditions, all clients must access the monitor object only through Many applications contain objects that are accessed concur- these methods. Each method executes in the thread of rently by multiple client threads. For concurrent applications the client that invokes it because a monitor object does to execute correctly, therefore, it is often necessary to syn- not have its own thread of control.1 chronize and schedule access to these objects. In the pres- For instance, the consumer handler’s message queue in ence of this problem, the following three requirements must the Gateway application can be implemented as a mon- be satisfied: itor object. 1. Synchronization boundaries should correspond to object methods: Object-oriented programmers are accus- Synchronized methods tomed to accessing an object only through its interface meth- ods in order to protect an object’s data from uncontrolled Synchronized methods implement the thread-safe ser- changes. It is relatively straightforward to extend this object- vices exported by a monitor object. To prevent race oriented programming model to protect an object’s data from conditions, only one synchronized method can execute uncontrolled concurrent changes, known as race conditions. within a monitor at any point in time, regardless of the Therefore, an object’s method interface should define its syn- number of threads that invoke the object’s synchronized chronization boundaries. methods concurrently or the number of synchronized methods in the object’s class. 2. Objects, not clients, should be responsible for their own method synchronization: Concurrent applications For instance, the put and get operations on the con- are harder to program if clients must explicitly acquire sumer handler’s message queue should be synchronized and release low-level synchronization mechanisms, such as methods to ensure that routing messages can be inserted semaphores, mutexes, or condition variables. Thus, objects and removed simultaneously by multiple threads with- should be responsible for ensuring that any of their methods out corrupting a queue’s internal state. requiring synchronization are serialized transparently, i.e., 1 In contrast, an active object [2] does have its own thread of control. without explicit client intervention. 2 Monitor lock 1. Synchronized method invocation and serialization: When a client invokes a synchronized method on a moni- Each monitor object contains its own monitor lock. tor object, the method must first acquire its monitor lock. A Synchronized methods use this monitor lock to seri- monitor lock cannot be acquired as long as another synchro- alize method invocations on a per-object basis. Each nized method is executing within the monitor object. In this synchronized method must acquire/release the monitor case, the client thread will block until it acquires the monitor lock when the method enters/exits the object, respec- lock, at which point the synchronized method will acquire tively. This protocol ensures the monitor lock is held the lock, enter its critical section, and perform the service whenever a method performs operations that access or implemented by the method. Once the synchronized method modify its object’s state. has finished executing, the monitor lock must be released so For instance, a Thread Mutex [3]couldbeusedto that other synchronized methods can access the monitor ob- implement the message queue’s monitor lock. ject. Monitor condition 2. Synchronized method thread suspension: If a syn- chronized method must block or cannot otherwise make im- Multiple synchronized methods running in separate mediate progress, it can wait on one of its monitor condi- threads can cooperatively schedule their execution se- tions, which causes it to “leave” the monitor object temporar- quences by waiting for and notifying each other via ily [5]. When a synchronized method leaves the monitor monitor conditions associated with their monitor object. object, the monitor lock is released automatically and the Synchronized methods use monitor conditions to deter- client’s thread of control is suspended on the monitor con- mine the circumstances under which they
Recommended publications
  • Design and Performance of a Modular Portable Object Adapter for Distributed, Real-Time, and Embedded CORBA Applications
    Design and Performance of a Modular Portable Object Adapter for Distributed, Real-Time, and Embedded CORBA Applications Raymond Klefstad, Arvind S. Krishna, and Douglas C. Schmidt g fklefstad, krishnaa, schmidt @uci.edu Electrical and Computer Engineering Dept. University of California, Irvine, CA 92697, USA Abstract of distributed computing from application developers to mid- dleware developers. It has been used successfully in large- ZEN is a CORBA ORB designed to support distributed, real- scale business systems where scalability, evolvability, and in- time, and embedded (DRE) applications that have stringent teroperability are essential for success. memory constraints. This paper discusses the design and per- Real-time CORBA [5] is a rapidly maturing DOC middle- formance of ZENs portable object adapter (POA) which is ware technology standardized by the OMG that can simplify an important component in a CORBA object request broker many challenges for DRE applications, just as CORBA has (ORB). This paper makes the following three contributions for large-scale business systems. Real-time CORBA is de- to the study of middleware for memory-constrained DRE ap- signed for applications with hard real-time requirements, such plications. First, it presents three alternative designs of the as avionics mission computing [6]. It can also handle ap- CORBA POA. Second, it explains how design patterns can be plications with stringent soft real-time requirements, such as applied to improve the quality and performance of POA im- telecommunication call processing and streaming video [7]. plementations. Finally, it presents empirical measurements ZEN [8] is an open-source Real-time CORBA object re- based on the ZEN ORB showing how memory footprint can quest broker (ORB) implemented in Real-time Java [9].
    [Show full text]
  • Active Object Systems Redacted for Privacy Abstract Approved
    AN ABSTRACT OF THE THESIS OF Sungwoon Choi for the degree of Doctor of Philosophyin Computer Science presented on February 6, 1992. Title: Active Object Systems Redacted for Privacy Abstract approved. v '" Toshimi Minoura An active object system isa transition-based object-oriented system suitable for the design of various concurrentsystems. An AOS consists of a collection of interacting objects, where the behavior of eachobject is determined by the transition statements provided in the class of that object.A transition statement is a condition-action pair, an equational assignmentstatement, or an event routine. The transition statements provided for eachobject can access, besides the state of that object, the states of the other objectsknown to it through its interface variables. Interface variablesare bound to objects when objects are instantiated so that desired connections among objects are established. The major benefit of the AOS approach is thatan active system can be hierarchically composed from its active software componentsas if it were a hardware system. An AOS provides better encapsulation andmore flexible communication protocols than ordinary object oriented systems, since control withinan AOS is localized. ©Copyright by Sungwoon Choi February 6, 1992 All Rights Reserved Active Object Systems by Sungwoon Choi A THESIS submitted to Oregon State University in partial fulfillment of the requirements for the degree of Doctor of Philosophy Completed February 6, 1992 Commencement June 1992 APPROVED: Redacted for Privacy Professor of Computer Science in charge ofmajor Redacted for Privacy Head of Department of Computer Science Redacted for Privacy Dean of Gradu to School Or Date thesis presented February 6, 1992 Typed by Sungwoon Choi for Sungwoon Choi To my wife Yejung ACKNOWLEDGEMENTS I would like to express my sincere gratitude tomy major professor, Dr.
    [Show full text]
  • Servant Documentation Release
    servant Documentation Release Servant Contributors February 09, 2018 Contents 1 Introduction 3 2 Tutorial 5 2.1 A web API as a type...........................................5 2.2 Serving an API.............................................. 10 2.3 Querying an API............................................. 28 2.4 Generating Javascript functions to query an API............................ 31 2.5 Custom function name builder...................................... 38 2.6 Documenting an API........................................... 38 2.7 Authentication in Servant........................................ 43 3 Cookbook 51 3.1 Structuring APIs............................................. 51 3.2 Serving web applications over HTTPS................................. 54 3.3 SQLite database............................................. 54 3.4 PostgreSQL connection pool....................................... 56 3.5 Using a custom monad.......................................... 58 3.6 Basic Authentication........................................... 60 3.7 Combining JWT-based authentication with basic access authentication................ 62 3.8 File Upload (multipart/form-data)............................... 65 4 Example Projects 69 5 Helpful Links 71 i ii servant Documentation, Release servant is a set of packages for declaring web APIs at the type-level and then using those API specifications to: • write servers (this part of servant can be considered a web framework), • obtain client functions (in haskell), • generate client functions for other programming
    [Show full text]
  • An Efficient Implementation of Guard-Based Synchronization for an Object-Oriented Programming Language an Efficient Implementation of Guard-Based
    AN EFFICIENT IMPLEMENTATION OF GUARD-BASED SYNCHRONIZATION FOR AN OBJECT-ORIENTED PROGRAMMING LANGUAGE AN EFFICIENT IMPLEMENTATION OF GUARD-BASED SYNCHRONIZATION FOR AN OBJECT-ORIENTED PROGRAMMING LANGUAGE By SHUCAI YAO, M.Sc., B.Sc. A Thesis Submitted to the Department of Computing and Software and the School of Graduate Studies of McMaster University in Partial Fulfilment of the Requirements for the Degree of Doctor of Philosophy McMaster University c Copyright by Shucai Yao, July 2020 Doctor of Philosophy (2020) McMaster University (Computing and Software) Hamilton, Ontario, Canada TITLE: An Efficient Implementation of Guard-Based Synchro- nization for an Object-Oriented Programming Language AUTHOR: Shucai Yao M.Sc., (Computer Science) University of Science and Technology Beijing B.Sc., (Computer Science) University of Science and Technology Beijing SUPERVISOR: Dr. Emil Sekerinski, Dr. William M. Farmer NUMBER OF PAGES: xvii,167 ii To my beloved family Abstract Object-oriented programming has had a significant impact on software development because it provides programmers with a clear structure of a large system. It encap- sulates data and operations into objects, groups objects into classes and dynamically binds operations to program code. With the emergence of multi-core processors, application developers have to explore concurrent programming to take full advan- tage of multi-core technology. However, when it comes to concurrent programming, object-oriented programming remains elusive as a useful programming tool. Most object-oriented programming languages do have some extensions for con- currency, but concurrency is implemented independently of objects: for example, concurrency in Java is managed separately with the Thread object. We employ a programming model called Lime that combines action systems tightly with object- oriented programming and implements concurrency by extending classes with actions and guarded methods.
    [Show full text]
  • Concurrency Patterns for Easier Robotic Coordination
    Concurrency Patterns for Easier Robotic Coordination Andrey Rusakov∗ Jiwon Shin∗ Bertrand Meyer∗† ∗Chair of Software Engineering, Department of Computer Science, ETH Zurich,¨ Switzerland †Software Engineering Lab, Innopolis University, Kazan, Russia fandrey.rusakov, jiwon.shin, [email protected] Abstract— Software design patterns are reusable solutions to Guarded suspension – are chosen for their concurrent nature, commonly occurring problems in software development. Grow- applicability to robotic coordination, and evidence of use ing complexity of robotics software increases the importance of in existing robotics frameworks. The paper aims to help applying proper software engineering principles and methods such as design patterns to robotics. Concurrency design patterns programmers who are not yet experts in concurrency to are particularly interesting to robotics because robots often have identify and then to apply one of the common concurrency- many components that can operate in parallel. However, there based solutions for their applications on robotic coordination. has not yet been any established set of reusable concurrency The rest of the paper is organized as follows: After design patterns for robotics. For this purpose, we choose six presenting related work in Section II, it proceeds with a known concurrency patterns – Future, Periodic timer, Invoke later, Active object, Cooperative cancellation, and Guarded sus- general description of concurrency approach for robotics pension. We demonstrate how these patterns could be used for in Section III. Then the paper presents and describes in solving common robotic coordination tasks. We also discuss detail a number of concurrency patterns related to robotic advantages and disadvantages of the patterns and how existing coordination in Section IV.
    [Show full text]
  • Active Object
    Active Object an Object Behavioral Pattern for Concurrent Programming R. Greg Lavender Douglas C. Schmidt [email protected] [email protected] ISODE Consortium Inc. Department of Computer Science Austin, TX Washington University, St. Louis An earlier version of this paper appeared in a chapter in the book ªPattern Languages of Program Design 2º ISBN 0-201-89527-7, edited by John Vlissides, Jim Coplien, and Norm Kerth published by Addison-Wesley, 1996. : Output : Input Handler Handler : Routing Table : Message Abstract Queue This paper describes the Active Object pattern, which decou- 3: enqueue(msg) ples method execution from method invocation in order to : Output 2: find_route(msg) simplify synchronized access to a shared resource by meth- Handler ods invoked in different threads of control. The Active Object : Message : Input pattern allows one or more independent threads of execution Queue Handler 1: recv(msg) to interleave their access to data modeled as a single ob- ject. A broad class of producer/consumer and reader/writer OUTGOING OUTGOING MESSAGES GATEWAY problems are well-suited to this model of concurrency. This MESSAGES pattern is commonly used in distributed systems requiring INCOMING INCOMING multi-threaded servers. In addition,client applications (such MESSAGES MESSAGES as windowing systems and network browsers), are increas- DST DST ingly employing active objects to simplify concurrent, asyn- SRC SRC chronous network operations. 1 Intent Figure 1: Connection-Oriented Gateway The Active Object pattern decouples method execution from method invocation in order to simplify synchronized access to a shared resource by methods invoked in different threads [2]. Sources and destinationscommunicate with the Gateway of control.
    [Show full text]
  • Servant Documentation Release
    servant Documentation Release Servant Contributors March 07, 2016 Contents 1 Introduction 3 2 Tutorial 5 2.1 A web API as a type...........................................5 2.2 Serving an API..............................................9 2.3 Querying an API............................................. 25 2.4 Generating Javascript functions to query an API............................ 28 2.5 Documenting an API........................................... 30 3 Helpful Links 35 i ii servant Documentation, Release servant is a set of packages for declaring web APIs at the type-level and then using those API specifications to: • write servers (this part of servant can be considered a web framework), • obtain client functions (in haskell), • generate client functions for other programming languages, • generate documentation for your web applications • and more... All in a type-safe manner. Contents 1 servant Documentation, Release 2 Contents CHAPTER 1 Introduction servant has the following guiding principles: • concision This is a pretty wide-ranging principle. You should be able to get nice documentation for your web servers, and client libraries, without repeating yourself. You should not have to manually serialize and deserialize your resources, but only declare how to do those things once per type. If a bunch of your handlers take the same query parameters, you shouldn’t have to repeat that logic for each handler, but instead just “apply” it to all of them at once. Your handlers shouldn’t be where composition goes to die. And so on. • flexibility If we haven’t thought of your use case, it should still be easily achievable. If you want to use templating library X, go ahead. Forms? Do them however you want, but without difficulty.
    [Show full text]
  • Design Patterns
    Design Patterns Andy Verkeyn April 2001 - March 2003 Third version [email protected] http://allserv.rug.ac.be/~averkeyn Contents Contents ______________________________________________________________________ 1 Foreword _____________________________________________________________________ 1 Introduction___________________________________________________________________ 1 Iterator _______________________________________________________________________ 2 Adapter (class)_________________________________________________________________ 4 Adapter (object) ________________________________________________________________ 5 Decorator _____________________________________________________________________ 7 Singleton _____________________________________________________________________ 9 Chain of responsibility _________________________________________________________ 12 Observer_____________________________________________________________________ 14 Command____________________________________________________________________ 16 Bridge_______________________________________________________________________ 18 Factory method _______________________________________________________________ 20 Abstract factory _______________________________________________________________ 21 Proxy _______________________________________________________________________ 22 Other patterns used in Java _____________________________________________________ 24 Foreword This text is mainly taken from the excellent book “Design patterns: Elements of reusable Object- Oriented
    [Show full text]
  • MULTI-ACTIVE OBJECTS and THEIR APPLICATIONS 1. Introduction
    Logical Methods in Computer Science Vol. 13(4:12)2017, pp. 1–41 Submitted Nov. 01, 2016 https://lmcs.episciences.org/ Published Nov. 21, 2017 MULTI-ACTIVE OBJECTS AND THEIR APPLICATIONS LUDOVIC HENRIO AND JUSTINE ROCHAS Universit´eC^oted'Azur, CNRS, I3S, France e-mail address: [email protected], [email protected] Abstract. In order to tackle the development of concurrent and distributed systems, the active object programming model provides a high-level abstraction to program concurrent behaviours. There exists already a variety of active object frameworks targeted at a large range of application domains: modelling, verification, efficient execution. However, among these frameworks, very few consider a multi-threaded execution of active objects. Introducing controlled parallelism within active objects enables overcoming some of their limitations. In this paper, we present a complete framework around the multi-active object programming model. We present it through ProActive, the Java library that offers multi-active objects, and through MultiASP, the programming language that allows the formalisation of our developments. We then show how to compile an active object language with cooperative multi-threading into multi-active objects. This paper also presents different use cases and the development support to illustrate the practical usability of our language. Formalisation of our work provides the programmer with guarantees on the behaviour of the multi-active object programming model and of the compiler. 1. Introduction Systems and applications nowadays run in a concurrent and distributed manner. In general, existing programming models and languages lack adequate abstractions for handling the concurrency of parallel programs and for writing distributed applications.
    [Show full text]
  • Department of Veterans Affairs Text Integration Utilities (TIU)
    Department of Veterans Affairs Text Integration Utilities (TIU) Technical Manual April 2021 Office of Information & Technology (OIT) Product Development Revision History Date Description Author/Project Manager April 2021 Patch TIU*1.0*330: Under Section 1.3.1, Recently Released Patches: • Added description of Patch TIU*1.0*330 which adds a new Document Class (EHRM CUTOVER (PARENT FACILITY NAME) and two new note titles • Complete 508 accessibility Globally updated dates on Title page and in footers November 2020 Patch TIU*1*336 Corrected the oversight in ALLMEDS parameter #1: Added Clinic Meds to it, 219 This correction, related to TIU*1.0*290, was incorporated into the TIU*1.0*336 release.. Made the manual 508-compliant. November 2020 Patch TIU*1*328 Under Section 1.3.1, Recently Released Patches, added paragraph describing Patch TIU*1*328 update. Under Section 11, Glossary, added Prescription Drug-Monitoring Program PDMP acronym. October 2020 Patch TIU*1.0*333 This patch fixes an issue with HL7 message for the OEHRM project. Specifically, when an addendum is signed it was previously completing the parent consult when the parent document was not yet completed. This was remedied to complete only the addendum. See Recently Released Patches – October 2020 update September 2020 Patch TIU*1*290 – Added to the entry about ALLMEDS, 219 Added a Protocols section, including some information regarding what to do if the TIU ACTIONS RESOURCE device becomes backed up, 160 April 2021 Text Integration Utilities (TIU) v1.0 Technical Manual i Added an overview of changes for TIU under Recently Released Patches: page 3 June 2020 Patch TIU*1*290 – Updated Copy/Paste Tracking information for Basic TIU Parameters on page 15, Document Parameter Edit on page 53, and Copy/Paste Tracking Reports ..
    [Show full text]
  • IC Coder's Club Design Patterns (In C++)
    Andrew W. Rose Simple exercise You have been tasked to move this pile from A to B Simple exercise You have been tasked to move this pile from A to B You have three resources available to you: a) Bare hands b) A stack of timber c) A horse and cart Simple exercise You have been tasked to move this pile from A to B You have three resources available to you: a) Bare hands b) A stack of timber c) A horse and cart How do you achieve the task in the quickest, least-painful way, which won’t leave you up-to-your-neck in the produce you are moving, nor smelling of it? Software analogy a) Bare hands b) A stack of timber c) A horse and cart Software analogy a) Bare hands b) A stack of timber c) A horse and cart Do a task manually Software analogy a) Bare hands b) A stack of timber c) A horse and cart Design Do a task the tools manually yourself Software analogy a) Bare hands b) A stack of timber c) A horse and cart Benefit from Design Do a task someone the tools manually else’s yourself hard work Software analogy a) Bare hands b) A stack of timber c) A horse and cart Benefit from someone else’s This is the purpose of design patterns! hard work Motivation I will, in fact, claim that the difference between a bad programmer and a good one is whether he considers his code or his data structures more important: Bad programmers worry about the code; good programmers worry about data structures and their relationships.
    [Show full text]
  • Plinycompute: a Platform for High-Performance, Distributed, Data-Intensive Tool Development
    PlinyCompute: A Platform for High-Performance, Distributed, Data-Intensive Tool Development Jia Zou, R. Matthew Barnett, Tania Lorido-Botran, Shangyu Luo, Carlos Monroy Sourav Sikdar, Kia Teymourian, Binhang Yuan, Chris Jermaine Rice University Houston, Texas, USA Abstract This paper describes PlinyCompute, a system for development of high-performance, data-intensive, dis- tributed computing tools and libraries. In the large, PlinyCompute presents the programmer with a very high-level, declarative interface, relying on automatic, relational-database style optimization to figure out how to stage distributed computations. However, in the small, PlinyCompute presents the capable systems programmer with a persistent object data model and API (the “PC object model”) and associated memory management system that has been designed from the ground-up for high performance, distributed, data- intensive computing. This contrasts with most other Big Data systems, which are constructed on top of the Java Virtual Machine (JVM), and hence must at least partially cede performance-critical concerns such as memory management (including layout and de/allocation) and virtual method/function dispatch to the JVM. This hybrid approach—declarative in the large, trusting the programmer’s ability to utilize PC object model efficiently in the small—results in a system that is ideal for the development of reusable, data-intensive tools and libraries. Through extensive benchmarking, we show that implementing complex objects manipulation and non-trivial, library-style computations on top of PlinyCompute can result in a speedup of 2× to more than 50× or more compared to equivalent implementations on Spark. 1 Introduction Big Data systems such as Spark [70] and Flink [11, 27] have effectively solved what we call the “data munging” problem.
    [Show full text]