COS 318: Operating Systems Message Passing
Total Page:16
File Type:pdf, Size:1020Kb
Load more
Recommended publications
-
Network/Socket Programming in Java
Network/Socket Programming in Java Rajkumar Buyya Elements of C-S Computing a client, a server, and network Request Client Server Network Result Client machine Server machine java.net n Used to manage: c URL streams c Client/server sockets c Datagrams Part III - Networking ServerSocket(1234) Output/write stream Input/read stream Socket(“128.250.25.158”, 1234) Server_name: “manjira.cs.mu.oz.au” 4 Server side Socket Operations 1. Open Server Socket: ServerSocket server; DataOutputStream os; DataInputStream is; server = new ServerSocket( PORT ); 2. Wait for Client Request: Socket client = server.accept(); 3. Create I/O streams for communicating to clients is = new DataInputStream( client.getInputStream() ); os = new DataOutputStream( client.getOutputStream() ); 4. Perform communication with client Receiive from client: String line = is.readLine(); Send to client: os.writeBytes("Hello\n"); 5. Close sockets: client.close(); For multithreade server: while(true) { i. wait for client requests (step 2 above) ii. create a thread with “client” socket as parameter (the thread creates streams (as in step (3) and does communication as stated in (4). Remove thread once service is provided. } Client side Socket Operations 1. Get connection to server: client = new Socket( server, port_id ); 2. Create I/O streams for communicating to clients is = new DataInputStream( client.getInputStream() ); os = new DataOutputStream( client.getOutputStream() ); 3. Perform communication with client Receiive from client: String line = is.readLine(); Send to client: os.writeBytes("Hello\n"); -
Efficient Inter-Core Communications on Manycore Machines
ZIMP: Efficient Inter-core Communications on Manycore Machines Pierre-Louis Aublin Sonia Ben Mokhtar Gilles Muller Vivien Quema´ Grenoble University CNRS - LIRIS INRIA CNRS - LIG Abstract—Modern computers have an increasing num- nication mechanisms enabling both one-to-one and one- ber of cores and, as exemplified by the recent Barrelfish to-many communications. Current operating systems operating system, the software they execute increasingly provide various inter-core communication mechanisms. resembles distributed, message-passing systems. To sup- port this evolution, there is a need for very efficient However it is not clear yet how these mechanisms be- inter-core communication mechanisms. Current operat- have on manycore processors, especially when messages ing systems provide various inter-core communication are intended to many recipient cores. mechanisms, but it is not clear yet how they behave In this report, we study seven communication mech- on manycore processors. In this report, we study seven anisms, which are considered state-of-the-art. More mechanisms, that are considered state-of-the-art. We show that these mechanisms have two main drawbacks that precisely, we study TCP, UDP and Unix domain sock- limit their efficiency: they perform costly memory copy ets, pipes, IPC and POSIX message queues. These operations and they do not provide efficient support mechanisms are supported by traditional operating sys- for one-to-many communications. We do thus propose tems such as Linux. We also study Barrelfish message ZIMP, a new inter-core communication mechanism that passing, the inter-process communication mechanism of implements zero-copy inter-core message communications and that efficiently handles one-to-many communications. -
D-Bus, the Message Bus System Training Material
Maemo Diablo D-Bus, The Message Bus System Training Material February 9, 2009 Contents 1 D-Bus, The Message Bus System 2 1.1 Introduction to D-Bus ......................... 2 1.2 D-Bus architecture and terminology ................ 3 1.3 Addressing and names in D-Bus .................. 4 1.4 Role of D-Bus in maemo ....................... 6 1.5 Programming directly with libdbus ................. 9 1 Chapter 1 D-Bus, The Message Bus System 1.1 Introduction to D-Bus D-Bus (the D originally stood for "Desktop") is a relatively new inter process communication (IPC) mechanism designed to be used as a unified middleware layer in free desktop environments. Some example projects where D-Bus is used are GNOME and Hildon. Compared to other middleware layers for IPC, D-Bus lacks many of the more refined (and complicated) features and for that reason, is faster and simpler. D-Bus does not directly compete with low level IPC mechanisms like sock- ets, shared memory or message queues. Each of these mechanisms have their uses, which normally do not overlap the ones in D-Bus. Instead, D-Bus aims to provide higher level functionality, like: Structured name spaces • Architecture independent data formatting • Support for the most common data elements in messages • A generic remote call interface with support for exceptions (errors) • A generic signalling interface to support "broadcast" type communication • Clear separation of per-user and system-wide scopes, which is important • when dealing with multi-user systems Not bound to any specific programming language (while providing a • design that readily maps to most higher level languages, via language specific bindings) The design of D-Bus benefits from the long experience of using other mid- dleware IPC solutions in the desktop arena and this has allowed the design to be optimised. -
Retrofitting Privacy Controls to Stock Android
Saarland University Faculty of Mathematics and Computer Science Department of Computer Science Retrofitting Privacy Controls to Stock Android Dissertation zur Erlangung des Grades des Doktors der Ingenieurwissenschaften der Fakultät für Mathematik und Informatik der Universität des Saarlandes von Philipp von Styp-Rekowsky Saarbrücken, Dezember 2019 Tag des Kolloquiums: 18. Januar 2021 Dekan: Prof. Dr. Thomas Schuster Prüfungsausschuss: Vorsitzender: Prof. Dr. Thorsten Herfet Berichterstattende: Prof. Dr. Michael Backes Prof. Dr. Andreas Zeller Akademischer Mitarbeiter: Dr. Michael Schilling Zusammenfassung Android ist nicht nur das beliebteste Betriebssystem für mobile Endgeräte, sondern auch ein ein attraktives Ziel für Angreifer. Um diesen zu begegnen, nutzt Androids Sicher- heitskonzept App-Isolation und Zugangskontrolle zu kritischen Systemressourcen. Nutzer haben dabei aber nur wenige Optionen, App-Berechtigungen gemäß ihrer Bedürfnisse einzuschränken, sondern die Entwickler entscheiden über zu gewährende Berechtigungen. Androids Sicherheitsmodell kann zudem nicht durch Dritte angepasst werden, so dass Nutzer zum Schutz ihrer Privatsphäre auf die Gerätehersteller angewiesen sind. Diese Dissertation präsentiert einen Ansatz, Android mit umfassenden Privatsphäreeinstellun- gen nachzurüsten. Dabei geht es konkret um Techniken, die ohne Modifikationen des Betriebssystems oder Zugriff auf Root-Rechte auf regulären Android-Geräten eingesetzt werden können. Der erste Teil dieser Arbeit etabliert Techniken zur Durchsetzung von Sicherheitsrichtlinien -
How to XDP (With Snabb)
How to XDP (with Snabb) Max Rottenkolber <[email protected]> Tuesday, 21 January 2020 Table of Contents Intro 1 Creating an XDP socket 2 Mapping the descriptor rings 4 Binding an XDP socket to an interface 7 Forwarding packets from a queue to an XDP socket 8 Creating a BPF map . 9 Assembling a BPF program . 9 Attaching a BPF program to an interface . 12 Packet I/O 14 Deallocating the XDP socket 18 Intro Networking in userspace is becoming mainstream. Recent Linux kernels ship with a feature called eXpress Data Path (XDP) that intends to con- solidate kernel-bypass packet processing applications with the kernel’s networking stack. XDP provides a new kind of network socket that al- lows userspace applications to do almost direct I/O with network device drivers while maintaining integration with the kernel’s native network stack. To Snabb XDP looks like just another network I/O device. In fact the XDP ABI/API is so similar to a typical hardware interfaces that it feels a bit like driving a NIC, and not by accident. I think it is fair to describe XDP as a generic interface to Linux network drivers that also 1 has a virtual backend it integrates with: the Linux network stack. In that sense, XDP can be seen as a more hardware centric sibling of AF_PACKET. I can think of two reasons why XDP might be interesting for Snabb users: • While the Snabb community prefers NICs with open hardware interface specifications there are a lot of networking devices out there that lack open specifications. -
Message Passing Model
Message Passing Model Giuseppe Anastasi [email protected] Pervasive Computing & Networking Lab. (PerLab) Dept. of Information Engineering, University of Pisa PerLab Based on original slides by Silberschatz, Galvin and Gagne Overview PerLab Message Passing Model Addressing Synchronization Example of IPC systems Message Passing Model 2 Operating Systems Objectives PerLab To introduce an alternative solution (to shared memory) for process cooperation To show pros and cons of message passing vs. shared memory To show some examples of message-based communication systems Message Passing Model 3 Operating Systems Inter-Process Communication (IPC) PerLab Message system – processes communicate with each other without resorting to shared variables. IPC facility provides two operations: send (message ) – fixed or variable message size receive (message ) If P and Q wish to communicate, they need to: establish a communication link between them exchange messages via send/receive The communication link is provided by the OS Message Passing Model 4 Operating Systems Implementation Issues PerLab Physical implementation Single-processor system Shared memory Multi-processor systems Hardware bus Distributed systems Networking System + Communication networks Message Passing Model 5 Operating Systems Implementation Issues PerLab Logical properties Can a link be associated with more than two processes? How many links can there be between every pair of communicating processes? What is the capacity of a link? Is the size of a message that the link can accommodate fixed or variable? Is a link unidirectional or bi-directional? Message Passing Model 6 Operating Systems Implementation Issues PerLab Other Aspects Addressing Synchronization Buffering Message Passing Model 7 Operating Systems Overview PerLab Message Passing Model Addressing Synchronization Example of IPC systems Message Passing Model 8 Operating Systems Direct Addressing PerLab Processes must name each other explicitly. -
Improving Networking
IMPERIAL COLLEGE LONDON FINALYEARPROJECT JUNE 14, 2010 Improving Networking by moving the network stack to userspace Author: Matthew WHITWORTH Supervisor: Dr. Naranker DULAY 2 Abstract In our modern, networked world the software, protocols and algorithms involved in communication are among some of the most critical parts of an operating system. The core communication software in most modern systems is the network stack, but its basic monolithic design and functioning has remained unchanged for decades. Here we present an adaptable user-space network stack, as an addition to my operating system Whitix. The ideas and concepts presented in this report, however, are applicable to any mainstream operating system. We show how re-imagining the whole architecture of networking in a modern operating system offers numerous benefits for stack-application interactivity, protocol extensibility, and improvements in network throughput and latency. 3 4 Acknowledgements I would like to thank Naranker Dulay for supervising me during the course of this project. His time spent offering constructive feedback about the progress of the project is very much appreciated. I would also like to thank my family and friends for their support, and also anybody who has contributed to Whitix in the past or offered encouragement with the project. 5 6 Contents 1 Introduction 11 1.1 Motivation.................................... 11 1.1.1 Adaptability and interactivity.................... 11 1.1.2 Multiprocessor systems and locking................ 12 1.1.3 Cache performance.......................... 14 1.2 Whitix....................................... 14 1.3 Outline...................................... 15 2 Hardware and the LDL 17 2.1 Architectural overview............................. 17 2.2 Network drivers................................. 18 2.2.1 Driver and device setup....................... -
Message Passing
COS 318: Operating Systems Message Passing Jaswinder Pal Singh Computer Science Department Princeton University (http://www.cs.princeton.edu/courses/cos318/) Sending A Message Within A Computer Across A Network P1 P2 Send() Recv() Send() Recv() Network OS Kernel OS OS COS461 2 Synchronous Message Passing (Within A System) Synchronous send: u Call send system call with M u send system call: l No buffer in kernel: block send( M ) recv( M ) l Copy M to kernel buffer Synchronous recv: u Call recv system call M u recv system call: l No M in kernel: block l Copy to user buffer How to manage kernel buffer? 3 API Issues u Message S l Buffer (addr) and size l Message type, buffer and size u Destination or source send(dest, msg) R l Direct address: node Id, process Id l Indirect address: mailbox, socket, recv(src, msg) channel, … 4 Direct Addressing Example Producer(){ Consumer(){ ... ... while (1) { for (i=0; i<N; i++) produce item; send(Producer, credit); recv(Consumer, &credit); while (1) { send(Consumer, item); recv(Producer, &item); } send(Producer, credit); } consume item; } } u Does this work? u Would it work with multiple producers and 1 consumer? u Would it work with 1 producer and multiple consumers? u What about multiple producers and multiple consumers? 5 Indirect Addressing Example Producer(){ Consumer(){ ... ... while (1) { for (i=0; i<N; i++) produce item; send(prodMbox, credit); recv(prodMbox, &credit); while (1) { send(consMbox, item); recv(consMbox, &item); } send(prodMbox, credit); } consume item; } } u Would it work with multiple producers and 1 consumer? u Would it work with 1 producer and multiple consumers? u What about multiple producers and multiple consumers? 6 Indirect Communication u Names l mailbox, socket, channel, … u Properties l Some allow one-to-one mbox (e.g. -
Post Sockets - a Modern Systems Network API Mihail Yanev (2065983) April 23, 2018
Post Sockets - A Modern Systems Network API Mihail Yanev (2065983) April 23, 2018 ABSTRACT In addition to providing native solutions for problems, es- The Berkeley Sockets API has been the de-facto standard tablishing optimal connection with a Remote point or struc- systems API for networking. However, designed more than tured data communication, we have identified that a com- three decades ago, it is not a good match for the networking mon issue with many of the previously provided networking applications today. We have identified three different sets of solutions has been leaving the products vulnerable to code problems in the Sockets API. In this paper, we present an injections and/or buffer overflow attacks. The root cause implementation of Post Sockets - a modern API, proposed by for such problems has mostly proved to be poor memory Trammell et al. that solves the identified limitations. This or resource management. For this reason, we have decided paper can be used for basis of evaluation of the maturity of to use the Rust programming language to implement the Post Sockets and if successful to promote its introduction as new API. Rust is a programming language, that provides a replacement API to Berkeley Sockets. data integrity and memory safety guarantees. It uses region based memory, freeing the programmer from the responsibil- ity of manually managing the heap resources. This in turn 1. INTRODUCTION eliminates the possibility of leaving the code vulnerable to Over the years, writing networked applications has be- resource management errors, as this is handled by the lan- come increasingly difficult. -
Multiprocess Communication and Control Software for Humanoid Robots Neil T
IEEE Robotics and Automation Magazine Multiprocess Communication and Control Software for Humanoid Robots Neil T. Dantam∗ Daniel M. Lofaroy Ayonga Hereidx Paul Y. Ohz Aaron D. Amesx Mike Stilman∗ I. Introduction orrect real-time software is vital for robots in safety-critical roles such as service and disaster response. These systems depend on software for Clocomotion, navigation, manipulation, and even seemingly innocuous tasks such as safely regulating battery voltage. A multi-process software design increases robustness by isolating errors to a single process, allowing the rest of the system to continue operating. This approach also assists with modularity and concurrency. For real-time tasks such as dynamic balance and force control of manipulators, it is critical to communicate the latest data sample with minimum latency. There are many communication approaches intended for both general purpose and real-time needs [19], [17], [13], [9], [15]. Typical methods focus on reliable communication or network-transparency and accept a trade-off of increased mes- sage latency or the potential to discard newer data. By focusing instead on the specific case of real-time communication on a single host, we reduce communication latency and guarantee access to the latest sample. We present a new Interprocess Communication (IPC) library, Ach,1 which addresses this need, and discuss its application for real-time, multiprocess control on three humanoid robots (Fig. 1). There are several design decisions that influenced this robot software and motivated development of the Ach library. First, to utilize decades of prior development and engineering, we implement our real-time system on top of a POSIX-like Operating System (OS)2. -
Message Passing
Message passing CS252 • Sending of messages under control of programmer – User-level/system level? Graduate Computer Architecture – Bulk transfers? Lecture 24 • How efficient is it to send and receive messages? – Speed of memory bus? First-level cache? Network Interface Design • Communication Model: Memory Consistency Models – Synchronous » Send completes after matching recv and source data sent » Receive completes after data transfer complete from matching send – Asynchronous Prof John D. Kubiatowicz » Send completes after send buffer may be reused http://www.cs.berkeley.edu/~kubitron/cs252 4/27/2009 cs252-S09, Lecture 24 2 Synchronous Message Passing Asynch. Message Passing: Optimistic Source Destination Source Destination (1) Initiate send Recv Psrc, local VA, len (1) Initiate send (2) Address translation on Psrc Send Pdest, local VA, len (2) Address translation (3) Local/remote check Send (P , local VA, len) (3) Local/remote check dest (4) Send-ready request Send-rdy req (4) Send data (5) Remote check for (5) Remote check for posted receive; on fail, Tag match allocate data buffer posted receive Wait Tag check Data-xfer req (assume success) Processor Allocate buffer Action? (6) Reply transaction Recv-rdy reply (7) Bulk data transfer Source VA Dest VA or ID Recv P , local VA, len Time src Data-xfer req Time • More powerful programming model • Constrained programming model. • Wildcard receive => non-deterministic • Deterministic! What happens when threads added? • Destination contention very limited. • Storage required within msg layer? • User/System boundary? 4/27/2009 cs252-S09, Lecture 24 3 4/27/2009 cs252-S09, Lecture 24 4 Asynch. Msg Passing: Conservative Features of Msg Passing Abstraction Destination • Source knows send data address, dest. -
Inter-Process Communication
QNX Inter-Process Communication QNX IPC 2008/01/08 R08 © 2006 QNX Software Systems. QNX and Momentics are registered trademarks of QNX Software Systems in certain jurisdictions NOTES: QNX, Momentics, Neutrino, Photon microGUI, and “Build a more reliable world” are registered trademarks in certain jurisdictions, and PhAB, Phindows, and Qnet are trademarks of QNX Software Systems Ltd. All other trademarks and trade names belong to their respective owners. 1 IntroductionIntroduction IPC: • Inter-Process Communication • two processes exchange: – data – control – notification of event occurrence IPC Thread Thread Process A Process B QNX IPC 2008/01/08 R08 2 © 2006, QNX Software Systems. QNX and Momentics are registered trademarks of QNX Software Systems in certain jurisdictions NOTES: 2 IntroductionIntroduction QNX Neutrino supports a wide variety of IPC: – QNX Core (API is unique to QNX or low-level) • includes: – QNX Neutrino messaging – QNX Neutrino pulses – shared memory – QNX asynchronous messaging • the focus of this course module – POSIX/Unix (well known, portable API’s ) • includes: – signals – shared memory – pipes (requires pipe process) – POSIX message queues (requires mqueue or mq process) – TCP/IP sockets (requires io-net process) • the focus of the POSIX IPC course module QNX IPC 2008/01/08 R08 3 © 2006, QNX Software Systems. QNX and Momentics are registered trademarks of QNX Software Systems in certain jurisdictions NOTES: API: Application Programming Interface 3 InterprocessInterprocess Communication Communication Topics: Message Passing Designing a Message Passing System (1) Pulses How a Client Finds a Server Server Cleanup Multi-Part Messages Designing a Message Passing System (2) Client Information Structure Issues Related to Priorities Designing a Message Passing System (3) Event Delivery Asynchronous Message Passing QNET Shared Memory Conclusion QNX IPC 2008/01/08 R08 4 © 2006, QNX Software Systems.