10/8/15

DISTRIBUTED SYSTEMS

MESSAGE ORIENTED COMMUNICATIONS

Dr. Jack Lange Department University of Pittsburgh

Fall 2015

Outline n Message Oriented Communication n Sockets and Socket API n Interface – MPI n Architecture and Design Issues n Message Oriented Persistent Communication n Message Queues n IBM WebSphereMQ n Stream Oriented Communication n Continuous Media Requirements n Flow Specification and QoS Support n Delay and Jitter Control n Packet Loss Control n Conclusion

1 10/8/15

MESSAGE ORIENTED COMMUNICATION SOCKETS

Sockets n A Socket is a communication end point n Processes can send and Host or Host or Server receive messages to and Server from its socket n Abstraction over the Controlled by Process Application communication end point socket Developer socket that is by the local OS for a TCP with TCP with specific transport protocol buffers, buffers, variables variables n Sending process relies on transport infrastructure which brings message to socket at receiving process

2 10/8/15

Socket Arguments – Protocol n Function socket – int socket ( int family, int type, int protocol) n Return > 0 if success and -1 if error

Family! Type! Protocol! Actual Protocol!

AF_INET! SOCK_DGRAM! IPPROTO_UDP! UDP!

AF_INET! SOCK_STREAM! IPPROTO_TCP! TCP!

AF_INET! SOCK_RAW! IPPROTO_ICMP! ICMP!

AF_INET! SOCK_RAW! IPPROTO_RAW! raw IP!

Sockets Creation n The system call socket() is used to create a new socket n Int sockfd = socket(PF_INET, SOCK_STREAM, 0); n PF_INET = Protocol Family (Internet) n SOCK_STREAM = TCP n TCP is a reliable protocol n SOCK_DGRAM = UDP n UDP is a “best effort” protocol, no guaranteed reliability

3 10/8/15

Overview of the Sockets Interface

Client! Server! Socket() Socket()

Bind() open_listenfd open_clientfd Listen() Connection! Request! connect Accept()

Client! Write() Read() Server! Await connection! Session! request from! read() write() next client!

EOF! Close() Read()

Close()

MESSAGE ORIENTED COMMUNICATION

MESSAGE PASSING INTERFACE

4 10/8/15

What is MPI? n A message-passing specifications for parallel , clusters, and heterogeneous networks n Extended message-passing model n Communication modes n Standard, synchronous, buffered, and ready. n Designed for parallel applications and tailored to transient communications n Provides access to advanced parallel hardware n It assumes that serious failures are fatal and do not require automatic recovery n For example, process crashes or network disconnect

Group and Context n MPI assumes communication takes place with a group of processes – (groupID, procID) n Use in lieu of a transport-level address n MPI ties the concepts of process group and communication context into a communicator ! ! Group! Context! ! ! Communicator

5 10/8/15

MPI Abstractions n A process group is high-level abstraction, visible to users n A context is a system-defined object that uniquely identifies a communicator – Mechanism to isolate messages in distinct libraries and the user programs from one another n A message sent in one context can't be received in other contexts. n The communication context is low-level abstraction, not visible to users n A communicator is a data object that specializes the scope of a communication. n MPI_COMM_WORLD is an initial communicator, which is predefined and consists of all the processes running when program execution begins

MPI Communication Semantics n Point-to-point communication is the basic concept of MPI standard and fundamental for send and receive operations for typed data with associated message tag. n Using the point-to point communication, messages can be passed to another process with explicit message tag and implicit communication context. n Each process can carry out its own code in MIMD style, sequential or multi-threaded. n MPI is made -safe by not using global state.

6 10/8/15

MPI Communication Primitives

Primitive Meaning

MPI_bsend Append outgoing message to a local send buffer

MPI_send Send a message and wait until copied to local or remote buffer

MPI_ssend Send a message and wait until receipt starts

MPI_sendrecv Send a message and wait for reply

MPI_isend Pass reference to outgoing message, and continue

MPI_issend Pass reference to outgoing message, and wait until receipt starts

MPI_recv Receive a message; block if there are none

MPI_irecv Check if there is an incoming message, but do not block

MESSAGE ORIENTED COMMUNICATION

MESSAGE QUEUING SYSTEMS

7 10/8/15

Message-Queuing Model n MQM, aka Message Oriented , provides support for persistent asynchronous communication n MOM offers intermediate-term message storage capacity, without requiring either the sender of receiver to be active

Sender Running! Sender Running! Sender Passive! Sender Passive!

Receiver Running! Receiver Passive! Receiver Running! Receiver Passive!

MOM Basic Semantics and Interface

n MOM supports loosely-coupled in time semantics n Senders are given only guarantees that message will eventually be inserted in the receiver’s queue n No guarantee about if or when will messages be read n Recipient-behavior dependent n Basic interface to a queue in a message-queuing system.

Primitive Meaning

Put Append a message to a specified queue

Get Block until the specified queue is nonempty, and remove the first message

Poll Check a specified queue for messages, and remove the first. Never block. Install a handler to be called when a message is put into the specified Notify queue.

8 10/8/15

MQM General Architecture n Source and destination queues are distributed across multiple machines n The relationship between queue-level addressing and network-level addressing.

MQM Generalized Architecture – Routers n The general organization of a message-queuing system with routers.

2-29

9 10/8/15

Message Brokers n Common message format, typically used in traditional networks, does not generally work for MQM systems n Mostly due to the diversity of applications, which reduces the best common format to a sequence of bytes n Brokers – in a message-queuing system.

MQM Brokers Functionalities n Message brokers act as an application-level gateway n Their main purpose is to convert incoming messages to a format that can be understood by the recipient n Properly match end-of-record delimiters between applications n More advanced broker are designed to handle conversations between two different database applications n Difficult task, as information and semantic “mapping” between two different applications may not always be possible n Brokers are commonly used in Enterprise Application Integration for mediation n Publish/Subscribe is the typical communication model n Brokers “matches” messages to applications’ interests

10 10/8/15

WebSphere MQ – MQ Example

n General organization of IBM's WebSpher MQ message-queuing system – Queue Managers (QM), Message Channels (MC), and Message Channel Agents (MCA)

2-31

WebSphereMQ MCs and MCAs

n Message channels are an abstraction of transport level connections. n A MC is a unidirectional, reliable connection between a sending and a receiving QM n Internet-based MCs are implemented as TCP connections n The two ends of a MC are managed by a MCA

Attribute Description

Transport type Determines the transport protocol to be used

FIFO delivery Indicates that messages are to be delivered in the order they are sent

Message length Maximum length of a single message Setup retry Specifies maximum number of retries to start up the remote MCA count Delivery retries Maximum times MCA will try to put received message into queue

11 10/8/15

WebSphereMQ Message Transfer n Generalized organization queuing network using routing tables and aliases. n Routing Table Entry: , where DestMQ is the name of the destination QM and senQ is the name of the local send queue

Message Transfer Primitives n WebSphereMQ Interface Primitives

Primitive Description

MQopen Open a (possibly remote) queue

MQclose Close a queue

MQput Put a message into an opened queue

MQget Get a message from a (local) queue

12 10/8/15

MESSAGE ORIENTED COMMUNICATION

STREAM ORIENTED COMMUNICATION

Stream Communication Model

n Support for data streams, time-dependent information such as audio and video, requires communication models that go beyond those that support independent and complete units of information

General Architecture for Streaming Stored Media over a Network!

13 10/8/15

Data Streams – Point-to-Point n Setting up a stream directly between two devices.

2-35.2

Data Streams -- Multicasting n An example of multicasting a stream to several receivers.

14 10/8/15

Streams, flows and QoS n The Quality of Service (QoS) of an application is the “look and feel” of how the stream will flow n QoS Support depends on two components n Specification of the flow/stream n Enforcing the QoS requirements specification of the supported flows n The QoS support implementation depends on many, many factors, n It may require the “Flow Admission Protocol” n Given a new flow specifications, and a set of currently supported flows, a decision has to be made whether the new flow is to be accepted or rejected

Streams and Quality of Service

n QoS Flow Specification Parameters n The required bit rate at which data should be transported. n The maximum delay until a session has been set up n The maximum end-to-end delay . n The maximum delay variance, or jitter. n The maximum round-trip delay.

15 10/8/15

QoS Specification and Services

n A flow specification model and services

Characteristics of the Input Service Required

§ Maximum data unit size (bytes) § Loss sensitivity (bytes) § Token bucket rate (bytes/sec) § Loss interval (µsec) § Token bucket size (bytes) § Burst loss sensitivity (data § Maximum transmission rate (bytes/sec) units) § Minimum delay noticed (µsec) § Maximum delay variation (µsec) § Quality of guarantee

Specifying QoS – Token Bucket

n The principle of a token bucket algorithm.

16 10/8/15

Enforcing QoS – Binding Jitter n Buffers can be used to reduce jitter n Jitter represents the delay variation between consecutive packets n Typically measured as the variance of packet interarrival times

Enforcing QoS – Mitigating Packet Loss n The effect of packet loss (a) non interleaved transmission and (b) interleaved transmission.

17 10/8/15

Setting Up a Stream

n Resource reservation in a distributed system n OS must support mechanism n Policy can be implemented at the user level

• Reservations • Statistical (good on average) vs guaranteed (good for important stuff) • No standards, much research, no prevalent results

Synchronization Mechanisms n Synchronization of data and of flows is difficult, mainly due to the unpredictability of computing systems n Interrupts may disturb media continuity n Priority inversion may occur – High priority task blocked by a low priority task n Different flows must be synchronized n Application-level Synchronization n Hardware can be used to help specially when synchronization requirements are strict n Separate streams should be played within 30 μ-seconds

18 10/8/15

Synchronization Mechanisms n Synchronization supported by high-level interfaces. n Middleware multiplexes all substreams into a single stream and demultiplex the streams at the receiver. n Synchronization is handled at multiplexing/ demultiplexing point n Example: MPEG

Conclusion

n Message Oriented Communication n Sockets and Socket API n Message Passing Interface – MPI n Architecture and Design Issues n Message Oriented Persistent Communication n Message Queues n IBM WebSphereMQ n Stream Oriented Communication n Continuous Media Requirements n Flow Specification and QoS Support n Delay and Jitter Control and Reliability

19