4/14/2017

IPC, Threads, Races, Critical Sections Inter- Communication

7A. Inter-Process Communication • the exchange of data between processes 3T. Threads • Goals 7B. The Critical Section Problem – simplicity – convenience – generality – efficiency – security/privacy – robustness and reliability • some of these turn out to be contradictory

IPC, Threads, Races and Critical Sections 1 IPC, Threads, Races and Critical Sections 2

OS Support For IPC Typical IPC Operations • channel creation and destruction Wide range of semantics • • write/send/put – may appear to be another kind of file – insert data into the channel – may involve very different APIs • read/receive/get • provide more powerful semantics – extract data from the channel • more accurately reflect complex realities • channel content query • Connection establishment mediated by the OS – how much data is currently in the channel – to ensure authentication and authorization • connection establishment and query • Data exchange mediated by the OS – control connection of one channel end to another – to protect processes from one-another – who are end-points, what is status of connections – to ensure data integrity and authenticity

IPC, Threads, Races and Critical Sections 3 IPC, Threads, Races and Critical Sections 4

IPC: messages vs streams IPC: flow-control

• streams • queued messages consume system resources – a continuous stream of bytes – buffered in the OS until the receiver asks for them – read or write few or many bytes at a time • many things can increase required buffer space – write and read buffer sizes are unrelated – fast sender, non-responsive receiver – stream may contain app-specific record delimiters • must be a way to limit required buffer space • Messages (aka datagrams) – back-pressure: block sender or refuse message – a sequence of distinct messages – receiver side: drop connection or messages – each message has its own length (subject to limits) – this is usually handled by network protocols – message is typically read/written as a unit • mechanisms to report stifle/flush to sender – delivery of a message is typically all-or-nothing

IPC, Threads, Races and Critical Sections 5 IPC, Threads, Races and Critical Sections 6

1 4/14/2017

IPC: reliability and robustness Simplicity: pipelines • reliable delivery (e.g. TCP vs UDP) – networks can lose requests and responses • data flows through a series of programs • a sent message may not be processed – ls | grep | sort | mail – receiver invalid, dead, or not responding – macro processor | complier | assembler • When do we tell the sender "OK"? • data is a simple byte stream – queued locally? added to receivers input queue? – buffered in the operating system – receiver has read? receiver has acknowledged? – no need for intermediate temporary files • how persistent is system in attempting to deliver? – retransmission, alternate routes, back-up servers, ... • there are no security/privacy/trust issues • do channel/contents survive receiver restarts? – all under control of a single user – can new server instance pick up where the old left off? • error conditions – input: End of File output: SIGPIPE

IPC, Threads, Races and Critical Sections 7 IPC, Threads, Races and Critical Sections 8

Generality: sockets half way: mail boxes, named pipes

• connections between addresses/ports • client/server rendezvous point – connect/listen/accept – a name corresponds to a service – lookup: registry, DNS, service discovery protocols • many data options – a server awaits client connections – reliable (TCP) or best effort data-grams (UDP) – once open, it may be as simple as a pipe – streams, messages, remote procedure calls, … – OS may authenticate message sender • complex flow control and error handling – retransmissions, timeouts, node failures • limited fail-over capability – possibility of reconnection or fail-over – if server dies, another can take its place • trust/security/privacy/integrity – but what about in-progress requests? – we have a whole lecture on this subject • client/server must be on same system

IPC, Threads, Races and Critical Sections 9 IPC, Threads, Races and Critical Sections 10

Ludicrous Speed – Shared Memory IPC: synchronous and asynchronous • synchronous operations • shared read/write memory segments – writes block until message sent/delivered/received – mmap(2) into multiple address spaces – reads block until a new message is available – perhaps locked in physical memory – easy for programmers, but no parallelism – applications maintain circular buffers • asynchronous operations – OS is not involved in data transfer – writes return when system accepts message • no confirmation of transmission/delivery/reception • simplicity, ease of use … your kidding, right? • requires auxiliary mechanism to learn of errors • reliability, security … caveat emptor! – reads return promptly if no message available • requires auxiliary mechanism to learn of new messages • generality … locals only! • often involves "wait for any of these" operation

IPC, Threads, Races and Critical Sections 11 IPC, Threads, Races and Critical Sections 12

2 4/14/2017

a brief history of threads What is a thread?

• processes are very expensive • strictly a unit of execution/scheduling – to create: they own resources – each thread has its own stack, PC, registers – to dispatch: they have address spaces • multiple threads can run in a process • different processes are very distinct – they all share the same code and data space – they cannot share the same address space – they all have access to the same resources – they cannot (usually) share resources – this makes the cheaper to create and run • not all programs require strong separation • sharing the CPU between multiple threads – cooperating parallel threads of execution – user level threads (w/voluntary yielding) – all are trusted, part of a single program – scheduled system threads (w/preemption)

IPC, Threads, Races and Critical Sections 13 IPC, Threads, Races and Critical Sections 14

When to use processes Using Multiple Processes: cc • running multiple distinct programs # shell script to implement the cc command cpp $1. | cc1 | ccopt > $1.s • creation/destruction are rare events as $1.s • running agents with distinct privileges ld /lib/crt0.o $1.o /lib/libc.so • limited interactions and shared resources mv a.out $1 rm $1.s $1.o • prevent interference between processes • firewall one from failures of the other

IPC, Threads, Races and Critical Sections 15 IPC, Threads, Races and Critical Sections 16

When to use threads Using Multiple Threads: telnet

• parallel activities in a single program netfd = get_telnet_connection(host); pthread_create(&tid, NULL, writer, netfd); • frequent creation and destruction reader(netfd); • all can run with same privileges pthread_join(tid, &status); ... • they need to share resources reader( fd ) { int cnt; char buf[100]; while( cnt = read(0, buf, sizeof (buf) > 0 ) • they exchange many messages/signals write(fd, buf, cnt); • no need to protect from each other } writer( fd ) { int cnt; char buf[100]; while( cnt = read(fd, buf, sizeof (buf) > 0 ) write(1, buf, cnt); }

IPC, Threads, Races and Critical Sections 17 IPC, Threads, Races and Critical Sections 18

3 4/14/2017

Kernel vs User-Mode Threads Thread state and thread stacks

• Does OS schedule threads or processes? • each thread has its own registers, PS, PC • Advantages of Kernel implemented threads • each thread must have its own stack area – multiple threads can truly run in parallel • max size specified when thread is created – one thread blocking does not block others – a process can contain many threads – OS can enforce priorities and preemption they cannot all grow towards a single hole – OS can provide atomic sleep/wakeup/signals – • Advantages of library implemented threads – thread creator must know max required stack size – fewer system calls – stack space must be reclaimed when thread exits – faster context switches • procedure linkage conventions are unchanged – ability to tailor semantics to application needs

IPC, Threads, Races and Critical Sections 19 IPC, Threads, Races and Critical Sections 20

UNIX stack space management Thread Stack Allocation

0x00000000 code segment data segment stack segment thread thread thread code data stack 1 stack 2 stack 3

0x00000000 0xFFFFFFFF stack

Data segment starts at page boundary after code segment 0x0120000 Stack segment starts at high end of address space 0xFFFFFFFF Unix extends stack automatically as program needs more.

Data segment grows up; Stack segment grows down Both grow towards the hole in the middle. They are not allowed to meet .

IPC, Threads, Races and Critical Sections 21 IPC, Threads, Races and Critical Sections 22

Thread Safety - Reentrancy – Shared Data/Events

• thread-safe routines must be reentrant • threads operate in a single address space – any routine can be called by multiple threads – automatic (stack) locals are private – concurrent or interspersed execution – signals can also cause reentrancy – storage (from thread-safe malloc) can be private • state cannot be saved in static variables – read-only data causes no problems – e.g. errno … getting around C scalar returns – shared read/write data is a problem – e.g. optarg … implicit session state • transient state can be safely allocated on stack • signals are sent to processes • persistent session state must be client-owned – delivered to first available thread – open returns a descriptor – chosen recipient may not have been expecting it – descriptor is passed to all subsequent operations • a call to exit(2) terminates all threads

IPC, Threads, Races and Critical Sections 23 IPC, Threads, Races and Critical Sections 24

4 4/14/2017

Synchronization - evolution of problem The benefits of parallelism

• batch processing - serially reusable resources • improved throughput – process A has tape drive, process B must wait – blocking of one activity does not stop others – process A updates file first, then process B • improved modularity • cooperating processes – exchanging messages with one-another – separating complex activities into simpler pieces – continuous updates against shared files • improved robustness • shared data and multi-threaded computation – the failure of one thread does not stop others – handlers, symmetric multi-processors • a better fit to emerging paradigms – parallel algorithms, preemptive scheduling – client server computing, web based services • network-scale distributed computing – our universe is cooperating parallel processes

IPC, Threads, Races and Critical Sections 25 IPC, Threads, Races and Critical Sections 26

What's the big deal? Race Conditions • sequential program execution is easy • shared resources and parallel operations – first instruction one, then instruction two, ... – where outcome depends on execution order – execution order is obvious and deterministic – these happen all the time, most don’t matter • independent parallel programs are easy • some race conditions affect correctness – if the parallel streams do not interact in any way – conflicting updates (mutual exclusion) – check/act races (sleep/wakeup problem) • cooperating parallel programs are hard – multi-object updates (all-or-none transactions) – if the two execution streams are not synchronized – distributed decisions based on inconsistent views • results depend on the order of instruction execution each of these classes can be managed • parallelism makes execution order non-deterministic • • results become combinatorially intractable – if we recognize the race condition and danger

IPC, Threads, Races and Critical Sections 27 IPC, Threads, Races and Critical Sections 28

Non-Deterministic Execution What is "Synchronization"

• processes block for I/O or resources • true parallelism is imponderable • time-slice end preemption – pseudo-parallelism may be good enough • identify and serialize key points of interaction • interrupt service routines • there are two interdependent problems • unsynchronized execution on another core – critical section serialization • queuing delays – asynchronous completions • time required to perform I/O operations • message transmission/delivery time • they are often discussed as a single problem – many mechanisms simultaneously solve both – solution to either requires solution to the other

IPC, Threads, Races and Critical Sections 29 IPC, Threads, Races and Critical Sections 30

5 4/14/2017

A Synchronization Problem Problem 1: Critical Sections (multi-thread, shared memory, circular buffer) • a resource shared by multiple threads write(buf, toSend): read(buf, desired): – multiple concurrent threads, processes or CPUs while toSend > 0 while desired > 0 – interrupted code and interrupt handler wait(nextWrite < endOfBuffer) wait (nextWrite > lastRead) free = endOfBuffer – nextWrite avail = nextWrite – lastRead • use of the resource changes its state count = min(free, toSend) count = min(avail, desired) – contents, properties, relation to other resources copy(buf, nextWrite, count) copy(lastRead, buf, count) – updates are non-atomic (or non-global) nextWrite += count lastRead += count • correctness depends on execution order toSend -= count; if lastRead == endOfBuffer lastRead = startOfBuffer – when scheduler runs/preempts which threads nextWrite = startOfBuffer – true (e.g. multi-processor) parallelism desired -= count – relative timing of independent events Critical Section Await Event Signal Event – leading to “ indeterminate” results

IPC, Threads, Races and Critical Sections 31 IPC, Threads, Races and Critical Sections 32

Reentrant & MT-safe code Critical Section - updating a file • consider a simple recursive routine: int factorial(x) { tmp = factorial( x-1 ); return x*tmp} Process #1 Process #2 • consider a possibly multi-threaded routine: remove( "database"); void debit(amt) {tmp = bal-amt; if (tmp >=0) bal = tmp)} fd = create( "database" ); write(fd, newdata, length); neither would work if tmp was shared/static fd = open( "database", READ); • close(fd); count = read(fd, buffer, length); – must be dynamic, each invocation has own copy ... – this is not a problem with read-only information • some variables must be shared – and proper sharing often involves critical sections

IPC, Threads, Races and Critical Sections 33 IPC, Threads, Races and Critical Sections 34

What could go wrong with an add? Achieving Mutual Exclusion

thread #1 thread #2 pthread_mutex_t ; counter = counter + 1; counter = counter + 1; pthread_mutex_init(&lock, NULL); mov counter, %eax add $0x1, %eax … mov counter, %eax if (pthread_mutex_lock(&lock) == 0) { add $0x1, %eax counter = counter + 1; mov %eax, counter pthread_mutex_unlock(&lock); mov %eax, counter }

IPC, Threads, Races and Critical Sections 35 IPC, Threads, Races and Critical Sections 36

6 4/14/2017

Recognizing Critical Sections Critical Sections in Operating System

• generally involves updates to object state • Shared data used by concurrent threads – may be updates to a single object – process state variables – may be related updates to multiple objects – resource pools • generally involves multi-step operations – device driver state – object state inconsistent until operation finishes • logical parallelism – preemption compromises object or operation – created by preemptive scheduling • correct operation requires mutual exclusion – asynchronous – only one thread at a time has access to object(s) • physical parallelism – client 1 completes before client 2 starts – shared memory, symmetric multi-processors

IPC, Threads, Races and Critical Sections 37 IPC, Threads, Races, Critical Sections 38

Two Types of Atomicity Assignments

• Before or After (mutual exclusion) • Project – A enters critical section before B starts – assemble and bring up your Edison – A enters critical section after B completes • Reading • All or None (atomic transactions) – AD 27.3-4 (synchronization APIs) – an update that starts will complete w/o interruption – AD 28 (locks and implementation) – an uncompleted update has no effect – AD 30-30.1 condition variables mutex

IPC, Threads, Races and Critical Sections 39 IPC, Threads, Races and Critical Sections 40

IPC: communication fan-out

• point-to-point/unicast (1->1) – channel carries traffic from one sender to one receiver • multi-cast (1->N) Supplementary Slides – messages are sent to specified receivers or group • broadcast (1->N) – messages are sent to all receivers in a community • publish/subscribe (N->M) – messages are distributed/filtered based on content – routing can be at sender, receiver, and in-between

IPC, Threads, Races and Critical Sections 42

7 4/14/2017

IPC: in-band vs. out-of-band IPC examples: UNIX sockets • more powerful than pipes • in-band messages – can be bound to various protocols – messages delivered in same order as sent • tcp ... reliable stream, network protocol – message n+1 won't be seen till after message n • udp ... unreliable datagrams, network protocol • out-of-band messages • unix ... named pipes – more versatile connection options – messages that leap ahead of queued traffic • connect, listen, accept, broadcast, multicast • often used to announce errors or cancel requests • both stream and message semantics – use priority to “cut” ahead in the queue – read/write ... synchronous stream • priority must be honored on each link in the path – send/recv ... synchronous datagrams – deliver them over a separate channel • a separate message channel, or perhaps a • socket is destroyed when creator dies

IPC, Threads, Races and Critical Sections 43 IPC, Threads, Races and Critical Sections 44

IPC examples: mail boxes Synchronization – a Parable • named message queues • consider the “Garden of Eden” – we were warned of the “knowledge of good and evil” – associated with a particular receiving process • which would forever end our innocent lives in paradise – any process can send messages to any mailbox – but we were then tempted to eat that fruit • additional semantics vary with implementations • it would open our eyes, and give us God-like knowledge – we ate, lost our innocence, were banished from paradise – trusted identification of sending process • consider the “Garden of computation” – synchronous and asynchronous options – we were warned about cooperating parallel processes – confirmation of delivery (or receipt) • which would forever cost us our innocence – contents of queue may survive a kill and restart • but we also saw the power that knowledge could unleash – again we took the bait, and lost our innocence • messages typically buffered in the OS • programming, once simple, has been made impossibly difficult – some flow control is usually provided

IPC, Threads, Races and Critical Sections 45 IPC, Threads, Races and Critical Sections 46

8