15-213 Lectures 22 and 23: Synchronization

Carnegie Mellon Carnegie Mellon Introduction

Last Time:
Threads Introduction to Computer Systems
Concepts 15-213/18-243, fall 2009
Synchronization with Semaphores 23 rd Lecture, Nov 19
Today: Synchronization in Greater Depth
Producer Consumer
Buffer Pools and Condition Variables Instructors:
Message Queues Roger B. Dannenberg and Greg Ganger
Message Passing

2

Carnegie Mellon Carnegie Mellon Notifying With Semaphores Producer-Consumer on a Buffer That Holds One Item

producer shared consumer /* buf1.c - producer-consumer int main() { thread buffer thread on 1-element buffer */ pthread_t tid_producer; #include “csapp.h” pthread_t tid_consumer;

#define NITERS 5 /* initialize the semaphores */
Common synchronization pattern: Sem_init(&shared.empty, 0, 1);
Producer waits for slot, inserts item in buffer, and notifies consumer void *producer(void *arg); Sem_init(&shared.full, 0, 0);
Consumer waits for item, removes it from buffer, and notifies producer void *consumer(void *arg); /* create threads and wait */ struct { Pthread_create(&tid_producer, NULL, int buf; /* shared var */ producer, NULL);
Examples sem_t full; /* sems */ Pthread_create(&tid_consumer, NULL,
Multimedia processing: sem_t empty; consumer, NULL); } shared; Pthread_join(tid_producer, NULL);
Producer creates MPEG video frames, consumer renders them Pthread_join(tid_consumer, NULL);
Event-driven graphical user interfaces
Producer detects mouse clicks, mouse movements, and keyboard hits exit(0); and inserts corresponding events in buffer }
Consumer retrieves events from buffer and paints the display 3 4

Carnegie Mellon Carnegie Mellon Producer-Consumer (cont) Counting with Semaphores Initially: empty = 1, full = 0
Remember, it’s a non-negative integer

/* producer thread */ /* consumer thread */
So, values greater than 1 are legal void *producer(void *arg) { void *consumer(void *arg) {
Lets repeat thing_5() 5 times for every 3 of thing_3() int i, item; int i, item; /* thing_5 and thing_3 */ int main() { for (i=0; i

Carnegie Mellon Carnegie Mellon Counting with semaphores (cont) Producer-Consumer, Circular Buffer Initially: five = 5, three = 3

/* thing_5() thread */ /* thing_3() thread */ /* bufn.c - producer-consumer int main() { void *five_times(void *arg) { void *three_times(void *arg) { on n-element buffer */ pthread_t tid_producer; int i; int i; #include “csapp.h” pthread_t tid_consumer;

while (1) { while (1) { #define NITERS 100 /* initialize the semaphores */ for (i=0; i<5; i++) { for (i=0; i<3; i++) { #define N 20 /* buffer size */ Sem_init(&shared.empty, 0, N); /* wait & thing_5() */ /* wait & thing_3() */ Sem_init(&shared.full, 0, 0); P(&five); P(&three); void *producer(void *arg); thing_5(); thing_3(); void *consumer(void *arg); /* create threads and wait */ } } Pthread_create(&tid_producer, NULL, V(&three); V(&five); struct { producer, NULL); V(&three); V(&five); int buf [N] ; /* shared var */ Pthread_create(&tid_consumer, NULL, V(&three); V(&five); sem_t full; /* sems */ consumer, NULL); } V(&five); sem_t empty; Pthread_join(tid_producer, NULL); return NULL; V(&five); } shared; Pthread_join(tid_consumer, NULL); } } return NULL; exit(0); } }

7 8

Carnegie Mellon Carnegie Mellon Producer-Consumer, Circular Buffer (cont) Do You Need Semaphores? Initially, shared.full = 0, shared.empty = N /* data structure */ /* producer */ /* producer thread */ /* consumer thread */ int shared; while (shared != 0) ; void *producer(void *arg) { void *consumer(void *arg) { shared = any_non_zero_data; int i, item; int i, item; /* initially */ shared = 0; for (i=0; i

/* write item to buf */ /* consume item */ P(&shared.empty); printf("consumed %d\n“, item);
Notes: shared.buf [i%N] = item; } V(&shared.full); return NULL;
Producer notifies Consumer by writing non-zero } }
Consumer notifies Producer by writing zero return NULL; }
Don’t try this at home! (loops, hot-spot)
This is just a mental warm-up

9 10

Carnegie Mellon Carnegie Mellon Do You Need Semaphores? (cont) Sharing With Dependencies

/* data structure */ /* producer */
Threads interact through a resource manager #define N 20 int i = (shared.tail + 1) % N; struct { if (i == shared.head) return FAIL;
(Standard terminology: monitor ) int head; shared.buf[i] = produced_data;
One thread may need to wait for something another int tail; tail = i; /* atomic update */ int buf[N]; thread provides } shared; /* consumer */ if (shared.tail == shared.head)
Example 1: previous circular buffer /* initiallize */ return FAIL;
But that had a simple semaphore solution shared.head = 0; data_to_consume = shared.buf[shared.head]; Shared.tail = 0; /* atomic update: */
Example 2: memory buffer pool shared.head = (shared.head + 1) % N;
Multiple threads checking out and returning buffers
Notes:
Might have to wait for buffer of the right size to show up
Producer notifies Consumer by updating tail
Consumer notifies Producer by updating head
Data copied in before notifying consumer
Ultimately, our solution will be: mutex and condition
Data copied out before notifying producer variable

What about out-of-order writes? 11 12 15-213 Lectures 22 and 23: Synchronization

Carnegie Mellon Carnegie Mellon Naïve Solution What If Resource Is Unavailable?

/* protected access to buffer structures */ /* protected access to buffer structures */ mutex_t mutex; mutex_t mutex; Mutex_init(&mutex); Mutex_init(&mutex);

/* get a buffer */ /* get a buffer */ Mutex_lock(&mutex); Mutex_lock(&mutex); Find a suitable buffer while (! Find_a_suitable_buffer ()) { Mutex_unlock(&mutex); Mutex_unlock(); Mutex_lock(); } /* return a buffer */ Reserve the suitable buffer Mutex_lock(&mutex); Mutex_unlock(&mutex); Return buffer to pool Mutex_unlock(&mutex); /* return a buffer */ Mutex_lock(&mutex); Return buffer to pool
Notes: Mutex_unlock(&mutex);
mutex_lock is functionally similar to P()
mutex_unlock is similar to V()
Bad: spins rather than yielding processor to threads
Analog of Mutex_init() is initializing semaphore to 1 holding the very resources we are waiting for!

13 14

Carnegie Mellon Carnegie Mellon What If Resource Is Unavailable? (cont) How About Waiting for a Notification?

/* protected access to buffer structures */ /* protected access to buffer structures */ mutex_t mutex; mutex_t mutex; Mutex_init(&mutex); Mutex_init(&mutex); int waiting = FALSE; /* get a buffer */ sem_t rsrc; Mutex_lock(&mutex); Sem_init(&rsrc, 0, 0); while (! Find_a_suitable_buffer ()) { Mutex_unlock(); Yield(); Mutex_lock(); /* get a buffer */ } Mutex_lock(&mutex); Reserve the suitable buffer while (! Find_a_suitable_buffer ()) { Mutex_unlock(&mutex); waiting = TRUE; Mutex_unlock(&mutex); P(&rsrc); Mutex_lock(&mutex); /* return a buffer */ } Mutex_lock(&mutex); Reserve the suitable buffer Return buffer to pool Mutex_unlock(&mutex); Mutex_unlock(&mutex); /* return a buffer */ Mutex_lock(&mutex);
Better, but not really acceptable. Polls until buffer is Return
Better, buffer but to not pool really acceptable. Polls until buffer is available. if available.(waiting) { waiting = FALSE; V(&rsrc); } Mutex_unlock(&mutex); 15 16

Carnegie Mellon Carnegie Mellon How About Waiting for a Notification? Condition Variable

/* protected access to buffer structures */ mutex_t mutex;
Yet another synchronization mechanism Mutex_init(&mutex);
Not a semaphore, not a mutex int waiting = FALSE; sem_t rsrc;
Not really a “variable” Problems:Sem_init(&rsrc, 0, 0);
Essentially a queue of waiting/sleeping/suspended threads • This will not work with multiple waiting threads /* get a buffer */
Operations: Mutex_lock(&mutex);• waiting is only a Boolean while (! Find_a_suitable_buffer ()) {
Cond_wait puts thread on the queue • Hardwaiting to =analyze TRUE;
Cond_signal wakes up one thread on the queue (if any) Mutex_unlock(&mutex); P(&rsrc); Mutex_lock(&mutex); } • Need a more general approach
Cond_broadcast wakes up all threads on the queue Reserve the suitable buffer
Special feature of Condition Variables: Mutex_unlock(&mutex);• Special-case “hacks” invite obscure bugs
Cond_wait atomically puts thread on queue and releases a mutex lock /* return a buffer */
Waking up automatically reacquires the lock (!) Mutex_lock(&mutex); Return
Better, buffer but to not pool really acceptable. Polls until buffer is if available.(waiting) { waiting = FALSE; V(&rsrc); } Mutex_unlock(&mutex); 17 18 15-213 Lectures 22 and 23: Synchronization

Carnegie Mellon Carnegie Mellon Sharing with Condition Variable Another Problem: Readers and Writers

/* protected access to buffer structures */ mutex_t mutex;
This is a general pattern:
Consider multiple readers and multiple writers of some Mutex_init(&mutex); enter mutex, shared data. cond_t bufcv; loop testing for what you need Cond_init(&bufcv); to proceed,
Writers must access the data exclusively (no other readers or writers at the same time) /* get a buffer */ cond_wait() if you cannot, Mutex_lock(&mutex); finally finish up and leave
Readers must exclude writers, but multiple readers can while (! Find_a_suitable_buffer ()) { mutex; Cond_wait(&condcv, &mutex); read at the same time }
Other operations within
How would you implement Readers and Writers? Reserve the suitable buffer the monitor always: Mutex_unlock(&mutex); cond_signal or cond_broadcast
How would you give priority to Writers? /* return a buffer */ if they possibly enable a
How would you prevent Writers from preventing Readers Mutex_lock(&mutex); blocked thread Return buffer to pool from (ever) making progress? Cond_broadcast(&condcv);
Note: posix allows spurious Mutex_unlock(&mutex); wakeups to occur, so always retest condition in a loop

19 20

Carnegie Mellon Carnegie Mellon Synchronization and Message Passing Message Passing Implementation

Advantages of threads over processes seem to require
Many implementations are possible

shared memory (and all the associated problems)
Rare to see primitives in languages or systems

There’s no requirement that threads share variables
Usually built using synchronization primitives

Threads can communicate through messages even in a
Design Issues: shared address space
Do you send actual message data or just a pointer to it?
In fact, shared address space allows for lightweight
Data structures to use for messages and queues message passing
How to name recipients of messages

Let’s consider:
How to implement message passing
Solutions to some common synchronization problems

Note: message passing is not covered in the textbook

21 22

Carnegie Mellon Carnegie Mellon Simple Message Implementation Note About Message Types

Messages are send to and received from a mailbox
We can’t really work with messages of type void * .

Mailbox is just a linked list of pointers to messages
Typically use something like this:

Implementation (interim version, no synchronization): enum Msg_type {start, stop, task1, task2 }; /* assume Queue datatype */ mbox_t my_mb; typedef struct { queue_t q; } mbox_t; typedef struct { /* sending a message */ enum Msg_type tag; void mb_init(mbox_t *mb) { void *msg = malloc(MSG_SIZE); union { queue_init(&(mb->q)); } … fill in msg with data … struct { … } start_data; mb_snd(&my_mb, msg); struct { … } stop_data; void mb_snd(mbox_t *mb, void *msg) { … do not free msg! … struct { … } task1_data; mb->q.enqueue(msg); } struct { … } task2_data; /* receiving a message */ } void *mb_rcv(mbox_t *mb) { void *msg = mb_rcv(&my_mb); } Message; if (mb->q.empty()) return NULL; if (msg) { return mb->q.dequeue(); } … use data in *msg … free(msg); int mb_empty(mbox_t *mb) { } return mb->q.empty(); } 23 24 15-213 Lectures 22 and 23: Synchronization

Carnegie Mellon Carnegie Mellon Synchronization Synchronization (cont)

Problem 1: Shared access to Mailboxes (and Queues)
Problem 2: Waiting for a message: rewrite with condition var /* assume Queue datatype */ /* assume Queue datatype */ typedef struct { Queue q; void *mb_rcv(mbox_t *mb) { typedef struct { Queue q; void *mb_rcv(mbox_t *mb) { sem_t s; } mbox_t; void *m; mutex_t s; void *m; P(&(mb->s)); cond_t rdy;} mbox_t; mutex_lock(&(mb->s)); if (mb->q.empty()) { while (mb->q.empty()) { void mb_init(mbox_t *mb) { V(&(mb->s)); cond_wait(&(mb->rdy), queue_init(&(mb->q)); return NULL; void mb_init(mbox_t *mb) { &(mb->s)); Sem_init(&(mb->s, 0, 1)); } } queue_init(&(mb->q)); } m = mb->q.dequeue(); Mutex_init(&(mb->s)); m = mb->q.dequeue(); V(&(mb->s)); Cond_init(&(mb->rdy)); } mutex_unlock(&(mb->s)); void mb_snd(mbox_t *mb, void *msg) { } } P(&(mb->s)); mb->q.enqueue(msg); int mb_empty(mbox_t *mb) { void mb_snd(mbox_t *mb, void *msg) { int mb_empty(mbox_t *mb) { V(&(mb->s)); } int empty; mutex_lock(&(mb->s)); int empty; P(&(mb->s)); mb->q.enqueue(msg); mutex_lock(&(mb->s)); empty = mb->q.empty(); cond_signal(&(mb->rdy)); empty = mb->q.empty(); V(&(mb->s)); mutex_unlock(&(mb->s)); } mutex_unlock(&(mb->s)); return empty; } return empty; }

25 26

Carnegie Mellon Carnegie Mellon Example Message Passing Application Beware of Optimizing Compilers!

Consider an Audio Player Code From Book
User interface #define NITERS 100000000 Generated Code
sends filename, EQ, volume, UI /* shared counter variable */ movl cnt, %ecx position to audio thread unsigned int cnt = 0; movl $99999999, %eax
displays song pos. and spectrum .L6: /* thread routine */ leal 1(%ecx), %edx
Audio thread void *count(void *arg) decl %eax
reads, decodes, plays audio Audio { movl %edx, %ecx int i; jns .L6
computes spectrum data for (i = 0; i < NITERS; i++) movl %edx, cnt
Possible implementation cnt++; return NULL;
2 mailboxes: one for UI, one for Audio thread
Compiler moved access to cnt } out of loop
Each thread: check for msgs, do work, repeat
Global variable cnt shared
Only shared accesses to cnt
No shared variables except for mailboxes: between threads occur before loop (read) or after
No shared access to screen
Multiple threads could be trying (write)
Display updates unlikely to block time-critical to update within their iterations
What are possible program audio thread outcomes?

27 28

Carnegie Mellon Carnegie Mellon Controlling Optimizing Compilers! Threads Summary Revised Book Code #define NITERS 100000000
Threads provide another mechanism for writing Generated Code concurrent programs /* shared counter variable */ movl $99999999, %edx volatile unsigned int cnt = 0; .L15:
Threads are very popular movl cnt, %eax /* thread routine */ incl %eax
Somewhat cheaper than processes void *count(void *arg) decl %edx
Easy to share data between threads { movl %eax, cnt int i; jns .L15
Make use of multiple cores for parallel algorithms for (i = 0; i < NITERS; i++) cnt++;
However, the ease of sharing has a cost: return NULL;
Easy to introduce subtle synchronization errors }
Tread carefully with threads!
Declaring variable as volatile
Shared variable read and written forces it to be kept in memory each iteration
For more info:
D. Butenhof, “Programming with Posix Threads”, Addison-Wesley, 1997

29 30