Parallel Systems Events and Futures Chris Rossbach + Calvin Lin CS380p Outline for Today • Asynchronous Programming Models • Events • Futures CS380P Events+Futures 2 Review: Parallel Programming Models CS380P Processes + Threads 3 Review: Parallel Programming Models • Concrete model: • CPU(s) execute instructions sequentially CS380P Processes + Threads 3 Review: Parallel Programming Models • Concrete model: • CPU(s) execute instructions sequentially • Dimensions: • How to specify computation • How to specify communication • How to specify coordination/control transfer CS380P Processes + Threads 3 Review: Parallel Programming Models • Concrete model: • CPU(s) execute instructions sequentially • Dimensions: • How to specify computation • How to specify communication • How to specify coordination/control transfer • Techniques/primitives • Threads/Processes • Message passing vs shared memory • Preemption vs Non-preemption CS380P Processes + Threads 3 Review: Parallel Programming Models • Concrete model: • CPU(s) execute instructions sequentially • Dimensions: • How to specify computation • How to specify communication • How to specify coordination/control transfer • Techniques/primitives • Threads/Processes • Message passing vs shared memory • Preemption vs Non-preemption • Dimensions/techniques not always orthogonal CS380P Processes + Threads 3 Review: Execution Context Management “Task” == “Flow of Control” “Stack” == Task State CS380P Processes + Threads 4 Review: Execution Context Management “Task” == “Flow of Control” “Stack” == Task State Task Management CS380P Processes + Threads 4 Review: Execution Context Management “Task” == “Flow of Control” “Stack” == Task State Task Management • Preemptive • Interleave on uniprocessor • Overlap on multiprocessor CS380P Processes + Threads 4 Review: Execution Context Management “Task” == “Flow of Control” “Stack” == Task State Task Management • Preemptive • Interleave on uniprocessor • Overlap on multiprocessor • Serial • One at a time, no conflict CS380P Processes + Threads 4 Review: Execution Context Management “Task” == “Flow of Control” “Stack” == Task State Task Management • Preemptive • Interleave on uniprocessor • Overlap on multiprocessor • Serial • One at a time, no conflict • Cooperative • Yields at well-defined points • E.g. wait for long-running I/O CS380P Processes + Threads 4 Review: Execution Context Management “Task” == “Flow of Control” “Stack” == Task State Task Management Stack Management • Preemptive • Manual • Interleave on uniprocessor • Inherent in Cooperative • Overlap on multiprocessor • Changing at quiescent points • Serial • Automatic • One at a time, no conflict • Inherent in pre-emptive • Cooperative • Downside: Hidden concurrency assumptions • Yields at well-defined points • E.g. wait for long-running I/O CS380P Processes + Threads 4 UI Programming CS380P Events+Futures 5 UI Programming CS380P Events+Futures 5 UI Programming CS380P Events+Futures 6 UI Programming CS380P Events+Futures 6 UI Programming CS380P Events+Futures 6 UI Programming CS380P Events+Futures 6 UI Programming CS380P Events+Futures 6 UI programming CS380P Events+Futures 7 Over 1000 last UI programming time I checked! CS380P Events+Futures 7 Over 1000 last UI programming time I checked! void OnMove() { … } void OnSize() { … } void OnPaint() { … } CS380P Events+Futures 7 UI Programming Distilled CS380P Events+Futures 8 UI Programming Distilled Pros CS380P Events+Futures 8 UI Programming Distilled Pros • Simple imperative programming CS380P Events+Futures 8 UI Programming Distilled Pros • Simple imperative programming • Good fit for uni-processor CS380P Events+Futures 8 UI Programming Distilled Pros Cons • Simple imperative programming • Good fit for uni-processor CS380P Events+Futures 8 UI Programming Distilled Pros Cons • Simple imperative programming •Awkward/verbose • Good fit for uni-processor CS380P Events+Futures 8 UI Programming Distilled Pros Cons • Simple imperative programming •Awkward/verbose • Good fit for uni-processor •Obscures available parallelism CS380P Events+Futures 8 UI Programming Distilled Pros Cons • Simple imperative programming •Awkward/verbose • Good fit for uni-processor •Obscures available parallelism CS380P Events+Futures 8 UI Programming Distilled CS380P Events+Futures 9 UI Programming Distilled How can we parallelize this? CS380P Events+Futures 9 Parallel UI Implementation 1 CS380P Events+Futures 10 Parallel UI Implementation 1 CS380P Events+Futures 10 Parallel UI Implementation 1 CS380P Events+Futures 10 Parallel UI Implementation 1 CS380P Events+Futures 10 Parallel UI Implementation 1 DoThisProc DoThatProc OtherThing CS380P Events+Futures 11 Parallel UI Implementation 1 DoThisProc DoThatProc Pros/cons? OtherThing CS380P Events+Futures 11 Parallel UI Implementation 1 DoThisProc DoThatProc Pros/cons? Pros: • Encapsulates parallel work Cons: OtherThing • Obliterates original code structure • How to assign handlers→CPUs? • Load balance?!? • Utilization CS380P Events+Futures 11 Parallel GUI Implementation 2 CS380P Events+Futures 12 Pros/cons? Parallel GUI Implementation 2 CS380P Events+Futures 12 Pros/cons? Parallel GUI Implementation 2 Pros: • Preserves programming model • Can recover some parallelism Cons: • Workers still have same problem • How to load balance? • Shared mutable state a problem CS380P Events+Futures 12 Pros/cons? Parallel GUI Implementation 2 Pros: • Preserves programming model • Can recover some parallelism Cons: • Workers still have same problem • How to load balance? • Shared mutable state a problem Extremely difficult to solve without changing the whole programming model…so CS380P Events+Futures change it 12 Event-based Programming: Motivation CS380P Events+Futures 13 Event-based Programming: Motivation • Threads have a *lot* of down-sides: • Tuning parallelism for different environments • Load balancing/assignment brittle • Shared state requires locks → • Priority inversion • Deadlock • Incorrect synchronization • … CS380P Events+Futures 13 Event-based Programming: Motivation • Threads have a *lot* of down-sides: • Tuning parallelism for different environments • Load balancing/assignment brittle • Shared state requires locks → • Priority inversion • Deadlock • Incorrect synchronization • … • Events: restructure programming model to have no threads! CS380P Events+Futures 13 Event Programming Model Basics CS380P Events+Futures 14 Event Programming Model Basics • Programmer only writes events CS380P Events+Futures 14 Event Programming Model Basics • Programmer only writes events • Event: an object queued for a module (think future/promise) CS380P Events+Futures 14 Event Programming Model Basics • Programmer only writes events • Event: an object queued for a module (think future/promise) • Basic primitives • create_event_queue(handler) → event_q • enqueue_event(event_q, event-object) • Invokes handler (eventually) CS380P Events+Futures 14 Event Programming Model Basics • Programmer only writes events • Event: an object queued for a module (think future/promise) • Basic primitives • create_event_queue(handler) → event_q • enqueue_event(event_q, event-object) • Invokes handler (eventually) • Scheduler decides which event to execute next • E.g. based on priority, CPU usage, etc. CS380P Events+Futures 14 Event-based programming CS380P Events+Futures 15 Event-based programming CS380P Events+Futures 15 Event-based programming CS380P Events+Futures 15 Event-based programming CS380P Events+Futures 15 Event-based programming Runtime CS380P Events+Futures 15 Event-based programming Runtime CS380P Events+Futures 15 Event-based programming Runtime Is the problem solved? CS380P Events+Futures 15 Another Event-based Program CS380P Events+Futures 16 Another Event-based Program CS380P Events+Futures 16 Another Event-based Program Blocks! CS380P Events+Futures 16 Another Event-based Program Burns CPU! Blocks! CS380P Events+Futures 16 Another Event-based Program Uses Other Handlers! Burns CPU! Blocks! (call OnPaint?) CS380P Events+Futures 16 No problem! Just use more events/handlers, right? CS380P Events+Futures 17 Continuations, BTW CS380P Events+Futures 18 Stack-Ripping CS380P Events+Futures 19 Stack-Ripping CS380P Events+Futures 19 Stack-Ripping CS380P Events+Futures 19 Stack-Ripping Stack-based state out-of-scope! Requests must carry state CS380P Events+Futures 19 Threads vs Events • Thread Pros • Event Pros • Overlap I/O and computation • Easier to create well-conditioned system • Easier to express dynamic change in level of • While looking sequential parallelism • Intermediate state on stack • Control flow naturally expressed • Thread Cons • Event Cons • Synchronization required • Difficult to program • Overflowable stack • Control flow between callbacks obscure • Stack memory pressure • When to deallocate memory • Incomplete language/tool/debugger support • Difficult to exploit concurrent hardware CS380P Events+Futures 20 Threads vs Events • Thread Pros • Event Pros • Overlap I/O and computation • Easier to create well-conditioned system • Easier to express dynamic change in level of • While looking sequential parallelism • Intermediate state on stack • Control flow naturally expressed Language-level • Thread Cons Futures: •theEvent Cons • Synchronization required sweet spot?• Difficult to program • Overflowable stack • Control flow between callbacks obscure • Stack memory pressure • When to deallocate memory • Incomplete language/tool/debugger support • Difficult to exploit concurrent hardware CS380P Events+Futures 20 Threads vs Events • Thread