Concurrency Welcome to Cs378

Parallel Systems Events and Futures Chris Rossbach + Calvin Lin CS380p Outline for Today

• Asynchronous Programming Models • Events • Futures

CS380P Events+Futures 2 Review: Parallel Programming Models

CS380P Processes + Threads 3 Review: Parallel Programming Models

• Concrete model: • CPU(s) execute instructions sequentially

CS380P Processes + Threads 3 Review: Parallel Programming Models

• Concrete model: • CPU(s) execute instructions sequentially • Dimensions: • How to specify computation • How to specify communication • How to specify coordination/control transfer

CS380P Processes + Threads 3 Review: Parallel Programming Models

• Concrete model: • CPU(s) execute instructions sequentially • Dimensions: • How to specify computation • How to specify communication • How to specify coordination/control transfer • Techniques/primitives • Threads/Processes • Message passing vs shared memory • Preemption vs Non-preemption

CS380P Processes + Threads 3 Review: Parallel Programming Models

CS380P Processes + Threads 3 Review: Execution Context Management “Task” == “Flow of Control” “Stack” == Task State

CS380P Processes + Threads 4 Review: Execution Context Management “Task” == “Flow of Control” “Stack” == Task State Task Management

CS380P Processes + Threads 4 Review: Execution Context Management “Task” == “Flow of Control” “Stack” == Task State Task Management • Preemptive • Interleave on uniprocessor • Overlap on multiprocessor • Serial • One at a time, no conflict

CS380P Processes + Threads 4 Review: Execution Context Management “Task” == “Flow of Control” “Stack” == Task State Task Management Stack Management • Preemptive • Manual • Interleave on uniprocessor • Inherent in Cooperative • Overlap on multiprocessor • Changing at quiescent points • Serial • Automatic • One at a time, no conflict • Inherent in pre-emptive • Cooperative • Downside: Hidden concurrency assumptions • Yields at well-defined points • E.g. wait for long-running I/O

CS380P Processes + Threads 4 UI Programming

CS380P Events+Futures 5 UI Programming

CS380P Events+Futures 6 UI Programming

CS380P Events+Futures 6 UI programming

CS380P Events+Futures 7 Over 1000 last UI programming time I checked!

void OnMove() { … } void OnSize() { … }

void OnPaint() { … }

CS380P Events+Futures 7 UI Programming Distilled

CS380P Events+Futures 8 UI Programming Distilled

Pros

CS380P Events+Futures 8 UI Programming Distilled

Pros • Simple imperative programming

CS380P Events+Futures 8 UI Programming Distilled

Pros • Simple imperative programming • Good fit for uni-processor CS380P Events+Futures 8 UI Programming Distilled

Pros Cons • Simple imperative programming • Good fit for uni-processor CS380P Events+Futures 8 UI Programming Distilled

Pros Cons • Simple imperative programming •Awkward/verbose • Good fit for uni-processor CS380P Events+Futures 8 UI Programming Distilled

Pros Cons • Simple imperative programming •Awkward/verbose • Good fit for uni-processor •Obscures available parallelism CS380P Events+Futures 8 UI Programming Distilled

CS380P Events+Futures 9 UI Programming Distilled How can we parallelize this?

CS380P Events+Futures 9 Parallel UI Implementation 1

CS380P Events+Futures 10 Parallel UI Implementation 1

DoThisProc

DoThatProc

OtherThing

CS380P Events+Futures 11 Parallel UI Implementation 1

DoThisProc

DoThatProc Pros/cons?

OtherThing

CS380P Events+Futures 11 Parallel UI Implementation 1

DoThisProc

DoThatProc Pros/cons?

Pros: • Encapsulates parallel work Cons: OtherThing • Obliterates original code structure • How to assign handlers→CPUs? • Load balance?!? • Utilization

CS380P Events+Futures 11 Parallel GUI Implementation 2

CS380P Events+Futures 12 Pros/cons? Parallel GUI Implementation 2

CS380P Events+Futures 12 Pros/cons?

Parallel GUI Implementation 2 Pros: • Preserves programming model • Can recover some parallelism Cons: • Workers still have same problem • How to load balance? • Shared mutable state a problem

CS380P Events+Futures 12 Pros/cons?

Extremely difficult to solve without changing the whole programming model…so

CS380P Events+Futures change it 12 Event-based Programming: Motivation

CS380P Events+Futures 13 Event-based Programming: Motivation

• Threads have a *lot* of down-sides: • Tuning parallelism for different environments • Load balancing/assignment brittle • Shared state requires locks → • Priority inversion • Deadlock • Incorrect synchronization • …

CS380P Events+Futures 13 Event-based Programming: Motivation

• Events: restructure programming model to have no threads!

CS380P Events+Futures 13 Event Programming Model Basics

CS380P Events+Futures 14 Event Programming Model Basics

• Programmer only writes events

CS380P Events+Futures 14 Event Programming Model Basics

• Programmer only writes events • Event: an object queued for a module (think future/promise)

CS380P Events+Futures 14 Event Programming Model Basics

• Programmer only writes events • Event: an object queued for a module (think future/promise) • Basic primitives • create_event_queue(handler) → event_q • enqueue_event(event_q, event-object) • Invokes handler (eventually)

CS380P Events+Futures 14 Event Programming Model Basics

CS380P Events+Futures 14 Event-based programming

CS380P Events+Futures 15 Event-based programming

Runtime

CS380P Events+Futures 15 Event-based programming

Runtime

CS380P Events+Futures 15 Event-based programming

Runtime

Is the problem solved? CS380P Events+Futures 15 Another Event-based Program

CS380P Events+Futures 16 Another Event-based Program

Blocks!

CS380P Events+Futures 16 Another Event-based Program

Burns CPU! Blocks!

CS380P Events+Futures 16 Another Event-based Program

Uses Other Handlers! Burns CPU! Blocks! (call OnPaint?) CS380P Events+Futures 16 No problem! Just use more events/handlers, right?

CS380P Events+Futures 17 Continuations, BTW

CS380P Events+Futures 18 Stack-Ripping

CS380P Events+Futures 19 Stack-Ripping

Stack-based state out-of-scope! Requests must carry state

CS380P Events+Futures 19 Threads vs Events

• Thread Pros • Event Pros • Overlap I/O and computation • Easier to create well-conditioned system • Easier to express dynamic change in level of • While looking sequential parallelism • Intermediate state on stack • Control flow naturally expressed • Thread Cons • Event Cons • Synchronization required • Difficult to program • Overflowable stack • Control flow between callbacks obscure • Stack memory pressure • When to deallocate memory • Incomplete language/tool/debugger support • Difficult to exploit concurrent hardware

CS380P Events+Futures 20 Threads vs Events

• Thread Pros • Event Pros • Overlap I/O and computation • Easier to create well-conditioned system • Easier to express dynamic change in level of • While looking sequential parallelism • Intermediate state on stack • Control flow naturally expressed Language-level • Thread Cons Futures: •theEvent Cons • Synchronization required sweet spot?• Difficult to program • Overflowable stack • Control flow between callbacks obscure • Stack memory pressure • When to deallocate memory • Incomplete language/tool/debugger support • Difficult to exploit concurrent hardware

CS380P Events+Futures 20 Threads vs Events

• Thread Pros • Event Pros • Overlap I/O and computation • Easier to create well-conditioned system • Easier to express dynamic change in level of • While looking sequential parallelism • Intermediate state on stack • Control flow naturally expressed Language-level • Thread Cons Futures: •theEvent Cons • Synchronization required sweet spot?• Difficult to program • Overflowable stack • Control flow between callbacks obscure • Stack memory pressure • When to deallocate memory • Incomplete language/tool/debugger support • Difficult to exploit concurrent hardware

CS380P Events+Futures 20 Futures & Promises

CS380P Events+Futures 21 Futures & Promises

• Values that will eventually become available

CS380P Events+Futures 21 Futures & Promises

• Values that will eventually become available • Time-dependent states: • Completed/determined • Computation complete, value concrete • Incomplete/undetermined • Computation not complete yet

CS380P Events+Futures 21 Futures & Promises

CS380P Events+Futures 21 Java Example

CS380P Events+Futures 22 Java Example

• CompletableFuture is a container for Future object type

CS380P Events+Futures 22 Java Example

• CompletableFuture is a container for Future object type • cf is an instance

CS380P Events+Futures 22 Java Example

• CompletableFuture is a container for Future object type • cf is an instance • runAsync() accepts • Lambda expression • Anonymous function • Functor

CS380P Events+Futures 22 Java Example

• CompletableFuture is a container for Future object type • cf is an instance • runAsync() accepts • Lambda expression • Anonymous function • Functor • runAsync() immediately returns a waitable object (cf) • Where (on what thread) does the lambda expression run? CS380P Events+Futures 22 Futures and Promises: Why two kinds of objects?

CS380P Events+Futures 23 Futures and Promises: Why two kinds of objects? Promise: “thing to be done”

Future: encapsulation (something to give caller)

CS380P Events+Futures 23 Futures and Promises: Why two kinds of objects? Promise: “thing to be done”

Future: encapsulation (something to give caller)

CS380P Promise Events+Futuresto do something in the future 23 Futures vs Promises

CS380P Events+Futures 24 Futures vs Promises

• Future: read-only reference to uncompleted value • Promise: single-assignment variable that the future refers to • Promises complete the future with: • Result with success/failure • Exception Language Promise Future Algol Thunk Address of async result Java CompletableFuture Future

C#/.NET TaskCompletionSource Task

JavaScript Deferred Promise CS380P C++ std::promise Events+Futures std::future 24 Futures vs Promises

C#/.NET TaskCompletionSource Task

JavaScript Deferred Promise CS380P C++ std::promise Events+Futures std::future 24 Mnemonic: Promise to do something Futures vs Promises Make a promise for the future

C#/.NET TaskCompletionSource Task

JavaScript Deferred Promise CS380P C++ std::promise Events+Futures std::future 24 Putting Futures in Context My unvarnished opinion

CS380P Events+Futures 25 Putting Futures in Context My unvarnished opinion Futures:

CS380P Events+Futures 25 Putting Futures in Context My unvarnished opinion Futures: • abstraction for concurrent work supported by • Compiler: abstractions are language-level objects • Runtime: scheduler, task queues, thread-pools are transparent • Programming remains mostly imperative • Threads of control peppered with asynchronous/concurrent tasks

CS380P Events+Futures 25 Dataflow: a better abstraction?

Matrix: A Matrix: B

gemm Matrix: C

gemm

• nodes → computation • edges → communication • Expresses parallelism explicitly • Minimal specification of data movement: runtime does it. • asynchrony is a runtime concern (not programmer concern) • No specification of compute→device mapping: like threads! CS380P Events+Futures 26