Threads ICS332 — Operating Systems

Spring 2018

Henri Casanova ([email protected]) Threads Concurrent computing: several computations are performed during overlapping time periods As opposed to sequential execution, which is one computation to completion, followed by one computation to completion, etc.

Concurrency: A feature of a program that can do multiple things “at the same time”

A program is concurrent if it consists of units that can be executed independently or partially out of order without changing the output of the program

Concurrent Computing

Henri Casanova ([email protected]) Threads Concurrency: A feature of a program that can do multiple things “at the same time”

A program is concurrent if it consists of units that can be executed independently or partially out of order without changing the output of the program

Concurrent Computing

Concurrent computing: several computations are performed during overlapping time periods As opposed to sequential execution, which is one computation to completion, followed by one computation to completion, etc.

Henri Casanova ([email protected]) Threads A program is concurrent if it consists of units that can be executed independently or partially out of order without changing the output of the program

Concurrent Computing

Concurrency: A feature of a program that can do multiple things “at the same time”

Henri Casanova ([email protected]) Threads Concurrent Computing

Concurrency: A feature of a program that can do multiple things “at the same time”

A program is concurrent if it consists of units that can be executed independently or partially out of order without changing the output of the program

Henri Casanova ([email protected]) Threads I want to output a boolean array where each element is true if and only if the corresponding integer in the input array is > 50 { false, true, false, true, true ...} Assume that it takes one millisecond to test an integer value and update the output array Sequential programming: Iterating through the array would take 10,000 milliseconds. Concurrent programming: If I create 10 “units of execution” to compute 1000 output values, i.e., 1/10-th of the work, each unit takes 1,000 milliseconds Now if I can execute these 10 units independently (on 10 CPUs), the whole execution takes 1,000 milliseconds, i.e., 10 times faster In practice, we can’t go exactly 10 times faster due to various overheads and bottlenecks But we will go much faster than sequential, provided be have multiple CPUs (which we all do in this multicore era!)

Concurrent Computing: Example

Consider an input array of 10,000 integers with values between 0 and 100 { 23, 56, 7, 68, 68 ...}

Henri Casanova ([email protected]) Threads Assume that it takes one millisecond to test an integer value and update the output array Sequential programming: Iterating through the array would take 10,000 milliseconds. Concurrent programming: If I create 10 “units of execution” to compute 1000 output values, i.e., 1/10-th of the work, each unit takes 1,000 milliseconds Now if I can execute these 10 units independently (on 10 CPUs), the whole execution takes 1,000 milliseconds, i.e., 10 times faster In practice, we can’t go exactly 10 times faster due to various overheads and bottlenecks But we will go much faster than sequential, provided be have multiple CPUs (which we all do in this multicore era!)

Concurrent Computing: Example

Henri Casanova ([email protected]) Threads Sequential programming: Iterating through the array would take 10,000 milliseconds. Concurrent programming: If I create 10 “units of execution” to compute 1000 output values, i.e., 1/10-th of the work, each unit takes 1,000 milliseconds Now if I can execute these 10 units independently (on 10 CPUs), the whole execution takes 1,000 milliseconds, i.e., 10 times faster In practice, we can’t go exactly 10 times faster due to various overheads and bottlenecks But we will go much faster than sequential, provided be have multiple CPUs (which we all do in this multicore era!)

Concurrent Computing: Example

Henri Casanova ([email protected]) Threads Concurrent programming: If I create 10 “units of execution” to compute 1000 output values, i.e., 1/10-th of the work, each unit takes 1,000 milliseconds Now if I can execute these 10 units independently (on 10 CPUs), the whole execution takes 1,000 milliseconds, i.e., 10 times faster In practice, we can’t go exactly 10 times faster due to various overheads and bottlenecks But we will go much faster than sequential, provided be have multiple CPUs (which we all do in this multicore era!)

Concurrent Computing: Example

Henri Casanova ([email protected]) Threads In practice, we can’t go exactly 10 times faster due to various overheads and bottlenecks But we will go much faster than sequential, provided be have multiple CPUs (which we all do in this multicore era!)

Concurrent Computing: Example

Consider an input array of 10,000 integers with values between 0 and 100 { 23, 56, 7, 68, 68 ...} I want to output a boolean array where each element is true if and only if the corresponding integer in the input array is > 50 { false, true, false, true, true ...} Assume that it takes one millisecond to test an integer value and update the output array Sequential programming: Iterating through the array would take 10,000 milliseconds. Concurrent programming: If I create 10 “units of execution” to compute 1000 output values, i.e., 1/10-th of the work, each unit takes 1,000 milliseconds Now if I can execute these 10 units independently (on 10 CPUs), the whole execution takes 1,000 milliseconds, i.e., 10 times faster

Henri Casanova ([email protected]) Threads Concurrent Computing: Example

Henri Casanova ([email protected]) Threads To make programs more responsive Structuring a program as concurrent activities can make it more responsive because while one activity blocks waiting for some event, another can do something e.g., in some server, spawn a new activity to answer each client request e.g., in some GUI, while one activity is updating some display another activity is listening for mouse clicks

What is Concurrency Used for?

Make programs faster Running multiple activities at once can use the machine more eﬀectively because there are multiple hardware components e.g., while one activity computes on the CPU, another activity sends data to the network card e.g., while one activity computes on one CPU, another computes on another CPU (this is the example in the previous slide)

Henri Casanova ([email protected]) Threads What is Concurrency Used for?

To make programs more responsive Structuring a program as concurrent activities can make it more responsive because while one activity blocks waiting for some event, another can do something e.g., in some server, spawn a new activity to answer each client request e.g., in some GUI, while one activity is updating some display another activity is listening for mouse clicks

Henri Casanova ([email protected]) Threads But because the OS virtualizes memory, processes don’t share memory naturally You can make it happen with special system calls, but it’s cumbersome This can make it diﬃcult to program processes that have complicated cooperative behaviors

And so come threads...

Concurrency with Processes

Henri Casanova ([email protected]) Threads And so come threads...

Concurrency with Processes

We have already talks about concurrency After all it’s the 2nd “easy piece” in our textbook Processes run concurrently on the computer And therefore, they can be used for better responsiveness and speed They have been used for those objectives a lot say until the early 90’s And still used a lot, e.g., see our programming assignment But because the OS virtualizes memory, processes don’t share memory naturally You can make it happen with special system calls, but it’s cumbersome This can make it diﬃcult to program processes that have complicated cooperative behaviors

Henri Casanova ([email protected]) Threads Concurrency with Processes

And so come threads...

Henri Casanova ([email protected]) Threads Multi-threaded process: Concurrent execution of diﬀerent parts of the same running program

Each thread has its own: Thread ID (assigned by the OS) Program Counter (which instruction it currently executes) Registers Set (which values are stored in registers) Stack (bookkeeping of its function/method invocations) The above fully deﬁnes “what a thread is doing right now” But it shares with other threads in the same process the code/text section the data segment (global variables) the list of open ﬁle descriptors (at the moment of thread creation) the heap the signal behaviors (handlers)

Threads

A thread is a basic unit of CPU utilization within a process

Henri Casanova ([email protected]) Threads Each thread has its own: Thread ID (assigned by the OS) Program Counter (which instruction it currently executes) Registers Set (which values are stored in registers) Stack (bookkeeping of its function/method invocations) The above fully deﬁnes “what a thread is doing right now” But it shares with other threads in the same process the code/text section the data segment (global variables) the list of open ﬁle descriptors (at the moment of thread creation) the heap the signal behaviors (handlers)

Threads

A thread is a basic unit of CPU utilization within a process Multi-threaded process: Concurrent execution of diﬀerent parts of the same running program

Henri Casanova ([email protected]) Threads The above fully deﬁnes “what a thread is doing right now” But it shares with other threads in the same process the code/text section the data segment (global variables) the list of open ﬁle descriptors (at the moment of thread creation) the heap the signal behaviors (handlers)

Threads

A thread is a basic unit of CPU utilization within a process Multi-threaded process: Concurrent execution of diﬀerent parts of the same running program

Henri Casanova ([email protected]) Threads But it shares with other threads in the same process the code/text section the data segment (global variables) the list of open ﬁle descriptors (at the moment of thread creation) the heap the signal behaviors (handlers)

Threads

A thread is a basic unit of CPU utilization within a process Multi-threaded process: Concurrent execution of diﬀerent parts of the same running program

Henri Casanova ([email protected]) Threads Threads

A thread is a basic unit of CPU utilization within a process Multi-threaded process: Concurrent execution of diﬀerent parts of the same running program

Henri Casanova ([email protected]) Threads code data heap

r s r s r s

Thread

Multi-Threaded Process

Threads: Typical representation

code data heap

registers stack

Single-Threaded Process

Henri Casanova ([email protected]) Threads code data heap

r s r s r s

Multi-Threaded Process

Threads: Typical representation

code data heap

registers stack

Thread

Single-Threaded Process

Henri Casanova ([email protected]) Threads Threads: Typical representation

code data heap code data heap

registers stack r s r s r s

Thread

Single-Threaded Process Multi-Threaded Process

Henri Casanova ([email protected]) Threads Multi-Threaded Program in Execution 1/3

Threads can execute diﬀerent parts of code

Henri Casanova ([email protected]) Threads Multi-Threaded Program in Execution 1/3

Threads can execute diﬀerent parts of code

Henri Casanova ([email protected]) Threads Multi-Threaded Program in Execution 1/3

Threads can execute diﬀerent parts of code

Henri Casanova ([email protected]) Threads Multi-Threaded Program in Execution 1/3

Threads can execute diﬀerent parts of code

Henri Casanova ([email protected]) Threads Multi-Threaded Program in Execution 2/3

Threads can execute the same part of code

Henri Casanova ([email protected]) Threads Multi-Threaded Program in Execution 3/3

... Or anything...

Henri Casanova ([email protected]) Threads Economy Creating a thread is cheap Slightly cheaper than creating a process in MacOSX / Linux Much cheaper than creating a process in Windows Context-switching between threads is cheaper than between processes So if you can do with threads what you can do with processes, then you likely can go a bit faster

Advantages of Threads vs. Processes

Resource Sharing Threads “naturally” share memory Provides a direct Shared Memory IPC mechanism No need for other complicate IPC mechanisms Having concurrent activities in the same address space is very powerful It makes it possible to implement all kinds of concurrent behaviors

Henri Casanova ([email protected]) Threads Advantages of Threads vs. Processes

Economy Creating a thread is cheap Slightly cheaper than creating a process in MacOSX / Linux Much cheaper than creating a process in Windows Context-switching between threads is cheaper than between processes So if you can do with threads what you can do with processes, then you likely can go a bit faster

Henri Casanova ([email protected]) Threads Threads may be more memory-constrained than processes Since they execute in the same process address space But that’s typically not a big deal

Threads do not beneﬁt from memory protection This is a feature, since you want them to see the same memory But it can cause really, really diﬃcult bugs More about this in the Synchronization module...

Drawbacks of Threads vs. Processes

If one thread fails with an error/exception which is not managed, all threads (and therefore the whole process) fails

Henri Casanova ([email protected]) Threads Threads do not beneﬁt from memory protection This is a feature, since you want them to see the same memory But it can cause really, really diﬃcult bugs More about this in the Synchronization module...

Drawbacks of Threads vs. Processes

If one thread fails with an error/exception which is not managed, all threads (and therefore the whole process) fails

Threads may be more memory-constrained than processes Since they execute in the same process address space But that’s typically not a big deal

Henri Casanova ([email protected]) Threads Drawbacks of Threads vs. Processes

If one thread fails with an error/exception which is not managed, all threads (and therefore the whole process) fails

Threads may be more memory-constrained than processes Since they execute in the same process address space But that’s typically not a big deal

Henri Casanova ([email protected]) Threads Answer: NO Many applications consist of multiple (multi-threaded) processes A well-known example of fork()’s usefulness: the Google Chrome Web browser Google Chrome calls fork() each time you open a tab! Each tab is a (multi-threaded) process And thus writing Chrome code is a bit harder because processes need to use process IPC instead of just “seeing” the same memory In fact, let’s look at this Firefox technical page (and click on Motivation)

Is “Concurrency with Processes” obsolete?

Given the advantages of threads, should I just not care about concurrency with processes? i.e., using fork(), waitpid(), popen(), etc.

Henri Casanova ([email protected]) Threads A well-known example of fork()’s usefulness: the Google Chrome Web browser Google Chrome calls fork() each time you open a tab! Each tab is a (multi-threaded) process And thus writing Chrome code is a bit harder because processes need to use process IPC instead of just “seeing” the same memory In fact, let’s look at this Firefox technical page (and click on Motivation)

Is “Concurrency with Processes” obsolete?

Henri Casanova ([email protected]) Threads Is “Concurrency with Processes” obsolete?

Given the advantages of threads, should I just not care about concurrency with processes? i.e., using fork(), waitpid(), popen(), etc. Answer: NO Many applications consist of multiple (multi-threaded) processes A well-known example of fork()’s usefulness: the Google Chrome Web browser Google Chrome calls fork() each time you open a tab! Each tab is a (multi-threaded) process And thus writing Chrome code is a bit harder because processes need to use process IPC instead of just “seeing” the same memory In fact, let’s look at this Firefox technical page (and click on Motivation)

Henri Casanova ([email protected]) Threads About once every 6 months in my software development eﬀorts I end up having to use fork for something Example: my own simulation project (WRENCH) in which I had to re-use code from another research group (BatSched) – December 2017 Working with Google Test (the software testing infrastructure from Google Inc.), had to create a process to avoid broken tests from breaking the whole test framework – March 2017

Bottom Line: don’t drink the “I don’t care about fork() because I’ll only be doing threads” Kool-Aid

Is “Concurrency with Processes” obsolete? (2)

In real-world settings you often have to put together diﬀerent software products to make up a full system Some of these software products may just be executables Read/write on the network, read/write on stdin/stdout, read/write on ﬁles, They don’t have APIs you can use, and so you can’t just write the code that you wishes your threads to do... Or don’t want to!! (see our Processes homework assignment) Conclusion: you have to use fork/exec to achieve your goals

Henri Casanova ([email protected]) Threads Bottom Line: don’t drink the “I don’t care about fork() because I’ll only be doing threads” Kool-Aid

Is “Concurrency with Processes” obsolete? (2)

About once every 6 months in my software development eﬀorts I end up having to use fork for something Example: my own simulation project (WRENCH) in which I had to re-use code from another research group (BatSched) – December 2017 Working with Google Test (the software testing infrastructure from Google Inc.), had to create a process to avoid broken tests from breaking the whole test framework – March 2017

Henri Casanova ([email protected]) Threads Is “Concurrency with Processes” obsolete? (2)

Bottom Line: don’t drink the “I don’t care about fork() because I’ll only be doing threads” Kool-Aid

Henri Casanova ([email protected]) Threads Threads can be supported solely in User Space, or User Threads You can actually write your own threads management system and the OS will never know about it

All modern OSes support threads in Kernel Space or Kernel Threads The kernel provides data structures and functionality to handle threads

User Threads vs. Kernel Threads

Let’s now talk about how the OS implements threads

Henri Casanova ([email protected]) Threads All modern OSes support threads in Kernel Space or Kernel Threads The kernel provides data structures and functionality to handle threads

User Threads vs. Kernel Threads

Let’s now talk about how the OS implements threads

Threads can be supported solely in User Space, or User Threads You can actually write your own threads management system and the OS will never know about it

Henri Casanova ([email protected]) Threads User Threads vs. Kernel Threads

Let’s now talk about how the OS implements threads

Threads can be supported solely in User Space, or User Threads You can actually write your own threads management system and the OS will never know about it

All modern OSes support threads in Kernel Space or Kernel Threads The kernel provides data structures and functionality to handle threads

Henri Casanova ([email protected]) Threads Main advantage of Kernel Threads + Can use multiple cores: Since the Kernel is aware of the multiple threads, it can put them on diﬀerent cores + Threads don’t have to cooperate: The Kernel is in charge of scheduling (see upcoming module) − Higher overhead: since system calls are involves to create/terminate threads

User Threads vs. Kernel Threads: Advantages and Drawbacks

User Threads + Low overhead: fast thread creation/termination, fast switching between threads (because no OS involvement) - Can’t use multiple cores: The OS doesn’t know anything about user-level threads - Threads must cooperate: They should play nice and give up the CPU willingly to let other threads run (again, the OS knows nothing about them and thus can’t decide who runs when and for how long) And if one thread blocks, the others could all be stuck (e.g., sleep(1000))

Henri Casanova ([email protected]) Threads User Threads vs. Kernel Threads: Advantages and Drawbacks

Main advantage of Kernel Threads + Can use multiple cores: Since the Kernel is aware of the multiple threads, it can put them on diﬀerent cores + Threads don’t have to cooperate: The Kernel is in charge of scheduling (see upcoming module) − Higher overhead: since system calls are involves to create/terminate threads

Henri Casanova ([email protected]) Threads User/Kernel Threads Models

Historically, there have been various ways in which to combine user threads and kernel threads

4 diﬀerent models: Many-to-One Model One-to-One Model Many-to-Many Model Two-Level Model

Henri Casanova ([email protected]) Threads Examples (User-level Threads): Java Green Threads (obsolete), GNU Portable Threads (obsolete-2006), Python threading In some cases, they are very useful when you know what you’re doing For instance, I use them in the simulation research project I’ve mentioned in the past (SimGrid)

Threads Models: Many-to-One Model

This is basically: use only user-level user thread threads Advantage: multi-threading is eﬃcient and low-overhead The drawbacks: can’t use multi-core, thread must cooperate

K kernel thread

Henri Casanova ([email protected]) Threads In some cases, they are very useful when you know what you’re doing For instance, I use them in the simulation research project I’ve mentioned in the past (SimGrid)

Threads Models: Many-to-One Model

K kernel thread

Henri Casanova ([email protected]) Threads Threads Models: Many-to-One Model

This is basically: use only user-level user thread threads Advantage: multi-threading is eﬃcient and low-overhead The drawbacks: can’t use multi-core, thread must cooperate Examples (User-level Threads): Java Green Threads (obsolete), GNU Portable Threads (obsolete-2006), Python threading In some cases, they are very useful when you know what you’re doing For instance, I use them in the simulation research project I’ve mentioned in the past (SimGrid) K kernel thread

Henri Casanova ([email protected]) Threads Examples: Linux, MacOS X, Windows, Solaris 9 and later

Threads Models: One-to-One Model

This is basically: use only user thread kernel-level threads Drawbacks: higher overhead Advantages: Can use multiple cores, threads don’t need to cooperate

K K K K kernel thread

Henri Casanova ([email protected]) Threads Threads Models: One-to-One Model

This is basically: use only user thread kernel-level threads Drawbacks: higher overhead Advantages: Can use multiple cores, threads don’t need to cooperate Examples: Linux, MacOS X, Windows, Solaris 9 and later K K K K kernel thread

Henri Casanova ([email protected]) Threads Examples: Solaris 9 and earlier, WinNT/2k

Threads Models: Many-to-Many Model

user thread

A compromise... Not all threads must cooperate Multiple cores can be used Not all thread creation involves the kernel

K K K kernel thread

Henri Casanova ([email protected]) Threads Threads Models: Many-to-Many Model

user thread

A compromise... Not all threads must cooperate Multiple cores can be used Not all thread creation involves the kernel Examples: Solaris 9 and earlier, WinNT/2k

K K K kernel thread

Henri Casanova ([email protected]) Threads Letting the user decide is usually not a good thing (unless they know what they are doing)

Threads Models: Two-Level Model

user thread Let the user decide! System calls to say “Bind this thread to its own kernel thread” Examples: IRIX (obsolete), HP-UX (old ones), Tru64 UNIX (obsolete) Obsolete?

K K K K kernel thread

Henri Casanova ([email protected]) Threads Threads Models: Two-Level Model

user thread Let the user decide! System calls to say “Bind this thread to its own kernel thread” Examples: IRIX (obsolete), HP-UX (old ones), Tru64 UNIX (obsolete) Obsolete? Letting the user decide is usually not a good thing (unless they know what they are doing)

K K K K kernel thread

Henri Casanova ([email protected]) Threads Let’s look at Java...

Threads Libraries

Thread libraries provide users with ways to create threads in their own programs C/C++: Pthreads (implemented by the kernel) C/C++: OpenMP (layer above Pthreads) Java: Java threads (implemented by the JVM, which relies on the kernel threads implementation) Python: threading / multiprocessing packages (watch out though threading uses the many-to-one model) JavaScript: no multithreading (the multiprocessing is performed by the “web application” — See Web Workers (Working draft of W3C standard))

Henri Casanova ([email protected]) Threads Threads Libraries

Let’s look at Java...

Henri Casanova ([email protected]) Threads There is a Thread class There is a Runnable interface There is a Callable interface There is an ExecutorService interface

Let’s see how to deﬁne/run threads using these...

Java Threads

Java makes it pretty easy to deﬁne and run threads Java does it like it does everything else, by providing Thread-related classes and interfaces

Henri Casanova ([email protected]) Threads Let’s see how to deﬁne/run threads using these...

Java Threads

Java makes it pretty easy to deﬁne and run threads Java does it like it does everything else, by providing Thread-related classes and interfaces

There is a Thread class There is a Runnable interface There is a Callable interface There is an ExecutorService interface

Henri Casanova ([email protected]) Threads Java Threads

Java makes it pretty easy to deﬁne and run threads Java does it like it does everything else, by providing Thread-related classes and interfaces

There is a Thread class There is a Runnable interface There is a Callable interface There is an ExecutorService interface

Let’s see how to deﬁne/run threads using these...

Henri Casanova ([email protected]) Threads Java Threads: The Thread class — All versions of Java

Using the Thread class: Implementing a subclass that extends Thread Override the run() method (if you forget this, nothing will work, but Java won’t complain!) Call the (never overridden) start() method to start the thread

Thread subclass public class MyThread extends Main program Thread { MyThread() { // Constructor public class super(); SomeProgramUsingMyThread { ... public static void main(...) { } MyThread myThread = new MyThread(); @Override myThread.start(); public void run() { ... // At this point2 threads ... // What the thread should are running do } } }

Henri Casanova ([email protected]) Threads Java Threads: The Runnable interface — All versions of Java

Implement the Runnable interface Override the run() method (if you forget this, nothing will work, but Java won’t complain!) Call the (never overridden) start() method to start the thread

Class that implements Runnable Main program public class MyRunnable implements Runnable { MyRunnable() { // Constructor public class SomeProgramUsingMyRunnable { ... public static void main(...) { } MyRunnable myRunnable = new MyRunnable(); Thread thread = new Thread(myRunnable); @Override thread.start(); public void run() { ... // At this point2 threads are running ... // What the thread should do } } }

Henri Casanova ([email protected]) Threads Java Threads: The Callable interface — Java 1.5+

Implement the Callable interface Override the call() method Add a return type and checked exception to call() Start the thread through an ExecutorService (see next slide)

Main program (a Callable that return a MyAwesomeClass object) public class MyCallable implements Callable { MyCallable() { // Constructor ... }

@override public MyAwesomeClass call() throws MyOopsException { ... return myAwesomeObject; // instance of MyAwesomeClass }

Henri Casanova ([email protected]) Threads Java Threads: ExecutorServices

Classes derived from ExecutorServices implement convenient thread management options: i.e., Executors.newSingleThreadExecutor(), Executors.newFixedThreadPool(nThreads), and others...

Main program (a Callable that return a MyAwesomeClass object) public class SomeProgramUsingMyCallable { public void doSomething() { MyCallable myThread3 = new MyCallable(); ExecutorService es = Executors.newFixedThreadPool(42); Future result = es.submit(myThread3); ... es.shutdown();

try { es.awaitTermination(timeout, timeUnit); } catch (InterruptedException e) { ... } ... if (result != null) { MyAwesomeClass myAwesomeObject = result.get(); } ... }

Henri Casanova ([email protected]) Threads To launch or spawn a Thread/Runnable it is necessary to call the start() method (instead of run()) If you call run(), it’s just a regular method call that does not create a thread but wil happily run the method’s code!! The start() method hides all the “details” of thread creation (basically it places system calls) ExecutorServices are powerful, and very convenient to run things in threads without having to do much work Just create Callable objects and submit them to the executor service

Of course there are many tutorials and a lot of documentation about all these things, and many many bells and whistles Let’s now see a few basic example...

Java Threads Takeaways

It’s wise to use Runnable, rather than extend the Thread class

Henri Casanova ([email protected]) Threads The start() method hides all the “details” of thread creation (basically it places system calls) ExecutorServices are powerful, and very convenient to run things in threads without having to do much work Just create Callable objects and submit them to the executor service

Of course there are many tutorials and a lot of documentation about all these things, and many many bells and whistles Let’s now see a few basic example...

Java Threads Takeaways

Henri Casanova ([email protected]) Threads ExecutorServices are powerful, and very convenient to run things in threads without having to do much work Just create Callable objects and submit them to the executor service

Of course there are many tutorials and a lot of documentation about all these things, and many many bells and whistles Let’s now see a few basic example...

Java Threads Takeaways

Henri Casanova ([email protected]) Threads Of course there are many tutorials and a lot of documentation about all these things, and many many bells and whistles Let’s now see a few basic example...

Java Threads Takeaways

It’s wise to use Runnable, rather than extend the Thread class To launch or spawn a Thread/Runnable it is necessary to call the start() method (instead of run()) If you call run(), it’s just a regular method call that does not create a thread but wil happily run the method’s code!! The start() method hides all the “details” of thread creation (basically it places system calls) ExecutorServices are powerful, and very convenient to run things in threads without having to do much work Just create Callable objects and submit them to the executor service

Henri Casanova ([email protected]) Threads Java Threads Takeaways

Of course there are many tutorials and a lot of documentation about all these things, and many many bells and whistles Let’s now see a few basic example...

Henri Casanova ([email protected]) Threads Java Threads: Example #1

Printings 1’s public class HelloWorldRunnable implements Runnable { private int index; public HelloWorldRunnable(int index) { this.index = index; } @Override public void run() { for (int i=0;i<10000;i++) { System.out.print(this.index); } } }

public class MyProgram { public static void main(String[] args) { HelloWorldRunnable helloRunnable = new HelloWorldRunnable(1); Thread helloThread = new Thread(helloRunnable); helloThread.start(); } }

Henri Casanova ([email protected]) Threads We now have two threads The main thread The newly created thread Both threads are running The main thread does not do anything The new thread prints a bunch of ’1’ to the to terminal In Java, when both threads terminate, the process terminates The main thread terminates when it returns from main() The new thread terminates when it returns from run() Let’s have the ﬁrst thread do something as well...

Java Threads: Example #1 — What should we expect?

The previous program runs as a Java process i.e., as a thread running inside the JVM, in fact We call it the main thread When the start() method is called, the main thread creates a new thread

Henri Casanova ([email protected]) Threads Both threads are running The main thread does not do anything The new thread prints a bunch of ’1’ to the to terminal In Java, when both threads terminate, the process terminates The main thread terminates when it returns from main() The new thread terminates when it returns from run() Let’s have the ﬁrst thread do something as well...

Java Threads: Example #1 — What should we expect?

Henri Casanova ([email protected]) Threads In Java, when both threads terminate, the process terminates The main thread terminates when it returns from main() The new thread terminates when it returns from run() Let’s have the ﬁrst thread do something as well...

Java Threads: Example #1 — What should we expect?

Henri Casanova ([email protected]) Threads Let’s have the ﬁrst thread do something as well...

Java Threads: Example #1 — What should we expect?

The previous program runs as a Java process i.e., as a thread running inside the JVM, in fact We call it the main thread When the start() method is called, the main thread creates a new thread We now have two threads The main thread The newly created thread Both threads are running The main thread does not do anything The new thread prints a bunch of ’1’ to the to terminal In Java, when both threads terminate, the process terminates The main thread terminates when it returns from main() The new thread terminates when it returns from run()

Henri Casanova ([email protected]) Threads Java Threads: Example #1 — What should we expect?

Henri Casanova ([email protected]) Threads Java Threads: Example #1a

Printings 1’s and 0’s public class HelloWorldRunnable implements Runnable { private int index; public HelloWorldRunnable(int index) { this.index = index; } @Override public void run() { for (int i=0;i<10000;i++) { System.out.print(this.index); } } } public class MyProgram { public static void main(String[] args) { HelloWorldRunnable helloRunnable = new HelloWorldRunnable(1); Thread helloThread = new Thread(helloRunnable); helloThread.start(); for (int i=0;i<10000;i++) { System.out.print(0); } } }

Henri Casanova ([email protected]) Threads Answer: Impossible to tell for sure If you know the details of the implementation of the JVM on your host, and you know your OS and hardware well, perhaps you can have some idea of what it might look like ... but it’s not very useful because it will look diﬀerent on a diﬀerent setup (it’s not portable) Let’s have a look at a few executions...

Java Threads: Example #1a — What should we expect?

Now we have the main thread printing to the screen and the new thread printing to the screen Question: what will the output be?

Henri Casanova ([email protected]) Threads Java Threads: Example #1a — What should we expect?

Now we have the main thread printing to the screen and the new thread printing to the screen Question: what will the output be? Answer: Impossible to tell for sure If you know the details of the implementation of the JVM on your host, and you know your OS and hardware well, perhaps you can have some idea of what it might look like ... but it’s not very useful because it will look diﬀerent on a diﬀerent setup (it’s not portable) Let’s have a look at a few executions...

Henri Casanova ([email protected]) Threads The execution is non-deterministic Something decides when a thread runs In this case the JVM and the OS together Deciding when a thread runs is called thread scheduling

Java Threads: Example #1a

Henri Casanova ([email protected]) Threads Something decides when a thread runs In this case the JVM and the OS together Deciding when a thread runs is called thread scheduling

Java Threads: Example #1a

The execution is non-deterministic

Henri Casanova ([email protected]) Threads In this case the JVM and the OS together Deciding when a thread runs is called thread scheduling

Java Threads: Example #1a

The execution is non-deterministic Something decides when a thread runs

Henri Casanova ([email protected]) Threads Deciding when a thread runs is called thread scheduling

Java Threads: Example #1a

The execution is non-deterministic Something decides when a thread runs In this case the JVM and the OS together

Henri Casanova ([email protected]) Threads Java Threads: Example #1a

The execution is non-deterministic Something decides when a thread runs In this case the JVM and the OS together Deciding when a thread runs is called thread scheduling

Henri Casanova ([email protected]) Threads Major diﬃculty: You may not be able to reproduce a bug because each execution is diﬀerent! Therefore, you may think your code is working, but that’s because you haven’t been able to observe the bug yet... If you run is 10,000 times and don’t see the bug, you still cannot be sure that the bug will not happen

Multi-Threading Programming Challenge

Major challenge: You cannot make any assumption about thread scheduling, since the OS is in charge And what the OS does depends on the hardware and on other running processes

Henri Casanova ([email protected]) Threads Multi-Threading Programming Challenge

Major challenge: You cannot make any assumption about thread scheduling, since the OS is in charge And what the OS does depends on the hardware and on other running processes

Major diﬃculty: You may not be able to reproduce a bug because each execution is diﬀerent! Therefore, you may think your code is working, but that’s because you haven’t been able to observe the bug yet... If you run is 10,000 times and don’t see the bug, you still cannot be sure that the bug will not happen

Henri Casanova ([email protected]) Threads In a nutshell: Threads everywhere Kernel threads that run application threads Threads in the JVM that do some work for the JVM

Let’s write a Java program that does nothing and count threads (ps auxM)

Java Threads / Kernel Threads

The JVM is itself multi-threaded! The JVM has a thread scheduler for application threads, which are mapped to kernel threads Several application threads could be mapped to the same kernel thread That thread scheduler runs itself in a dedicated thread The OS is in charge of scheduling kernel threads

Henri Casanova ([email protected]) Threads Java Threads / Kernel Threads

In a nutshell: Threads everywhere Kernel threads that run application threads Threads in the JVM that do some work for the JVM

Let’s write a Java program that does nothing and count threads (ps auxM)

Henri Casanova ([email protected]) Threads In Java, a thread can call Thread.yield(), which says “I am willingly giving up the CPU now” But it is still not deterministic! Programs should NEVER rely on yield() for correctness (it’s more a hint to the JVM, and can help for interactivity) In Java, the Thread class has a setPriority() (and a getPriority()) method Thread priorities are integers ranging between Thread.MIN PRIORITY and Thread.MAX PRIORITY, the greater the integer, the higher the priority Again, these are hints and you can’t rely on them (and they don’t work at all on some JVMs!!) All the above are basically “hints that may have some eﬀect”, nothing more So they don’t really “solve” anything for sure

Java Threads: Relinquishing control with yield()

Henri Casanova ([email protected]) Threads In Java, the Thread class has a setPriority() (and a getPriority()) method Thread priorities are integers ranging between Thread.MIN PRIORITY and Thread.MAX PRIORITY, the greater the integer, the higher the priority Again, these are hints and you can’t rely on them (and they don’t work at all on some JVMs!!) All the above are basically “hints that may have some eﬀect”, nothing more So they don’t really “solve” anything for sure

Java Threads: Relinquishing control with yield()

Henri Casanova ([email protected]) Threads All the above are basically “hints that may have some eﬀect”, nothing more So they don’t really “solve” anything for sure

Java Threads: Relinquishing control with yield()

At this point, it seems that we throw a bunch of threads in, the OS “shakes the bag”, and we don’t really know what happens To some extent this is true, but we have ways to inﬂuence what happens control In Java, a thread can call Thread.yield(), which says “I am willingly giving up the CPU now” But it is still not deterministic! Programs should NEVER rely on yield() for correctness (it’s more a hint to the JVM, and can help for interactivity) In Java, the Thread class has a setPriority() (and a getPriority()) method Thread priorities are integers ranging between Thread.MIN PRIORITY and Thread.MAX PRIORITY, the greater the integer, the higher the priority Again, these are hints and you can’t rely on them (and they don’t work at all on some JVMs!!)

Henri Casanova ([email protected]) Threads Java Threads: Relinquishing control with yield()

Henri Casanova ([email protected]) Threads TERMINATED

start() RUNNABLE

RunningRUNNABLENot running

All waiting states

TIMED WAITING BLOCKED WAITING

Java Threads: Thread States — Thread::getState()

NEW

Henri Casanova ([email protected]) Threads TERMINATED

RUNNABLE

RunningRUNNABLENot running

All waiting states

TIMED WAITING BLOCKED WAITING

Java Threads: Thread States — Thread::getState()

NEW

start()

Henri Casanova ([email protected]) Threads TERMINATED

RUNNABLE

Running Not running

All waiting states

TIMED WAITING BLOCKED WAITING

Java Threads: Thread States — Thread::getState()

NEW

start()

RUNNABLE

Henri Casanova ([email protected]) Threads TERMINATED

RUNNABLE

All waiting states

TIMED WAITING BLOCKED WAITING

Java Threads: Thread States — Thread::getState()

NEW

start() RUNNABLE

Running Not running

Henri Casanova ([email protected]) Threads TERMINATED

RUNNABLE

All waiting states

TIMED WAITING WAITING

Java Threads: Thread States — Thread::getState()

NEW

start() RUNNABLE

Running Not running

BLOCKED

Henri Casanova ([email protected]) Threads TERMINATED

RUNNABLE

All waiting states

TIMED WAITING

Java Threads: Thread States — Thread::getState()

NEW

start() RUNNABLE

Running Not running

BLOCKED WAITING

Henri Casanova ([email protected]) Threads TERMINATED

RUNNABLE

All waiting states

Java Threads: Thread States — Thread::getState()

NEW

start() RUNNABLE

Running Not running

TIMED WAITING BLOCKED WAITING

Henri Casanova ([email protected]) Threads TERMINATED

RUNNABLE

All waiting states

Java Threads: Thread States — Thread::getState()

NEW

start() RUNNABLE

Running Not running

TIMED WAITING BLOCKED WAITING

Henri Casanova ([email protected]) Threads TERMINATED

RUNNABLE

Java Threads: Thread States — Thread::getState()

NEW

start() RUNNABLE

Running Not running

All waiting states

TIMED WAITING BLOCKED WAITING

Henri Casanova ([email protected]) Threads TERMINATED

RUNNABLE

Java Threads: Thread States — Thread::getState()

NEW

start() RUNNABLE

Running Not running

All waiting states

TIMED WAITING BLOCKED WAITING

Henri Casanova ([email protected]) Threads RUNNABLE

Java Threads: Thread States — Thread::getState()

NEW TERMINATED

start() RUNNABLE

Running Not running

All waiting states

TIMED WAITING BLOCKED WAITING

Henri Casanova ([email protected]) Threads RUNNABLE

Java Threads: Thread States — Thread::getState()

NEW TERMINATED

start() RUNNABLE

Running Not running

All waiting states

TIMED WAITING BLOCKED WAITING

Henri Casanova ([email protected]) Threads Java Threads: Thread States

Does the previous ﬁgure remind you of anything?

Henri Casanova ([email protected]) Threads Process States of course!

New Terminated

Descheduled Accepted Exit

Ready Running

Scheduled I/O Complete I/O Initiated

Waiting

Henri Casanova ([email protected]) Threads Threads States vs Process States

NEW TERMINATED New Terminated start() RUNNABLE Descheduled Accepted Exit

RUNNABLE Running Not running Ready Running

Scheduled I/O Complete I/O Initiated

All waiting states

TIMED WAITING BLOCKED WAITING Waiting

Henri Casanova ([email protected]) Threads In old OSes (Solaris 4), threads were actually called lightweight processes (since they do not require as much resources allocation that regular/heavyweight processes created by fork())

Linux/MacOS X Threads

Linux does not distinguish between processes and threads: they are called tasks Kernel data structure: task struct We already looked at it when we talked about processes The clone() syscall is used to create a task It can be invoked with several options, called the ﬂag Each option speciﬁes one thing that the child may share with the parent fork() is just a particular set of options Preserved as a system call for backward compatibility

Henri Casanova ([email protected]) Threads Linux/MacOS X Threads

Henri Casanova ([email protected]) Threads Java Threads: Wait for a thread to terminate

Given a Thread object, calling its join() method means “waiting for that thread to terminate” Useful to give some work to do to a thread, and at some point wait for that work to be done before moving on

Waiting for a tread termination with join() ... Thread thread = new Thread(new MyRunnable()); thread.start(); ... try { thread.join();// will block until my Runnable object returns from run() } catch (InterruptedException e) { ... }

Henri Casanova ([email protected]) Threads Java Threads: Wait for a thread to terminate (2)

For an ExecutorService object, the method is awaitTermination()

Waiting for a tread termination with ExecutorService(s) ... Future future = executorService.submit(new MyRunnable());

try { // will block until Runnable returns from run() executorService.awaitTermination(); } catch (InterruptedException e) { ... }

// One can also test for termination as follows if (future.isDone()) { ... }

Henri Casanova ([email protected]) Threads Instead, you can do deferred cancellation The thread has to periodically check whether it should terminate e.g., by checking some boolean variable This is because asynchronous cancellation is dangerous e.g., what if the thread was in the middle of writing a ﬁle?? And Java avoid all danger as much as possible

Java Thread Cancellation

Henri Casanova ([email protected]) Threads This is because asynchronous cancellation is dangerous e.g., what if the thread was in the middle of writing a ﬁle?? And Java avoid all danger as much as possible

Java Thread Cancellation

Henri Casanova ([email protected]) Threads Java Thread Cancellation

It is not possible to perform an asynchronous cancellation of a thread (i.e., a thread tells another thread to stop right now) The Thread::stop() method is deprecated In other words: you cannot kill a Java thread Instead, you can do deferred cancellation The thread has to periodically check whether it should terminate e.g., by checking some boolean variable This is because asynchronous cancellation is dangerous e.g., what if the thread was in the middle of writing a ﬁle?? And Java avoid all danger as much as possible

Henri Casanova ([email protected]) Threads Programming with threads is known to be diﬃcult, and a lot of techniques/tools are available In this course we focus more on how the OS implements threads than how the user uses threads

Quiz next week on this module... Homework Assignment #4...

Conclusion

Henri Casanova ([email protected]) Threads Conclusion

Multi-threading today is everywhere, in part due to multi-core architectures Let’s do a ps axuM on my Mac OSX laptop and see how many processes are multi-threaded... When I did this while back writing this slides I got 531 processes and 1126 threads. 217 processess where multithreaded, with up to one process having 56 threads (Firefox with only one tab open!!!) Programming with threads is known to be diﬃcult, and a lot of techniques/tools are available In this course we focus more on how the OS implements threads than how the user uses threads

Quiz next week on this module... Homework Assignment #4...

Henri Casanova ([email protected]) Threads