Fully Automatic and Precise Detection of Thread Safety Violations
Michael Pradel and Thomas R. Gross
Department of Computer Science ETH Zurich
1 Motivation
Thread-safe classes: Building blocks for concurrent programs
2 Motivation
Thread-safe classes: Building blocks for concurrent programs
2 Motivation
Thread-safe classes: Building blocks for concurrent programs
2 Example from JDK
StringBuffer b = new StringBuffer() b.append("abc")
Thread 1 Thread 2 b.insert(1, b) b.deleteCharAt(1)
3 Example from JDK
StringBuffer b = new StringBuffer() b.append("abc")
Thread 1 Thread 2 b.insert(1, b) b.deleteCharAt(1)
! IndexOutOfBoundsException
Confirmed as bug: Issue #7100996 3 Example from JDK
StringBuffer b = new StringBuffer() b.append("abc") ThreadHow 1 toThread test 2 b.insert(1,thread b) b.deleteCharAt(1) safety?
! IndexOutOfBoundsException
Confirmed as bug: Issue #7100996 3 Goal
Automatic and precise bug detection
Thread Class Tool safety violations
4 Goal
Automatic and precise bug detection
Tests Thread Class Tool safety violations Formal False specs positives
4 Goal
Automatic and precise bug detection
Thread Class Tool safety violations
4 Thread-Safe Classes
“behaves correctly when accessed from multiple threads ... with no additional synchronization ... (in the) calling code” page 18
5 Thread-Safe Classes
“behaves correctly when accessed from multiple threads ... with no additional synchronization ... (in the) calling code” page 18
5 Thread-Safe Classes
“behaves correctly when accessed from multiple threads ... with no additional synchronization ... (in the) calling code” page 18
5 Thread-Safe Classes
“behaves correctly when accessed from multiple threads ... with no additional synchronization ... (in the) calling code” page 18
“operations ... behave as if they occur in some serial order that is consistent with the order of the method calls made by each of the individual threads” StringBuffer API documentation, JDK 6 5 Thread-Safe Classes
“behaves correctly when accessed from multiple threads ... with no additional synchronization ... (in the) calling code” page 18
“operations ... behave as if they occur in some serial order that is consistent with the order of the method calls made by each of the individual threads” StringBuffer API documentation, JDK 6 5 Example
StringBuffer b = new StringBuffer() Thread 1 Thread 2 b.append("a") b.append("c") b.append("b")
6 Example
StringBuffer b = new StringBuffer() Thread 1 Thread 2 b.append("a") b.append("c") b.append("b")
"abc" "cab" "acb" "ac"
6 Example
StringBuffer b = new StringBuffer() Thread 1 Thread 2 b.append("a") b.append("c") b.append("b")
"abc" "cab" "acb" "ac"
6 Example
StringBuffer b = new StringBuffer() Thread 1 Thread 2 b.append("a") b.append("c") b.append("b")
"abc" "cab" "acb" "ac"
6 Example
StringBuffer b = new StringBuffer() Thread 1 Thread 2 b.append("a") b.append("c") b.append("b")
"abc" "cab" "acb" "ac"
6 Example
StringBuffer b = new StringBuffer() Thread 1 Thread 2 b.append("a") b.append("c") b.append("b")
"abc" "cab" "acb" "ac"
6 Approach
Class Generate a under concurrent test test (CUT) Execute
Thread safety Bug oracle
7 Approach
Class Generate a under concurrent test test (CUT) Execute
Thread safety Bug oracle
7 Approach
Class Generate a under concurrent test test (CUT) Execute
Thread safety Bug oracle
7 Generating Concurrent Tests
Example:
StringBuffer b = new StringBuffer() b.append("abc") Thread 1 Thread 2 b.insert(1, b) b.deleteCharAt(1)
8 Generating Concurrent Tests
Sequential prefix: Example: Create and set up CUT instance
StringBuffer b = new StringBuffer() b.append("abc") Thread 1 Thread 2 b.insert(1, b) b.deleteCharAt(1)
8 Generating Concurrent Tests
Example:
StringBuffer b = new StringBuffer() b.append("abc") Thread 1 Thread 2 b.insert(1, b) b.deleteCharAt(1) Concurrent suffixes: Use shared CUT instance 8 Test Generation Algorithm
1. Create prefix
Instantiate CUT
Call methods
2. Create suffixes for prefix
Call methods on shared CUT instance
3. Prefix + two suffixes = test
9 Creating a Prefix
1. Create prefix
Instantiate CUT
Call methods
StringBuffer b = new StringBuffer() b.append("abc") Thread 1 Thread 2 b.insert(1, b) b.deleteCharAt(1)
10 Creating a Prefix
1. Create prefix Randomly
Instantiate CUT select a constructor Call methods
StringBuffer b = new StringBuffer() b.append("abc") Thread 1 Thread 2 b.insert(1, b) b.deleteCharAt(1)
10 Creating a Prefix
1. Create prefix Randomly
Instantiate CUT select a constructor Call methods
StringBuffer b = new StringBuffer() b.append("abc") Thread 1 Thread 2 b.insert(1, b) b.deleteCharAt(1)
10 Creating a Prefix
1. Create prefix After adding a call:
Instantiate CUT Execute
Call methods
StringBuffer b = new StringBuffer() b.append("abc") Thread 1 Thread 2 b.insert(1, b) b.deleteCharAt(1)
10 Creating a Prefix
1. Create prefix After adding a call:
Instantiate CUT Execute
Call methods
StringBuffer b = new StringBuffer() b.append("abc") Thread 1 Thread 2 b.insert(1, b) b.deleteCharAt(1)
10 Creating a Prefix
1. Create prefix Instantiate CUT Randomly
Call methods select a method
StringBuffer b = new StringBuffer() b.append("abc") Thread 1 Thread 2 b.insert(1, b) b.deleteCharAt(1)
10 Creating a Prefix
1. Create prefix Instantiate CUT Randomly
Call methods select a method
StringBuffer b = new StringBuffer() b.append(b.append("abc")/* String */) Thread 1 Thread 2 b.insert(1, b) b.deleteCharAt(1)
10 Creating a Prefix
1. Create prefix Arguments: a) Take available object Instantiate CUT b) Call method returning
Call methods required type c) Random value StringBuffer b = new StringBuffer() b.append(b.append("abc")/* String */) Thread 1 Thread 2 b.insert(1, b) b.deleteCharAt(1)
10 Creating a Prefix
1. Create prefix Arguments: a) Take available object Instantiate CUT b) Call method returning
Call methods required type c) Random value StringBuffer b = new StringBuffer() b.append("abc") Thread 1 Thread 2 b.insert(1, b) b.deleteCharAt(1)
10 Creating a Prefix
1. Create prefix After adding a call:
Instantiate CUT Execute
Call methods
StringBuffer b = new StringBuffer() b.append("abc") Thread 1 Thread 2 b.insert(1, b) b.deleteCharAt(1)
10 Creating a Prefix
1. Create prefix After adding a call:
Instantiate CUT Execute
Call methods
StringBuffer b = new StringBuffer() b.append("abc") Thread 1 Thread 2 b.insert(1, b) b.deleteCharAt(1)
10 Creating a Prefix
1. Create prefix
Instantiate CUT
Call methods
StringBuffer b = new StringBuffer() b.append("abc") Thread 1 Thread 2 b.insert(1, b) b.deleteCharAt(1)
10 Creating Suffixes
2. Create suffixes for prefix
Call methods on shared CUT instance
11 Creating Suffixes
2. Create suffixes for prefix
Call methods on shared CUT instance
StringBuffer b = new StringBuffer() b.append("abc")
b.insert(1, b)
11 Creating Suffixes
2. Create suffixes for prefix Randomly Call methods on select a shared CUT instance method StringBuffer b = new StringBuffer() b.append("abc")
b.insert(1, b)
11 Creating Suffixes
2. Create suffixes for prefix Randomly Call methods on select a shared CUT instance method StringBuffer b = new StringBuffer() b.append("abc")
b.insert(1,b.insert(/* int b) */, /* CharSequence */)
11 Creating Suffixes
Arguments: 2. Create suffixes a) Take available object for prefix b) Call method returning
Call methods on required type shared CUT instance c) Random value StringBuffer b = new StringBuffer() b.append("abc")
b.insert(1,b.insert(/* int b) */, /* CharSequence */)
11 Creating Suffixes
Arguments: 2. Create suffixes a) Take available object for prefix b) Call method returning
Call methods on required type shared CUT instance c) Random value StringBuffer b = new StringBuffer() b.append("abc")
b.insert(1,b.insert(-5, b) b)
11 Creating Suffixes
2. Create suffixes After adding a call: for prefix Execute
Call methods on shared CUT instance
StringBuffer b = new StringBuffer() b.append("abc")
b.insert(1,b.insert(-5, b) b)
11 Creating Suffixes
2. Create suffixes After adding a call: for prefix Execute
Call methods on shared CUT instance
StringBuffer b = new StringBuffer() b.append("abc")
b.insert(1,b.insert(-5, b) b)
! 11 Creating Suffixes
Arguments: 2. Create suffixes a) Take available object for prefix b) Call method returning
Call methods on required type shared CUT instance c) Random value StringBuffer b = new StringBuffer() b.append("abc")
b.insert(1,b.insert(/* int b) */, /* CharSequence */)
11 Creating Suffixes
Arguments: 2. Create suffixes a) Take available object for prefix b) Call method returning
Call methods on required type shared CUT instance c) Random value StringBuffer b = new StringBuffer() b.append("abc")
b.insert(1, b)
11 Creating Suffixes
2. Create suffixes After adding a call: for prefix Execute
Call methods on shared CUT instance
StringBuffer b = new StringBuffer() b.append("abc")
b.insert(1, b)
11 Creating Suffixes
2. Create suffixes After adding a call: for prefix Execute
Call methods on shared CUT instance
StringBuffer b = new StringBuffer() b.append("abc")
b.insert(1, b)
11 Creating Suffixes
2. Create suffixes for prefix
Call methods on shared CUT instance
StringBuffer b = new StringBuffer() b.append("abc")
b.insert(1, b)
11 Creating Suffixes
2. Create suffixes for prefix
Call methods on shared CUT instance
StringBuffer b = new StringBuffer() b.append("abc")
b.insert(1, b) b.deleteCharAt(1)
11 Creating Suffixes
2. Create suffixes After adding a call: for prefix Execute
Call methods on shared CUT instance
StringBuffer b = new StringBuffer() b.append("abc")
b.insert(1, b) b.deleteCharAt(1)
11 Creating Suffixes
2. Create suffixes After adding a call: for prefix Execute
Call methods on shared CUT instance
StringBuffer b = new StringBuffer() b.append("abc")
b.insert(1, b) b.deleteCharAt(1)
11 Creating Suffixes
2. Create suffixes for prefix
Call methods on shared CUT instance
StringBuffer b = new StringBuffer() b.append("abc")
b.insert(1, b) b.deleteCharAt(1)
11 Creating a Test
3. Prefix + two suffixes = test
12 Creating a Test
3. Prefix + two suffixes = test
StringBuffer b = new StringBuffer() b.append("abc")
b.insert(1, b) b.deleteCharAt(1)
12 Creating a Test
3. Prefix + two suffixes = test
Spawn new thread for each suffix
StringBuffer b = new StringBuffer() b.append("abc") Thread 1 Thread 2 b.insert(1, b) b.deleteCharAt(1)
12 Approach
Class Generate a under concurrent test test (CUT) Execute
Thread safety Bug oracle
13 Approach
Class Generate a under concurrent test test (CUT) Execute
Thread safety Bug oracle
13 Thread Safety Oracle
Does the test execution expose a thread safety violation?
Focus on exceptions and deadlocks
Compare concurrent execution to linearizations 14 Assumptions
Concurrency-only crashes are undesired
Matches definition of thread safety
Control over all input to tests
Sequential execution: Deterministic
15 Linearizations
Put all calls into one thread
Preserve order of calls within a thread
16 Linearizations
Put all calls into one thread
Preserve order of calls within a thread
1 2 3
1 1 3 3 2 2
3 1 2 16 The Oracle
Execute concurrently
Exception or No deadlock? Yes Execute linearization All linearizations checked Same No Thread safety failure? violation
Yes 17 The Oracle
Execute concurrently
Exception or No deadlock? Yes Execute linearization All linearizations checked Same No Thread safety failure? violation
Yes 17 The Oracle
Execute concurrently
Exception or No deadlock? Yes Execute linearization All linearizations checked Same No Thread safety failure? violation
Yes 17 The Oracle
Execute concurrently
Exception or No deadlock? Yes Execute linearization All linearizations checked Same No Thread safety failure? violation
Yes 17 The Oracle
Execute concurrently
Exception or No deadlock? Yes Execute linearization All linearizations checked Same No Thread safety failure? violation
Yes 17 The Oracle
Execute concurrently
Exception or No deadlock? Yes Execute linearization All linearizations checked Same No Thread safety failure? violation
Yes 17 The Oracle
Execute concurrently
Exception or No deadlock? Yes Execute linearization All linearizations checked Same No Thread safety failure? violation
Yes 17 The Oracle
Execute concurrently
Exception or No deadlock? Yes Execute linearization All linearizations checked Same No Thread safety failure? violation
Yes 17 Example
StringBuffer b = new StringBuffer() b.append("abc") Thread 1 Thread 2 b.insert(1, b) b.deleteCharAt(1)
18 Example
StringBuffer b = new StringBuffer() b.append("abc") Thread 1 Thread 2 b.insert(1, b) b.deleteCharAt(1) !
18 Example
StringBuffer b = new StringBuffer() b.append("abc") Thread 1 Thread 2 b.insert(1, b) b.deleteCharAt(1) !
StringBuffer b = .. b.append("abc") b.insert(1, b) b.deleteCharAt(1) 18 Example
StringBuffer b = new StringBuffer() b.append("abc") Thread 1 Thread 2 b.insert(1, b) b.deleteCharAt(1) !
StringBuffer b = .. StringBuffer b = .. b.append("abc") b.append("abc") b.insert(1, b) b.deleteCharAt(1) b.deleteCharAt(1) b.insert(1, b) 18 Example
StringBuffer b = new StringBuffer() b.append("abc") Thread 1 Thread 2 b.insert(1, b) b.deleteCharAt(1) ! Thread safety violation
StringBuffer b = .. StringBuffer b = .. b.append("abc") b.append("abc") b.insert(1, b) b.deleteCharAt(1) b.deleteCharAt(1) b.insert(1, b) 18 Properties of the Oracle
Sound but incomplete*
All reported violations are real
Cannot guarantee thread safety
Independent of bug type
Data races
Atomicity violations
Deadlocks
* with respect to incorrectness 19 Implementation
20 Evaluation
1. Effectiveness in finding bugs
2. Performance
Setup:
Thread-safe classes from six Java libraries (e.g., JDK, Apache DBCP)
Intel Xeon (8x3GHz)
21 Bugs
Found 15 bugs and 0 false positives
9 known bugs
6 previously unknown bugs
E.g., in JDK and Apache DBCP
22 Example: Apache DBCP
DataSource ds = new DataSource() Thread 1 Thread 2 ds.setDataSourceName("a") ds.close()
23 Example: Apache DBCP
DataSource ds = new DataSource() Thread 1 Thread 2 ds.setDataSourceName("a") ds.close()
! ConcurrentModificationException Reason: Unsynchronized use of thread-unsafe collection
Confirmed as bug: Issue #369 23 Kinds of Failures
12 of 15 failures are implicit (VM or JDK)
Most common:
NullPointerException
ConcurrentModificationException
24 Performance
25 Performance
8 hours
25 Conclusion
Concurrency: More and more important
Need tools to test thread-safe classes
This work:
Fully automatic testing
Only real bugs reported
26 Thank you!
Try it:
Fully Automatic and Precise Detection of Thread Safety Violations Michael Pradel and Thomas R. Gross, ETH Zurich 27