Runtime monitoring CO444H Ben Livshits 1 Basic Instrumentation • Insert additional code into the program • This code is designed to record important events as they occur at runtime • Some examples • A particular function is being hit or a statement is being hit • This leads to function-level or line-level coverage • Each allocation to measure overall memory allocation 2 Levels of Instrumentation • Native code • Instrument machine code • Tools like LLVM are often used for rewriting • Bytecode • Common for languages such as Java and C# • A variety of tools are available for each bytecode format • JoeQ is in this category as well, although it’s a lot more general • Source code • Common for languages like JavaScript • Often the easiest option – parse the code and add more statements 3 Runtime Code Monitoring •Three major examples of monitoring • Purify/Valgrind • Detecting data races • Detecting memory leaks 4 5 Memory Error Detection Purify • C and C++ are not type-safe • The type system and the runtime fail to enforce type safety • What are some of the examples? • Possible to read and write outside of your intended data structures • Write beyond loop bounds • Or object bounds • Or overwrite the code pointer, etc. 6 Track Each Byte of Memory • Three states for every byte of tracker memory • Unallocated: cannot be read or written • Allocated but not initialized: cannot be read • Allocated and initialized: all operations are allowed 7 Instrumentation for Purify • Check the state of each byte at every access • Binary instrumentation: • Add code before each load and store • 2 bits per byte of memory (3 different states) • 25% memory overhead as a result (8+2) 8 Red Zones • Leave buffer space between allocated objects that is never allocated – red zones • Red zones are unallocated chunks of memory • Guarantees that walking off the end of an array hits unallocated memory 9 Aging Free Memory • When memory is freed, do not reallocate it immediately • Wait until the memory has “aged” somewhat • This helps with catching dangling pointer errors • Red zones are and aging are easily implemented in the malloc library 10 Summary of Purify • Used quite widely • Started with Purify • Now people use Valgrind • An open-source tool • What is the overhead? • Can you use these in production? 11 12 Data Race Detection Data Races • Data races are miltithreaded bugs • At least two threads share a variable or memory location • At least one threat writes to the variable • This is similar to what we did for loop analysis • Races are to be avoided • Typical bug patterns in multithreaded code • Sources of non-determinism • Very hard to reproduce bugs • Why? 13 Not All Races Are Made Equal • We can have data races that involve writes that don’t lead to anything particularly bad • x=1 by two threats – doesn’t matter which one gets to execute first 14 Looking for Data Races • Event A happens before event B if • B follows A in a single thread • A in thread a and B is in thread B, event c such that • c is a sync event after A in a and before B in b • There is a natural partial order on events 15 Early Days of Race Detection • First race tools that is based on happens-before • Monitor all data references • Watch for • Access of v in thread a • Access of v in thread b • No intervening sync between a and b 16 Issues with This Approach • Can be expensive • The approach is fundamentally unsound, i.e. prone to • We need to do a lot of false negatives instrumentation: • Can miss data races • Requires access to all • Needs to be tested with shared variables many execution • All synchronization schedules points 17 What Happens Here? • Thread a • Thread 2 • y=y+1 • lock(m) • lock(m) • unlock(m) • unlock(m) • y=y+1 • How many schedules are there to explore? 18 What Else Can We Do? • What is the proper • Enforce this discipline: programming • Any access to a shared discipline? variable is protected by at least one lock • Most likely, we need to • Any access that is not guard access to shared protected by locks is an variables with locks error 19 Which Lock? • How do we know which • Lock inference: lock protects a • It must be one of the variable? locks that is held at the • A program may have time of accessing the many unrelated locks variable • Links between shared • Initialize C(v) to the set of variables and locks may all locks in the program not be very clear • On access to v by threat t • At runtime, we don’t • C v ← C(푣) ∩ want to do extensive 푙표푐푘푠_ℎ푒푙푑(푡) analysis because of • If C(v) is empty, print an overhead error 20 Complications • It’s not this simple • Uninitialized data • We need to think about • Data initialized by the owner • Uninintialized data • No need to lock access • Read-shared data before initialization • Read-write locks • When does initialization happen? • No good answer at runtime 21 More Complications • Some data is only read • We don’t have to worry about shared reads • We don’t have to update locksets until • More than one thread has the value • At least one thread is writing the value • Keep the lockset algorithm as before but only infer locksets for shared- modified state locations 22 Read-Write Locks • Support a single writer • For each location read but multiple readers • 퐶 푣 ← 퐶 푣 ∩ • Some lock must be held 푙표푐푘푠_ℎ푒푙푑(푡) either in write mode or read mode for all accesses of a shared • For each location write location • 퐶 푣 ← 퐶 푣 ∩ • We separate between 푤푟푡푒_푙표푐푘푠_ℎ푒푙푑(푡) read and write mode locks 23 Implementation Details • Instrument the • Every memory word has a program at the binary shadow word (32 bits) level • 30 bits designed for the lockset key • Could also be done at the level of the source • Sets of locks that are encoded using an integer key in a hashtable • Depends on having not many distinct sets of locks • 2 bits for state in the DFA 24 This is the Basis for a Tool Called Eraser • Works quite well • Can find lots of errors with relatively few runs • However, the overhead is dramatic • 10-30x slowdown • Could be optimized with the help of a static analysis 25 26 Memory Leak Detection Looking for Memory Leaks • Generally, very difficult to find • They manifest themselves over time • Sometimes, it takes hours or days in a long-running program to find a slow memory leak • An issue in production code when these things are not found in testing 27 Basic Idea • Approach: • What is a memory leak • Look for memory leaks using in Java? techniques that are borrowed from garbage • Object that haven’t collection been accessed for a • Any allocated memory that long time has no more pointers to it is • Track the time of considered to be a leak allocation, track the last • It’s possible to run a garbage collector, don’t free any access time, periodically garbage, just detect it and report unused objects report 28 Difficult in C and C++ • While in Java, we can easily tell what portions of the heap are accessible, in C and C++ that is a difficult task • Some of the possibilities: • No pointers to a malloced block at all – garbage • No pointers to the head of a malloced block – likely garbage • How do we identify what is reachable in C/C++? 29.