Fast As a Shadow, Expressive As a Tree: Hybrid Memory Monitoring for C

Fast as a Shadow, Expressive as a Tree: Hybrid Memory Monitoring for C

Nikolai Kosmatov1

with Arvid Jakobsson2, Guillaume Petiot1 and Julien Signoles1

SASEFOR, November 24, 2015 A.Jakobsson, N.Kosmatov, J.Signoles (CEA) Hybrid Memory Monitoring for C 2015-11-24 1 / 48 Outline Context and motivation Frama-C, a platform for analysis of C code Motivation The memory monitoring library An overview Patricia trie model Shadow memory based model The Hybrid model Design principles Illustrating example Dataﬂow analysis An overview How it proceeds Evaluation Conclusion and future work

A.Jakobsson, N.Kosmatov, J.Signoles (CEA) Hybrid Memory Monitoring for C 2015-11-24 2 / 48 Context and motivation Frama-C, a platform for analysis of C code Outline Context and motivation Frama-C, a platform for analysis of C code Motivation The memory monitoring library An overview Patricia trie model Shadow memory based model The Hybrid model Design principles Illustrating example Dataﬂow analysis An overview How it proceeds Evaluation Conclusion and future work

A.Jakobsson, N.Kosmatov, J.Signoles (CEA) Hybrid Memory Monitoring for C 2015-11-24 3 / 48 Context and motivation Frama-C, a platform for analysis of C code A brief history

I 90’s: CAVEAT, Hoare logic-based tool for C code at CEA

I 2000’s: CAVEAT used by Airbus during certiﬁcation process of the A380 (DO-178 level A qualiﬁcation)

I 2002: Why and its C front-end Caduceus (at INRIA)

I 2006: Joint project on a successor to CAVEAT and Caduceus

I 2008: First public release of Frama-C (Hydrogen) I Today: Frama-C Sodium (v.11)

I Multiple projects around the platform I A growing community of users. . . I and of developers

A.Jakobsson, N.Kosmatov, J.Signoles (CEA) Hybrid Memory Monitoring for C 2015-11-24 4 / 48 Context and motivation Frama-C, a platform for analysis of C code Frama-C at a glance

I A Framework forModularAnalysis ofC code

I Developed at CEA LIST in collaboration with INRIA Saclay

I Released under LGPL license

I ACSL annotation language I Extensible plugin oriented platform

I Collaboration of analyses over same code I Inter plugin communication through ACSL formulas I Adding specialized plugins is easy

I http://frama-c.com/ [Kirchner et al. FAC 2015]

A.Jakobsson, N.Kosmatov, J.Signoles (CEA) Hybrid Memory Monitoring for C 2015-11-24 5 / 48 Context and motivation Frama-C, a platform for analysis of C code ACSL: ANSI/ISO C Speciﬁcation Language

I Based on the notion of contract, like in Eiﬀel, JML

I Allows users to specify functional properties of programs

I Allows communication between various plugins

I Independent from a particular analysis

I Manual at http://frama-c.com/acsl

Basic Components

I First-order logic

I Pure C expressions I C types + Z (integer) and R (real) I Built-in predicates and logic functions particularly over pointers: \valid(p) \valid(p+0..2), \separated(p+0..2,q+0..5), \block_length(p)

A.Jakobsson, N.Kosmatov, J.Signoles (CEA) Hybrid Memory Monitoring for C 2015-11-24 6 / 48 Context and motivation Frama-C, a platform for analysis of C code Example: a C program annotated in ACSL

/*@ r e q u i r e s n>=0 && \ v a l i d ( t + ( 0 . . n −1)); a s s i g n s \nothing ; e n s u r e s \ r e s u l t != 0 <==> ( \ f o r a l l integer j ; 0 <= j < n ==> t [ j ] == 0 ) ; */ i n t a l l z e r o s ( i n t t [ ] , i n t n ) { i n t k ; /*@ loop invariant 0 <= k <= n ; loop invariant \ f o r a l l integer j ; 0<=j t [ j ]==0; loop assigns k ; loop variant n−k ; */ f o r ( k = 0 ; k < n ; k++) i f ( t [ k ] != 0) r e t u r n 0 ; r e t u r n 1 ; Can be proven } in Frama-C/WP

A.Jakobsson, N.Kosmatov, J.Signoles (CEA) Hybrid Memory Monitoring for C 2015-11-24 7 / 48 Context and motivation Frama-C, a platform for analysis of C code Main Frama-C plugins

VALUE Aora¨ı Jessie WP Agen Abstract Interpretation Speciﬁcation Generation Mthread Deductive Veriﬁcation Slicing Concurrency Spare code Formal Methods E-ACSL Code Transformation Frama-C Plugins PathCrawler Dynamic Analysis STADY Semantic constant folding LTEST Browsing of unfamiliar code SANTE

Metrics computation Scope & Data-ﬂow browsing

Impact Analysis Variable occurrences

A.Jakobsson, N.Kosmatov, J.Signoles (CEA) Hybrid Memory Monitoring for C 2015-11-24 8 / 48 Context and motivation Motivation Outline Context and motivation Frama-C, a platform for analysis of C code Motivation The memory monitoring library An overview Patricia trie model Shadow memory based model The Hybrid model Design principles Illustrating example Dataﬂow analysis An overview How it proceeds Evaluation Conclusion and future work

A.Jakobsson, N.Kosmatov, J.Signoles (CEA) Hybrid Memory Monitoring for C 2015-11-24 9 / 48 Context and motivation Motivation The C language is risky!

I Low-level operations

I Widely used for critical software

I Lack of security mechanisms Runtime errors are common:

I Division by 0

I Invalid array index

I Invalid pointer

I Non initialized variable

I Out-of-bounds shifting

I Arithmetical overﬂow

I ...

A.Jakobsson, N.Kosmatov, J.Signoles (CEA) Hybrid Memory Monitoring for C 2015-11-24 10 / 48 Context and motivation Motivation Memory safety violations in C programs Spacial safety violations:

I out-of-bounds access, unintialized/manufactured/null pointer access Temporal safety violations:

I dangling stack/heap pointer access (use-after-free), multiple dealloc’s Many advanced techniques and tools proposed, relying on:

I object metadata [Jones & Kelly, VEE’97], [Ruwase & Lam, NDSS’04], [Xu et al, FSE’04], [Djurjati & Adve, ICSE’06], [Arkitidis et al, USENIX’09]

I fat pointers Safe C [Austin et al, PLDI’94], FailSafe C [Oiwa, PLDI’09]

I combined techniques CAWDOR [SCAM’12], MemSafe [Soft.,Pract.Exp.,2013]

I shadow memory Valgrind [PLDI’07], AddressSanitizer [USENIX ATC’12]

Our objective is different: efficiently support runtime checking of memory related annotations for an expressive specification language

A.Jakobsson, N.Kosmatov, J.Signoles (CEA) Hybrid Memory Monitoring for C 2015-11-24 11 / 48 Context and motivation Motivation E-ACSL Language

E-ACSL, an executable subset of ACSL:

I it is veriﬁable in ﬁnite time, suitable for runtime assertion checking

I limitations: only bounded quantiﬁcation, no axioms, no lemmas

I Includes builtin memory-related predicates

E-ACSL2C is a runtime veriﬁcation tool for E-ACSL speciﬁcations: 0 I it translates annotated program p into another program p 0 I p exits with error message if an annotation is violated 0 I otherwise p and p have the same behavior

[Delahaye et al. SAC 2013]

A.Jakobsson, N.Kosmatov, J.Signoles (CEA) Hybrid Memory Monitoring for C 2015-11-24 12 / 48 Monitoring level Byte Byte Block Block Block

Block-level information such as block length and base address is not needed to verify byte-level predicates.

Context and motivation Motivation Motivation

Support eﬃcient runtime checking for memory-related E-ACSL constructs for a pointer p such as:

E-ACSL keyword Its semantics \valid(p) True iff p is valid \initialized(p) True iff ∗p has been initialized \block length(p) Length of p’s memory block \base address(p) Base address of p’s memory block \offset(p) Offset of p in its memory block

A.Jakobsson, N.Kosmatov, J.Signoles (CEA) Hybrid Memory Monitoring for C 2015-11-24 13 / 48 Context and motivation Motivation Motivation

Support eﬃcient runtime checking for memory-related E-ACSL constructs for a pointer p such as:

E-ACSL keyword Its semantics Monitoring level \valid(p) True iff p is valid Byte \initialized(p) True iff ∗p has been initialized Byte \block length(p) Length of p’s memory block Block \base address(p) Base address of p’s memory block Block \offset(p) Offset of p in its memory block Block

Block-level information such as block length and base address is not needed to verify byte-level predicates.

A.Jakobsson, N.Kosmatov, J.Signoles (CEA) Hybrid Memory Monitoring for C 2015-11-24 13 / 48 Context and motivation Motivation Example C program p annotated with E-ACSL i n t a [ ] = {1 ,2 ,3 , 4} , l e n = 4 , i , * a i n v ;

/*@ assert len>0;*/

a inv = malloc( s i z e o f ( i n t )* l e n ) ; f o r ( i = l e n − 1 ; i >= 0 ; i ) { /*@ assert\ v a l i d (a+i);*/

a i n v [ l e n − i − 1 ] = a [ i ] ; } // array a_inv is inversed f r e e ( a i n v ) ;

ai nv

A.Jakobsson, N.Kosmatov, J.Signoles (CEA) Hybrid Memory Monitoring for C 2015-11-24 14 / 48 Context and motivation Motivation Example C program p0 translated by E-ACSL2C (simpliﬁed) i n t a [ ] = {1 ,2 ,3 , 4} , l e n = 4 , i , * a i n v ;

/*@ assert len>0;*/ e a c s l a s s e r t ( l e n > 0 ) ; a inv = malloc( s i z e o f ( i n t )* l e n ) ; f o r ( i = l e n − 1 ; i >= 0 ; i ) { /*@ assert\ v a l i d (a+i);*/ i n t e a c s l v a l i d = v a l i d ( a + i , s i z e o f ( i n t )); e a c s l a s s e r t ( e a c s l v a l i d ) ; a i n v [ l e n − i − 1 ] = a [ i ] ; } // array a_inv is inversed f r e e ( a i n v ) ;

ai nv

A.Jakobsson, N.Kosmatov, J.Signoles (CEA) Hybrid Memory Monitoring for C 2015-11-24 15 / 48 The memory monitoring library An overview Outline Context and motivation Frama-C, a platform for analysis of C code Motivation The memory monitoring library An overview Patricia trie model Shadow memory based model The Hybrid model Design principles Illustrating example Dataﬂow analysis An overview How it proceeds Evaluation Conclusion and future work

A.Jakobsson, N.Kosmatov, J.Signoles (CEA) Hybrid Memory Monitoring for C 2015-11-24 16 / 48 The memory monitoring library An overview The memory monitoring library, an overview

E-ACSL2C performs a non-invasive C code instrumentation of p into p0:

0 I p records memory block (object) metadata in the store

I validity information (including base address, size) whenever a new block is allocated I initialization information whenever a byte is assigned 0 I p queries the store to evaluate memory related E-ACSL constructs

The memory monitoring library provides primitives for record/query operations.

A.Jakobsson, N.Kosmatov, J.Signoles (CEA) Hybrid Memory Monitoring for C 2015-11-24 17 / 48 The memory monitoring library An overview Instrumented program p0 with memory monitoring i n t a [ ] = {1 ,2 , 3 , 4} , l e n = 4 , i , * a i n v ; s t o r e block(& a,16); s t o r e block(& len ,4); s t o r e block(& i ,4); s t o r e b l o c k (& a i n v , 4 ) ; /*@ assert len>0;*/ e a c s l a s s e r t ( l e n > 0 ) ; a i n v = e a c s l m a l l o c ( s i z e o f ( i n t )* l e n ) ; f o r ( i = l e n − 1 ; i >= 0 ; i ) { /*@ assert\ v a l i d (a+i);*/ i n t e a c s l v a l i d = v a l i d ( a + i , s i z e o f ( i n t )); e a c s l a s s e r t ( e a c s l v a l i d ) ; a i n v [ l e n − i − 1 ] = a [ i ] ; } // array a_inv is inversed e a c s l f r e e ( a i n v ) ; d e l e t e b l o c k (& a i n v ) ; d e l e t e b l o c k (& i ) ; d e l e t e block(& len); d e l e t e b l o c k (& a ) ;

A.Jakobsson, N.Kosmatov, J.Signoles (CEA) Hybrid Memory Monitoring for C 2015-11-24 18 / 48 The memory monitoring library An overview Instrumented program p0 with memory monitoring i n t a [ ] = {1 ,2 , 3 , 4} , l e n = 4 , i , * a i n v ; s t o r e block(& a,16); s t o r e block(& len ,4); s t o r e block(& i ,4); s t o r e b l o c k (& a i n v , 4 ) ; /*@ assert len>0;*/ e a c s l a s s e r t ( l e n > 0 ) ; a i n v = e a c s l m a l l o c ( s i z e o f ( i n t )* l e n ) ; f o r ( i = l e n − 1 ; i >= 0 ; i ) { /*@ assert\ v a l i d (a+i);*/ i n t e a c s l v a l i d = v a l i d ( a + i , s i z e o f ( i n t )); e a c s l a s s e r t ( e a c s l v a l i d ) ; a i n v [ l e n − i − 1 ] = a [ i ] ; } // array a_inv is inversed e a c s l f r e e ( a i n v ) ; d e l e t e b l o c k (& a i n v ) ; d e l e t e b l o c k (& i ) ; d e l e t e block(& len); d e l e t e b l o c k (& a ) ; Memory model: {(a, 16)} A.Jakobsson, N.Kosmatov, J.Signoles (CEA) Hybrid Memory Monitoring for C 2015-11-24 18 / 48 The memory monitoring library An overview Instrumented program p0 with memory monitoring i n t a [ ] = {1 ,2 , 3 , 4} , l e n = 4 , i , * a i n v ; s t o r e block(& a,16); s t o r e block(& len ,4); s t o r e block(& i ,4); s t o r e b l o c k (& a i n v , 4 ) ; /*@ assert len>0;*/ e a c s l a s s e r t ( l e n > 0 ) ; a i n v = e a c s l m a l l o c ( s i z e o f ( i n t )* l e n ) ; f o r ( i = l e n − 1 ; i >= 0 ; i ) { /*@ assert\ v a l i d (a+i);*/ i n t e a c s l v a l i d = v a l i d ( a + i , s i z e o f ( i n t )); e a c s l a s s e r t ( e a c s l v a l i d ) ; a i n v [ l e n − i − 1 ] = a [ i ] ; } // array a_inv is inversed e a c s l f r e e ( a i n v ) ; d e l e t e b l o c k (& a i n v ) ; d e l e t e b l o c k (& i ) ; d e l e t e block(& len); d e l e t e b l o c k (& a ) ; Memory model: {(a, 16), (&len, 4)} A.Jakobsson, N.Kosmatov, J.Signoles (CEA) Hybrid Memory Monitoring for C 2015-11-24 18 / 48 The memory monitoring library An overview Instrumented program p0 with memory monitoring i n t a [ ] = {1 ,2 , 3 , 4} , l e n = 4 , i , * a i n v ; s t o r e block(& a,16); s t o r e block(& len ,4); s t o r e block(& i ,4); s t o r e b l o c k (& a i n v , 4 ) ; /*@ assert len>0;*/ e a c s l a s s e r t ( l e n > 0 ) ; a i n v = e a c s l m a l l o c ( s i z e o f ( i n t )* l e n ) ; f o r ( i = l e n − 1 ; i >= 0 ; i ) { /*@ assert\ v a l i d (a+i);*/ i n t e a c s l v a l i d = v a l i d ( a + i , s i z e o f ( i n t )); e a c s l a s s e r t ( e a c s l v a l i d ) ; a i n v [ l e n − i − 1 ] = a [ i ] ; } // array a_inv is inversed e a c s l f r e e ( a i n v ) ; d e l e t e b l o c k (& a i n v ) ; d e l e t e b l o c k (& i ) ; d e l e t e block(& len); d e l e t e b l o c k (& a ) ; Memory model: {(a, 16), (&len, 4), (&i, 4)} A.Jakobsson, N.Kosmatov, J.Signoles (CEA) Hybrid Memory Monitoring for C 2015-11-24 18 / 48 The memory monitoring library An overview Instrumented program p0 with memory monitoring i n t a [ ] = {1 ,2 , 3 , 4} , l e n = 4 , i , * a i n v ; s t o r e block(& a,16); s t o r e block(& len ,4); s t o r e block(& i ,4); s t o r e b l o c k (& a i n v , 4 ) ; /*@ assert len>0;*/ e a c s l a s s e r t ( l e n > 0 ) ; a i n v = e a c s l m a l l o c ( s i z e o f ( i n t )* l e n ) ; f o r ( i = l e n − 1 ; i >= 0 ; i ) { /*@ assert\ v a l i d (a+i);*/ i n t e a c s l v a l i d = v a l i d ( a + i , s i z e o f ( i n t )); e a c s l a s s e r t ( e a c s l v a l i d ) ; a i n v [ l e n − i − 1 ] = a [ i ] ; } // array a_inv is inversed e a c s l f r e e ( a i n v ) ; d e l e t e b l o c k (& a i n v ) ; d e l e t e b l o c k (& i ) ; d e l e t e block(& len); d e l e t e b l o c k (& a ) ; Memory model: {(a, 16), (&len, 4), (&i, 4), (&a inv, 4)} A.Jakobsson, N.Kosmatov, J.Signoles (CEA) Hybrid Memory Monitoring for C 2015-11-24 18 / 48 The memory monitoring library An overview Instrumented program p0 with memory monitoring i n t a [ ] = {1 ,2 , 3 , 4} , l e n = 4 , i , * a i n v ; s t o r e block(& a,16); s t o r e block(& len ,4); s t o r e block(& i ,4); s t o r e b l o c k (& a i n v , 4 ) ; /*@ assert len>0;*/ e a c s l a s s e r t ( l e n > 0 ) ; a i n v = e a c s l m a l l o c ( s i z e o f ( i n t )* l e n ) ; f o r ( i = l e n − 1 ; i >= 0 ; i ) { /*@ assert\ v a l i d (a+i);*/ i n t e a c s l v a l i d = v a l i d ( a + i , s i z e o f ( i n t )); e a c s l a s s e r t ( e a c s l v a l i d ) ; a i n v [ l e n − i − 1 ] = a [ i ] ; } // array a_inv is inversed e a c s l f r e e ( a i n v ) ; d e l e t e b l o c k (& a i n v ) ; d e l e t e b l o c k (& i ) ; d e l e t e block(& len); d e l e t e b l o c k (& a ) ; Memory model: {(a, 16), (&len, 4), (&i, 4), (&a inv, 4), (a inv, 16)} A.Jakobsson, N.Kosmatov, J.Signoles (CEA) Hybrid Memory Monitoring for C 2015-11-24 18 / 48 The memory monitoring library An overview Instrumented program p0 with memory monitoring i n t a [ ] = {1 ,2 , 3 , 4} , l e n = 4 , i , * a i n v ; s t o r e block(& a,16); s t o r e block(& len ,4); s t o r e block(& i ,4); s t o r e b l o c k (& a i n v , 4 ) ; /*@ assert len>0;*/ e a c s l a s s e r t ( l e n > 0 ) ; a i n v = e a c s l m a l l o c ( s i z e o f ( i n t )* l e n ) ; f o r ( i = l e n − 1 ; i >= 0 ; i ) { /*@ assert\ v a l i d (a+i);*/ i n t e a c s l v a l i d = v a l i d ( a + i , s i z e o f ( i n t )); e a c s l a s s e r t ( e a c s l v a l i d ) ; a i n v [ l e n − i − 1 ] = a [ i ] ; } // array a_inv is inversed e a c s l f r e e ( a i n v ) ; d e l e t e b l o c k (& a i n v ) ; d e l e t e b l o c k (& i ) ; d e l e t e block(& len); d e l e t e b l o c k (& a ) ; Memory model: {(a, 16), (&len, 4), (&i, 4), (&a inv, 4), (a inv, 16)} A.Jakobsson, N.Kosmatov, J.Signoles (CEA) Hybrid Memory Monitoring for C 2015-11-24 18 / 48 The memory monitoring library An overview Instrumented program p0 with memory monitoring i n t a [ ] = {1 ,2 , 3 , 4} , l e n = 4 , i , * a i n v ; s t o r e block(& a,16); s t o r e block(& len ,4); s t o r e block(& i ,4); s t o r e b l o c k (& a i n v , 4 ) ; /*@ assert len>0;*/ e a c s l a s s e r t ( l e n > 0 ) ; a i n v = e a c s l m a l l o c ( s i z e o f ( i n t )* l e n ) ; f o r ( i = l e n − 1 ; i >= 0 ; i ) { /*@ assert\ v a l i d (a+i);*/ i n t e a c s l v a l i d = v a l i d ( a + i , s i z e o f ( i n t )); e a c s l a s s e r t ( e a c s l v a l i d ) ; a i n v [ l e n − i − 1 ] = a [ i ] ; } // array a_inv is inversed e a c s l f r e e ( a i n v ) ; d e l e t e b l o c k (& a i n v ) ; d e l e t e b l o c k (& i ) ; d e l e t e block(& len); d e l e t e b l o c k (& a ) ; Memory model: {(a, 16), (&len, 4), (&i, 4), (&a inv, 4), (a inv, 16)} A.Jakobsson, N.Kosmatov, J.Signoles (CEA) Hybrid Memory Monitoring for C 2015-11-24 18 / 48 The memory monitoring library An overview Instrumented program p0 with memory monitoring i n t a [ ] = {1 ,2 , 3 , 4} , l e n = 4 , i , * a i n v ; s t o r e block(& a,16); s t o r e block(& len ,4); s t o r e block(& i ,4); s t o r e b l o c k (& a i n v , 4 ) ; /*@ assert len>0;*/ e a c s l a s s e r t ( l e n > 0 ) ; a i n v = e a c s l m a l l o c ( s i z e o f ( i n t )* l e n ) ; f o r ( i = l e n − 1 ; i >= 0 ; i ) { /*@ assert\ v a l i d (a+i);*/ i n t e a c s l v a l i d = v a l i d ( a + i , s i z e o f ( i n t )); e a c s l a s s e r t ( e a c s l v a l i d ) ; a i n v [ l e n − i − 1 ] = a [ i ] ; } // array a_inv is inversed e a c s l f r e e ( a i n v ) ; d e l e t e b l o c k (& a i n v ) ; d e l e t e b l o c k (& i ) ; d e l e t e block(& len); d e l e t e b l o c k (& a ) ; Memory model: {(a, 16), (&len, 4), (&i, 4), (&a inv, 4), (a inv, 16)} A.Jakobsson, N.Kosmatov, J.Signoles (CEA) Hybrid Memory Monitoring for C 2015-11-24 18 / 48 The memory monitoring library An overview Instrumented program p0 with memory monitoring i n t a [ ] = {1 ,2 , 3 , 4} , l e n = 4 , i , * a i n v ; s t o r e block(& a,16); s t o r e block(& len ,4); s t o r e block(& i ,4); s t o r e b l o c k (& a i n v , 4 ) ; /*@ assert len>0;*/ e a c s l a s s e r t ( l e n > 0 ) ; a i n v = e a c s l m a l l o c ( s i z e o f ( i n t )* l e n ) ; f o r ( i = l e n − 1 ; i >= 0 ; i ) { /*@ assert\ v a l i d (a+i);*/ i n t e a c s l v a l i d = v a l i d ( a + i , s i z e o f ( i n t )); e a c s l a s s e r t ( e a c s l v a l i d ) ; a i n v [ l e n − i − 1 ] = a [ i ] ; } // array a_inv is inversed e a c s l f r e e ( a i n v ) ; d e l e t e b l o c k (& a i n v ) ; d e l e t e b l o c k (& i ) ; d e l e t e block(& len); d e l e t e b l o c k (& a ) ; Memory model: {(a, 16), (&len, 4), (&i, 4), (&a inv, 4), (a inv, 16)} A.Jakobsson, N.Kosmatov, J.Signoles (CEA) Hybrid Memory Monitoring for C 2015-11-24 18 / 48 The memory monitoring library An overview Instrumented program p0 with memory monitoring i n t a [ ] = {1 ,2 , 3 , 4} , l e n = 4 , i , * a i n v ; s t o r e block(& a,16); s t o r e block(& len ,4); s t o r e block(& i ,4); s t o r e b l o c k (& a i n v , 4 ) ; /*@ assert len>0;*/ e a c s l a s s e r t ( l e n > 0 ) ; a i n v = e a c s l m a l l o c ( s i z e o f ( i n t )* l e n ) ; f o r ( i = l e n − 1 ; i >= 0 ; i ) { /*@ assert\ v a l i d (a+i);*/ i n t e a c s l v a l i d = v a l i d ( a + i , s i z e o f ( i n t )); e a c s l a s s e r t ( e a c s l v a l i d ) ; a i n v [ l e n − i − 1 ] = a [ i ] ; } // array a_inv is inversed e a c s l f r e e ( a i n v ) ; d e l e t e b l o c k (& a i n v ) ; d e l e t e b l o c k (& i ) ; d e l e t e block(& len); d e l e t e b l o c k (& a ) ; Memory model: {(a, 16), (&len, 4), (&i, 4), (&a inv, 4), (a inv, 16)} A.Jakobsson, N.Kosmatov, J.Signoles (CEA) Hybrid Memory Monitoring for C 2015-11-24 18 / 48 The memory monitoring library An overview The problem:

How to store and extract this information eﬃciently?

A.Jakobsson, N.Kosmatov, J.Signoles (CEA) Hybrid Memory Monitoring for C 2015-11-24 19 / 48 The memory monitoring library Patricia trie model Outline Context and motivation Frama-C, a platform for analysis of C code Motivation The memory monitoring library An overview Patricia trie model Shadow memory based model The Hybrid model Design principles Illustrating example Dataﬂow analysis An overview How it proceeds Evaluation Conclusion and future work

A.Jakobsson, N.Kosmatov, J.Signoles (CEA) Hybrid Memory Monitoring for C 2015-11-24 20 / 48 The memory monitoring library Patricia trie model Model 1: Patricia trie A Patricia trie (compact preﬁx trie) is an optimized preﬁx trie, where

I leaves contain stored keys (here, block base addresses) I any internal node contains greatest common preﬁx of all its successors Example of a Patricia trie a) before, and b) after inserting 0010 0111

a) 0010 **** b) 0010 ****

0010 0110 0010 1*** 0010 011* 0010 1***

0010 1001 0010 1101 0010 0110 0010 0111 0010 1001 0010 1101

Advantages of a Patricia trie: supports byte- and block-level predicates

I sorted structure (e.g. closest predecessor searched for \base_addr) I size of metadata not limited Disadvantage:

I look-up of metadata in O(k) steps (word length k=32 or 64)

A.Jakobsson, N.Kosmatov, J.Signoles (CEA) Hybrid Memory Monitoring for C 2015-11-24 21 / 48 The memory monitoring library Patricia trie model Experiments: comparison to other data structures

Our implementation with a Patricia trie is in average

I 2500 times faster than linked lists

I 200 times faster than unbalanced binary search trees

I linear worst case complexity !

I 27 times faster than Splay trees (can be 3 times slower or >500 times faster depending on examples)

I Splay trees move recently accessed elements to the top I it pays if frequent successive accesses to the same blocks I waste of time if successive accesses to diﬀerent blocks (ex. big matrix multiplication)

A.Jakobsson, N.Kosmatov, J.Signoles (CEA) Hybrid Memory Monitoring for C 2015-11-24 22 / 48 The memory monitoring library Patricia trie model Optimized records and queries in the store

Queries in the store intensively use greatest common preﬁx computation to decide which branch to follow or which common predecessor to add

Here, the Patricia trie stores binary numbers as keys (base addresses)

I a linear search to compute greatest common preﬁx can be avoided,

I eﬃcient logarithmic dichotomic search can be used instead

Our greatest common preﬁx uses dichotomic search optimized by

I bit operations

I pre-computed masks

I pre-computed next step indices (no need for (high+low)/2)

A.Jakobsson, N.Kosmatov, J.Signoles (CEA) Hybrid Memory Monitoring for C 2015-11-24 23 / 48 The memory monitoring library Patricia trie model Optimized records and queries in the store, cont’d

Greatest common preﬁx mask by dichotomic search for 8-bit words:

typedef unsigned char byte; // index 0 1 2 3 4 5 6 7 8 byte masks[] = {0x00,0x80,0xC0,0xE0,0xF0,0xF8,0xFC,0xFE,0xFF}; int longer [] = { 0, -1, 3, -3, 6, -5, 7, 8, -8}; int shorter[] = { 0, 0, 1, -2, 2, -4, 5, -6, -7}; byte gtCommonPrefixMask(byte a, byte b) { byte nxor = ~(a ^ b); // a bit = 1 iff this bit is equal in a and b int i = 4; // search starts in the middle of the word while(i > 0) // if more comparisons needed if (nxor >= masks[i]) i = longer[i]; // if first i bits equal,try a longer prefix else i = shorter[i]; // otherwise, try a shorter prefix return masks[-i]; // if i<=0, masks[-i] is the answer }

A.Jakobsson, N.Kosmatov, J.Signoles (CEA) Hybrid Memory Monitoring for C 2015-11-24 24 / 48 The memory monitoring library Patricia trie model Experiments: optimized greatest common preﬁx

Our optimized greatest common preﬁx makes the execution of the instrumented code

I in average 2.7 times faster

I going up to 4.7 times faster on some examples

compared to a linear search optimized only by bit operations

[Kosmatov et al. RV 2013]

A.Jakobsson, N.Kosmatov, J.Signoles (CEA) Hybrid Memory Monitoring for C 2015-11-24 25 / 48 The memory monitoring library Shadow memory based model Outline Context and motivation Frama-C, a platform for analysis of C code Motivation The memory monitoring library An overview Patricia trie model Shadow memory based model The Hybrid model Design principles Illustrating example Dataﬂow analysis An overview How it proceeds Evaluation Conclusion and future work

A.Jakobsson, N.Kosmatov, J.Signoles (CEA) Hybrid Memory Monitoring for C 2015-11-24 26 / 48 The memory monitoring library Shadow memory based model Model 2: Shadow memory Inspired by the shadow memory technique:

I The shadow memory is a large array of metadata

I Each byte of user memory (allocated or not) corresponds to one element of this array

I Mapping user memory to shadow memory in constant time

Memory Shadow

b1 sh1

b2 sh2

b3 sh3 ......

A.Jakobsson, N.Kosmatov, J.Signoles (CEA) Hybrid Memory Monitoring for C 2015-11-24 27 / 48 The memory monitoring library Shadow memory based model Model 2: Shadow memory, continued

Program memory Stack ... Shadow for the stack Shadow for the heap ... Heap

Advantage of Shadow memory: I constant-time lookups of metadata Disadvantages: I size of metadata is limited, only supports byte-level predicates I cannot cover all memory in general (but does for most programs) A.Jakobsson, N.Kosmatov, J.Signoles (CEA) Hybrid Memory Monitoring for C 2015-11-24 28 / 48 The Hybrid model Design principles Outline Context and motivation Frama-C, a platform for analysis of C code Motivation The memory monitoring library An overview Patricia trie model Shadow memory based model The Hybrid model Design principles Illustrating example Dataﬂow analysis An overview How it proceeds Evaluation Conclusion and future work

A.Jakobsson, N.Kosmatov, J.Signoles (CEA) Hybrid Memory Monitoring for C 2015-11-24 29 / 48 The Hybrid model Design principles Proposed solution: Hybrid model

Objective: Reconcile the beneﬁts of the tree-based and shadow-memory-based storage.

Idea: Use both models in tandem. Place blocks in one or the other model, depending on:

I the size of the block,

I the metadata needed for that block.

A.Jakobsson, N.Kosmatov, J.Signoles (CEA) Hybrid Memory Monitoring for C 2015-11-24 30 / 48 The Hybrid model Design principles Design principles of the Hybrid model, 1/3

Storage P1 Metadata of all memory locations that require block-level monitoring should be stored in the Patricia trie store. P2 For memory locations that require byte-level monitoring, their metadata are typically stored in the shadow memory store. P3 If a block does not completely belong to the interval of addresses supported by the shadow memory, its metadata should be stored in the Patricia trie store. P4 For blocks longer than a (parameterizable) constant length C, the metadata should be stored in the Patricia trie store.

A.Jakobsson, N.Kosmatov, J.Signoles (CEA) Hybrid Memory Monitoring for C 2015-11-24 31 / 48 The Hybrid model Design principles Principles of the Hybrid model, 2/3

Evaluation P5 The evaluation of block-level predicates should always query the Patricia trie. P6 For predicates that require byte-level monitoring, the evaluation ﬁrst tries to ﬁnd the required information in the shadow memory store. If the block is unknown in the shadow mem ry store, it queries the Patricia trie store.

A.Jakobsson, N.Kosmatov, J.Signoles (CEA) Hybrid Memory Monitoring for C 2015-11-24 32 / 48 The Hybrid model Design principles Principles of the Hybrid model, 3/3

Deallocation P7 The deallocation of an existing block ﬁrst tries to remove the block metadata from the shadow memory store. If the block is unknown in the shadow memory store, it queries and tries to remove it from the Patricia trie store.

A.Jakobsson, N.Kosmatov, J.Signoles (CEA) Hybrid Memory Monitoring for C 2015-11-24 33 / 48 The Hybrid model Illustrating example Outline Context and motivation Frama-C, a platform for analysis of C code Motivation The memory monitoring library An overview Patricia trie model Shadow memory based model The Hybrid model Design principles Illustrating example Dataﬂow analysis An overview How it proceeds Evaluation Conclusion and future work

A.Jakobsson, N.Kosmatov, J.Signoles (CEA) Hybrid Memory Monitoring for C 2015-11-24 34 / 48 The Hybrid model Illustrating example Proposed solution: Hybrid model, example

Hybrid model

Patricia trie char a[128]; char b; 0010 **** 0010 0110 0010 1*** /*@ assert 0010 1001 0010 1101 \block_length(b) == 1 ; */ char c;

Shadow memory /*@ assert b1 sh1 \block_length(a) == 128; */ b2 sh2

b3 sh3 char *p = &a[0]; ...... /*@ assert \valid(p) ; */

Advantages of the Hybrid: I supports byte- and block-level predicates I as fast as the shadow, when only byte-level monitoring is needed

A.Jakobsson, N.Kosmatov, J.Signoles (CEA) Hybrid Memory Monitoring for C 2015-11-24 35 / 48 The Hybrid model Illustrating example Proposed solution: Hybrid model, example

Hybrid model

Patricia trie char a[128]; char b; 0010 **** 0010 0110 0010 1*** /*@ assert 0010 1001 0010 1101 \block_length(b) == 1 ; */ char c;

Shadow memory /*@ assert b1 sh1 \block_length(a) == 128; */ b2 sh2

b3 sh3 char *p = &a[0]; ...... /*@ assert \valid(p) ; */

Advantages of the Hybrid: I supports byte- and block-level predicates I as fast as the shadow, when only byte-level monitoring is needed

A.Jakobsson, N.Kosmatov, J.Signoles (CEA) Hybrid Memory Monitoring for C 2015-11-24 35 / 48 The Hybrid model Illustrating example Proposed solution: Hybrid model, example

Hybrid model

Patricia trie char a[128]; char b; 0010 **** 0010 0110 0010 1*** /*@ assert 0010 1001 0010 1101 \block_length(b) == 1 ; */ char c;

Shadow memory /*@ assert b1 sh1 \block_length(a) == 128; */ b2 sh2

b3 sh3 char *p = &a[0]; ...... /*@ assert \valid(p) ; */

Advantages of the Hybrid: I supports byte- and block-level predicates I as fast as the shadow, when only byte-level monitoring is needed

A.Jakobsson, N.Kosmatov, J.Signoles (CEA) Hybrid Memory Monitoring for C 2015-11-24 35 / 48 The Hybrid model Illustrating example Proposed solution: Hybrid model, example

Hybrid model

Patricia trie char a[128]; char b; 0010 **** 0010 0110 0010 1*** /*@ assert 0010 1001 0010 1101 \block_length(b) == 1 ; */ char c;

Shadow memory /*@ assert b1 sh1 \block_length(a) == 128; */ b2 sh2

b3 sh3 char *p = &a[0]; ...... /*@ assert \valid(p) ; */

Advantages of the Hybrid: I supports byte- and block-level predicates I as fast as the shadow, when only byte-level monitoring is needed

A.Jakobsson, N.Kosmatov, J.Signoles (CEA) Hybrid Memory Monitoring for C 2015-11-24 35 / 48 The Hybrid model Illustrating example Proposed solution: Hybrid model, example

Hybrid model

Patricia trie char a[128]; char b; 0010 **** 0010 0110 0010 1*** /*@ assert 0010 1001 0010 1101 \block_length(b) == 1 ; */ char c;

Shadow memory /*@ assert b1 sh1 \block_length(a) == 128; */ b2 sh2

b3 sh3 char *p = &a[0]; ...... /*@ assert \valid(p) ; */

Advantages of the Hybrid: I supports byte- and block-level predicates I as fast as the shadow, when only byte-level monitoring is needed

A.Jakobsson, N.Kosmatov, J.Signoles (CEA) Hybrid Memory Monitoring for C 2015-11-24 35 / 48 The Hybrid model Illustrating example Proposed solution: Hybrid model, example

Hybrid model

Patricia trie char a[128]; char b; 0010 **** 0010 0110 0010 1*** /*@ assert 0010 1001 0010 1101 \block_length(b) == 1 ; */ char c;

Shadow memory /*@ assert b1 sh1 \block_length(a) == 128; */ b2 sh2

b3 sh3 char *p = &a[0]; ...... /*@ assert \valid(p) ; */

Advantages of the Hybrid: I supports byte- and block-level predicates I as fast as the shadow, when only byte-level monitoring is needed

A.Jakobsson, N.Kosmatov, J.Signoles (CEA) Hybrid Memory Monitoring for C 2015-11-24 35 / 48 The Hybrid model Illustrating example Proposed solution: Hybrid model, example

Hybrid model

Patricia trie char a[128]; char b; 0010 **** 0010 0110 0010 1*** /*@ assert 0010 1001 0010 1101 \block_length(b) == 1 ; */ char c;

Shadow memory /*@ assert b1 sh1 \block_length(a) == 128; */ b2 sh2

b3 sh3 char *p = &a[0]; ...... /*@ assert \valid(p) ; */

Advantages of the Hybrid: I supports byte- and block-level predicates I as fast as the shadow, when only byte-level monitoring is needed

A.Jakobsson, N.Kosmatov, J.Signoles (CEA) Hybrid Memory Monitoring for C 2015-11-24 35 / 48 Dataﬂow analysis An overview Outline Context and motivation Frama-C, a platform for analysis of C code Motivation The memory monitoring library An overview Patricia trie model Shadow memory based model The Hybrid model Design principles Illustrating example Dataﬂow analysis An overview How it proceeds Evaluation Conclusion and future work

A.Jakobsson, N.Kosmatov, J.Signoles (CEA) Hybrid Memory Monitoring for C 2015-11-24 36 / 48 Dataﬂow analysis An overview Dataﬂow analysis

Goal: reduce irrelevant instrumentation

I Backward dataﬂow analaysis I Computes an over-approximation

I of left-values to be monitored

I and their observation purpose (validity, initialization, . . . )

I Ignores (approximates) oﬀsets

I Parameterized by an alias analysis

[Jakobsson et al. JFLA 2015]

A.Jakobsson, N.Kosmatov, J.Signoles (CEA) Hybrid Memory Monitoring for C 2015-11-24 37 / 48 Dataﬂow analysis An overview Instrumented program p0 after pre-analysis: what is useful? i n t a [ ] = {1 ,2 , 3 , 4} , l e n = 4 , i , * a i n v ; s t o r e block(a,16); s t o r e block(& len ,4); s t o r e block(& i ,4); s t o r e b l o c k (& a i n v , 4 ) ; /*@ a s s e r t l e n > 0 ; */ e a c s l a s s e r t ( l e n > 0 ) ; a i n v = e a c s l m a l l o c ( s i z e o f ( i n t )* l e n ) ; f o r ( i = l e n − 1 ; i >= 0 ; i −−) { /*@ a s s e r t \ v a l i d ( a + i ) ; */ i n t e a c s l v a l i d = v a l i d ( a + i , s i z e o f ( i n t )); e a c s l a s s e r t ( e a c s l v a l i d ) ; a i n v [ l e n − i − 1 ] = a [ i ] ; } e a c s l f r e e ( a i n v ) ; d e l e t e b l o c k (& a i n v ) ; d e l e t e b l o c k (& i ) ; d e l e t e block(& len); d e l e t e b l o c k ( a ) ;

A.Jakobsson, N.Kosmatov, J.Signoles (CEA) Hybrid Memory Monitoring for C 2015-11-24 38 / 48 Dataﬂow analysis How it proceeds Outline Context and motivation Frama-C, a platform for analysis of C code Motivation The memory monitoring library An overview Patricia trie model Shadow memory based model The Hybrid model Design principles Illustrating example Dataﬂow analysis An overview How it proceeds Evaluation Conclusion and future work

A.Jakobsson, N.Kosmatov, J.Signoles (CEA) Hybrid Memory Monitoring for C 2015-11-24 39 / 48 Dataﬂow analysis How it proceeds Illustrative example

void f(int c) { char *p; intx=0; if (c) { p = (char*)malloc(sizeof(char) * 4); for(int k = 0; k < 4; i++) *(p+k) = (char)k; } else { x = 1; p = (char*)&x; } *(p +3) = 5; /*@ assert ∀ integer k; 0<=k<4 =⇒ \valid(p+k) ∧ \initialized(p+k); */ }

A.Jakobsson, N.Kosmatov, J.Signoles (CEA) Hybrid Memory Monitoring for C 2015-11-24 40 / 48 Dataﬂow analysis How it proceeds Illustrative example, continued

0 x=0 2 1 7 ?c ?!c p=alloc(4) x=1 3 8 k=0 p=&x 4 9 ?!(k<4) ?k<4 5 10 *(p+k)=k *(p+3)=5 6 11 ?check(p+k) 12

A.Jakobsson, N.Kosmatov, J.Signoles (CEA) Hybrid Memory Monitoring for C 2015-11-24 41 / 48 p 7→ v, i;&x 7→ i p 7→ v, i;&x 7→ i p 7→ v, i;&x 7→ i p 7→ i;&x 7→ i p 7→ i p 7→ v, i;&x 7→ i p 7→ v, i;&x 7→ i

p 7→ v, i;&x 7→ i p 7→ i;&x 7→ i p 7→ i;&x 7→ i p 7→ i