Technical report UIUCDCS-R-2007-2836, March 2007 MOP: An Efficient and Generic Runtime Verification Framework ∗ Feng Chen Grigore Ros¸u University of Illinois at Urbana-Champaign {fengchen,grosu}@cs.uiuc.edu Abstract 1. Introduction Monitoring-Oriented Programming (MOP 1) [19, 16, 20, 17] Runtime verification [28, 41, 10] aims at combining test- is a formal framework for software development and analy- ing with formal methods in a mutually beneficial way. The sis, in which the developer specifies desired properties using idea underlying runtime verification is that system require- definable specification formalisms, along with code to ex- ments specifications, typically formal and referring to tem- ecute when properties are violated or validated. The MOP poral behaviors and histories of events or actions, are rig- framework automatically generates monitors from the spec- orously checked at runtime against the current execution of ified properties and then integrates them together with the the program, rather than statically against all hypothetical user-defined code into the original system. executions. If used for bug detection, runtime verification The previous design of MOP only allowed specifications gives a rigorous means to state and test complex temporal without parameters, so it could not be used to state and mon- requirements, and is particularly appealing when combined itor safety properties referring to two or more related ob- with test case generation [5] or with steering of programs jects. In this paper we propose a parametric specification- [34]. A large number of runtime verification techniques, al- formalism-independent extension of MOP, together with an gorithms, formalisms, and tools such as Tracematches [2], implementation of JavaMOP that supports parameters. In PQL [37], PTQL [26], MOP [17], Hawk/Eraser [21], MAC our current implementation, parametric specifications are [34], PaX [27], etc., have been and are still being developed, translated into AspectJ code and then weaved into the appli- showing that runtime verification is increasingly adopted not cation using off-the-shelf AspectJ compilers; hence, MOP only by formal methods communities, but also by program- specifications can be seen as formal or logical aspects. ming language designers and software engineers. Our JavaMOP implementation was extensively evaluated We present a parametric extension together with a ma- on two benchmarks, Dacapo [13] and Tracematches [8], ture, optimized and thoroughly evaluated implementation of showing that runtime verification in general and MOP in monitoring-oriented programming (MOP). MOP was first particular are feasible. In some of the examples, millions proposed in 2003 [19] as a software development and anal- of monitor instances are generated, each observing a set of ysis framework based on runtime verification intuitions and related objects. To keep the runtime overhead of monitor- techniques. It was further described and extended in [16, 20, ing and event observation low, we devised and implemented 17], but, up to now, it was not able to handle parameters in a decentralized indexing optimization. Less than 8% of the specifications, and was not shown, through large-scale per- experiments showed more than 10% runtime overhead; in formance tests measuring run-time overhead, to be feasible most cases our tool generates monitoring code as efficient as in practice. An implementation of JavaMOP was carried out the hand-optimized code. Despite its genericity, JavaMOP is to support these, together with decentralized monitor index- empirically shown to be more efficient than runtime verifica- ing algorithms for reducing the runtime overhead. tion systems specialized and optimized for particular speci- As shown in this paper, MOP is expressive, generic, and fication formalisms. Many property violations were detected efficient. MOP logic-plugins encapsulate monitor synthesis during our experiments; some of them are benign, others in- algorithms for logics of interest; these allow users comfort- dicate defects in programs. Many of these are subtle and hard able with formal notation to declare properties using high- to find by ordinary testing. level or application-specific requirements specification for- malisms. Specifications using any of the logic-plugins are allowed to have parameters; this way, multiple monitor in- stances for the same property can coexist, one per collection of objects of interest. MOP also allows its users to imple- ment monitors manually, using the full strength of the target ∗ This material is based upon work supported by the National Science Foun- dation under Grant No. 0448501 and Grant No. 0509321. Any opinions, language. In other words, MOP supports and encourages the findings, and conclusions or recommendations expressed in this material use of formal specifications, but it does not require it. Since are those of the author(s) and do not necessarily reflect the views of the the safety properties are precisely the monitorable ones [40], National Science Foundation. MOP can therefore handle any safety property. 1 Not to be confused with “meta-object protocol” [33]. 1 2007/4/3 Technical report UIUCDCS-R-2007-2836, March 2007 class Resource { /*+MonitorAspect+*/ /*@ public aspect MonitorAspect { scope = class /*+ Generated by JavaMOP for javamop.monitor PTLTL_0 */ logic = PTLTL public boolean[] Resource.PTLTL_0_pre = new boolean[1]; { public boolean[] Resource.PTLTL_0_now = new boolean[1]; event authenticate: end(exec(* authenticate())); pointcut PTLTL_0_Init(Resource thisObject): event access: begin(exec(* access())); execution(Resource.new(..)) && target(thisObject); formula: access -> <*> authenticate; after(Resource thisObject): PTLTL_0_Init(thisObject) { } boolean authenticate = false; violation handler { @this.authenticate(); } boolean access = false; @*/ thisObject.PTLTL_0_now[0] = authenticate; void authenticate() {...} } void access() {...} pointcut PTLTL_0_authenticate0(Resource thisObject): ... target(thisObject) && execution(* Resource.authenticate()); } after (Resource thisObject) returning: PTLTL_0_authenticate0(thisObject) { Figure 1. MOP specification for resource safety boolean authenticate = false; boolean access = false; 1.1 Examples authenticate = true; thisObject.PTLTL_0_pre[0] = thisObject.PTLTL_0_now[0]; Let us consider a simple and common safety property for thisObject.PTLTL_0_now[0] = authenticate || thisObject.PTLTL_0_pre[0]; a shared resource, namely that any access to the resource if (access && ! thisObject.PTLTL_0_now[0]){ should be authenticated. For simplicity, suppose that all thisObject.authenticate(); } } the operations on the shared resource are implemented pointcut PTLTL_0_access0(Resource thisObject): in the class Resource, including methods access() and target(thisObject) && execution(* Resource.access()); before (Resource thisObject): authenticate(). Then the safety property can be speci- PTLTL_0_access0(thisObject) { fied as a trivial “always past” linear temporal logic (LTL) boolean authenticate = false; boolean access = false; formula over method invocations, namely access = true; access -> <*> authenticate, thisObject.PTLTL_0_pre[0] = thisObject.PTLTL_0_now[0]; thisObject.PTLTL_0_now[0] = authenticate || stating that “if access then authenticate held in the past” thisObject.PTLTL_0_pre[0]; (“<*>” reads “eventually in the past”); the “always” part is if (access && ! thisObject.PTLTL_0_now[0]){ thisObject.authenticate(); } implicit, since MOP properties are continuously monitored. } Using MOP like in Figure 1, one can enforce this policy /* Generated code ends +*/ } to hold in any system that manages the resource via the Figure 2. Generated monitor for the property in Figure 1 Resource class; by “enforce” we mean that MOP ensures that the system will satisfy the property even though it was formula, and insert the monitor with the recovery handler not originally programmed (intentionally or not) to satisfy it. into appropriate points of the program. The first line of the MOP specification in Figure 1 states There are two important observations regarding the ex- that this property is a class invariant, i.e., it should hold ample above, each reflecting a crucial aspect of MOP: in the scope of this class (specification attributes are dis- cussed in Section 4.1). The second line chooses a desired 1. By synthesizing monitoring code from specifications and formalism to express the corresponding formal requirement, automatically integrating it together with the recovery in this case past-time LTL (PTLTL); MOP allows users code at relevant points in the program, the developer to “plug-and-play” new specification formalisms, provided can and should have quite a high confidence that the that they respect the standardized interface of logic-plugins resource is used correctly throughout the system. In fact, (these are discussed in Section 3.1). The content enclosed if we trust that the MOP tool generates and integrates the by the curly brackets is specific to the chosen formalism. monitoring code correctly, then we can also trust that the For PTLTL, the user needs to first build an abstraction that resulting system is correct w.r.t. this safety property, no maps runtime events into logical elements, e.g., the in- matter how complicated the system is. vocation of authenticate() being mapped to an event 2. Suppose that authentication-before-access was not a re- authenticate. Using the elements produced by the ab- quirement of the system originally, but that it became a straction, a PTLTL formula is given