Repframe: an Efficient and Transparent Framework For
Total Page:16
File Type:pdf, Size:1020Kb
REPFRAME: An Efficient and Transparent Framework for Dynamic Program Analysis Heming Cui+*, Rui Gu*, Cheng Liu*, and Junfeng Yang* +The University of Hong Kong *Columbia University Abstract sis frameworks can be classified into two approaches de- pending on how a framework transfers an application’s Dynamic program analysis frameworks greatly improve execution states to an analysis tool. Traditional analysis software quality as they enable a wide range of pow- frameworks [8, 31, 34, 38, 41] take a “fully-coupled” ap- erful analysis tools (e.g., reliability, profiling, and log- proach: a framework in-lines an analysis with the execu- ging) at runtime. However, because existing frameworks tion, then the analysis can inspect all or most execution run only one actual execution for each software applica- states. However, this approach can easily let the anal- tion, the execution is fully or partially coupled with an ysis slow down the actual execution. To improve per- analysis tool in order to transfer execution states (e.g., formance, leveraging a fact that many analyses can be accessed memory and thread interleavings) to the anal- done asynchronously with the actual execution, some re- ysis tool, easily causing a prohibitive slowdown for the cent frameworks [10, 20, 22, 35, 42, 43] take a second execution. To reduce the portions of execution states that “partially-decoupled” approach: only analysis-critical require transfer, many frameworks require significantly execution states (e.g., effective memory addresses and carving analysis tools as well as the frameworks them- thread interleavings) are transferred to the analysis run- selves. Thus, these frameworks significantly trade off ning on other CPU cores, then the actual execution can transparency with analysis tools and allow only one type be less perturbed by the analysis. of tools to run within one execution. This paper presents REPFRAME, an efficient and Unfortunately, despite these great effort, most exist- transparent framework that fully decouples execution ing analysis frameworks are still hard to deploy in pro- and analysis by constructing multiple equivalent execu- duction runs, mainly due to three problems. The first tions. To do so, REPFRAME leverages a recent fault- problem is still performance. Traditional frameworks tolerant technique: transparent state machine replica- fully couple the analysis with the actual execution us- tion, which runs the same software application on a set ing the shadow memory approach, so whenever an anal- of machines (or replicas), and ensures that all replicas ysis does some heavyweight work, the execution is see the same sequence of inputs and process these inputs slowed down (e.g., a popular race detection tool Thread- with the same efficient thread interleavings automati- Sanitizer [41], which uses a traditional framework, in- cally. In addition, this paper discusses potential direc- curs 20X∼100X slowdown for many programs). Recent tions in which REPFRAME can further strengthen exist- frameworks that take the “partially-decoupled” approach ing analyses. Evaluation shows that REPFRAME is easy have shown to run 4X∼8X times faster [22, 43] than to run two asynchronous analysis tools together and has traditional frameworks because they transfer fewer ex- reasonable overhead. ecution states. However, the overall slowdown of these recent frameworks are still prohibitive in their own eval- 1 Introduction uation because the amount of execution states (e.g., effective memory addresses [22] and thread interleav- Dynamic program analysis frameworks greatly improve ings [43]) transferred to analysis tools are still enormous. software quality as they enable a wide range of powerful analysis tools (e.g., data race detectors [34, 41, 43] for The second problem is that recent frameworks with multithreaded applications) at runtime. Existing analy- the “partially-decoupled” approach heavily trade off transparency with analysis tools. To be able to transfer Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or fewer execution states to an analysis tool, these frame- distributed for profit or commercial advantage and that copies bear this notice works require heavily carving the analysis tool as well and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is per- as the transferred execution states. For example, a re- mitted. To copy otherwise, or republish, to post on servers or to redistribute to cent framework [43] for race detection tools leverages lists, requires prior specific permission and/or a fee. Request permissions from [email protected]. the record-replay technique [24, 25, 29] to reduce anal- ysis work in the actual execution, but this framework re- APSys ’15, July 27-28, 2015, Tokyo, Japan. 2015 ACM. ISBN 978-1-4503-3554-6/15/07$15.00. quires significantly carving race detection tools into sev- http://dx.doi.org/10.1145/2797022.2797033 eral phases to adapt to record-replay, which makes the 1 tools largely different from typical ones (Note that an cusses several types of analyses in which REPFRAME analysis tool itself is already quite difficult to implement has the potential to strengthen and speedup existing tools and maintain.) themselves via REPFRAME’s replication architecture. In The third problem is that existing frameworks, includ- addition, this paper points out that existing race detec- ing both traditional ones and recent ones, have not shown tion tools can actually speedup REPFRAME. In a word, to run multiple types of analysis tools together. Multiple REPFRAME and existing analysis tools form a mutually analysis tools bring multiple powerful guarantees, which beneficial eco-system. is attractive to today’s applications. However, the tra- Third, despite much effort, state-of-the-art still lacks ditional “fully-coupled” frameworks typically take the evaluation to show that different types of analysis shadow memory approach to hold analysis results for tools can actually run together. To address this chal- each memory byte in the actual execution, then different lenge, we evaluated REPFRAME on a popular paral- analyses may require different copies of shadow mem- lel anti-virus scanning server ClamAV [11] with three ory per byte. It is not trivial to extend these frameworks replicas (each has 24 cores) in Linux. Our evaluation to support multiple copies of shadow memory. Consider- shows that REPFRAME is able to transparently run two ing “partially-decoupled” frameworks, it is not trivial to analysis tools together: one is a heavyweight analy- extend them to support multiple types of analysis tools sis tool, the Helgrind race detector [34]; the other is a within one execution either, because these frameworks lightweight analysis tool, DynamoRio’s code coverage heavily carve the transferred execution states, which are tool drcov [8]. Moreover, REPFRAME incurred merely hard to reuse for different types of analysis tools. 2.1% overhead over the actual execution. In sum, a fundamental reason for the three problems The main contribution of REPFRAME is the idea of in existing analysis frameworks is that they run only one applying transparent state machine replication to fully actual execution, so an analysis tool has to be fully or decouple an application’s actual execution and analy- partially coupled with the execution in order to inspect sis tools, which benefits applications, analysis tools, and execution states. Well, what if one can construct multiple frameworks. Note that: (1) unlike CRANE,REPFRAME equivalent executions efficiently and transparently? does not aim to provide fault-tolerance, but aims to con- To address this question, this paper presents struct multiple equivalent executions so that applications REPFRAME, an efficient, transparent dynamic pro- can enjoy efficient and transparent analyses; and (2) gram analysis framework by leveraging a technique REPFRAME does not aim to replace traditional “fully- called transparent state machine replication, presented coupled” frameworks, but aims to compensate them be- in CRANE [16]. This technique runs the same multi- cause REPFRAME can easily run them on replicas. threaded application on a set of machines (or replicas) In the remaining of this paper, x2 introduces the back- transparently without modifying these applications, and ground of the CRANE system. x3 gives an overview of enforces that all replicas perform the same execution REPFRAME, including its deployment model, its check- state transitions. To do so, CRANE combines two point design for analysis tools, and its potential bene- techniques, state machine replication (or SMR) and fits. x4 presents evaluation results, x5 introduces related deterministic multithreading (or DMT). SMR ensures work, and x6 concludes. that all replicas always see the same sequence of inputs as long as majority of replicas agree on these inputs. 2C RANE Background DMT efficiently enforces the same thread interleavings CRANE’s deployment model is similar to a typical across replicas on each input. SMR’s. In a CRANE-replicated system, a set of 2f+1 Leveraging CRANE,REPFRAME can just run anal- machines (nodes) are set up within a LAN, and each ysis tools on some replicas while the other replicas node runs an instance of CRANE containing the same still run actual executions and process client requests server program. Once the CRANE system starts, one fast. However, to achieve this goal, REPFRAME must node becomes the primary node which proposes the or- address three practical challenges. First, even asyn- der of requests to execute, and the others become backup chronous analysis tools (e.g., race detectors) sometimes nodes which follow the primary’s proposals. An arbi- may decide to roll back to previous execution states and trary number of clients in LAN or WAN send network discard malicious inputs that triggered harmful events. requests to the primary and get responses. If failures oc- To address this challenge, REPFRAME provides a trans- cur, the nodes run a leader election to elect a new leader parent application-level checkpoint mechanism with ex- and continue.