Software Behaviour Correlation in a Redundant and Diverse Environment Using the Concept of Trace Abstraction Abdelwahab Hamou-Lhadj, Syed Shariyar Murtaza, Mario Couture, Raphael Khoury Waseem Fadel, Ali Mehrabian System of Systems Section, Software Analysis and Software Behaviour Analysis (SBA) Research Lab, Robustness Group, Defence Research and Concordia University, Montréal, QC, Canada Development Canada, Valcartier, Québec, Canada {abdelw, s_eskand, w_fadel, al_meh}@ece.concordia.ca {mario.couture, raphael.khoury}@drdc-rddc.gc.ca ABSTRACT intrusion detection. Redundancy alone, however, has been shown Redundancy and diversity has been shown to be an effective to be ineffective since an attack can propagate to other nodes and approach for ensuring service continuity (an important compromise the whole system. To address this issue, the nodes requirement for autonomic systems) despite the presence of should support some sort of diverse design. Studies have shown anomalies due to attacks or faults. In this paper, we focus on that it is difficult for an attacker to compromise multiple diverse operating system (OS) diversity, which is useful in helping a nodes with the same attack [19]. system survive kernel-level anomalies. We propose an approach There are different ways in which diversity can be introduced in a for detecting anomalies in the presence of OS diversity. We computing infrastructure including the use of system architectures achieve this by comparing kernel-level traces generated from [12, 40], automatic diversity through randomization [30], design instances of the same application deployed on different OS. Our diversity using N-version programming [15], and so on. A trace correlation process relies on the concept of trace abstraction, thorough survey of redundancy and diversity techniques for in which low-level system events are transformed into higher-level security is presented in [20]. To detect anomalies, most of these concepts, freeing the trace from OS-related events. We show the techniques rely on comparing the output generated by the diverse effectiveness of our approach through a case study, in which we instances providing the same input. This design, as noted by selected Linux and FreeBSD as target OS. We also report on Giffin et al. in [21], makes these methods vulnerable to attacks lessons learned, setting the ground for future research. that mimic the original system behaviour by returning the correct service response. To overcome this issue, Gao et al. proposed to Categories and Subject Descriptors compare the control flow (represented as execution traces) of D.4 [Operating Systems]: Security and Protection – information diverse processes running the same input using a behavioural flow controls, invasive software, security kernels. distance [17] and Hidden Markov Models [18]. Despite the authors‘ efforts, their proposed techniques do not overcome the inherit complexity associated with the semantic variations of General Terms traces coming from different platforms. In many ways the Security, Reliability, Algorithms. problem can be thought of as analogous to that of comparing two sentences from different languages. Keywords Redundancy and diversity, Anomaly Detection, Dynamic In this paper, we propose a new approach for anomaly detection in Analysis, Trace abstraction, Autonomic systems. a diverse environment. Our approach relies on the concept of trace abstraction, which is the process of transforming a trace of low- level events into higher-level concepts by abstracting out details 1. INTRODUCTION pertaining to the computing platform. In other words, the resulting Redundancy — the process of having multiple instances of the abstract trace contains operations that are agnostic to the platform same application run on redundant nodes, is a key component of from which the trace is generated. For example, the content of an system resilience in the presence of an security breach. If one event-based trace generated from reading a file on disk can vary node is down (due to an attack for example), a backup (and significantly from one operating system to another. The aim of presumably healthy) instance takes over the load and provides trace abstraction is to transform these low-level events into a services. Monitoring of the divergence between the behaviors of higher concept, such as ‗read file‘, making it possible to compare each instance has also been shown to be an effective method of the traces despite the environment in which they have been generated. Permission to make digital or hard copies of all or part of this work for The focus of this study is on operating system (OS) diversity. OS personal or classroom use is granted without fee provided that copies are diversity is an effective way to improve the overall resilience of not made or distributed for profit or commercial advantage and that copies the system in the presence of kernel-level attack threats. For bear this notice and the full citation on the first page. To copy otherwise, example, if an attack is designed to exploit Linux vulnerability, it or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. will most likely fail to compromise a Windows system since both RACS’13, October 1–4, 2013, Montreal, QC, Canada. systems exhibit different flaws. In this study, we limit ourselves to Copyright 2013 ACM 978-1-4503-2348-2/13/10 …$15.00. two nodes for simplicity reasons (although the concepts presented DRDC-RDDC-2016-N024 LQWKLVSDSHUFDQHDVLO\EHH[WHQGHGWRPXOWLSOH26 7KHVHOHFWHG ZKLFKLVQHFHVVDU\ IRUWKHHIIHFWLYHXVHRIVHYHUDOWUDFHDQDO\VLV RSHUDWLQJ V\VWHPV DUH /LQX[ 8EXQWXDQG )UHH%6' 1DWXUDOO\ WHFKQLTXHV LQFOXGLQJ WUDFH FRUUHODWLRQ VHH > @ IRU PRUH RWKHURSHUDWLQJV\VWHPVFDQEHXVHG2XUFKRLFHLVPRWLYDWHGE\ GLVFXVVLRQRQWKHXVHRIWUDFHRIDEVWUDFWLRQWHFKQLTXHVIRUWUDFH WKHIROORZLQJFULWHULD FRPSUHKHQVLRQ +RZHYHU XQOLNH D UXOHEDVHG PRGHO WUDFH DEVWUDFWLRQFDXVHVORVVRILQIRUPDWLRQ7KLVLQIRUPDWLRQPLJKWEH x $OWKRXJK /LQX[ DQG )UHH%6' GLIIHU LQWHUQDOO\ WKH\ ERWK QHHGHG WR GHWHFW VRPH DWWDFNV )XWXUH ZRUN VKRXOG IRFXV RQ GHULYH IURP 8QL[ 6LPLODU FRQYHQWLRQV KDYH EHHQ XVHG WR GHWHUPLQLQJWKHOHYHORIGHWDLOVWKDWDEVWUDFWWUDFHVVKRXOGFRQWDLQ GHYHORSERWKV\VWHPV7KLVSHUPLWVWKHUHXVHRIH[SHUWLVH WRUHGXFHWKHHIIHFWRIORVWLQIRUPDWLRQ x %RWK V\VWHPV DUH RSHQ VRXUFH DQG IUHH 7KLV LV YHU\ LPSRUWDQW LQ WKH FRQWH[W RI VHFXULW\ VLQFH PRUH DGYDQFHG Application Application VHFXULW\PHFKDQLVPVPLJKWUHTXLUHLQYHVWLJDWLQJWKHVRXUFH Attack Linux (LTTng) FreeBSD (DTrace) FRGHRUHYHQPRGLI\LQJLW7KLVZRXOGQRWEHSRVVLEOHLID Script SURSULHWDU\V\VWHP VXFKDV:LQGRZV LVXVHG x %RWKV\VWHPVHQMR\DODUJHRQOLQHFRPPXQLW\VXSSRUWZLWK H[WHQVLYHGRFXPHQWDWLRQ:HXVHGRQOLQHGRFXPHQWDWLRQWR Trace1 Trace2 XQGHUVWDQGWKHV\VWHPFDOOPHFKDQLVPVRIERWKV\VWHPVDQG EHDEOHWRFRPSDUHWUDFHVJHQHUDWHGIURPWKHLUNHUQHOV x %RWK V\VWHPV KDYH EXLOWLQ WUDFLQJ FDSDELOLWLHV :H XVHG /77QJ WR WUDFH WKH /LQX[ NHUQHO DQG '7UDFH WR WUDFH Trace )UHH%6' Abstraction 7KLVDUWLFOHPDNHVWZRNH\FRQWULEXWLRQVWRWKHVFLHQWLILFOLWHUDWXUH RQ LQWUXVLRQ GHWHFWLRQ )LUVW LWSURSRVHV D QHZ WUDFH DEVWUDFWLRQ DOJRULWKPWKDWDOORZVWKHWUDQVODWLRQRIORZOHYHOV\VWHPVSHFLILF WUDFHVLQWRDEVWUDFWWUDFHVWKDWFDSWXUHWKHEHKDYLRXURIWKHWDUJHW Trace V\VWHP LQ D VHPDQWLFVEDVHG DQG V\VWHP DJQRVWLF UHSUHVHQWDWLRQ Correlation 6HFRQGWKHSDSHUVKRZVKRZWKLVUHSUHVHQWDWLRQFDQEHXVHGIRU LQWUXVLRQGHWHFWLRQE\FRUUHODWLQJWKHVLPXOWDQHRXVH[HFXWLRQVRI WZRGLYHUVHV\VWHPV7KLVFRUUHODWLRQOHYHUDJHVWKHIDFWWKDWLWLV GLIILFXOWWRVLPXOWDQHRXVO\DWWDFNWZRGLIIHUHQWV\VWHPVLQRUGHUWR Correlation EXLOGPRUHVHFXUHV\VWHPV Report 7KH UHPDLQGHU RI WKLV SDSHU LV VWUXFWXUHG DV IROORZV 7KH QH[W VHFWLRQGHYHORSVWKHPHWKRGRORJ\ZHDGRSWLQWKLVVWXG\7KLVLV Figure 1. Correlating traces using trace abstraction IROORZHG LQ VHFWLRQ E\ D FDVH VWXG\ ZKLFK KLJKOLJKWV WKH VWUHQJWKVDQGOLPLWDWLRQVRIWKHDSSURDFK6HFWLRQSUHVHQWVDQ 7UDFH*HQHUDWLRQ RYHUYLHZRIWKHUHOHYDQWOLWHUDWXUH6HFWLRQGUDZVWKHFRQFOXVLRQ 7KHWUDFHVXVHGLQWKLVVWXG\DUHV\VWHPFDOOWUDFHVZKLFKGHSLFW WRWKHVWXG\DQGRXWOLQHVGLUHFWLRQVIRUIXWXUHUHVHDUFK WKH LQWHUDFWLRQV EHWZHHQ XVHU DSSOLFDWLRQV DQG WKH NHUQHO :H FKRVHV\VWHPFDOOWUDFHVEHFDXVHWKH\KDYHEHHQXVHGH[WHQVLYHO\ $3352$&+ LQWKHDUHDRILQWUXVLRQGHWHFWLRQ HJ>@ 7KH 2XU DSSURDFK LV VKRZQ LQ )LJXUH 7KH ILUVW LQVWDQFH RI WKH FRPPRQ DSSURDFK LV WR EXLOG D UHIHUHQFH PRGHO XVLQJ YDULRXV DSSOLFDWLRQ UXQV XQGHU QRUPDO FRQGLWLRQV DQG LV LQWHQGHG WR PDFKLQH OHDUQLQJ WHFKQLTXHV IURP WUDFHV RI V\VWHP FDOOV GXULQJ FDSWXUH KHDOWK\ EHKDYLRXU 7KH VHFRQG LQVWDQFH LV GHOLEHUDWHO\ QRUPDOH[HFXWLRQRIWKHV\VWHP XVXDOO\LQDODEHQYLURQPHQW $ LQIHFWHG E\ D VLPXODWHG DWWDFN 7UDFHV JHQHUDWHG IURP WKHVH IDXOW GHWHFWLRQ WHFKQLTXH FDQ WKHQ EH GHYHORSHG E\ REVHUYLQJ LQVWDQFHVGXULQJRSHUDWLRQDUHILUVWDEVWUDFWHGRXWXVLQJRXUWUDFH XVLQJ PRQLWRULQJ FDSDELOLWLHV DQ\ GHYLDWLRQV RI WKH GHSOR\HG DEVWUDFWLRQSURFHVVDQGWKHQFRPSDUHG$VPHQWLRQHGHDUOLHUWKH V\VWHPIURPWKHEDVHOLQHPRGHO 7KHVHVWXGLHVKRZHYHUGR QRW DEVWUDFWLRQSURFHVVWXUQVDUDZWUDFHLQWRDPRUHGHVFULSWLYHDQG WDNHLQWRDFFRXQWGLYHUVLW\ PHDQLQJIXOVHTXHQFHRIRSHUDWLRQVUDWKHUWKDQDVHTXHQFHRIORZ OHYHOHYHQWV :HXVHGWZRWUDFHUVQDPHO\/77QJDQG'WUDFHZKLFKJHQHUDWH WUDFHVIRU/LQX[DQG%6'UHVSHFWLYHO\/77QJLVDWUDFHUWKDWZDV $QDOWHUQDWLYHDSSURDFKZRXOGEHWRGHVLJQPDSSLQJUXOHVWRPDS GHYHORSHG WR H[WUDFW LQIRUPDWLRQ IURP WKH /LQX[ NHUQHO XVHU V\VWHPFDOOVLQ/LQX[WRWKHLUFRUUHVSRQGLQJRQHVLQ%6'DQGXVH VSDFH OLEUDULHV DQG XVHU DSSOLFDWLRQV E\ UXQQLQJ D UHFRPSLOHG WKHVH UXOHV DV D UHIHUHQFH PRGHO WR JXLGH WKH WUDFH FRUUHODWLRQ LQVWUXPHQWHGYHUVLRQRIWKHNHUQHO>@,QWKLVVWXG\ZHLQVWDOOHG SURFHVV 7KH FKDOOHQJH KRZHYHU LV WR EXLOG DQ DGHTXDWH VHW