TROGUARD: Context-Aware Protec6on Against Web-Based Socially Engineered Trojans Rui Han, Alejandro Mesa, University of Miami Mihai Christodorescu, QualComm Research Saman Zonouz, Rutgers University Mo#va#on • Waterfall screen saver Trojan 2 Mac OS threats Rank Name Percentage 1 Trojan.OSX.FakeCo.a 52% 2 Trojan-DownloaDer.OSX.Jahlav.d 8% 3 Trojan-DownloaDer.OSX.Flashfake.aI 7% 4 Trojan-DownloaDer.OSX.FavDonw.c 5% 5 Trojan-DownloaDer.OSX.FavDonw.a 2% 6 Trojan-DownloaDer.OSX.Flashfake.ab 2% 7 Trojan-FakeAV.OSX.Defma.gen 2% 8 Trojan-FakeAV.OSX.Defma.f 1% 9 Exploit.OSX.Smid.b 1% 10 Trojan-DownloaDer.OSX.Flashfake.af 1% McAfee an6virus solu6on: hVp:www.securelist.com 3 Example Malwares Malware Descripons Plaorm TrojanClicker.VB Trojan socially Windows engineered as adobe .395 flash update and Mac OS X Trojan or Adware socially Windows, Mac Faked An6-Virus engineered as an6-virus so]ware OS X, and Linux Malware socially Android Opfake Browser engineered as Opera Browser Legi6mate applicaons Mac OS X and WireLuker socially engineered with ad-wares and Trojan iOS 4 ContribUons • Answer the ques6on: “Is this program doing what I expected it to do?” • Bridge the seman6c gap between func6onality classes and low level behaviors • Built on 100 Linux app profiles • High detec6on rate on 50 Trojan apps 5 TROGUARD ArchItectUre TROGUARD Offline Dynamic Application Functionality Application Functionality Functionality Class Profile Database Feature Tracing Generation Extraction Application Functionality Profile Database Online Inference of Download Perceived Website Functionality Class Real-Time Classification Application Downloaded Functionality Application Tracing sandbox Alert 6 Key Premise • TROGUARD detects Trojans based on the premise that applicaons with similar func#onalies expose similar system-level behaviors • Applicaons with similar func6onali6es belong to a funconalItY class, they should exhibit common system level behaviors • Learn web-browser behavior of well know instances(e.g., Firefox and Chrome) • Compare the web-browser profile with the behaviors of unknown downloaded web-browser app 7 Func#onalItY class • It represents both user’s understanding of so]ware category and the system’s observaon of a so]ware execu6on behavior Func#onalItY class sopedia.com download.cnet.com tucows.com GraphIc EdItor Ar6s6c so]ware Graphic Design SW Design tools Game Games Games Games Browser Internet Browsers Internet Instant Messenger Communicaons Communicaons MeDIa PlaYer MP3/Audio So]ware AuDIo EDItor Mul6media Audio/Video ViDeo EDItor Text EdItor office Produc6vity so]ware Business IDE Programming Developer Tools Develop/Web Calculator Uli6es Uli6es Home/Educaon 8 ApplIca#ons Class StUDIeD ApplIca#ons 1. Graphic Editor gimp, pinta, imagej, Inkscape, koloUrpaInt, rawtherapee, mypaInt, gpaint, gnome-paint, pencil 2. Games sol, wesnoth, glchess, neverball, kmahjongg, sUpertUxkart, hedge- wars, pingus, frozen-bUbble, eboard 3. Browser chrome, firefox , opera, epIphanY, miDori, chromium, netsurf, arora, xxxterm, rekonq 4. Instant skype, kmess, emesene, kopete, pidgin, psi, gajim, empathy, amsn, qum Messenger 5. MeDIa PlaYer smplayer, vlc, audacious, quodli- bet, gmusicbrowser, qmmp, abraca, amarok, guayadeqUe, aqualung 6. AuDIo EDItor audacity, avidemux, dvbcut, og- gconvert, kwave, wavbreaker, mp3splt-gtk, mhwaveedIt, fillmore, soundconverter 7. Video Editor openshot, lives, iriverter, kino, pivI, viDeocut, winff, arista-gtk, kdenlIve, curlew 8. Text Editor kile, geanY, texmaker, callIgra- words, soffice.bin, lyx, tea, jed, emacs, vi 9. IDE anjuta, coDelIte, coDeblocks, net- beans, monoDevelop, kdevelop, spyder, monkeYstUDIo, Drracket, idle 10. Calculator grpn, gcalctool, EDenMath, speed- crunch, kcalc, keUrocalc, extcalc, gip, 9 galculator, gnome-genius Func#onalItY TracIng • Manual tes6ng • Run 60 seconds for each applicaon • System call trace • User-space informaon User interacvity Resource consumpBon IP addresses and port number 10 FeatUre Extrac#on • Processing tracing data • Four groups of feature file system Network resource usage user interacvity • Intermediate feature 11 IntermeDIate FeatUres Example: if( libssl3.so & fd = sYs_socket(AF_INET, ..) &sys_write(fd, ..) & sys_reaD(fd, ..) ) HTTP = trUe 12 TROGUARD ArchItectUre TROGUARD Offline Dynamic Application Functionality Application Functionality Functionality Class Profile Database Feature Tracing Generation Extraction Application Functionality Profile Database Online Inference of Download Perceived Website Functionality Class Real-Time Classification Application Downloaded Functionality Application Tracing Alert 13 Web Page AnalYsIs • Give the explicit func6onality class • Web page contents analysis • OCR to extract the texts in the images • Analysis based on keywords 14 User Interface • Browser extension for web page analysis 15 Sandboxing • SELinux sandbox • One policy for each app class • Automacally generated by parsing all the logs from an app class 16 Classifier EvalUaon • 600 data points (10 second each) • 10 fold cross validaon • 5 classifiers with different feature group • Precision • Recall • Confusion Matrix 17 Precision 100% 90% 80% 70% 60% 50% Precision 40% 30% 20% 10% 0 File Network CPU-Mem Interaction All Different Attributes Domains Browser Game Graphic-Editor Calculator Office IDE Video-Editor Average IM Media-Player Audio-Editor 18 Recall 100% 90% 80% 70% 60% 50% Recall 40% 30% 20% 10% 0 File Network CPU-Mem Interaction All Different Attributes Domains Browser Game Graphic-Editor Calculator Office IDE Video-Editor Average IM Media-Player Audio-Editor 19 ConfUsIon MatrIces File features Network features Resource usage features User interac6vity features 20 ConfUsIon MatrIces 21 IntermeDIate FeatUre ResUlts 54 5 0 0 0 0 0 0 0 0 90% 56 0 0 1 0 0 3 0 0 0 93% 58 1 0 0 0 1 0 0 0 97% 0 58 0 0 0 2 0 0 97% 0 0 0 57 0 1 0 0 1 95% 0 0 1 0 42 8 0 0 0 0 0 5 5 0 70% 60 0 0 0 0 0 100% 0 0 0 0 57 0 0 3 0 0 0 0 0 0 95% 60 0 0 0 0 0 100% 0 0 0 0 54 0 0 0 90% 0 0 6 0 0 0 22 23 100% Text analysis Web Page AnalYsIs AccUracY 90% OCR analysis 100 Web page, 20 categories • 80% 70% 60% 50% Accuracy 40% 30% 20% 10% 0 Anti-virusEbookEmailMediaThemes PlayerDownloaderDriverCalculatorGameOfficeBrowserVideoAudio EditorDatabase EditorIDE P2P AppIM GraphicsPDF ReaderNetworkEducation Case StUDY • 10 benign apps ×5 payload = 50 Trojans Func#onalItY Class ApplIca#on MetasploIt Payload 1. Graphic Editor gpaint 2. Games eboard 3. Browser xxxterm 4. Instant Messenger psi linux/x86/shell_bind_tcp 5. Media Player qmmp linux/x86/shell/reverse_tcp linux/x86/vncinject/bind_tcp 6. Audio Editor winff linux/x86/meterpreter/bind_tcp 7. Video Editor fillmore linux/x86/download_exec 8. Text Editor tea 9. IDE spyder 10. Calculator gnome-genius 24 Case StUDY • Predefined acceptance rate 0.8 Trojans" Benign"Apps" 1" 0.8" 0.6" Rate" 0.4" 0.2" 0" True" False" Precision" Recall" F9Measure" Posi1ve" Posi1ve" 25 Symbolic Execuon • Tested Core U6li6es (four func6onality classes) Dirlist Filetype Userinfo Systeminfo • Features collected from symbolic execu6on give us 52% precision • Features collected from user execu6on give us 76% precision 26 User Execuon VS Symbolic Execuon 27 Performance OverheaD • CPU usage: No-LTTng LTTng 28 Performance OverheaD • Memory usage: No-LTTng LTTng 29 Performance OverheaD • Disk throughput: No-LTTng LTTng 30 Performance OverheaD • Network throughput: No-LTTng LTTng 31 ConclUsions • TROGUARD detects Trojans based on the premise that applicaons with similar funconalIes expose similar system-level behaviors • TROGUARD can detect Trojan applicaon download by bridging the gap between the user perceived func6ons and genuine so]ware func6ons 32 RuI Han [email protected] 33 ConclUsions • TROGUARD detects Trojans based on the premise that applicaons with similar funconalIes expose similar system-level behaviors • TROGUARD can detect Trojan applicaon download by bridging the gap between the user perceived func6ons and genuine so]ware func6ons 34 Symbolic Execuon Code Coverage 35 SE Code Coverage Evoluon 36 .