In Lieu of Swap: Analyzing Compressed RAM in Mac OS X and Linux

In Lieu of Swap: Analyzing Compressed RAM in Mac OS X and Linux

In Lieu of Swap: Analyzing Compressed RAM in Mac OS X and Linux Golden G. Richard III University of New Orleans and Arcane Alloy, LLC [email protected] Andrew Case The Volatility Project [email protected] 2 Who? Professor of Computer Science and University Research Professor, Director, GNOCIA, University of New Orleans hp://www.cs.uno.edu/~golden Digital forensics, OS internals, reverse engineering, offensive compuKng, pushing students to the brink of destrucKon, et al. Founder, Arcane Alloy, LLC. hp://www.arcanealloy.com Digital forensics, reverse engineering, malware analysis, security research, tool development, training. Co-Founder, Partner / Photographer, High ISO Music, LLC. hp://www.highisomusic.com Rock stars. Heavy Metal. Earplugs. © 2014 by Golden G. Richard III and Andrew Case 3 Forensics, Features, Privacy • New OS features may negavely forensics • e.g.: – Nave whole disk encrypKon – Encrypted swap – Secure Recycle Bin / Trash faciliKes – Secure erasure during disk formang • Today’s topic: Compressed RAM • Claim: posively impacts forensics • First some background © 2014 by Golden G. Richard III and Andrew Case 4 Where’s the Evidence? Files and Filesystem ApplicaOon Windows Deleted Files metadata metadata registry Print spool Hibernaon Temp files Log files files files Browser Network Slack space Swap files caches traces RAM: OS and app data Memory analysis “Tradional” structures storage forensics © 2014 by Golden G. Richard III and Andrew Case 5 Memory Analysis on RAM Dump • Processes (dead/alive) e.g., discover unauthorized programs • Open files e.g., detect keystroke loggers, hidden processes • Network connections e.g., find backdoors, connections to contraband sites • Volatile registry contents • Volatile application data e.g., plaintext for encrypted material, chat messages, email fragments • Other OS structures e.g., analysis of memory allocators to retrieve interesting structures, filesystem cache, etc. • Encryption keys e.g., keys for whole disk encryption schemes What’s missing? © 2014 by Golden G. Richard III and Andrew Case 6 Memory Analysis: Swap Files “TradiKonal” forensics: capture contents of storage devices, including the swap file swap file on disk Swap file contains RAM “overflow”—if you don’t have this data, you probably don’t have a complete memory dump Live forensics / memory analysis: capture RAM contents RAM 00000000000000000000000000000000 0 11110010010011010010010101010101 4096 If you acquire both RAM and disk contents, 01000001010010010010001000100010 … … memory smearing is likely and the swap file 111111111111111111111111111111111 may be (very) out of sync with the memory . dump 010101010101010101010101010101011 2GB © 2014 by Golden G. Richard III and Andrew Case 7 Forensically, Swap Files are a Mess Data trapped order in unsanized RAM dump Swap file blocks allocated to swap file encrypted swap void swap_crypt_ctx_initialize ( void ) { unsigned int i; 554-88-2345 if ( swap_crypt_ctx_initialized == FALSE ) { for (i = 0; i < ( sizeof ( swap_crypt_key ) / [email protected] sizeof ( swap_crypt_key [0])); i++) { swap_crypt_key [i] = random(); } aes_encrypt_key ( murder (const unsigned char *) swap_crypt_key , SWAP_CRYPT_AES_KEY_SIZE , &swap_crypt_ctx.encrypt); struct { ... swap_crypt_ctx_initialized = TRUE ; int ip; } … } } netstat; © 2014 by Golden G. Richard III and Andrew Case 8 Background: Memory Management 2GB • Need to allocate RAM to: – Operang system – Individual applicaons – All applicaons • Possibly fairly, or some other policy • OS’s need to manage memory efficiently to feed RAM-hungry apps • Ever-increasing “need” for more memory © 2014 by Golden G. Richard III and Andrew Case 9 Virtual Memory • Provides conKguous, linear address space for processes • Virtual address space is divided into fixed-size pages – Physical memory is divided into pages of same size – On x86 Intel, these are normally 4K • OS keeps track of free and allocated pages • Processes get only as many pages as they need © 2014 by Golden G. Richard III and Andrew Case 10 Logical vs. Physical Addresses • Virtual address: Address whose scope is a parKcular process or OS kernel • Physical address: Locaon in physical RAM Process P2 0 Process P1 RAM 4096 Process P3 0 0 … 4096 4096 0 … 4096 … … … … … . … . 4GB . 2GB 4GB . 4GB © 2014 by Golden G. Richard III and Andrew Case 11 Paging: (Very) Simplified View • Page tables (per process) are used to translate logical to physical addresses • Pages for a parKcular process are generally not in conKguous order in RAM Process P1 P1 page table RAM 0 01000001010010010010001000100010 2 0 4096 11110010010011010010010101010101 1 11110010010011010010010101010101 4096 … 01000001010010010010001000100010 … … … . 2GB 4GB © 2014 by Golden G. Richard III and Andrew Case 12 64-bit Mode: 4-level Page Tables: 4K Pages © 2014 by Golden G. Richard III and Andrew Case 13 Stretching RAM • Want to allocate more memory to processes than total amount of RAM • Extend RAM by swapping pages to disk and retrieving as necessary • Allows more (and bigger) processes to execute • Workable, because: – Not all execuKng processes are acKve at once – Processes typically have “working sets” of pages, e.g., memory they’re acKvely using “now” © 2014 by Golden G. Richard III and Andrew Case 14 Virtual Memory with Swapping For good performance, can’t overdo it--the pipe swap file between RAM and disk is 100MB/sec on disk small Otherwise: THRASHING Process P1 P1 page table RAM 0 01000001010010010010001000100010 2 00000000000000000000000000000000 0 4096 11110010010011010010010101010101 1 11110010010011010010010101010101 4096 … 01001001010100111111111111111000 Not present 01000001010010010010001000100010 … … 11111111111111111111000011111111 Not present … 111111111111111111111111111111111 . 010101010101010101010101010101011 2GB 4GB © 2014 by Golden G. Richard III and Andrew Case 15 Virtual Memory with Swapping SSDs substanKally increase size of pipe swap file 500GB/sec or so on SSD Process P1 P1 page table RAM 0 01000001010010010010001000100010 2 00000000000000000000000000000000 0 4096 11110010010011010010010101010101 1 11110010010011010010010101010101 4096 … 01001001010100111111111111111000 Not present 01000001010010010010001000100010 … … 11111111111111111111000011111111 Not present … 111111111111111111111111111111111 . 010101010101010101010101010101011 2GB 4GB © 2014 by Golden G. Richard III and Andrew Case 16 Virtual Memory with Swapping up to 1.2GB/sec+ swap file on PCIe (e.g., new Mac Pro, flash storage newest iMacs) Process P1 P1 page table RAM 0 01000001010010010010001000100010 2 00000000000000000000000000000000 0 4096 11110010010011010010010101010101 1 11110010010011010010010101010101 4096 … 01001001010100111111111111111000 Not present 01000001010010010010001000100010 … … 11111111111111111111000011111111 Not present … 111111111111111111111111111111111 . 010101010101010101010101010101011 2GB 4GB © 2014 by Golden G. Richard III and Andrew Case 17 Avoiding Swapping by Compressing RAM • OpKmizaon: – “Hoard” some RAM, not available to processes – When RAM is running low, compress pages in memory instead of swapping to disk • Why bother? – SSDs are fast – PCIe solid state storage is “crazy” fast – Memory is ever cheaper • So…why? © 2014 by Golden G. Richard III and Andrew Case 18 It’s Worth It • Ultraportable laptops sKll RAM limited – e.g., current Macbook Air: Base 4GB, Max 8GB RAM • Virtualizaon, even on laptops • Malware analysts, VMWare Workstaon, Fusion, etc. • Swapping can cause serious performance issues • New Mac Pro memory bandwidth: 60GB/sec • Max swap bandwidth: 1.2GB/sec • Modern CPUs compress and decompress very efficiently © 2014 by Golden G. Richard III and Andrew Case 19 Virtual Memory with Compressed RAM In Mac OS X Mavericks and Linux, set aside swap file some RAM and compress pages that would on SSD have been swapped due to memory pressure Process P1 P1 page table RAM 0 01000001010010010010001000100010 2 00000000000000000000000000000000 4096 11110010010011010010010101010101 1 11110010010011010010010101010101 … 01001001010100111111111111111000 Not present 01000001010010010010001000100010 … 11111111111111111111000011111111 Not present 111111111111111111111111111111111 0100100101 . 0100100101 compressed . 0100100101 0100100101 pages 4GB © 2014 by Golden G. Richard III and Andrew Case 20 © 2014 by Golden G. Richard III and Andrew Case 21 It’s Back • RAM Doubler -- ~20 years ago • Historically RAM compression not very effecKve – Processors too slow to offset compress/decompress costs – Poor integraon with OS • Old academic paper (2003) on compression schemes for RAM: – M. Simpson, R. Barua, S. Biswas, Analysis of Compression Algorithms for Program Data, Tech. rep., U. of Maryland, ECE department, 2003. • Compression scheme in Mavericks (WKdm) was invented in 1999: – P. R. Wilson, S. F. Kaplan, and Y. Smaragdakis, The Case for Compressed Caching in Virtual Memory Systems, Proceedings of the USENIX Annual Technical Conference, 1999. • Now, super fast CPUs, lots of cores, Kght OS integraon © 2014 by Golden G. Richard III and Andrew Case 22 Compressed RAM: Forensic Impact Currently, “simultaneous” capture results in lots of smearing swap file on disk With compressed RAM, may be li;le or no data swapped out—capturing RAM captures (some or all) “swap”! PHYSICAL RAM No support in current tools—compressed 00000000000000000000000000000000 0 data opaque or ignored 11110010010011010010010101010101 4096 01000001010010010010001000100010 … 111111111111111111111111111111111 … Our tools enable analysis of . compressed RAM 010101010101010101010101010101011

View Full Text

Details

  • File Type
    pdf
  • Upload Time
    -
  • Content Languages
    English
  • Upload User
    Anonymous/Not logged-in
  • File Pages
    42 Page
  • File Size
    -

Download

Channel Download Status
Express Download Enable

Copyright

We respect the copyrights and intellectual property rights of all users. All uploaded documents are either original works of the uploader or authorized works of the rightful owners.

  • Not to be reproduced or distributed without explicit permission.
  • Not used for commercial purposes outside of approved use cases.
  • Not used to infringe on the rights of the original creators.
  • If you believe any content infringes your copyright, please contact us immediately.

Support

For help with questions, suggestions, or problems, please contact us