RL Android Internals A Confectioner's Cookbook Volume I: The Power User's View With updates for Android M, Preview Release 1 (6/2015) By Jonathan Levin Cambridge, MA In memoriam: Frank R. Dye. I missed you by a day, and will miss you for a lifetime. Android Internals - A Confectioner's Cookbook - Volume I - The Power User's View © 2015 by Jonathan Levin. All rights reserved. No part of this work may be reproduced, transmitted in any form or any means, electronic or mechanical, including photocopying, recording, or by any information storage or retrieval system, without the prior written permission of the author. Printed in Cambridge, MA, USA First Printing5-Mp1-as ISBN-10: 9910555-2-4 ISBN-13: 978-0-9910555-2-4 Publisher: Technologeeks.com Production Editor: Jonathan Levin Interior Design: Jonathan Levin Technical Reviewers: Moshe Kravchik, Arie Haenel Cover Illustration: Dino Tsiopanos For information on distribution, translations, or bulk sales, please contact Jonathan Levin, at: Jonathan Levin Phone: (617)-3000-667 Email: [email protected] Web: www.NewAndroidBook.com Company and product names mentioned in this work may be trademarked by their respective owners. Any and every such name is used solely in an editorial fashion, to the benefit of the trademark owner, with no intention whatsoever of infringement. The Android robot is reproduced and/or modified from work created and shared by Google, according to the terms of the Creative Commons 3.0 Attribution License. Limit of Liability/Disclaimer of Warranty: The author neither makes or implies any representations or warranties with respect to the accuracy or completeness of the contents of this work. Further, this paragraph explictly and specifically serves to disclaim all warranties, including without limitation warranties of fitness for a particular purpose. No warranty may be created or extended by sales or promotional materials. The advice and strategies contained herein may not be suitable for every situation. This work is sold with the understanding that the author is not engaged in rendering legal, accounting, or other professional services. If professional assistance is required, the services of a competent professional person should be sought. The author shall not be liable for damages arising herefrom. The fact that an organization or Web site is referred to in this work as a citation and/or a potential source of further information does not mean that the author endorses the information the organization or Web site may provide or recommendations it may make. Further, readers should be aware that Internet Web sites listed in this work may have changed or disappeared between when this work was written and when it is read. Information in this book is distributed on an "As Is" basis, with no warranty. Android is a shifting landscape, with versions being released faster than any work can keep up with. Every effort and precaution has been taken to update this work to reflect up to and Including Android Lollipop (5.0.2). Bear in mind, however, that different versions of Android may add, modify, or remove APIs and features with no notice. The author hereby disclaims any liability to any and every person or entity with respect to any loss or damage, whether caused or alleged to be caused directly or indirectly, by this work. To report errors or omissions, please contact the author. This copy is specifically licensed for RedLattice, Inc, Ashburn VA, for training conducted 6/2015 and not meant to be used outside the purposes of said specific training About the [Author/Editor/Formatter/Publisher] I started in my teens by hacking (mostly in the good sense of the word) and trying to figure out how things worked back in 1993, from an XT with a 2400 baud modem onto a shell I'm not even sure was mine. With no help per se, I had to NBONBO and then NBO .. A lot has happened in the two decades since, and from UNIX to Linux, I got to Windows and OS X. Over the years, I took up consulting and training - initially, in security. I then realized security is largely a projection of internals. Finally, I gathered a few good men and started Technologeeks.com - which is where I presently pass my time and function as CTO. Authoring is somewhat new: I first took on Apple's OSes with "Mac OS X and iOS Internals" (Wiley, 2012), which was well received. It was a painful process, but the dopamine rush was awesome. Now an addict, I took on Android, and the first part of the result is what you see before you. This was also my first foray into the world of self publishing. With this done, I'm turning to the 2nd Edition of MOXiI. About the Technical Reviewers Moshe Kravchik of Cisco helped review my previous book, and immediately volunteered to help with this one. With his eye for detail and accuracy, he helped me lay out the contents of this book, expanded them with many a question a reader would pose, and caught my (oh-so-many) typos, or areas wherein I wasn't clear enough. For that, my thanks! Arie Haenel, also of Cisco, is hands-down the most gifted reverse engineer I've ever known, and certainly one of the outright smartest people, period. A reviewer of my previous book, it was only natural I would seek his help for this one. He's already got the dubious pleasure of reviewing the 2nd Edition of MOXiI, as well. Table Of Contents 0. About This Book............................................................ viii 1. Introduction...................................................................... 1 Android version history Cupcake Donut Éclair Froyo Gingerbread Honeycomb Ice Cream Sandwich JellyBean KitKat Lollipop M (Preview) Android vs. Linux Not just another Linux Distribution And then came Android. Commonalities and Divergences from Linux The Android Frameworks The Dalvik Virtual Machine JNI Native Binaries Bionic Android Native Libraries External Native Libraries Hardware Abstraction Layer The Linux Kernel Android Derivatives Google offshoots Non-Google ports Pondering the Way Ahead 64-Bit compatibility Android RunTime (ART) Split-Screen Android as a desktop OS Android and Project ARA 2. Partitions & FileSystems.................................................. 2 Partitions The need for separate partitions The GUID Partition Table Flash Storage Systems File Systems Android Device Partitions Android Filesystem Contents The root file system /system /data /cache /vendor The SD card Protected Filesystems OBB - Opaque Binary Blobs ASec - Android Secure Storage The Linux pseudo-Filesystems cgroupfs debugfs functionfs (/dev/usb-ffs/adb) procfs (/proc) pstore (/sys/fs/pstore) selinuxfs (/sys/fs/selinux) sysfs (/sys) 3. Boot, Backup & Recovery............................................... Android Images The Boot Loader Boot Images The Kernel The RAM disk System and Data Partitions The Boot Process Firmware Boot Kernel Boot Shutdown & Reboot Application Backup & Restore Command line tools Local backups Monitoring backup operations System Recovery & Updates Over-The-Air (OTA) Updates Customizing images Resources for image modding 4. Init.............................................................................. 9 The roles and responsbilities of init System Properties The .rc Files Putting it all together: The flow of JOJU Init and USB The Other Roles of init ueventd watchdogd 5. Daemons...................................................................... 11 Core Services adbd servicemanager healthd lmkd (L) logd (L) vold Network Services netd mdnsd mtpd racooon rild Graphics and Media Services surfaceflinger bootanimation mediaserver drmserver Other Services installd keystore debuggerd sdcard zygote[64] 6. The Framework Service Architecture.............................. 1 Revisiting servicemanager The Service Calling Pattern Advantages and Disadvantages Serialization and AIDL The Binder A Little History So what, exactly, is Binder? Using Binder Tracing Binder system_server Startup and Flow Modifying startup behavior 7. Android Through a Linux Lens....................................... 17 /proc, revisited The symlinks: cwd, exe, root fd fdinfo status User mode memory management Virtual Memory classification and lifecycle Memory statistics Out Of Memory conditions Tracing system calls The UPPMCPYQT tool wchan and syscall The TUSBDF utility 8. Security........................................................................ Threat Modeling Mobile Security Security at the Linux Level Linux Permissions Linux Capabilities SELinux Other noteworthy features Security at the Dalvik Level Dalvik Level Permissions Dalvik Code Signing User Level Security The Lock Screen Multi-User Support Key Management Certificate Management Secret and Private Key Management Storage Security /data Encryption Secure Boot Rooting Android Boot-To-Root Rooting via Exploiting Security Aspects of Rooting About This Book Overview If you got this book, no doubt you recognize the importance of Android. From a start-up started back in 2003, it has been assimilated by Google, and morphed into one of its largest arms. Taking Apple's iOS head on (some would say, too closely), it has not only achieved hegemony overmobile operating systems worldwide (with a staggering 82% of the market as this book goes toprint) but has also permeated other platforms, becoming an operating system for wearable devices,TVs, and embedded devices. Android is open source and freely available, meaning anyone can get it, and adopt it to any platform - indeed, it owes its overwhelming popularity to this. It's surprising, however, that some seven years since