Android boot and its optimization Taichun Yuan 0 1 / 1 9 / 2 0 1 5 TM Confidential and Proprietary Updated on 10/22/2015 Preface • This slide will provide a “go through” for Android’s boot process flow and a simple introduction to some directions of speeding up the boot. • Major purpose of this slide is to provide some tips/hints on how to do Android boot optimization. It’s NOT a completed solution. • The Android boot flow will get uboot/Linux involved, so, we will cover the boot optimization on uboot and Linux as well. • All discussion will be based on the following configurations: − Android 4.4.3 release for i.MX6 which contains a 3.10.53 Linux kernel. − Sabre-SDP board for i.MX6Q, boot with eMMC. TM Confidential and Proprietary 1 Agenda • Overview of the system boot flow. • Ideas for uboot/kernel optimization. • Android boot flow. • Android boot optimization − Popular directions and solutions. − Measurement and analyzing tool. • A try on IMX6 TM Confidential and Proprietary 2 Overview of the system boot flow TM Confidential and Proprietary 3 Image structure (no HAB) Before we start with overall boot flow, we need to be clear about the image structure: 1. uboot.imx: contains IVT/DCD data for DDR initialization and uboot itself. IVT/DCD cmdline 2. boot.img (defined in bootimg.h): uboot uboot zImage a) cmdline: preset kernel command line. Boot (p1) b) zImage: Linux kernel. ramdisk c) ramdisk: RAM based file system image. Used as Recovery (p2) rootfs for Android. dtb d) dtb: The device tree blob. exists in boot.img as the 2-Stage image. 3. recovery.img: has a same structure as boot.img. Data (p4) EXT (no image) Normally, only the ramdisk is different from the boot.img. 4. Data partition: holding user data, we don’t have image built for this. Only format it when downloading. 5. system.img: holding system binaries and resources. 6. Others: a) misc: for BCB（Bootloader Control Block） which is used as communication between main system and bootloader when doing recovery boot. We’re not System (p5) EXT using this mechanism, SNVS_LPGPR is used instead. b) cache: for recovery. Holding command from main system and all materials for doing upgrade. Others (p…) EXT/Raw (no image) FLASH TM Confidential and Proprietary 4 Boot flow PC Pointer i.MX6 1. Core reset. PC Jump to Boot CPU Core Data copy ROM for some basic initialization Control and boot decision. Reset * Boot determination 2. Boot ROM read IVT/DCD header process ignored here from flash. uboot 3. Boot ROM parse IVT/DCD and init Boot ROM Boot (p1) the DDR. uboot_head Recovery (p2) 4. Uboot will be loaded into DDR. PC then jumps to it. iRAM 5. Uboot loads boot.img from flash, parse it and place Data (p4) kernel/dtb/ramdisk to specific uboot location. Then jumps to kernel’s kernel bootstrap loader. dtb 6. Kernel boots. After initialization, it ramdisk will jumps to the “init” in the system_mnt Mount ramdisk. System (p5) 7. “init” process will then done a lot of setup, including mounting system/user/… partitions. It will then start some core services for Others (p…) Android. DDR RAM FLASH * Does NOT represent the real memory layout, illustration only TM Confidential and Proprietary 5 Boot flow – time breakdown • Current status on pre-built GA image. Power ON ROM bootloader Kernel boot Android init Home screen is shown ? ms ? ms 830 ms 5.35 s ~15 s ∞ Boot Power uboot kernel Android init home ROM * The time between “Power ON” and “uboot” may have connection with specific platform design, such as power ramp time and boot device type. * The “Android init” here covers not only the “init” process, but also all initialization between kernel and home screen. TM Confidential and Proprietary 6 Ideas for uboot/kernel optimization TM Confidential and Proprietary 7 Before we start • We need some way to measure the boot time: − Host side timestamp. Recommended: “ExtraPutty”. − Kernel printk timestamp: “CONFIG_PRINTK_TIME=y”. − Stop watch, LED, … TM Confidential and Proprietary 8 Ideas for boot optimization • Size / built-in modules • build-in modules’ init speed • Kernel: un-compressed kernel • Kernel: built-in vs. module • Other ideas TM Confidential and Proprietary 9 Size / built-in modules • What to do with size? − Reduce loading time − Reduce init time by removing un-necessary modules. • How? − Define target application: this is to make sure the image contains minimal set of software needed for the board/application. − Breakdown: we need some tools… • Device Tree? − The concept of DT is making the kernel bigger than ever since it needs to be a “super set” which contains all possible code and let DTB “describe” / “choose” which device to support. − In this slide, we will NOT touch DTB because it’s defined as hardware description and the “hardware” can NOT be slimmed. TM Confidential and Proprietary 10 Size / built-in modules – breakdown – size Small tips for breakdown (applicable for uboot & kernel): find . -maxdepth 2 -name "built-in.o" | xargs size | sort -n -r -k 4 Sample result for kernel: • “text” is the code size • “data” & “bss” are the data size (init & un-init) • “dec” is the sum of the previous columns in decimal while “hex” is the HEX value. Notes: 1. Without “-maxdepth 2”, a full list for all targets under current folder will be listed. Or, you can use this command step by step down into specific folder. 2. With “-name ‘*.o’,” similar list can be generated for separate object file. Useful for specific module. Example: find drivers/usb/gadget/ -maxdepth 2 -name "*.o" | xargs size | sort -n -r -k 4 TM Confidential and Proprietary 11 Size / built-in modules – breakdown – size cont’d • After some text processing and importing to XLS, size list for original uboot and kernel: uboot kernel text data bss dec filename text data bss dec filename 132565 8948 108051 249564 ./common/built-in.o 4107005 318121 129768 4554894 ./drivers/built-in.o 16690 624 196908 214222 ./fs/built-in.o 3835172 0 0 3835172 ./firmware/built-in.o 30212 51060 1423 82695 ./drivers/built-in.o 2087224 20732 11540 2119496 ./fs/built-in.o 18882 46 12143 31071 ./net/built-in.o 1772363 75086 64244 1911693 ./net/built-in.o 30098 0 30 30128 ./lib/built-in.o 640947 26512 353108 1020567 ./kernel/built-in.o 4733 0 0 4733 ./disk/built-in.o 425995 42924 2948 471867 ./sound/built-in.o 283642 18150 25532 327324 ./mm/built-in.o 263907 11194 20 275121 ./crypto/built-in.o 175051 10592 11496 197139 ./security/built-in.o 93555 25255 2300 121110 ./lib/built-in.o 100275 2082 1220 103577 ./block/built-in.o 19434 14837 152 34423 ./init/built-in.o 29304 760 8 30072 ./ipc/built-in.o 516 0 0 516 ./usr/built-in.o This kind of table can then be used as a guide for slimming the binary. Using “menuconfig” to do the slimming is recommended. TM Confidential and Proprietary 12 Size / built-in modules – breakdown – nm • “nm” can be used to analyze the symbol size in the kernel image. nm --size-sort -r vmlinux Sample result for kernel: • “t”/”T” = “text”, code section • “d”/”D” = “data”, initialized data • “b”/”B” = “bss”, un-initialized data • “r”, read-only data section • … There’s a script in kernel source tree to diff two kernel images using “nm” command: <kernel-src>/scripts/bloat-o-meter vmlinux.default vmlinux.altconfig TM Confidential and Proprietary 13 Size / built-in modules – Findings Some most significant hot-spots listed here (not a full list): • Uboot: − Some drivers and modules can be removed, such as: usb, un-used cmd_* commands, fastboot (android), network related (MII_*), un-used env_* device for environment storage (use default), display support (if no splash screen required). − Build-in file systems can all be removed: FAT/EXT since Android boot doesn’t require FS support from uboot. • Kernel: − EPD firmware can be removed/moved if not used. − Video in/out related drivers can be slimmed a lot. − Some un-used USB gadget drivers can be removed. − Many un-used FS can be removed: UBIFS, NFS, JFFS2, UDF, … − Other un-used components/sub-componets, such as MTD, ata, input, hid, … TM Confidential and Proprietary 14 build-in modules’ init speed • Most of the time spent during boot is consumed by “initialize”. • For both of uboot and kernel, major work for init is called “initcall”: − Uboot: common/board_f.c (“init_sequence_f” array) common/board_r.c (“init_sequence_r” array) We can only add timestamp print manually to breakdown the time for each init step. − Kernel: All init routine defined with Linux init API, such as “module_init()”, “late_initcall()”, etc. A simple way to debug the time, add the following line into kernel cmdline and check the “dmesg” output: initcall_debug TM Confidential and Proprietary 15 build-in modules’ init speed, cont’d • The message print itself will impact the boot time, but it can provide some trend information. Sample output: After process/sort: initcall init_static_idmap+0x0/0xe8 returned 0 after 0 usecs Start time Function name Return usec initcall init_workqueues+0x0/0x36c returned 0 after 0 usecs 6.165906 imx_serial_init+0x0/0x48 0 4038545 initcall init_mmap_min_addr+0x0/0x28 returned 0 after 0 usecs 1.999378 mxcfb_init+0x0/0xc 0 1470073 10.309506 imx_wm8962_driver_init+0x0/0xc 0 1212897 … 8.471315 gpu_init+0x0/0x12c 0 341488 initcall init_mtd+0x0/0xfc returned 0 after 532 usecs 8.103319 sdhci_esdhc_imx_driver_init+0x0/0xc 0 177161 initcall init_mtdblock+0x0/0xc returned 0 after 4 usecs 8.962669 wm8962_i2c_driver_init+0x0/0x10 0 169909 initcall init_ladder+0x0/0xc returned 0 after 2757 usecs 7.499705 ov5640_init+0x0/0x40 0 124343 … 7.698344 mma8x5x_driver_init+0x0/0x10 0 121189 7.081894 isl29023_init+0x0/0x10 0 114222 7.361302 ov5642_init+0x0/0x40 0 104195 • Now it will be straight forward to check who has consumed most of time.