AOSP Mini-Conference Linaro Welcome ● Main difference between the miniconference and regular Connect talks: Let’s be more interactive! ● One additional purpose of the miniconference: Bring together the various groups inside Linaro that work on the AOSP codebase: ○ LMG -- probably the most obvious use of AOSP ○ LHG -- Android TV ○ Potentially LITE -- Brillo ○ Kernel, Toolchain, … -- need to support both regular Linux and AOSP use ENGINEERS ○ Are there other groups (member engineering teams, maybe) here? AND DEVICES What is your use of the AOSP code base? WORKING TOGETHER Filesystem analysis Satish Patel <[email protected]> File System analysis ● Filesystems investigated: ext4, btrfs, f2fs, nilfs, squashfs ● Variants: encryption enabled/disabled, compression off/zlib/lz4 ● File system analysis briefing (ongoing changes) ○ https://docs.google.com/a/linaro.org/document/d/1jam-PlV9iefnOqujzYWZoY8U9d9GnmPwda 3MItxsPsU/edit?usp=sharing ● Challenges ○ Fixed build support for f2fs image generation (core.mk & image size alignment to 4096) ○ Fixed sparse raw image generation issue ■ Need to use for btrfs and nilfs ○ Image generation for btrfs, nilfs, squashfs etc.. ○ Benchmark porting - bonnie, iozone ○ Partition overload scripts and long run impact scripts ENGINEERS AND DEVICES WORKING TOGETHER Filesystems - A Brief Feature/FS ext4 f2fs btrfs nilfs squashfs Introduction Most used in linux Flash Friendly B/Better/Butter New Compress read based system File System File System Implementation only File System of LFS I-node Hashed B-Tree Linear B+ Tree B-Tree Block Size Extent Fixed Extent Fixed Fixed Type Unix like File Log File Copy On Write Log File UFS Structure Structure Structure Allocation Delayed Immediate Delayed Immediate NA Journal Ordered, WriteBack NA NA NA NA Ubuntu, Most Moto Series Suse Enterprise Ubuntu,NixOs Live mobiles CDs,Android ENGINEERS AND DEVICES WORKING TOGETHER Filesystems - A Traditional Layer WebKit Sqlite Video/Image Application - file access - dir operations Memory Management - file indexing and management - security operations Logical File System(ext4, f2fs, btrfs etc..) OS - data operation on Basic File System physical device - buffering if required - no management Device Driver 1 Device Driver 2 Device Driver n Storage 1 Storage 2 Storage n ENGINEERS AND DEVICES WORKING TOGETHER Filesystems - Basic Types LSF- Log File Structure COW - Copy On Write Image courtesy: http://dboptimizer.com/wp-content/uploads/2013/06/Screen-Shot-2013-06-03-at-10.28.44-AM.png http://tinou.blogs.com/.a/6a00d83451ce5a69e2016302fe0458970d-500wi ENGINEERS AND DEVICES WORKING TOGETHER Filesystems - Test Environment ● Hikey - 96Board ○ 1GB RAM ○ Cortex-A53 Octa Core ○ eMMC ■ Popular on embedded Device ■ Cheap & Flexible http://www.96boards.org/product/hikey/ ■ Fast read & random seek ■ Domains - navigation, eReaders, smartphones, industrial loggers, entertainment devices etc.. ● AOSP + Linaro PatchSet (branch : r55, kernel 4.4) ● F2FS, Ext4, Squashfs, btrfs, nilfs ● Benchmarks ○ Vellamo, RL bench, androbench ○ Bonnie (ported for Android) ○ Iozone (ported for Android) ○ Overload and long run test - in progress!! ENGINEERS AND DEVICES WORKING TOGETHER Filesystems - Challenges ● Fixed build support for f2fs image generation (core.mk & image size alignment to 4096) ● Fixed sparse raw image generation issue ● Need to use for btrfs and nilfs ● Image generation for btrfs, nilfs, squashfs etc. (raw -> format -> sparse) ● Benchmark porting - bonnie, iozone ● Partition overload scripts and long run impact scripts ENGINEERS AND DEVICES WORKING TOGETHER Filesystems - Results • Given ranking based on performance for each benchmark and test – Average rank for iozone test (span over various record length) • Few more points to consider – Performance impact as filesystem ages – CPU utilization • O_SYNC (-+r option iozone) : requires that any write operations block until all data and all metadata have been written to persistent storage. This ensure file integrity (rather than data integrity with O_DSYNC flag) ENGINEERS AND DEVICES WORKING TOGETHER Filesystems - iozone average (full test) ● Write - btrfs (lzo/zlib) wins ● Read - ext4 performance is comparable to btrfs Note: nilfs failed to complete full iozone test ENGINEERS AND DEVICES WORKING TOGETHER Filesystems - small read and write (64K) ● Small records/file F2FS wins with sync option ● For read NILFS has better performance on cache read ENGINEERS AND DEVICES WORKING TOGETHER Filesystems - 1MB file test ● Ext4 outperform on all read operations ● F2FS has good score (with sync flag) ENGINEERS AND DEVICES WORKING TOGETHER Filesystems - 512MB, 4MB ● Write - btrfs (lZO), with sync flag ZLIB wins the race - not sure why? ● 4MB file read EXT4 ENGINEERS AND DEVICES WORKING TOGETHER Filesystems - bonnie results Low the better ● Btrfs (lzo, zlib) gives good number but.. ○ At the cost of CPU eating.. ○ No of kworker threads are more. Coming up next ● F2FS/Ext4 has fair amount of CPU usage on read/write ENGINEERS AND DEVICES ● F2FS outperform on char operation - do we have usecase ? WORKING TOGETHER Filesystems - hdparm ● Squashfs is better ( after btrfs ) ENGINEERS AND DEVICES WORKING TOGETHER Filesystems - speed variation Low the better ● Btrs wins for avg. speed ● But, speed (read/write) deviation is very less for f2fs ENGINEERS AND DEVICES WORKING TOGETHER Filesystems - disk access ● Disk reads are more for f2fs ( use of less buffered i/o) ● Nilfs disk read are less ● More writes for btrfs ( might be due background write activities, for snapshot handling) ● High disk utilization in case of nilfs ● NilFS if we do not run gc - 1000 runs, system went to out of disk space ENGINEERS AND DEVICES WORKING TOGETHER Filesystems - btrfs low lights Though BTRFS has good performance • High CPU Utilization: More kernel threads • For small data (<1MB), btrfs under perform over f2fs and ext4. Not recommended where small i/o transaction with sync is expected. E.g. frequent calls to DB entries. • Btrfs does not force all dirty data to disk on every fsync or O_SYNC operation ( risk on power/crash recovery) • Yet to test effect on long run test ?? ENGINEERS AND DEVICES WORKING TOGETHER File System analysis - Summary ● All relative rank graphs is available at ○ https://docs.google.com/a/linaro.org/spreadsheets/d/1ctknBBVWUjrIZwS8OQcb5L8gCd CLuzktJgx-K_CMgt0/edit?usp=sharing ● F2FS/Ext4 Wins for ○ Small File Access (4K-1MB) + DB Access with disk data integrity ○ Potential use case: Industrial monitoring system, Consumer Phone, Health monitoring system ● NilFS outperforms for SQLite operations ○ Only cache here is, metadata/data gets updated later once get written to log file ( kind of extended version of fdatasync over fsync) ○ Can be useful for power backed system and continuous log recording of small data (upto 4K) but with good amount of storage ○ It quickly fill up the space if GC is not called in between. On 5GB space, it just went out of space for 1000 runs of iozone test. Do not recommended for “Embedded System” ● SquashFS : Good buffered I/O read ○ Can be used for read only partitions ( system libraries and ro database) ENGINEERS AND DEVICES WORKING TOGETHER File System analysis - Summary ● BTRFS : Large file + large RAM ○ LZO - Outperforms for block write/read operations ( > 4MB) ○ Potential use case: ■ In flight entertainment system ( mostly for movies/songs/images etc..) ■ Portable streaming & recording devices ( should be power backed up) ○ Low lights: ■ High cpu utilization ( more no# of threads) ■ Not recommended where small i/o transaction with sync is expected ■ Risk on power failure recovery (Not high, but sometimes corrupt itsself) ● Hybrid use of different file systems on multiple partitions can improve overall performance e.g. ○ large read/write (movies, extra download) on BTRFS partition ○ All small read/write (docs, images) on f2fs/ext4 partition ○ All database access insert/update/delete on f2fs/nilfs partition ● Note: Yet to perform impact on file system as it ages ENGINEERS AND DEVICES WORKING TOGETHER Filesystems - Todo List ● Perform long run test (3-4 days, with various operations) and measure the impact ● Partition overload testing - impact on low disk availability ● Encryption impact ● Overhead of overlayfs etc. if we need to add drivers, HALs etc. for a specific piece of hardware to /system when otherwise using a common /system with HAL consolidation ● Any other ? ENGINEERS AND DEVICES WORKING TOGETHER Filesystems - Some points of discussion • Any other filesystems (out-of-tree, perhaps) we should look into? • Impact of storage technology (devices might start using NVMe) • Best way to measure filesystem longevity ENGINEERS AND DEVICES WORKING TOGETHER Thanks! Questions? <[email protected]> HAL Consolidation Rob Herring <[email protected]> HAL Consolidation - one build, many devices ● Goal is one Android build/filesystem per cpu architecture while maintaining configurability for device specific builds: http://tinyurl.com/zscbbrx ● A directory per feature for features more than just a config variable ● KConfig based configuration for features ● Supporting DB410c, HiKey, Nexus 7, QEMU, RaspberryPi 3 ● Tablet/phone or TV targets ● Next platforms or targets to add? ● Possible next config features: ○ Anything the next device needs ○ Any feature Linaro is working on ○ Custom compiler and compiler flags ○ Kernel build integration ○ malloc selection ENGINEERS AND DEVICES ○ f2fs filesystem WORKING TOGETHER HAL Consolidation - Graphics (Done) ● CI job for Mesa Android builds ● GBM based