Build Community Android Distribution and Ensure the Quality Jim Huang ( 黃敬群 ) <[email protected]> Developer & Co-Founder, 0xlab http://0xlab.org/ Oct 28, 2011 / ELC Europe (Prague) Rights to copy © Copyright 2011 0xlab http://0xlab.org/ [email protected] Attribution – ShareAlike 3.0 You are free Corrections, suggestions, contributions and translations are welcome! to copy, distribute, display, and perform the work to make derivative works Latest update: Oct 28, 2011 to make commercial use of the work Under the following conditions Attribution. You must give the original author credit. Share Alike. If you alter, transform, or build upon this work, you may distribute the resulting work only under a license identical to this one. For any reuse or distribution, you must make clear to others the license terms of this work. Any of these conditions can be waived if you get permission from the copyright holder. Your fair use and other rights are in no way affected by the above. License text: http://creativecommons.org/licenses/by-sa/3.0/legalcode 0x1ab = 162+16x10+11 = 427 (founded on April 27, 2009) 0xlab is another Hexspeak. About Me (1) Come from Taiwan (2) Contributior of Android Open Source Project (AOSP) (3) Developer, Linaro (4) Contributed to GNU Classpath / Kaffe, Linux internationalization (i18n), Openmoko Commercial Partners of 0xlab ARM / Linaro Contribute to Linaro Android since the first line of code AzureWave Build wireless networking & image processing solutions Mediatek Android based consumer products Open Embedded Software Foundation Contribute to the reference implementation Eventually, partners can benefit from open http://0xlab.org/partners.html source efforts and our experience. Agenda (1) Build Android distribution (2) Lesson learned from AOSP (3) Ensure the Quality (4) Bring enhancements back to Community Build Android Distribution based on non typical open source projects The reason why we built community Android Distribution: Initially, we just wanted to enable wireless connectivity features on Android for our hardware partnters. But, we never thought that it was difficult to do things efficiently. 0xdroid: enhanced Android distribution Beagleboard DevKit8000 We suffered from performance and usability issues in AOSP. 0xdroid is basically the environment where we can develop and experiment. What does 0xdroid deliver? • Hardware enablement: Beagleboard (TI OMAP3), Pandaboard (TI OMAP4), Snowball (ST-Ericsson Ux500; on-going) • Provide full source code for HAL • Usability: software cursor, window manager fix, large screen tweaks, network connectivity fix • Performance: ARM specific optimizations, graphics enhancement • Features: Bluetooth HID (keyboard/mouse), external modem, 3D effects, customized Launcher But, 0xlab is not really making yet another Android distribution. We wish to help community. 0xdroid is just a testbed (or reference implementation), and the valuable changes should be merged in upstream or other community projects. Strategy and Policy • open source efforts to improve AOSP • We focus on small-but-important area of Android. – toolchain, libc, dynamic linker, skia, software GL, system libraries, HAL, UX • Develop system utilities for Android – benchmark, black-box testing tool, validation infrastructure • Feature driven development – Faster boot/startup time, Bluetooth profile, visual enhancements • Submit and share changes to... – AOSP, CyanogenMod, Android-x86, and Linaro Working Model by 0xlab Rowboat CyanogenMod Android-x86 Lesson Learned from AOSP Let's go Upstream! Unfortunately, contributing to AOSP is an __art__. You never know how Google thinks of your patches exactly, even through Gerrit (code review system). Problems We faced • AOSP looks like “An Open Source Pretender" • No public roadmap – Therefore, we ignore the modifications against Android framework. • The merged changes usually show up in next 1 or 2 public release. – It is really hard to introduce/track the relevant changes. • Not clear discussions on android-contrib mailing- list. Sometimes, you have to have private communications to Google engineers. • Version control / Code Review on invisible repositories (internal and far-away GIT tree) Change represented in Gerrit gerrit upstream Change 123 Patchset 1 dev-b dev-a Commit cdeb34 Create Local % git checkout -b topic-branch Branch Patchset 2 Commit 41dbe5 work Google provides the great code Push % git push gerrit \ review tool for AOSP, but... to HEAD:refs/for/master Gerrit Flow of AOSP submitted changes Create Local Branch Fix work commit rejected rebased! Push to Review Upstream repo Gerrit approved/ submitted Here “upstream” means AOSP master and Google internal tree. NoNo explicitexplicit changechange historyhistory AndroidAndroid stylestyle openopen sourcesource modelmodel Gap between Goolge internal GIT/Perforce and AOSP Git After Gingerbread, it gets much clear for toolchain part prebuilt/prebuilt/ commitcommit 81cce608ab19dcd0aaf7d08d57a4460229e43c4581cce608ab19dcd0aaf7d08d57a4460229e43c45 Author:Author: JingJing YuYu <[email protected]><[email protected]> Date:Date: TueTue DecDec 1414 10:55:2310:55:23 20102010 -0800-0800 PatchedPatched toolchaintoolchain toto fixfix aa fewfew gccgcc andand binutilsbinutils bugs.bugs. SourcesSources toto buildbuild thisthis toolchaintoolchain areare listedlisted onon arm-eabi-4.4.3/SOURCESarm-eabi-4.4.3/SOURCES linux-x86/toolchain/arm-eabi-4.4.3/SOURCESlinux-x86/toolchain/arm-eabi-4.4.3/SOURCES build/build/ syncedsynced toto commitcommit 4cc02faaa7e8828f9458b1828a6f85e7791ae2aa4cc02faaa7e8828f9458b1828a6f85e7791ae2aa Author:Author: JimJim HuangHuang <[email protected]><[email protected]> Date:Date: FriFri AugAug 2020 23:30:3723:30:37 20102010 +0800+0800 AndAnd rollbackrollback thethe followingfollowing 33 patches.patches. commitcommit de263c26a7680529baca731c003bc58b68d72511de263c26a7680529baca731c003bc58b68d72511 Author:Author: JingJing YuYu <[email protected]><[email protected]> Date:Date: ThuThu AugAug 1212 15:52:1515:52:15 20102010 -0700-0700 AlthoughAlthough wewe cancan checkcheck gitgit log,log, wewe stillstill havehave nono ideaidea whywhy theythey changedchanged Observed AOSP Working Model • “master” branch in Android is the bridge between AOSP and Google internal tree. There are many contributions merged from companies, organizations, and individuals. But no efficient code review available for non-existing repositories. And, only few Google engineers do review changes. • Master branch = the latest AOSP + Partial changes by Google (bug-fixes from internal tree) –Not fully verified codebase. • The best hints are the opinions written by Google engineers inside Gerrit. –Send patches if possible My interpretation of Android: Hardware is Revolution; Sotware is basically Evolution; Android is Hardware-driven Software Revolution Functional View (Android 1.5) ApplicationsApplications Gallery Phone Web Browser Google Maps ・・・・・ AndroidAndroid FrameworkFramework Activity Window Content Notification View System Manager Manager Manager Manager Package Telephony Resource Location Manager Manager Manager Manager SystemSystem LibraryLibrary AndroidAndroid RuntimeRuntime SurfaceFlingerSurfaceFlinger OpenCOREOpenCORE SQLite Class Library DalvikDalvik VirtualVirtual MachineMachine OpenGL|ESOpenGL|ES AudioFlingerAudioFlinger WebKit SGLSGL OpenSSL bionicbionic libclibc Freetype LinuxLinux KernelKernel Functional View (Android 2.3) ApplicationsApplications Gallery Phone Web Browser Google Maps ・・・・・ AndroidAndroid FrameworkFramework Activity Window Content Notification View System Manager Manager Manager Manager Package Telephony Resource Location Manager Manager Manager Manager RenderScript OpenGL|ES 2.x accelerated. SystemSystem LibraryLibrary AndroidAndroid RuntimeRuntime Drop 2D accel SurfaceFlingerSurfaceFlinger StageFright SQLite Class Library V8 bridge DalvikDalvik VirtualVirtual MachineMachine GLES 2.0 OpenGL|ESOpenGL|ES AudioFlingerAudioFlinger WebKit SMP improvements SkiaSkia OpenSSL bionicbionic libclibc Freetype JIT compiler Skia supports GPU backend SMP fixes In Android ICS LinuxLinux KernelKernel TheThe overalloverall designdesign isis consistant,consistant, butbut thethe currentcurrent modelmodel preventsprevents fromfrom diversediverse communitycommunity contributions.contributions. AOSP statistics for Gingerbread (Dec 2010) 4204 google.com 1354 android.com 17 openbossa.org 98 sonyericsson.com 11 nxp.com 71 gmail.com 11 linux.org.tw 39 codeaurora.org 10 ti.com 39 samsung.com 10 acer.com.tw 38 intel.com 8 themaw.net 32 nokia.com 8 garmin.com 32 holtmann.org 7 snpe.rs 29 0xlab.org 7 motorola.com 25 trusted-logic.com 7 mc.pp.se 7 googlemail.com The number are commits since Froyo release. However, the valuable changes from community such as CyanogenMod are usually absent due to long-time review process. Ensure the Quality when building custom Android distribution and merging changes from community Mission in our development: Improve UX in SoC UX = User Experience SoC = Integrated Computing Anywhere Quality in custom Android Distribution • 0xlab delivers the advantages of open source software and development. – Quality relies on two factors: continuous development + strong user feedback • Several utilities are developed to ensure the quality and released as open source software. – 0xbench (Android benchmarking tool) – ASTER (Android System Testing Environment and Runtime) – LAVA (Linaro Automated Validation Architecture) • In the meanwhile, performance is improved by several patches against essential components. LAVA: Automated Validation