[MOBILE PLATFORMS] Jinghao Shi, Edwin Santos, and Geoffrey Challen University at Buffalo Editors: Sharad Agarwal and Marco Gruteser WHY AND HOW TO USE PHONELAB While smartphone app marketplaces have enabled large-scale app-level experimentation, medium-scale experimentation with the platform code implementing the app interface and providing core device services remains difficult for academic researchers. But this is where many of the ideas currently being explored by the mobile systems community must be evaluated—including new networking protocols, security and privacy mechanisms, storage abstractions, and energy management strategies. To enable these experiments, we built and are operating PhoneLab, a 175-smartphone testbed where real users run experimental Android platform builds on their primary devices. We are eager to make PhoneLab useful to the mobile systems community. To aid in this effort, this article discusses why PhoneLab might be useful for your research and provides an overview of how to use the testbed, including examples drawn from our group’s current projects. 32 GetMobile October 2015 | Volume 19, Issue 4 [MOBILE PLATFORMS] he world’s two billion mobile played a significant role in democratizing enormous number of users, like the Carat2 smartphones together constitute the mobile app development by allowing a single personalized energy management app, which T largest, most powerful, and most developer to easily reach a global audience. was installed more than 750K times. We widely distributed system ever built. In In the same way, these marketplaces are expect app marketplaces to continue to play many ways, the incredible success of the also playing an increasingly important a vital role in allowing the mobile systems smartphone is built on decades of research role in enabling large-scale mobile systems community to evaluate ideas at scale. by the mobile systems community. But new research. Researchers can now make But what about the smartphone platform challenges and opportunities alike await experimental apps available to billions itself? Platform codebases are the millions next-generation mobile apps and systems of users, and rely on the fact that even a of lines of code that are responsible for that attempt to harness this increasingly tiny fraction of citizen scientists willing to performing crucial tasks such as determining powerful and pervasive network, and participate in their project can amount to a the device’s location, choosing which researchers must continue to find ways representative, statistically significant and available network to use, and managing to make these ubiquitous devices even globally distributed sample of thousands limited resources such as energy – tasks more effective. or even tens of thousands of users. As an of obvious interest to the mobile systems An important part of the smartphone example, the Netalyzr tool1 distributed by research community. Smartphone platforms success story has been the emergence of ICSI has been downloaded 24,000 times cannot be altered via apps distributed by app mobile app marketplaces, such as the Google by users in 120 countries. And if the app marketplaces, and they also limit the access Play Store. These distribution channels have provides a tangible benefit it may attract an of these apps to useful information and device capabilities – sometimes for security or privacy reasons, but at other times simply because the information is considered too difficult or not useful enough to expose. Microsoft’s and Apple’s platform codebases are proprietary, making experimentation impossible without insider support. But while the Android Open Source Project (AOSP) makes it possible to create fully functional custom Android platform images for select devices, many roadblocks still await researchers attempting to deploy modified platform images at even modest scale. Experimental participants must be recruited and, if their behavior is to be representative, convinced to swap their primary smartphone for (and port their number to) a new device. Data collection tools must be built and tested, and the platform changes themselves must be developed and deployed. These experimental barriers are making it difficult for today’s academic mobile systems research community to have a real impact on the most important parts of the most successful mobile systems platform: the smartphone. To simplify smartphone platform experi- mentation we are operating PhoneLab, a 1 http://netalyzr.icsi.berkeley.edu/ Photo, istockphoto.com Photo, 2 http://carat.cs.berkeley.edu/ Volume 19, Issue 4 33 [MOBILE PLATFORMS] smartphone platform testbed (https://www. demographics. We do know that we have and location homogeneity, PhoneLab is a phone-lab.org). PhoneLab is both a group participants in many different departments relatively poor environment to investigate of people and an ongoing community effort on campus, providing a reasonable level of this challenge. to maintain a well-instrumented version on-campus spatial coverage which can be of the AOSP useful for research purposes. important to certain studies. EXPERIMENTAL PROCESS PhoneLab consists of 175 participants that To simplify testbed administration, we To use PhoneLab, you first download and run custom Android platform images on operate PhoneLab as a Sprint corporate- modify our fork of the AOSP. Next, you their primary smartphone. Our PhoneLab liable service plan. PhoneLab participants work with PhoneLab administrators and platform image is open to the mobile pay $45 per month for an all-you-can-eat developers to test your changes. After your systems community to instrument and Sprint service plan which includes unlimited changes are validated to be safe and effec- modify. We provide the participants, reliable voice, messaging and data. Technically tive, they are distributed to participants data collection tools, and increasingly well resources are pooled and capped over all and data collection begins. Finally, after instrumented AOSP platform sources. You participants, but in practice the caps are verifying that your experiment has been provide the exciting research questions and generous enough to never limit participant cleared for human subjects compliance, you new ideas that help us learn more about and consumption. Cooperation from Sprint will be provided the data that it generated. improve these ubiquitous mobile devices. allows us to create an attractive price point Here we review each step in the process in In this article our goal is to provide for our participants, while they offload all more detail. an overview of PhoneLab’s capabilities in payment processing and customer support the hope that you will find it helpful in to us. Participants purchase service directly Instrumenting or Modifying enabling your research studies. We start from PhoneLab in six-month installments, the PhoneLab Platform: with some basic information about the with invoice and payment processing done Experimenting on PhoneLab begins with testbed, describe the process of performing using Square. Sprint’s coverage in the Buffalo making changes to the Android platform. a PhoneLab experiment, and then use some area is fairly good, including 4G LTE mobile PhoneLab hosts a mirror of the AOSP examples drawn from our ongoing projects data coverage in many locations. allowing experimenters to use Google’s to provide tangible examples of what Currently all participants use the Google Android well-documented platform build PhoneLab can do. Nexus 5 smartphone running a custom An- process as well as their repo and Gerrit droid platform image based on the Android source control tools. Only the repoURL PARTICIPANTS, PLANS, PHONES Open Source Project (AOSP). Currently used during the process of downloading PhoneLab is a smartphone platform testbed our PhoneLab branch is based on AOSP the platform sources is different – the rest located at the University at Buffalo (UB). version 5.1 “Lollipop,” an upgrade from of the workflow is identical. 175 participants – primarily UB, students, version 4.4.4 that participants received in We divide PhoneLab experiments faculty, and staff – receive discounted Sprint mid-September 2015. In previous academic into two categories: instrumentation and service and a Google Nexus 5 smartphone years participants use the Samsung Nexus S modification. Instrumentation consists of to use as their primary device. In exchange, (2012-13), Samsung Galaxy Nexus (2013- additional logging that records but does not they run a modified Android platform 14), and Google Nexus 5 (2014-15). Com- alter how Android operates. Modifications image containing instrumentation and pared to these previous smartphones the modify Android itself, possibly by altering potentially novel changes and features. Nexus 5 seems far superior, and has helped existing services or adding new features. PhoneLab participants were originally address battery lifetime and network quality Of course, modification experiments will required to be UB affiliates. Recently complaints registered by first- and second- usually need to generate data to evaluate we have relaxed that policy, but almost year participants. their changes, and so will require instru- all participants are still UB students, We are aware that in some ways mentation as well. faculty and staff. We have not yet polled PhoneLab represents an unusually homo- Instrumentation changes are usually the demographics of our 2014-15 geneous environment for smartphone ex- fairly well-contained and unlikely to participants,