Good Afternoon. My name is Lars Bishop, and I’m an engineer in NVIDIA’s Developer Technologies team. I focus on our Android application developers and on 3D rendering, especially on mobile platforms like NVIDIA’s SHIELD. 1 The basis of this session is my team’s own experiences working with two groups of people; first, our direct support interactions with Android TV and Android Mobile game and app developers. Many of these topics originally came in as developer questions. These formed the depth of the session. On the other hand, the session is also based on many discussions with and bug reports filed by our own NVIDIA software QA team, who test hundreds upon hundreds of game titles and apps on NVIDIA’s various mobile platforms. These interactions formed the breadth of this talk; frequent issues we see in a wide range of titles. We’re going to cover a mix of the basics and some more advanced Android feature topics. We’ve found getting the basics right from the start is the best way to have time for the advanced bits. 2 First, we want to discuss a few of the new features that Android Marshmallow brings to both mobile and Android TV developers. Along with that, there are some behavior changes that have caught quite a few developers off-guard; we’ll discuss them as well. 3 Android M’s new features cover a wide range. One of the most important to Android TV developers will be core support for 4K display. While a lot of the 4K talk is around streaming media apps, it is even easier for games to take advantage of this. Also, M adds adoptable storage, a feature that makes SD cards and USB drives even more useful on Android, both mobile and TV. Low-latency audio improves the impact of game sounds. 4 The first thing many folks do when they consider supporting Android M is to target M’s API level, 23 in order to gain access to the Java APIs for the new features. But this is a double-edged sword. Android works hard to avoid breaking or changing existing app behavior when an older-API app is installed on a new OS. But all bets are off if the app builds against the new OS’s API level. That rebuilding is a form of implicit “contract”; the developer gets access to the new APIs in their app, but they also agree to any documented behavior changes in the new platform. This caused issues in a number of games we tested. In a few cases, the developer seemed to have upgraded to the new API level as a matter of course when the SDK shipped, but did not actually test on Android M devices. As more and more devices shipped with M, the hidden issues that were lurking in their apps came out. The rule here is simple; if you build on an API level, test on devices of that API level. If you also export that your app will run on older OSes, then test on those and make sure your use of any newer APIs is conditional. 5 Android M adds a new Java API for supported display modes. Using these APIs, an application can determine if 4K resolutions are supported on the platform, grab the ID of a 4K mode and use it. Note that this only affects Surface Views, not core Android UI views. But Surface Views are the basis upon which all 3D rendering is done on Android, so games are covered here. There’s only two problems, really. One is a practical and likely temporary one. Some devices that can support 4K and are attached to 4K displays do not yet export the 4K modes via this API. So if 4K support is not indicated by these APIs, it may still be available to a game… The other issue is NativeActivity. 6 NativeActivity apps do not have direct access to the window parameters before creation, so the Java APIs for 4K aren’t applicable. Luckily, there is another method for native apps. The app can test the sys display-size property; if the display lists a 4K resolution, then there are native APIs that will let you create a 4K rendering surface. If the 4K mode is available, the app can simply use the set buffers geometry call they already use in their native activity setup code to set the desired resolution. And they will be able to render and display to 4K. Do not set the buffer size to 4K if the property indicates that the display is still 1080 P. If you do that, you’ll pay the price of rendering to 4K, but the scaler will just filter it to 1080 P and you will have wasted the perf. Note that this does require user intervention on their device right now. The user must enable 4K display on their device via the Android display HDMI settings panel. In fact, the app _could_ even redirect the user to that resolution control panel page directly with an intent; the package is listed on this slide. 7 If you build against API 23 for Android M, you implicitly sign on to a BIG change; permissions. Unlike the older system where users were shown a laundry list of every permission your app requested in a play store dialog, most of which they didn’t understand anyway, API 23 actually requires some permissions to be explicitly requested of and accepted by the user at run time! If you don’t do this, the APIs related to the permission will fail. This runtime request requirement is now true of all permissions that Android has declared as “dangerous”. As an example, Read Write external storage is now considered dangerous. That sounds pretty extreme, since historically a huge number of games listed this permission. But there’s more here in the details. 8 As of API level 19 (which seems so far ago now…) Android stopped requiring the read write external storage permission for most app. Apps that only accessed paths returned by the get external directories functions no longer needed those permissions. Those directories were considered “safe”. Only apps accessing non- app-specific directories in external storage had to request the permissions. Well, API 23 took this a step further; apps needing this wide-ranging access to external storage were now declaring a dangerous permission and have to make the request directly to the user via the request UI APIs. However, in practice, exceedingly few games need external storage beyond the per-app directories. So apps using those can simply stop requiring the external storage permission at all 9 Related to this permissions change is the fact that Android M now makes it possible to “adopt” external storage like SD cards and USB drives. The benefit of this is that the expandable storage can be used for a wider range of application data than before. For users, the complication is that Android encrypts the storage device as a part of the adoption process. For developers, the main complication is that storage paths change. So apps that cached fixed paths to resources found that under Android M, they could no longer find their data, and they often simply crashed. Basically, the fix here is to store and load application resource data and cache data from the paths exported via Context and Application Info and to do so on each access, and not cache expected paths for later use. 10 M was designed to improve user identity security, too. The biggest change that many games will see is that if their rolled their own D R M via bluetooth or Wifi MAC address, the functions returning those MAC addresses will return the same ID for all devices on M. In other words, those functions will be useless for identifying a device. In addition, apps that used the property “settings secure android I D” as a unique I D have another issue on their hands – Google specifically disallows using that I D to identify a user or device for targeted ads. They require that apps use the Google Play Advertising I D, because the user can choose regenerate that I D and wipe their advertising identity at any time. Using any other I D for targeted ads will get your app pulled from the play store. 11 Historically, Android had a bad reputation with respect to audio latency. In the early days, this was, frankly, deserved. However, as of Android L, and especially on Android M with NVIDIA Tegra-based systems, this latency has fallen dramatically. The fast paths used to be quite narrow, and using the wrong audio format, sampling rate, et cetera would quickly have you back on the road to high lag. However, with M, those fast paths are wider. The details are beyond what we have time for here, but Google has an excellent set of resources and samples that detail these new paths. The documentation there is focused on pro-audio developers, such as those writing guitar effect emulators and the like, but the output latency sections are absolutely appropriate for game developers. There is a feature tag for low-latency audio, but games should not require it. It is designed to be used with apps that are unusable when audio latency is not perfect. Games can usually handle either case; the sound is just more effective when it is low latency. 12 Next, lets cover a little bit on developing Android TV apps focusing on the core User TV experience.