Optimizing Mobile Display Brightness by Leveraging Human Visual Perception Matthew Schuchhardt Susmit Jha Northwestern University Intel Corporation [email protected] [email protected] Raid Ayoub Michael Kishinevsky Gokhan Memik Intel Corporation Intel Corporation Northwestern University [email protected] [email protected] [email protected] ABSTRACT that after two Samsung display design iterations, normalized Modern smart phones and tablets are battery-constrained full-brightness per-pixel power efficiency only improves by by their mobility; this constraint is heavily factored into 5.7%; this is reduced even further when the current trend any design decision made on the device. Furthermore, the toward increasingly larger screens is considered. Furthermore, display is one of the most power-consuming subsystems. they find that the most significant contributor to power Adaptive display brightness systems attempt to address this consumption is the size of the screen itself. With larger high display power consumption by setting the brightness devices, although the battery tends to be bigger, the screen is depending on the surrounding ambient light levels. also larger, and considering the lack of significantgenerational In this work, we run a series of user studies aimed at improvements in power consumption, there is no reason to gauging the accuracy of these adaptive brightness models. expect that display power will not continue to be a significant These studies compare subjective satisfaction and readability consumer of power in mobile devices. metrics on a common smartphone display and find that the Because the display consumes a significant amount of adaptive brightness system isn't well-tuned to user prefer­ power, adaptive brightness systems are one way that dis­ ences. We also find that subjective ratings and readability play manufacturers attempt to reduce the display's power are closely correlated with one another across different dis­ envelope while improving user satisfaction with the display's play brightness levels, which can be used to give a better brightness. Adaptive brightness systems control a display's understanding of how screen brightness levels impact users. brightness via ambient light sensors which give an indication Additionally, we note that operating systems need to re­ of incident environmental light on the screen. These adap­ duce the power envelope of the device for a variety of reasons. tive brightness systems can significantly reduce the power This is done without understanding the impact of these deci­ usage compared to a display which is constantly set to a high sions on user satisfaction, and can significantly degrade user brightness level. Also, adaptive brightness systems have the experience. In this work, we propose a user-aware method advantage of causing less eye strain by avoiding setting the of reducing display power consumption, which allows display brightness too high when in a dim lighting environment [12]. power to be throttled while understanding the resulting im­ In this work, we wish to gauge how effective these adaptive pact on the user. By considering this impact on the user, brightness systems actually are at satisfying users' brightness we devise an optimal dimming scheme which can reduce requirements. To do this, we design a user study which the time-weighted readability degradation by upwards of allows us to gather subjective display brightness satisfaction 21.5%. ratings and readability data for a series of users at a selection of ambient light and screen brightness levels. From these results, we find that the default automatic brightness system 1. INTRODUCTION that comes shipped with the device isn't well-tailored to the In modern mobile devices, the touchscreen display has group of users we studied. taken front and center as the most important interface that Furthermore, we find that subjective user ratings are users interact with. Not only does the touchscreen display closely correlated with an objective readability metric, espe­ act as the primary means of output to the user, but with the cially in brighter lighting environments. This strong, direct absence of hardware keyboards and buttons on many mobile correlation is interesting because it suggests that by using devices, it is now the primary means of input as well. With this relationship, it's possible to understand how much of an this elevated station that touchscreen displays have come impact selecting a given brightness level will have on a user, to enjoy in smartphones and tablets, more and more focus without needing to directly gather subjective ratings from is being placed on the design and features of the display. the user. The trend of mobile displays is increasingly toward larger, We also study the impact that various brightness throttling brighter, higher-resolution displays [10], and this trend is schemes have on users. System power may need to be reduced undoubtedly a sight for sore eyes everywhere. below the standard power envelope for many reasons. For However, this trend toward larger and brighter displays example, one common reason for reducing display brightness does not come without cost; although many manufacturers is low-battery situations. When the battery falls under a tote the energy efficiency of their displays, Chen et al. [3] find critical threshold, many smartphones reduce the display's that generational improvements in OLED displays account brightness to help extend the device's remaining on-screen for minor improvements in power consumption. They find 978-1-4673-8320-2/15/$3l.00 ©20 15 IEEE 11 time. Additionally, there are systems such as system-level Luminance is defined as the intensity of light per unit area, dynamic thermal management [9], which reduce system power and is colloquially referred to as brightness in the context consumption in an attempt to reduce the system's total heat of mobile displays. Contrast ratio is related to luminance; levels. Any OS subsystem can cause power throttling events, generically defined, contrast ratio compares the luminance but the underlying issue with these systems is that they do of the colors black and white (Lblack and Lwhite) as they ap­ not consider the impact that these power savings decisions pear on the display. Lblack is never exactly zero on an LCD have on the user. display; there is always some amount of light which leaks To explore this further, we conduct another study which through the liquid crystal matrix, even on an image which collects data on what lighting environments are commonly is intended to be fully black. Similarly, LWhite is limited encountered throughout the day by a typical user. Using by how bright the LCD's backlight is. The contrast ratio these results along with the previous readability and ratings describes the relative difference between these luminances. data, we directly analyze how much power can be saved with However, there is not a single well-defined metric which de­ a specified degradation in readability. We also compare a scribes contrast ratio, and there are a number of different user-agnostic system which degrades the display by a con­ equations which can represent this. One common formula stant brightness level or constant brightness percentage, to for a display's contrast ratio is described in Equation (1): our proposed user-aware system which degrades the display's brightness more in brighter environments. Any backlight Lwhite - Lblack C = (1) dimming power savings decision will impact users to some Lwhite degree, but by considering how these decisions affectthe user, There are some complexities as to how this is measured it's possible to reduce the average impact over a period of and reported in consumer devices (such as dynamic contrast time. Using our user-aware brightness throttling scheme can ratio, static contrast ratio, etc.), but as far as light perception achieve an 8% average system power reduction while only is considered, this definition is sufficient. It is important to degrading average readability by 3.2%; this is 21.5% less of a note, and are not entirely self-contained to the degradation than the next best user-agnostic method. Even Lblack Lwhite display itself; the device's ambient light can have a signifi­ more importantly, this method allows us to directly under­ cant impact on these values. For instance, in a completely stand how the power decisions will affect users, which makes darkened room, the contrast ratio is precisely defined as these power decisions more well-informed than arbitrarily in Equation (1). However, consider a lighting environment throttling system power. which involves a significant amount of ambient light. Since In summary, we make the following primary contributions: the display's optical characteristics necessarily don't absorb 100% of incoming ambient light, some of that light is reflected • We find a correlation between subjective user ratings and an objective readability metric back at the viewer. This reflected light reduces the resulting contrast ratio, as evidenced in Equation (2), with E repre­ - We run user studies which gather subjective ratings senting the environmental illuminance and p representing the and visual discernment task data across multiple light­ display's reflective coefficient. Contrast ratio has a significant ing environments impact on discerning visual detail, and this relationship of - We present analysis which shows a strong correlation contrast ratio with ambient light explains why it is easier to between objective visual discernment and subjective read backlit displays in darker ambient environments. This is ratings also the basis for the inclusion of adaptive brightness systems - We perform the first analysis on the impact of adap­ in mobile devices. Adaptive brightness systems use ambient tive screen brightness mechanisms on readability light sensors to help control the display's brightness: in dim • We create a model for screen brightness which maximizes environments, the display reduces the display's brightness, time-averaged screen readability and in brighter environments, the display is set to higher - We compare existing adaptive brightness systems to brightness levels.