Qualcomm Defines Premium- Smartphone Cameras By Aakash Jani Senior Analyst August 2021 www.linleygroup.com Qualcomm Defines Premium-Smartphone Cameras By Aakash Jani, Senior Analyst, The Linley Group The Snapdragon 888 and 888+ are the latest flagship processors from Qualcomm, offering a collection of industry-leading camera technologies that provide a lead over competitors on DXOmark’s camera and video benchmark. These processors enable feature-rich 4K HDR video and photography, AI-driven “3A” technology, and industry- leading hardware to allow one-inch camera sensors. Qualcomm sponsored this white paper, but the opinions and analysis are the author’s. In recent user surveys conducted by Qualcomm, consumers ranked camera quality as the second-most important feature when buying a new premium smartphone ($1,000+); battery life was first, as Figure 1 shows. Additionally, 60% said a premium camera would determine their next purchase. As smartphones become more feature driven, OEMs are increasingly designing their products around tangible capabilities, such as camera quality. Figure 1. Features determining consumer premium-smartphone purchases. Just behind battery life, camera quality is the second-most important factor when purchasing a smartphone. Additionally, new camera-driven capabilities, such as 8K resolution, help satisfy buyers’ desire for the latest technology and maximum overall performance. (Data source: Qualcomm) Raw camera quality (megapixels or zoom magnification) is no longer enough. Consumers are basing their purchasing decisions on premium features in addition to a flagship camera. Weekend campers need quick auto focus for the chance encounter with local wildlife. At-home chefs require built-in bokeh to accentuate their plating. Future pro athletes need highlight reels for prospective colleges. ©2021 The Linley Group - 1 - Qualcomm Defines Premium Smartphone-Cameras Smartphone cameras became the main driver for many customers’ decisions because of their imminent crossover with DSLR quality. They deliver double-digit increases in processing power year over year while continually updating features through software patches. Bulky DSLR cameras, by contrast, make smaller updates only about every two years. As a result, premium smartphones are already similar to mainstream DLSRs and are rapidly catching up to high-end prosumer models. Qualcomm enables smartphone OEMs to deliver these desirable features by providing industry-leading hardware and software expertise to facilitate camera design. Its latest premium Snapdragon mobile chips, the 888 and 888+, combine a triple-core image processor with an industry-leading AI engine that carries a rating of 32 trillion INT8 operations per second (TOPS). The company built the camera platform to process 2.7 gigapixels per second, supporting 4K HDR video and photography. Color and Resolution Technology Qualcomm introduced 4K video to smartphones in 2013 with the Snapdragon 800, the first Android chip to support that resolution. Five years later, the company added 10-bit color, allowing phones to capture 4K HDR video in one billion colors. In last year’s Snapdragon 865, it continued to make incremental upgrades to its 4K capability, including 120fps slow-motion video capture, and it introduced 8K video capture (at 30fps). The company delivered 10-bit color for video while competitors were still stuck at 8 bits, allowing Snapdragon to provide finer gradation between colors and tones. The difference between the two is most evident when shooting twilight landscapes: compared with 8-bit color, 10-bit color enables dark-blue and black regions to make smoother transitions from shadowy landscapes to the emerging sun. Early in Snapdragon’s development, Qualcomm acknowledged the shortcomings of the JPEG format, which is limited to 8-bit color. The 855 was the first Snapdragon to provide hardware support for HEIF photo and video capture. HEIF photos were still limited to 8 bits, but HEVC video was 10 bits. The Snapdragon 888 ultimately became the first smartphone processor to use HEIF to take photos and HEVC to record videos in 10-bit format for extended color capabilities. High-dynamic-range (HDR) photography has grown in popularity as more monitors and smartphone displays implement it. The technology captures differences in light with greater fidelity than standard definition range (SDR). Revisiting the twilight example, an HDR-enabled smartphone camera will capture interactions between light rays and clouds that an SDR-enabled camera would miss. Each smartphone OEM has a unique approach to HDR, which produces differences in the types of lighting it can handle. Smartphone vendors either use spatial, such as qHDR, or temporally multiplexed, such as MFHDR, sensors. The former includes specific pixels for different exposures, and the final image is typically created at one-fourth of the full ©2021 The Linley Group, Inc. - 2 - Qualcomm Defines Premium Smartphone-Cameras sensor resolution. These sensors excel at reducing ghosting artifacts but are limited in dynamic range. MFHDR sensors capture the exposure frames at different times, and the resultant frames are stitched together to produce a final full-resolution image. This method offers strong dynamic range, but the time lag can produce ghosting, as Figure 2 shows. As a compromise, OEMs currently use (and will continue to use in the near future) sHDR sensors that offer a compromise between qHDR and MFHDR. To offer its OEMs flexibility, the Snapdragon image processor supports all of these methods. Figure 2. HDR-sensor comparison. Relative to MFHDR, qHDR sensors reduce ghosting, but they have lower dynamic range; OEMs may therefore choose sHDR sensors as a compromise. Snapdragon image processors natively support all three of these types. (Images captured by Sony Semiconductor Solutions and processed by Qualcomm) AI Enables 3A and More Photographers can manually adjust the focus, exposure, and white balance of standard DSLR cameras in accordance with the subject to improve the quality of the shot. They can also remove blur from the foreground image by calibrating the camera’s diopter through the lens, optimizing focus. Exposure determines how bright or dark an image appears and is controlled by the aperture, shutter speed, and sensor integration time and gain. Overexposure leads to washed-out pictures, while underexposure produces grungy and murky pictures. White balance is closely tied with color temperature; it adds the opposite color to an image to rebalance the temperature. In daylight, the same photo with different white-balance settings can appear anywhere from gray to blue. ©2021 The Linley Group, Inc. - 3 - Qualcomm Defines Premium Smartphone-Cameras Qualcomm automated these three aspects of photography, using AI to increase photo quality. Its auto-exposure, auto-focus, and auto-white-balance (3A) effort began in 2014, employing dual LEDs for ambient-light matching and an auto-focus algorithm that prioritized face matching. Over the past seven years, the company hit 3A milestones by adding features such as scene-change awareness, motion-aware auto exposure, and IR/RGB-assisted auto white balance. These improvements can turn a smartphone user from an amateur photographer to a near professional. Qualcomm trained many of them by studying people’s eye movements as they viewed different images. Instead of always focusing on the foreground, Snapdragon products can distinguish between aspects of a scene (e.g., mountains or people) and auto focus accordingly. After talking with Snapdragon Insider and expert photographer Ryan Resatka, we learned about some important features that caught professionals’ eyes. Resatka was immediately enamored with the color profile in 888-powered devices. Color profiles act as a unique fingerprint for the chip vendor; the 888’s is cohesive and enables a device to capture authentic colors, requiring less postprocessing. Canon DSLR cameras set the standard for color profiles, but by implementing better color and white-balance technology, the 888 comes closer to that level. For Resatka, color is the most important feature when choosing a camera: smartphone, or DSLR. Additionally, he found that the auto focus of the 888 platform was extraordinarily accurate and that the auto white balance pairs well with the HDR sensors. The device allowed him to capture detailed shots even when shooting straight into the sun. For professional photographers, a phone comes in handy when their main camera runs out of battery or breaks. Snapdragon-based models create RAW files that meet the high standards of these specialists while providing an easy-to-use interface for amateurs and prosumers. Qualcomm integrates AI into more than just its 3A platform: it also applied its neural- network expertise to low-light photography and video. Software developer BlinkAI used Snapdragon’s sixth-generation AI Engine in the Xiaomi Mi 11 smartphone to capture video in as little as 0.1 lux, an Android first. As Figure 3 shows, the video reveals interactions between light and water, preserving both bright and dim areas. ©2021 The Linley Group, Inc. - 4 - Qualcomm Defines Premium Smartphone-Cameras Figure 3. Low-light video still. Through a partnership with BlinkAI, Qualcomm and Xiaomi released the first Android phone with low-light-video capabilities. (Image source: BlinkAI) Snapdragon Leads on DXOmark Smartphones are now the primary tool for amateur photography, so objective evaluations and rankings are important. DXOmark has become the leader