Multi-Camera Applications Mehran Ayat Sr. Dir. Product Marketing and Amateur Photographer To go beyond human vision 3 InvenSense Developers Conference 2016 Human Vision facts (sort of ) 1. An angle of view of about 140° by 80° – Actual focal length of ~22mm with a view of ~90° 2. Perceive ~576 MP @120° angle of view – View extends to ~180 ° with higher equivalent resolution – Perception is via a combination of eyes and brain 3. Sensitivity equivalence of 800 ISO for adapted eye – Not a single shot frame, rather a video stream – It takes ~half hour for eyes to adapt to the dark – In day time sensitivity is 600X less at about 1 ISO! 4. Dynamic range of over 1 million to 1 – A range of ~12.5 - 15 stops (doubling or halving of light) – Seeing a faint star with moon in your eyes is 12.5 stops Sources: Clarkvision.com and Wikipedia 4 InvenSense Developers Conference 2016 Multi-Camera Applications (some!) Apple iPhone 7+ Huawei P9 Giroptic 360cam Facebook Surround360 Light L16 Nikon KeyMission 360 5 InvenSense Inc. Company Confidential Why 2+ cameras instead of one? 1. Improve image quality where space is limited – Smartphones most critical dimension is height 2. More than one focal length and Optical zoom – Wide, Very wide, normal and telephoto options 3. Wrap around view (360°) for Sports, AR, VR – One camera cannot see behind its back 4. Depth perception for Bokeh, AR and refocusing – Much easier to drive a depth map with 2+ cameras 5. The wholly grail! DSLR quality in your pocket – Cheap camera modules thanks to Smartphones 6. Combination of some or all of the above! 6 InvenSense Developers Conference 2016 Image Sensor size, matters! 4.5 x 3.4 mm Full frame DSLR image sensor is 56X the size of smartphones A full frame sensor is equivalent to 35 mm film 7 InvenSense Developers Conference 2016 Dual-Camera for image Quality • Dual-identical cameras – Same 12 MP image sensor – Same lens / focal length – One without RGB filter: Monochrome – No Optical Image Stabilization Huawei P9 • Pro – Better low light with 2X+ light absorption – Allows depth perception and refocusing • Con – Single camera with OIS gathers 4X to RGB Filter 8X the light absorption in a phone – Pixel level alignment of image sensors 8 InvenSense Inc. Company Confidential Dual-Camera: Wide, Tele, Zoom • I7+: Normal and Tele camera – Optical zoom, up to 2X with no image quality penalty, same resolution – Normal camera has OIS with good iPhone 7+, Wide with OIS low light image quality Normal (tele), no OIS • LG G5 – Normal lens with OIS, good low light – Super wide view camera, no OIS LG G5: Wide 16MP w. OIS – No zooming ability between cameras Super Wide, 8MP, 135°, no OIS ~7+ Wide ~7+ Normal 9 InvenSense Inc. Company Confidential Multi-Camera Virtual Reality • Images from 3 x 185° view cameras • 360° x 300° view with onboard stitching • Six-axis sensor for image stabilization • VR compatible with common viewers Giroptic 360cam 10 InvenSense Inc. Company Confidential Depth Map for Bokeh • i7+ Tele lens focal length of 52mm – ”Normal” perspective ~50mm Focal L. – A DSLR ”Portrait” lens is 75 to 135mm • I7+ “Bokeh” with SW update – Bokeh is “the quality” of out of focus area of an image – DSLR tele lens w. aperture ≤ 2.8 achieves good bokeh – A dual-camera lens can drive bokeh with depth map 11 InvenSense Inc. Company Confidential Computational Imaging: Light L16 • Digital computation instead of optical processing – E.g. Panorama, HDR & Light field (intensity + direction) • Combine 16 cameras to achieve 35-150 mm FL • Promises DSLR level image quality in your pocket • Folded optics: Long focal length, in inch thick form • Android based, heavy computational power 12 InvenSense Developers Conference 2016 What does it all mean to me? • Multi-cameras are way of the future in mobile space • Ability to extract depth and direction of light as opposed to light intensity only with single camera • Depth perception is a big challenge for AR – Ability to line up real objects with virtual objects • API-reference with camera raw output to provide unique applications by developer community 13 InvenSense Developers Conference 2016 Thank You.