Quantum-Inspired Computational Imaging Yoann Altmann, Stephen Mclaughlin, Miles J
Total Page:16
File Type:pdf, Size:1020Kb
RESEARCH ◥ emerging technologies are now relying on REVIEW SUMMARY sensors that can detect just one single photon, the smallest quantum out of which light is made. These detectors provide a “click,” just OPTICAL IMAGING like a Geiger detector that clicks in the pres- ence of radioactivity. We have now learned to use these “click” detectors to make cameras Quantum-inspired that have enhanced properties and applica- tions. For example, videos ◥ computational imaging ON OUR WEBSITE can be created at a tril- Read the full article lion frames per second, Yoann Altmann, Stephen McLaughlin, Miles J. Padgett, Vivek K Goyal, at http://dx.doi. making a billion-fold jump Alfred O. Hero, Daniele Faccio* org/10.1126/ in speed with respect to science.aat2298 standard high-speed cam- .................................................. eras. These frame rates BACKGROUND: Imaging technologies, which 10 billion photographs were taken per year. are sufficient, for example, to freeze light in extend human vision capabilities, are such a Facilitated by the explosion in internet usage motion in the same way that previous photo- natural part of our current everyday experience since the 2000s, this year we will approach graphy techniques were able to freeze the that we often take them for granted. However, 2 trillion images per year—nearly 1000 images motion of a bullet—although light travels a the ability to capture images with new kinds of for every person on the planet. This upsurge is billion times faster than a supersonic bullet. Downloaded from sensing devices that allow us to see more than enabled by considerable advances in sensing By fusing this high temporal resolution to- what can be seen by the unaided eye has a rel- and data storage and communication. At the gether with single-photon sensitivity and ad- atively recent history. same time, it is driving the desire for imaging vanced computational analysis techniques, a In the early 1800s, the first ever photograph technology that can further exceed the capabil- new generation of imagingdevicesisemerging, was taken: an unassuming picture that required ities of human vision and incorporate higher- together with an unprecedented technological days of exposure to obtain a very grainy image. level aspects of visual processing. leap forward and new imaging applications http://science.sciencemag.org/ In the late 1800s, a photograph was used for that were previously difficult to imagine. the first time to see the movement of a running ADVANCES: Beyond consumer products, re- For example, full three-dimensional (3D) horse that the human eye alone could not see. search labs are producing new forms of imag- images can be taken of a scene that is hidden In the following years, photography played a ingthatlookquitedifferentfromanything behind a wall, the location of a person or car pivotal role in recording human history, rang- we were used to and, in some cases, do not can be precisely tracked from behind a corner, ing from influencing the creation of the first resemble cameras at all. or images can be obtained from a few photons national parks in the United States all the way Light is typically detected at relatively high transmitted directly through an opaque mate- to documenting NASA’s Apollo 11 mission to intensities, in the spectral range and with frame rial. Inspired by quantum techniques, it is also put a man on the Moon. In the 1990s, roughly rates comfortable to the human eye. However, possible to create cameras that have just one pixel or that combine information from mul- tiple sensors, providing images with 3D and on August 16, 2018 spectral information that was not otherwise possible to obtain. OUTLOOK: Quantum-inspired imaging tech- niques combined with computational approaches and artificial intelligence are changing our per- spective of what constitutes an image and what can or cannot be imaged. Steady progress is being made in the direction of building cam- eras that can see through fog or directly inside the human body with groundbreaking poten- tial for self-driving cars and medical diagnosis. Other cameras are being developed that can form 3D images from information with less than one photon per pixel. Single-photon cam- eras have already made their way into widely sold smartphones where they are currently used for more mundane purposes such as focusing the camera lens or detecting whether the phone is being held close to one’s ear. This technol- ogy is already out of the research laboratories and is on the way to delivering fascinating im- aging systems. ▪ The list of author affiliations is available in the full article online. *Corresponding author. Email: [email protected] Quantum-based imaging systems are being developed to image through opaque media Cite this article as Y. Altmann et al., Science 361, eaat2298 PHOTO: KEVIN J. MITCHELL (e.g., fog or human tissue) that scatter light in all directions. (2018). DOI: 10.1126/science.aat2298 Altmann et al., Science 361, 660 (2018) 17 August 2018 1of1 RESEARCH ◥ of light when using single-photon detectors has REVIEW stimulated the birth of new computational tech- niques such as “first-photon imaging,” which hints at the ultimate limits of information to be OPTICAL IMAGING gained from detecting just one photon. Single-pixel and ghost imaging Quantum-inspired Although most imaging techniques that have emerged recently are based on classical detec- tors and cameras, some of these approaches have computational imaging been inspired by or have a tight connection with 1 1 2 3 similar ideas in quantum imaging. A prime ex- Yoann Altmann , Stephen McLaughlin , Miles J. Padgett , Vivek K Goyal , 1 4 2 ampleisghostimaging(Fig.1)( ), originally Alfred O. Hero , Daniele Faccio * thought to be based purely on quantum princi- ples but now recognized as being dependent Computational imaging combines measurement and computational methods with the on spatial correlations that can arise from both aim of forming images even when the measurement conditions are weak, few in number, quantum and classical light (2). The realization or highly indirect. The recent surge in quantum-inspired imaging sensors, together with that this technique does not require quantum a new wave of algorithms allowing on-chip, scalable and robust data processing, has light led to a merging of the fields of computa- induced an increase of activity with notable results in the domain of low-light flux imaging tional ghost imaging (3) and work on single-pixel and sensing. We provide an overview of the major challenges encountered in low-illumination 4 cameras ( ), as well as to an overall increase of Downloaded from (e.g., ultrafast) imaging and how these problems have recently been addressed for activity in the field. In its quantum version, ghost imaging applications in extreme conditions. These methods provide examples of the imaging refers to the use of parametric down- future imaging solutions to be developed, for which the best results are expected to arise conversion to create pairs of photons with corre- from an efficient codesign of the sensors and data analysis tools. lated positions. If we detect the position of one photon with a standard camera and illuminate omputational imaging is the fusion of com- imaging with single-photon detectors can im- an object with the other position-correlated pho- putational methods and imaging techniques prove 3D reconstruction and provide richer in- ton, it is sufficient to detect only the reflectance http://science.sciencemag.org/ with the aim of producing better images, formation about a scene. or transmittance of the object with a single-pixel C where “better” has a multiplicity of mean- We discuss how single-photon detection tech- detector—i.e., to measure the correlation count ings. The development of new imaging nologies are transforming imaging capabilities between the beams to then reconstruct a full sensors and, in particular, instruments with single- with single-photon–sensitive cameras that can image by repeating the measurement with many photon sensitivity, combined with a new wave of take pictures at the lowest light levels and with different photon pairs (each of which will be computational algorithms, data handling capa- the ability to create videos reaching a trillion randomly distributed because of the random bility, and deep learning, has resulted in a surge frames per second. This improvement has en- nature of the correlated photon pair generation of activity in this field. abledthecaptureofimagesoflightbeamstrav- process) (5, 6). It is now acknowledged that the One clear trend is a shift away from increasing eling across a scene and provided opportunities quantum properties of the correlated photons the number of megapixels and toward fusing to observe image distortions and peculiar rela- play no role in the image reconstruction process: camera data with computational processing and, tivistic temporal inversion effects that are due Thermal light split into two beams using a beam on August 16, 2018 if anything, decreasing the number of pixels, to the finite speed of light. The ability to cap- splitter can be used equally effectively, albeit at potentially to a single pixel. The incoming data ture light in flight also underpins some of the a higher photon flux (7). Rather than using a may therefore not actually look like an image applications mentioned above—for example, the beam splitter, it is possible to use a spatial light in the conventional sense but are transformed ability to view a 3D scene from around a corner. modulator to create a pattern where the copy is into one after a series of computational steps Probabilistic modeling of the particlelike nature simply the computer memory. This approach and/or modeling of how the light travels through the scene or the camera. This additional layer of computational processing frees us from the Bucket detector, An chains of conventional imaging techniques and removes many limitations in our imaging Object capability.