Microoptical Multi Aperture Imaging Systems Dissertation zur Erlangung des akademischen Grades doctor rerum naturalium (Dr. rer. nat.) vorgelegt dem Rat der Physikalisch-Astronomischen Fakultät der Friedrich-Schiller-Universität Jena von Diplomphysiker Andreas Brückner geboren am 09. August 1980 in Magdeburg Gutachter 1. Prof. Dr. rer. nat. habil. Andreas Tünnermann, Friedrich-Schiller-Universität Jena 2. Prof. Dr. rer. nat. habil. Stefan Sinzinger, Technische Universität Ilmenau 3. Prof. Dr. Hans Peter Herzig, École Polytechnique Fédérale de Lausanne, Schweiz Tag der Disputation: 22.12.2011 Contents i Contents 1 Introduction 1 2 Fundamentals 4 2.1ApplicationsofMiniaturizedImagingSystems................ 4 2.2IntroductiontoDigitalVisionSystems.................... 5 2.2.1 ObjectiveLens............................. 6 2.2.2 ImageSensor.............................. 14 2.2.3 ImageProcessing............................ 19 2.3ScalingLimitsofSingleApertureImagingOptics.............. 20 3 Multi Aperture Imaging Systems 25 3.1NaturalArchetypesandStateoftheArtTechnicalSolutions........ 25 3.1.1 AppositionCompoundEyes...................... 26 3.1.2 SuperpositionCompoundEyes..................... 30 3.1.3 CompoundEyesofStrepsiptera.................... 32 3.2ClassificationofMultiApertureImagingSystems.............. 33 3.2.1 BasicParametersofMultiApertureImagingOptics(MAO).... 33 3.2.2 ConceptsforFieldofViewSegmentation............... 36 3.2.3 MultiApertureSuper-Resolution................... 41 3.2.4 ReasonsforaThicknessReductioninMAO............. 42 3.3MethodologyfortheOpticalDesignandSimulationofMAO........ 43 3.3.1 ParaxialModel............................. 45 3.3.2 ParaxialMatrixFormalism....................... 46 3.3.3 AnalyticalModeling.......................... 47 3.3.4 NumericalSimulation.......................... 50 4 Artificial Compound Eyes with Electronic Stitching of Segments 53 4.1ArtificialNeuralSuperpositionEye(ANSE)................. 53 4.1.1 IncreasedInformationCapacityofAPCOs.............. 53 4.1.2 FabricationandIntegration...................... 55 4.1.3 DemonstrationofIncreasedSensitivityandColorImaging..... 58 4.1.4 ContrastEnhancementbyImageDeconvolution........... 61 4.2ElectronicClusterEye(eCLEY)........................ 62 4.2.1 OpticalDesignforThinWafer-LevelCameraOptics......... 64 4.2.2 FabricationofeCLEYwithVGAresolution............. 66 Contents ii 4.2.3 SoftwareDistortionCorrectionandImageStitching......... 67 4.2.4 ExperimentalCharacterizationofeCLEYVGA........... 69 4.2.5 ImprovementsoftheeCLEY...................... 72 4.3SummaryofElectronicStitchingofSegments................ 74 5 Artificial Compound Eyes with Optical Stitching of Segments 75 5.1OpticalClusterEye(oCLEY)......................... 75 5.1.1 DesignofanArrayofMicro-TelescopesonSmallestFormat.... 75 5.1.2 PrototypeFabrication......................... 78 5.1.3 CharacterizationofoCLEY...................... 78 5.2Ultra-ThinArrayMicroscope......................... 81 5.2.1 HighResolutionImagingofLargeObjectFields........... 82 5.2.2 FabricationofaSystemwithAdaptableLateralFormat....... 83 5.2.3 ExperimentalCharacterization..................... 85 5.3MicroopticalGaborSuperlens(μoGSL).................... 88 5.3.1 Approach for an Ultra-Compact Objective with High Sensitivity . 88 5.3.2 DemonstrationofaPrototypeSystem................. 90 5.4SummaryofOpticalStitchingofSegments.................. 94 6 Conclusions and Outlook 96 Bibliography 99 Appendix 106 Abbreviations and Symbols 122 Acknowledgments 128 Zusammenfassung 130 Ehrenwörtliche Erklärung 131 Lebenslauf 132 Wissenschaftliche Veröffentlichungen 133 Die Menschheit existiert nicht um der Wissenschaft willen, sondern die Wissenschaft sollte betrieben werden, um die Bedingungen der Menschheit zu verbessern. Mankind does not exist for science’s sake, but science should be used to improve the conditions of mankind. Jozef Maximilián Petzval (1807-1891) 1 1 Introduction Vision is by far the most important sense of human beings. It enables the exact orientation in the 3D environment, reaches up to several kilometers in distance and has always been a transport medium of human’s memories. The majority of man-made optical imaging sys- tems follows the design principles of single aperture optics which is the basic architecture of all known mammalian eyes [1]. Single aperture refers to the fact that all light which is recorded in the image passes through a single clear aperture of the optical system. Even the ancestor of the photographic camera, the ’Camera obscura’ which had no lens at all, belongs to this type [2, 3]. In the second half of the 19th century, the manufacturing of optical glasses of constant quality, enabled the spread of matured optical systems such as telescopes, microscopes, and photographic objectives into scientific, medical, and con- sumer applications [4]. About one hundred years later, the fabrication of opto-electronic image sensors by photolithographic techniques known from micro-electronics engineering and the development of precise molding of plastic lenses using high grade optical polymers have paved the way for the miniaturization of imaging systems [5]. Shrinking pixel sizes enabled the increase of resolution in image space while reducing the image sensor size and the fabrication of highly precise aspherical lenses led to the development of compact and cheap optical imaging systems. Hence, digital cameras are now an integral part of various electronic products such as laptops, tablet PCs, and mobile phones but also automotive safety systems, video endoscopes, and point-of-care diagnostics to name only a few. At present, miniaturized digital imaging systems are rarely smaller than 4 mm2 in footprint size with a thickness of 3 mm at 1.75 μm pixel pitch. A further downscaling yields pixel sizes below the diffraction limit, and thus a reduced spatial resolution as well as increased noise. Although this might be tolerable in consumer digital cameras, the effect on the performance of subsequent image processing and analysis is not acceptable in machine vision, industrial inspection, and medical imaging. Additionally, technological limits apply such as tight mechanical tolerances which severely increase the costs for a hybrid integration of small lenses. Lower f-numbers are hard to achieve, so that the diffraction limit cannot be shifted, leading to insufficient performance with standard fabrication techniques [6, 7]. For a further miniaturization of optical imaging systems, it is worth looking at some of the fascinating approaches which have been present in the tiniest creatures in nature for millions of years. The evolutionary solution of choice is found in the vision systems of invertebrates - the compound eyes. Here, a large number of extremely small vision systems (called ommatidia) on a curved base capture the visual information of a large field of view (FOV) in parallel at the sacrifice of spatial resolution. Each ommatidium itself has a small diameter and a low information capacity when compared to the single aperture eye. However, due to the large number of channels, a high information capacity of the overall multi aperture objective can be achieved. Since the dawn of digital image sensors, several technical derivatives of compound eyes 2 have been realized in order to miniaturize man-made vision systems [8–14]. However, since the major challenge for a technical adoption of natural compound eyes is the required precision of fabrication and assembly, none of these attempts has led to a successful tech- nology, since macroscopic fabrication methods were exploited for the manufacturing of microoptical structures. First examples of a realization by well adapted microoptical tech- nologies provided the thinnest known optical sensors which were promising even though the achieved resolution was not yet appropriate for technical applications [15–20]. The aim of this thesis is the investigation of the prospects of multi aperture imaging optics (MAOs) in a combination of opto-electronics, optics, and image processing. Therefore, promising design rules of natural compound eyes are studied and adopted for technical so- lutions. In contrast to prior work, where the size advantage is often obtained at the sacrifice of resolution, here, the highest degree of miniaturization should be achieved while main- taining optical system parameters (such as resolution and field of view) which are relevant for industrial applications. General principles of insect vision such as (1) simplification of the design, (2) spatial segmentation of the field of view, (3) spectral segmentation, and (4) overlap of the FOVs of adjacent optical channels are applied to yield new approaches to miniaturized vision systems with high resolution which overcome the basic scaling limits of single aperture imaging optics. A classification is provided that distinguishes the dif- ferent state-of-the-art and novel approaches with respect to their working principle and application potential. The scaling behavior as well as new and faster methods for the simulation and optimization of MAOs are examined. The discussion is focused on optical systemswhichcanbefabricatedinparallelbyhighly precise microoptics technology based on wafer-level techniques. The analysis includes new methods for the characterization in order to investigate the potential and differences of the various demonstrated approaches. Chapter 2 starts out with an overview of applications of small single aperture digital imaging systems followed by an introduction of the basic properties