Panoramic Monocentric Lens Imaging

UNIVERSITY OF CALIFORNIA SAN DIEGO

Panoramic monocentric lens imaging

A dissertation submitted in partial satisfaction of the requirements for the degree Doctor of Philosophy

Electrical Engineering (Photonics)

Igor Stamenov

Committee in charge: Professor Joseph Ford, Chair Professor Thomas Bewley Professor David Kriegman Professor Truong Nguyen Professor George Papen

2014

Igor Stamenov, 2014

The Dissertation of Igor Stamenov is approved, and it is acceptable in

quality and form for publication on microfilm and electronically:

______

Chair

University of California San Diego

2014

iii Dedication

To my family

iv Table of Contents

Signature Page ...... iii Dedication ...... iv Table of Contents ...... v List of Figures ...... viii List of Tables ...... xvi Acknowledgements ...... xviii Vita ...... xx Abstract of the Dissertation ...... xxi 1. Introduction ...... 1 1.1 Wide-field of view imaging ...... 1 1.2 Monocentric wide-field of view imaging ...... 2 1.3 Introduction to fiber-coupled monocentric lens imaging ...... 6 1.4 Thesis organization ...... 9 2. Optimization of simple monocentric lenses ...... 11 2.1 Two-glass symmetric (2GS) monocentric lens background ...... 11 2.2 Theoretical analysis of monocentric lenses ...... 14 2.2.1 Focus of monocentric lenses ...... 14 2.2.2 Design optimization of monocentric lenses ...... 18 2.3 Specific monocentric lens cases ...... 29 2.3.1 AWARE2 monocentric lens analysis ...... 29 2.3.2 SCENICC F/1.71 12mm focal length lenses ...... 33 2.3.3 Comparison with conventional wide-angle lenses ...... 40 2.4 Chapter summary ...... 42 3. Optimization of high-performance monocentric lenses ...... 45 3.1 Limits of simple monocentric lenses ...... 45 3.2 Options for improving monocentric lenses ...... 48 3.2.1 Review of monocentric lens architectures ...... 48 3.2.2 Review of monocentric lens design methods ...... 52

3.3 Advanced design algorithms and results ...... 54 3.3.1 Improved 2GS global search using 5 wavelengths ...... 54 3.3.2 Three-glass symmetric (3GS) global search ...... 58 3.3.3 Seeded Hammer Optimization ...... 62 3.3.4 Five-dimensional 4GA-8 near global optimization ...... 64 3.4 Lens complexity and performance tradeoff ...... 83 3.5 Specific lens design examples ...... 87 3.5.1 Water-immersed lens (f=12mm, F/1.79, 380-550nm) ...... 87 3.5.2 Night vision lens (f=16mm, F/1.2, 500-900nm) ...... 89 3.5.3 Short-wave IR lens (f=12mm, F/1.19, 900-1500mm) ...... 90 3.5.4 Medium scale lens (f=112mm, F/2.33, 486-656nm) ...... 92 3.5.5 Large scale lens (f=280mm, F/2.8, 450-700nm) ...... 93 3.6 Chapter summary ...... 95 4. Panoramic monocentric imaging using fiber-coupled focal planes ...... 97 4.1 Introduction ...... 97 4.2 Fiber-coupled imager components ...... 97 4.2.1 Monocentric lens design and fabrication ...... 97 4.2.2 Characterization of monocentric lens image formation and focus . 100 4.2.3 Fiber bundle and CMOS image sensor integration ...... 104 4.3 Narrow field (single sensor) fiber-coupled image transfer ...... 107 4.3.1 Fiber-coupled image transfer performance ...... 107 4.3.2 Monocentric fiber-coupled vs. a conventional lens image forming 110 4.4 Wide field (multi-sensor) fiber coupled monocentric imaging ...... 113 4.4.1 Fiber-coupled monocentric lens prototype ...... 113 4.4.2 Fiber-coupled monocentric imager vs. wide angle DSLR camera . 114 4.5 Conclusions ...... 118 5. Broad spectrum monocentric lens ...... 120 5.1 Broad spectrum 4GA-8 monocentric lens prototype ...... 120 5.2 Imaging performance and focus of the 4GA-8 VNIR lens ...... 122 5.3 Broad-spectrum “Letterbox 2” prototype camera ...... 128 5.3.1 Components and assembly ...... 128 5.3.2 Indoors operation (VIS illumination spectrum) ...... 130

vi 5.3.3 Outdoors operation (VIS and VNIR illumination spectrum) ...... 132 6. Conclusions and future directions: Towards Terapixel video imaging ...... 134 6.1 Long term perspective: system level performance metrics ...... 134 6.2 Total resolution scaling in a fully monocentric camera system ...... 137 6.2.1 A panoramic monocentric lens imaging 2.6 Terapixels ...... 141 6.3 Conclusion ...... 143 References ...... 144

vii List of Figures

Figure 1.1: (a) Optical layout of a 2.4 Gigapixel monocentric multiscale lens, and (b) the

same image field transferred by tapered fiber bundles instead of multiple

relay optics...... 3

Figure 1.2: Monocentric lens imaging with (a) a physical aperture stop at the center of

the objective lens, and (b) a "virtual" stop accomplished by limiting the

numerical aperture of the image transfer which, as drawn, are non-imaging

optical fibers...... 5

Figure 1.3: Possible geometries of the panoramic fiber-coupled monocentric imager with

(a) single straight fiber bundle, (b) single curved fiber bundle and (c) multiple

straight fiber bundles ...... 7

Figure 2.1: First and third order consideration of monocentric lens refocus ...... 15

Figure 2.2: Third order aberration theory applied to monocentric lens design...... 19

Figure 2.3: Monocentric lens real ray trace variables...... 22

Figure 2.4: Example of dependence of criterion Q on the radius r1 ...... 25

Figure 2.5: Image formation in the monocentric lens...... 26

Figure 2.6: AWARE2 lens and global optimum solution ...... 30

Figure 2.7: MTF and ray aberrations performance comparison of the (a) fabricated

AWARE2 prototype and (b) new AWARE2 design candidate...... 32

Figure 2.8: Correlation between polychromatic mean square wavefront deformation and

MTF @ 200lp/mm for candidates...... 35

viii Figure 2.9: Highest ranked design solution (high index center glass)...... 36

Figure 2.10: MTF performance of monocentric lens (a) focused at infinity (design) and

(b) refocused at flat object at 0.5m ...... 38

Figure 2.11: The top member of the third family (lower center glass index) operating in

(a) the physical aperture stop, including a 73° field angle to illustrate the

effect of aperture vignetting and (b) ‘‘virtual" aperture stop mode, with

uniform response up to 80° ...... 39

Figure 2.12: Comparison of two conventional wide field lenses with a monocentric-

waveguide lenses. All have a 12 mm focal length, 120° field of view, and

similar light collection, but the monocentric lens provides higher resolution in

a compact volume...... 41

Figure 2.13: Systematic diagram of photographic lens setup families, including

monocentric multiscale and waveguide imagers. Figure adapted from [43]. 43

Figure 3.1: MTF performance curves showing the limits of the globally optimized 2GS

monocentric geometries. The examples are derived from an initial high-

performing lens (a) which is pushed to improve spectral bandwidth (b),

numerical aperture (c) or focal length (d) ...... 47

Figure 3.2: Monocentric lens design space showing glass only (upper half) and glass

with air gap (lower half) regions divided by the seven preferred design

architectures in between ...... 50

Figure 3.3: Optimization of preferred monocentric lens geometries ...... 54

Figure 3.4: New f=70mm AWARE 2 2GS candidates ...... 57

Figure 3.5: Monocentric three-glass symmetric (3GS) architecture ...... 58

ix Figure 3.6: Top 12mm F/1.7 435-850nm 3GS monocentric lens candidate (a)

optimization space, (b) MTF comparison curves with top 2GS candidate and

Figure 3.7: MTF performance comparison of globally optimized 2GS and 3GS

monocentric lenses for extension of the original lens (a) specifications. The

plots show only on-axis MTF, to allow comparison of 2GS and 3GS

architectures...... 61

Figure 3.8: MTF performance curves of the 4GA-8 lens geometries derived from the

original lens specifications through seeded Hammer optimization...... 63

Figure 3.9: Four-glass asymmetric with airgap (4GA-8) monocentric lens architecture 65

Figure 3.10: Gradient descent method applied for normal minimum and degraded

minimum shape ...... 68

Figure 3.11: The optimization criterion ravine of minimums (projection onto 3D space)

...... 70

Figure 3.12: Optimization procedure inside the 4GA-8 optimization space...... 73

Figure 3.13: MTF curves for 12mm, F/1.7, 400-1000nm lenses obtained through seeded

Hammer search and near global 5-dimensional optimization (shown on

layout)...... 78

Figure 3.14: Spectral response of front and back-illuminated silicon sensor ...... 78

Figure 3.15: MTF curves for 12mm, F/1.7 lens operating with 400-1000 nm front-

illuminated silicon sensor sensitivity spectrum ...... 80

Figure 3.16: MTF curves for 12mm, F/1.7 lens operating with 435-1000nm back-

illuminated silicon sensor sensitivity spectrum ...... 82

x Figure 3.17: Longitudinal aberrations of the (a) top 3GS and (b) top 4GA-8 architecture

12mm F/1.7 monocentric lenses for 400-1000nm spectral band...... 83

Figure 3.18: Monocentric lens geometries optimization behavior for two different scales

operating in 486-656nm spectral range ...... 84

Figure 3.19: Monocentric objective lens performance tradeoff for different scales and

three spectral bands (486-656nm, 435-850nm, 400-1000nm) ...... 86

Figure 3.20: Underwater monocentric lens (9mm/12mm image/object space focal length,

F/1.79, 380-550nm) ...... 88

Figure 3.21: Gen III Night Vision monocentric lens (16mm focal length, F/1.2, 500-

900nm)...... 90

Figure 3.22: SWIR monocentric lens (12mm focal length, F/1.19, 900-1500nm) ...... 91

Figure 3.23: Medium scale monocentric lens 5GA-10 diffraction limited candidate

possible to fabricate (112mm focal length, F/2.33, 486-656nm) ...... 93

Figure 3.24: Polychromatic MTF comparison of previously reported 5GA-10 optimal

solution for the Gigagon 40GPixel lens and a simpler 4GA-8 solution

(280mm focal length, F/2.8, 450-700nm) ...... 94

Figure 4.1: Two-glass symmetric (2GS) f=12mm 470-650nm F/1.7 monocentric lens

design layout (top row) with MTF performance curves, and a photograph of

the fabricated F/1.35 prototype. Increasing aperture (bottom row) improves

light collection at the expense of a spot size and MTF ...... 101

Figure 4.2: Experimental confirmation of monocentric lens focusing capability. Axial

translation of the center ball lens maintains a fixed radius hemispherical

image of a flat object in focus (0.5m and 1.0m conjugates shown)...... 103

xi Figure 4.3: Commercial focal plane integration with the fiber bundle. (a) cross section of

OV5653 wafer-fabricated CMOS sensor, (b) sensor with cover glass

removed; (c, and d) shaped glass fiber bundle output and input faces, ready to

be attached to CMOS sensor and meniscus lens...... 105

Figure 4.4: Assembled fiber-coupled OV5653 sensor, which was then cross-sectioned by

diamond saw to inspect adhesive thickness and uniformity. Optical

micrograph (center) and SEM measurement (right) of the fiber bundle/sensor

interface reveals the optical adhesive layer was less than 2µm thick...... 106

Figure 4.5: White light impulse response of the two-glass monocentric (a) lens only,

measured with a Keyence VHX 1000 microscope and (b) the lens with the

fiber-coupled Omnivision OV5653 1.75µm sensor attached to it...... 107

Figure 4.6: Fiber-coupled image transfer effect on visible spectrum (LED illumination)

MTF lens performance of the fabricated 2GS monocentric imager prototype.

Top row: performance of glass monocentric objective lens only. Center row:

objective with the fiber-bundle in oil contact ...... 109

Figure 4.7: Side-by-side lens -60° off-axis performance comparison: conventional wide-

angle vs fiber-coupled monocentric lens. In both cases partial field of view

was sensed by the Omnivision OV5653 1.75µm 5Mpixel CMOS sensor.

Canon lens suffers from distortion and strong chromatic aberration...... 112

Figure 4.8: Single row 30Mpixel fiber-coupled monocentric imager CAD models of (a)

the exploded imager system showing 2GS monocentric lens, electronic

focusing mechanism and fiber coupled Omnivision sensors forming the

seamless array for 126°x16° FOV curved image sensing ...... 114

xii Figure 4.9: Comparison of the conventional DSLR Canon camera with wide-angle lens

to fiber-coupled monocentric camera. On the right: photo of the laboratory

characterization system...... 115

Figure 4.10: Raw unprocessed image data acquired with the monocentric fiber-coupled

prototype. Vignetting, field gaps, fiber bundle imperfections and moiré

patterns are visible in the image...... 116

Figure 4.11: Indoors comparison photos of the performance between Canon EOS 5D

Mark II DSLR camera with Canon 8-15mm F/4 fisheye lens and the 2GS

fiber-coupled F/1.0 monocentric prototype. Canon photo is cropped to match

the prototype’s letterbox 8:1 field of view...... 118

Figure 5.1: MTF performance and the 4GA-8 12mm F1.7 broad-spectrum monocentric

lens design with a photo of the fabricated prototype...... 121

Figure 5.2: 2GS and 4GA8 monocentric lens on-axis performance comparison in VIS

(470-650nm) and VNIR (435-1000nm) spectrum range...... 123

Figure 5.3: Broad spectrum 4GA-8 lens focusing problem: accumulated tolerancing

errors result in image formation in air, 57µm outside the mounting meniscus.

Custom, reoptimized meniscus was needed to correct the error...... 124

Figure 5.4: Focusing and performance of the 4GA-8 lens with matched meniscus lens

(1m object distance, visible LED illumination) ...... 125

Figure 5.5: Focusing and performance of the 4GA-8 lens with matched meniscus lens

(1m object distance, broad spectrum illumination) ...... 126

Figure 5.6: Cut fiber bundles before and after integration with OV5653 sensors.

Configurations on the right show intended sequence and relative positioning

xiii of individual bundles after their placing in contact with the mounting

meniscus...... 129

Figure 5.7: UV-curing process (left) of fiber coupled sensors to the lens mounting

meniscus, fully assembled “Letterbox 2” 4GA-8 monocentric fiber-coupled

prototype (center) and whole system enclosed in the box (right) ready for

outdoor testing...... 130

Figure 5.8: Indoor panorama image acquired by 4GA-8 “Letterbox 2” monocentric fiber-

coupled imager prototype. Visible LED light (VIS spectrum) illumination

used. (Top) raw unprocessed image, (middle) calibrated white-balanced

image with no interpolation ...... 131

Figure 5.9: Panoramic images acquired outdoors with 4GA-8 fiber-coupled “Letterbox

2” prototype with and without IR cutoff filter. High resolution imaging

performance is maintained from VIS illumination range through VNIR. ... 132

Figure 6.1: Map of modern camera systems in F-number vs sensor pitch space. Cameras

marked in black, with dedicated optics, can achieve >120° field of view

imaging...... 135

Figure 6.2: Chart of modern camera systems showing total system resolution with sensor

pixel pitch and optics F-number tradeoffs...... 136

Figure 6.3: Total resolution scaling with system size for fully spherical 120°x120°

monocentric imager ...... 140

Figure 6.4: The layout and performance curves of a 30mm F/1.7 monocentric lens

capable of imaging 2.3Gpixel (with optimum 1.1µm pixel pitch sensor array)

...... 141

xiv Figure 6.5: The layout and performance curves of a gigantic 3000mm F/5.6 monocentric

lens capable of imaging 2.6TPixel (with optimum 3.3µm pixel pitch sensor

array) ...... 142

xv List of Tables

Table 2.1: Optical prescription of the fabricated AWARE2 lens...... 29

Table 2.2: Optical prescription of the top design solution for the AWARE2 lens...... 31

Table 2.3: Top solutions for a F#/1.7 f=12 mm monocentric lens...... 34

Table 2.4: Optical prescription for refocusing of the monocentric lens, showing multiple

configurations for three object distances ...... 36

Table 3.1: Updated list of top solutions for the SCENICC F/1.7 f=12mm 470-650nm

120° MC lens (prescriptions shown pertain to glass combinations marked in

bold)...... 56

Table 3.2: The array of local minimums over the main ravine in 4GA-8 optimization

space. Radii shown pertain to the areas at the bottom of the ravine reached

after optimization ...... 72

Table 3.3: Optical prescription of the 400-1000nm F/1.7 f=12mm MC lens example

solution A=5.63g ...... 75

Table 3.4: Optical prescription of the 400-1000nm F/1.7 f=12mm MC lens example

solution B=5.71g ...... 75

Table 3.5: Optical prescription of the 400-1000nm F/1.7 f=12mm MC lens example

solution C=5.72g ...... 76

Table 3.6: Families of solutions for F/1.7 12mm monocentric 4GA-8 400-1000nm lens

obtained through near-global search...... 77

xvi Table 3.7: Optical prescription of the 400-1000nm F/1.7 f=12mm monocentric lens near

global solution ...... 77

Table 3.8: Optical prescription of the 400-1000nm F/1.7 f=12mm monocentric lens

operating with front-illuminated silicon sensor ...... 79

Table 3.9: Optical prescription of the 435-1000nm F/1.7 f=12mm monocentric lens

operating with back-illuminated silicon sensor ...... 81

Table 3.10: Optical prescription of the water immersed monocentric lens...... 88

Table 3.11: Optical prescription of the Night Vision monocentric lens...... 89

Table 3.12: Optical prescription of the short-wave infrared (SWIR) monocentric lens. . 91

Table 3.13: Optical prescription of the f=112mm monocentric lens candidate...... 92

Table 3.14: Optical prescription of the optimal 40GPix monocentric 4GA-8 lens ...... 94

Table 4.1: Optical prescription of the 470-650nm F/1.7 f=12mm 120° 2GS MC lens

prototype ...... 99

Table 5.1: Finalized optical prescription of the 435-1000nm F/1.7 f=12mm second

generation broad-spectrum SCENICC monocentric lens (4GA-8 architecture)

...... 121

Table 6.1: Maximum resolution 120°x120° 2GS monocentric systems specs...... 139

Table 6.2: Optical prescription of 120°x120° f=30mm F/1.7 486-656nm 2.3GPixel lens.

...... 140

Table 6.3: Optical prescription of 120°x120° f=3000mm F/5.6 486-656nm Teragon lens.

...... 142

xvii Acknowledgements

I would like to thank all the people who contributed in some way to the work described in this thesis. First and foremost, I thank Professor Joseph Ford for believing in me, accepting me in his group and his support during last four years. During my studies, he contributed to a rewarding graduate school experience by giving me intellectual freedom in my work, supporting my attendance at various conferences, engaging me in new ideas, teaching me how to create, discuss, present and lead. His guidance has proved to be invaluable.

Almost every result described in this thesis was accomplished with the help and support of colleagues and collaborators. Specifically, I would like to thank Dr. Ilya

Agurok for introducing me to the art of old-school lens design and help in developing the theory of monocentric lens design. The help of fellow labmates was invaluable in setting and performing the experiments. I would like to thank members of the Distant Focus

Corporation for excellent work on design and fabrication of the components for monocentric camera prototypes.

Finally, I would like to acknowledge family and friends who supported and believed in me during my work on the thesis. First and foremost I would like to thank

Mom, Dad and my sister Ivana for their constant love and support. Slaven, Andreja, Eric,

Justin, Brett, Kate, Stephen, Ash, Glenn, Nojan, Max and Salman made my time here at

UCSD a lot more creative and fun.

xviii Right before embarking on this trip, I was lucky to meet Aleksandra, who joined me to share the same dream, and I thank her for her friendship, love, and unyielding support.

Chapter 2, in full, is a reprint of the material as it appears in “Optimization of two-glass monocentric lenses for compact panoramic imagers: general aberration analysis and specific designs,” by I. Stamenov, I. Agurok and J. Ford, Applied Optics, Vol. 51,

No. 31, 2012. The dissertation author was the primary investigator and author of this paper.

Chapter 3, in full, is a reprint of the material as it appears in “Optimization of high-performance monocentric lenses,” by I. Stamenov, I. Agurok and J. Ford, Applied

Optics, Vol. 52, No. 34, 2013. The dissertation author was the primary investigator and author of this paper.

Chapter 4, in full, is a reprint of the material as it appears in “Panoramic monocentric imaging using fiber-coupled focal planes,” by I. Stamenov, A. Arianpour, S.

J. Olivas, I. Agurok, A. R. Johnson, R. A. Stack, R. L. Morrison and J. Ford, Optics

Express, Vol. 22, No. 26, 2014. The dissertation author was the primary investigator and author of this paper.

xix Vita

2007 B.S./M.S. in Electrical Engineering, School of Electrical Engineering, University of Belgrade.

2014 Ph.D. in Electrical Engineering (Photonics), University of California – San Diego, Department of Electrical and Computer Engineering.

xx Abstract of the Dissertation

ABSTRACT OF THE DISSERTATION

Panoramic monocentric lens imaging

Igor Stamenov

Doctor of Philosophy in Electrical Engineering (Photonics)

University of California, San Diego, 2014

Professor Joseph Ford, Chair

In this dissertation, a framework for panoramic monocentric lens imaging theory is established and specific lens design optimization algorithms are developed. The global and near-global lens optimization procedures are demonstrated on a number of cases.

Furthermore, two wide-angle fiber-coupled monocentric imagers, targeting visible and visible near infrared (VNIR) imaging, were built, tested and compared to an existing state-of-the-art conventional camera system. Lastly, general wide-angle monocentric system performance scaling is discussed, with future directions in development of this class of imagers.

xxi 1. Introduction

1.1 Wide-field of view imaging

Imagers that require the combination of wide field of view, high angular resolution and large light collection present a difficult challenge in optical system design.

Geometric lens aberrations increase with aperture diameter, numerical aperture and field of view, and scale linearly with focal length. This means that for a sufficiently short focal length, it is possible to find near diffraction-limited wide angle lens designs, including lenses mass-produced for cellphone imagers. However, obtaining high angular resolution

(for a fixed sensor pixel pitch) requires a long focal length for magnification, as well as a large numerical aperture to maintain resolution and image brightness. This combination is difficult to provide over a wide-angle range. Conventional lens designs for longer focal length wide-angle lenses represent a tradeoff between competing factors of light

collection, volume, and angular resolution. For example, conventional reverse-telephoto 1

and "fisheye" lenses provide extremely limited light collection compared to their large clear aperture and overall volume [1]. However, the problem goes beyond the lens itself.

Solving this lens design only leads to a secondary design constraint, in that the total resolution of such wide-angle lenses can easily exceed 100 Megapixels. This is beyond the current spatial resolution and communications bandwidth of a single cost-effective sensor [2], especially for video output at 30 frames per second or more.

1 2

1.2 Monocentric wide-field of view imaging

One early solution to wide-angle imaging was "monocentric" lenses [3,4], using only hemispherical or spherical optical surfaces, which share a single center of curvature.

This symmetry yields zero coma or astigmatism over a hemispherical image surface, and on that surface provides a field of view limited only by vignetting from the central aperture stop. The challenge of using a curved image surface limited the practical application of this type of lens, but there has been a resurgence of interest in monocentric lens imaging. In 2009, Krishnan and Nayar proposed an omnidirectional imager using a spherical ball lens contained within a spherical detector shell [5]. In 2010 Ford and

Tremblay [6] proposed using a monocentric lens as the objective in a multiscale imager system [7], where overlapping regions of the spherical image surface are relayed onto conventional image sensors, and where the mosaic of sub-images can be digitally processed to form a single aggregate image. Cossiart and Nayar demonstrated a closely related configuration using a glass ball and single element relay lenses, recording and digitally combining overlapping images from 5 adjacent image sensors [8]. And recently a Gigapixel monocentric multiscale imager has been demonstrated which integrates a 2- dimensional mosaic of sub-images [9], [10], using the optical layout shown in Figure 1.1

(a) [11].

Monocentric lenses and spherical image formation provides favorable scaling to long focal lengths, and have been shown capable of two orders of magnitude higher space-bandwidth product (number of resolvable spots) than conventional flat field systems of the same physical volume [12].

(a) (b)

-60° -60°

-30° -30°

-5° -5°

5° 5°

30° 30°

60° 60° focal objective objective planes image surface focal fiber single planes bundles microcamera

Figure 1.1: (a) Optical layout of a 2.4 Gigapixel monocentric multiscale lens, and (b) the same image field transferred by tapered fiber bundles instead of multiple relay optics.

In early monocentric lens cameras, the usable field of view was limited by the vignetting and diffraction from the central lens aperture, as well as the ability of recording media to conform to a spherical image surface. However, the system aperture stop need not be located in the monocentric lens. The detailed design of the multiscale monocentric lens [11] shows that locating the aperture stop within the secondary (relay) imagers enables uniform relative illumination and resolution over the full field. This design maintains F/2.4 light collection with near-diffraction limited resolution over a

120° field of view. With 1.4µm pitch sensors, this yields an aggregate resolution of 2.4

Gigapixels.

Such wide and uniform fields can also be achieved via the conceptually simpler

"waveguide" approach shown in Figure 1.1(b). Instead of relay optics, the spherical image surface is transferred to conventional planar image sensors using one or more multimode fiber bundles [13]. Fused fiber faceplates are commercially available with high spatial resolutions and light collection (2.5 microns between fiber cores, and numerical aperture of 1), and can be fabricated as straight or tapered [14]. Straight fiber bundles can project sufficiently far to allow space for packaging of CMOS image sensors, while tapered fiber bundles can provide the 3:1 demagnification used in the relay optics in Figure 1.1(a). Fiber bundles introduce artifacts from multiple sampling of the image

[15], which can be mitigated but not eliminated through post-detection image processing

[16]. In addition, the edges between adjacent fiber bundles can introduce "seams" in the collected image, whose width depends on the accuracy of fiber bundle fabrication.

However, waveguide transfer can reduce overall physical footprint and significantly increase light collection: In the multiscale optics structure, field overlap at the center of three relay optics must be divided between three apertures, while the waveguide can transfer all light energy from a given field angle to a single sensor.

As shown in Figure 1.2, in both systems stray light can be controlled using a physical aperture stop at the center of the monocentric lens (Figure 1.2(a)), or through a

"virtual stop" achieved by limiting light transmission in the image transfer optics (Figure

1.2(b)). In the case of relay imaging, this is done using a physical aperture stop internal to the relay optics. In the case of fiber transfer, this can be done by restricting the numerical aperture (NA) of the fiber bundles. Straight fiber bundles with a lower index difference between core and cladding glasses are commercially available with a numerical aperture

5 of 0.28. Alternately, high index contrast bundles with a spatial taper to a smaller output face can provide a controlled NA. Such bundles have the original output NA, but conservation of étendue reduces the input NA of a tapered fiber bundle by approximately the ratio of input to output diameter [17]. For example, a 3:1 taper of input to output width with a 1.84 core and 1.48 cladding index, yields approximately 0.3 input NA.

physical aperture stop (a) (b)

0° 0°

30° 30°

60° 60° fiber 80° 80° bundles

Figure 1.2: Monocentric lens imaging with (a) a physical aperture stop at the center of the objective lens, and (b) a "virtual" stop accomplished by limiting the numerical aperture of the image transfer which, as drawn, are non-imaging optical fibers.

The field of view of monocentric "virtual stop" imagers can be extraordinarily wide. A physical aperture at the center of the objective lens is projected onto the field angle. At 60° incidence (120° field of view), the aperture is elliptical and reduced in width by 50%, with a corresponding decrease in light transmission and diffraction-limited resolution. As Figure 1.2(b) shows, however, moving the aperture stop to the light transfer enables uniform illumination and resolution over a 160° field of view, where at extreme angles the input light illuminate the back surface of the monocentric objective.

The lens as drawn in Figure 1.2(b) indicates that the image transfer optics perform all stray light control, and does not show non-sequential paths from surface reflections. In practice, an oversized physical aperture or other light baffles can be used to block most of the stray light, while the image transfer optics provide the final, angle-independent apodization. While such practical optomechanical packaging constraints can limit the practical field of view, the potential for performance improvement over a conventional

"fisheye" lens is clear.

Despite the structural constraints, even a simple two-glass monocentric objective lens can provide high angular resolution over the spherical image surface. With waveguide image transfer, overall system resolution is directly limited by objective lens resolution. In multiscale imagers, geometric aberrations in the objective can be corrected by the fabricating objective-specific relay optics. However, compensating for large aberrations in the primary lens tends to increase the complexity and the precision of fabrication of the relay optics. Since each multiscale imager requires many sets of relay optics (221 sets in the 120° field 2.4 Gigapixel imager design), minimizing relay optic complexity and fabrication tolerance can significantly reduce system cost. So for both structures, it is useful to optimize the objective lens resolution.

1.3 Introduction to fiber-coupled monocentric lens imaging

As previously mentioned, monocentric lenses consist entirely of hemispherical optical surfaces that share a common center of curvature. Because the single point of symmetry eliminates most of the aberrations, monocentric lenses are capable of generating a high-resolution wide-angle image on a spherical image surface. Such

7 monocentric lenses were historically one of the first lenses used for panoramic imaging

[4], and nearly a century later this approach was revived for high-resolution aerial imaging [18].

low$ low$ loss$ blur$ blur$ good$ 60°# 60°# 60°# blur$ 30°# 30°# 30°#

0°# 0°# 0°# good$ good$ good$

Figure 1.3: Possible geometries of the panoramic fiber-coupled monocentric imager with (a) single straight fiber bundle, (b) single curved fiber bundle and (c) multiple straight fiber bundles

However, both of these imagers were limited in utility due to their use of curved film negatives, which are difficult to fabricate and process. As mentioned, another approach for sensing the monocentric image surface is to transfer it over a dense array of optical fibers [14], which can be polished with a curved input face and a planar output face, so that the image can be sensed by focal planes fabricated with conventional wafer processing. This approach was successfully used by Lawrence Livermore to make a

F/1.7, 17.5mm focal length monocentric imager using a fiber-coupled CCD focal plane to sense a 60˚ field of view [19,20]. A larger imager used a F/2.8, 250mm focal length monocentric objective with an array of 23 intensified CCD sensors [21], where each

384x576 sensor captured a 7.5x11.5˚ segment to cover 75% of the overall 60˚ field of view. Both of these imagers used fiber bundles, which highly oversampled their 23µm pixel monochrome CCD sensors, eliminating the potential for Moiré sampling effects.

However, none of the previous imagers have made use of the primary advantage of monocentric lenses: the combination of high spatial resolution with a wider field of view than conventional focal plane objectives. The goal of the current work (Chapter 4) is to demonstrate that a panoramic fiber-coupled monocentric lens imager can be implemented with current CMOS focal planes, where the pixel pitch is typically less than the minimum

2.5µm pitch supported by commercial fiber bundle suppliers, and compare the resulting imager resolution and physical volume to a conventional wide-field imager.

A single straight fiber bundle can couple the spherical image to a single focal plane, as shown in Figure 1.3(a) and as implemented in reference [20]. However, Fresnel refraction at the angled fiber input face limits input fiber coupling and strongly affects the light emission from the fiber output aperture, and so limits the achievable field of view.

Even for fiber bundles with N.A. = 1, the maximum full field of view with 50% of peak efficiency is approximately 55˚ [22]. Uniformly efficient coupling across a wide field of view can be accomplished using a 3-dimensional waveguide as shown in Figure 1.3(b), where multi-mode fibers are curved to point each input aperture towards the center of the lens and adiabatically couple the propagating modes to the planar output face. This structure is the topic of on-going research in fiber modeling and system fabrication.

However, a straightforward approach to wide-field imaging is to use multiple straight fiber bundles to divide the field (Figure 1.3(c)) between multiple sensors for image acquisition. If the bundles are segmented so that each bundle covers a ±15˚ field, the signal coupling is uniform to within approximately 97% and the overall imager field of view is limited only by the conventional cosine projection losses from the objective lens internal aperture stop. The remaining challenges are physical integration of the optical

9 fiber bundles to minimize information loss at the seams and in the interface to the focal plane image sensors.

1.4 Thesis organization

The dissertation is organized as follows: simple two-glass symmetric (2GS) monocentric architecture is analyzed in Chapter 2. In particular, rigorous Siedel 3rd order aberration analysis is performed with a proof of monocentric lens focusing capability.

Exact raytracing equations are derived and global optimization algorithms based on them are described. At the end of the chapter, global optimization is demonstrated on specific monocentric lens designs for visible spectrum imaging.

Limits of simple 2GS monocentric geometry are discussed in Chapter 3 and options for high-performance monocentric imaging are explored. In Chapter 3 we categorize the monocentric lens design space of moderate complexity, provide procedures for optimum and near-optimum lens design with complexity and performance tradeoff considerations. Preferred monocentric architectures are identified and advanced systematic optimization procedures are demonstrated for selected wide-angle imaging applications: underwater, night vision, short wave infrared (SWIR), and large scale (10-

40GPixel) monocentric imaging.

Chapter 4 shows experimental results with first generation 2GS visible monocentric lens including focusing and high-resolution image forming. Fiber-coupled high-resolution image transfer to flat focal planes is demonstrated and the performance of the F/1.0 120° field of view 30MPixel prototype is compared to a conventional wide angle DSLR camera system.

Chapter 5 shows an ongoing work with F1.7 broad-spectrum monocentric lens and its operation in second-generation fiber-coupled monocentric imager prototype.

Panoramic imaging is demonstrated, both indoor and outdoor in visible and VNIR spectral band.

Chapter 6 is primarily concerned with performance scaling of the monocentric lens systems and it concludes with framework for a continuing study in the field of panoramic monocentric imaging.

2. Optimization of simple monocentric lenses

2.1 Two-glass symmetric (2GS) monocentric lens background

T. Sutton offered the first 2GS water-filled design of the monocentric lens in 1859

[3,4], with a wide field of view but high F-number (about 30) due to the lack of correction of spherical and chromatic aberrations. To solve this problem, in 1942, J. G.

Baker proposed a glass combination with a high index flint glass for the outer shell and low index crown for internal ball lens [3,18], resulting in a monocentric but not front-to- back symmetric lens structure with aberrations well corrected for a moderately high F# of

3.5. The design of a compact and high resolution endoscope lens with a two-glass symmetrical monocentric lens was published by Waidelich in 1965 [13]. An endoscopic lens with a 2 mm focal length and F# of 1.7 has a diffraction limited image quality, but 11 scaling in focal length to > 1 cm for photographic applications with the focus about 12 mm required the increase of F# to about 3.5. More recently, a two-glass monocentric lens was used as the projection objective in a Kodak autostereoscopic display [23, 24]. In the

Waidelich and Kodak designs, like in Baker lens, the outer shell meniscus was made from a flint glass and the internal ball lens from a crown glass with the moderate difference in Abbe number. As will be shown later, this was nearly the ideal combination.

11 12

The outer meniscuses had a higher nd and the difference in refraction index between outer and internal ball lenses did not exceed 0.08. This low index difference probably was inherited from the Cooke triplet design [1]. We will show that differences in refraction indices larger than 0.2 are needed to approach diffraction-limited performance in lower

F# photographic lenses. Nevertheless these designs have demonstrated that purely monocentric lenses can achieve high quality achromatic imaging.

Optical systems are now typically designed by computer numeric optimization in commercial software like Zemax and CodeV. The highly constrained monocentric lenses seem well suited to a global optimization search to identify the best glass combination and radii. However, we have found that "blind" optimization of monocentric lenses, even a simple two-glass lens, often overlooks the best solutions. This is especially true for large NA designs, where the ray angles are steep and the optimization space has multiple deep local minima. There are also a large number of glasses to consider. The available glass catalog was recently increased by the publication of a number of new glasses from

Hoya to 559 glasses [25]. The more advanced optimization algorithms take significant processing time. Even for a 2-glass lens it is impractical to use them to search all 312,000 potential combinations, so the best design may be overlooked. Fortunately, the symmetry of monocentric lenses permits a relatively straightforward mathematical analysis of geometrical optic aberrations, as well as providing some degree of intuitive understanding of this overall design space. Combining "old school" analysis with computer sorting of glass candidates can enable a global optimization for any specific focal length and spectral bandwidth desired.

In this chapter, we provide a detailed analysis for the design of two-glass monocentric lenses. We begin with the first order paraxial and Seidel third order analysis of the focus of wide-field monocentric imagers, showing that despite the highly curved focal surface, axial translation of monocentric lenses can maintain focus of a planar object field from infinite to close conjugates. We will optimize these lenses operating with a more larger and so more general “virtual” stop, because introducing a comparably- sized physical stop will only tend to create vignetting at the field points and cut off some of the most highly aberrated rays. We continue by demonstrating the systematic optimization of these lenses by the following process.

For a specified focal length, numerical aperture and wavelength range:

(1) Compute and minimize 3rd order Seidel spherical and longitudinal

chromatism aberrations to find approximate surface radii for valid

glass combination.

(2) Optimize lens prescriptions via exact ray tracing of multiple ray

heights for the central wavelength,

(3) Calculate the polychromatic mean square wavefront deformation, and

generate ranked list of all lens candidates,

(4) Confirm ranking order by comparing polychromatic diffraction MTF

curves.

To verify this method, we redesign the objective from our 2.4 Gigapixel multiscale imager, and find that the global optimization process yields the original design

(and fabricated) lens, as well as additional candidates with improved internal image surface resolution. We then apply the design methodology to a new system, an ultra-

14 compact fiber-coupled imager with a 12 mm focal length and uniform resolution and light collection over more than 120° field of view. We show that this design compares favorably to a more conventional imager using a "fisheye" wide field lens, and conclude with observations on future directions and challenges for this type of imager.

2.2 Theoretical analysis of monocentric lenses

2.2.1 Focus of monocentric lenses

Photographic lenses are normally focused by moving them closer or further from the image plane, but this appears impractical for the deep spherical image surface in a wide-field monocentric lens. For the 70 mm focal length objective of Figure 1.1, a 1mm axial translation to focus on an object at 5 m range brings the image surface only 0.5 mm closer to the objective for an object at a 60° field angle, and 0.17mm closer for an object at an 80° field. This seems to imply that the lens can only focus in one direction at time, and needs 3 dimensional translation to do so. In the monocentric multiscale imager, the primary lens position is fixed, and the secondary imagers are used for independent focus on each region of the scene [11], [26]. However, introducing optomechanical focus mechanism for each secondary imager constrains the lens design, and adds significantly to the overall system bulk and cost. More fundamentally, the monocentric-waveguide imager shown in Figure 1.2(b) has no secondary imagers, and cannot be focused in this way, which initially appears a major disadvantage. In fact, however, axial translation of monocentric lenses maintains focus on a planar object across the full field of view.

Consider the geometry of an image formation in the monocentric lens structure shown in Figure 2.1.

For a focal length f with refocusing from object at infinity to the closer on-axis object at distance d, assuming d >> f , the image surface shift Δx is [1]

f 2 f 2 Δx = ≈ (2.1) d − f d

Δx

1 2 3 4

B’ n =1 n =1 1 5 Δx Q Binf α2

h1 n3 β4 A α1 β3 α5 Δx A’ Δx d β2 O A’ β1 inf

h0 n2=n4

B R=f R=f aperture stop ≡ entrance pupil ≡ exit pupil

Δx

5(∞) 5(d)

Figure 2.1: First and third order consideration of monocentric lens refocus

For the off-axis field point B, having a field angle and the angle of principal ray

β1 , the distance OB to the object is d / cos(β1 ) and the refocusing image point shift Δx'

( Binf B' ) will be

2 cos(β1 ) Δx'(β1 ) Δx'(β1 ) = f = Δx ⋅cos(β1 ) → BinfQ = = Δx = Ainf 'A' (2.2) d cos(β1 )

which means that for refocusing, the spherical image surface 5(∞) is axially translated on segment Δx to the position 5(d).

As will be shown later, for our 12 mm focus monocentric lens this approximation works well for up to the closest distance of 500 mm, and reasonably well for objects at a

100 mm range. So for a planar object at any distance above some moderate minimum, the geometry of refocusing the monocentric lens is in accordance with first order paraxial optics.

The most general analytic tool for lens aberration analysis and correction is classical 3rd order Seidel theory [27,28]. In Seidel theory astigmatism and image curvature are bound with the coefficients C and D. Referring to the variables defined in

Figure 2.1, the coefficients C and D are shown in Eq. (2.3) and Eq. (2.4) can be expressed as [27,28,29]:

2 m ⎛ ⎞ 1 ⎜ β − β ⎟ ⎛ α α ⎞ C = h s+1 s s+1 − s (2.3) s ⎜ 1 1 ⎟ ⎜ ⎟ 2 ∑ ⎝ ns+1 ns ⎠ s=1 ⎜ − ⎟ ⎝ ns+1 ns ⎠

⎛ 1 1 ⎞ m − 1 ⎝⎜ n n ⎠⎟ D = s s+1 + C (2.4) 2 ∑ rs s=1

th where ri is radius of i surface and ni is the preceding index of refraction. From

Figure 2.1 it is clear that chief ray angles to optical axis at each surface are identical

(β1=β2=β3=β4) for any axial position of the monocentric optics relative to image surface.

Therefore, coefficient C remains zero while focusing to planar object surfaces by means of axial movement of the monocentric objective lens. According to [27],

1 ⎛ 1 1 ⎞ 1 1 C = ⎜ − ⎟ and D = (2.5) 4nim ⎝ Rt Rs ⎠ 2nim Rs

where Rt is the tangential image surface radius, Rs is the sagittal image surface radius, and nim is the image space refraction index. So if C is defined as zero during refocusing then Rt will be equal to Rs and the image surface will stay spherical and maintain the image radius Rim = Rt = Rs. Also, from Eq. (2.4) it is clear that the coefficient

D will remain constant. Because D remains constant, Eq. (2.5) also show that Rs and hence radius Rim will stay unchanged as well. Third order aberration theory couples the image curvature with astigmatism, while coma and spherical aberrations need to be corrected at this surface [27, 28, and 30]. In other words, third order Seidel aberration theory also indicates that simple axial refocusing of monocentric lens structures preserves the image surface radius, maintaining focus for a planar object onto a spherical image surface over a wide range of object distances. In a purely monocentric geometry there is zero coma, and no additional coma will be introduced by translation [1, 28, and 30].

Suppose that spherical aberration is corrected at infinity. Third order spherical aberration does not have the term, which depends on the object distance [27, 28, and 31]. We expect only a minor change in spherical aberration during refocusing due to the small changes in

2 terms (αi+1–αi) [27, 28], which constitutes the Seidel spherical aberration coefficient.

This is confirmed by the ZEMAX simulations shown in Chapter 2.3 for an optimal f = 12 mm lens solution.

2.2.2 Design optimization of monocentric lenses

Imaging optics are conventionally designed in two steps [1, 27, 28, and 30]. First, monochromatic design at the center wavelength achieves a sufficient level of aberrations correction. The second step is to correct chromatic aberration, usually by splitting some key components to achieve achromatic power: a single glass material is replaced with two glasses of the same refraction index at the center wavelength, but different dispersions. However, this process cannot easily be applied to the symmetric 2-glass monocentric lens shown in Figure 1.2. For a given glass pair and focal length, lens has only four prescription parameters to be optimized (two glasses and two radii), and the radii are constrained with the monocentric symmetry and desired focal length. From this perspective monocentric lenses resemble conventional spherical doublets. An approach for global optimization of such lenses was proposed in [32], and our approach for a global search for optimal monocentric lenses, considering all glass combinations, is closely related. As in [32] we want to analyze all valid glass combinations. However, unlike aplanatic doublets [33] there is not an optimal analytical solution for monocentric lenses, so a modified process is needed. In addition, analysis in [32] is restricted to fifth order aberration estimates [34], while we want to extend the design process to include full analytic raytracing and MTF calculation.

We define a systematic search process beginning with a specified focal length,

F/number, and wavelength range. Optimization of each glass combination was done in three steps. The first step is to determine the solution (if it exists) to minimize third order

Seidel geometrical and chromatic aberrations [27, 28, and 29]. A monocentric lens operating in the “fiber” stop mode has only two primary third order aberrations –

spherical aberration (Seidel wave coefficient W040) and longitudinal chromatism W020, which is defocus between blue and red paraxial foci. The sum of the absolute values of the third order coefficients provide a good approach for a first-pass merit (cost) function, and an analytical solution for third order coefficients allows this cost function to be quickly calculated. The monocentric lens shown in Figure 2.2 is defined with 6 variables, the two radii r1 and r2, and the index and Abbe number for each of two glasses: the outer glass n2 and v2, and the inner glass n3 and v3. Raytracing of any collimated ray can use the

Abbe invariant:

ni+1 − ni ri = hi (2.6) ni+1α i+1 − niα i

ni ni+1 − ni α i+1 = α i + hi (2.7) ni+1 ni+iri

where αi are the angles between marginal rays and the optical axis. The raytracing proceeds surface by surface and at each step for the input ray angle αi and ray height hi the output angle αi+1 can be calculated by Eq. (2.7).

1 2 3 4 n2

α1=0 α2 A α3 α4 h α5 1 h 2 n3 h3 h4 0 r2 r1

Figure 2.2: Third order aberration theory applied to monocentric lens design.

The ray height at the next surface is

hi+1 = hi −α i+1di (2.8)

where di is the thickness between surfaces i+1 and i. For the monocentric lens

d1 = r1 − r2 , d2 = 2r2 and d3 = r1 − r2 . The ray trace of the marginal input ray having α1=0 gives

1 2 ⎛ 1 ⎞ 2 ⎛ 1 1 ⎞ = ⎜1− ⎟ + ⎜ − ⎟ (2.9) f r1 ⎝ n2 ⎠ r2 ⎝ n2 n3 ⎠

For a given glass combination and focal length f Eq. (2.9) constrains the radius r2 to the radius r1 for all subsequent calculations.

The Seidel spherical aberration coefficient B, according to [27, 28, 29], is

2 4 ⎛ ⎞ 1 ⎜ α −α ⎟ ⎛ α α ⎞ B = h s+1 s s+1 − s (2.10) s ⎜ 1 1 ⎟ ⎜ ⎟ 2 ∑ ⎝ ns+1 ns ⎠ s=1 ⎜ − ⎟ ⎝ ns+1 ns ⎠

Equivalently, the spherical wave aberration W040 for the marginal ray is:

2 4 ⎛ ⎞ 1 ρ=1 1 ⎜ α −α ⎟ ⎛ α α ⎞ W = Bρ 4 = h s+1 s s+1 − s (2.11) 040 s ⎜ 1 1 ⎟ ⎜ ⎟ 4 8 ∑ ⎝ ns+1 ns ⎠ s=1 ⎜ − ⎟ ⎝ ns+1 ns ⎠ where a clockwise positive angle convention is used.

The starting position for raytracing is h1 = f × NA and α1=0. Consequently, applying the Abbe invariant for each surface we can substitute ray angles and heights with the system constructional parameters. Thus, from Abbe invariant for 1st surface we get

n2 −1 n2 −1 r1 = h1 or α 2 = h1 (2.12) n2α 2 n2r1

So α2 can be determined from the input ray height and prescription parameters of the first surface. The value of h2 is found from angle α2 (see Eq. (2.8)), and so on. Using this iterative process, and the relation between r1 and r2 from Eq. (2.9), we get:

⎛ 2 2 2 ⎞ (n2 − 3n2n3 + n3 ) (n2 −1)(n2 − n3 )(n3 −1) ⎜ − 3 2 − 2 2 3 + ⎟ ⎜ 32 f (n2 − n3 ) 4n2 (n2 − n3 ) r1 ⎟ W = h 4 ⎜ ⎟ (2.13) 040 1 ⎜ ⎟ ⎜ 2 2 2 2 2 ⎟ (n2 −1) (n2 + n2n3 + n3 ) (n2 −1)(n2 + n2n3 + n3 ) ⎜ + − ⎟ ⎜ 2 2 2 2 2 ⎟ ⎝ 8 fn2 (n2 − n3 ) r1 16 f n2 (n2 − n3 ) r1 ⎠

The defocus coefficient W020 between images in blue and red light is equal to –

L0/2 [28,30] where

⎛ ⎞ 2 (n3 −1)(2 f (1− n2 ) + n2r1 ) (n2 −1)(2 f (1− n3 ) + n3r1 ) L0 = −2W020 = h1 ⎜ − ⎟ (2.14) ⎝ f (n2 − n3 )r1n3v3 f (n2 − n3 )r1n2v2 ⎠

Eq. (2.14) is sufficiently accurate for the visible (photographic) spectral range, where the dispersion is approximately linear. Design for extended visible waveband (400-

700nm) requires calculations in two separate sub-bands with custom defined Abbe numbers to compensate for the increased nonlinearity of the glass dispersion curve. We define E(r1) = |W040|+|W020| as a merit function for 3rd order aberrations, which is continuous-valued and has a single global minimum identifying the approximate (near optimum) radius r1 for each valid two glass combination. Afterwards, radius r2 is calculated from Eq. (2.9).

The result is an algebraic expression for the 3rd order aberrations of the solution - if any - for a given glass combination. In our examples, we performed this calculation for

22 each of the 198,000 combinations of the 446 glasses which were available as of April

2012 in the combined Schott, Ohara, Sumita and Hoya glass catalogs [35, 36, 37, 38].

This yields a list of qualified candidates (those forming an image on or outside of the outer glass element), ranked by third order aberrations. However, this ranking is insufficiently accurate for a fully optimized lens.

Because of high numerical aperture, monocentric systems tend to have strong 5th and 7th order aberrations. That makes 3rd order analysis only a first approximation towards a good design. Fortunately, the two-glass monocentric lens system has an exact analytical ray trace solution in compact form, and the more accurate values of the lens prescription parameters can be found from a fast exact raytracing of several ray heights.

n1=1 n2=n4 n5=1

A φ1 φ2 B φ1' n3 C φ2' φ3' D φ3 h φ4 φ4' φ22 φ33 φ44 E φ1 φ55 θh O r2 r1

Figure 2.3: Monocentric lens real ray trace variables.

The variables for raytracing of rays with arbitrary input height h are shown in

Figure 2.3, where ϕi are the angles between the ray and the surface normal. We can write

h sin(φ1 ) = (2.15) r1

From Snell's law we have:

h sin(φ1 ') = (2.16) r1n2

Applying the sine law for the triangle ABO yields

r1 r1 h h sin(φ2 ) = sin(φ1 ') = = (2.17) r2 r2 r1n2 r2n2

Again from Snell's law

n2 n2 h h sin(φ2 ') = sin(φ2 ) = = (2.18) n3 n3 r2n2 r2n3

Triangle OBC has equal sides OB and OC. So,

n3 n3 h h sin(φ3 ') = sin(φ3 ) = = (2.19) n2 n2 r2n3 r2n2

From triangle OCD we find

r2 h sin(φ4 ) = sin(φ3 ') = (2.20) r1 r1n2

Then finally

h sin(φ4 ') = n2 sin(φ4 ) = (2.21) r1

One interesting result is that for the monocentric lens there is an invariant in the form:

φ4 ' ≡ φ1 (2.22)

Next, the segment OE=S can be found by applying the sine theorem to the triangle OED:

r1 sin(φ4 ') S = (2.23) sin ⎣⎡180° − (180° −φ4 ') − (180° −φ1 −φ22 −φ33 −φ44 )⎦⎤

r1 sin(φ4 ') S = (2.24) sin(−180° + 2φ1 +φ22 +φ33 +φ44 )

From Eq. (2.15)

⎛ h ⎞ (2.25) φ1 = arcsin⎜ ⎟ ⎝ r1 ⎠

From the triangles OAB, OBC and OCD we have

⎛ h ⎞ ⎛ h ⎞ (2.26) φ22 = φ2 −φ1 ' = arcsin⎜ ⎟ − arcsin⎜ ⎟ ⎝ r2n2 ⎠ ⎝ r1n2 ⎠

⎛ h ⎞ (2.27) φ33 = 180° − 2φ2 ' = 180° − 2arcsin⎜ ⎟ ⎝ r2n3 ⎠

⎛ h ⎞ ⎛ h ⎞ (2.28) φ44 = φ3 '−φ4 = arcsin⎜ ⎟ − arcsin⎜ ⎟ ⎝ r2n2 ⎠ ⎝ r1n2 ⎠

Finally,

h S = (2.29) ⎪⎧ ⎡ ⎛ h ⎞ ⎛ h ⎞ ⎛ h ⎞ ⎛ h ⎞ ⎤⎪⎫ sin ⎨2 arcsin − arcsin + arcsin − arcsin ⎬ ⎢ ⎜ r ⎟ ⎜ r n ⎟ ⎜ r n ⎟ ⎜ r n ⎟ ⎥ ⎩⎪ ⎣ ⎝ 1 ⎠ ⎝ 1 2 ⎠ ⎝ 2 2 ⎠ ⎝ 2 3 ⎠ ⎦⎭⎪

From Eq. (2.29) the longitudinal aberration for the ray with input height hi is given by

ΔS(hi ) = S(hi ) − f (2.30)

Radius r2 is bound to radius r1 with Eq. (2.9). With a given focal length and combination of refractive indices, the longitudinal aberration ΔS is actually a function of

a single variable r1. Finally, for the more accurate monochromatic optimization of the radius r1, we obtain a more accurate cost function Q:

3 3 Q = Abs ΔS h ,λ + Abs ⎡ΔS h ,λ − ΔS h ,λ ⎤ (2.31) ∑ ( ( i )) ∑∑ ⎣ ( j ) ( k )⎦ i=1 j=1 k≠ j where the first term of Eq. (2.31) minimizes longitudinal aberrations and the second term minimizes aberrations derivatives. In Eq. (2.31) for visible light operation, λ is the nd line located in center of photographic waveband and three input ray heights are: h1=f×NA, h2=0.7h1 and h3=0.4h1, respectively. With such steep ray angles Q is a strongly varying function, which is why a fast automated optimization can overlook the optimal radius values for a given glass pair. Figure 2.4 shows the dependence of the Q on radius r1 for a representative case, one of the best glass pairs (S-LAH79 and S-LAH59) for the 12 mm focal length lens described in Chapter 2.3.

Q 120

100

80 Q 3.0 60 2.0 40 1.0 0.0 20 9 10 11 12 r1 [mm]

2 4 6 8 10 12 r1 [mm]

Figure 2.4: Example of dependence of criterion Q on the radius r1

The monochromatic image quality criterion Q has several minimums over the possible range for r1 radius. The preliminary solution for r1 radius for this glass pair obtained from the third order aberrations minimization was 8.92 mm, close to the global minimum solution for Q found at 9.05 mm. This shows how the first optimization step provided a good starting point for r1 radius, avoiding the time consuming investigation of low quality solutions in the multi-extremum problem illustrated by Figure 2.4.

Optimization with the criterion in Eq. (2.31) gives the optimal solution by means of minimum geometrical aberrations, but it is not sufficient to provide reliable sorting of monocentric lens solutions. The system polychromatic mean square wavefront deformation is better correlated with the Strehl ratio and other diffraction image quality criteria [27]. So, in the third step, the wavefront deformation is calculated and expanded into Zernike polynomials. The polychromatic mean square wavefront deformation is calculated and used as a criterion for creating the ranked list of monocentric lens solutions by means of their quality. In the monocentric lens geometry the aperture stop is located at the center of the lens, where entrance and exit pupils coincide as well. This is shown in Figure 2.5.

Figure 2.5: Image formation in the monocentric lens.

For an arbitrary ray the lateral aberrations ΔY are bound to the wavefront deformation as

∂W λ ΔY = − (2.32) ∂ρ A where λ is wavelength, W is the wavefront deformation expressed in wavelengths; ρ is reduced ray pupil coordinate which varies from zero at the pupil center to unity at the edge; defining A as the back numerical aperture, and ΔY as the lateral aberration in mm

[27, 39]. From Figure 2.5 we have

ΔY ∂W λ ΔS(ρ) = = − (2.33) Aρ ∂ρ A2ρ where Aρ is directional cosine of the angle q or coordinate of the ray in the image space.

Expansion of the wavefront deformation into fringe Zernike polynomials up to 7th order is given in Table 9.1 of [27]. From the Eq. (2.33)

2 A 2 −ΔS(ρ,λi ) = 4C20 + C40 (24ρ −12) + λi (2.34) 4 2 6 4 2 +C60 (120ρ −120ρ + 24) + C80 (560ρ − 840ρ + 360ρ − 40)

The values for ΔS(ρ,λi) are calculated with fast raytracing [Eq.(2.30)] for nine rays with reduced coordinate heights ρj =1, 0.95, 0.9, 0.85, 0.8, 0.75, 0.7, 0.6, 0.5 and for three wavelengths 470 nm, 550 nm and 650 nm. Then coefficients Cn0(λi) are calculated from the least square criterion [40]:

2 2 ⎡ A 2 ⎤ −ΔS(ρ,λi ) − 4C20 (λi ) − C40 (λi ) 24ρ −12 − ⎢ λ ( ) ⎥ 9 ⎢ i ⎥ ⎢ 4 2 ⎥ −C60 (λi )(120ρ −120ρ + 24) − = min (2.35) ∑ ⎢ ⎥ j=1 ⎢ 6 4 2 ⎥ −C80 (λi )(560ρ − 840ρ + 360ρ − 40) ⎢ ⎥ ⎣ ⎦

We found that optimal geometrical aberrations solution from step two does not have complete correlation with diffraction quality criterions and has a small general defocus. In order to prevent general defocus of the image surface, we introduce a small

new shift dS of the back focal distance which makes the new coefficient C20 (λ2=550nm) equal to zero

λ2 dS = 4C20 (λ2 ) 2 (2.36) A

new This means that the system will have a slightly adjusted focus f = f+dS, and the

new only difference from before will be in coefficients C20 :

2 new A C20 (λi ) = C20 (λi ) − dS (2.37) 4λi

Finally, according to [27], (Chapter 9.1, Equation (23), and Chapter 9.2, Equation

(13)) the system polychromatic mean square wavefront deformation (ΔΦ)2 is:

3 2 2 2 2 ⎧ C new C C C ⎫ 2 2 2 1 ⎪⎣⎡ 20 (λi )⎦⎤ ⎣⎡ 40 (λi )⎦⎤ ⎣⎡ 60 (λi )⎦⎤ ⎣⎡ 80 (λi )⎦⎤ ⎪ (ΔΦ) = Φ − (Φ) = ⎨ + + + ⎬ (2.38) 2 ∑ 3 5 7 9 i=1 ⎩⎪ ⎭⎪

2 where Φ2 is the average value of squared wave aberration and (Φ) is the squared average wave aberration. In our examples, the top 50 solutions for different glasses combinations were sorted in the ranked list by polychromatic mean square wavefront deformation, then each was imported into ZEMAX optical design software and quickly optimized for the best modulation transfer function (MTF) performance at the 200 lp/mm. This frequency was chosen because the smallest fibers bundle receiver 24AS available from Schott has 2.5 microns pitch [14]. This close to the optimal design,

29 however, the MTF performance is well behaved, and a similar result is found with a wide range of MTF values.

2.3 Specific monocentric lens cases

2.3.1 AWARE2 monocentric lens analysis

The goal of this analysis is a monocentric lens optimization process to find the best possible candidates for fabrication. These candidates are then subject to other materials constraints involved in the final selection of a lens design, including mechanical aspects such as differential thermal expansion or environmental robustness, as well as practical aspects like availability and cost. The process described above appears to provide a comprehensive list of candidate designs. However, the best test of a lens design process is to compare the results to one generated by the normal process of software based lens design. To do this, we used the constraints of a monocentric objective which was designed in the DARPA AWARE program; specifically, the AWARE-2 objective lens [26] which was designed by a conventional software optimization process, then fabricated, tested, and integrated into the AWARE2 imager.

Table 2.1: Optical prescription of the fabricated AWARE2 lens.

Surface Type Radius Thickness Glass Semi-Diameter

OBJ Standard Infinity Infinity 1 Standard 31.80000 13.61300 S-NBH8 29.69700 2 Standard 18.18700 18.18700 F_SILICA 17.39400

STO Standard Infinity 18.18700 F_SILICA 16.95200

4 Standard -18.18700 15.19100 S-NBH8 16.95200 5 Standard -33.37800 36.87800 30.13600 IMA Standard -70.25600 60.84437

The lens has a 70 mm focal length and image space F# of 3.5, using a fused silica core and a S-NBH8 glass outer shell. The optical prescription is shown in Table 2.1, and the layout in Figure 2.6(a). The global optimization method identified this candidate lens system, as well as multiple alternative designs (glass combinations) which provide a similar physical volume and improved MTF. The optical prescription of the top-ranked solution is shown in the Table 2.2, and the lens layout is shown in Figure 2.6(b).

(a) (b)

S-NBH8 N-KZFS2

0° F_SILICA 0° S-FPL53 S-NBH8 20° 20° 40° 40°

60° 60°

Figure 2.6: AWARE2 lens and global optimum solution

The new candidate appears physically very similar to the fabricated lens.

However, the MTF and ray aberrations for manufactured prototype and the top design solution are compared in Figure 2.7. The new candidate lens is significantly closer to diffraction limited resolution, has lower chromatism and polychromatic mean square wavefront deformation than the actually fabricated lens. It is important to recognize that the resolution of the AWARE-2 imager system includes the microcamera relay optics.

The relay optics corrected for aberrations in the primary, as well as provided flattening of

31 the relayed image field onto the planar image sensors. In fact, the overall AWARE-2 system optics design [11] was diffraction limited. However, conducting the systematic design process on a relatively long focal length system, where geometrical aberrations are influential on resolution, served as a successful test of the design methodology.

Table 2.2: Optical prescription of the top design solution for the AWARE2 lens.

Surface Type Radius Thickness Glass Semi-Diameter

OBJ Standard Infinity Infinity 1 Standard 34.91298 17.95482 N-KZFS2 34.91298 2 Standard 16.95816 16.95816 S-FPL53 16.95816

STO Standard Infinity 16.95816 S-FPL53 7.00075

4 Standard -16.95816 17.95482 N-KZFS2 16.95816 5 Standard -34.91298 35.30018 34.91298 IMA Standard -70.21316 70.00000

TS Diff. Limit TS Diff. Limit TS 0.00 (deg) (a) TS 0.00 (deg) (b) 1.0 1.0 0.9 T-tangential 0.9 S-sagittal 0.8 OTF-optical transfer function 0.8 0.7 0.7 0.6 0.6 0.5 0.5 0.4 0.4 0.3 0.3 0.2 0.2 Modulus of the OTF Modulus of the OTF 0.1 0.1 0.0 0.0 0 30 60 90 120 150 180 210 240 270 300 0 30 60 90 120 150 180 210 240 270 300 Spatial Frequency in cycles per mm Spatial Frequency in cycles per mm

OBJ: 0.00 (deg) OBJ: 60.00 (deg) OBJ: 0.00 (deg) OBJ: 60.00 (deg) ey ey ex ey ey ex

Py Py Px Py Py Px λ[µm] λ[µm] 0.486 0.486 0.587 0.587 0.656 0.656 Maximum Scale: ± 10.000 µm. Maximum Scale: ± 10.000 µm.

Figure 2.7: MTF and ray aberrations performance comparison of the (a) fabricated AWARE2 prototype and (b) new AWARE2 design candidate.

2.3.2 SCENICC F/1.71 12mm focal length lenses

Our specific goal was to design an imager with at least 120° field of view and resolution and sensitivity comparable to the human eye (1 arc minute), resulting in about

100 Megapixels total resolution. Assuming we use the waveguide configuration of Figure

1.1(b) with 2.5 micron pitch NA=1 fiber bundles, we defined the goal of a 12 mm focal length lens with diffraction limited operation in the photographic spectral band (0.47 to

0.65µm) and the numerical aperture of 0.29 (F/# 1.71). We designed the monocentric lens assuming fiber stop operation mode (Figure 1.2(b)) as that is the more demanding design: the physical aperture stop and vignetting of the field beams will increase diffraction at wider field angles, but can only reduce geometrical optic aberrations.

The result of the design process was an initial evaluation of 198,000 glass pair systems, of which some 56,000 candidates passed the initial evaluation and were optimized using exact ray tracing to generate the final ranked list. The entire process took only 15 minutes to generate using single-threaded Matlab optimization code running on a

2.2GHz i7 Intel processor. Part of this list is shown in Table 2.3. Because different optical glass manufacturers produce similar glasses the solutions have been combined in families, and the table shows the first seven of these families. We show radii for a primary glass and list several substitutions glasses in parenthesis. Design with the substitutions glasses result in small changes in radii but substantially the same performance. The table shows the computed polychromatic mean square wavefront deformation, the criterion for the analytical global search, and the value for the MTF at

200 lp/mm (Nyquist sampling) found following Zemax optimization of the same

34 candidate glasses. If the design process has worked we would expect that these metrics would be strongly correlated.

Table 2.3: Top solutions for a F#/1.7 f=12 mm monocentric lens. MT Fast exact ZEMAX radii Internal glass F at raytracing [mm] # Outer glass (substitution glasses (ΔΦ)2 200l [mm] shown in parentheses) p/m R1 R2 R1 R2 m K-LASFN9 1 S-LAH79 9.049 3.765 0.0057 9.068 3.792 0.65 (TAF5, S-LAH59) S-LAL59 (K-LaKn12, 2 S-LAH79 8.049 3.772 0.0089 8.074 3.807 0.58 TAC4) K-VC80 (S-LAL13, 3 S-LAH79 M-LAC130, 7.581 3.738 0.0098 7.593 3.756 0.55 P-LAK35, LAC13) N-LASF45 (N- 4 K-PSFN2 8.871 3.773 0.0116 8.901 3.819 0.59 LASF45HT, S-LAM66) TAF4 (N-LAF21, 5 S-LAH79 8.700 3.782 0.0120 8.714 3.801 0.58 TAF1, S-LAH64) 6 K-PSFN203 BAFD8 (S-BAH28) 7.869 3.769 0.0124 7.886 3.791 0.54

7 P-SF68 BAFD8 (S-BAH28) 7.917 3.765 0.0135 7.934 3.787 0.53

Figure 2.8 illustrates by showing the correlation for representative samples of the top 200 solutions, and in fact we find the identical sequence for all 200 of the top candidates. The best performance monocentric lens solution (family number 1) demonstrates near diffraction-limited resolution over the photographic visible operational waveband (470 – 650nm). It uses S-LAH79 for the outer shell glass and K-LASFN9 for the inner glass. To provide a central aperture stop, it is necessary to fabricate the center lens as two hemispherical elements. Because optical glass K-LASFN9 has a high refractive index of 1.81, the interface between the two hemispherical elements can cause reflections at large incidence angles unless the interface is index matched, and the index of optical cements is limited. For example the Norland UV-cure epoxy NOA164 has an index of 1.64. This results in a critical angle of 65°, and a maximum achievable field of

35 view of ±55°. For this glass system, it is preferable to fabricate the center lens as a single spherical element and operate the system in the "virtual iris" mode, where the system can provide a maximum field of view of ±78.5°. The optical layout, MTF and ray fan diagrams of the top solution operating in “virtual” stop mode are shown in Figure 2.9.

Figure 2.8: Correlation between polychromatic mean square wavefront deformation and MTF @ 200lp/mm for candidates.

Table 2.4 shows the detailed optical prescription of the monocentric lens example.

The lens operation is shown in three configurations. The distance to the object plane is changed from infinity to 1m and then to 0.5m. The back focal distance (thickness 5 in

Table 2.4) is changed as well. The back focal distance for object at infinity is 2.92088 mm, for 1 m object distance 3.06156 mm and for the 0.5m object distance 3.20050 mm.

TS Diff. Limit TS 0.0000 (deg) 1.0 0.9 S-LAH79 0.8 0.7 K-LaSFn9 0.6 0° 0.5 0.4 0.3

40° Modulus of the OTF 0.2 0.1 0.0 60° 0 20 40 60 80 100 120 140 160 180 200 80° Spatial Frequency in cycles per mm

OBJ: 0.0000, 0.0000 (deg) ey ex λ[µm] λ[µm] 0.470 0.470 0.550 0.550 0.650 0.650

Py Px

Maximum Scale: Maximum Scale: ± 5.000 µm. ± 5.000 µm.

Figure 2.9: Highest ranked design solution (high index center glass).

Table 2.4: Optical prescription for refocusing of the monocentric lens, showing multiple configurations for three object distances

Surface Type Radius Thickness Glass Semi-Diameter

OBJ Standard Infinity 1.0E+10 / 1000.000 /500.000 1 Standard 9.06762 5.27607 S-LAH79 9.06000 2 Standard 3.79155 3.79155 K-LASFN9 3.79000

STO Standard Infinity 3.79155 K-LASFN9 1.94188

4 Standard -3.79155 5.27607 S-LAH79 3.79000 5 Standard -9.06762 2.92088 / 3.06156 / 3.20318 9.06000 IMA Standard -11.9885 - 11.0000

The MTF for two extreme object positions is shown in Figure 2.10. After refocusing to closer object distance, the image surface loses concentricity with the lens but still retain the original radius, as was shown in Chapter 2.2.1. The loss of concentricity enables conjugation between planar objects at a finite distance to the spherical image surface, and so implies the ability to focus onto points anywhere in the object space. A concentric system designed at a specific finite object radius can be conjugated only with a specific concentric spherical image surface, and focus of a concentric lens system onto spherical objects at different distances would require the change in the image surface radius. At closer object distances, angle α1 (Figure 2.1) increases from zero to some small finite value, which changes spherical aberration (Eq.

(2.10)) and MTF slightly away from optimum value. As predicted by third order aberration analysis, we maintain image quality close to the original infinite conjugate design during refocusing operation, over a wide range of field angles. Note that adequate simulation of the “virtual stop” operational mode in ZEMAX requires the rotation of the aperture stop at off-axis field angles, as well as rotation on this angle image surface around its center of curvature. In this file, a distinct configuration has been defined for each field angle.

The members of the third family of Table 2.3 have a glass with a significantly lower refraction index of the central glass ball, 1.69, which can be index matched with standard UV cure epoxies. This enables the lens to be fabricated as two halves, and assembled with a physical aperture stop at the center. While members of this family have slightly lower image quality performance they can fully operate over ±65° field of view in both “virtual” stop and “aperture” stop modes. The optical layout, MTF and the ray fan

38 diagrams of the top member of the third family operating in the “aperture stop” mode for the object located at infinity is shown in Figure 2.11(a) The ray fan graphs are shown for the axial and 60° field points at the image surface. The scale of ray fans is ±10µm. The design has been modified to include a 10µm thick layer of NOA164 glue between the two glasses, and in the central plane.

Figure 2.10: MTF performance of monocentric lens (a) focused at infinity (design) and (b) refocused at flat object at 0.5m

TS Diff. Limit TS 65.00 (deg) TS 0.00 (deg) TS 73.00 (deg) TS 30.00 (deg) (a) 1.0 0.9 0.8 S-LAH79 0.7 0.6 0° K-VC80 0.5 0.4 S-LAH79 0.3 30° 0.2 0.1 0.0 60° 0 20 40 60 80 100 120 140 160 180 200 73° Spatial Frequency in cycles per mm

OBJ: 0.00 (deg) OBJ: 60.00 (deg) ey λ[µm] ey ex 0.470 0.550 0.650

Py Py Px

Maximum Scale: Maximum Scale: ± 10.000 µm. ± 10.000 µm.

TS Diff. Limit TS 0.0000, 0.0000 (deg) (b) 1.0 0.9 0.8 S-LAH79 0.7 0.6 0° K-VC80 0.5 0.4 0.3 30° 0.2 0.1 0.0 0 20 40 60 80 100 120 140 160 180 200 60° 80° Spatial Frequency in cycles per mm

Figure 2.11: The top member of the third family (lower center glass index) operating in (a) the physical aperture stop, including a 73° field angle to illustrate the effect of aperture vignetting and (b) ‘‘virtual" aperture stop mode, with uniform response up to 80°

The optical layout and MTF of the top member of the third family operating in the

“fiber stop” mode is shown in Figure 2.11(b). The “virtual” stop mode has a uniform high image quality performance all over the field. The “aperture” stop mode suffers from drop in performance at the edge of the field due to the pupil vignetting and aberrations in the optical cement layer. This design is useful, however, as operation in the “aperture” stop mode simplifies requirements to the image transfer and detection. In this case the input aperture of the fibers bundle can exceed back aperture of the optics, as the physical aperture stop provides all stray light filtering needed, and the image transfer can be done with relay optics or with standard Schott fiber bundles with NA=1 and 2.5 micron pitch.

2.3.3 Comparison with conventional wide-angle lenses

The architecture of monocentric lens is intrinsically compact: the principal

(central) rays of all field angles are orthogonal to the front (powered) optical surface, and are directly focused to an image surface which is always substantially perpendicular to the incident light. Conventional wide field of view imagers require a lens which conveys wide field input to a contiguous planar image surface. Extreme wide angle "Fisheye" lenses use a two stage architecture in the more general class of reverse-telephoto lenses

[1, 41], where the back focal distance is greater than the focal length. The front optic is a wide aperture negative lens to insure acceptance of at least a fraction of light incident over a wide angle range. The negative power reduces the divergence of principal rays of the input beams, so the following focusing positive component operates with a significantly reduced field of view, although it must provide sufficient optical power to compensate for the negative first element. This retro-telephoto architecture results in a

41 physically bulky optic. Figure 2.12 shows two conventional wide-field lenses on the same scale as the f = 12 mm design example above.

TS Diff. Limit TS 0,0 deg TS 0,60 deg TS 0,30 deg 1.0 0.8 0.6 0.4 0.2 0.0 Modulus the of OTF 0 40 80 120 160 200 [lp/mm] TS Diff. Limit TS 0,0 deg TS 0,60 deg Weight ~ 371g 1.0 TS 0,30 deg 0.8 F#2.0 0.6 5mm 0.4 30mm 0.2

Modulus the of OTF 0.0 Weight ~ 28g 0 40 80 120 160 200 F#2.6 TS Diff. Limit [lp/mm] TS 0,0 deg 4.5mm TS 0,30 deg TS 0,60 deg 1.0 7mm F#1.7 0.8 0.6 Weight ~16g 0.4 (without bundles) 0.2

Modulus the of OTF 0.0 20mm 0 40 80 120 160 200 63mm [lp/mm] 136mm

Figure 2.12: Comparison of two conventional wide field lenses with a monocentric-waveguide lenses. All have a 12 mm focal length, 120° field of view, and similar light collection, but the monocentric lens provides higher resolution in a compact volume.

At the top of the figure is a 13-element "fisheye" lens taken directly from the

"Zebase" catalog of standard lens designs (F_004), then scaled from 31 mm to the 12 mm focal length. The smaller lens at the center of Figure 2.12 is based on the prescription for a 9-element wide field lens specifically designed to minimize volume [42]. This design was intended for a shorter focal length, and the aberrations are not well corrected when scaled to 12 mm. When this design was optimized, even allowing aspheric surfaces, the components tended toward the same overall physical shape and volume of the lens above.

Both of these lenses are substantially larger than the monocentric lens design, even including the volume of the fiber bundles, and they collect less light, even including for coupling and transmission losses in the fiber bundles.

2.4 Chapter summary

This whole chapter describe a general perspective on monocentric lens based imagers, including discussion of stray light control, focus, and a method for global search of optimal design solutions for 2-glass symmetric monocentric lenses. We proved that the monocentric lenses can be focused in the conventional way by axial moving the lens regarding a fixed radius spherical image surface, indicating feasibility of panoramic lenses using a fiber bundle image transfer. We checked our design approach by comparing the result for a 70 mm focal length lens, and concluded that it is an effective approach to identify top candidate solutions for specific applications. Practical constraints will determine the final selection. For the “virtual iris” stray light filtering a spherical high index central glass solutions can be used. With a conventional aperture stop, where the central ball lens consists of two hemispherical elements and a physical stop at the center, reflections from the internal surface can limit extreme field angle, favoring solutions with a lower index center lens.

Looking further at a specific design example, we identified high- and low-index center glass solutions for F/1.7 12 mm focal length monocentric lenses with (at least) a

120° field of view, and showed that this lens compares favorably to conventional fisheye and projection lens solutions to panoramic imaging. It is reasonable to ask whether the specific designs used for the comparison were optimal, and in fact there is clearly room

43 for improvement in these specific designs. However, the standard lens categorization shown in Figure 2.13 indicates that monocentric lenses can enter a domain of light collection and field of view which is not otherwise addressable [43].

Half field angle Full field angle log w log 2w SCENICC design virtual stop Fish-eye (F#1.7 160°) 100° 200° objective Topogon SCENICC design Metrogon Monocentric aperture stop Distagon (F#1.7 130°) Flektogon 50° Hologon 100° Biogon AWARE2 design Sonnar (F#2.4 120° ) 20° Tessar 40° Double- Landscape Cooke - Gauss objective triplet 10° Telephoto 20° Petzval- Petzval Projection

5.0° 10°

0.02 0.05 0.10 0.20 0.50 1.00 NA f/25 f/10 f/5.0 f/2.5 f/1.0 f/0.5 F#

Figure 2.13: Systematic diagram of photographic lens setup families, including monocentric multiscale and waveguide imagers. Figure adapted from [43].

We conclude that monocentric lenses offer a significant potential for panoramic imaging, and deserve further investigation. Following chapters show the systematic design of more complex monocentric lens structures and practical technologies for image transfer from the spherical image surface to planar image sensors.

No. 31, 2012. The dissertation author was the primary investigator and author of this paper.

3. Optimization of high-performance monocentric lenses

3.1 Limits of simple monocentric lenses

Monocentric imaging lenses, as discussed in previous chapter, can produce a high-resolution image on a spherical image surface. Since high-resolution spherical detectors are not currently available, in practical application this image surface is optically transferred onto multiple conventional focal planes. As mentioned before, this can be done by relay through multiple adjacent sets of secondary optics, as in monocentric multi-scale imagers [6,10,11]. Alternately, it can be done via imaging fiber bundles with curved input and flat output faces, as in the monocentric fiber-coupled imagers [13,19,21,20,44,45]. These successful demonstrations motivate a more systematic exploration of the capabilities of the monocentric lens. 45

In a lens with centered spherical or hemispherical surfaces, off-axis aberrations of coma and astigmatism are cancelled [1], but we need to correct spherical and chromatic aberration, and their combination spherochromatism. Reducing spherochromatism is difficult, especially with large apertures. But despite the monocentric constraint, and even with a small number of degrees of freedom, it is possible to obtain a number of useful designs [11,12,23,44].

45 46

In a previous work [44], discussed in Chapter 2, the general aberration analysis of two-glass symmetric (2GS) monocentric lenses and results of a 2GS global search algorithm were shown. The algorithm was applied for a specific example, a 12mm focal length, F/1.7, 120° field of view lens operating in 470-650nm visible waveband. The algorithm described identified the optimum diffraction limited design (Table 2.4 and

Figure 3.1(a)) and a number of additional families of high-performing solutions.

However, if we substantially increase the lens spectrum, light collection or the scale, even after repeated 2GS global search we will not achieve desired performance (Figure 3.1(b)-

(d)). The 2GS monocentric architecture reaches its limits.

Achieving a similar level of performance with these extended operating specifications demands more complex monocentric lens architectures, with more degrees of freedom. This is especially true when increasing more than one of these performance metrics.

In this chapter, we show methods and algorithms for advanced monocentric lens design. We categorize the monocentric lens design space, provide procedures for optimum and near-optimum lens design with complexity and performance tradeoff considerations, and in the Chapter 3.5 provide specific lens designs for selected applications: underwater near UV, night vision (NV), short wave infrared (SWIR), and medium and large scale (10-40GPixel) monocentric imaging.

TS 0.00 (deg) TS 60.00 (deg) TS 0.00 (deg) TS 60.00 (deg) TS 30.00 (deg) TS 30.00 (deg) 1 1

S-LAH79 S-LAH60 K-LaSFn9 K-GFK68 Modulus of the OTF 1cm 1cm RMS spot/Airy disc = 95% RMS spot/Airy disc = 295% 0 0 0 40 80 120 160 200 0 40 80 120 160 200 [lp/mm] [lp/mm] Optimized SCENICC 2-glass lens 3.3x Spectrum: 400-1000nm (a) (f=12mm, F/1.7, 470-650nm) (b) (f=12mm, F/1.7)

TS 0.00 (deg) TS 60.00 (deg) TS 0.00 (deg) TS 60.00 (deg) TS 30.00 (deg) TS 30.00 (deg) 1 1

K-PSFn173 S-LAH79 LAH80 S-LAH59 Modulus of the OTF 1cm 1cm RMS spot/Airy disc = 700% RMS spot/Airy disc = 422% 0 0 0 40 80 120 160 200 0 40 80 120 160 200 [lp/mm] [lp/mm] (c) 2x Signal: F/1.2 (d) 3x Scale: f=36mm (f=12mm, 470-650nm) (F/1.7, 470-650nm)

Figure 3.1: MTF performance curves showing the limits of the globally optimized 2GS monocentric geometries. The examples are derived from an initial high-performing lens (a) which is pushed to improve spectral bandwidth (b), numerical aperture (c) or focal length (d) [using a 3x scale change to the illustration]. For each, the resolution of the 2-glass structure drops well below the diffraction limit, indicating a need for greater complexity.

3.2 Options for improving monocentric lenses

3.2.1 Review of monocentric lens architectures

The simplest monocentric lens architecture is a simple glass ball [5,8] with inset aperture stop. Historically, the more common approach was an achromatic 2-glass symmetric architecture (2GS), used by Sutton in 1856, Baker in 1942 [3] and more recently by Brady and co-workers [10,46] and Ford and co-workers [6,11]. A more complex three-glass symmetric structure (3GS) with third order aberration analysis was designed and fabricated by Oakley [47] for a panoramic spherical retro reflector.

However, high order aberrations at large apertures were not systematically corrected.

Our goal was to push the performance of the existing lens and identify the limit of what monocentric lenses can or cannot do. We started by using commercial lens design software to explore the monocentric lens design space by a systematic increase of degrees of freedom in the system, while maintaining the monocentricity constraint, to identify the major configurations, which showed the most promise. We constrained the focal length and did a lens optimization for all options with a given number of degrees of freedom

(i.e. glass choice, surface radius) and compared performance to the diffraction limit.

Glass as an optical material has at least two description parameters, the index of refraction and the Abbe number. To model the dispersion over a broader spectral range would require an expression with even more free parameters. But since we don’t have the ability to create a glass with arbitrary index and dispersion, the choice of an existing glass material represents only a single degree of freedom. We use their accurate models described by Sellmeier, Extended or Schott glass model formulas.

Figure 3.2 summarizes the result, showing one hundred different geometries, and the seven preferred design architectures (drawn with a larger scale). Those preferred architectures were labeled as:

1GS: One-glass symmetric with 1 degree of freedom (DOF)

2GS: Two-glass symmetric with 3 DOF

3GS: Three-glass symmetric with 5 DOF

3GA-7: Three-glass asymmetric with air gap and 7 DOF

4GA-8: Four-glass asymmetric with air gap and 8 DOF

4GA-9: Four-glass asymmetric with air gap and 9 DOF

5GA-10: Five-glass asymmetric with air gap and 10 DOF

The 1GS, 2GS, 3GS and 4GA-8 geometries were chosen for rigorous analysis and investigation, because they offered the best performance for their structural complexity.

The simplest 1GS geometry is a symmetric glass ball, with only one degree of freedom (1 DOF). When the desired monocentric system is being designed, focal length input constrains one of the radii, so the choice of glass in this structure remains as the single variable. Since there is no chromatism correction, this architecture is suitable mostly for monochromatic imagers with a relatively large F-number. In air, if we allow this geometry to become asymmetric, or increase degrees of freedom to two – the optimizer will converge back to the original 1GS structure as depicted in the upper half of

Figure 3.2. A similar outcome results if we introduce an air gap and push up to the maximum of six degrees of freedom (lower part of Figure 3.2).

The next logical step was to make the achromatic lens with an additional glass, which yields the 2GS geometry with three degrees of freedom. As in 1GS geometry, optimization after lens splitting and introducing an air gap will converge back to the simpler 2GS geometry, while taking the upper “glass modification only” path on the chart by allowing all radii to vary will land in a 5DOF two-glass architecture that performs substantially the same as the simplest 2GS structure.

1 GLASS 1 GLASS + AIR GAP Gradient index 2 GLASS 2 GLASS + AIR GAP (Luneburg) lens 3 GLASS 3 GLASS + AIR GAP

4 GLASS 4 GLASS + AIR GAP Glass only design space

1GS 2GS 3GS 3GA-7 4GA-8 4GA-9 5GA-10

X X X

Preferred designs

Glass + air gap design space

1 2 3 4 5 6 7 8 9 10 Degrees of freedom

Figure 3.2: Monocentric lens design space showing glass only (upper half) and glass with air gap (lower half) regions divided by the seven preferred design architectures in between

Adding the third and fourth glass in the monocentric structure and breaking the symmetry goes deep into the upper “glass modification only” region of the design space and offers only marginal improvement over 2GS and 3GS structures, not justifying the cost of manufacture. Therefore, simple symmetry breaking and glass adding is not productive. The upper half of the diagram is only partially populated – if we allow more glasses and more degrees of freedom, architecture will essentially converge to some variant of the Luneburg lens solution [27].

On the other hand, starting from 5DOF structure with three glasses and introducing an air gap also doesn’t appear to help, as we continue along the lower part of

Figure 3.2 (glass with air gap path) up to the 6th degree of freedom. Then just a simple step over to the 7 degrees of freedom 3GA-7 architecture gives a substantial increase in performance, as shown with red arrow in Figure 3.2. Further derivatives 4GA-8, 4GA-9 and 5GA-10 just keep up with the same trend. Out of these asymmetric structures with an air gap, 4GA-8 is the most attractive one to pursue with an addition of 5GA-10 for larger scale lenses where the maximum glass slab size plays an important role.

Looking at this comprehensive monocentric lens design space chart, an interesting fact is that simply adding the degrees of freedom at some point does not help. This is somewhat counter-intuitive. For the two, four and six degrees of freedom cases no preferred monocentric lens structure exists. All symmetry breaking attempts in this ≤6

DOF area inevitably converge back to the symmetric structures when the lens is designed for the use in air. A slight change to this rule applies only when the lens has different medium in object and image space (e.g., an immersed lens), where a 4 DOF 2-glass structure with symmetric core becomes the preferred design.

Our next goal was to find specific high-performance designs. To do this we developed global search algorithms for symmetric geometries, and systematic search methods for the asymmetric geometries with an air gap, as described in the following.

3.2.2 Review of monocentric lens design methods

Throughout the exploration of monocentric lens design space, several methods and optimization algorithms were developed. In Chapter 2.2.2 a global optimization algorithm for the two-glass symmetric (2GS) architecture was presented. Now a similar approach was used in one-glass (1GS) and three-glass (3GS) symmetric architectures, and a similar global search algorithm was developed. Since all these global optimization routines are essentially brute force calculations (for all possible glass combinations), with further increase in the number of degrees of freedom the cost of computing became prohibitive. Therefore, we developed systematic search methods. All the methods use spectral band, focal length and F-number as an input for the desired system, and a predefined pool of commercially available glasses. These included the Schott, Ohara,

Hoya, Sumita catalogs as well as CAF2 and fused silica, totaling 604 different materials available as of April 2013. Hikari, CDGM and NHG manufacturers were not used because almost all of their glasses represent duplicate replacements of the glasses already included.

The optimization methods used, in order of increasing complexity and computation time, were:

• 1GS global optimization algorithm (seconds to complete)

• 2GS global optimization algorithm (minutes to complete)

• 3GS global optimization algorithm (days to complete)

• 2GS seeded Hammer search (hours to days)

• 4GA-8 architecture 5-D “near global” search (up to 3 weeks)

Preliminary results of these methods were presented in [48], but will be described in more detail here. Global optimization algorithms for 1GS, 2GS and 3GS architectures are multi-threaded exact ray trace routines implemented in MATLAB. These check all possible glass choices (604 for 1GS, 364,816 for 2GS and more than 220 million combinations for 3GS geometry). They were executed on PC workstations with two Intel

3.1GHz Xeon E5-2687W or four Intel 2.7GHz Xeon E5-4650 Sandy Bridge based processors (16/32 CPU cores systems). The 3GS global optimization algorithm was also rewritten and tested on Kepler based NVIDIA K20 Tesla and K5000 Quadro GPU cards, with speed improvements on the order of 70x, effectively cutting down the computing time required from days to hours.

The 2GS seeded Hammer search approach used glass combinations of the top

2GS candidates obtained through global search, then imported in ZEMAX. The lens symmetry was broken and an air gap manually introduced.

Additional glass layers were added one by one, as shown in Figure 3.3, with optimization at each step. In practice, the 2GS geometry was optimized by human assistance through ZEMAX Hammer search to 3GA-7 and then to 4GA-8 architecture or even more complex ones, if needed. The most complicated design approach, which was guaranteed to give the best result for the 4GA-8 geometry, was a five dimensional “near

54 global” search algorithm also implemented in MATLAB. Like the seeded Hammer search, the 5D optimization algorithm starts with the core identified as the best 2GS candidates, and then the algorithm tries all combinations for the additional three glasses used in the 4GA-8 geometry. Because of 5-dimensional optimization space complexity, this algorithm requires up to three weeks to complete running continuously on the 32CPU core workstation. We used this algorithm at the end of the lens design procedure, to determine the absolute best candidate for manufacture and test.

1GS 2GS 3GS 3GA-7 4GA-8 4GA-9 5GA-10

Global search Systematic search

Figure 3.3: Optimization of preferred monocentric lens geometries

3.3 Advanced design algorithms and results

3.3.1 Improved 2GS global search using 5 wavelengths

In the previous work [44] we reported a general aberration analysis of two-glass symmetric (2GS) monocentric lenses, and applied a global search algorithm to identify the best design for a visible waveband monocentric lens (see Chapter 2). To generate a ranked list of all lens candidates this algorithm used a three-step optimization for each possible two-glass combination: minimization of third order Seidel spherical and longitudinal chromatism aberrations, exact ray trace for multiple ray heights at the central

55 wavelength, and the calculation of polychromatic mean square wavefront deformation.

The algorithm described was sufficiently accurate for the visible (photographic) spectral range, where the glass dispersion curve is approximately linear.

However, to look for solutions in an extended waveband we modified the existing

3λ algorithm by replacing the first two steps by exact ray trace with five equally spaced wavelengths over the desired spectrum. The modified exact ray trace cost function became:

5 ⎧ 3 3 ⎫ ⎪ ⎪ (Q = Abs ΔS h ,λ + Abs ⎡ΔS h ,λ − ΔS h ,λ ⎤ (3.1) ⎨ ( ( i m )) ⎣ ( j m ) ( k m )⎦⎬ ∑ ⎪∑ ∑∑ ⎪ m=1 ⎩ i=1 j=1 k≠ j ⎭

where ΔS is longitudinal aberration of the ray of height hi. Also, we modified the polychromatic mean square wavefront deformation calculation for an increased number of wavelengths and increased the Zernike polynomials expansion to the 9th order. The system polychromatic mean square wavefront deformation (ΔΦ)2 became:

5 ⎧ new 2 2 2 2 2 ⎫ 2 1 ⎪⎣⎡C20 (λi )⎦⎤ ⎣⎡C40 (λi )⎦⎤ ⎣⎡C60 (λi )⎦⎤ ⎣⎡C80 (λi )⎦⎤ ⎣⎡C100 (λi )⎦⎤ ⎪ (ΔΦ) = ⎨ + + + + ⎬ (3.2) 5 ∑ 3 5 7 9 11 i=1 ⎩⎪ ⎭⎪

We then used the improved 2GS global search 5λ algorithm with the updated glass catalog to look again for the optimal designs for f=12mm F/1.7 470-650nm camera and a longer focal length lens needed for the AWARE 2 Gigapixel imager, with f =

70mm, F/3.5 and a 486-656nm spectrum.

For the f=12mm case, the previously optimal top family of solutions remained on top, while quite a few intermediate (but still inferior) families were generated (Table 3.1).

For simplicity, since there are many similar glasses in the catalogs, a number of glasses

56 that have a refraction index within ±0.03 and an Abbe number within ±2 of the glasses shown we considered as replacement glasses and omitted from the table.

Table 3.1: Updated list of top solutions for the SCENICC F/1.7 f=12mm 470-650nm 120° MC lens (prescriptions shown pertain to glass combinations marked in bold). Fast exact ray ZEMAX tracing optim. radii MTF at # Outer glass Internal glass (ΔΦ)2 [mm] [mm] 200lp/mm R1 R2 R1 R2 K-LaSFn9, TAF5, 1 S-LAH79 9.060 3.781 0.00547 9.068 3.792 0.648 S-LAH59 TAFD55 K-LaFK50, S-YGH52, 2 8.529 3.800 0.00618 8.533 3.807 0.622 (LASF35) M-TAC60… M(C)-TAF1, TAF5, N-LASF46A/B K-LaFK50(T), 3 (TAFD25, 9.057 3.614 0.00622 9.065 3.630 0.609 S-LAH59, K-LaSFn9, L-LAH86) S-LAH65(V)… 4 L-NBH54 K-LaFn9, S-LAM55 8.949 3.629 0.00633 8.960 3.649 0.606 K-GIR79 K-LaFK50T, M(C)- 5 (LAH80, TAF1, N-LAF21, 9.290 3.435 0.00637 9.300 3.456 0.591 N-LASF9) K-LaSFn16, TAF4… M-TAFD305, 6 TAFD40 9.533 3.730 0.00749 9.537 3.735 0.626 L-LAH85V, L-LAH83 M-TAF101, N-LAF21, K- LaSFn16, TAF4, 7 S-LAH79 8.470 3.802 0.00757 8.477 3.814 0.626 M-TAF1, TAC4, K-LaKn12… N-LASF45(HT), 8 K-PSFn5 9.348 3.600 0.00793 9.357 3.617 0.607 S-LAM66 N-LAF2, K-LaF2, LAF2, 9 TAFD40 S-LAM2, K-LaFn11, 8.191 3.792 0.00795 8.194 3.796 0.592 S-LAM61… K-LaK9, K-LaK12, N- LAK12, S-LAL12…; LASF35 10 K-VC80, K-LaK13, 7.487 3.746 0.00856 7.492 3.752 0.561 (S-LAH79) P-LAK35, L-LAL13, S- LAL13… N-LASF46A/B N-LAK12, K-LaK9, 11 (TAFD25, N-LAK12, S-LAL12, 7.813 3.701 0.00886 7.817 3.707 0.534 L-LAH86) LAC12, L-LAL12…

On the other hand, the longer f=70mm lens benefited significantly from the increased number of materials, and we identified two better candidate families than previously reported: K-VC82/P-LAF37/S-BAH11/K-LasFN10 with a CAF2 core (as shown in Figure 3.4(a)) and M-LAF81/MP-LAF81/L-LAM69/S-LAH60 with K-GFK68 core (as shown in Figure 3.4(b)).

TS 0.00 (deg) TS 0.00 (deg) 1 1 RMS spot/Airy disc = 54% RMS spot/Airy disc = 29%

M-LAF81 K-VC82 K-GFK68 CAF2

Modulus of the OTF 1cm 1cm

0 0 0 60 120 180 240 300 0 60 120 180 240 300 [lp/mm] [lp/mm] (a) (b)

Figure 3.4: New f=70mm AWARE 2 2GS candidates

The monocentric 2GS global search generates a full list of ranked solutions, which is why a global search algorithm is more powerful than simply doing guided hammer/global searches in commercial optical design software like ZEMAX or CODEV.

From a ranked list the lens designer can quickly choose the specific designs subject to specific constraints such as lens volume, differential thermal expansion, or glass material availability.

3.3.2 Three-glass symmetric (3GS) global search

After reaching the 2GS architecture limits, we explored the 3GS architecture shown in Figure 3.5. Similar to 2GS geometry, from the first order principles the focal length is given by [44,47]:

1 2 ⎛ 1 ⎞ 2 ⎛ 1 1 ⎞ 2 ⎛ 1 1 ⎞ = ⎜1− ⎟ + ⎜ − ⎟ + ⎜ − ⎟ (3.3) f r1 ⎝ n2 ⎠ r2 ⎝ n2 n3 ⎠ r3 ⎝ n3 n4 ⎠

For each chosen glass combination, one radius is a function of the other two radii and the predefined focal length target value.

n2=n6 n1=1 n7=1 n3=n5 φ A 1 φ2 φ ' φ3 1 φ2' n4 φ3' φ4' φ ' φ 5 4 φ5 φ h 6 φ6'

φ1 θh E

r3 O r2 r1

Figure 3.5: Monocentric three-glass symmetric (3GS) architecture

From the real ray trace geometrical equations we obtained:

h OE = ⎪⎧ ⎡ ⎛ h ⎞ ⎛ h ⎞ ⎛ h ⎞ ⎛ h ⎞ ⎛ h ⎞ ⎛ h ⎞ ⎤⎪⎫ sin ⎨2 arcsin − arcsin + arcsin − arcsin + arcsin − arcsin ⎬ ⎢ ⎜ r ⎟ ⎜ r n ⎟ ⎜ r n ⎟ ⎜ r n ⎟ ⎜ r n ⎟ ⎜ r n ⎟ ⎥ ⎩⎪ ⎣ ⎝ 1 ⎠ ⎝ 1 2 ⎠ ⎝ 2 2 ⎠ ⎝ 2 3 ⎠ ⎝ 3 3 ⎠ ⎝ 3 4 ⎠ ⎦⎭⎪

(3.4)

The longitudinal aberration for the ray with input height hi is given by

ΔS(hi ) = OE(hi ) − f (3.5)

Finally, we constructed the following merit function for 3GS geometry optimization:

9 8 9 8 Q = p ⋅ Abs ΔS h ,λ == p ⋅ Abs ΔS p ⋅ f ⋅ NA,λ ( 3.6) ∑∑ j ( ( j i )) ∑∑ j ( ( j i )) i=1 j=1 i=1 j=1

Where f is the focal length of the lens, NA the numerical aperture and p=[1, 0.9,

0.8, 0.7, 0.6, 0.5, 0.4, 0.3] are the pupil zones used to calculate eight ray heights. To allow for extended spectral bands and material dispersion curves, a ray trace is done for nine equally spaced wavelengths inside the spectrum of interest.

TS 0.00 (deg) 8 1

7 SF1 ] LASF35 TAFD55 3GS M-TAF1 M-TAF1 2GS nm mm [ 2 850 - r

6 435

1cm Modulus of the OTF 2GS RMS/Airy disc = 144% 3GS RMS/Airy disc = 120% 5 0 8 9 10 11 0 40 80 120 160 200 -20 0 20 r1 [mm] (a) (b) Optimal 2GS/3GS lens [lp/mm] (c) Chromatic Focal Shift[µm] Figure 3.6: Top 12mm F/1.7 435-850nm 3GS monocentric lens candidate (a) optimization space, (b) MTF comparison curves with top 2GS candidate and (c) apochromatic shaped focal shift curve

In the 3GS geometry there is around 220 million glass combinations, and the optimization problem is inherently two-dimensional. In an attempt to reduce the computing time to reasonable limits, we identified and made use of an interesting fact about 3GS geometries. In the 2-dimensional optimization space of the 3GS monocentric system, if the glass choice is viable, areas of minimum merit function (high performance) look like a long and nearly linear ravine (Figure 3.6(a)).

So it is possible to fix the radius of the second glass shell at two points with some reasonable values, cross the ravine at two r2 levels to get its orientation, and then trade the two-dimensional optimization problem for one-dimensional track along the ravine. This increased the computational efficiency and made it possible for the global search to run in

24 to 48 hours on a high performance workstation (4x2.7Hz Intel Xeon E5-4650).

Because the ravine is substantially flat at the bottom, we had the freedom to choose the second radius in 3GS system, and this was helpful in avoiding excessively thin shell solutions, which are impractical to fabricate. Figure 3.6(b) shows the comparison of 2GS and 3GS top candidates for the 12mm F/1.7 monocentric lens operating in the extended visible (435-850nm) waveband. Both solutions are strong apochromats (Figure 3.6(c)), but unfortunately the 3GS geometry offered only modest performance improvement in

MTF and the spot size. A similar result is observed when the global 3GS search is applied in all scenarios for the f=12mm imager lens variants discussed in Chapter 3.1.

3GS global search generates a number of high ranked solutions that have nearly identical performance, and it is difficult to say which one is the absolute best performer. Some solutions have slightly better MTF but worse RMS spot size, and vice versa. The candidates shown in Figure 3.7 are chosen by MTF performance. An interesting fact about the 3GS geometry is that all good solutions are always derivatives from the good

2GS candidates. In other words, glass core materials of the top candidates identified in

2GS global search also form top 3GS solutions. That kind of behavior was observed in all design scenarios. Therefore the “quick track” to global 3GS solutions may be the exploration of all glass combinations, constrained only by the limited number of materials for the core.

TS 0.00 (deg) TS 0.00 (deg) 1 1

SF1 S-LAH79 2GS 3GS S-LAH59 N-KZFS11 TAFD45 K-GFK68 3GS 2GS

Modulus of the OTF 2GS RMS/Airy disc = 95% 2GS RMS/Airy disc = 295% 1cm 3GS RMS/Airy disc = 116% 1cm 3GS RMS/Airy disc = 256% 0 0 0 40 80 120 160 200 0 40 80 120 160 200 [lp/mm] [lp/mm] Optimized SCENICC 3-glass lens 3.3x Spectrum: 400-1000nm (a) (f=12mm, F/1.7, 470-650nm) (b) (f=12mm, F/1.7)

TS 0.00 (deg) TS 0.00 (deg) 1 1

SF1 S-NPH1 S-LAH79 K-PSFn215 L-LAH86 S-LAH59 3GS 3GS 2GS 2GS

Modulus of the OTF 2GS RMS/Airy disc = 700% 2GS RMS/Airy disc = 422% 1cm 3GS RMS/Airy disc = 558% 1cm 3GS RMS/Airy disc = 347% 0 0 0 40 80 120 160 200 0 40 80 120 160 200 [lp/mm] [lp/mm] (c) 2x Signal: F/1.2 3x Scale: f=36mm (f=12mm, 470-650nm) (d) (F/1.7, 470-650nm)

Figure 3.7: MTF performance comparison of globally optimized 2GS and 3GS monocentric lenses for extension of the original lens (a) specifications. The plots show only on-axis MTF, to allow comparison of 2GS and 3GS architectures.

This effectively reduces the CPU computing time from days to hours or, in case of GPU computing, from hours to minutes. After a number of global 3GS searches and

62 comparisons, we finally concluded that over the scale of apertures and waveband parameters we considered here, the 3GS architecture does not offer a significant performance improvement over the 2GS architecture.

3.3.3 Seeded Hammer Optimization

For the desired monocentric lens specification, as a start, a 2GS global search was performed and the full list of ranked candidates was created. Then, the multiple top candidate prescriptions were imported to ZEMAX and manual lens splitting and air gap introduction were performed: first guiding the candidate optimization to 3GA-7 structure, and then to 4GA-8 structure. All glass materials were substitution variables except the core, and within hours (sometimes even minutes) Hammer search would find a useful solution. We must emphasize that in order for that to happen, the most important thing is a good starting core material for the given design. Without 2GS global search algorithm input, both ZEMAX and CODEV may have a hard time converging to the best solutions if the starting design core material is not close to the ideal, especially for the low F- number cases.

The reason for that was the shape of multiple local minimums in the monocentric design space as discussed in Chapter 2.2.2. Optimized 4GA-8 structures through the

ZEMAX Hammer optimization seeded from top 2GS candidates of the modified 12mm imager specification lenses are shown in Figure 3.8. The original lens (Figure 3.8(a)) was substantially diffraction-limited in both the 2GS and 3GS geometry, so with the original design specification only a slight improvement is seen.

TS 0.00 (deg) TS 60.00 (deg) TS 0.00 (deg) TS 60.00 (deg) TS 30.00 (deg) TS 30.00 (deg) 1 1

S-LAH79 K-LaSFn9 TAF4 TAFD30 K-GFK68 L-LAH86 K-VC91 KZFS12 Modulus of the OTF 1cm 1cm RMS spot/Airy disc = 63% RMS spot/Airy disc = 114% 0 0 0 40 80 120 160 200 0 40 80 120 160 200 [lp/mm] [lp/mm] Optimized SCENICC 4GA-8 lens 3.3x Spectrum: 400-1000nm (a) (f=12mm, F/1.7, 470-650nm) (b) (f=12mm, F/1.7)

TS 0.00 (deg) TS 60.00 (deg) TS 0.00 (deg) TS 60.00 (deg) TS 30.00 (deg) TS 30.00 (deg) 1 1

K-PSFn215 S-LAH79 N-LASF46B S-LAH59 S-LAH79 K-LaSFn9 SF6 TAFD55 Modulus of the OTF 1cm 1cm RMS spot/Airy disc = 454% RMS spot/Airy disc = 232% 0 0 0 40 80 120 160 200 0 40 80 120 160 200 [lp/mm] [lp/mm] (c) 2x Signal: F/1.2 3x Scale: f=36mm (f=12mm, 470-650nm) (d) (F/1.7, 470-650nm)

Figure 3.8: MTF performance curves of the 4GA-8 lens geometries derived from the original lens specifications through seeded Hammer optimization.

For the other three cases, however, the more complex lens achieved substantial performance improvement. Hammer optimization is not global, and even higher

64 performance designs may exist, so the 4GA-8 architecture is a promising choice for high- performance lenses. This lens is sufficiently complex that an exhaustive and truly global optimization is impractical, but in the following section we describe a 5-dimensional

4GA-8 monocentric architecture optimizer to identify “near global” lens designs.

3.3.4 Five-dimensional 4GA-8 near global optimization

A useful solution is to break the front/rear symmetry and introduce an asymmetric air gap between the crown and flint glass core [19,49]. Introducing such an air gap is a common method used for control of spherochromatism [30,31,39]. This approach yields the 4-glass air gap asymmetric geometry, which improves performance on extended spectral bands, larger apertures, and longer focal length systems. The four-glass with air gap (4GA-8) lens architecture is shown in Figure 3.9.

Attempts to optimize the four-glass architecture with ZEMAX software shows that the result of optimization strongly depends on the initial starting point position. Some results obtained from very different starting points in the radii space showed good image quality but others were trapped in lower quality pockets. Such behavior of the commercial lens design software indicates that the optimization space of the 4GA-8 monocentric lenses has some specific features that must be investigated and special optimization algorithms to be developed. As for the 2GS and 3GS architectures, for lens quality evaluation we used fast exact monocentric lens ray tracing.

n5 n1=1 n6=1 n2 n4 A α1 α2 B α1' C α2' α3' h α3 D n3 α4 α4' α22 α5 E α5' α33 α44 α55 α6 F α α66 α6' 1 α77 G O R3=R2 R2

R1 R5

Figure 3.9: Four-glass asymmetric with airgap (4GA-8) monocentric lens architecture

For the input ray at height h it gives the value for the length OG:

⎪⎧ ⎛ h ⎞ ⎛ h ⎞ ⎛ h ⎞ ⎛ h ⎞ ⎛ h ⎞ h sin ⎨arcsin⎜ ⎟ + arcsin⎜ ⎟ + arcsin⎜ ⎟ − arcsin⎜ ⎟ − 2arcsin⎜ ⎟ + ⎩⎪ ⎝ R1 ⎠ ⎝ R2n2 ⎠ ⎝ R2n4 ⎠ ⎝ R1n2 ⎠ ⎝ R2n3 ⎠ OG = (3.7) ⎛ h ⎞ ⎛ h ⎞ ⎛ h ⎞ ⎛ h ⎞ ⎛ h ⎞ ⎛ h ⎞ ⎪⎫ + arcsin⎜ ⎟ − arcsin⎜ ⎟ − arcsin⎜ ⎟ + arcsin⎜ ⎟ + arcsin⎜ ⎟ − arcsin⎜ ⎟ ⎬ ⎝ R4 ⎠ ⎝ R4n4 ⎠ ⎝ R5 ⎠ ⎝ R5n5 ⎠ ⎝ R6 ⎠ ⎝ R6n5 ⎠ ⎭⎪

The longitudinal aberration ΔS(hi) for this ray will be:

ΔS(hi ) = OG(hi ) − f (3.8)

where f is the focal length. To form the optimization criterion the results of fast exact ray tracing with Eq. (3.7) and Eq. (3.8) were used. The entrance heights h of these rays are:

hi = NA⋅ f ⋅ pi (3.9)

Where pi is an array of reduced rays heights at the pupil. The array is defined as:

p = ⎣⎡ 1 0.97 0.88 0.8 0.7 0.6 0.5 0.4 0.05 ⎦⎤ (3.10)

For the optimization criterion C the following sum was used:

9 9 2 ⎡ ΔS(hi )⎤ 2 p ⎡ S h , S h , ⎤ C = ⎢ j ⋅ ⎥ + ⎣Δ ( 9 λ1 ) − Δ ( 9 λ9 )⎦ + ∑∑ λ j (3.11) i=1 j=1 ⎣ ⎦ 2 2 + ⎣⎡ΔS(h3,λ1 ) − ΔS(h3,λ9 )⎦⎤ + ⎣⎡ΔS(h1,λ1 ) − ΔS(h1,λ9 )⎦⎤

Where λj is the wavelength in microns used for weighting. The first term of the criterion C equation is a sum of squared values proportional to lateral aberrations and the following three members are squared chromatic longitudinal aberrations differences at the pupil reduced rays heights 1, 0.88 and 0.05. The longitudinal chromatic difference at the reduced pupil height 0.05 is similar to the classical chromatic focus shift. Pupil points with reduced pupil height 1 and 0.88 are critical for the spherochromatism correction. For optimization of any monocentric lens operating in an extended waveband we used nine wavelengths. For example, for criterion calculations for monocentric lenses designed to operate with a front-illuminated silicon CMOS or CCD sensor, we used the waveband 0.4 to 1.0 micrometers, divided into eight equal segments at nine wavelength values. This criterion demonstrated a good correlation with modulation transfer function (MTF) for all types of monocentric lenses operating in extended wavebands.

The starting point for the 4GA-8 systematic search is to make use of the core from multiple 2GS top candidates as seeds for further optimization. The most promising glass

K-GFK68 was chosen as a basic core glass for the systematic solution search, and then the other glasses were replaced in all possible combinations. For each glass combination the search of minimum of the criterion C (Eq. (3.11)) was performed and the optimized system was found. With the chosen glass combination we have five radii to optimize. In fact there are seven radii in the optical scheme, including the image surface radius

(Figure 3.9), but the third radius is equal and opposite to the second radius because the use of the central ball lens, and the image surface radius must match the focal length.

Investigation of the criterion C behavior showed the multi extremum nature of the function being optimized. But in our case, on top of this problem, we have a number of linear combinations between optimization parameters, or in other words, lines and surfaces in the optimization space over which the criterion does not change or changes very slowly. The optimization process is stuck somewhere in these ravines depending on the starting point position. Such areas are multidimensional ravines or saddle type stationary areas.

For each glass combination the minimum value of the criterion C is located at different positions in the radii space, but all glass combinations still have the characteristic general shape of solutions in the 5D radii space. For every glass combination the contour surfaces with the constant values of criterion C around the minimums appear as a thin, "pancake-shaped" volumes in the 5D radii space. These thin pancake volumes are pierced with a net of saddle type ravines, and are connected over the main ravine. Applying conventional optimization methods results in slow convergence to

68 a solution trapped in the saddle type ravines, rather than the global minimum [50,51]. The behavior of the gradient method in such cases is illustrated in Figure 3.10 (Adapted from

[51]). Figure 3.10(a) shows the gradient method behavior in the general case of the normal minimum shape. The gradient descent direction at any step is directed orthogonally to the criterion contour line and straight descent is continued up to the point when it reaches another lower value contour line for which the direction of descent would be tangential.

(0) X (0) (a) X0 (b) 0

(1) X0

X (2) 0 (2) (3) X0 (1) X0 X0

xmin

xLmin

Figure 3.10: Gradient descent method applied for normal minimum and degraded minimum shape

The process converges quickly to the minimum in just a few steps. In the case of a degraded (stretched) minimum with the strong linear dependence between optimization parameters, as shown in Figure 3.10(b) the gradient method begins to oscillate. In [52] the method of conjugated gradients was proposed. It was shown that apexes of segmented

69 lines in the gradient method are located on the lines showing the direction to the minimum and after several steps we can create these lines by using the least square method. The move along these lines will facilitate a fast descent to the minimum.

While the method of conjugated gradients can help in a number of degraded minimum cases, our situation is more sophisticated. The minimum volume has the shape of a thin pancake in the 5D space, with the walls close to being parallel, such that

i i+1 i+1 i+2 gradient descent segments X0 X0 and X0 X0 will practically coincide. The points

i i+2 i+4 X0 , X0 , X0 will be located so close to each other that we will not be able to reliably connect them with the line. Moreover, all the points inside the five dimensional thin pancake minimum area are saddle-like points. At every point we have Hesse matrix

[50,53] having one nearly zero negative eigenvalue, demonstrating a strong linear dependence between the first (R1) and last (R6) radii, while other eigenvalues will be strongly positive. The saddle type nature of the area of the minimum solution is another reason that conventional optimization methods are likely to be trapped at different points inside the pancake, where the specific end-point depends sensitively on the initial starting point of the optimization. In this situation even the method of conjugated gradients fails.

The optimization of our lens architecture required the development of special methods, which we will illustrate using the example of a lens with the following glass combination: P-LASF47, K-GFK68, K-LASFN6 and N-KZFS11. This glass combination demonstrated sufficiently good performance during our search for the optimal solution and was chosen as an example to demonstrate the optimization procedure.

Our search for the near minimum begins with a gradient descent [50] to the closest local minimum from the average radii solution for this architecture (Figure 3.9).

The optimization of this glass combination begins from the average radii combination at point M0=(7.0, 2.9, -2.9, -4.2, -4.5, -7.8, -12.0), in millimeters. The value of the criterion

C [Eq.(3.11)] at this point is 13.23. The local gradient descent method quickly arrives to the point inside pancake area with radii array M1=(7.06457, 2.95860, -2.95860, -4.16102,

-4.46400, -7.74390), which again are shown in millimeters, and criterion value at this point is C=0.00709. The contour lines graphs of the criterion C [Eq. (3.11)] in the plane section of radii R1-R6 of the 5-dimensional space is shown in Figure 3.11. (a) (b)

0.25 -7.73 3D plot of the optimization criterion 0.20

X=7.0646 Y=-7.7439 0.15

] -7.74

Level=0.0070922 mm

[ 0.10

6 R -7.75 0.05

0 -7.76 −7.72 −7.73 7.08 7.05 7.06 7.07 7.08 −7.74 7.07 R1[mm] −7.75 7.06 Contour plot of the optimization criterion −7.76 7.05 7.04

Figure 3.11: The optimization criterion ravine of minimums (projection onto 3D space)

The minimum area is a thin long strip, which is the section of a five dimensional thin pancake. The minimum point M1 is shown with information box in Figure 3.11(a).

We used the Hesse matrix eigenvectors [53] to find the direction of the strip. The five eigenvalues of the Hesse matrix at this point are: -0.01024, 0.25782, 0.91727, 25.44 and

21148.91. Note that radius number three does not participate in the optimization because it is equal to the negative value of the second radius. The two eigenvectors having

71 smallest eigenvalues are: E1 = (-0.6007, 0.00309, -0.00223, 0.00967, -0.79988) and E2 =

(0.31650, 0.12960, -0.50686, -0.75252, -0.24462), where the eigenvector direction cosines are related to the radii R1, R2, R4, R5 and R6 respectively. The section of criterion C shows that the main ravine is related to the eigenvector E1. We will name this ravine as the main virtual ravine (Figure 3.12).

Knowing that gradient methods are descending in the direction orthogonal to the contours of constant criterion C value, it is not surprising that gradient methods from any initial point come toward the main virtual ravine but to different locations over this ravine depending on the location of the initial point. The 3D graph of the criterion C with dependence on the first and last radii is shown in Figure 3.11(b). The value of the criterion function C in the direction orthogonal to the ravine quickly reaches the value of

0.4, with radii space step size as small as 0.02 mm. We will use the main ravine as an entrance area into the pancake space. The ravine is located very close to the straight line associated with the eigenvector E1 (the black dotted line in Figure 3.12). We travel over this straight line that is defined by the first eigenvector direction at the point of the minimum M1 and make a number of local optimizations, which will quickly come to the actual lowest point of the ravines.

Following over the direction of eigenvector E1 we made 17 steps with 0.05 increments in the radii space (8 steps in the direction of lower first radius and 8 steps in the opposite direction) and, making the gradient descent from each point, obtained the array of minimums. Table 3.2 shows the central ten minimums (ravine bottom points) of this scan with minimum M1 over the main ravine at place number 9. Table 3.2 shows that

72 minimums are located over a deep and slightly curved ravine, with a strong linear dependence between the first and last radii.

Table 3.2: The array of local minimums over the main ravine in 4GA-8 optimization space. Radii shown pertain to the areas at the bottom of the ravine reached after optimization

5 6 7 8 9 10 11 12 13 14

Cinitial 0.0132 0.0089 0.0074 0.0071 0.0071 0.0071 0.0075 0.0092 0.0135 0.0220

r1 6.945 6.975 7.005 7.035 7.065 7.010 7.125 7.155 7.186 7.216

r2 2.959 2.959 2.959 2.959 2.959 2.959 2.958 2.959 2.956 2.957

r3 -2.959 -2.959 -2.959 -2.959 -2.959 -2.959 -2.959 -2.963 -2.956 -2.957

r4 -4.162 -4.162 -4.161 -4.161 -4.161 -4.161 -4.161 -4.161 -4.161 -4.161

r5 -4.462 -4.462 -4.463 -4.463 -4.464 -4.464 -4.465 -4.465 -4.466 -4.466

r6 -7.904 -7.864 -7.824 -7.784 -7.744 -7.704 -7.664 -7.624 -7.584 -7.544

Cfinal 0.0072 0.0071 0.0071 0.0071 0.0071 0.0071 0.0072 0.0073 0.0075 0.0078

The body of the pancake shaped minimum is located over the 3D sphere inside the 5D space, and this sphere is orthogonal to the ravine, as shown in Figure 3.11(b). The directions of the minimum increment of the criterion C at each bottom point over the ravine is a direction of the second eigenvector while the first one is still directed over the ravine. The tunnels into the pancake (black dashed lines in Figure 3.12) [53] are located over the directions of second eigenvectors. These vectors are orthogonal to the ravines

(blue dash-dot lines in Figure 3.12).

To find the point of the criterion C minimum we crossed the ravine structure inside the pancake area of solutions from the point M1 in the direction of the second eigenvector E2. Then we initiated local gradient descents with the step of 0.025 mm.

Values of C over this line after local gradient descents are 0.00709 (point M1), then

[0.00622, 0.00558, 0.00504, 0.00470, 0.00439, 0.00412, 0.000400, 0.00393. 0.00392,

0.00399]. The ravine with the minimum value of criterion C is at the point M2, having

C=0.00392 and the array of radii is M2= (7.13558, 2.98846, -2.98846, -4.27410, -

4.63395, -7,79880). Then we made 17 steps along eigenvector E1 of this ravine with the short gradient descent. The values of C over ravine are [0.00431, 0.00424, 0.00413,

0.00405, 0.00401, 0.00399, 0.00394, 0.00395, 0.00393, 0.00392, 0.00391, 0.00394,

0.00395, 0.00396, 0.00404, 0.00413, 0.00418]. The minimum value of C=0.00391 is at the point M3 with array of radii M3=(7.10544, 2.98890, -2.98890, -4.27469, -4.63355, -

7.83869).

Starting point M0 13.23 MR0

M3 Increasing radius R1 M2 0.00391 0.00392 M4 M5 0.00385 (0.00384) ML0

E2 (0.00337)

12 8 11 7 10 6 4 5 (0.00402) 3

Left area of solutions Central pancake area of solutions (R1 ≈ 6.5mm) (R1 ≈ 7.0mm) Right area of solutions (R1 ≈ 7.5mm)

Figure 3.12: Optimization procedure inside the 4GA-8 optimization space.

We performed this whole operation in cycles until the step when the minimum is located at the initial point of the last cycle. The whole optimization path is shown by the solid red line in Figure 3.12.

Values of C around M3 are [0.00426, 0.00425, 0.00405, 0.00391, 0.00385,

0.00388, 0.00394], where the new minimum point M4 has C=0.00385, and array of radii

M4=(7.09785, 2.98510, -2.98510, -4.26180, -4.61467, -7.83319). The next step along this new ravine associated with the point M4 gives the point M5 at one step from M4 with C=0.00384. The array of radii at M5 is M5=(7.08293, 2.98531, -2.98531, -4.26206,

-4.61453, -7.85324).

The next crossing of the ravines did not succeed and the minimum C point remained at the point M5, indicating that we had approached the limit of this process.

The next step was to substitute the array of radii at the point M5 with C= 0.00384 into

ZEMAX software, where we obtained the MTF value of 0.54 at 200 lp/mm frequency.

After that we used the standard Zemax process for a local optimization of the optical prescription to obtain the maximum MTF. The final results are shown in Table 3.3. For the MTF optimized scheme MTF at 200 lp/mm is 0.567. It is slightly better than we had at the optimum point of criterion C (Eq. (3.11)). The construction of a low burden computer criterion from the results of raytracing, which will perfectly correlate with MTF performance, is still an open problem [51,54]. However, we consider that our criterion is in good correlation with MTF, allowing us to sort the results of the search for 4GA-8 architecture. The optimization process from the different initial point MR0 having

R1=7.5 mm shows another solution inside the neighboring pancake area on the right

(Figure 3.12), with the value of C=0.00337.

Table 3.3: Optical prescription of the 400-1000nm F/1.7 f=12mm MC lens example solution A=5.63g Radius Thickness Glass Semi-diameter OBJ Infinity Infinity Infinity 1 7.05918 4.10337 P-LASF47 6.75121 2 2.95581 2.95581 K-GFK68 2.93356 STO Infinity 2.95581 K-GFK68 2.20989 4 -2.95581 1.26426 K-LASFN6 2.91646 5 -4.22007 0.35576 4.05564 6 -4.57583 3.27037 N-KZFS11 4.34140 7 -7.84620 4.14501 7.12060 IMA -11.9913 10.38526

The optimized MTF at frequency 200 lp/m for this solution is 0.575. The optical prescription is shown in the Table 3.4.

Table 3.4: Optical prescription of the 400-1000nm F/1.7 f=12mm MC lens example solution B=5.71g Radius Thickness Glass Semi-diameter OBJ Infinity Infinity Infinity 1 7.26696 4.34654 P-LASF47 6.93839 2 2.92041 2.92041 K-GFK68 2.89944 STO Infinity 2.92041 K-GFK68 2.20692 4 -2.92041 1.29818 K-LASFN6 2.88460 5 -4.21859 0.41330 4.05796 6 -4.63189 3.00543 N-KZFS11 4.39123 7 -7.63733 4.35056 6.95098 IMA -11.98789 10.38221

Similarly, the next optimization process from the initial point ML0 having R1=6.5 mm shows another solution inside the neighboring pancake on the left with C=0.00402.

The optimized MTF at frequency 200 lp/m for this solution is 0.562. The optical

76 prescription is shown in the Table 3.5. In the global search among these three feasible solutions we prefer the solutions of first type shown in Table 3.3, as they have the smallest weight.

Table 3.5: Optical prescription of the 400-1000nm F/1.7 f=12mm MC lens example solution C=5.72g Radius Thickness Glass Semi-diameter OBJ Infinity Infinity Infinity 1 6.56181 3.55968 P-LASF47 6.30232 2 3.00213 3.00213 K-GFK68 2.97882 STO Infinity 3.00213 K-GFK68 2.21707 4 -3.00213 1.16917 K-LASFN6 2.95718 5 -4.17130 0.24397 4.00443 6 -4.41527 4.06139 N-KZFS11 4.19853 7 -8.47720 3.51334 7.62830 IMA -11.99054 10.38469

The procedure described above was applied on all other glass combinations, and our near-global search resulted in 350 top solutions that are grouped in seven distinctive families (Table 3.6). Glasses are considered replacement glasses if their index of refraction is within ±0.03 range and Abbe number in ±3 range of glasses shown in the table.

The example solution discussed before, shown in Table 3.3 - Table 3.5 belongs to the first family of solutions. Figure 3.13 shows the optical layout of the lens and MTF curves of the top solution from the first family and compares it with the seeded Hammer result from Figure 3.8(b). The prescription of this near-global optimum solution is shown in Table 3.7. Upon inspection of the full solutions list, seeded Hammer solution was

77 located with C=0.006354 near-global search criterion value and obviously far outside the top solutions families.

Table 3.6: Families of solutions for F/1.7 12mm monocentric 4GA-8 400-1000nm lens obtained through near-global search. Near-global Family 1st glass 2nd glass 3rd glass 4th glass MTF at search (core) (meniscus) 200lp/mm criterion I P-LASF50 K-GFK68 TAF1 E-ADF10 0.003253 0.583 II NBFD11 K-GFK68 TAF3 KZFS12 0.003572 0.569 III L-LAM72 K-GFK68 TAF1 S-NBH53 0.003591 0.567 IV TAF2 K-GFK68 P-LASF50 N-KZFS11 0.003808 0.577 V L-LAH83 K-GFK68 NBF1 KZFS12 0.003958 0.555 VI TAFD30 K-GFK68 P-LASF50 N-KZFS11 0.003970 0.559 VII P-LASF51 K-GFK68 S-LAH58 N-KZFS11 0.003978 0.553

Table 3.7: Optical prescription of the 400-1000nm F/1.7 f=12mm monocentric lens near global solution Radius Thickness Glass Semi-diameter OBJ Infinity Infinity Infinity 1 7.05656 4.20075 P-LASF50 6.75008 2 2.85581 2.85581 K-GFK68 2.83814 STO Infinity 2.85581 K-GFK68 2.20709 4 -2.85581 1.30890 TAF1 2.82507 5 -4.16471 0.42696 4.00744 6 -4.59167 3.54357 E-ADF10 4.35045 7 -8.13525 3.84990 7.34868 IMA -11.98514 10.37980

TS 0.00 (deg)

P-LASF50 K-GFK68 TAF1 E-ADF10

Modulus of the OTF the of Modulus 1cm Hammer RMS/Airy disc = 114% 5D RMS/Airy disc = 125% 0 0 40 80 120 160 200 [lp/mm]

Figure 3.13: MTF curves for 12mm, F/1.7, 400-1000nm lenses obtained through seeded Hammer search and near global 5-dimensional optimization (shown on layout).

100 [%] 80

Back illuminated CCD 60

20 Front illuminated CCD Relative Quantum Efficiency 0 200 400 600 800 1000 1200 Wavelength [nm]

Figure 3.14: Spectral response of front and back-illuminated silicon sensor

Our near global 5D search improved the MTF performance at 200lp/mm over the seeded Hammer solution by a 16% margin. Figure 3.14 shows graphs of sensitivities of the front and back-illuminated silicon sensors [55].

Our next goal was to modify the 400-1000nm 4GA-8 solution from Table 3.3 to operate with front-illuminated silicon sensor. We constructed ZEMAX merit function as a function keeping at minimum radii of point spread functions at all nine wavelengths

(operators REAR), maximizing MTF at frequencies 100, 160 and 200 lp/mm and keeping the focal length at 12 mm (operator EFFL). Substitution of the wavelength weights for the front-illuminated silicon sensor and quick reoptimization in ZEMAX gave the optical prescription shown in Table 3.8.

Table 3.8: Optical prescription of the 400-1000nm F/1.7 f=12mm monocentric lens operating with front-illuminated silicon sensor Radius Thickness Glass Semi-diameter OBJ Infinity Infinity Infinity 1 7.04107 4.08409 P-LASF47 6.73508 2 2.95699 2.95699 K-GFK68 2.93476 STO Infinity 2.95699 K-GFK68 2.21103 4 -2.95699 1.26003 K-LASFN6 2.91758 5 -4.21702 0.35134 4.05282 6 -4.56835 3.30232 N-KZFS11 4.33494 7 -7.87067 4.12587 7.14073 IMA -11.99654 10.38982

The image quality is practically diffraction limited. The MTF of the lens is shown in the Figure 3.15. At the 200 lp/mm the lens has 90% level of diffraction limited resolution. Back-illuminated silicon sensors are sensitive to as low as 200 nm

80 wavelength. We found that achromatization in 200-1050nm waveband is out of ability of the 4GA-8 architecture at this scale and aperture.

TS 0.00 (deg) TS 60.00 (deg) TS 30.00 (deg) 1

P-LASF47 K-GFK68 K-LaSFn6 N-KZFS11 Modulus of the OTF the of Modulus 1cm RMS spot/Airy disc = 108% 0 0 40 80 120 160 200 [lp/mm]

Figure 3.15: MTF curves for 12mm, F/1.7 lens operating with 400-1000 nm front-illuminated silicon sensor sensitivity spectrum

In order to avoid the use of expensive coatings we decided to cut off the UV spectrum by using the mounting meniscus made from the Schott GG435 absorptive glass.

Using an additional mounting meniscus at the curved image surface is optional and has little impact on imaging system performance.

The optical prescription of the monocentric lens operating with the back- illuminated silicon sensor is shown in the Table 3.9. GG435 color glass refraction indices were measured with Filmetrix F10-RT refractometer. The MTF and the layout of the lens are shown in the Figure 3.16. The image quality is close to diffraction limited.

Table 3.9: Optical prescription of the 435-1000nm F/1.7 f=12mm monocentric lens operating with back-illuminated silicon sensor Radius Thickness Glass Semi-diameter OBJ Infinity Infinity Infinity 1 6.92285 4.01591 NBFD11 6.62605 2 2.90694 2.90694 K-GFK68 2.88753 STO Infinity 2.90694 K-GFK68 2.21436 4 -2.90694 1.27374 K-LASFN17 2.87138 5 -4.18068 0.38453 4.02581 6 -4.56521 2.27558 KZFS12 4.33527 7 -6.84078 3.13995 6.28471 8 -9.98074 2.00000 GG435 8.75173 IMA -11.98074 10.37606

Both lenses have the core glass K-GFK68 with a very high coefficient of thermal expansion, TCE=12.9, while surrounding glasses have low TCE coefficients. For example, the front-illuminated silicon sensor lens shown in Table 3.8: P-LASF47 glass has TCE= 6.04 and K-LASFN6 glass has TCE= 5.9. Normally the TCE difference less than 1.5 for cemented surfaces can be recommended for outdoor optics [56]. Recently

Norland Products Inc. offered extremely low psi modulus NOA 76 [57] optical cement, which can be used for glass pairs with such high CTE differences.

ZEMAX thermal modeling of the schemes shown in Table 3.8 and Table 3.9 with a 10µm thick layer of NOA 76 optical cement for differential thermal expansion shows that the lenses can operate in a wide temperature range of -20 C to +50 C without image quality degradation.

TS 0.00 (deg) TS 60.00 (deg) TS 30.00 (deg) 1

NBFD11 K-GFK68 K-LaSFn17 KZFS12 GG435 Modulus of the OTF the of Modulus 1cm RMS spot/Airy disc = 129% 0 0 40 80 120 160 200 [lp/mm]

Figure 3.16: MTF curves for 12mm, F/1.7 lens operating with 435-1000nm back-illuminated silicon sensor sensitivity spectrum

Only a 0.02 mm back focal length adjustment is required. Since monocentric lenses were originally designed to be used with refocusing [44], this procedure does not pose a problem.

Finally, if we ask ourselves why the 4GA-8 architecture has such capabilities for achieving the high performance monocentric designs, the answer lays in the asymmetry and the presence of an air-gap. To support that claim, in Figure 3.17 longitudinal aberrations over the pupil for 3GS and 4GA-8 architectures are shown. Symmetric designs like 2GS and 3GS always have the shape of the longitudinal aberrations as in

Figure 3.17(a).

3GS 4GA-8 mm 50 . 3 Pupil radius (a) (b)

-50 0 50 -50 0 50 0.40 0.55 0.70 0.85 1.00 0.40 0.55 0.70 0.85 1.00 0.48 0.62 0.78 0.92 [µm] 0.48 0.62 0.78 0.92 [µm] Longitudinal Aberration Longitudinal Aberration

Figure 3.17: Longitudinal aberrations of the (a) top 3GS and (b) top 4GA-8 architecture 12mm F/1.7 monocentric lenses for 400-1000nm spectral band.

This single bending curve, as the aperture grows, cannot be controlled, so the spherochromatism and zonal spherical aberration become dominant and limit the performance. On the other hand, presence of an air gap and broken symmetry help correcting those aberrations and adds an additional bending to the curve as in Figure

3.17(b).

3.4 Lens complexity and performance tradeoff

To explore the maximum achievable performance of the monocentric lens geometries, we began with the design constraint of a 12mm focal length 120° FOV imager and the visible waveband of 486-656nm and optimized the lens design to increase the aperture as much as possible, subject to a pre-defined performance metric. This performance constraint was to require that the MTF was at least 70% of the diffraction limit at 200 lp/mm (the highest spatial frequency needed for Nyquist sampling of a 2.5

84 micron pitch Schott 24AS optical fiber faceplate) while simultaneously requiring that the

RMS spot size radius had to be less than 1.5x the Airy disc radius (which maintained

MTF at lower spatial frequencies). While this metric is somewhat arbitrary, the trends of the results are indicative of a wide range of related performance metrics, as applied to the available degrees of freedom. The results are shown in Figure 3.18.

1 1

F/# 3GS 2GS 2GAS 3GA-6 4GA-8 3GA-7

2 2

2GS 2GAS 3GS 3GA-6 4GA-8 3GA-7 3 3

4 4 1GS 1G-2 5 5

6 6 f=12mm 7 7 8 8 9 1GS 1G-2 f=112mm 9 10 10

1 2 3 4 5 6 7 8 Degrees of freedom

Figure 3.18: Monocentric lens geometries optimization behavior for two different scales operating in 486-656nm spectral range

Ill chosen geometries, labeled in blue in Figure 3.18. (1G-2, 2GAS, 3GA-6), converge to simpler ones and do not enable an increase in aperture size. The geometries

85 labeled in black (1GS, 2GS, 3GS) show results obtained by our global optimization algorithm, while geometries labeled in red (3GA-7, 4GA8) are the result of the seeded

Hammer optimization, starting from top 2GS candidates as seed designs.

Adding the second glass is helpful in controlling the chromatic aberration, so there is a large increase in achievable F-number in moving from geometry 1GS to 2GS.

This is equivalent to going from a singlet lens to a cemented doublet achromat. Adding the third glass in the 3GS geometry gives marginal chromatic aberrations improvement over 2GS, whereas the other degrees of freedom (specifically, 2GAS with 4 DOF, and

3GA-6 with 6 DOF) provide no improvement).

Breaking the symmetry and introducing an air gap with the 7 DOF and 8 DOF architectures allows us to further increase the aperture, and still meet the desired MTF performance. Similar behavior is observed for the longer 112mm focal length lens, which is also shown in Figure 3.18.

The final step was to explore the design space for 4 different focal length scales

{f=12mm (the SCENICC program lens scale), 40mm, 70mm (AWARE2 program [10]) and 112mm (AWARE10 program [58])}, and at each scale look at the maximum aperture for visible, extended visible and visible-NIR spectral wavebands, subject to the MTF performance constraint described above. As before, we used the global optimization for the 2-glass and 3-glass symmetric systems (2GS, 3GS). For the 3 and 4-glass air gap candidates (3GA-7, 4GA-8), the combination of our systematic 5th dimensional optimizer with ZEMAX hammer optimization was used. The entire set of results is summarized in Figure 3.19.

Asymmetric 4-glass air-gap lens (4GA-8) Specific results of Hammer optimization

Asymmetric 3-glass air-gap lens (3GA-7) Specific results of Hammer optimization

Symmetric 3-glass lens structure (3GS) Optimal solutions found by global search

Symmetric 2-glass lens structure (2GS)

Optimal solutions found by global search Increasing lens → Increasing aperture lens

*AWARE prototype objectives alone (not including relay optics) were not designed to satisfy the MTF performance constraint.

Figure 3.19: Monocentric objective lens performance tradeoff for different scales and three spectral bands (486-656nm, 435-850nm, 400-1000nm)

The clear trend is maintained over all three spectral bands and it shows the necessity of lens structure complexity increase if we wish to push the scale and/or the aperture size. For the time being, area above pink 4GA-8 line remains terra incognita where the complexity and manufacturing cost increase outweighs the performance improvements. One exception to that rule is 5GA-10 architecture, which can be quite useful for medium scale (>100mm) and large scale designs, and will be demonstrated in an example in Chapter 3.5.4. 2GS global search still remains a powerful tool in monocentric lens design, both for supplying the final or seed designs for the more complex architectures. While the 5-dimensional near global search gives substantially

87 better results over the 2GS seeded Hammer search, completing it on 32 CPU cores at this point is very time consuming and should be used only as a final attempt to squeeze out the maximum performance capability from the lens before manufacturing. Implementing the code for the 5D near global search on GPUs would possibly reduce the search time by a factor of hundred cutting it down from weeks to hours.

3.5 Specific lens design examples

3.5.1 Water-immersed lens (f=12mm, F/1.79, 380-550nm)

Water in natural reservoirs is highly scattering and lossy due to suspended particles, necessitating large NA optics for imaging at appreciable sub-surface distances.

The seawater transmission window is 300 - 600 nm, compatible with back-illuminated silicon sensors (Figure 3.14). Underwater optics may also need to resist high pressure on the front lens. The large aperture of classical wide field underwater optics [1,59] must be protected by a window or dome, resulting in large weight and bulk.

Underwater objectives like the Gidrorussar model 11 (f=23mm) and 12

(f=11.9mm), which can achieve F/2 with a 40°-60° overall field, have a thick cemented doublet as a front lens [59] and combine the Topogon structure with a Petzval projection lens for distortion compensation and image flattening over the 430 - 656 nm spectrum.

A 9mm focal length F/1.79 monocentric underwater lens was optimized for the

0.38 to 0.55 micron spectral band. Table 3.10 shows the optical prescription of the four- glass with air gap (4GA-8) lens. The front lens is a cemented solid ball, which can support high pressure. The MTF of the underwater monocentric lens, shown in the Figure

3.20, achieves 70% of the diffraction limit over a 120˚ full field of view.

Table 3.10: Optical prescription of the water immersed monocentric lens. Radius Thickness Glass Semi-diameter OBJ Infinity Infinity SEAWATER Infinity 1 8.30000 2.00000 SEAWATER 7.61630 2 6.03999 3.48360 TAFD30 5.71884 3 2.55639 2.55639 N-LAF21 2.53267 STO Infinity 2.55639 N-LAF21 1.87234 5 -2.55639 1.45793 LASF35 2.52255 6 -4.01433 0.15699 3.84362 7 -4.17132 2.61177 K-GIR79 3.96995 8 -6.78309 5.26472 6.24176 IMA -12.04781 10.43447

TS 0.00 (deg) TS 60.00 (deg) TS 30.00 (deg) 1

TAFD30 N-LAF21 LASF35 K-GIR79

Modulus of the OTF the of Modulus Seawater 1cm

RMS spot/Airy disc = 154% 0 0 40 80 120 160 200 [lp/mm]

Figure 3.20: Underwater monocentric lens (9mm/12mm image/object space focal length, F/1.79, 380-550nm)

3.5.2 Night vision lens (f=16mm, F/1.2, 500-900nm)

Typical night vision goggles use image intensifiers operating from 700 to 900 nm, and a lightweight (~40g) 25mm focal length F/1.2 objective lens covering a 40° field of view [60].

Table 3.11: Optical prescription of the Night Vision monocentric lens.

Radius Thickness Glass Semi-diameter

OBJ Infinity Infinity Infinity

1 12.21344 7.19969 K-PSFN173 11.72988

2 5.01376 5.01376 K-GIR79 4.97624

STO Infinity 5.01376 K-GIR79 3.67166

4 -5.01376 2.76625 K-PSFN173 4.92548

5 -7.78001 0.26457 7.36222

6 -8.04458 5.49005 KZFS12 7.55792

7 -13.53463 2.43761 12.03582

IMA -15.97224 13.83509

We found it possible to design a 120° wide field of view monocentric lens to satisfy these optical specifications, but the weight was 170g. Scaling the lens to a 16 mm focal length matched the 40g target weight. The resulting optical prescription is shown in

Table 3.11. The F/1.2 lens operates from 500 - 900 nm, which spans the entire spectral band of the Gen III GaAsP photocathode. The lens resolves 100 lp/mm (Figure 3.21), well over the 70 lp/mm supported by image intensifiers. While the resulting imager would provide some 50% lower angular resolution, it could operate over a 120° wide field, triple that of standard night vision optics.

TS 0.00 (deg) TS 60.00 (deg) TS 30.00 (deg) 1

K-PSFn173 K-GIR79 K-PSFn173 KZFS12 Modulus of the OTF the of Modulus 1cm RMS spot/Airy disc = 253% 0 0 20 40 60 80 100 [lp/mm]

Figure 3.21: Gen III Night Vision monocentric lens (16mm focal length, F/1.2, 500-900nm).

3.5.3 Short-wave IR lens (f=12mm, F/1.19, 900-1500mm)

Skylight glow in the SWIR band provides a reliable light source for night imaging even in the absence of starlight [61]. This light can be sensed using uncooled InGaAs focal planes such as those available from Sensors Unlimited Corp. [62]. A 2GS seeded

Hammer search produced the 12 mm focal length F/1.19 lens prescription shown in Table

3.12, with MTF and lens layout as shown in the Figure 3.22.

The lens provides nearly diffraction-limited resolution at 200 lp/mm, compared to the 33 lp/mm Nyquist frequency of the 15µm pixels in the 1280x1024 Sensors Unlimited focal plane, and so monocentric optics will support advanced imaging even for the next generation SWIR sensors. The 15g weight of this lens compares favorably with the existing wearable night vision optics.

Table 3.12: Optical prescription of the short-wave infrared (SWIR) monocentric lens. Radius Thickness Glass Semi-diameter OBJ Infinity Infinity Infinity 1 9.64853 6.08158 M-FDS2 9.22685 2 3.56695 3.56695 K-GIR79 3.55045 STO Infinity 3.56695 K-GIR79 2.81170 4 -3.56695 2.13160 K-PSFN215 3.52510 5 -5.69855 0.20974 5.40924 6 -5.90829 4.27312 N-SF4 5.56372 7 -10.18141 1.79011 9.05080 IMA -11.97152 10.36887

TS 0.00 (deg) TS 60.00 (deg) TS 30.00 (deg) 1

M-FDS2 K-GIR79 K-PSFn215 N-SF4 Modulus of the OTF the of Modulus 1cm RMS spot/Airy disc = 153% 0 0 40 80 120 160 200 [lp/mm]

Figure 3.22: SWIR monocentric lens (12mm focal length, F/1.19, 900-1500nm)

3.5.4 Medium scale lens (f=112mm, F/2.33, 486-656nm)

A 10GPixel monocentric multi-scale imager prototype was designed and built for the AWARE program [58]. The system was a 2GS monocentric objective lens of F2 and

S-BSL7 glass, designed to work in combination with the secondary relay optics. While the objective lens itself was not diffraction limited (with RMS spot size 11 times larger than the Airy disc) the whole system was designed to be diffraction-limited. We identified better stand-alone objective designs in the 2GS and 3GS architecture, but these lenses are not easily fabricated due to size limitations of available glass slabs. Size restrictions may provide a hard constraint on the materials choice for the outer shells in large monocentric lenses.

Table 3.13: Optical prescription of the f=112mm monocentric lens candidate. Radius Thickness Glass Semi-diameter OBJ Infinity Infinity Infinity 1 70.6495 25.0385 F2 66.0262 2 45.6110 20.0201 N-LASF31A 43.4394 3 25.5910 25.5910 K-GFK68 25.0475 STO Infinity 25.5910 K-GFK68 15.0900 5 -25.5910 11.0541 N-LASF31A 24.8256 6 -36.6451 5.7785 34.6924 7 -42.4236 34.4507 N-BK7 39.4128 8 -76.8743 35.0734 68.5323 IMA -111.9476 96.9500

Under this constraint, the 4GA-8 monocentric architecture was unable to achieve a diffraction limited f=112mm objective, so we used a 5GA-10 architecture, choosing N-

BK7 and F2 glasses for the outer shells as they can be melted in blanks up to 500mm thick [63]. Such lenses are more expensive to fabricate than the two-glass objective, but

93 they are compatible with fiber-optic image transfer. Prescription of the lens is shown in

Table 3.13, and layout with MTF performance in Figure 3.23. Similar designs can be obtained using Ohara S-FPM2 glass instead of Sumita K-GFK68 glass for the core material.

TS 0.00 (deg) TS 60.00 (deg) TS 30.00 (deg) 1

N-LASF31A K-GFK68 F2 N-LASF31A N-BK7 Modulus of the OTF the of Modulus

100mm RMS spot/Airy disc = 98% 0 0 40 80 120 160 200 [lp/mm]

Figure 3.23: Medium scale monocentric lens 5GA-10 diffraction limited candidate possible to fabricate (112mm focal length, F/2.33, 486-656nm)

3.5.5 Large scale lens (f=280mm, F/2.8, 450-700nm)

In 2010, Brady and Marks [49] presented a 5GA-10 architecture monocentric lens design capable of up to 40 Gigapixels overall resolution. In the journal publication that followed [64], the group used a statistical approach to conduct a global search for this lens using the Schott glass catalog and a specific merit function.

Table 3.14: Optical prescription of the optimal 40GPix monocentric 4GA-8 lens Radius Thickness Glass Semi-diameter OBJ Infinity Infinity Infinity 1 122.6509 65.5070 N-LAK33B 116.1051 2 57.1440 57.1440 CAF2 56.0001 STO Infinity 57.1440 CAF2 34.8902 4 -57.1440 16.2656 N-LAK8 55.6673 5 -73.4095 11.4827 70.3637 6 -84.8922 31.3242 N-LAF34 80.0069 7 -116.2164 163.6565 107.5578 IMA -279.8730 242.3780

TS 0.00 (deg) 1 4GA-8 RMS/Airy disc = 188% 5GA-10 RMS/Airy disc = 185%

4GA-8

5GA-10

280mm Modulus of the OTF the of Modulus

0 0 160 320 480 640 800 [lp/mm]

Figure 3.24: Polychromatic MTF comparison of previously reported 5GA-10 optimal solution for the Gigagon 40GPixel lens and a simpler 4GA-8 solution (280mm focal length, F/2.8, 450- 700nm)

We performed our search using the merit function of our 5D near-global optimizer using the same pool of glasses (Schott catalog, as of October 2010) and

95 obtained the 4GA-8 geometry solution shown in Figure 3.24 and Table 3.14. The resulting 4GA-8 lens offers comparable size and weight, and slightly better MTF performance, than the 5GA-10 lens. It is necessary to note, however, that both lenses would be difficult to fabricate due to the physical thickness of the glass elements required.

3.6 Chapter summary

This chapter describes an investigation of wide field of view monocentric lenses using architectures with complexity ranging from a simple glass ball to moderately complex structures with up to 10 degrees of design freedom following the focal length constraint. A 2-glass symmetric lens structure works well for applications with a moderate spectral range, focal length, and numerical aperture. However, for applications that substantially increase one or more of these specifications, we show that the best performance in a moderate complexity lens is achieved with four-glass structures with an air gap between meniscus elements behind a spherical glass core (the 4GA-8 architecture).

To help identify the best specific designs for such 4GA-8 lenses, we presented a systematic optimization method derived from the global optimization of a 2-glass lens, and demonstrate its capability for several case studies. We conclude that the general class of monocentric objective lenses offers practical high-performance options for a variety of wide-angle imaging systems.

Chapter 3, in full, is a reprint of the material as it appears in “Optimization of high-performance monocentric lenses,” by I. Stamenov, I. Agurok and J. Ford, Applied

Optics, Vol. 52, No. 34, 2013. The dissertation author was the primary investigator and author of this paper.

4. Panoramic monocentric imaging using fiber-coupled focal planes

4.1 Introduction

In this Chapter 4 we show the design, construction and operation of a 12mm focal length wide-aperture fiber-coupled monocentric imager that acquires a 30Mpixel panoramic image with a 126˚ x 16° field of view and compare its performance to a conventional wide angle DSLR camera. Chapter 4.2 describes the design and stand-alone characterization of the prototype monocentric objective lens, and describes the integration of 1.75µm pitch back-side illuminated CMOS focal planes with shaped fiber bundles. Chapter 4.3 contains characterization of the fiber-coupled image transfer, comparing resolution to a full size "benchmark" wide-angle lens imaging onto the same

CMOS sensors. Chapter 4.4 describes the integration of 6 fiber-coupled sensors with the 97 focus optomechanics and the resulting panoramic images, comparing performance to a conventional DSLR imager. Chapter 4.5 contains our conclusions.

4.2 Fiber-coupled imager components

4.2.1 Monocentric lens design and fabrication

We have investigated methods for identifying optimal two-glass symmetric (2GS) monocentric objective lenses [44] as well as more complex monocentric structures [] that

97 98 reduce higher order aberrations. The optimal solution for a 12mm F/1.7 470-650nm lens is a S-LAH79 glass shell (nd=2.003) and a spherical glass core of K-LaSFn9 (or equivalently, S-LAH59 or TAF5, nd=1.816). This lens requires a fixed aperture stop at the center of symmetry. An aperture inside a ball lens may be introduced by directly cutting the aperture into a spherical glass element [22], but this is not a lens fabrication technique available from our preferred lens fabricator. The aperture can also be introduced by cementing two hemispherical elements, where one has been fabricated with an aperture stop. Light will be incident at large angles on the interface between the two high-index hemispheres, so the index of the adhesive must be sufficiently high to limit internal reflection.

Using the highest index Norland UV-cure epoxy (NOA164 with index of 1.64) to bond the K-LaSFn9 glass of index 1.81 introduces a total internal reflection at a 65° ray angle incidence at the ball center, which limits the achievable field of view of a F/1.7 lens to ±55°. Therefore, we preferred a lower index core glass. A suitable replacement solution from the global list of ranked candidates had a S-LAL13 core glass material

(n=1.69) [44,65]. We added a fused silica meniscus at the focal surface for the fiber bundle mounting purposes and re-optimized the solution (Table 4.1). Norland 61 optical cement (n=1.56) was used. The fused silica meniscus does not have a significant impact on the lens optical performance, but it facilitates the alignment and mounting of multiple fiber bundles to meet the shared spherical image surface. Layout of the optimized lens and the Modulation transfer function (MTF) performance curves obtained with Zemax lens design software are shown in Figure 4.1. Three monocentric lens prototypes were fabricated. The design tolerances of the lens were defined using the highest of three

99 standard options offered by the lens fabricator, Optimax Systems. The surface radii were accurate to 0.1% of the radii (2 fringes at the surfaces used), with 20µm for lens element decentration, and 25µm for center element thickness. Of these, the center thickness tolerance was the most sensitive parameter. One of the lenses was fabricated with the maximum practical F/1.0 aperture, intended for demonstrations of high light collection imaging. This first prototype imager was intended to demonstrate the light collection capabilities of monocentric lenses. The two remaining lenses were fabricated with an aperture stop at F/1.35, wider than the highest resolution F/1.7 aperture.

Table 4.1: Optical prescription of the 470-650nm F/1.7 f=12mm 120° 2GS MC lens prototype

Radius Thickness Glass Semi-diameter OBJ Infinity Infinity Infinity 1 7.15000 3.54539 S-LAH79 7.15000 2 3.59900 0.01000 NOA61 3.59900 3 3.58900 3.58900 S-LAL13 3.58900 4 Infinity 0.00500 NOA61 2.41614 STO Infinity 0.00500 NOA61 2.39185 6 Infinity 3.58900 S-LAL13 2.38000 7 -3.58900 0.01000 NOA61 3.50000 8 -3.59900 3.54539 S-LAH79 3.51000 9 -7.15000 2.48990 6.60000 10 -9.63500 2.40000 F_SILICA 8.80000 IMA -12.03400 0.00000 11.00000

Figure 4.1 shows the F/1.7 design MTF and photographs of the fabricated F/1.35 lens, next to a penny to illustrate the size of the 2-glass monocentric core (which provides image formation) and the structural mounting meniscus. The bottom row shows the impact on resolution when the aperture size grows to increase light collection. As

100 discussed in the Zemax v.13 manual (August 2014, p. 149), the FFT MTF calculation is based on scalar diffraction theory, and "The vectorial nature of the light is not accounted for. This is significant in systems that are very fast, around F/1.5 (in air) or faster."

Accordingly, the MTF curves shown in Fig. 2 were calculated using the Zemax FFT

MTF function for F/1.7 lens and Huygens MTF function for the F/1.35 and F/1.0 cases.

4.2.2 Characterization of monocentric lens image formation and focus

Monocentric lenses designed for optimal performance at infinite object distance can be focused like any other lens by means of axial translation [44], where the object field is a flat surface perpendicular to the optical axis. To confirm that experimentally, the monocentric lens was mounted on a 5-axis translation stage with one axis aligned to the optical system axis for focus adjustment. A custom-built boresight optical aligner was used to center the ball lens and the meniscus. The curved image plane was sensed with three identical relay-imaging systems, which allowed us to simultaneously record, with high spatial resolution, the center and the extreme ±60˚ field positions.

The relay imaging systems used Mitutoyo M Plan Apo 20X long working distance apochromatic objectives with 0.42 NA, sufficient to test the monocentric lenses down to a F/1.2 wide aperture. Standard InfinitubeTM 200mm lens tubes were attached to

C-mount NIR Thorlabs DCC3240N cameras (1280x1024, 5.3µm pixel pitch sensor,

0.265µm equivalent after 20X magnification) to record the monochrome image. The resolution of the relay imaging system was under 1.7µm, as compared to the 2.6µm RMS spot diameter of the monocentric objective. So the overall system MTF including relay imaging is reasonably indicative of the MTF of the monocentric objective lens.

TS D iff. Li mi t TS 60.00 (de g) TS 0.00 (de g) TS 30.00 (de g) As#fabricated#lens:## 1.0 f=12mm,#F/1.35,# 0.9 F/1.7 470=650nm# 0.8 0.7 S-LAH79 0.6 S-LAL13 0.5 Penny,#below,#is#at#same#scale#as#photo# 0.4 0.3 0.2 Modulus of the OTF 0.1 F_SILICA 1cm RMS spot/Airy disc = 227% 0.0 0 80 160 240 320 400 Spatial Frequency [lp/mm]

TS 0.00 (de g) TS 60.00 (de g) TS 0.00 (de g) TS 60.00 (de g) TS 30.00 (de g) TS 30.00 (de g) 1.0 1.0 0.9 F/1.35 0.9 F/1.0 0.8 0.8 0.7 0.7 0.6 0.6 0.5 0.5 0.4 0.4 0.3 0.3 0.2 0.2 Modulus of the OTF 0.1 0.1 RMS spot/Airy disc = 3486% RMS spot/Airy disc = 39550% 0.0 0.0 0 80 160 240 320 400 0 80 160 240 320 400 Spatial Frequency [lp/mm] Spatial Frequency [lp/mm]

Figure 4.1: Two-glass symmetric (2GS) f=12mm 470-650nm F/1.7 monocentric lens design layout (top row) with MTF performance curves, and a photograph of the fabricated F/1.35 prototype. Increasing aperture (bottom row) improves light collection at the expense of a spot size and MTF performance. In top right corner of the MTF graphs the lens entrance pupil is shown, as observed from the front. The object is at

infinity. Moving to 1m and refocusing does not significantly change the curves. 101

102

Three chrome-on-glass USAF targets were used as test objects at 0.5m and 1.0m away for three different field angles (Figure 4.2). Microscope objectives in the image relay arms were kept at fixed focus on the meniscus surface, while the ball lens was moved along the optical axis providing the focusing function to the monocentric lens.

Images sensed by an enhanced NIR sensor in the DCC3240N Thorlabs cameras were acquired as raw 8-bit monochrome bitmaps. The white background behind the targets was illuminated by a visible 4200˚K spectrum LED light source. Lens MTF curves were calculated using Imatest "Master" software. We used MTF50 as a metric of resolution, the spatial frequency where the transferred MTF has a 50% contrast. The lens showed an on-axis MTF50 performance of approximately 180-190 lp/mm. The reduction in resolution from the expected design performance was a result of the fabricated lens having a wider aperture (F/1.35 instead of F/1.7). The MTF curves for 1m object distance in Figure 4.2 are well matched to Zemax generated curves in Figure 4.1 except for tangential performance at ±60° where the experiment shows a better result. Focusing of the lens was also confirmed for effectively infinite object conjugates by taking the assembly outdoors for testing at more than 30m ranges. While monocentric lenses are unusual in some aspects, as for planar focal surface lenses, the image depth of field is governed by the operating F/# (or numerical aperture) and field incidence angle (i.e. the chief ray) at the image-surface. The depth of field of the F/1.35 monocentric lens was similar to a more conventional F/1.4 lens with a planar focal surface. The monocentric lens is not telecentric, and so a defocused off-axis blur function (the bokeh) is laterally shifted. However, operating the lens outdoors in sunlight required the use of an external

UV-IR blocking filter, as the lens was optimized for the visible spectrum.

Figure 4.2: Experimental confirmation of monocentric lens focusing capability. Axial translation of the center ball lens maintains a fixed radius hemispherical image of a flat object in focus (0.5m and 1.0m conjugates shown). Three apochromatic microscope objectives were used 103 for simultaneous relay imaging at three different field angles.

104

The measured travel range for the ball lens required to focus to the wide flat object from infinity to 0.5m was 240µm. This was the minimum travel range required for the prototype’s optomechanical focusing mechanism.

4.2.3 Fiber bundle and CMOS image sensor integration

The monocentric straight fiber bundle configuration (Figure 1.3(a)) is limited by a

± 30° field of view [20,22]. To retain uniform illumination, we choose to use a row of six fiber bundles, each covering a 21° horizontal and 16° vertical FOV (26.2˚ diagonal), for an overall 126° by 16˚ (126.1° diagonal) field of view. The vertical field of view was constrained by the 4:3 factor of the selected image sensor, an Omnivision 5653 5Mpixel back-side illuminated (BSI) CMOS image sensor, with 1.75 µm pitch pixels covered by microlenses and a Bayer RGB color filter. Each sensor is interfaced through a rigi-flex printed circuit board (PCB) that provides a USB-2.0 interface to a host computer for either streaming video or full-resolution capture. The OV5653 image sensor was intended for high production volume compact imagers such as smartphone cameras, and so was available only in a chip-scale surface-mount ball grid array package. The package has a

400µm thick glass cover attached directly to the silicon chip by a thin patterned epoxy grid. This cover had to be removed to allow direct contact between the fiber bundle and image sensor. For removal of the cover glass, we placed the PCB-mounted sensors into a temporary fixture for accurate leveling, then used a computer-controlled dicing saw to cut through the 400µm thick glass layer and stop within the 25µm adhesive (Figure 4.3(a)).

After the cover glass was removed, it was essential to completely clean the surface from

105 any residual debris, especially glass fragments, that might otherwise introduce a gap between the fiber bundle and image sensor.

The fiber bundles were made of Schott 24AS fiber optic material with a 2.5µm pitch between 5-sided optical fibers arranged into a 4-fiber unit cell with a diamond- shaped absorptive center [14]. The fiber bundle input faces were ground and polished to match the 12.043mm radius of curvature of the mounting meniscus, with an additional

20µm allocated for adhesive thickness. The output faces were polished to form a flat pedestal (Figure 4.3(c)) to fit into the recess formed after the cover glass was removed from the OV5653 sensor. The pedestal was cut to cover slightly more than the

Omnivision sensor (4.592 x 3.423 mm) active area and to avoid mechanical interference around it. Assembling six adjacent fiber bundles can provide seamless image stitching across the full field, but only if the fiber bundles are precisely shaped to align the input image with the sensor pixels, and have a sharp edge transition between the spherical input face and angled sidewalls. The first set of shaped bundles were made with a lateral misalignment of the input image area, such that for several of the bundles, a vertical section of image ranging from 9 to 85 pixels in width was not sensed.

(a)+ (b)+ (c)+ (d)+

400%µm%% cover%glass:%% Dicing&saw&

25%µm%glue% 200%µm%% CMOS%chip%

Chip%Scale+Packaged+Focal+Plane+ Decapped+OV5653+BSI+sensor+ Cut+fiber+bundle+ Cut+fiber+bundle+ Glass%cover%on%CMOS%sensors% 1.75%µm%pixels%with%lenslets%and% Output%face%pedestal% Curved%input%face% Bayer%color%filters%%

Figure 4.3: Commercial focal plane integration with the fiber bundle. (a) cross section of OV5653 wafer-fabricated CMOS sensor, (b) sensor with cover glass removed; (c, and d) shaped glass fiber bundle output and input faces, ready to be attached to CMOS sensor and meniscus lens.

106

This fabrication error was corrected for a second-generation prototype that also incorporates different optics and focus actuation. This prototype is currently being characterized, and will be reported separately. The flat polished pedestal side of each fiber bundle was placed in contact with a clean decapped sensor using Norland NOA72

UV curable optical adhesive, and the pedestal reinforced using an additional layer of the same flexible epoxy. The impulse response of all sensors was characterized at the center and edges to confirm that the sensor assembly did not incorporate any particles that would lead to nonuniform response. Cross sectioning and optical and electron microscope images of one of the working fiber-coupled sensors revealed the epoxy thickness to be under 2µm across the sensor aperture (Figure 4.4). The CMOS sensor micro lens and color filter add an additional 3µm thickness between the fiber bundle and active sensor surface.

Spherical& 2.4um&ﬁber& input&face& bundle&cores& Epoxy+2+µm+ Lenslet+RGB+ Fiber& ﬁlter+~3µm+ bundle& BSI+CMOS+~8µm+

Circuit%board%

CMOS%Sensor% 1000um& 10um&

Figure 4.4: Assembled fiber-coupled OV5653 sensor, which was then cross-sectioned by diamond saw to inspect adhesive thickness and uniformity. Optical micrograph (center) and SEM measurement (right) of the fiber bundle/sensor interface reveals the optical adhesive layer was less than 2µm thick.

All six fiber-coupled sensors were found capable of transmitting an incident focused signal through a single fiber core and coupling the signal to a 3x3 sensor pixel area, consistent with the relative areas of the fiber core and image sensor pixel, and the total 5µm thick volume of isotropic index media between the sensor and fiber bundle.

107

The impulse response of the fiber-coupled sensor is locally space-variant due to the fiber core structure, and the sensed image is impacted by Moiré effects from sampling of the quasi-periodic fiber bundle by the CMOS sensor pixels.

(a)$ (b)$

2.6μm&spot&size&& 3x1.75μm& diameter& &pixels& (3.8μm&including&weak&halo)&

RGB&interpolated&

Monocentric$$ Keyence$VHX1000$ Monocentric$$ Fiber:coupled$ lens$ microscope$ lens$ sensor$

Figure 4.5: White light impulse response of the two-glass monocentric (a) lens only, measured with a Keyence VHX 1000 microscope and (b) the lens with the fiber-coupled Omnivision OV5653 1.75µm sensor attached to it.

4.3 Narrow field (single sensor) fiber-coupled image transfer

4.3.1 Fiber-coupled image transfer performance

To observe the effect of each step of the fiber-coupled image sensing, we bench- mounted the F/1.35 two-glass monocentric lens so that a central region of the spherical image could be recorded by a Keyence VHX1000 digital microscope, and then the same scene recorded with a fiber-coupled CMOS sensor. We measured the impulse response using a white light LED source, and found that the impulse response of the lens alone matched the PSF spot size diameter predicted by Zemax OpticStudio (1st dark ring

108 diameter = 2.6µm) as shown in Figure 4.5(a). With the fiber-coupled sensor in oil contact with the monocentric lens we observe local space variance of the impulse response due to the varying relative position of the quasiperiodic 2.5µm pitch fiber core relative to the

1.75µm pitch RGB color filtered sensor pixels. A detailed characterization of the space- variant impulse response of the fiber-coupled image sensor has been reported separately

[66]. As we move the white spot around the field of view, it subsequently hits the different areas of the fiber bundle surface (individual cores or cladding between them), resulting in light containment that is always within a 4x4 to 3x3 pixel area (Figure

4.5(b)). Note that the brightness of the sensed image in Figure 4.5(b) has been increased to make visible the relatively low signal in the adjacent pixels.

Next, we looked at the effect of fiber transfer on a color image. The 2GS monocentric lens itself achieves an on-axis MTF50 performance close to 200lp/mm

(Figure 4.2). To understand how much of this raw image information can be preserved through the fiber transfer and image sensing we placed the monocentric lens looking down into a 90° fold mirror, and set up a scene at 1m range that included a star pattern chart for slant-edge MTF testing.

Figure 4.6 shows the physical arrangement and image data acquired for three cases:

a) Microscope relay image of a region of the spherical image formed on the rear

surface of the meniscus lens of the bare monocentric lens objective

b) Microscope relay image of the planar rear surface of a fiber bundle with a

spherically polished front surface index-matched to the meniscus lens.

c) 5 Mpixel digitally sampled image from a fiber-coupled CMOS sensor.

109

In the top row of Figure 4.6, the depth of field of the lens used determines the depth of field, and therefore the fraction of the overall field of view captured by the monocentric lens which is visible in the microscope's image. The MTF calculation at maximum magnification confirms the previous results, achieving 170 lp/mm MTF50 performance. In the center row, the 2.5µm polished bare fiber bundle curved input face transfers the image to its planar exit surface. Maximum magnification reveals the fiber bundle structure, but the slant edge MTF measured is still excellent, with a MTF50 resolution of 110 lp/mm. The transferred image shows local defects of the fiber structure, which can be mitigated using digital image processing techniques [66] for sensor calibration.

30X$0.6X$VH#Z20R/Z20W$$ 100X$5X$VH#Z100' 1000X$20X$VH#Z100UR/Z100UW' MTF50$170lp/mm$$

Spherical$image$surface$

0.6X$ 5X$ 20X$ MTF50$110lp/mm$ Op>cal$ ﬁber$ bundle$

Transferred$image$

Fiber$coupled$CMOS$sensor$ MTF50$70lp/mm$

Figure 4.6: Fiber-coupled image transfer effect on visible spectrum (LED illumination) MTF lens performance of the fabricated 2GS monocentric imager prototype. Top row: performance of glass monocentric objective lens only. Center row: objective with the fiber-bundle in oil contact with the mounting meniscus. Bottom row: single 5Mpixel sensor fiber-coupled system.

The bottom row shows the performance of the fiber-coupled imaging system. The microscope relay optics from the second case is replaced by a 1.75µm 5Mpixel

110

Omnivision sensor that is UV cured to the bundle. Upon acquisition, the image was auto white-balanced in Photoshop. The image demonstrates good overall color transfer, and the Moiré artifacts of the image sensing are not apparent except at high magnification.

The achieved MTF50 resolution of this raw image is 70lp/mm. While this resolution is low compared to the bare monocentric lens, it still compares favorably to a conventional wide-angle lens, as shown by the following section.

4.3.2 Monocentric fiber-coupled vs. a conventional lens image forming

The next step was to test the off-axis monocentric fiber-coupled image performance and to compare the image quality with the conventional of-the-shelf wide- angle lens. A fair comparison between the monocentric fiber-coupled lens and the conventional lens requires three metrics: field of view, F-number and resolution. Since equivalent lenses with a 120° wide field of view and F-number as low as F/1.35 do not exist, we chose as our benchmark lens a high-quality example of a retro-telephoto lens, the Canon EF 8-15mm F/4L “Fisheye” USM zoom lens, intended for use with digital single lens reflex (DSLR) cameras. This lens was constrained by the DSLR requirement for back focal length, and it also provided "zoom" adjustment of focal length, whereas the monocentric lens is a fixed focal length (prime) lens. However, the best lens resulting from a search for high resolution and low F/# prime lenses with the required 12 mm focal length resulted in the best candidate a 12mm F/1.6 SLR Magic Hyper Prime Cine lens built for micro-four-thirds cameras. This lens was smaller, but provided only an 84° diagonal field of view, and the two sample lenses we characterized yielded a substantially lower resolution at all field angles than the Canon EF lens.

111

To match the monocentric lens, we set the Canon lens focal length to 12mm and the minimum F-number of F/4. The same Omnivision OV5653 1.75µm sensor was used to acquire the images in both systems. Since the single Omnivision 1/3.2” format sensor cannot cover the full frame Canon lens field of view, we focused the lens at the center of the field of view, and used a lateral translation stage to move the sensor and sample and test the performance at different field positions. Similarly, we focused the monocentric lens at the center of the field of view, and repositioned a single fiber-coupled OV5653 sensor on the monocentric meniscus to vary the lateral field angle. We set up the flat test scene 1m away (60° object was at 2m) and acquired image data at 0° (on-axis) and 60° off-axis.

The exposure time for the Canon F/4 lens was 4x (-60° off-axis) to 6x (0° on axis) longer than for the F/1.35 fiber coupled lens, which is consistent with the difference in aperture combined with the modest addition of loss from absorption in the fiber bundle.

The image resolution at 0˚ was similar for both imagers, with an MTF50 between 70 and

80 lp/mm, as expected from our measurements (Figure 4.6) and Canon specifications for the EF lens series. However, the fiber-coupled monocentric lens showed considerably better performance at the edge of the field. Figure 4.7 shows the photo of the setup and photos of the scene at -60° taken through both of the lenses. The photo taken with the fiber-coupled lens is raw (not digitally processed with calibration data) and so the individual fiber cores imperfections are clearly visible, especially on the white portions of the scene.

The Canon benchmark lens reaches MTF50 of 47lp/mm and image suffers from strong chromatic aberration and distortion. The fiber-coupled monocentric lens achieves

112

MTF50 of 74lp/mm in an image free of distortion and lateral color. More significantly, it achieves this level of performance at more than 2 aperture stops lower than the Canon

(F/1.4 vs. F/4) and within a dramatically smaller volume.

1 Canon%EF% Canon%benchmark%:"MTF50%44lp/mm% 8415mm%F/4% FC%monocentric:"MTF50%78lp/mm% “Fisheye”%lens% F1.7%diﬀrac8on%limit%

2GS% monocentric% objec8ve%lens% F4%diﬀrac8on%limit% SFR (MTF) -60° Fiber4coupled% 0 CMOS%sensor% 0 100 200 300 lp 400 !mm Frequency [ cycles/mm ]

Fiber4coupled%monocentric%lens%output% Canon%benchmark%lens%output%

Figure 4.7: Side-by-side lens -60° off-axis performance comparison: conventional wide-angle vs fiber-coupled monocentric lens. In both cases partial field of view was sensed by the Omnivision OV5653 1.75µm 5Mpixel CMOS sensor. Canon lens suffers from distortion and strong chromatic aberration. The fiber-coupled monocentric lens has a 4x increase in sensitivity and 10x reduction in volume.

113

4.4 Wide field (multi-sensor) fiber coupled monocentric imaging

4.4.1 Fiber-coupled monocentric lens prototype

The next step was to assemble 6 fiber-coupled sensors and the lens glass into the self-contained 126°x 16° field of view “Letterbox” imager prototype. We chose to use the

F/1.0 2GS monocentric lens part for maximum light collection. We designed and fabricated the compact optomechanical mount shown in Figure 4.8. The meniscus and 6 adjacent sensors are rigidly attached to an outer frame, while the central ball lens is mounted on a flexure that allows axial translation for focus. The sensors connect through small controller boards with USB-2.0 connection to a single host computer which acquired and displayed simultaneous images at 3 fps at full resolution, including basic image processing for sensor calibration (described in [66].) The same USB connection allows remote focus actuation via a stepper motor. However, the stepper motor was not used in this first system integration, and focus was performed manually. The test interface was MDOSim software, developed by Distant Focus for Windows and Linux platforms. An exploded solid CAD model of the assembly is shown in Figure 4.8(a). The ball lens is placed in a flexure mount and centered with the meniscus. Assembly allowed a 300µm travel range to focus the lens from 0.5m to infinity. All 6 sensors were assembled with UV-cured Norland NOA74 optical adhesive between the fiber bundles and meniscus lens, then permanently fixed in position. Details about the optomechanical mount and the motion system will be described in more detail separately [67].

114

Moun5ng) Fiber+coupled) (a)$ )meniscus) (b)$ Fiber+coupled)focal)) OV5653)sensors) plane)array) Lens)ﬂexure) mount) Lens)assembly) Monocentric)) 2+glass)lens)

Manual) Stepper)motor)) axial)focus) axial)focus) FFC)sockets)for)sensor))control) over)the)USB)controller)boards)

Figure 4.8: Single row 30Mpixel fiber-coupled monocentric imager CAD models of (a) the exploded imager system showing 2GS monocentric lens, electronic focusing mechanism and fiber coupled Omnivision sensors forming the seamless array for 126°x16° FOV curved image sensing and (b) as built assembly of the integrated imager system with manual focusing option.

4.4.2 Fiber-coupled monocentric imager vs. wide angle DSLR camera

For a direct imager system comparison, we combined the Canon EF lens with the full frame Canon DSLR EOS 5D Mark II camera and mounted it side-by-side to our

Letterbox fiber-coupled prototype, as shown in Figure 4.9. For characterizing wide-angle imagers we printed a 6.10m by 2.40m poster with repeating resolution and image quality test patterns and mounted it on the wall of the lab, such that both cameras were placed

1.6m away from the wall and would image the entire poster within a 126° horizontal field of view. The poster was illuminated uniformly with 4 evenly-spaced white LED studio lights. The monocentric fiber-coupled prototype was connected to a desktop computer supporting 6 USB hosts. This enabled parallel capture and storage of full resolution image data, as well as streaming real time 30Mpixel color imagery to six 30” high definition WQXGA monitors (Figure 4.9), each monitor displaying the output of an individual sensor. The USB 2.0 standard limited the frame rate to 2-3 frames per second

115 at full 5 Mpixel resolution. The monitor resolution was 4Mpix, so the actual real time display was limited to 24Mpixel real time resolution.

Figure 4.9: Comparison of the conventional DSLR Canon camera with wide-angle lens to fiber- coupled monocentric camera. On the right: photo of the laboratory characterization system.

The initial raw snapshots of the full field captured by the prototype reveal variable width vertical bars in the sensed image field, as well as a small vertical mismatch between adjacent image sectors (Figure 4.10). The gaps in the image originate from misplaced pedestal positions on the back of the bundles, as well as any physical gap between the side facets of the shaped bundles. Imperfections in the surface of the fiber bundle and Moiré effects are also visible, as is a global cosine squared relative illumination falloff from vignetting of the central aperture and relative illumination. Note that the monocentric image structure orients the light collection towards the sensed image, and so has substantially better field uniformity than the conventional fourth power cosine dependence of illumination of a paraxial lens [68]. The MTF50 performance measured at the center of the field was around 25-30 lp/mm, and at ±60° off-axis was in the range of 50-80 lp/mm. The drop in resolution expected from the F/1 objective lens only (Figure 4.1) is due to the additional elements in the image sensor (the fiber bundle and Bayer color-filtered CMOS sensor).

116

%60°# %30°# 0°# 30°# 60°#

MTF50!! MTF50!! MTF50!! T:57lp/mm!!S:70lp/mm! T/S:!30lp/mm! T:50lp/mm!!S:79lp/mm!

Figure 4.10: Raw unprocessed image data acquired with the monocentric fiber-coupled prototype. Vignetting, field gaps, fiber bundle imperfections and moiré patterns are visible in the image.

Full width images from both the monocentric and benchmark images are shown in

Figure 4.11, including enlargements of selected regions across the field of view. The fiber-coupled image has been digitally processed using the global processing techniques described in [66], including compensation for fiber Moiré and local defects as well as relative illumination, along with interpolation across the vertical gaps in the sensed image field. Like most camera manufacturers, Canon compensates for sensor nonuniformities and also applies a proprietary edge-enhancement algorithm even to their raw images, similar to an unsharp mask even in RAW format, which makes the photos more visually appealing in high contrast areas, but such edge-enhancements render inaccurate MTF calculations of the Canon DSLR images. While both lenses have the same 12 mm focal length and similar horizontal field of view, the comparison between the 22 Mpixel DSLR and 30Mpixel Letterbox imager is imperfect. The Canon camera’s 22Mpixel full resolution output is sensed by 6.4µm pixels, and 89% of the image data was discarded when the image was cropped to match the letterbox field of view of our monocentric

117 prototype. On the other hand, the digital to analog conversion (ADC) in the Canon image sensor is substantially better, yielding 12 bits signal depth as opposed to the 8-bit depth of the OV5653 sensor transmitted by the MDOSim interface.

Comparing the two images, the detail in the enlargements demonstrate the superior resolution of the monocentric fiber-coupled prototype, especially at the edge of the field where the image quality of the DSLR image is clearly limited by lens resolution

(especially lateral color) as well as sensor sampling. Colors are richer and more vivid in the Canon photo, presumably due at least in part to the 12-bit Canon ADC. A halo is visible at the center of the monocentric imager field, which reduces contrast at lower spatial frequencies. This was the consequence of a wide-open F/1.0 aperture and resulting increase in stray light. In early versions of this prototype where only small a portion of the image surface was covered by fiber-coupled sensors, we observed significant stray light from internal scatter of the other image fields, which would otherwise have been absorbed by the image sensor. The lens models indicated this halo was to be expected with the F/1 aperture, but not with a reduced aperture. We confirmed this by acquiring on-axis images with no visible halo using an external fixed aperture stop set at F/2 and

F/4 centered in front of the F/1 monocentric lens. The 2-glass monocentric lens was designed to operate over the 470-650nm spectral range, and so it requires an external spectral filter for outdoor photography. The second fiber-coupled monocentric imager prototype, currently being assembled and characterized, uses a F/1.7 4-glass monocentric lens that resolves the visible through near infrared spectrum [69], fiber bundles with much more accurate surface position and edge quality, and includes the stepper motor actuation for focus.

118

22Mpixel(benchmark(Canon(camera(image,(cropped(to(2.2Mpixel(over(le9erbox(ﬁeld(

30(Mpixel(processed(ﬁber>coupled(monocentric("le9erbox"(prototype(image(

Figure 4.11: Indoors comparison photos of the performance between Canon EOS 5D Mark II DSLR camera with Canon 8-15mm F/4 fisheye lens and the 2GS fiber-coupled F/1.0 monocentric prototype. Canon photo is cropped to match the prototype’s letterbox 8:1 field of view. 4.5 Conclusions

This chapter establishes the potential of fiber-coupled monocentric imagers in next-generation high-resolution panoramic imaging. Such imagers may find applications as high-performance versions of the popular GoPro-type video recorders, or as an extremely compact cinematic camera to provide panoramic and/or stereo imaging for emerging immersive media. We confirmed the ability of monocentric lenses to focus

119 wide-angle images over a range of object depths, and the feasibility of high-resolution fiber-coupled curved image transfer across the 126° field of a self-contained "letterbox" format imager. This first prototype acquires a fraction of the full image data resolved by the monocentric objective lens. However, the full image could be acquired by a straightforward extension of the structure, with multiple image sensor rows arranged into a 2-D array. Such a camera would provide an F/1.7 85 Mpixel image with a 120˚x120˚ field of view, and with the optics, optomechanics, and sensors held within a volume similar to the current prototype. The compact size is a primary advantage of this class of imager. The volume of our conventional benchmark lens imager is about 465 cm3, approximated by a cylinder containing the 78 mm diameter lens barrel and extending to the image sensor (83 mm to flange, then 44 mm to focal plane). The volume of a fiber- coupled monocentric imager can be approximated by a 7.3mm hemisphere containing the convex front lens on a cylinder 47mm in diameter by 27mm long containing the focus actuator and fiber bundles and sensors covering the same full field of view.

Ongoing research is being directed towards optimization of the fiber bundle microstructure for enhanced spatial resolution, digital image processing techniques to further improve image appearance, and exploration of full-field imagers including curved fiber-bundle geometries.

Chapter 4, in full, is a reprint of the material as it appears in “Panoramic monocentric imaging using fiber-coupled focal planes,” by I. Stamenov, A. Arianpour, S.

J. Olivas, I. Agurok, A. R. Johnson, R. A. Stack, R. L. Morrison and J. Ford, Optics

Express, Vol. 22, No. 26, 2014. The dissertation author was the primary investigator and author of this paper.

5. Broad spectrum monocentric lens

5.1 Broad spectrum 4GA-8 monocentric lens prototype

In Chapter 2 we discussed the algorithms for optimum 2GS monocentric lens design and we used them to find the most suitable candidate for the prototype fabrication.

The fabricated 2GS prototype performance was demonstrated in Chapter 4. This procedure worked for a 12mm focal length F/1.7 lens that operated in visible waveband

(470-650nm). Pushing the lens operation performance further, necessitated the use of the more advanced monocentric lens design techniques, discussed in Chapter 3.

The second-generation broad-spectrum lens kept the 12mm focal length and F1.7 aperture, but required the operating spectral bandwidth to cover the full visible through near-infrared (VNIR) range. The preliminary design for 435-1000nm spectrum range lens, was shown in Table 3.9 and Figure 3.16. During the final design stages, by request from the selected lens fabricator (Optimax), the glass pool was restricted to OHARA and 120

SCHOTT glasses, which resulted in changing the glass core from Sumita K-GFK68 to the closest Ohara S-FPM2 glass. The whole five-dimensional 4GA-8 near global optimization was repeated, using the more restricted glass catalog. The finalized 4GA-8 lens prescription for fabrication is shown in Table 5.1. Simulated MTF curves for the designed lens are shown in Figure 5.1. Optimax fabricated three 4GA-8 prototype samples, with fixed aperture at F/1.68. As will be shown in the following section, the first

120 121 air gap thickness and the adjacent S-NBH8 glass element centration proved to be the most critical for lens performance, both on and off-axis.

Table 5.1: Finalized optical prescription of the 435-1000nm F/1.7 f=12mm second generation broad-spectrum SCENICC monocentric lens (4GA-8 architecture) Radius Thickness Glass Semi-diameter OBJ Infinity Infinity Infinity 1 6.78312 3.77312 S-LAH52 6.50000 2 3.01000 0.01000 NOA76 3.00000 3 3.00000 3.00000 S-FPM2 3.00000 STO Infinity 3.00000 S-FPM2 2.20653 5 -3.00000 0.01000 NOA76 3.00000 6 -3.01000 1.07204 N-LASF41 3.00000 7 -4.08204 0.17113 3.92737 8 -4.25317 2.56520 S-NBH8 4.06447 9 -6.81838 3.15535 6.47500 10 -9.97373 2.00000 GG435 9.10000 IMA -11.97373 0.00000 10.37472

1.0 0.9 4GA$8& 0.8 S-FPM2 N-LaSF41 0.7 0.6 S-LAH52 0.5 As&fabricated&lens:&12mm,&F1.68& 0.4 VNIR&(435$1050nm)&&& & &&&& TS 0.00 (deg) 0.3 TS 30.00 (deg) 0.2 TS 60.00 (deg) penny%

Modulus of the OTF S-NBH8 GG435 (same%scale)% 0.1 1cm RMS spot/Airy disc = 202% 0.0 0 40 80 120 160 200 Spatial Frequency [lp/mm]

Figure 5.1: MTF performance and the 4GA-8 12mm F1.7 broad-spectrum monocentric lens design with a photo of the fabricated prototype.

122

For optical element bonding, Norland 76 optical adhesive was used.

5.2 Imaging performance and focus of the 4GA-8 VNIR lens

Initially, we tested the on-axis MTF performance of both 2GS and 4GA-8 lenses with VIS and VNIR laboratory illumination. We used a NIR 5.5µm pixel pitch CMOS camera for sensing and an apochromatic microscope objective for relay imaging, as in

Section 4.2.2. Out of three fabricated 4GA-8 lens prototypes, two matched the expected performance levels, and the best one was chosen for further testing.

On-axis MTF results (Figure 5.2) show that the 2GS and the 4GA-8 lens both work well in the visible spectrum, reaching 45% and 58% contrast at 200lp/mm, respectively. With VNIR illumination, the 2GS lens performance drops to 20% contrast while the 4GA lens maintains high contrast at 47% at 200lp/mm. This confirms that the monocentric 4GA lens should resolve the full silicon sensor-operating band for VNIR imaging.

The next step was to confirm the focusing operation of the 4GA-8 lens over the full field. This is where a problem with the airgap thickness and centration was identified.

With the ball lens centered upon the mounting meniscus, a symmetric 57µm image defocus at ±60° field was observed. From Zemax simulations we knew that the air gap thickness inside the ball assembly and the centration of the adjacent glass element was critical. Using Zemax simulation with as-fabricated surface data from the lens passport we concluded that accumulated lens surface tolerancing errors of the same sign, together with an airgap thickness which was slightly out of spec and with a 5µm S-NBH8 element decentration, all combined to result in image forming outside the outer meniscus surface.

CEV$"Gen1"$Lens$ Test%sensor:% 2$glass$symmetric$design$$ CEV$"Gen$2"$Lens$ 4Wglasses,$asymmetric$airWgap$$ %%Microscope% (1st$CEV$lab$experiments)$ to$correct$spherochroma[sm$ %%relay%to%2Mpix% 12$mm$focal$length,$F/1.35$ %%VNIR%CMOS%focal%plane% 12$mm$focal$length,$F/1.68$ 1cm%

Visible'470+700nm'spectrum'(externally'ﬁltered'source)'op;cal'performance:'' Both'the'original'2+glass'and'the' new'4+glass'lenses'both'work'well.' Halogen$lamp$w/ND1$+$ 470$W$670nm$ﬁlters$

58%$contrast$ 45%$contrast$ @200lp/mm$ @200lp/mm$

0.4$ 0.5$ 0.6$ 0.7$ 0.8$ 0.9$ 1.0$ 150x100pixels$ 150x100pixels$ Wavelength$[μm]$

Visible'+'Near+Infrared'spectrum'(unﬁltered)'op;cal'performance:'' 2+glass'lens'performance'drops,'but'' 4+glass'lens'maintains'high'contrast' Halogen$lamp$with$ ND1$aIenuator$

47%$contrast$ @200lp/mm$ 20%$contrast$ @200lp/mm$ 30%$contrast$ @$300lp/mm$ 150x100pixels$ 150x100pixels$ 0.4$ 0.5$ 0.6$ 0.7$ 0.8$ 0.9$ 1.0$ Wavelength$[μm]$

123 Figure 5.2: 2GS and 4GA8 monocentric lens on-axis performance comparison in VIS (470-650nm) and VNIR (435-1000nm) spectrum range.

124

Simply put, the originally designed GG435 meniscus was not matched to the as- fabricated ball lens assembly. Fortunately it was possible to design a new meniscus lens specifically optimized to mitigate the as-fabricated lens assembly. Each of the 4GA-8 lens prototypes had a similar problem, and each required a custom fabricated GG435 meniscus lens for error correction. Figure 5.2 shows the problem in detail, with our (on- axis) best performing 4GA-8 prototype lens as an example. The top right part of the

Figure 5.2 shows images formed on the back of the meniscus, sensed through relay imaging arms at -60°, 0° and 60° respectively, when the monocentric lens is focused at object 1m away. The bottom right images show the off-axis scene in focus, after 57µm of refocusing travel for the microscope relay imaging arms. The left portion of the figure shows the geometry of the fabrication tolerance problem together with MTF performance levels on the new image surface (marked in red).

0°(

REFOCUSED( 57um(refocus(needed(at(±60°( Image&forms&on&a&surface&outside&meniscus&

146lp/mm( 247lp/mm( 151lp/mm( <60°( 60°( 133lp/mm( 271lp/mm( ((92lp/mm( Monocentric&lens&resolu3on&test&bench&

Figure 5.3: Broad spectrum 4GA-8 lens focusing problem: accumulated tolerancing errors result in image formation in air, 57µm outside the mounting meniscus. Custom, reoptimized meniscus was needed to correct the error.

1.0m%ﬂat%object%distance%% 1.0% MTF50%%147lp/mm% 1.0% MTF50%%244lp/mm% 1.0% MTF50%%155lp/mm% MTF50%%119lp/mm% MTF50%%262lp/mm% MTF50%%%%94lp/mm% 0.8% 0.8% 0.8%

0.6% 0.6% 0.6%

0.4% 0.4% 0.4%

0.2% 0.2% 0.2% tangen5al/sagi:al% tangen5al/sagi:al% tangen5al/sagi:al% 0% 0% 0% 0% 100% 200% 300% 0% 100% 200% 300% 0% 100% 200% 300%

LED$IKAN$light$ 1.0m% source$4200K$ NIR$CMOS$$sensi=vity$

&60°% +60°% Resolu5on%tes5ng%relay%imaging%system% 0.4$ 0.5$ 0.6$ 0.7$ 0.8$ 0.9$ 1.0$ Wavelength$[μm]$ 125 Figure 5.4: Focusing and performance of the 4GA-8 lens with matched meniscus lens (1m object distance, visible LED illumination)

1.0m%ﬂat%object%distance%% 1.0% MTF50%%143lp/mm% 1.0% MTF50%%247lp/mm% 1.0% MTF50%%172lp/mm% MTF50%%128lp/mm% MTF50%%257lp/mm% MTF50%%%%93lp/mm% 0.8% 0.8% 0.8%

0.6% 0.6% 0.6%

0.4% 0.4% 0.4%

0.2% 0.2% 0.2% tangen5al/sagi:al% tangen5al/sagi:al% tangen5al/sagi:al% 0% 0% 0% 0% 100% 200% 300% 0% 100% 200% 300% 0% 100% 200% 300%

Halogen(lamp(with( 1.0m% ND1(a2enuator( NIR(CMOS((sensi

&60°% +60°% Resolu5on%tes5ng%relay%imaging%system% 0.4( 0.5( 0.6( 0.7( 0.8( 0.9( 1.0( Wavelength([μm]( 126 Figure 5.5: Focusing and performance of the 4GA-8 lens with matched meniscus lens (1m object distance, broad spectrum illumination)

127

The meniscus used in these experiments had R1=10.027mm and R2=12.013mm, with center thickness CT=2.017mm. The slight residual performance difference on the opposite off-axis fields is attributed to decentration of the lens core relative to the S-

NBH8 element. Upon Zemax reoptimization and new fabrication order, the new

“matched” GG435 meniscus had radii R1=10.187mm, R2=12.051mm and CT=1.972mm.

The focusing experiment was repeated with the new meniscus and two illumination sources: a visible light LED array and a VNIR spectrum from a halogen lamp with neutral density (ND) filter in front. Figure 5.4 shows the image formation on the back of the meniscus and corresponding MTF curves. The on-axis performance is now close to the original design performance, while off axis performance is slightly lower, due to the decentering error of the cemented element adjacent to the airgap coupled with larger airgap thickness than in the design. These errors cannot be compensated with a custom mounting meniscus, but may be avoided with tighter assembling tolerances in future prototype fabrications. Figure 5.5 shows similar behavior for broad-spectrum (400-1000nm) illumination on USAF test targets for the same field positions. It is important to emphasize that GG435 mounting meniscus acts as an UV- cutoff filter at 435nm.

The 4GA-8 broad-spectrum lenses have a strong potential for low light and VNIR wide field of view surveillance applications, and can be used with both monochrome or color fiber-coupled sensors. A color sensor would require an additional optical band pass filter for accurate color reproduction, possibly in form of a front protective dome made from optical filter glass like SCHOTT BG40 or similar.

128

The 4GA-8 monocentric lens geometry, opens the combination of broad spectral waveband and high light collection, but has drawback. The manufacturing requires the air gap thickness tolerance to be 10µm or less, while maintaining element centration error below 5µm. Otherwise, a custom mounting meniscus fabrication is needed for each 4GA-

8 ball assembly, and this partially restores off-axis performance. That said, the optical performance of this lens is, to our knowledge, unprecedented in flat-field objectives of any cost or complexity.

5.3 Broad-spectrum “Letterbox 2” prototype camera

5.3.1 Components and assembly

As with first 2GS prototype, described in Chapter 4, we fabricated six fiber bundles, curved and polished on input side and cut to a pedestal and polished flat on output side. Photos of unattached fiber bundles and their intended configuration are shown in Figure 5.6. Back surface around the pedestal was painted black to minimize stray light. As before, fiber bundles were bonded to decapped Omnivision sensors using optical cement (see bottom left in Figure 5.6). Impulse responses of all fiber-coupled sensors were checked at the center and at all four corners of each sensor and all were within 3x3 pixel area.

The optomechanical mount for the second generation 4GA-8 lens was upgraded with electronic focus option together with the new aluminum mount for new GG435 meniscus and 4GA-8 ball lens holder. Because of major CAD model revisions, a mechanical interference between six fiber bundles and the aluminum mount was discovered late in the prototype assembly process. To avoid additional delays, it was

129 decided to proceed the assembly with five out of six fiber bundles, reducing the horizontal field of view to 105°. At the same time a seam in the middle of the field was avoided too.

Top$view$

Front$view$ Una8ached$ﬁber$bundles$

Fiber2coupled$sensors$

Back$view$

Figure 5.6: Cut fiber bundles before and after integration with OV5653 sensors. Configurations on the right show intended sequence and relative positioning of individual bundles after their placing in contact with the mounting meniscus.

To ease the fiber-coupled sensors mounting and integration of the whole 4GA-8 monocentric prototype, a 3D-printed part was made and used to keep individual fiber- coupled sensors in place while ensuring minimum gap at the seams (Figure 5.7, left).

Strong UV light source was used to perform the cure of the optical cement between the fiber bundles and meniscus. Additional optical cement was used to reinforce the whole structure and applied between the fiber bundles and aluminum mount.

130

4GA$8&MC& lens&

Electronic& focus&

Figure 5.7: UV-curing process (left) of fiber coupled sensors to the lens mounting meniscus, fully assembled “Letterbox 2” 4GA-8 monocentric fiber-coupled prototype (center) and whole system enclosed in the box (right) ready for outdoor testing.

Assembled and fully cured “Letterbox 2” 4GA-8 broad-spectrum prototype is shown in Figure 5.7 center (no enclosure) and right (inside box enclosure). Initial results with photos taken indoor and outdoor will be shown in following subsections.

Upon prototype use over a course of few days the problem of optical cement bond longevity was discovered. Large difference in temperature expansion coefficients (TCEs) for the glass material and aluminum mount caused delamination of fiber bundles from the meniscus lens. The result were defects in the field of view, changing dynamically with temperature requiring the fix in form of a different material instead of aluminum for holding the fiber bundles in place (invar or similar).

5.3.2 Indoors operation (VIS illumination spectrum)

The first light generating indoor panorama photo is shown in Figure 5.8. Visible

(VIS) illumination spectrum was used with LED studio light sources (4200K color temperature). Five fiber-coupled sensors covered 105°x16° total field of view.

131

RAW$unprocessed$image$in$MDOSim$

Dot$–$divided$image$in$MDOSim,$auto$levels$in$Photoshop$(NO$interpola;on$&$NO$ver;cal$adjustment)$

Seam%removal%

Figure 5.8: Indoor panorama image acquired by 4GA-8 “Letterbox 2” monocentric fiber-coupled imager prototype. Visible LED light (VIS spectrum) illumination used. (Top) raw unprocessed image, (middle) calibrated white-balanced image with no interpolation and (bottom) zoomed regions showing the available resolution and almost virtually no information loss at the seams.

Second fiber bundle fabrication solved the seamless imaging problem that was present in previous 2GS prototype, as discussed in Chapter 4.4. This resulted in field loss no larger than three pixels, which will be reconstructed by various interpolation techniques. An interesting fact is that a very small portion of field of view, incident at the seam between two fiber bundles, gets coupled through the glue line through bundle sides to both neighboring sensors. This fact is expected to be helpful in information restoration during later stages of image processing, subject of ongoing research efforts.

132

5.3.3 Outdoors operation (VIS and VNIR illumination spectrum)

The “Letterbox 2” broad-spectrum 4GA-8 imager was taken outside and in bright, clear sky day the panoramic image was acquired both with IR cutoff filter in front of the lens and without (see Figure 5.9). Image quality and resolution is virtually the same like with previous 2GS “Letterbox 1” prototype, when IR filter is used [70]. Author of this dissertation with USAF test chart is shown in frame at the edge of the field at 155m away from the camera. All spectrum components below 435nm were absorbed by GG435 meniscus material. The second photo shows the scene without IR filter, as sensed with color sensors in VNIR spectrum. High resolution performance is maintained, as expected.

Sunlight)illumina,on)with) UV/IR)cut)ﬁlter)(VIS))

U V# c IR#cut# u t#

Sunlight)illumina,on)with) UV)block)only)(VNIR))

Figure 5.9: Panoramic images acquired outdoors with 4GA-8 fiber-coupled “Letterbox 2” prototype with and without IR cutoff filter. High resolution imaging performance is maintained from VIS illumination range through VNIR.

133

This outdoor experiment, together with MTF resolution tests in laboratory conditions

(Chapter 5.2), demonstrated high-resolution imaging capabilities of the 4GA-8 fiber- coupled monocentric lens, both in visible (VIS) and VNIR spectrum ranges. Because of mechanical interference error, only 105° horizontal field of view was covered. An additional problem, which started to develop during second outdoor experiment, was a delamination of optical fiber bundles from the mounting meniscus.

Next 4GA-8 prototype will have all six fiber bundles in place (to cover full 126° field of view) and optomechanical mount in contact with the bundles from similar TCE material, to avoid delamination. Authors are aware that full VNIR imaging capabilities of

4GA-8 lens require native monochrome sensors, instead of RGB. Ongoing research efforts are focused on single large sensor/curved fiber bundle architecture (see Figure

1.3(b)) both with color and monochrome operation. Results with upgraded prototype will be reported separately.

6. Conclusions and future directions: Towards Terapixel video imaging

6.1 Long term perspective: system level performance metrics

Current digital image sensors cannot be fabricated with a combination of high spatial resolution and deep spherical curvature, but curved image sensors have been the subject of intense investigation [71]. Recently Sony announced a full frame and 2/3” image sensors [72]. So it is worth looking forward to see the potential performance of a camera with direct sensing of the monocentric objective’s image.

Designing a camera system raises an important question of the performance metrics and its targeted value. Modern compact camera systems have a tendency of having high-resolution sensors outresolving the lenses used. The primary reason is the use of total sensor pixel count for camera advertising, regardless of the full utilization of 134

this resolution by the lens. It is informative to plot the F/# vs the sensor pitch for camera systems. Figure 6.1 shows several modern state-of-the art camera systems capable of

>120° field of view imaging (marked in black), as well as two cell phone cameras

(marked in red) on a map relating the utilized sensor pixel pitch with the optics F- number. For DSLR systems, the lowest available F-number wide-angle lenses were used.

134 135

The 2GS SCENICC Letterbox prototype camera system discussed in Chapter 4.4 is also shown in the chart. The equation describing the relation of the Airy disc diameter with F-number:

DAiry = 2.44 ⋅λ ⋅ F / # (5.1)

is shown as a line dividing the space into sensor limited and lens performance limited domain.

Sensor pixel pitch (µm) 10 9 8 7 6 5 4 3 2 1

Sensor limited SCENICC Letterbox SCENICC Full

2 Iphone 6 Lumia 1520 r e Nikon D810 b GoPro HERO3+

m 3 u Hasselblad H5D-50c n

- Panasonic GH4 F 4 Canon 5D Mark III er et m ia d 5 c is d y ir 6 A 7 8 Lens limited 9 10

Figure 6.1: Map of modern camera systems in F-number vs sensor pitch space. Cameras marked in black, with dedicated optics, can achieve >120° field of view imaging.

In terms of this simple relation, the professional medium-format and full-frame

DSLR cameras (Hasselblad, Canon, Nikon) have well balanced the wide-angle lens

136 optics F-numbers with size of the sensor pixels (5.5-6.4µm). On the other hand, more compact imaging systems like cell phone cameras tend to have much lower pixel pitch sensors (down to 1.1µm), outresolving the lens by a large margin.

If we add a total system resolution as a third dimension to the plot, we get the chart shown in Figure 6.2.

Figure 6.2: Chart of modern camera systems showing total system resolution with sensor pixel pitch and optics F-number tradeoffs.

137

The highest resolution off-the-shelf camera is a medium format Hasselblad camera with 50MPixel sensor, as compared to the single-row SCENICC Letterbox prototype camera’s 30MPixel resolution. The data point “SCENICC Full” pertains to the next step in image sensing, a monocentric camera prototype using 5 rows of 4 sensors, for a total of 85MPixels.

Monocentric camera systems have a clear potential of delivering high resolution imaging in a small system volume. To understand the upper limit of a monocentric camera resolution (and the upper limit of the chart in Figure 6.2) we decided to explore the raw image information capacity through appropriately chosen performance metrics as a function of system scale and the aperture setting.

6.2 Total resolution scaling in a fully monocentric camera system

To characterize the full available camera system resolution, one must take into consideration both lens and sensor performance to compute the total available “true” image resolution. With that approach we avoid both extremes – strong lens-limited or strong sensor-limited performance - and give an accurate metric of the actual image information capacity. DSLR camera manufacturers use MTF vs. field curves at specific spatial frequencies (10 and 30lp/mm for full frame sensors, 20 and 40lp/mm for micro- four-thirds format) and observe the image contrast across the image height starting at the center of the sensor and ending at the far corner.

Observation of various DSLR and machine vision camera systems’ MTF charts revealed an empirical trend, that the wide-angle lens at some given F-number setting is considered “good” if the lens MTF vs. field curve at spatial frequency roughly equal to

138 one third of the sensor Nyquist frequency does not fall below 30-50% at the extreme field angle. If so, the lens is considered “matched” to the sensor such that the camera system provides a “true” full sensor resolution over the given field of view without information oversampling. That is why full frame camera systems have around 6µm pixel pitch sensors, and deliver up to 23Mpixel resolution, while micro four-thirds system uses sensors over some 26% of the full frame area with 3.75µm pixel pitch and achieves up to

16Mpixel total resolution. As mentioned before, compact cameras and cell phone cameras are not good example of this trend, because their sensors surpass the optics performance.

To calculate the “true” available system resolution of a monocentric camera the following system optimization procedure was chosen:

- For all designs, the system field of view was set to circular 120°x120° and the

operating spectrum to photographic (486-656nm)

- The scale of the system was set (focal length) with the desired F-number

- Optimum 2GS design was found using the algorithm presented in Chapter 3

- Record highest spatial frequency at which at least 50% contrast is observed

(usually this occurs at the extreme field angle)

- Chose equivalent curved sensor pixel pitch that the Nyquist frequency is 3x

larger then the frequency recorded in previous step

- Total pixel count is calculated over the hemispherical image surface

This procedure is by no means flawless; especially because the equivalent spherical sensor is an abstract term. But at this point we are interested in calculating the system’s total information capacity, without discussing the mechanism of the curved

139 image sensing. One limitation is set: the chosen pixel pitch could not be smaller than

1.1µm, regardless of the Airy disc diameter.

The procedure above was repeated for various F-numbers and the scale was increased until the optimal sensor pixel pitch reached 6.5µm value. Results are shown in

Figure 6.3.

Table 6.1: Maximum resolution 120°x120° 2GS monocentric systems specs. Sensor pixel pitch Focal length Total resolution F-number [µm] [mm] [GPixel] 1 1.0 1.24 4 0.03 2 1.4 1.23 15 0.47 3 1.7 1.10 30 2.34 4 2.8 1.50 150 31.4 5 4.0 2.44 600 191 6 5.6 3.29 3000 2611 7 8.0 6.33 5550 2412

For each F-number there is an optimal focal length value where the total resolution reaches its maximum and does not increase anymore. For F/1.7 lens, total maximum resolution is achieved at 2.3GPixel and focal length of 30mm (Table 6.1). An additional trend is obvious – a lower F-number system has higher total resolution provided it didn’t reach the architecture limit (saturation zone in graph where the total resolution remains constant regardless of the focal length scale increase).

As an example, we observe the 2GS lens candidate for F/1.7 as it reaches its architectural limit at f=30mm. Camera based on this specific example could use 1.1µm sensors covering the 2827mm2 of image area surface (roughly 3.3x larger than full frame sensor area) and totaling in 2.3GPixel resolution.

140

1010000000 26126111

2412 1000 2412 ) 1000 l

e 191

x 191 i ] l P e 101000 x G

i 31

( 31

P 0 . G [ 8 10 120°x120°&ﬁeld&of&view&

10 F n 2.32.3 o i t @

u 1 l 1 0.47

n 0.47 o o i s t T oTtoatla rle rseosloultuiotino n@ @ F 8F.80.0 e u r l

0.1 Total resolution @ F5.6 l Total resolution @ F5.6 o 0.1 0.003.3033 a s

t Total resolution @ F4.0

e Total resolution @ F4.0 o r

l T 0.01 T oTtoatla rle rseosloultuiotino n@ @ F 2F.28.8 a 0.01 t Total resolution @ F1.7

o Total resolution @ F1.7 T T oTtoatla rle rseosloultuiotino n@ @ F 1F.14.4 1E1-E3-3 T oTtoatla rle rseosloultuiotino n@ @ F 1F.10.0

11 1010 101000 10100000 1010000000 fofocacal lle lennggthth ( m(mmm) )

Figure 6.3: Total resolution scaling with system size for fully spherical 120°x120° monocentric imager

Table 6.2: Optical prescription of 120°x120° f=30mm F/1.7 486-656nm 2.3GPixel lens.

Radius Thickness Glass Semi-diameter

OBJ Infinity Infinity Infinity

1 22.602 13.225 S-LAH79 21.404

2 9.378 9.378 S-LAH59 9.214

STO Infinity 9.378 S-LAH59 5.604

4 -9.378 13.225 S-LAH79 8.858

5 -30.137 7.372 20.102

IMA -29.975 25.960

141

Lens layout, MTF curves and point-spread function of this lens is shown in Table

6.2 and Figure 6.4. Similar scaling behavior is expected in more complex (4GA-8, 5GA-

10) monocentric geometries for broad-spectrum imaging.

f=30mm$ FFT$MTF$vs.$Field$ F/1.7$ 30$lp/mm$ 50$lp/mm$ S"LAH79( 100$lp/mm$ 0°( S"LAH59( 150$lp/mm$

30°( 60°( 10mm$ PSF$Cross$Sec:on$ Weight$$ 0.21kg$

Figure 6.4: The layout and performance curves of a 30mm F/1.7 monocentric lens capable of imaging 2.3Gpixel (with optimum 1.1µm pixel pitch sensor array)

6.2.1 A panoramic monocentric lens imaging 2.6 Terapixels

An interesting fact is that for photographic spectrum 2GS monocentric architecture with hemispherical equivalent “optimal sensor” there is a maximum achievable image resolution value. That value is around 2.600.000MPixels and is

142 achieved with F/5.6 3000mm lens using 3.3µm sensors (Figure 6.3). This immense amount of information would be extraordinarily difficult to capture.

Table 6.3: Optical prescription of 120°x120° f=3000mm F/5.6 486-656nm Teragon lens.

Radius Thickness Glass Semi-diameter

OBJ Infinity Infinity Infinity

1 1480.962 772.221 N-KZFS2 1344.260

2 708.741 708.741 S-FPL53 660.928

STO Infinity 708.741 S-FPL53 193.904

4 -708.741 772.220 N-KZFS2 648.844

5 -148.962 1518.665 1315.173

IMA -2999.627 2597.756

f=3000mm# FFT#MTF#vs.#Field# F/5.6# 10#lp/mm# N"KZFS2( 30#lp/mm#

0°( S"FPL53( 50#lp/mm#

30°( 60°( 1m# PSF#Cross#Sec;on# Weight## 27350kg#

Figure 6.5: The layout and performance curves of a gigantic 3000mm F/5.6 monocentric lens capable of imaging 2.6TPixel (with optimum 3.3µm pixel pitch sensor array)

143

It would require 52000 50MPixel sensors interfaced over an array of field flatteners or fiber bundles to the hemispherical image surface. The size and especially the cost of such system would be prohibitively high. The prescription of a 3000mm F/5.6

486-656nm monocentric lens is shown in Table 6.3. Optical layout, MTF curves and point-spread function of this lens is shown in Figure 6.5.

6.3 Conclusion

The work presented in this dissertation explored a promising domain of wide- angle lenses architecture – monocentric lenses that provide extraordinary combination of high light collection and high resolution panoramic imaging at various scales and operating spectral bands. An extensive design space exploration and mapping was performed, with a theory behind the procedures and algorithms for the systematic lens design. This structure was validated with two compact lens prototypes, and two imagers that demonstrated that fiber-coupled imaging transfer is an effective way to access the monocentric lens capabilities. The final chapter provided previously-unpublished calculations that show fiber-coupled monocentric cameras scale favorably in terms of resolution and volume to any conventional wide-angle camera system, with the potential to deliver as high as Terapixel panoramic (spherical focal surface) imaging. There is currently a fast-moving trend towards immersive media and augmented reality hardware, and we hope that monocentric imaging may assume an important role in future commercial systems.

References

1. W. Smith, “Modern lens design, 2nd edition,” (McGraw Hills, 2005).

2. T. Yamashita, R. Funatsu, T. Yanagi, K. Mitani, Y. Nojiri, and T. Yoshida, "A Camera System Using Three 33-megapixel CMOS Image Sensors for UHDTV2," SMPTE Mot. Imag. J. 120(8), pp. 24-31, November (2011).

3. R. Kingslake, "A history of the photographic lens," (Academic Press, 1989), pp. 49- 67.

4. T. Sutton, “Panoramic Photography,” The Photographic Journal, Vol. 6, pp. 184- 188, (March, 1860).

5. G. Krishnan and S.K. Nayar, "Towards A True Spherical Camera," SPIE Human Vision and Electronic Imaging, Jan, (2009).

6. J. E. Ford and E. Tremblay, "Extreme Form Factor Imagers," in Imaging Systems, OSA Technical Digest, paper IMC2, June (2010).

7. D. J. Brady and N. Hagen, “Multiscale lens design,” Optics Express 17(13): p. 10659-10674, (2009).

8. O. Cossairt, D. Miau, S. K. Nayar, "Gigapixel Computational Imaging," IEEE International Conference on Computational Photography (2011). 144 9. H. Son, D. L. Marks, E. J. Tremblay, J. Ford, J. Hahn, R. Stack, A. Johnson, P. McLaughlin, J. Shaw, J. Kim, and D. J. Brady, "A Multiscale, Wide Field, Gigapixel Camera," in Imaging Systems Applications, OSA Technical Digest, paper JTuE2, (2011).

10. D. J. Brady, M. E. Gehm, R. A. Stack, D. L. Marks, D. S. Kittle, D. R. Golish, E. M. Vera and S. D. Feller, "Multiscale gigapixel photography," Nature 486, 386-389, (2012).

11. E. J. Tremblay, D. L. Marks, D. J. Brady, and J. E. Ford, "Design and Scaling of Monocentric Multiscale Imagers," Applied Optics 51(20), pp. 4691–4702, (2012).

144 145

12. P. Milojkovic, J. Mait, “Space-bandwidth scaling for wide field-of-view imaging,”, Applied Optics, Vol.51, Issue 4, pp. A36-A47, (2012).

13. J. A. Waidelich Jr., "Spherical Lens Imaging Device," US patent 3,166,623 issued January (1965).

14. J. J. Hancock, "The Design, Fabrication, And Calibration Of A Fiber Filter Spectrometer" PhD Thesis, University of Arizona, (2012); see also product datasheets posted on Schott Fiber Optics website, "Schott Fiber Optic Faceplates" (faceplates_us_march_2011.pdf) and "Schott Fused Imaging Fiber Tapers" (Tapers- US-October_2011.pdf).

15. Renee Drougard, "Optical Transfer Properties of Fiber Bundles," JOSA volume 54, issue 7, pp. 907-914, (1964).

16. Jae-Ho Han, Junghoon Lee, and Jin U. Kang, "Pixelation effect removal from fiber bundle probe based optical coherence tomography imaging," Optics Express, Vol. 18, Issue 7, pp. 7427-7439, (2010).

17. Y. F. Li and J. W. Y. Lit, "Transmission properties of a multimode optical-fiber taper," J. Opt. Soc. Am. A 2, 462-468, (1985).

18. “Spherical Camera,” Popular Mechanic Magazine, published by H.H. Windsor, Vol. 99, No. 3, pp.94-95, (March, 1953).

19. T. S. Axelrod, N. J. Colella, A. G. Ledebuhr, “ The Wide-Field-of-View Camera”, in Energy and Technology Review, Lawrence Livermore National Laboratory, (December, 1988).

20. J. F. Kordas, I. T. Lewis, B. A. Wilson, D. P. Nielsen, H. Park, R. E. Priest, R. Hills, M. J. Shannon, A.G. Ledebuhr, L. D. Pleasance, “Star tracker stellar compass for the Clementine mission”. Proc. SPIE 2466, Space Guidance, Control, and Tracking II, 70 (June 12, 1995).

21. C. Akerlof, M. Fatuzzo, B. Lee, R. Bionta, A. Ledebuhr, H. Park, S. Barthelmy, T. Cline, N. Gehrels. "Gamma‐ray optical counterpart search experiment (GROCSE)." AIP Conference Proceedings 307(1): 633-637 (1994).

22. A. Arianpour, I. Agurok, N. Motamedi, and J. Ford, “Enhanced field of view fiber- coupled image sensing,” in International Optical Design Conference 2014, OSA Technical Digest (online) (Optical Society of America, 2014), paper IM2A.4.

146

23. J.M. Cobb, D. Kessler, J. Agostinelli, ”Optical design of a monocentric autostereoscopic immersive display”, SPIE Proc., Vol. 4832, pp. 80-90, (2002).

24. J.M. Cobb, D. Kessler, J.E. Roddy, “Autostereoscopic optical apparatus”, US Patent 6,871,956, (2005).

25. Hoya glass catalog, 6/20/2012, http://www.hoya-opticalworld.com/english/

26. D. Marks, E. Tremblay, J. Ford, D. Brady, ”Multicamera aperture scale in monocentric gigapixel cameras,” Appl. Optics, Vol. 50, No. 30, pp. 5824-5833, (2011).

27. M. Born, E. Wolf, “Principles of Optics 7th expanded edition”, (Cambridge University Press, 1999).

28. V. Churilovskiy, ”The theory of chromatism and third order aberrations”, (Mashinostroenie, 1968).

29. J. Sasian, “Theory of sixth-order wave aberrations”, Applied Optics, Vol. 49, No. 16, D16-D95, (2010).

30. R. Kingslake, R.B. Johnson, “Lens design fundamentals 2nd edition”, (SPIE Press, 2010)

31. G. G. Slyusarev, “Aberrations and optical design theory 2nd edition”, (Adam Hilger Ltd, 1984).

32. J. L. Rayces, M. Rosete-Aguilar, ” Selection of glasses for achromatic doublets with reduced secondary spectrum. I. Tolerances conditions for secondary spectrum, spherochromatism, and fifths- order spherical aberrations”, Applied Optics, Vol. 40, No. 31, 5663-5676, (2001).

33. I. Gardner,” Application of algebraic aberration equations to optical design”, Scientific papers of the Bureau of Standards, No. 550, (1927).

34. H.A. Buchdahl, “Optical aberrations coefficients”, (Dover Publications, 1968).

35. Schott glass catalog, http://www.us.schott.com/advanced_optics/english/download/ schott_optical_glass_catalogue_excel_june_2012.xls

36. Ohara glass catalog, http://www.oharacorp.com/xls/glass-data-2012.xls

147

37. Sumita glass catalog, http://www.sumita-opt.co.jp/ja/goods/data/glassdata.xls

38. Hoya glass catalog, http://www.hoyaoptics.com/pdf/MasterOpticalGlass.xls

39. M. M. Rusinov, ”Handbook of computational optics,” Chapter 23, (Mashinostroenie, 1984).

40. I. Agurok, “Method of “truss” approximation in wavefront testing”, SPIE Proc. Vol. 3782, pp. 337-348, (1999).

41. J. Kulmer, M. Bauer, “Fisheye lens designs and their relative performance,” SPIE Proc. Vol. 4093, pp. 360-369, (2000).

42. M. Horimoto, US Patent 4,412,726.

43. H. Gross, F. Blechinger, B.Achtner, “Handbook of Optical Systems Volume 4 – Survey of Optical Instruments”, (Wiley, 2008).

44. I. Stamenov, I. Agurok, and J. Ford, "Optimization of two-glass monocentric lenses for compact panoramic imagers: general aberration analysis and specific designs," Appl. Opt. 51, 7648-7661 (2012).

45. J. Ford, I. Stamenov, S. Olivas, G. Schuster, N. Motamedi, I. Agurok, R. Stack, A. Johnson, and R. Morrison, "Fiber-coupled Monocentric Lens Imaging," in Imaging and Applied Optics, J. Christou and D. Miller, eds., OSA Technical Digest (online) (Optical Society of America, 2013), paper CW4C.2.

46. H. Son, D. L. Marks, E. J. Tremblay, J. Ford, J. Hahn, R. Stack, A. Johnson, P. McLaughlin, J. Shaw, J. Kim, and D. J. Brady, "A Multiscale, Wide Field, Gigapixel Camera," in Imaging Systems Applications, OSA Technical Digest, paper JTuE2, (2011).

47. J. Oakley, "Whole-angle spherical retroreflector using concentric layers of homogeneous optical media," Appl. Opt. 46, 1026-1031 (2007).

48. I. Stamenov, I. Agurok, and J. Ford, "Capabilities of monocentric objective lenses," in Imaging and Applied Optics, J. Christou and D. Miller, eds., OSA Technical Digest (online) (Optical Society of America, 2013), paper ITu3E.4.

49. D. Marks and D. Brady, "Gigagon: A Monocentric Lens Design Imaging 40 Gigapixels," in Imag. Sys. OSA technical Digest (CD) (2010).

148

50. P. Gill, W. Murray, M. Wright,” Practical optimization” Academic Press, 1981.

51. S. A. Rodionov, “Computer lens design”, Mashinostroenie, 1982.

52. D. Feder, “Automatic optical design”, Applied Optics, Vol. 2, Iss. 12, 1963, pp 1209-1226.

53. G. Golub, C. Van Loan, ”Matrix Computations”, The John Hopkins University Prees, 1989.

54. R. Shannon, The Art and Science of Optical Design, (Cambridge University Press, 1997).

55. www.andor.com

56. M. Kruger, B. Panov, B. Kulagin, G. Pogarev, Y. Kruger, A. Levinson, “ Handbook of opto-mechanics”, Moscow, 1963.

57. www.norlandprod.com

58. D. L. Marks, H. S. Son, J. Kim and D. Brady, “Engineering a gigapixel monocentric multiscale camera”, Opt. Eng. 51(8), 083202 (Aug 07, 2012).

59. M. Rusinov, “Composition of optical systems”, Mashinostroenie, 1989.

60. www.morovision.com

61. D. Dayton, J. Gonglewsky, C. Arnauld, I. Mons, D. Burns, “SWIR sky glow cloud correlation with NIR and visible clouds: an urban and rural comparison”, AFRL Kirtland, May 1, 2009.

62. www.sensorsinc.com

63. www.schott.com, TIE-41 Large Optical Glass Blanks, Technical information document.

64. N. Zheng, S. C. Schmidler, D. Marks and D. Brady, “Computer experiment and global optimization of layered monocentric lens systems”, Optik – International Journal for Light and Electron Optics, Vol. 123, Issue 14, 1249-1259, (July 2012)

149

65. I. Stamenov, I. Agurok, and J. Ford, “Optimization of high-performance monocentric lenses,” Appl. Opt. 52, 8287-8304 (2013).

66. S. Olivas, N. Nikzad, I. Stamenov, A. Arianpour, G. Schuster, N. Motamedi, W. Mellette, R. Stack, A. Johnson, R. Morrison, I. Agurok, and J. Ford, "Fiber Bundle Image Relay for Monocentric Lenses," in Computational Optical Sensing and Imaging 2014, OSA Technical Digest (online) (Optical Society of America, 2014), paper CTh1C.5.S.

67. A. R. Johnson, J. Pessin, J. E. Ford, I. Stamenov, A. Arianpour, and R. A. Stack, “Optomechanical Design with Wide Field of View Fiber-Coupled Image Systems”, to be presented at the 2014 OSA Frontiers in Optics Meeting.

68. B. G. Grant, “Field Guide to Radiometry”, SPIE Field Guides, Vol.23, SPIE Press, Bellingham, USA (2011)

69. I. Stamenov, S. Olivas, A. Arianpour, I. Agurok, A. Johnson, R. Stack, and J. Ford, “Broad-spectrum fiber-coupled monocentric lens imaging,” in International Optical Design Conference 2014, OSA Technical Digest (online) (Optical Society of America, 2014), paper IM3B.5

70. S. Olivas, I. Stamenov, A. Arianpour, I. Agurok, A. Johnson, R. Stack, and J. Ford, “Image processing for cameras with fiber bundle image relay”, submitted do Applied Optics, (2014)

71. Y. M. Song, Y. Xie, V. Malyarchuk, J. Xiao, I. Jung, K. Choi, Z. Liu, H. Park, C. Lu, R. Kim, R. Li, K. B. Crozier, Y. Huang and J. A. Rogers, “Digital cameras with designs inspired by the arthropod eye”, Nature Vol. 497, 95-99 (2013)

72. K. Itonaga, US Patent 8,878,116