space. Most imaging systems with conventional Multi- telecentric exhibit varying magnification for objects at different distances from the lens. This causes several problems for machine vision and other Bo Wu applications. (i) Objects closer to the lens appear Lei Ye to be larger than those farther from the lens; The Hong Kong Polytechnic University, China for example, in an image of a cylindrical pipe the top and bottom crown edges appear to be concentric even though the two circles are A multi-aperture telecentric lens is a single perfectly identical. (ii) Object shapes vary with telecentric lens with multiple aperture stops their distance from the center of the field of that enable it to capture multidirectional parallel view (FOV); for example, circles near the center light rays. Unlike conventional optical systems in of the FOV appear to be egg-shaped when photogrammetry or computer vision, which use moved toward the periphery. (iii) Some features a pair of stereo for three-dimensional or objects may be hidden by objects closer to (3-D) reconstruction, a multi-aperture telecen- the lens. In contrast, a telecentric lens removes tric lens uses image(s) obtained from the light these perspective or parallax errors to produce rays separated by different aperture stops for 3-D an orthographic projection that provides the reconstruction. A multi-aperture telecentric lens same magnification at all distances. Pioneered by is ideal for use in a variety of machine vision Moore (1973), telecentric lenses have been pur- applications, such as noncontact measurement sued by researchers in various settings (Watanabe and inspection systems, 3-D reconstruction of and Nayar 1996; Bai and Sadoulet 2007). In close-range targets, and intelligent vision systems particular, telecentric lenses are commonly used in robotic equipment. in machine vision applications, where software The telecentric lens is discussed here firstly. analysis is simplified and more accurate due to Detailed configurations of the multi-aperture the reduction of parallax. Imaging systems with telecentric lens are then discussed. Key aspects of telecentric lenses have made it possible to reach using a multi-aperture telecentric lens for 3-D dimensional measurement accuracies that can reconstruction – including its geometric model be better than those generated by contact and and an evaluation of the potential accuracy – are laser-based methods (Djidel et al. 2006). then presented. There are three types of telecentric lens. The first is an object-space telecentric lens, in which Telecentric lens the aperture stop is placed at the front focal plane of the lens, resulting in an entrance pupil loca- tion at infinity. A shift in the object plane does A telecentric lens is a compound lens used in not affect image magnification. Such lenses are an imaging system to make objects appear to be used in machine vision systems because image the same size independent of their location in magnification is independent of the objects’

The International Encyclopedia of Geography. Edited by Douglas Richardson, Noel Castree, Michael F. Goodchild, Audrey Kobayashi, Weidong Liu, and Richard A. Marston. © 2017 John Wiley & Sons, Ltd. Published 2017 by John Wiley & Sons, Ltd. DOI: 10.1002/9781118786352.wbieg0106 MULTI-APERTURE TELECENTRIC LENS

o

Figure 1 An idealized double telecentric lens. distance or position in the field of view. The stop placed at the common focal plane of the second is an image-space telecentric lens, in front and rear elements such that it only passes which the aperture stop is placed at the rear the light rays that are parallel to the optical axis. focal plane of the lens, resulting in an In summary, telecentric lenses have the fol- location at infinity. A shift in the image plane lowing advantages over conventional lenses: (i) does not affect image magnification. Such lenses constant magnification independent of shift in are used in image sensors that do not tolerate a object and/or image planes; (ii) low distortion, wide range of angles of incidence. The third is normally in the range of 0.1% for high-quality a double telecentric lens, in which the aperture telecentric lenses; (iii) reduction or elimination stop is placed at the common focal plane, result- of perspective error; (iv) increased image resolu- ing in both the entrance and exit pupils being tion; and (v) uniform image plane illumination. located at infinity. Shifting either the image However, they also have several disadvantages. or object planes does not affect magnification First, more optical elements are used than in given that double-telecentric systems are afocal. conventional lens systems due to the complex Double telecentric lenses have magnification design. Second, large aperture optical elements that is more precisely constant than those that in the region of telecentricity are required to are only object-space telecentric because the provide a nonvignetted FOV. The large aperture intersection position of the principal ray on and more optical elements lead to increases the detector does not change, which allows for in the cost and weight of the imaging system. the precise measurement of objects regardless Third, traditional telecentric lenses use fixed of their positions. Figure 1 illustrates an ide- focal lengths, which result in fixed FOVs. An alized double telecentric lens with two thin investigation of varying FOVs requires the use positive-powered elements, with the aperture of several fixed magnification lenses. Zinter

2 MULTI-APERTURE TELECENTRIC LENS and Sanson (2001) presented an endeavor to view between . Image disparities from develop a telecentric zoom lens for this type of multiple perspectives in the focal plane facilitated task. Finally, it is not possible to obtain depth the derivation of 3-D information. To take information from the telecentric images due to advantage of the orthographic projection of the the orthographic projection of the telecentric telecentric lens and the image disparities from lens. Despite the disadvantages inherent in the multiple, multi-aperture telecentric lenses have telecentric lens design due to its increased com- been investigated for use in 3-D reconstruction plexity, the numerous benefits make telecentric in recent years (Kim and Kanade 2011). lenses a popular choice in a variety of applica- A multi-aperture telecentric lens has mul- tions. The last drawback evokes the development tiple aperture stops rather than one, as in a of a multi-aperture telecentric lens. conventional telecentric lens. Figure 2 illustrates an idealized multi-aperture telecentric lens. There are two aperture stops O and O′ on Multi-aperture telecentric lens the focal plane, of which O is located at the focal point of the lens and the other O′ is at a In applications such as machine vision, the 2-D distance from the focal point. The aperture stop information and the 3-D depth information from O selectively passes light rays that are parallel to the scene must be extracted simultaneously.Stereo the optical axis because O is located at the focal vision using multiple cameras is the conventional point of the lens and only passes the rays. The ′ approach to infer depth information based on aperture stop O selectively passes light rays that are parallel to each other but not parallel to the parallax from multiple perspectives, which has ′ a long history in the fields of photogrammetry optical axis. Assuming O is an infinitely small aperture stop, the light rays selected by O′ are and computer vision. In recent years, a vari- ′ ety of techniques, such as motion parallax and parallel to the vector from the lens center to O depth-from-focus, have been implemented in (blue dashed line in Figure 2). 3-D imaging systems. However, these systems In a conventional stereo vision system, as illus- are relatively expensive and require complex trated in Figure 3, two cameras are used to form stereo images and the depth information of any calibration and geometric processing. As object in the scene can be derived from the dis- mentioned, the telecentric lens has the desirable parity of the same object imaged on the stereo property of orthographic projection, which images. From Figure 3, for an object point P, its makes it easier to measure or compare an object’s disparity is: physical length independently from its depth in relation to the camera. However, it is not possible f d = x1 + x2 = B (1) to obtain depth information from the image, as Z there is no foreshortening effect in telecentric where f is the of the camera, B is images. The multiple aperture technique has been the baseline between the stereo cameras, and Z used in imaging systems to provide 3-D informa- is the depth of point P. In equation 1, both B tion. For example, Fife, Gamal, and Wong (2006) and f are fixed, and thus the disparity d is linearly presented an comprising an array proportional to the inverse of the depth. of apertures each with its own local integrated In a simplified multi-aperture telecentric optics and pixel array. A lens focused the image lens system (1× magnification), as illustrated in above the sensor, creating overlapping fields of Figure 4, the distance between the two aperture

3 MULTI-APERTURE TELECENTRIC LENS ( ) stops is B, which can be considered the baseline Z d = 4 − B (2) between the two aperture stops. The focal length f of the lens is f. The image plane is located at a distance f from the focal plane. A spatial auxiliary In equation 2, both the baseline length, B,and coordinate system is used with its origin at focal length, f, are fixed; thus the disparity, d, the center of the image plane. Its z-axis aligns is linearly proportional to the depth, Z,ofthe object point. When Z = 4f,thedisparity,d, with the optic axis and points to the front of becomes zero. It should be noted that d can also the lens while its y-axis points up. Its x-axis is be negative. perpendicular to the plane determined by its A multi-aperture telecentric lens system has the z-andy-axes. For an object point P located at following major advantages over a conventional (0, Y, Z), the light rays selected by the aperture stereo vision system, as follows: ′ stops O and O pass through the points p1 and p2 at the image plane. The distance between p1 and p is the disparity of the same point P imaged on 2 1 The disparity given by the multi-aperture the image plane through the two aperture stops. telecentric system is linearly proportional The coordinates of p1 and p2 on the image plane to the depth, which is a critical difference can be determined as: from the conventional stereo vision system ( ( ) ) Z in which the disparity is proportional to (0, −Y) and 0, −Y − 4 − B the inverse depth, as already mentioned. f This property results in a more accurate and respectively. Therefore, the disparity is: stable determination of depth information.

o

o′

Figure 2 An idealized multi-aperture telecentric lens with two aperture stops O and O ′.

4 MULTI-APERTURE TELECENTRIC LENS

Image plane 1

f Camera 1 x 1

p 1

B Z P

p 2 x 2 f Camera 2

Image plane 2

Figure 3 A conventional stereo vision system. 2 The absolute location of an object can be inferred from Figure 3. This results point other than the depth can be eas- in less disparity in the image space, that is, ily determined based on the image captured less accuracy in depth determination. In by the aperture stop at the focal point, which the multi-aperture telecentric system, the maintains the advantages of a conventional angle between the rays remains constant telecentric lens. because the disparity is linearly propor- 3 When an object point goes farther, the tional to the depth, that is, the accuracy of angle between light rays in the conventional depth determination remains at the same stereo vision system becomes narrower, as level.

5 MULTI-APERTURE TELECENTRIC LENS

Y

Z

X P

Z o

ff B

o′ p 1 p 2

Lens Image plane

Focal plane

Figure 4 A simplified multi-aperture telecentric lens system (1× magnification).

4 Unlike the conventional stereo vision sys- Figure 4, for example, the geometric model can tem, the multi-aperture telecentric system be derived as: does not require any photometric calibration u +u ⎧X =− 1 2 or intensity adjustment between cameras ⎪ 2 because it is a single-lens imaging sensor. ⎨Y = (v1 ) (3) ⎪ ⎩Z = 4 − d f B 3-D Reconstruction and accuracy potential using a multi-aperture where (X, Y, Z) are the 3-D coordinates of telecentric system the object point P, (u1, v1)and(u2, v2)arethe image coordinates on the images acquired from the aperture stops O and O′, respectively, and The geometric model for 3-D reconstruction might differ among multi-aperture telecen- d = v1 –v2. From equation 3, it can be seen that tric system types. Taking the simplified multi- the geometric model of the simplified multi- aperture telecentric lens system illustrated in aperture telecentric lens system for 3-D

6 MULTI-APERTURE TELECENTRIC LENS reconstruction is much simpler than other parallel to the baseline. Automatic and reliable machine vision systems. image matching can be expected for the multi- Camera calibration is an important step in aperture telecentric system. 3-D reconstruction using conventional stereo To evaluate the potential measurement accu- vision systems, but with the multi-aperture racy of the multi-aperture telecentric system, telecentric lens system, because the telecentric the simplified multi-aperture telecentric lens lens offers orthographic projection and the pro- system illustrated in Figure 4 is used for analysis. jection center is at infinity, the lens distortions Through an error-propagation derivation based for telecentric lens systems are very small (nor- on equation 3, the measurement errors can be mally in the range of 0.1%). Therefore, only the calculated as: offset of the principal point needs to be consid- ⎧ ered in the calibration. This can be achieved by σ2 = 1 σ2 + 1 σ2 ⎪ X 4 u1 4 u2 following the general methodology of calibrat- ⎨σ2 =σ2 (4) Y v1 ing the principal point in photogrammetry or ⎪ 2 2 ⎩σ2 = (4f −Z) σ2 + f σ2 computer vision by adding additional terms to Z B2 B B2 d the right-hand side of equation 3. The accurate σ σ σ principal point can be determined through where X, Y,and Z are the standard errors several control points measured in object and of measurement in the X, Y,andZ directions, respectively; σ is the standard error of the base- image space. In a high-quality multi-aperture B line length; σ is the disparity measurement error; telecentric lens system, the calibration procedure d σ , σ ,andσ are the measurement errors of might be ignored, as the influences of lens u1 u2 v1 the image coordinates. σ can be determined by distortion and the offset of the principal point d the accuracy of image matching. According to on the measurement accuracy are negligible. a theoretical analysis, pixel-level image match- Image matching is another important issue ing (correlation) can reach an accuracy of about in 3-D reconstruction. In the multi-aperture one-third of a pixel. Given the homogeneity of telecentric system, image matching involves the imaging sensor: identifying the image coordinates on the images √ acquired from the two aperture stops, which σ =σ =σ = 1∕2σ (5) belong to the same object point. The illustration u1 u2 v1 d of the multi-aperture telecentric lens in Figure 4 As equation 4 shows, the measurement errors reveals that for an object point, the light ray in the X and Y directions are fixed because should pass through two aperture stops and build the CCD sensor is fixed, whereas the measure- a pencil of the planes containing the baseline. ment error in the Z direction (depth) is related Because the image plane cuts the pencil of the to the baseline length, depth, focal length, planes, all of the correspondences should be on and disparity. To investigate the measurement a line – the epipolar line. For an image plane error in the Z direction, a theoretical analy- parallel to the focal plane, all of the epipolar sis is conducted based on the third equation lines are parallel to the baseline and parallel to in equation 4, referring to the parameters of a each other. Therefore, image matching is much commercially available telecentric lens as listed in easier than it is in conventional stereo vision Table 1. systems, as it only needs to search for image cor- Because the best working distance is 500 mm, respondences along the epipolar lines, which are distances ranging from 300 mm to 1000 mm

7 MULTI-APERTURE TELECENTRIC LENS

2 1m 1.8 0.9m 0.8m 1.6 0.7m 1.4 0.6m 0.5m 1.2 0.4m direction (mm) Z 0.3m 1

0.8

0.6

0.4 Measurement error in 0.2

0 10 15 20 25 30 35 40 45 50 55 60 Baseline length (mm)

Figure 5 The influence of baseline length on measurement accuracy inthe Z direction.

Table 1 Parameters of the telecentric lens used baseline length is about 30 mm (about 14% of in accuracy analysis. the focal length), a measurement accuracy in the Focal length (mm) 216 Z direction of 0.1% (measurement error divided by distance) can be expected around the best Lens diameter (mm) 322 working distance. Best working distance (from object to lens) 500 It should be noted that only a theoretical anal- (mm) ysis has been performed here. The real perfor- Pixel size (μm) 3.3 mance of the multi-aperture telecentric system in 3-D reconstruction requires further investigation based on experiments using a real multi-aperture are investigated. The baseline length error is telecentric system. assumed to be 10 μm. The measurement errors There are several promising future research in the Z direction are calculated for each work- topics related to multi-aperture telecentric ing distance under different baseline lengths systems. For example, it remains difficult to from 10 mm to 60 mm. The results are shown in obtain two separate images from the two Figure 5. aperture stops while maintaining relatively sim- Figure 5 shows that as the baseline length ple optical geometry. Previous studies have used increases, the measurement error in the Z complicated lens elements to separate images direction decreases, and the farther the working from multiple aperture stops (Yahav and Iddan distance is, the larger the decline. When the 2000). A more productive approach would be

8 MULTI-APERTURE TELECENTRIC LENS to design the apertures to be controllable and Moore, Duncan. 1973. “Telecentric lens.” US Patent the two apertures to open successively to acquire 3773404 A, filed June 30, 1972, published Novem- two images separately. However, the actual ber 20, 1973. http://www.google.com/patents/ implementation of this mechanism depends on (accessed October 24, 2015). future research. In addition, real applications Watanabe, Masahiro, and Shree Nayar. 1996. “Tele- in machine vision may require telecentric lens centric optics for computational vision.” Computer Vision – ECCV’96, Fourth European Conference systems with better magnification capabilities on Computer Vision, April 14–18, Cambridge, (e.g., 10× or 20×). The geometric model for UK, 439–451. Berlin: Springer-Verlag. multi-aperture telecentric systems with magni- Yahav, Giora, and Gavriel Iddan. 2000. “Tele- fication capabilities will be more complicated, centric 3D camera and method.” US Patent as it will involve more optical elements, which 6091905 A, filed June 20, 1996, published requires future research. July 18, 2000. http://www.google.com/patents/ US6091905 (accessed October 24, 2015). Zinter, John, and Mark Sanson. 2001. “Telecentric SEE ALSO: Photogrammetry: 3-D from zoom lens.” Proceedings of the Society of Photo-Optical imagery Instrumentation Engineers, Vol. 4487 – Zoom Lenses III, 130. DOI:10.1117/12.450416.

References

Bai, Hanxiang, and S.P. Sadoulet. 2007. “Large- Further reading format telecentric lens.” Proceedings of the Soci- ety of Photo-Optical Instrumentation Engineers, Vol. Adelson, Edward, and John Wang. 1992. “Single lens 6667 – Current Developments in Lens Design and stereo with a plenoptic camera.” IEEE Transactions Optical Engineering VIII, 666705. DOI:10.1117/12. on Pattern Analysis and Machine Intelligence, 14(2): 734447. 99–106. Djidel, Slimane, Justyna Gansel, Heather Camp- Edmund Optics Inc. 2009. “The Advantages of bell, and Alan Greenaway. 2006. “High-speed, 3- Telecentricity.” Imaging Resource Guide, Section dimensional, telecentric imaging.” Optics Express, 5.1. http://www.edmundoptics.com/technical- 14(18): 8269–8277. resources-center/imaging/what-is-telecentricity/ Fife, Keith, Abbas Gamal, and H.-S. Wong. 2006. (accessed October 24, 2015). “A 3D multi-aperture image sensor architecture.” Zomet, Assaf, and Shree Nayar. 2006. “Lensless Imag- Proceedings of the Custom Integrated Circuits Confer- ing with a Controllable Aperture.” IEEE Confer- ence, CICC ’06, September 10–13, San Jose, CA, ence on Computer Vision and Pattern Recognition, June 281–284. New York: IEEE. 17–23, New York,NY, 339–346. New York:IEEE Kim, Junsik, and Takeo Kanade. 2011. “Multi- Computer Society. aperture telecentric lens for 3D reconstruction.” Optics Letters, 36: 1050–1052.

9