From: AAAI-93 Proceedings. Copyright © 1993, AAAI (www.aaai.org). All rights reserved. Range Estimation From Focus Using a Non-frontal Imaging Camera* Arun Krislwan and Narendra Ahuja l3eckina~n Institute University Of Illinois 405 North Makhews Ave., Urbana. IL 61SO1, U.S.A. e-mail: arunki~visioii.csl.uiric.eclu, aliuja.~&~ision.csl.uiuc.eclu Abstract (lard mechanical adjustment of image plane loca- t,ion, and Curther, integrakion of the only remain- This paper is concerned with active sensing of ing mecllanical action of camera panning with fo- range information from focus. It describes a new cusing a.nd range estima.tion. Thus, ima.ging ge- type of camera whose image pla.ne is uot perpen- ometry a.nd optics a.re exploited to replace explicit dicular to the optical a,xis as is sta.nda.rd. This sequential computation. Since the camera imple- special imaging geometry eliminakes t’he usua.1 fo- ments a range from focus a,pproacli, the resulting cusing need of image plane movement. Camera. estinia.tes have tlke following characteristics as is movement, which is anywa,y necessary to process brue for any such a.pproa.ch [Das and Ahuja, 1990; large visual fields, integrakes panning, focusing, Das a.ncl Ahuja., 1992b]. The scene surfaces of in- and range estimakion. Thus the two stantla.rd terest must have texture so ima.ge sharpness can mechanical actions of focusing a.nd panning arc be nleasured. The confidence of the estimates im- replaced by pa.nning alone. Ra.nge estimakion is proves wi 1,h the amount of surface texture present. done at the speed of pa.nning.‘ An imple~nentat~ioll l:urther, tlie i*elial,ilit8y of estimates is inherently of the proposed camera. design is descril.wcl ant1 a function of the rauge to be estima.ted. IIow- experiments with range estimakion are reported. ever, range estimation for wide scenes using the proposed approach is faster than any traditional INTRODUCTION range from focus a.pproa.ch, thus eliminating one of the major clra.wbacks. This pa.per is concerned with active sensing of The next section describes in detail the pertinence range information from focus. It describes a new of range estimation from focus, and some prob- type of camera which integrakes the processes 01 lems t,lia.t cha.racterize previous range from focus image acquisition and range estimakion. The cam- approaches arid serve as the motivation for the era ca,n be viewed as a computa.tiona.1 sensor wllich work reported in I,his paper. The following sec- can perform high speed range estima tiou over tion 1,resent.s the new, proposed imaging geometry large scenes. Typically, the field of view of a call)- ~liose centerpiece is a. Glting of the image plane era is much smaller t&an the entire visi1a.l field of from the st#a.nclnrd frontoparallel orientation. It interest. Consequently, the camera. must pan t#o shows how the design achieves the results of search sequentially acquire images of the visual field, a over focus with high computational efficiency. The part at a. time, and for each pa.rt colllpute range rest of t(lie paper presents a. range from focus al- estimates by acquiring a.nd searching illlages over gorit lun that uses the proposed camera, followed many image plane locakions. Using the proposed by the results of an experiment demonstraking the approach, range can be computed a.t 18hc spee(l of fea.sibiIity of our method. The last’ section presents panning the camera. some concluding rema.rks. At the heart of the proposed design is a ctive coil- trol of imaging geometry to elimin a 1.(f tlw stall- BACKGROUND & MOTIVATION *The support of the National Science Foundatioll il.lld Range Estimation From Focus and Its Defence Advanced Research Projects -4gellcy under grant Utility IRI-89-02728 and U.S. Army Advance Const.ruct.io~~ l’ech- nology Center under grant. DXAL OS-87-Ii-OOOGis gril(c- Focus I~nsecl methods usually obtain a depth esti- fully acknowledged. nlate of a scene point by inecha.nically relocaking 830 Krishnan the ima*ge plane, thereby va.rying the focus dis- image pla.ne, while its sha.pe will be the same as tance (v). When the scene point a.ppears in sharp tha.t of the ima.ge plame. Figure l(a) shows the SF focus, the corresponding u a.ncl 2) values satisfy surface for a rectangu1a.r ima.ge plane. the standard lens law: d + $ = i. The depth As the ima.ge plane distance from the lens, V, value u for the scene point ca.n then be calcu- is changed, the SF surface moves away, or to- lated by knowing the values of the focal length and ward the ca.mera. As t,he entire range of v val- the focus distance [Pentland, 1987; Darrell and ues is tra.versed, the SF surface sweeps out a cone Wohn, 1988; Ens and Lawrence, 19911. To deter- shaped volume in three-dimensional space, hence- mine when a scene is ima.ged in sharp focus, sev- forth called the SF cone. The vertex angle of eral autofocus methods have been proposed in the the cone represents the magnification or scaling past [Horn, 1968; Sperling, 1970; Teneba,uill, 1971; achieved and is proportional to the f value. Fig- Jarvis, 1976; Ligthart a.nd Groen, 1982; Sch1a.g ure l(b) shows a frustum of the cone. et al., 1985; Krotkov et nl., 1986; Krotkov, 1986; Only those points of the scene within the SF cone Darrell and Wohn, 1988; Na.yar and Nalca.gawa, a.re ever imaged sharply. To increase the size of 19901. the imaged scene, the f va.lue used must be in- Like any other visual cue, ra,nge estimation from creased. Since in practice there is a limit on the focus is reliable under some conditions and not so usable range of f value, it is not possible to image in some other conditions. Therefore, to use the the entire scene in one viewing. The camera must cue appropriately, its shortcomings and strengths be panned to repea.tedly image different parts of must be recognized and the estimat#ion process t#he scene. If the solid angle of the cone is w, then should be suitably integra.tecl with other pro- to ima.ge an entire hemisphere one must clearly cesses using different cues, so as to achieve su- use at, least ?n viewing directions. This is a crude perior estimates under broa.cler conditions of in- lower boundy5ince it does not take into account terest [Abbott and Ahuja, 1990; Iirot~kov, 1989; the constra.intls imposed by the packing and tes- Das and Ahuja, 1992a.l. When a.ccurat#e depth in- sellability of the hemisphere surfa,ce by the shape formation is not needed, e.g., for obstacle avoicl- of the camera visua.1 field. ante during navigation, range estimates from fo- If specialized hardwa.re which can quickly identify cus or some other cue a.lone may suflice, eve]] focused regions in the image is used, then the time though it may be less accurate than that, obt,ainc4 required to obtain the depth estimates is bounded by an integrated ana.lysis of mult~iple cues. by tl1a.t required to make a.11pan angle changes and to process t,he data acquired for ea.ch pan angle. Motivation for Proposed Approach The goal of the a,pproach proposed in this paper The usual range from focus a.lgorithms involve two is to select t,lie appropriate 21 value for each scene point, without conducting a. dedicated mechanical mechanica. actions, those of panning a.nd for each search over a.11‘ [I values. The next section describes chosen pan angle finding the best 21 value. These how this is accomplished by slightly changing the steps ma.ke the algorithms slow. The purpose of camera geometry and exploiting this in conjunc- the first step is to acquire da.ta. over the ent,ire t.ion with the pan motion to accomplish the same visual field since cameras typica.lly have na.rrower field of view. This step is therefore essent,ia.l to result, as traditionally provided by the two me- chanical mot,ions. construct a range map of the entire scene. The proposed a.pproach is motiva,ted prima.rily by t#he desire to elimina.te the second step involving me- A NQN-FRONTAL IMAGING chanical control. CAMERA Consider the set of scene points that will be im- The following observations underlie the proposed aged with sharp focus for some const#a.nt value of approach. In a. normal camera, all points on the focal length and focus dista.nce. Let us call this image plane lie a.t, a fixed distance (v) from the set of points the SF surface’. For the conven- lens. So a.11scene points are always imaged with a tional ca,se where the image is formed on a, plane fixed va.lue of V, regardless of where on the image perpendicular to the optica. axis, and assuming plane they are ima.ged, i.e., regardless of the cam- that the lens has no optical a.berra.tions, this SF era pan angle. If we instea.d have an image surface surface will be a surface that is a.pproxima.t,ely pla- such that the different image surface points are at nar and normal to the optica. axis. The size of SF different dista.nces from the lens, then depending surface will be a scaled version of the size of the upon where on the imaging surface the image of a lActua.lly, the dep th-of-field effect will cause t.he SF sur- scene point is formed (i.e., depending on the pan face to be a 3-D volume.