Sequential Estimation in Robot Vision*
Total Page:16
File Type:pdf, Size:1020Kb
PEER.REVIEWED ARIIGTE SequentialEstimation in RobotVision* Armln Gruen and Thomas P. Kersten Abstract of on-line bundle triangulation is closely tied to tle image Highly time-constrained robot uision applications rcquirc a coordinate measurement process involving a human opera- careful tuning and optimized interaction of a system's hard- tor. The main purposo of this fast sequential estimation is ware components, algorithmic complexity, software engineer- that of blunder detection at an early stage of the measure- ing, and task performance, The high accwacy processing of ment process with the utilization of quick r€measurement possibilities and better blunder control capabilities.Impor- full-ftame image sequencesfor image analysis and object space positioning is very time consuming. In both of tant characteristics of this application are, on ttre one hand, feature ("solution these processes,sequential estimation algorithms offer valua- t}le constaatly varying size of the state vector vec- ble alternatives to simultaneous approaches. This paper in- tor" in least-squaresadjustment terminology) of bundle ad- troduces an efficient estimation algorithm based on Givens justment consisting of tle exterior orientation pararrretersof photographs,the object point parameters,and, possibly, ad- transformations for use in point positioning and updating camera orientation data. In a test, an easy-to-use standard ditional parametprs for self-calibration. On tle other hand, the full covariance matrix of all system parameters is, if at video camera has been applied for image frame generation. The results of camera calibration and an accuracy test using all, only needed at the termination of the process. A third a 3o testfield are presented. The computing times of sequen- distinctive characteristic is the high and typically sparse pat- tial point positioning and camera ofientation arc given and terns of the matrices involved in tle estimation procedure (design matrix of observation equatious and normal equation in part compared to the values for the simultaneous adjust- ment. This clearly indicates the superior performance of the matrices of least squares).Given ttrese system characteristics, sequential procedurc. a number of sequential estimation algorithms have been compared to each other in the past. First, the Triangular Fac- lntroduction tor Update (rru) algorithm, which updates directly the upper Image sequencesplay an important rolo in photogramme$, triangle of tle reduced normal equations, was fou-ud to per- machine vision, aad robot vision. While in classical photo- form much better than the Kalman form of updating, both in grammetry, especially in aerial applications, data acquisition terms of computing times and storage requirement (Gruen, and processingis largely separated,this is no longer the case 1982; Wyatt, 1982).Later, the Givens transformationswere in modern applications where non-photographicsensor tech- found to be superior, in general, to the TFU (Runge, 1987; nology and digital processing techniquos are employed. Fast Holm, 1989), botl in computational performance and in the methods for data reduction are required, in particular, in ease of mechanization and software implementation. In the highly time-constrained robotics applications, but are also meantime, t}.e Givens algorithm has been implemented in a very often of advantage in less time-critical machine vision number of systems (Edmundson, 1991; Kersten et al., 7992). and digital photogrammetric projects. The classical data re- Already in the mid 80s, Gruen (1985a) envisioned semi-auto- duction processconsists of two major stages:image meas- matic or fully automatic digital real-time triangulation sys- urement and three-dimensional (ao) point positioning. tems for the future. We argue nowadays that machine vision These processcomponents are in general separatedfrom and, in particular, robot vision could draw substantial advan- each other. In each case,simultaneous algorithms can be re- tages from theso sequential approaches. formulated into sequential form for better time perform- This fact has been obviously acknowledgedby the com- ance, puter vision community, where, among others, two recent In image processing, well-known sequential formulations dovelopments are of particular interest. In Matthies ef o1, exist for incremental convolution operations (used in Iinear (1989), the Kalman filter is used to estimate a depth map filtering, resampling,image pyramid generation,etc.); in im- from image sequences.Typical for this approach is that age analysis, they are applied in the pixel location transfor- depth estimates and uncertaintv values are comDuted at each mations in orthophoto production (Baltsaviaset aI.,'J.997) pliel of miniffed (256 by 256 p;ls) video framei and that and in form of the Kalman filter in the tracking of line seg- these estimates are refined incrementally over time. The ments in image space(Deriche and Faugoras,1990). good performance of the Kalman filter is due to the fact that A well known example is that of on-line triangulation only one parameter (depth value) constitutes the state vector using sequential estimation techniques in point positioning and is updated. Errors in orientation and calibration of the with aerial photographs. Here the computational procedure video frdrnes and thus correlations between different ele- ments of the depth map are not considered. Also, only lat- eral motion of ttre CCDcamera is assumed. *Presented at the ISPRSCongress, Commission V, Washington, D.C., Photogrammetric Engineering & Remote Sensing, 2-14 August 1992. Vol. 0r, No. 1, January1995, pp.75-82, Institute of Geodesy and Photogrammetry, Swiss Federal In- 0099-7772/ 95/6 1 0 1-75$3.00/0 stitute of Technology,ETH-Hoenggerberg, CH-8093, Zurich, 1995 American Society for Photogrammetry Switzerland. and Remote Sensing PE&RS PEER.REVIEWED ARIICTE Zhang and Faugeras(1990) use the Kalman update Least-SquaresApproach for Estlmatlon mechanism to track object motion in a sequenceof stereo The Gauss-Markovmodel is the estimation model most frames.They track object features,called ntokens" (line seg- widely used in photogrammetriclinear or linearized estima- ments for example),in eD spacofrom frame to frame and es- tion problems. An observationvector I of dimension n X 1 is timate the motion Darametersof these tokens in a unified functionally related to a u X 1 parameter vector x through way (they also inte-gratea model of motion kinematics). (1) Their statevector consists,thus, of three angular velocity, l-e:Ax three translationalvelocity, and three translational accelera- The design matrix A is an n X u matrixwith n > u and Rank tion parametersof each object,and, in addition, six line seg- (A) : u. There is no need to work with rank-deficientdesign parameters gD ment for each line representation.As before, matrices in on-line triangulation. Rank deficient systems, the tokens are treated independentty here, which allows the causedby missing observations,generally do not allow for a statevector for estimation to remain small in size and may comprehensivemodel check. Observationsshould be accu- result in straightforwardparallelization for any number of to- mulated until the system is regular and can be solved using Kens. standard techniques.For rank deficiency causedby incom- (e.9,, Although we aro aware that very fast a video rate plete datum, seeGruen (rgesa). Sequentialleast-squares esti- of 25 Hz) solutions to tracking problems with affordable mation with pseudo-inversesis very costly (compare computer hardware still require substantialsimplification of Boullion and Odell (1977),p. 50 ffj. The vector e represents the measurementproblem at hand, we neverthelesspresent tho true errors. With the expectation E(e) = 0 and the disper- in this paper the generalsolution for sequentialpoint posi- sion operator D, we get tioning, basedon the bundle solution. The solution is object- point based.Other featuresmay be derived from these gD E(l) = 6*, (2a) point measurements,Any simplification in measurementar- D(l) : Cr : (zoP-t, and (2b) rangement,as for instancenon-moving sensors,may readily D(e) = C""= Ca. (zc) be derived from the generalconcept. Our solution may in- clude self-calibrationparameters for systematicerror model- The estimation of x and oo2is usually attempted as unbiased, ing and statisticaltests for blunder detection.The sequential minimum variance estimation performed by means of least estimation procedure applies Givens transformationsfor up- squares, and results in dating of the upper triangle of the reduced normal equations pammeter vector * : (Ar Pa;-' 4t (3a) (Blais, 1983; Gruen, 1985a).We use Givens transformations residualvectorv = Ai - I, and"t, (3b) as opposedto a Kalman update becausein robotics t}le co- vrPv variance update of the parametervector is not required at variance factor (2o (3c) every stageand then, if at all, only at relatively sparseincre- r ments, Moreover, the varying size of the parametervector The architecture of A is determined by the type of triangula- (addition and deletion of new object points, addition of tion method used. As explained previously, we chosethe frame exterior orientation parameters)leads to very poor bundle method for the pu{pose of generality and rigidity. computational performance of the Kalman filter. For bundle adjustment, Equation 1 can be written as The approachpresented here treats only the so point po- sitioning problem in a sequentialmode. A combination of se- -e:4x+&t-l;P (4a) quential algorithms