Automatic Orientation of Large Blocks of Oblique Images
Total Page:16
File Type:pdf, Size:1020Kb
International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XL-1/W1, ISPRS Hannover Workshop 2013, 21 – 24 May 2013, Hannover, Germany AUTOMATIC ORIENTATION OF LARGE BLOCKS OF OBLIQUE IMAGES a, b b E. Rupnik ∗, F. Nex , F. Remondino a GEO Department, Vienna University of Technology - [email protected] b 3D Optical Metrology (3DOM) Unit, Bruno Kessler Foundation (FBK), Trento, Italy (franex,remondino)@fbk.eu, http://3dom.fbk.eu Commission WG III/4 KEY WORDS: Urban, Orientation, Automation, Bundle, Block, Multi-sensor, High resolution ABSTRACT: Nowadays, multi-camera platforms combining nadir and oblique cameras are experiencing a revival. Due to their advantages such as ease of interpretation, completeness through mitigation of occluding areas, as well as system accessibility, they have found their place in numerous civil applications. However, automatic post-processing of such imagery still remains a topic of research. Configuration of cameras poses a challenge on the traditional photogrammetric pipeline used in commercial software and manual measurements are inevitable. For large image blocks it is certainly an impediment. Within theoretical part of the work we review three common least square adjustment methods and recap on possible ways for a multi-camera system orientation. In the practical part we present an approach that successfully oriented a block of 550 images acquired with an imaging system composed of 5 cameras (Canon Eos 1D Mark III) with different focal lengths. Oblique cameras are rotated in the four looking directions (forward, backward, left and right) by 45◦ with respect to the nadir camera. The workflow relies only upon open-source software: a developed tool to analyse image connectivity and Apero to orient the image block. The benefits of the connectivity tool are twofold: in terms of computational time and success of Bundle Block Adjustment. It exploits the georeferenced information provided by the Applanix system in constraining feature point extraction to relevant images only, and guides the concatenation of images during the relative orientation. Ultimately an absolute transformation is performed resulting in mean re-projection residuals equal to 0.6pix. 1 INTRODUCTION metric applications (point cloud generation, manual plotting). Although aerotriangulation has a long tradition, most of com- mercial solutions were designed only for nadir images (Jacobsen, The use of oblique images was first adopted in the 1920s for 2008), hence cannot cope with different camera viewing angles surveillance and reconnaissance purposes and during the last cen- and varying scale within images. Several papers dealing with the tury it was mainly used for military applications. Only in the oblique images’ orientation have been already presented (Wiede- last decade oblique imagery has become a standard technology mann and More, 2012, Gerke and Nyaruhuma, 2009) proposing for civil applications, thanks to the development of airborne dig- the use of additional constraints within bundle adjustment (rel- ital cameras and in particular the development of multi-camera ative position between images, verticality of lines in the scene, systems, as proposed by many companies (Leica RCD30, Pic- etc.), or simply aligning the (not adjusted) oblique cameras to the tometry, Midas, BlomOblique, IGI, UltraCam Osprey, etc.). The (adjusted) nadir ones with the use of GNSS/IMU information. virtue of oblique photography lies in its simplicity of interpre- Merely a single contribution has succeeded to automatically ori- tation and understanding for inexperienced users. These qual- ent a large block of oblique images with a commercial software ities allowed the use of oblique images in very different appli- (Fritsch et al., 2012). cations, such as monitoring services during mass events and en- In this paper, a fully automatic methodology for simultaneous vironmental accidents, (Petrie, 2008, Grenzdorfer¨ et al., 2008, orientation of large datasets of oblique and nadir images is de- Kurz et al., 2007), building detection and reconstruction (Xiao et scribed. The presented workflow relies only upon open-source al., 2012), building structural damage classification (Nyaruhuma software: a developed tool to analyse image connectivity and et al., 2012), road land updating (Mishra et al., 2008) and ad- Apero (Pierrot-Deseilligny and Clery, 2011) to orient the im- ministration services (Lemmens et al., 2008). What is more, age block. The connectivity tool was developed to exploit the oblique imagery can provide a great improvement in city mod- georeferenced information provided by the Applanix system in elling (Wang et al., 2008) and in some cases it can serve as a constraining feature point extraction to relevant images only, and good alternative to mobile mapping surveys or airborne LiDAR guideing the concatenation of images during the relative orienta- acquisitions. The reason is the density and the accuracy of point tion. clouds that can be generated with matching techniques (Fritsch In the following sections theoretical background of different least et al., 2012, Gerke, 2009, Besnerais et al., 2008). What is more, squares approaches to bundle block adjustment will be given. a filtered point cloud can be used to produce reliable meshes for Following this, possible ways of multi-camera orientation will visualization purposes (i.e. AppleC3). be discussed. The methodology part will include a detailed de- So far, most prominent cities in the world have been covered with scription of the connectivity algorithm as well as obtained results. georeferenced oblique images and these flights are usually re- Finally the conclusion and the future outlook of this work will be peated every few years to be always up-to-date (Karbo and Sim- presented. mons, 2007). Abundance and practical utility of this data is ap- parent, however, automated processing is still a challenge. This is primarily because complex image configurations are imposed by the acquisitions and because of shortcomings of on-board direct orientation sensors that do not satisfy the strict requirements of 299 International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XL-1/W1, ISPRS Hannover Workshop 2013, 21 – 24 May 2013, Hannover, Germany 2 THEORETICAL BACKGROUND where, 2.1 Least squares adjustment 2 3 ∂f1 (x) ∂f1 (x) ∂x1 ··· ∂xm Many measured quantities involved in Geomatics problems are 6 . 7 J = 6 . 7 random variables which normally assume different values accord- 4 . 5 ∂f ∂f ing to the different (and repeated) measurements performed to N (x) N (x) ∂x1 ··· ∂xm obtain them. Therefore a set of measurements (or observations) l can be expressed as l = l + v with l the adjusted measures and v r(x) = y f(x0) Jdx = e Jdx (4) the residuals (or corrections) to the measures. − − − Behind each problem there is a mathematical model, composed JTJdxGN = eJ (5) of two parts: a functional model (it describes the geometric char- acteristics of the problem) and a stochastic model (it deals with xk+1 = xk + αkdx (6) the statistical properties of all the element involved in the func- Let’s denote the relationship between observations y and parame- tional model). ters as f(x) . Setting (3) in (2) results in (4). Next, when inserted To increase the accuracy and provide an error check, the number in (1) and equating its derivative to zero it will give (5) (Engels et of measured quantities (observations l) in the functional model al., 2006). Provided that the Jacobian is of full rank, the solution is normally larger than the number of the unknowns - redundant to (5) gives us the search direction. It becomes also clear from measurements. Consequently we have an over-determined sys- (5) that the left hand-side of the equation represents normal equa- tem and a solution of the problem is derived with a least squares tions i.e. the search direction is the solution of linear least squares adjustment. The least squares method minimizes an objective and can be solved with direct algorithms. Once the direction is function (1), being the sum of the squares of the residuals (2) retrieved, an update dx (a.k.a. Gauss-Newton step) is calculated of the available observations. with (6), where the scalar α is called the step length and is chosen according to specified conditions (Nocedal and Wright, 2006). c(x) = r(x)Tr(x) (1) The Levenberg-Marquadt method is a further modification of the r(x) = y f(x) (2) Newton method. Here, the step generation is carried out by a − trust-region method rather than line-search as in Gauss-Newton. There are different techniques by which least squares can be solv- The algorithm builds a spherical region around current parameter ed. Firstly the model behind the problem must be identified with estimates and looks simultaneously for the direction and length of its number of observations, underlying parameters (unknowns) the step. On the contrary, line-search methods look independently and the relationship between them. If the model is linear, the ob- for the direction and α corresponding to the length (Nocedal and jective function is quadratic and independent of unknowns there- Wright, 2006). In practice, the normal equations in (5) are in- fore it is convex. The solution is then obtained with direct meth- teractively augmented by a damping factor (7). Owing to that, ods (Gauss elimination, Cholesky, QR, SVD, conjugate gradient) the algorithm operates as the Gauss-Newton for λ 0, steepest from the function’s gradient. descent direction for λ , and something in-between→ for the In case of non-linear least squares problems (e.g. collinearity remaining λ values. The→ damping ∞ is raised if the cost (1) is not model), it is practical to obtain linear equations. Linearization reduced, otherwise it is decreased. Through manipulation of the implies that approximate values for all parameters are known and damping, rank-deficient Jacobians can also be handled (Lourakis the most optimal values are computed in an iterative framework and Argyros, 2005).