Creating Simplified 3D Models with High Quality Textures

University of Wollongong Research Online Faculty of Engineering and Information Sciences - Faculty of Engineering and Information Sciences Papers: Part A 2015 Creating Simplified 3D oM dels with High Quality Textures Song Liu University of Wollongong, [email protected] Wanqing Li University of Wollongong, [email protected] Philip O. Ogunbona University of Wollongong, [email protected] Yang-Wai Chow University of Wollongong, [email protected] Publication Details Liu, S., Li, W., Ogunbona, P. & Chow, Y. (2015). Creating Simplified 3D Models with High Quality Textures. 2015 International Conference on Digital Image Computing: Techniques and Applications, DICTA 2015 (pp. 264-271). United States of America: The Institute of Electrical and Electronics Engineers, Inc.. Research Online is the open access institutional repository for the University of Wollongong. For further information contact the UOW Library: [email protected] Creating Simplified 3D oM dels with High Quality Textures Abstract This paper presents an extension to the KinectFusion algorithm which allows creating simplified 3D models with high quality RGB textures. This is achieved through (i) creating model textures using images from an HD RGB camera that is calibrated with Kinect depth camera, (ii) using a modified scheme to update model textures in an asymmetrical colour volume that contains a higher number of voxels than that of the geometry volume, (iii) simplifying dense polygon mesh model using quadric-based mesh decimation algorithm, and (iv) creating and mapping 2D textures to every polygon in the output 3D model. The proposed method is implemented in real-Time by means of GPU parallel processing. Visualization via ray casting of both geometry and colour volumes provides users with a real-Time feedback of the currently scanned 3D model. Experimental results show that the proposed method is capable of keeping the model texture quality even for a heavily decimated model and that, when reconstructing small objects, photorealistic RGB textures can still be reconstructed. Disciplines Engineering | Science and Technology Studies Publication Details Liu, S., Li, W., Ogunbona, P. & Chow, Y. (2015). Creating Simplified 3D Models with High Quality Textures. 2015 International Conference on Digital Image Computing: Techniques and Applications, DICTA 2015 (pp. 264-271). United States of America: The nI stitute of Electrical and Electronics Engineers, Inc.. This conference paper is available at Research Online: http://ro.uow.edu.au/eispapers/5447 Creating Simplified 3D Models with High Quality Textures Song Liu, Wanqing Li, Philip Ogunbona, Yang-Wai Chow Advanced Multimedia Research Lab University of Wollongong, Wollongong, NSW, Australia, 2522 fsl796,wanqing,philipo,[email protected] Abstract—This paper presents an extension to the KinectFu- information, which would increase the model complexity and sion algorithm which allows creating simplified 3D models with lower the rendering efficiency. In many cases, it is necessary high quality RGB textures. This is achieved through (i) creating to simplify a dense 3D model to achieve higher rendering model textures using images from an HD RGB camera that efficiency, especially for large scale rendering or on platforms is calibrated with Kinect depth camera, (ii) using a modified with limited processing powers such as cell phones and tablets. scheme to update model textures in an asymmetrical colour It is noteworthy that for many 3D models generated from volume that contains a higher number of voxels than that of the geometry volume, (iii) simplifying dense polygon mesh model existing 3D reconstruction systems, the quality of model tex- using quadric-based mesh decimation algorithm, and (iv) creating ture is directly related to the model complexity. Furthermore, and mapping 2D textures to every polygon in the output 3D decimation of the polygon mesh models will degrade the model model. The proposed method is implemented in real-time by texture. means of GPU parallel processing. Visualization via ray casting of both geometry and colour volumes provides users with a real- time feedback of the currently scanned 3D model. Experimental II. RELATED WORK results show that the proposed method is capable of keeping The problem of reconstructing geometry and texture of the model texture quality even for a heavily decimated model and that, when reconstructing small objects, photorealistic RGB real world has remained an active challenge in the field of textures can still be reconstructed. computer vision for decades. We now review some of the extant aproaches and the associated results. I. INTRODUCTION Conventional 3D reconstruction approaches usually do not Generating 3D models based on real-world environments consider model texture information, or represent model texture and with high quality textures is of great significance to many in a simple way. Chen and Medioni [5] average overlapping fields including civil engineering, 3D printing, game design, range images and connect points based on simple surface movie, virtual reality and preservation of cultural heritage topology to create polygon mesh models; model textures are artefacts. Various computer vision-based approaches have been totally ignored. Turk and Levoy [23] propose mesh zippering proposed to create 3D models and deal with the associated as an extension to Chen and Medioni’s work; they stitch classical problems such as simultaneous localization and map- polygon meshes to create 3D model without textures. Some ping (SLAM), and structure-from-motion (SFM). To date, point-based 3D reconstruction methods [20][24][12][22] use impressive progress has been made in this domain [6][2][8]. simple unstructured point representations that are directly Largely, many approaches use visual key points to build 3D captured from many range imaging devices. These methods models leading to sparse point cloud based 3D reconstruction. do not model connected surfaces which usually requires post Conventional dense reconstruction method [9][28] on the other processing to generate polygons. In most popular point-based hand usually require professional sensors such as high-fidelity 3D model rendering techniques [11][19][30], textures are sim- laser scanners or time-of-flight (ToF) depth cameras which are ply represented by colours attached to each point in the model. very expensive. The release of low-cost RGB-D cameras like the Microsoft The release of commodity RGB-D cameras such as the Kinect™ and Asus Xtion™ opens up new opportunities to Microsoft Kinect™ and Asus Xtion™ has made dense 3D 3D reconstruction in terms of providing easy access to depth reconstruction possible at an affordable cost. This, along with imaging. KinectFusion [17][15] adopts volumetric data struc- the KinectFusion algorithm [17][15] , has enabled real-time ture to store reconstructed scene surface [7][14] and realise dense 3D reconstruction using a low-cost RGB-D camera real-time reconstructions using GPU. Although model textures and GPU parallel processing. Subsequent efforts by other re- are not considered in the original KinectFusion algorithm, it searchers have led to the development of several KinectFusion- inspires multiple volumetric 3D reconstruction methods using based methods [26][21][4][18] that allow efficient 3D re- commodity RGB-D cameras that try to create dense polygon construction on a large scale and with higher reconstruction mesh with RGB textures. Among these KinectFusion-based quality. However, current KinectFusion-based methods tend methods, an open source C++ implementation of KinectFusion to deliver 3D models with high quality geometry but low from Point Cloud Library (PCL) [1], Whelan et al. [26] and quality texture. In other words, the works on improving model Bylow et al. [4] use a colour volume to store and update textures are less advanced. Moreover, 3D models created by RGB texture information. In their methods, model textures on dense 3D reconstruction usually contain significant redundant reconstructed 3D models are represented by colours on model vertices, and these colours are linearly interpolated within each and extended. Given updated geometry and colour volumes, polygon. This popular 3D model texture representation can 3D polygon mesh model with textures can be extracted using also be easily found in many other 3D reconstruction methods marching cube algorithm [16]; a method that aims to produce [21][25][18]. While the texture representation is straightfor- triangle polygons for visualization. ward and easy to implement, simplifying the model inevitably degrades the texture quality because it is determined by the IV. IMPROVED METHOD number of vertices in the model. The workflow of our improved method is shown in Fig1. Zhou and Koltun [29] reconstruct 3D models with high The similarity with the KinectFusion process is noticeable. quality colour textures using a commodity depth camera and However, in order to create simplified 3D models with high an HD RGB camera. HD model textures are refined using quality textures, the following major improvements are made: optimized camera poses in tandem with non-rigid correction functions for all images. Because model textures are also • HD RGB camera is added to achieve higher model represented by colours assigned to each vertex, the generation texture quality; of high quality textures requires increasing the number of • Colour volume integration scheme is revised to pro- vertices and polygons of a 3D model. This results in increased vide an asymmetrical

Load more