How to VisualSFM Jacob A. Morgan & Daniel J. Brogan Department of Civil & Environmental Engineering Colorado State University Fort Collins, Colorado January 2016 Contents 1 Introduction 1 1.1 Example Photographs.................................2 1.2 Tips for Acquiring Photographs............................3 2 Downloading VisualSFM4 2.1 VisualSFM.......................................4 2.2 PMVS/CMVS.....................................4 2.3 Customization.....................................5 3 VisualSFM Workflow7 3.1 Importing Images....................................7 3.2 Matching Images....................................8 3.3 Creating the Sparse Reconstruction..........................9 3.4 Specifying Ground Control Points........................... 10 3.5 Creating the Dense Reconstruction.......................... 12 3.6 Exporting the 3D Model................................ 13 4 Validation using CloudCompare 15 4.1 Downloading and Installing CloudCompare..................... 15 4.2 Importing Point Clouds................................ 15 4.3 Comparing Point Clouds................................ 18 1 Introduction This document was prepared by Jacob Morgan and Dan Brogan as a simple tutorial on how to download and use VisualSFM to create 3-dimensional (3D), scaled, and georeferenced point clouds. This tutorial includes a set of photographs as an example for the workflow. Structure from motion (SfM) is a technique that can be used for obtaining topographic data from digital imagery that is becoming rapidly popular in the geosciences. A number of soft- ware options are available for processing photographs to obtain 3D point clouds with XYZ coordinates, as well as RGB color values. Freely available software for SfM processing includes Bundler Photogrammetry Package (Snavely et al. 2006), SFMToolkit, PhotoSynth Toolkit, Python Photogrammetry Toolbox, VisualSFM (Wu 2013), and 3DF Samantha. Proprietary software includes Agisoft PhotoScan, Acute3D, PhotoModeler, and 3DF Zephyr. Web-based services include Photosynth, Arc3D (Tingdahl and Van Gool 2011), CMP SfM Web Service, and Autodesk 123D Catch. One of the more popular free options is VisualSFM, which was created by Changchang Wu by combining a few of his previous projects. More information about VisualSFM can be found on Dr. Wu's website (http://ccwu.me/vsfm). We are by no means experts in this technology. We suggest using the plethora of online resources to increase your own knowledge of these topics and methods. A good place to find general information about SfM methods for geomorphology is James Dietrich's Advanced Geographic Research blog (http://adv-geo-research.blogspot.com/). For general SfM references consider: • Furukawa, Y., and J. Ponce, 2010, Accurate, dense, and robust multiview stereopsis, IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(8): 1362{1376, doi:10.1109/TPAMI.2009.161. • Lowe, D.G., 2004, Distinctive image features from scale-invariant keypoints, International Journal of Computer Vision, 60(2): 91{110, doi:10.1023/B:VISI.0000029664.99615.94. • Snavely, N., S.M. Steitz, and R. Szeliski, 2006, Photo tourism: Exploring photo collections in 3D, in Proceedings of ACM SIGGRAPH 2006, ACM Transactions on Graphics, 25(3): 835{846, doi:10.1145/1141911.1141964. • Snavely, N., S.M. Steitz, and R. Szeliski, 2008, Modeling the world from internet photo col- lections, International Journal of Computer Vision, 80(2): 189{210, doi:10.1007/s11263- 007-0107-3. • Tingdahl, D., and L. Van Gool, 2011, A public system for image based 3D model gener- ation, in Computer Vision/Computer Graphics Collaboration Techniques, A. Gagalowicz and W. Philips (eds.), Springer-Verlag: Berlin, 262{273, doi:10.1007/978-3-642-24136- 9 23. • Ullman, S., 1979, The interpretation of structure from motion, Proceedings of the Royal Society of London, Biological Sciences, 203(1153), 405{426, doi:10.1098/rspb.1979.0006 • Wu, C., 2013, Towards linear-time incremental structure from motion, in 3DV 2013, 2013 International Conference on 3D Vision, Seattle, Wash., 127{134, doi:10.1109/3DV.2013.25. • Wu, C., S. Agarwal, B. Curless, and S.M. Seitz, 2011, Multicore bundle adjustment, 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 3057{3064, doi:10.1109/CVPR.2011.5995552. 1 For SfM references related to geoscience applications consider: • Clapuyt, F., V. Vanacker, and K. Van Oost, 2015, Reproducibility of UAV-based earth topography reconstructions based on Structure-from-Motion algorithms, Geomorphology, doi:10.1016/j.geomorph.2015.05.011. • Deitrich, J.T., 2014, Applications of Structure-from-Motion photogrammetry to fluvial geomorphology, PhD Dissertation, Department of Geography, University of Oregon, 109 p., available online. • Dietrich, J.T., 2016, Riverscape mapping with helicopter-based Structure-from-Motion photogrammetry, Geomorphology, 252: 144{157, doi:10.1016/j.geomorph.2015.05.008. • Favalli, M., A. Fornaciai, I. Isola, S. Tarquini, and L. Nannipieri, 2012, Multiview 3D re- construction in geosciences, Computers & Geosciences, 44: 168{176, doi:10.1016/j.cageo. 2011.09.012. • Fonstad, M.A., J.T. Dietrich, B.C. Courville, and P.E. Carbonneau, 2013, Topographic structure from motion: a new development in photogrammetric measurements, Earth Surface Processes and Landforms, 38(4): 421{430, doi:10.1002/esp.3366. • Gomez-Gutierrez, A., S. Schnabel, F. Berenguer-Sempere, F. Lavado-Contador, and J. Rubio-Delgado, 2014, Using 3D photo-reconstruction methods to estimate gully headcut erosion, Catena, 120: 90{101, doi:10.1016/j.catena.2014.04.004. • James, M.R., and S. Robson, 2012, Straightforward reconstruction of 3D surfaces and topography with a camera: Accuracy and geoscience application, Journal of Geophysical Research: Earth Surface, 117(F3), doi:10.1029/2011JF002289. • Javernick, L., J. Brasington, and B. Caruso, 2014, Modeling the topography of shallow braided rivers using Structure-from-Motion photogrammetry, Geomorphology, 213: 166{ 182, doi:10.1016/j.geomorph.2014.01.006. • Micheletti, N., J.H. Chandler, and S.N. Lane, 2015, Investigating the geomorphological potential of freely available and accessible structure-from-motion photogrammetry using a smartphone, Earth Surface Processes and Landforms, 40(4): 473{486, doi:10.1002/esp. 3648. • Westoby, M., J. Brasington, N.F. Glasser, M.J. Hambrey, and M.J. Reynolds, 2012, `Structure-from-Motion' photogrammetry: a low-cost, effective tool for geoscience ap- plications, Geomorphology, 179: 300{314, doi:10.1016/j.geomorph.2012.08.021. • Woodget, A.S., P.E. Carbonneau, F. Visser, and I.P. Maddock, 2015, Quantifying sub- merged fluvial topography using hyperspatial resolution UAS imagery and structure from motion photogrammetry, Earth Surface Processes and Landforms, 40(1): 47{64, doi: 10.1002/esp.3613. 1.1 Example Photographs The example dataset is comprised of photographs taken in Skin Gulch, Larimer County, Col- orado on 6 October 2015. The photographs were collected using a Canon Rebel T3i with a 24 mm prime lens mounted on a 12 ft painter's pole. A remote with a cable extension was used to initiate the camera shutter. 8 in × 8 in aluminum composite targets were pre-made using target designs from Agisoft PhotoScan and evenly placed on the ground throughout the region of interest. RTK-GPS surveying was used to determine the absolute position of these targets that can then be used to orient the 3D point cloud in real space. 2 1.2 Tips for Acquiring Photographs There are no hard and fast rules for taking photographs for SfM applications. There are, however, some tips that can increase the likelihood of successfully creating a point cloud from digital imagery. In general, convergent camera views produce better results than either divergent or parallel views (Fig.1). In fact, parallel views have been known to result in significant distortion in the point cloud. Figure 1: Convergent (left), divergent (center), and parallel (right) camera views. Graphics from J. Dietrich's Advanced Ge- ographic Research blog (http://adv-geo-research.blogspot. com/2014/02/camera-geometries-for-structure-from. html). Inevitably you will likely have some combination of these camera view types; that is okay. Once you have acquired your photographs it is best to go through them individually to remove any blurred images. NOTE! This tutorial has not been endorsed or approved by Changchang Wu or any other researcher associated with VisualSFM or any technologies on which it depends. 3 2 Downloading VisualSFM 2.1 VisualSFM VisualSFM software can be downloaded from the VisualSFM website (http://ccwu.me/vsfm). The homepage of the website has a section titled \Download," where the most recent version is available to download for Windows, Linux, and Mac (Fig.2). We use 64-bit Windows and so that is what this tutorial is set up to show; however, we believe the same general steps can be applied to other operating systems to achieve the same results. Figure 2: The VisualSFM website, showing both the Download section and Documentation section. Choose the appropriate operating system and version. The corresponding hyperlink will connect to a .zip folder containing all of the necessary files. Simply download and unzip this file to a convenient location on your computer. Free unzipping software is available via 7-Zip (http: //www.7-zip.org). After unzipping the file you should have a folder whose name corresponds to the operating system and version you chose (e.g. VisualSFM windows 64bit). This folder contains all the necessary files to simply run VisualSFM