MeshLab and Arc3D: Photo-Reconstruction and Processing of 3D meshes P. Cignoni, M Corsini, M. Dellepiane, G. Ranzuglia, (Visual Computing Lab, ISTI - CNR, Italy) M. Vergauven, L. Van Gool (K.U.Leuven ESAT-PSI - ETH Zürich D-ITET-BIWI) Intro • Arc3d + MeshLab – a complete free software pipeline for the 3D digital acquisition – based on standard photographic equipment –Arc3D • A free web based 3D reconstruction service, you upload photos and you get sequences of aligned depth maps – MeshLab • An open source mesh processing system for cleaning, aligning and merging meshes and range maps. Arc3D: Architecture 1. Record a sequence of images of a scene or object 2. Upload the images to the ARC server 3. The server computes the 3D reconstruction 4. Download the results from the ARC website 5. Process and Visualize the results with MeshLab Arc3D: How it works QuickTime™ and a TIFF (Uncompressed) decompressor are needed to see this picture. • The entire process is based on finding matches between images Arc3D: the output • For each submitted image a depth image is reconstructed – an image with the distance from the camera for each pixel QuickTime™ and a QuickTime™ and a TIFF (Uncompressed) decompressor TIFF (Uncompressed) decompressor – a quality estimation for each pixel are needed to see this picture. are needed to see this picture. • All these depth images must be – cleaned and filtered – integrated into a full model. A typical Epoch - CI Tool • MeshLab - an epoch result – Developed at ISTI - CNR • Just an example of the kind of tools that we are targeting for building the pieces a common infrastructure tool chain •Domain – Digital representations of 3D shapes •Objectives – Linking together input and output of different CH apps – Adapting data from one CH app to another (mesh processing!) MeshLab • Generic mesh processing tool • Open source GPL avail for win, mac and linux (scheduled for inclusion inside debian-science) – The system relies on a large GPL library for mesh processing • Aimed to the (almost) automatic processing of large unstructured 3d models – acquired 3D models are different from human built models • No structure, no particular need of a scene graph • Large datasets : millions of primitives MeshLab • Able to manage many different kind of mesh formats • Input format PLY, STL, OFF, OBJ, 3DS, COLLADA, PTX, X3D, VRML • Output format PLY, STL, OFF, OBJ, 3DS, COLLADA, X3D, VRML, DXF MeshLab dissemination • License GPL • Code size: 35k + 65k LOC (>20 py estimated ) • V1.0.0 (march 2007) – Downloaded >30k times, – Web site served more than 400k pages – more then 2000 faithful users (users that opened more than 50 meshes), hundreds of university and research centers using it all over the world • V1.1.0 (feb 2008) Downloaded 5k times in three weeks MeshLab typical tasks • Filtering – Removal of outliers according to various heuristics • Smoothing out noise – Various fairing algorithms • Editing – Selection of parts and removal of non interesting portions of the acquired data Remeshing • Subdivision Surface (loop and butterfly) • Reducing complexity through simplification – Both fast approximate and slower high quality algorithms – 500k 50k 5k Checking • Visual inspection – See through filters • Quality evaluation – Curvature dependent, … • Automatic marking of mesh inconsistencies – topological: non manifoldness – Geometric: self intersection Repairing • Hole Filling – Automatically interpolate small missing mesh portions filling small gaps MeshLab: Aligning and Merging • 3D Scanning Pipeline Tools – ICP based alignment – Including global alignment for error redistribution – Surface reconstruction algorithms for range map merging MeshLab Features • Aligning functionalities needed to bring together models reconstructed from different sequences: MeshLab Features • Merging tools can be used for blending all the reconstructed range maps into a single (eventually watertight) mesh • Various surface reconstruction algorithms – Poisson Surface Reconstruction • Watertight mesh, sometimes can invent too much – Volumetric distance field • Preserve and blend color information • Do not fill large holes MeshLab Recon. Sample • Arc du triomphe • >100 photos • ~1M triagles reconstructed model • No texture here to better show the geom quality. • Reconstructed with the distance field approach MeshLab Recon. Sample • One of the two lions in the Loggia della Signoria in Florence • >30 photos • 200k triangles reconstructed model • Hi quality color reconstructed • Reconstructed with the distance field approach MeshLab Recon. Sample • A Lion statue from the Portalada in Ripoll, a large romanic portal, near Barcelona • >60 photos • 600k triangles reconstructed model • No color but watertight • Reconstructed with the Poisson surface reconstruction alg. PhotoCloud Prototype • A new way to use the results of Arc3D – Do not aim to a clean 3D mesh model, but retain and exploit all the original photos information PhotoCloud Prototype Basic ideas – Mix a rough 3D with (many) High Res photos – Allow free navigation in 3D space and in the space of the images – Allow to to take measure on photos – Work in remote, everything (3D and photos) is stored on a standard web server • You can show your reconstruction results without requiring that people download giga sized datasets Conclusions • Download and play by yourself! • Both tools are ready to use and tested. – http://www.arc3d.be – http://meshlab.sourceforge.net • Thanks for your attention!.