Point-Based Computer Graphics
Eurographics 2003 Tutorial T1
Organizers
Markus Gross ETH Zürich
Hanspeter Pfister MERL, Cambridge
Presenters
Marc Alexa TU Darmstadt
Carsten Dachsbacher Universität Erlangen-Nürnberg
Markus Gross ETH Zürich
Mark Pauly ETH Zürich
Jeroen van Baar MERL, Cambridge
Matthias Zwicker ETH Zürich Contents Tutorial Schedule ...... 2 Presenters and Organizers Contact Information ...... 3 References ...... 4 Project Pages ...... 5
Tutorial Schedule Introduction (Markus Gross) Acquisition of Point-Sampled Geometry and Appearance (Jeroen van Baar) Point-Based Surface Representations (Marc Alexa) Point-Based Rendering (Matthias Zwicker)
Lunch
Sequential Point Trees (Carsten Dachsbacher) Efficient Simplification of Point-Sampled Geometry (Mark Pauly) Spectral Processing of Point-Sampled Geometry (Markus Gross) Pointshop3D: A Framework for Interactive Editing of Point-Sampled Surfaces (Markus Gross) Shape Modeling (Mark Pauly) Pointshop3D Demo (Mark Pauly) Discussion (all)
2
Presenters and Organizers Contact Information Dr. Markus Gross Professor Department of Computer Science Swiss Federal Institute of Technology (ETH) CH 8092 Zürich Switzerland Phone: +41 1 632 7114 FAX: +41 1 632 1596 [email protected] http://graphics.ethz.ch
Dr. Hanspeter Pfister Associate Director MERL - A Mitsubishi Electric Research Lab 201 Broadway Cambridge, MA 02139 USA Phone: (617) 621-7566 Fax: (617) 621-7550 [email protected] http://www.merl.com/people/pfister/
Jeroen van Baar MERL - A Mitsubishi Electric Research Lab 201 Broadway Cambridge, MA 02139 USA Phone: (617) 621-7577 Fax: (617) 621-7550 [email protected] http://www.merl.com/people/jeroen/
Matthias Zwicker Department of Computer Science Swiss Federal Institute of Technology (ETH) CH 8092 Zürich Switzerland Phone: +41 1 632 7437 FAX: +41 1 632 1596 [email protected] http://graphics.ethz.ch
Mark Pauly Department of Computer Science Swiss Federal Institute of Technology (ETH) CH 8092 Zürich
3 Switzerland Phone: +41 1 632 0906 FAX: +41 1 632 1596 [email protected] http://graphics.ethz.ch
Dr. Marc Stamminger Universität Erlangen-Nürnberg Am Weichselgarten 9 91058 Erlangen Germany Phone: +49 9131 852 9919 FAX: +49 9131 852 9931 [email protected]
Carsten Dachsbacher Universität Erlangen-Nürnberg Am Weichselgarten 9 91058 Erlangen Germany Phone: +49 9131 852 9925 FAX: +49 9131 852 9931 [email protected]
Dr. Marc Alexa Interactive Graphics Systems Group Technische Universität Darmstadt Fraunhoferstr. 5 64283 Darmstadt Germany Phone: +49 6151 155 674 FAX: +49 6151 155 669 [email protected] http://www.dgm.informatik.tu-darmstadt.de/staff/alexa/
References M. Alexa, J. Behr, D. Cohen-Or, S. Fleishman, D. Levin, C. Silva. Point set surfaces. Proceedings of IEEE Visualization 2001, p. 21-28, San Diego, CA, October 2001.
C. Dachsbacher, C. Vogelsang, M. Stamminger, Sequential point trees. Proceedings of SIGGRAPH 2003, to appear, San Diego, CA, July 2003.
O. Deussen, C. Colditz, M. Stamminger, G. Drettakis, Interactive visualization of complex plant ecosystems. Proceedings of IEEE Visualization 2002, Boston, MA, October 2002.
4 W. Matusik, H. Pfister, P. Beardsley, A. Ngan, R. Ziegler, L. McMillan, Image- based 3D photography using opacity hulls. Proceedings of SIGGRAPH 2002, San Antonio, TX, July 2002.
W. Matusik, H. Pfister, A. Ngan, R. Ziegler, L. McMillan, Acquisition and rendering of transparent and refractive objects. Thirteenth Eurographics Workshop on Rendering, Pisa, Italy, June 2002.
M. Pauly, R. Keiser, L. Kobbelt, M. Gross, Shape modelling with point-sampled geometry, to appear, Proceedings of SIGGRAPH 2003, San Diego, CA, July 2003.
M. Pauly, M. Gross, Spectral processing of point-sampled geometry. Proceedings of SIGGRAPH 2001, p. 379-386, Los Angeles, CA, August 2001.
M. Pauly, M. Gross, Efficient Simplification of Point-Sampled Surfaces. IEEE Proceedings of Visualization 2002, Boston, MA, October 2002.
H. Pfister, M. Zwicker, J. van Baar, M. Gross, Surfels - surface elements as rendering primitives. Proceedings of SIGGRAPH 2000, p. 335-342, New Orleans, LS, July 2000.
M. Stamminger, G. Drettakis, Interactive sampling and rendering for complex and procedural geometry, Rendering Techniques 2001, Proceedings of the Eurographics Workshop on Rendering 2001, June 2001.
L. Ren, H. Pfister, M. Zwicker, Object space EWA splatting: a hardware accelerated approach to high quality point rendering. Proceedings of the Eurographics 2002, to appear, Saarbrücken, Germany, September 2002.
M. Zwicker, H. Pfister, J. van Baar, M. Gross, EWA volume splatting. Proceedings of IEEE Visualization 2001, p. 29-36, San Diego, CA, October 2001.
M. Zwicker, H. Pfister, J. van Baar, M. Gross, Surface splatting. Proceedings of SIGGRAPH 2001, p. 371-378, Los Angeles, CA, August 2001.
M. Zwicker, H. Pfister, J. van Baar, M. Gross, EWA splatting. IEEE Transactions on Visualization and Computer Graphics.
M. Zwicker, M. Pauly, O. Knoll, M. Gross, Pointshop 3D: an interactive system for point-based surface editing. Proceedings of SIGGRAPH 2002, San Antonio, TX, July 2002
Project Pages • Rendering http://graphics.ethz.ch/surfels • Acquisition http://www.merl.com/projects/3Dimages/
5 • Sequential point trees http://www9.informatik.uni-erlangen.de/Persons/Stamminger/Research • Modeling, processing, sampling and filtering http://graphics.ethz.ch/points • Pointshop3D http://www.pointshop3d.com
6 Surf. Reps. for Graphics
Point-Based Computer Graphics Hierarchical splines Raise degree Wavelets Tutorial T1
Mesh processing Discrete (point based) Marc Alexa, Carsten Dachsbacher, methods representations
Markus Gross, Mark Pauly, Subdivision schemes Add connectivity Hanspeter Pfister, Marc Stamminger, Add operators Jeroen Van Baar, Matthias Zwicker Triangle meshes
Point-Based Computer Graphics Markus Gross 2
Polynomials... Polynomials -> Triangles
• Piecewise linear approximations 9 Rigorous mathematical concept • Irregular sampling of the surface 9 Robust evaluation of geometric entities • Forget about parameterization 9 Shape control for smooth shapes 9 Advanced physically-based modeling
2 Require parameterization Triangle meshes 2 Discontinuity modeling 2 Topological flexibility • Multiresolution modeling • Compression Refine h rather than p ! • Geometric signal processing
Point-Based Computer Graphics Markus Gross 3 Point-Based Computer Graphics Markus Gross 4
Triangles... Triangles -> Points
• From piecewise linear functions to 9 Simple and efficient representation Delta distributions 9 Hardware pipelines support ∆ 9 Advanced geometric processing is being in sight • Forget about connectivity 9 The widely accepted queen of graphics primitives
2 Sophisticated modeling is difficult Point clouds 2 (Local) parameterizations still needed 2 Complex LOD management • Points are natural representations within 3D acquisition systems 2 Compression and streaming is highly non-trivial • Meshes provide an articifical enhancement Remove connectivity ! of the acquired point samples
Point-Based Computer Graphics Markus Gross 5 Point-Based Computer Graphics Markus Gross 6
1 History of Points in Graphics The Purpose of our Course is …
• Particle systems [Reeves 1983] • Points as a display primitive [Whitted, Levoy 1985] • Oriented particles [Szeliski, Tonnesen 1992] I) …to introduce points as a versatile and • Particles and implicit surfaces [Witkin, Heckbert 1994] powerful graphics primitive • Digital Michelangelo [Levoy et al. 2000] • Image based visual hulls [Matusik 2000] II) …to present state of the art concepts • Surfels [Pfister et al. 2000] • QSplat [Rusinkiewicz, Levoy 2000] for acquisition, representation, • Point set surfaces [Alexa et al. 2001] processing and rendering of point • Radial basis functions [Carr et al. 2001] sampled geometry • Surface splatting [Zwicker et al. 2001] • Randomized z-buffer [Wand et al. 2001] III) …to stimulate YOU to help us to • Sampling [Stamminger, Drettakis 2001] • Opacity hulls [Matusik et al. 2002] further develop Point Based Graphics • Pointshop3D [Zwicker, Pauly, Knoll, Gross 2002]...?
Point-Based Computer Graphics Markus Gross 7 Point-Based Computer Graphics Markus Gross 8
Taxonomy Morning Schedule
• Introduction (Markus Gross) Rendering Acquisition (Zwicker) (Pfister, Stamminger) • Acquisition of Point-Sampled Geometry and Apprearance (Jeroen van Baar) • Point-Based Surface Representations (Marc Point-Based Graphics Alexa) • Point-Based Rendering (Matthias Zwicker)
Processing & Representation Editing (Alexa) (Gross, Pauly)
Point-Based Computer Graphics Markus Gross 9 Point-Based Computer Graphics Markus Gross 10
Afternoon Schedule
• Sequential point trees (Carsten Dachsbacher) • Efficient simplification of point-sampled geometry (Mark Pauly) • Spectral processing of point-sampled geometry (Markus Gross) • Pointshop3D: A framework for interactive editing of point-sampled surfaces (Markus Gross) • Shape modeling (Mark Pauly) • Pointshop3D demo (Mark Pauly) • Discussion (all)
Point-Based Computer Graphics Markus Gross 11
2 The Goal: To Capture Reality
Acquisition of Point-Sampled • Fully-automated 3D model creation of real Geometry and Appearance objects. • Faithful representation of appearance for Jeroen van Baar and Hanspeter Pfister, MERL these objects. [jeroen,pfister]@merl.com
Wojciech Matusik, MIT Addy Ngan, MIT Paul Beardsley, MERL Remo Ziegler, MERL Leonard McMillan, MIT
Point-Based Computer Graphics Hanspeter Pfister, MERL 1 Point-Based Computer Graphics Hanspeter Pfister, MERL 2
Image-Based 3D Photography Previous Work
• An image-based 3D scanning system. • Active and passive 3D scanners • Handles fuzzy, refractive, transparent objects. • Work best for diffuse materials. • Robust, automatic • Fuzzy, transparent, and refractive objects are difficult. • Point-sampled geometry based on the visual hull. • BRDF estimation, inverse rendering • Objects can be rendered in novel environments. • Image based modeling and rendering • Reflectance fields [Debevec et al. 00] • Light Stage system to capture reflectance fields • Fixed viewpoint, no geometry • Environment matting [Zongker et al. 99, Chuang et al. 00] • Capture reflections and refractions • Fixed viewpoint, no geometry
Point-Based Computer Graphics Hanspeter Pfister, MERL 3 Point-Based Computer Graphics Hanspeter Pfister, MERL 4
Outline Acquisition System
•Overview
¾ System Light Array
•Geometry r •Reflectance • Refraction & Transparency Multi-Color Monito
Cameras
Rotating Platform
Point-Based Computer Graphics Hanspeter Pfister, MERL 5 Point-Based Computer Graphics Hanspeter Pfister, MERL 6 Acquisition Process Acquisition Process
Alpha Visual Alpha Visual Mattes Hull Mattes Hull
Surface Lightfield Surface Reflectance Fields Surface Lightfield Surface Reflectance Fields
Point-Based Computer Graphics Hanspeter Pfister, MERL 7 Point-Based Computer Graphics Hanspeter Pfister, MERL 8
Acquisition Process Acquisition Process
Alpha Visual Alpha Visual Mattes Hull Mattes Hull
Surface Lightfield Surface Reflectance Fields Surface Lightfield Surface Reflectance Fields
Point-Based Computer Graphics Hanspeter Pfister, MERL 9 Point-Based Computer Graphics Hanspeter Pfister, MERL 10
Outline Acquisition
•Overview • For each viewpoint ( 6 cameras x 72 •System positions ) ¾ Geometry • Alpha mattes • Use multiple backgrounds [Smith and Blinn 96] •Reflectance • Reflectance images • Refraction & Transparency • Pictures of the object under different lighting (4 lights x 11 positions) • Environment mattes • Use similar techniques as [Chuang et al. 2000]
Point-Based Computer Graphics Hanspeter Pfister, MERL 11 Point-Based Computer Graphics Hanspeter Pfister, MERL 12 Geometry – Opacity Hull Geometry Example
• Visual hull: The maximal object consistent with a given set of silhouettes.
Point-Based Computer Graphics Hanspeter Pfister, MERL 13 Point-Based Computer Graphics Hanspeter Pfister, MERL 14
Approximate Geometry Surface Light Fields
• The approximate visual hull is augmented by • A surface light field is a function that radiance data to render concavities, assigns a color to each ray originating on a reflections, and transparency. surface. [Wood et al., 2000]
Point-Based Computer Graphics Hanspeter Pfister, MERL 15 Point-Based Computer Graphics Hanspeter Pfister, MERL 16
Shading Algorithm Color Blending
• A view-dependent strategy. • Blend colors based on angle between virtual camera and stored colors. • Unstructured Lumigraph Rendering [Buehler et al., SIGGRAPH 2001] • View-Dependent Texture Mapping [Debevec, EGRW 98]
Point-Based Computer Graphics Hanspeter Pfister, MERL 17 Point-Based Computer Graphics Hanspeter Pfister, MERL 18 Point-Based Rendering Geometry – Opacity Hull
• Point-based rendering using LDC tree, • Store the opacity of each observation at visibility splatting, and view-dependent each point on the visual hull [Matusik et al. shading. SIG2002].
Point-Based Computer Graphics Hanspeter Pfister, MERL 19 Point-Based Computer Graphics Hanspeter Pfister, MERL 20
Geometry – Opacity Hull Example
• Assign view-dependent opacity to each ray originating on a point of the visual hull.
B C Photo
A φ (θ,φ)
Red = invisible White = opaque A B C Black = transparent θ Point-Based Computer Graphics Hanspeter Pfister, MERL 21 Point-Based Computer Graphics Hanspeter Pfister, MERL 22
Example Example
Photo Photo
Visual Hull Visual Hull Opacity Hull
Point-Based Computer Graphics Hanspeter Pfister, MERL 23 Point-Based Computer Graphics Hanspeter Pfister, MERL 24 Example Results
• Point-based rendering using EWA splatting, A-buffer blending, and edge antialiasing.
Photo Surface Light Field
Visual Hull Opacity Hull
Point-Based Computer Graphics Hanspeter Pfister, MERL 25 Point-Based Computer Graphics Hanspeter Pfister, MERL 26
Results Video Results Video
Point-Based Computer Graphics Hanspeter Pfister, MERL 27 Point-Based Computer Graphics Hanspeter Pfister, MERL 28
Results Video Results Video
Point-Based Computer Graphics Hanspeter Pfister, MERL 29 Point-Based Computer Graphics Hanspeter Pfister, MERL 30 Opacity Hull - Discussion Point-Based Models
• View dependent opacity vs. geometry • No need to establish topology or trade-off. connectivity. • Sometimes acquiring the geometry is not • No need for a consistent surface possible. parameterization for texture mapping. • Sometimes representing true geometry • Represent organic models (feather, tree) would be very inefficient. much more readily than polygon models. • Opacity hull stores the “macro” effect. • Easy to represent view-dependent opacity and radiance per surface point.
Point-Based Computer Graphics Hanspeter Pfister, MERL 31 Point-Based Computer Graphics Hanspeter Pfister, MERL 32
Outline Light Transport Model
•Overview • Assume illumination originates from • Previous Works infinity. • The light arriving at a camera pixel can be •Geometry described as: ¾ Reflectance = ω ω ω • Refraction & Transparency C(x, y) ∫W ( )E( )d Ω
C(x,y) - the pixel value E-the environment W -the reflectance field
Point-Based Computer Graphics Hanspeter Pfister, MERL 33 Point-Based Computer Graphics Hanspeter Pfister, MERL 34
Surface Reflectance Fields Reflectance Functions
•6D function:W (P,ω ,ω ) = W (u ,v ;θ ,Φ ;θ ,Φ ) • For each viewpoint, 4D function: i i r r r i i r r ω = θ Φ i Wxy ( i ) W (x, y; i , i )
(θi,φi) P r φi
θi
Point-Based Computer Graphics Hanspeter Pfister, MERL 35 Point-Based Computer Graphics Hanspeter Pfister, MERL 36 Relighting Compression
New • Subdivide images into 8 x 8 pixel blocks. Illumination • Keep blocks containing the object (avg. Down- Surface reflectance compression 1:7) sample field • PCA compression (avg. compression 1:10)
x = V0 PCA
V1
a0 a1 a2 a3 a4 a5 V2
… Vn
Point-Based Computer Graphics Hanspeter Pfister, MERL 37 Point-Based Computer Graphics Hanspeter Pfister, MERL 38
Results The Library
Point-Based Computer Graphics Hanspeter Pfister, MERL 39 Point-Based Computer Graphics Hanspeter Pfister, MERL 40
Surface Reflectance Fields Outline
• Work without accurate geometry •Overview • Surface normals are not necessary • Previous Works • Capture more than reflectance •Geometry • Inter-reflections •Reflectance • Subsurface scattering ¾ Refraction & Transparency •Refraction • Dispersion • Non-uniform material variations • Simplified version of the BSSRDF
Point-Based Computer Graphics Hanspeter Pfister, MERL 41 Point-Based Computer Graphics Hanspeter Pfister, MERL 42 Acquisition Acquisition
• We separate the hemisphere into high • For each viewpoint ( 6 cameras x 72
resolution h and low resolution l. positions ) • Alpha mattes • Use multiple backgrounds [Smith and Blinn 96] Wh Wl • Reflectance images Low resolution T • Pictures of the object under different lighting L( (4 lights x 11 positions) C(x, y) = W (ξ )T(ξ )dξ + W) (ω )L(ω )dω • Environment mattes High resolution ∫ h ∫ l i i Ω Ω h l • Use similar techniques as [Chuang et al. 2000]
Point-Based Computer Graphics Hanspeter Pfister, MERL 43 Point-Based Computer Graphics Hanspeter Pfister, MERL 44
Low-Resolution Reflectance Field High-Resolution Reflectance Field
C(x, y) = W (ξ )T (ξ )dξ + W (ω )L(ω )dω C(x, y) = W (ξ )T (ξ )dξ + W (ω )L(ω )dω ∫ h ∫ l i i ∫ h ∫ l i i Ω Ω Ω Ω h l h l • Use techniques of environment matting [Chuang et al., SIGGRAPH 00].
n W (ω )L(ω )dω ≈ W L for n lights ∫ l i i ∑ i i Ω i=1 l
Point-Based Computer Graphics Hanspeter Pfister, MERL 45 Point-Based Computer Graphics Hanspeter Pfister, MERL 46
High-Resolution Reflectance Field Reproject h
• Approximate Wh by a sum of up to two • Project environment mattes onto the new Gaussians: environment. • Reflective G . • Environment mattes acquired was 1 ξ = + Wh ( ) a1G1 a2G2 parameterized on plane T (the plasma display). • Refractive G2. • We need to project the Gaussians to the new N environment map, producing new Gaussians. G1
Wh G2 T
Point-Based Computer Graphics Hanspeter Pfister, MERL 47 Point-Based Computer Graphics Hanspeter Pfister, MERL 48 View Interpolation Results
• Render low-resolution reflectance field. • Performance for 6x72 = 432 viewpoints • 337,824 images taken in total !! • High-resolution reflectance field: • Acquisition (47 hours) • Match reflected and refracted Gaussians. • Alpha mattes – 1 hour • Environment mattes – 18 hours V1 ~ • Reflectance images – 28 hours G1r • Processing N ~ G V2 2r • Opacity hull ~ 30 minutes • PCA Compression ~ 20 hours (MATLAB, unoptimized) ~ • Rendering ~ 5 minutes per frame G2t ~ •Size G1t • Opacity hull ~ 30 - 50 MB • Interpolate direction vectors, not colors. • Environment mattes ~ 0.5 - 2 GB • Reflectance images ~ Raw 370 GB / Compressed 2 - 4 GB • Determine new color along interpolated direction.
Point-Based Computer Graphics Hanspeter Pfister, MERL 49 Point-Based Computer Graphics Hanspeter Pfister, MERL 50
Results Results Ω Ω High-resolution h Low-resolutionl Combined
Point-Based Computer Graphics Hanspeter Pfister, MERL 51 Point-Based Computer Graphics Hanspeter Pfister, MERL 52
Ω Results Results – h
Point-Based Computer Graphics Hanspeter Pfister, MERL 53 Point-Based Computer Graphics Hanspeter Pfister, MERL 54 Ω Results – l Results – Combined
Point-Based Computer Graphics Hanspeter Pfister, MERL 55 Point-Based Computer Graphics Hanspeter Pfister, MERL 56
Results Results
Point-Based Computer Graphics Hanspeter Pfister, MERL 57 Point-Based Computer Graphics Hanspeter Pfister, MERL 58
Conclusions Future Directions
• Data driven modeling is able to capture • Real-time rendering and render any type of object. • Done! [Vlasic et al., I3D 2003] • Opacity hulls provide realistic 3D • Better environment matting graphics models. • More than two Gaussians • Our models can be seamlessly inserted • Better compression into new environments. • MPEG-4 / JPEG 2000 • Point-based rendering offers high image- quality for display of acquired models.
Point-Based Computer Graphics Hanspeter Pfister, MERL 59 Point-Based Computer Graphics Hanspeter Pfister, MERL 60 Acknowledgements
• Colleagues: • MIT: Chris Buehler, Tom Buehler • MERL: Bill Yerazunis, Darren Leigh, Michael Stern • Thanks to: • David Tames, Jennifer Roderick Pfister • NSF grants CCR-9975859 and EIA-9802220 • Papers available at: http://www.merl.com/people/pfister/
Point-Based Computer Graphics Hanspeter Pfister, MERL 61 Point-based Surface Reps
•MarcAlexa Point-Based Computer Graphics • Discrete Geometric Modeling Group • Darmstadt University of Technology • [email protected] Marc Alexa, Carsten Dachsbacher, Markus Gross, Mark Pauly, Hanspeter Pfister, Marc Stamminger, Matthias Zwicker
2
Motivation Overview
• Many applications need a definition of • Introduction & Basics surface based on point samples • Fitting Implicit Surfaces •Reduction •Up-sampling • Projection-based Surfaces • Interrogation (e.g. ray tracing) • Desirable surface properties •Manifold • Smooth • Local (efficient computation)
3 4
Introduction & Basics Terms: Regular/Irregular
•Terms • Regular (on a grid) or irregular (scattered) • Regular/Irregular, Approximation/Interpolation, Global/Local • Neighborhood (topology) is unclear for • Standard interpolation/approximation irregular data techniques •Triangulation, Voronoi-Interpolation, Least Squares (LS), Radial Basis Functions (RBF), Moving LS •Problems • Sharp edges, feature size/noise • Functional -> Manifold
5 6
1 Terms: Approximation/Interpolation Terms: Global/Local • Noisy data -> Approximation • Global approximation
• Local approximation •Perfect data -> Interpolation
• Locality comes at the expense of smoothness
7 8
Triangulation: Piecewise Triangulation linear • Exploit the topology in a triangulation • Barycentric interpolation on simplices (e.g. Delaunay) of the data (triangles)
• Interpolate the data points on the •given d+1 points xi with values fi and a triangles point x inside the simplex defined by xi α •Piecewise linear Æ C0 •Compute i from x = Σ α ·x and Σ α = 1 • Piecewise quadratic Æ C1? i i i i i •Then •… Σ α f = i i ·fi
9 10
Voronoi Interpolation Voronoi Interpolation
•computeVoronoidiagram • for any point x in space τ τ •addx to Voronoi diagram 2 1 n • Voronoi cell τ around x intersects original cells n2 1 τ τ i of natural neighbors ni x τ τ 5 T 3 n •interpolate λ (x)⋅()f + ∇f ⋅()x − x n 5 f(x) = ∑i i i i i 3 λ (x) τ ∑i i 4 n with τ ∩τ 4 λ (x) = i i τ ⋅ − x xi 11 12
2 Voronoi Interpolation Least Squares
Properties of Voronoi Interpolation: • Fits a primitive to the data • linear Precision • Minimizes squared distances between •local the pi’s and primitive g
• for d = 1 Æ f(x) piecewise cubic g(x) = a + bx + cx2 • f(x)∈ C1 on domain
• f(x,x1,...,xn ) is continuous in xi ( − ())2 min pi g pi g ∑ y x i
13 14
Least Squares - Example Least Squares - Example
• Primitive is a polynomial • Resulting system 2 T = ()⋅ j 2 T g(x) 1, x, x ,... c 0 = 2 p (p − (1, p , p ,...)c )⇔ ∑ ix iy ix ix 2 i • min ()p − ()1, p , p2 ,... cT ⇒ ∑ iy ix ix 1 x x2 Κ c y i 0 j 2 T 2 3 0 = 2 p ()p − ()1, p , p ,... c x x x c1 yx ∑ ix iy ix ix = i x2 x3 x4 c yx2 2 • Linear system of equations Μ Μ Μ Ο
15 16
Radial Basis Functions Radial Basis Functions
• Represent interpolant as •Solvep = w r( p − p ) jy ∑ i ix jx • Sum of radial functions r i
• Centered at the data points pi to compute weights wi ()= ()− • Linear system of equations f x ∑ wir pi x i r()0 r()p − p r()p − p Λ w p 0x 1x 0x 2x 0 0 y r()p − p r()0 r()p − p w1 p1 1x 0x 1x 2x = y ()− ()− () w p r p p r p p r 0 2 2 y 2x 0x 2x 1x Μ Μ Μ Ο
17 18
3 Radial Basis Functions Function Spaces!
• Solvability depends on radial function • Monomial, Lagrange, RBF share the • Several choices assure solvability same principle: • Choose basis of a function space •(thinr()d = d 2 log d plate spline) • Find weight vector for base elements by solving linear system defined by data − 2 2 •r()d = e d / h (Gaussian) points • h is a data parameter • Compute values as linear combinations • h reflects the feature size or anticipated •Properties spacing among points • One costly preprocessing step • Simple evaluation of function in any point
19 20
Function Spaces? Shepard Interpolation
d Σ φ •Problems •Approach forR : f(x)= i i(x) fi • Many points lead to large linear systems − • Evaluation requires global solutions x − x p φ = i with basis functions i (x) −p •Solutions x − x ∑ j j • RBF with compact support • Matrix is sparse •define f(x ) := f = lim f(x) i i xÆxi • Still: solution depends on every data point, though drop-off is exponential with distance • Local approximation approaches
21 22
Shepard Interpolation Shepard Interpolation
φ • f(x) is a convex combination of i, Localization: φ d ⊆ Σ φ ≡ because all i(R ) [0,1] and i i(x) 1. Σ µ φ •Set f(x)= ν(x)· (x)·f Îf(x) is contained in the convex hull of data x−i x i i i ()− i für x−x
23 24
4 Spatial subdivisions Spatial subdivisions
• Subdivide parameter domain into τ overlapping cells i with centroids ci −p • Compute Shepard weights x − c φ (x) = i i −p x − c ∑ j j and localize them using the radius of the cell • Interpolate/approximate data points in each cell by an arbitrary function fi Σ • The interpolant is given as f(x)= i µ φ i(x)· i(x)·fi
25 26
Moving Least Squares Moving Least Squares
• Compute a local LS approximation at t •The set • Weight data points based on distance ()= ( − ())2 ( − ) f t gt (t), gt : min pi g pi θ t pi g ∑ y x x to t i is a smooth curve, iff θ is smooth
g(x) = a + bx + cx2 t min (p − g()p )2θ( t − p ) ∑ iy ix ix i
27 28
Moving Least Squares Typical Problems
•Typical choices for θ: •Sharp corners/edges • θ()d = d −r • 2 2 θ()d = e−d / h • Note: θ = θ ( t − p ) is fixed i ix • Noise vs. feature size • For each t • Standard weighted LS problem • Linear iff corresponding LS is linear
29 30
5 Functional -> Manifold Implicits
• Standard techniques are applicable • Each orientable n-manifold can be if data represents a function embedded in n+1 – space • Idea: Represent n-manifold as zero- • Manifolds are more general set of a scalar function in n+1 – space ()< • No parameter domain •Inside: f x 0 • No knowledge about neighbors, Delaunay • On the manifold: f ()x = 0 triangulation connects non-neighbors •Outside: f ()x > 0
31 32
Implicits - Illustration Implicits from point samples
• Function should be zero in data points • ()= f pi 0 • Use standard 0 approximation techniques to find f •Trivial solution: f = 0 • Additional constraints •Image courtesy Greg Turk are needed
33 34
Implicits from point samples Implicits from point samples
+ • Constraints define + •Use normal inside and outside information − • Simple approach + (Turk, O’Brien) − − + • Normals could be • Sprinkle additional + − − information manually computed from scan − • Make additional •Or, normalshave to be + information soft estimated constraints +
35 36
6 Estimating normals Estimating normals
• Normal orientation • General fitting problem (Implicits are signed) − 2 () min q pi ,n θ q,pi • Use inside/outside n n =1 ∑ information from scan i n • Normal direction q • Problem is non-linear q by fitting a tangent because n is constrained to unit sphere • LS fit to nearest neighbors • Weighted LS fit •MLS fit
37 38
Estimating normals Implicits from point samples +1 • The constrained minimization problem • Compute non-zero +1 +1 − 2 anchors in the +1 min q pi ,n θi n =1 ∑ distance field i +1 +1 is solved by the eigenvector corresponding •Use normal to the smallest eigenvalue of information directly +1 +1 ()q − p 2 θ ()q − p 2 θ ()q − p 2 θ as constraints ∑ x ix i ∑ x iy i ∑ x iz i i i i +1 ()q − p 2 θ ()q − p 2 θ ()q − p 2 θ f ()p + n =1 +1 ∑ y ix i ∑ y iy i ∑ y iz i i i i i i +1 +1 ()q − p 2 θ ()q − p 2 θ ()q − p 2 θ ∑ z ix i ∑ z iy i ∑ z iz i i i i
39 40
Implicits from point samples Computing Implicits
+2+0.5 +1.5 • Compute non-zero • Given N points and normals p i , n i anchors in the and constraints f ()p = 0, f ()c = d distance field i i i • Compute distances −2.5 p = c +1 +1 •Let i+N i at specific points • An RBF approximation • Vertices, mid-points, ()= ()− etc. in a spatial f x ∑ wir pi x i subdivision +2 +2.5 +0.5 leads to a system of linear equations
41 42
7 Computing Implicits Computing Implicits
r()0 r()p − p r()p − p Λ •Practical problems: N > 10000 •Sparse matrices 0 1 0 2 r()p − p r()0 r()p − p 1 0 1 2 • Matrix solution becomes difficult ()− ()− () r p p0 r p2 p1 r 0 • Two solutions Μ Ο > → = = • Sparse matrices allow iterative solution • Needed: d c r(d) 0,r'(c) 0 • Smaller number of RBFs c c • Compactly supported RBFs
43 44
Computing Implicits RBF Implicits - Results
• Smaller number of RBFs • Images courtesy Greg Turk • Greedy approach (Carr et al.) • Start with random small subset • Add RBFs where approximation quality is not sufficient
45 46
RBF Implicits - Results Hoppe’s approach
• Images courtesy Greg Turk • Use linear distance field per point • Direction is defined by normal • In every point in space use the distance field of the closest point
47 48
8 Hoppe’s approach - smoother PuO Implicits
• Direction fields are • Construct a spatial +2+0.5 +1.5 interpolated using subdivision Voronoi • Compute local distance field interpolation −2.5 approximations +1 +1 • e.g. Quadrics • Blend them with local Shepard +2 weights +2.5 +0.5
49 50
PuO Implicits: Sharp features Multi-level PuO Implicits
• Subdivide cells based on local error
Corner function
Local analysis of Edge points and normals function
Piecewise quadric functions Standard quadric
51 52
Multi-level PuO Implicits Multi-level PuO Implicits
• Local computations • Aproximation at arbitrary accuracy • Insensitive to number of points • Local adaptation to shape complexity •Sensitive to output complexity
53 54
9 Implicits - Conclusions Projection
• Scalar field is underconstrained • Idea: Map space to surface • Constraints only define where the field is • Surface is defined as fixpoints of zero, not where it is non-zero mapping • Additional constraints are needed r • Signed fields restrict surfaces to be r’ unbounded • All implicit surfaces define solids
55 56
Surface definition Surface Definition
• Projection procedure (Levin) • Constructive definition H • Local polyonmial approximation • Input point r r •Compute a local • Inspired by differential geometry n reference plane r G • “Implicit” surface definition Hr=
57 58
Local Reference Plane Local Reference Plane
H = + Weight function •Computing reference plane r •Find plane H r q,n D based on distance to n r q, not r • − 2 ()− • Non-linear optimization problem min q pi ,n θ q pi q, n =1 ∑ •Minimize independent q i Hr d 2 / h2 n variables: • θ()d = e r •Overn for fixed distance r − q • h is feature size/ H point spacing n r r q •Along n for fixed direction n • Hr is independent of r’s distance • q changes -> the weights change q • Manifold property • Only iterative solutions possible
59 60
10 Local Reference Plane Projecting the Point
Hr •Practical computation n r • MLS polyonomial over Hr ( − − ( ))2 ()− • Minimize over n for fixed q • min q pi ,n G pi θ q pi G∈Π ∑ H r •Eigenvalueproblem q d i n • Translate q so that •LS problem r Hr r = q + r − q n H •r’=Gr(0) n r •Effectively changes r − q r q • Minimize along n for •Estimate normal fixed direction n q • Exploit partial derivative Gr
61 62
Spatial data structure Conclusions
• Regular grid based on support of θ • Projection-based surface definition • Each point influences only 8 cells • Surface is smooth and manifold •Each cell is • Surface may be bounded • Representation error mainly depends on an octree r • Distant octree cells point density are approximated • Adjustable feature size h allows to by one point in smooth out noise center of mass
63 64
11 Point-Based Rendering
• Introduction and motivation Point-Based Rendering • Surface elements •Rendering • Antialiasing Matthias Zwicker • Hardware Acceleration Computer Graphics Lab •Conclusions ETH Zürich
2
Motivation 1 Motivation 1
• Performance of 3D hardware has exploded (e.g., GeForce4: 136 million vertices per second) • Projected triangles are very small (i.e., cover only a few pixels) • Overhead for triangle setup increases (initialization of texture filtering, rasterization)
Quake 2, 1998 Nvidia, 2002 A simpler, more efficient rendering 10k triangles millions of triangles primitive than triangles?
3 4
Points as Rendering Motivation 2 Primitives • Modern 3D scanning devices • Point clouds instead of triangle meshes [Levoy and (e.g., laser range scanners) Whitted 1985] acquire huge point clouds • 2D vector versus pixel graphics • Generating consistent triangle meshes is time consuming and difficult
A rendering primitive for direct visualization of point clouds, without the need to triangle mesh (with point cloud generate triangle meshes? 4 million pts. [Levoy et al. 2000] textures)
5 6
1 Point-Based Surface Representation Surface Elements - Surfels
•Points are samples of the surface • Each point corresponds to a surface • The point cloud describes: element, or surfel, describing the surface in • 3D geometry of the surface • Surface reflectance properties (e.g., a small neighborhood diffuse color, etc.) • Basic surfels: • There is no additional information, such as • connectivity (i.e., explicit BasicSurfel { y neighborhood information between position points) position; color • texture maps, bump maps, etc. color; z } x
7 8
Surfels Surfels
• How to represent the surface between the • Surfels can be extended by storing additional points? attributes • This allows for higher quality rendering or advanced shading effects holes between the points ExtendedSurfel { position; normal color; position • Surfels need to interpolate the surface normal; between the points radius; color radius •A certain surface area is associated with etc... surfel disc each surfel }
9 10
Surfels Model Acquisition
• Surfels store essential information for • 3D scanning of physical objects rendering • See Pfister, acquisition •Surfels are primarily designed as a • Direct rendering of acquired point clouds point rendering primitive • No mesh reconstruction necessary • They do not provide a mathematically smooth surface definition (see [Alexa 2001], point set surfaces)
[Matusik et al. 2002]
11 12
2 Model Acquisition Model Acquisition
•Sampling synthetic objects • Processing and editing of point-sampled • Efficient rendering of complex models geometry • Dynamic sampling of procedural objects and animated scenes (see Stamminger, dynamic sampling)
spectral processing point-based surface editing [Pauly, Gross 2002] [Zwicker et al. 2002] (see Gross, spectral processing) (see Pauly, Pointshop3D) [Zwicker et al. 2001] [Stamminger et al. 2001]
13 14
Point Rendering Pipeline Point Rendering Pipeline
Point Frame- Image Cloud buffer Projection Shading Visibility Reconstruction
Image Projection Shading Visibility Reconstruction • Perspective projection of each point in the point cloud • Simple, pure forward mapping pipeline • Analogous to projection of triangle • Surfels carry all information through the pipeline („surfel stream“) vertices • No texture look-ups • homogeneous matrix-vector product • Framebuffer stores RGB, alpha, and Z • perspective division
15 16
Point Rendering Pipeline Point Rendering Pipeline
Image Image Projection Shading Visibility Projection Shading Visibility Reconstruction Reconstruction
• Per-point shading • Visibility and image reconstruction is tightly coupled • Conventional models for shading • Discard points that are occluded from the (Phong, Torrance-Sparrow, current viewpoint reflections, etc.) • Reconstruct continuous surfaces from projected points (antialiasing)
17 18
3 Visibility and Image Visibility and Image Reconstruction Reconstruction without visibility and with visibility and • Goal: avoid holes and 3D object space image reconstruction image reconstruction discard occluded surfels • Use surfel discs with radius r to cover surface completely
foreground point • Apply z-buffer to normal radius r occluded background point discard invisible surfels surfel disc surface discontinuity (“hole”)
19 20
Quad Rendering Primitive Visibility: Z-Buffering
• Rasterize a colored quad centered at the projected • No blending of rendering primitives point, use z-buffering • The quad side length is h, where h = 2 * r * s framebuffer •The scaling factors given by perspective pixel projection and viewport transformation z1 z >z • Hardware implementation: OpenGL GL_POINTS 1 2{ z screen space 2 colored quad y h z projected } point x y
x 21 22
Projected Disc Rendering Primitive Discussion
• Project surfel discs from object to screen space • Quad and projected disc primitive • Projecting discs results in ellipses in screen space • Ellipses adapt to the surface orientation • Simple, efficient screen space object space • Hardware support normal • Low image quality • Suitable for preview renderers (e.g. surfel disc Qsplat [Rusinkiewicz et al. 2000] ) y y • Problem: no blending of primitives z
x x projected surfel disc
23 24
4 Splatting Splatting
• A splat primitive consists of a colored point • The final color c(x,y) is computed by additive primitive and an alpha mask alpha blending, i.e., by computing the weighted sum color of splat i alpha of splat i at position (x,y) c w (x, y) ∑i i i y y = y c(x, y) = * w (x, y) ∑i i x x x • Normalization is necessary, because the weights do colored point alpha mask splat primitive not sum up to one with irregular point distributions primitive c w(x,y) c * w(x,y) (often a 2D w (x, y) ≠ 1 Gauss function) ∑i i
25 26
Splatting Splatting
•Extendedz-buffering without normalization with normalization surface 1 surface 2
z-buffer pixel surfel disc z varying brightness no artifacts z-threshold discard splats because of irregular accumulate point distribution splats
27 28
Extended Z-Buffering Splatting Comparison
elliptical circular splats surface DepthTest(x,y) { splats with min. radius splatting if (abs(splat z – z(x,y)) < threshold) { minif. c(x,y) = c(x,y) + splat color w(x,y) = w(x,y) + splat w(x,y) } else if (splat z < z(x,y)) { z(x,y) = splat z c(x,y) = splat color w(x,y) = splat w(x,y) } } magnif. 128 x 192 128 x 192 128 x 192 29 30
5 High Quality Splatting Aliasing in Computer Graphics
• High quality splatting requires careful • Aliasing = Sampling of continuous functions below the Nyquist frequency analysis of aliasing issues • To avoid aliasing, sampling rate must be twice as • Review of signal processing theory high as the maximum frequency in the signal • Application to point rendering • Aliasing effects: • Loss of detail • Surface splatting [Zwicker et al. 2001] • Moire patterns, jagged edges • Disintegration of objects or patterns • Aliasing in Computer Graphics • Texture Mapping • Scan conversion of geometry
31 32
Aliasing in Computer Occurrence of Aliasing Graphics • Aliasing: high frequencies in the input signal Spatial Domain Frequency Domain Spatial Domain Frequency Domain appear as low frequencies in the reconstructed signal
33 34
Aliasing-Free Reconstruction Antialiasing
Spatial Domain Frequency Domain Spatial Domain Frequency Domain •Prefiltering • Band-limit the continuous signal before sampling • Eliminates all aliasing (with an ideal low-pass filter) • Closed form solution not available in general • Supersampling • Raise sampling rate • Reduces, but does not eliminate all aliasing artifacts (in practice, many signals have infinite frequencies) • Simple implementation (hardware)
35 36
6 Resampling Resampling Filters
discrete input signal discrete output signal Object Space
warp color reconstructed input reconstruction kernels resampling position 1. 4. irregular spacing 2. 3.
37 38
Resampling Filters Resampling Filters
Object Space Screen Space Object Space Screen Space
sum of resampling filters
Screen Space Screen Space Screen Space Screen Space 2. Warp 2. Warp 4. Sample 4. Sample warped reconstruction kernel resampling filters
low-pass filter convolution 3. Filter 3. Filter 39 40
Resampling 2D Reconstruction Kernels
• 2D reconstruction kernels are given by surfel discs • Resampling in the context of surface with alpha masks rendering • Warping is equivalent to projecting the kernel from • Discrete input function = surface texture object to screen space (discrete 2D function) screen space object space normal • Warping = projecting surfaces to the image plane (2D to 2D projective surfel disc with alpha mask = mapping) y reconstruction y kernel z x warped reconstruction kernel x
41 42
7 Resampling Filters Mathematical Formulation
• A resampling filter is a convolution of a warped reconstruction filter and a low-pass resampling filter filter screen space “no information falls pixel grid inbetween the pixel −1 grid” convolution c(x, y) = c r (m (x, y)) ⊗ h(x, y) ∑k k k
pixel color warping function low pass filter
reconstruction kernel reconstruction kernel color warped low-pass filter resampling filter reconstruction (determined by (“blurred reconstruction kernel pixel grid) kernel”)
43 44
Gaussian Resampling Filters Mathematical Formulation
• Gaussians are closed under linear warping and convolution c(x, y) = c r (m−1(x, y)) ⊗ h(x, y) • With Gaussian reconstruction kernels ∑k k k and low-pass filters, the resampling Gaussian Gaussian filter is a Gaussian, too reconstruction kernel low-pass filter • Efficient rendering algorithms (surface splatting [Zwicker et al. 2001]) screen space screen space
45 46
Mathematical Formulation Algorithm
for each point P { project P to screen space; −1 c(x, y) = c r (m (x, y)) ⊗ h(x, y) shade P; ∑k k k determine resampling kernel G; = splat G; ckGk (x, y) ∑k } Gaussian resampling filter for each pixel { normalize; }
47 48
8 Properties of 2D Resampling Filters Results warped recon- low-pass resampling minification • High quality reconstruction and filtering struction kernel filter filter
magnification 200k points 4783k points 49 50
Results Hardware Implementation
transparent surfaces scanned objects • Based on the object space formulation of EWA filtering • Implemented using textured triangles • All calculations are performed in the programmable hardware (extensive use of vertex shaders) • Presented at EG 2002 ([Ren et al. 2002])
987k points [MERL/MIT Matusik et al.]
51 52
Surface Splatting Performance Conclusions • Software implementation • Points are an efficient rendering primitive for highly complex surfaces • 500 000 splats/sec on 866 MHz PIII • Points allow the direct visualization of real world data • 1 000 000 splats/sec on 2 GHz P4 acquired with 3D scanning devices • High performance, low quality point rendering is supported • Hardware implementation [Ren et al. 2002] by 3D hardware (tens of millions points per second) • Uses texture mapping and vertex shaders • High quality point rendering with anisotropic texture filtering is available • 3 000 000 splats/sec on GeForce4 Ti 4400 • 3 million points per second with hardware support • 1 million points per second in software • Antialiasing technique has been extended to volume rendering
53 54
9 Applications Future Work
• Direct visualization of point clouds • Dedicated rendering hardware • Real-time 3D reconstruction and rendering • Efficient approximations of exact EWA for virtual reality applications splatting • Hybrid point and polygon rendering systems • Rendering architecture for on the fly • Rendering animated scenes sampling and rendering • Interactive display of huge meshes • On the fly sampling and rendering of procedural objects
55 56
Acknowledgments References
• [Levoy and Whitted 1985] The use of points as a display primitive, • Hanspeter Pfister, Jeroen van Baar technical report, University of North Carolina at Chapel Hill, 1985 (MERL, Cambridge MA) • [Heckbert 1986] Fundamentals of texture mapping and image warping, Master‘s Thesis, 1986 •Markus Gross, Mark Pauly, CGL • [Grossman and Dally 1998] Point sample rendering, Eurographics workshop on rendering, 1998 •Liu Ren • [Levoy et al. 2000] The digital Michelangelo project, SIGGRAPH 2000 • [Rusinkiewicz et al. 2000] Qsplat, SIGGRAPH 2000 • [Pfister et al. 2000] Surfels: Surface elements as rendering primitives, SIGGRAPH 2000 • [Zwicker et al. 2001] Surface splatting, SIGGRAPH 2001 • [Zwicker et al. 2002] EWA Splatting, to appear, IEEE TVCG 2002 http://graphics.ethz.ch/surfels • [Ren et al. 2002] Object space EWA splatting: A hardware accelerated http://graphics.ethz.ch/pointshop3d approach to high quality point rendering, Eurographics 2002
57 58
10 Introduction
• point rendering Point-Based Computer Graphics • how adapt point densities? • for a given viewing position, how can we get n points that suffice for that viewer? Marc Alexa, Carsten Dachsbacher, • how render the points? Markus Gross, Mark Pauly, • given n points, how can we render an image from them ? Hanspeter Pfister, Marc Stamminger, Matthias Zwicker
2
Introduction Introduction
• how render the points? • even with hardware: • for (int i = 0; i < N; i++) • project point to pixel, set pixel color renderPointWithNormalAndColor (x[i],y[i],z[i],nx[i],ny[i],nz[i],…); • hardware solution (Radeon 9700 Pro) → 10 mio points per second • ~80 mio. points per second • for (int i = 0; i < N; i++) renderPoint(x[i],y[i],z[i]); • no hole filling → 20 mio points per second • software solution • float *p = {...} renderPoints(p); • ~8 mio. points per second → 80 mio points per second • hole filling • → best performance with sequential • hardware != software processing of large chunks !
3 4
Introduction Hierarchical Processing
•whatwewant: •Q-Splat • sequential processing and • Rusinkiewicz et al., Siggraph 2000 • adaptive point densities • hierarchical point rendering based on Bounding Sphere Hierarchy → precomputed point lists → render continuous segments only
rendered segment point list
© S. Rusinkiewicz 5 6
1 Hierarchical Processing Hierarchical Processing
•Q-Splathierarchy • Q-Splat recursive rendering render( Node n ) { R // compute screen size of node s = n.R / distanceToCamera( n ); // screen size too big? if ( s > threshold ) // → render children R R R forall children c render( c ); else // else draw node R R R renderPoint( n.xyz ); }
7 8
Hierarchical Processing Sequential Point Trees
• store with node d = n.R / 1 Pixel •notsequential min • no array, but tree structure • render( Node n ) { • most work on CPU // node too close? if ( distanceToCamera( n ) < n.dmin ) • CPU is bottleneck: ~8 mio points per second // → render children forall children c render( c ); → sequential version ? else // else draw node renderPoint( n.xyz ); }
9 10
Sequential Point Trees Sequential Point Trees
• node n is rendered if: • assume • n is not too close and • distToCam(n) ≈ distToCam(n.parent) • parent is not rendered • store with n •or • n.dmax = n.parent.dmin • distToCam( n ) < n.dmin • then a node is rendered if • distToCam( n.parent ) ≥ n.parent.dmin • n.dmin ≤ distToCam(n) < n.dmax •parentistooclose, butnodeisfar enough
11 12
2 Sequential Point Trees Sequential Point Trees
•exampletree • sequential version
• foreach tree node n if ( n.dmin < distToCam(n) && distToCam(n) < n.dmax ) renderPoint(n);
• how enumerate nodes?
13 14
Sequential Point Trees Sequential Point Trees
• sort nodes by • compute lower bound dbmin on dmax distToCam(n) with bounding volume
• all elements with dmax < dbmin can be skipped
point tree point • only prefix must be considered point tree point sequential
imax imax imax 15 16
Sequential Point Trees Sequential Point Trees
• culling by GPU necessary, because d is not constant • account for d ≠ d(parent): over object
•dmax = dmin(parent) + distance to parent • partially parent and some children selected • no visible artifacts from this
17 18
3 Sequential Point Trees Sequential Point Trees
• CPU does per frame: • CPU does first interval selection by dbmin • GPU does fine granularity selection •computedbmin
•searchlast nodeimax with dmax>dbmin sequential point tree • send first i points to GPU max prefix with dbmin < dmax • GPU then does for every node n • compute d = distToCam(n)
•ifn.dmin ≤ d ≤ n.dmax culled because d < n.dmin • render node rendered
culled because d > n.dmax
19 20
Sequential Point Trees Sequential Point Trees
•Result • better error measurement • culling by GPU: only 10 - 40% •in flatregions
• on a 2,4 GHz Pentium with Radeon 9700: •increasedmin, dmax • CPU-Load < 20% (usually much less) • render larger points • > 50 Mio points after culling
21 22
Sequential Point Trees Sequential Point Trees
•geometric • example • perpendicular error
•tangential error
23 24
4 Sequential Point Trees Sequential Point Trees
• also add texture criterion • if significant color variation in child • necessary for flat textured regions nodes: • modify tangential error • increase error to node diameter
• prevents washed out colors in flat regions
25 26
Sequential Point Trees Sequential Point Trees
• perpendicular, tangential, texture • combine errors • perpendicular e error p •tangential et •textureetex • scale with 1/(view distance) r if texture variation •ecom = 2 + 2 else • fits into sequential point trees ep et
• => screen error = ecom / viewDistance
27 28
Sequential Point Trees Sequential Point Trees
• can be combined with polygons • combine with polygonal rendering • for every triangle ε •computedmax (longest side / dmax = )
• remove all points from triangle with smaller dmax
•sorttrianglesfordmax • during rendering
• for every object, compute upper bound dbmax on distance
• send triangles with dmax < dbmax to GPU • on the GPU (vertex program)
•test d < dmax • cull by alpha-test
29 30
5 Sequential Point Trees
•pros •verysimple! •CPU-loadlow • most work moved to GPU • GPU runs at maximum efficiency • cons • no view frustum culling • currently: bad splatting support by GPU
31
6 Overview Efficient Simplification of •Introduction Point-sampled Surfaces • Local surface analysis • Simplification methods • Error measurement • Comparison
Point-Based Computer Graphics Mark Pauly 1 Point-Based Computer Graphics Mark Pauly 2
Introduction Introduction
• Point-based models are often sampled very densely • Example: Level-of-detail (LOD) rendering • Many applications require coarser approximations, e.g. for efficient •Storage •Transmission • Processing •Rendering
Ö We need simplification methods for reducing the 10k 20k 60k 200k 2000k complexity of point-based surfaces
Point-Based Computer Graphics Mark Pauly 3 Point-Based Computer Graphics Mark Pauly 4
Introduction Local Surface Analysis
• We transfer different simplification methods from • Cloud of point samples describes underlying triangle meshes to point clouds: (manifold) surface
• Hierarchical clustering • We need: • Iterative simplification • Mechanisms for locally approximating the • Particle simulation surface Ö MLS approach
• Depending on the intended use, each method has • Fast estimation of tangent plane and curvature its pros and cons (see comparison) Ö principal component analysis of local neighborhood
Point-Based Computer Graphics Mark Pauly 5 Point-Based Computer Graphics Mark Pauly 6
1 Neighborhood Neighborhood
• No explicit connectivity between samples (as with • K-nearest neighbors triangle meshes)
• Replace geodesic proximity with spatial proximity (requires sufficiently high sampling density!)
• Compute neighborhood according to Euclidean distance
• Can be quickly computed using spatial data- structures (e.g. kd-tree, octree, bsp-tree) • Requires isotropic point distribution
Point-Based Computer Graphics Mark Pauly 7 Point-Based Computer Graphics Mark Pauly 8
Neighborhood Neighborhood
• Improvement: Angle criterion (Linsen) • Local Delaunay triangulation (Floater)
• Project points onto tangent plane • Sort neighbors according to angle • Project points into tangent plane • Include more points if angle between • Compute local Voronoi diagram subsequent points is above some threshold
Point-Based Computer Graphics Mark Pauly 9 Point-Based Computer Graphics Mark Pauly 10
Covariance Analysis Covariance Analysis
• Covariance matrix of local neighborhood N: • Consider the eigenproblem:
T p − p p − p C⋅ v = λ ⋅ v , l ∈{0,1,2} i1 i1 l l l = Λ ⋅ Λ ∈ C , i j N p − p p − p • C is a 3x3, positive semi-definite matrix in in Ö All eigenvalues are real-valued Ö The eigenvector with smallest eigenvalue defines the = 1 • with centroid p ∑pi least-squares plane through the points in the N i∈N neighborhood, i.e. approximates the surface normal
Point-Based Computer Graphics Mark Pauly 11 Point-Based Computer Graphics Mark Pauly 12
2 Covariance Analysis Covariance Analysis
• Covariance ellipsoid spanned by the eigenvectors • The total variation is given as: scaled with corresponding eigenvalue − 2 = λ + λ + λ ∑ pi p 0 1 2 i∈N
• We define surface variation as: λ σ (p) = 0 , λ ≤ λ ≤ λ n λ + λ + λ 0 1 2 0 1 2 • Measures the fraction of variation along the surface normal, i.e. quantifies how strong the surface deviates from the tangent plane Ö estimate for curvature
Point-Based Computer Graphics Mark Pauly 13 Point-Based Computer Graphics Mark Pauly 14
Covariance Analysis Surface Simplification
• Comparison with curvature: • Hierarchical clustering • Iterative simplification • Particle simulation
original mean curvature variation n=20 variation n=50
Point-Based Computer Graphics Mark Pauly 15 Point-Based Computer Graphics Mark Pauly 16
Hierarchical Clustering Hierarchical Clustering
• Top-down approach using binary space partition: •2D example • Split the point cloud if: root • Size is larger than user-specified maximum or • Surface variation is above maximum threshold covariance • Split plane defined by centroid and axis of greatest ellipsoid split plane variation (= eigenvector of covariance matrix with centroid largest associated eigenvector) • Leaf nodes of the tree correspond to clusters • Replace clusters by centroid
Point-Based Computer Graphics Mark Pauly 17 Point-Based Computer Graphics Mark Pauly 18
3 Hierarchical Clustering Hierarchical Clustering
•2D example •2D example
Point-Based Computer Graphics Mark Pauly 19 Point-Based Computer Graphics Mark Pauly 20
Hierarchical Clustering Hierarchical Clustering
•2D example
43 Clusters 436 Clusters 4,280 Clusters
Point-Based Computer Graphics Mark Pauly 21 Point-Based Computer Graphics Mark Pauly 22
Hierarchical Clustering Iterative Simplification
• Adaptive Clustering • Iteratively contracts point pairs Ö Each contraction reduces the number of points by one • Contractions are arranged in priority queue according to quadric error metric (Garland and Heckbert) • Quadric measures cost of contraction and determines optimal position for contracted sample • Equivalent to QSlim except for definition of approximating planes
Point-Based Computer Graphics Mark Pauly 23 Point-Based Computer Graphics Mark Pauly 24
4 Iterative Simplification Iterative Simplification
• Quadric measures the squared distance to a set of •2D example planes defined over edges of neighborhood • Compute initial point-pair e = p − p e = e ×n • plane spanned by vectors 1 i and 2 1 contraction candidates
n • Compute fundamental quadrics e 2 p • Compute edge costs
e1
pi
Point-Based Computer Graphics Mark Pauly 25 Point-Based Computer Graphics Mark Pauly 26
Iterative Simplification Iterative Simplification
•2D example priority queue •2D example priority queue edge cost edge cost
6 0.02 6 0.02 2 0.03 2 0.03 14 0.04 14 0.04 5 0.04 5 0.04 9 0.09 9 0.09 1 0.11 1 0.11 13 0.13 13 0.13 3 0.22 3 0.22 11 0.27 11 0.27 10 0.36 10 0.36 7 0.44 7 0.44 4 0.56 4 0.56
Point-Based Computer Graphics Mark Pauly 27 Point-Based Computer Graphics Mark Pauly 28
Iterative Simplification Iterative Simplification
•2D example priority queue •2D example priority queue edge cost edge cost
6 0.02 2 0.03 2 0.03 14 0.04 14 0.04 5 0.06 5 0.06 9 0.09 9 0.09 1 0.11 1 0.11 13 0.13 13 0.13 3 0.23 3 0.23 11 0.27 11 0.27 10 0.36 10 0.36 7 0.49 7 0.49 4 0.56 4 0.56
Point-Based Computer Graphics Mark Pauly 29 Point-Based Computer Graphics Mark Pauly 30
5 Iterative Simplification Iterative Simplification
•2D example priority queue •2D example priority queue edge cost edge cost
2 0.03 14 0.04 14 0.04 5 0.06 5 0.06 9 0.09 9 0.09 1 0.11 1 0.11 13 0.13 13 0.13 3 0.23 3 0.23 11 0.27 11 0.27 10 0.36 10 0.36 7 0.49 7 0.49 4 0.56 4 0.56
Point-Based Computer Graphics Mark Pauly 31 Point-Based Computer Graphics Mark Pauly 32
Iterative Simplification Iterative Simplification
•2D example priority queue edge cost
11 0.27 10 0.36 7 0.49 4 0.56
original model simplified model remaining point pair (296,850 points) (2,000 points) contraction candidates
Point-Based Computer Graphics Mark Pauly 33 Point-Based Computer Graphics Mark Pauly 34
Particle Simulation Particle Simulation
• Resample surface by distributing particles • Initialization on the surface • Randomly spread particles • Particles move on surface according to •Repulsion F (p) = k(r − p − p )⋅(p − p ) inter-particle repelling forces • Linear repulsion force i i i • Particle relaxation terminates when Öonly need to consider neighborhood of radius r equilibrium is reached (requires damping) •Projection • Can also be used for up-sampling! • Keep particles on surface by projecting onto tangent plane of closest point • Apply full MLS projection at end of simulation
Point-Based Computer Graphics Mark Pauly 35 Point-Based Computer Graphics Mark Pauly 36
6 Particle Simulation Particle Simulation
•2D example •2D example • Initialization • randomly spread particles
Point-Based Computer Graphics Mark Pauly 37 Point-Based Computer Graphics Mark Pauly 38
Particle Simulation Particle Simulation
•2D example •2D example • Initialization • Initialization • randomly spread particles • randomly spread particles
•Repulsion •Repulsion • linear repulsion force • linear repulsion force = − − ⋅ − = − − ⋅ − Fi (p) k(r p pi ) (p pi ) Fi (p) k(r p pi ) (p pi )
Point-Based Computer Graphics Mark Pauly 39 Point-Based Computer Graphics Mark Pauly 40
Particle Simulation Particle Simulation
•2D example •2D example • Initialization • Initialization • randomly spread particles • randomly spread particles
•Repulsion •Repulsion • linear repulsion force • linear repulsion force = − − ⋅ − = − − ⋅ − Fi (p) k(r p pi ) (p pi ) Fi (p) k(r p pi ) (p pi ) •Projection •Projection • project particles onto • project particles onto surface surface
Point-Based Computer Graphics Mark Pauly 41 Point-Based Computer Graphics Mark Pauly 42
7 Particle Simulation Particle Simulation
• Adaptive simulation • User-controlled simulation • Adjust repulsion radius according to surface variation • Adjust repulsion radius according to user input Ö more samples in regions of high variation
variation simplified model uniform original selective estimation (3,000 points) Point-Based Computer Graphics Mark Pauly 43 Point-Based Computer Graphics Mark Pauly 44
Measuring Error Measuring Error
• Measure the distance between two point-sampled • d(q, S′) measures the distance of point q to surfaces using a sampling approach surface S ′ using the MLS projection operator with ∆ ′ = ′ linear basis functions • Maximum error: max (S, S ) maxq∈Q d(q, S ) Ö Two-sided Hausdorff distance ∆ ′ = 1 ′ • Mean error: avg (S, S ) ∑ d(q, S ) Q q∈Q Ö Area-weighted integral of point-to-surface distances •Q is an up-sampled version of the point cloud that describes the surface S
Point-Based Computer Graphics Mark Pauly 45 Point-Based Computer Graphics Mark Pauly 46
Measuring Error Comparison
• Error estimate for Michelangelo’s David simplified from 2,000,000 points to 5,000 points
original simplified upsampled error
Point-Based Computer Graphics Mark Pauly 47 Point-Based Computer Graphics Mark Pauly 48
8 Comparison Comparison
• Execution time as a function of input model • Execution time as a function of target size (reduction to 1%) model size (input: dragon, 535,545 points)
hierarchical clustering time 70 hierarchical clustering time 500 (sec) (sec) 450 iterative simplification 60 iterative simplification 400 50 350 particle simulation particle simulation 300 40 250 30 200 150 20 100 10 50 0 0 0 500 1000 1500 2000 2500 3000 3500 180 160 140 120 100 80 60 40 20 0
input size target size
Point-Based Computer Graphics Mark Pauly 49 Point-Based Computer Graphics Mark Pauly 50
Point-based vs. Mesh Comparison Simplification • Summary
Efficiency Surface Control Implementation Error Hierarchical + - - + Clustering Iterative - + o o Simplification point-based simplification with mesh reconstruction with subsequent Particle o + + - subsequent mesh reconstruction mesh simplification (QSlim) Simulation Ö point-based simplification saves an expensive surface reconstruction on the dense point cloud!
Point-Based Computer Graphics Mark Pauly 51 Point-Based Computer Graphics Mark Pauly 52
References
• Pauly, Gross: Efficient Simplification of Point- sampled Surfaces, IEEE Visualization 2002 •Shaffer, Garland: Efficient Adaptive Simplification of Massive Meshes, IEEE Visualization 2001 • Garland, Heckbert: Surface Simplification using Quadric Error Metrics, SIGGRAPH 1997 •Turk: Re-Tiling Polygonal Surfaces, SIGGRAPH 1992 • Alexa et al. Point Set Surfaces, IEEE Visualization 2001
Point-Based Computer Graphics Mark Pauly 53
9 Overview
•Introduction Spectral Processing of Point- • Fourier transform Sampled Geometry • Spectral processing pipeline • Applications Markus Gross • Spectral filtering • Adaptive subsampling •Summary
Point-Based Computer Graphics Markus Gross 2
Introduction Introduction
• Idea: Extend the Fourier transform to • Applications: manifold geometry • Spectral filtering: •Noise removal • Microstructure analysis • Enhancement
• Adaptive resampling: Ö Spectral representation of point-based objects • Complexity reduction Ö Powerful methods for digital geometry processing • Continuous LOD
Point-Based Computer Graphics Markus Gross 3 Point-Based Computer Graphics Markus Gross 4
Fourier Transform Fourier Transform
• 1D example: • Requirements: nk • Fourier transform defined on Euclidean domain N − j 2π X = x e N Öwe need a global parameterization n ∑ k spectral basis function k =1 • Basis functions are eigenfunctions of Laplacian output signal input signal operator Örequires regular sampling pattern so that basis functions can be expressed in analytical form (fast • Benefits: evaluation) • Sound concept of frequency • Limitations: • Extensive theory • Basis functions are globally defined • Fast algorithms ÖLack of local control
Point-Based Computer Graphics Markus Gross 5 Point-Based Computer Graphics Markus Gross 6
1 Approach Spectral Pipeline
• Split model into patches that: • are parameterized over the unit-square Ö mapping must be continuous and should minimize distortion • are re-sampled onto a regular grid Ö adjust sampling rate to minimize information loss • provide sufficient granularity for intended application (local analysis)
Öprocess each patch individually and blend processed patches
Point-Based Computer Graphics Markus Gross 7 Point-Based Computer Graphics Markus Gross 8
Patch Layout Creation Patch Layout Creation
• Iterative, local optimization method Clustering Ö Optimization • Merge patches according to quality metric: Φ = Φ ⋅Φ ⋅Φ ⋅Φ S NC B Reg Φ S Ö patch Size Φ NCÖ curvature Φ B Ö patch boundary Ö Ö Samples Clusters Patches Φ RegÖ spring energy regularization
Point-Based Computer Graphics Markus Gross 9 Point-Based Computer Graphics Markus Gross 10
Patch Layout Creation Patch Resampling
• Parameterize patches by orthogonal projection • Patches are irregularly sampled: onto base plane • Bound normal cone to control distortion of mapping using smallest enclosing sphere
Point-Based Computer Graphics Markus Gross 11 Point-Based Computer Graphics Markus Gross 12
2 Patch Resampling Spectral Analysis
• Resample patch onto regular grid using hierarchical • 2D discrete Fourier transform (DFT) push-pull filter (scattered data approximation) Ö Direct manipulation of spectral coefficients
• Filtering as convolution: F(x ⊗ y) = F(x)⋅ F(y)
Ö Convolution: O(N2) Ö multiplication: O(N)
• Inverse Fourier transform Ö Filtered patch surface
Point-Based Computer Graphics Markus Gross 13 Point-Based Computer Graphics Markus Gross 14
Spectral Filters Spectral Filters
• Smoothing filters • Microstructure analysis and enhancement ideal low-pass Gaussian low-pass original
transfer function: spectral domain
transfer function: spatial domain
Point-Based Computer Graphics Markus Gross 15 Point-Based Computer Graphics Markus Gross 16
Spectral Resampling Reconstruction
• Low-pass filtering • Filtering can lead to discontinuities at Ö Band-limitation patch boundaries Ö Create patch overlap, blend adjacent patches • Regular Resampling region of overlap Ö Optimal sampling rate Sampling rates (sampling theorem)
Ö Error control Power Spectrum Point positions (Parseval’s theorem) Normals
Point-Based Computer Graphics Markus Gross 17 Point-Based Computer Graphics Markus Gross 18
3 Reconstruction Timings
Clustering 9% • Blending the sampling rate Time
Patch Merging 38%
SDA 23%
blended sampling discretized pre-computed Analysis 4% rate in region of sampling rate sampling patterns patch overlap on regular grid Reconstruction 26%
Point-Based Computer Graphics Markus Gross 19 Point-Based Computer Graphics Markus Gross 20
Applications Applications
•Surface Restoration •Interactive filtering
Original Gaussian low-pass Wiener filter Patch layout
Point-Based Computer Graphics Markus Gross 21 Point-Based Computer Graphics Markus Gross 22
Applications Summary
• Adaptive Subsampling • Versatile spectral decomposition of point- based models • Effective filtering • Adaptive resampling • Efficient processing of large point-sampled models
4,128,614 pts. = 100% 287,163 pts. = 6.9%
Point-Based Computer Graphics Markus Gross 23 Point-Based Computer Graphics Markus Gross 24
4 Reference
• Pauly, Gross: Spectral Processing of Point-sampled Geometry, SIGGRAPH 2001
Point-Based Computer Graphics Markus Gross 25
5 Overview
•Introduction • Pointshop3D System Components • Point Cloud Parameterization An Interactive System for Point-based •Resampling Scheme Surface Editing •EditingOperators •Summary
Point-Based Computer Graphics Markus Gross 2
PointShop3D Concept
• Interactive system for point-based surface editing Parameterization • Generalizes 2D photo editing concepts and functionality to 3D point-sampled surfaces • Uses 3D surface pixels (surfels) as versatile display and modeling primitive
v
u
Resampling Editing Operator
Point-Based Computer Graphics Markus Gross 3 Point-Based Computer Graphics Markus Gross 4
Key Components Parameterization
• Point cloud parameterization Φ • Constrained minimum distortion • brings surface and brush into common reference frame parameterization of point clouds •Dynamic resamplingΨ • creates one-to-one correspondence of surface and brush samples • Editing operator Ω x(u) ∈ 2 = = ∈ ⊂ 3 • combines surface and brush samples u [0,1] ⇒ X (u) y(u) x P R z(u) S′ = Ω(Ψ(Φ(S)),Ψ(B))
modified surface original surface brush
Point-Based Computer Graphics Markus Gross 5 Point-Based Computer Graphics Markus Gross 6
1 Parameterization Parameterization
• Find mapping X that minimizes objective function:
brush points surface points
C(X ) = (X (p ) − x )2 + ε γ (u)du ∑ {j j ∫ j∈M {P contraints = matching minimum distortion = fitting constraints of feature points maximum smoothness distortion
Point-Based Computer Graphics Markus Gross 7 Point-Based Computer Graphics Markus Gross 8
Parameterization Parameterization
• Measuring distortion • Discrete formulation:
∂ 2 2 θ 2 r ~ n ∂U (x ) ∂U (x ) γ (u) = X (θ ,r) dθ = − 2 + ε i − i ∫ ∂ 2 u u C(U ) ∑∑∑(p j u j ) ~ θ r ∈=∈ ∂v ∂v j M iN1 j i j j
• Integrates squared curvature using local polar re-parameterization • Approximation: mapping is piecewise linear cos(θ ) X (θ , r) = X u + r u θ sin( )
Point-Based Computer Graphics Markus Gross 9 Point-Based Computer Graphics Markus Gross 10
Parameterization Parameterization
• Directional derivatives as extension of divided • Multigrid solver for efficient computation of differences based on k-nearest neighbors resulting sparse linear least squares problem
n 2 ~ = − = − 2 C(U ) ∑∑b j a j,iui b Au j i=1
Point-Based Computer Graphics Markus Gross 11 Point-Based Computer Graphics Markus Gross 12
2 Reconstruction Reconstruction
• Parameterized scattered data approximation
fitting functions weight functions Φ ∑ i (u)ri (u) X (u) = i r (u) ∑ i normalization factor i • Fitting functions • Compute local fitting functions using local reconstruction with weight functions in parameterizations linear fitting functions parameter space • Map to global parameterization using global parameter coordinates of neighboring points
Point-Based Computer Graphics Markus Gross 13 Point-Based Computer Graphics Markus Gross 14
Reconstruction Sampling
• Reconstruction with linear fitting functions is • Three sampling strategies: equivalent to surface splatting! • Resample the brush, i.e., sample at the original Ö we can use the surface splatting renderer to reconstruct our surface function (see chapter on rendering) surface points •This provides: • Resample the surface, i.e., sample at the brush • Fast evaluation points • Anti-aliasing (Band-limit the weight functions before • Adaptive resampling, i.e., sample at surface or sampling using Gaussian low-pass filter) brush points depending on the respective • Distortions of splats due to parameterization can be sampling density computed efficiently using local affine mappings
Point-Based Computer Graphics Markus Gross 15 Point-Based Computer Graphics Markus Gross 16
Editing Operators Editing Operators
•Painting •Sculpting • Texture, material properties, transparency • Carving, normal displacement
texture map displacement maps carved and texture mapped point-sampled surface
Point-Based Computer Graphics Markus Gross 17 Point-Based Computer Graphics Markus Gross 18
3 Editing Operators Summary
•Filtering • Pointshop3D provides sophisticated editing • Scalar attributes, geometry operations on point-sampled surfaces Öpoints are a versatile and powerful modeling primitive • Limitation: only works on “clean” models • sufficiently high sampling density • no outliers • little noise Örequires model cleaning (integrated or as pre- process)
Point-Based Computer Graphics Markus Gross 19 Point-Based Computer Graphics Markus Gross 20
Reference
• Zwicker, Pauly, Knoll, Gross: Pointshop3D: An interactive system for Point-based Surface Editing, SIGGRAPH 2002
•check out: www.pointshop3D.com
Point-Based Computer Graphics Markus Gross 21
4 Motivation
Shape Modeling • 3D content creation pipeline
Mark Pauly
Point-Based Computer Graphics Mark Pauly 2
Motivation Motivation
• Surface representations • Surface representations •Implicit surfaces •Implicit surfaces • Level sets • Level sets + Extreme deformations • Radial basis functions • Radial basis functions Hybrid Representation + Changes of topology • Algebraic surfaces • Algebraic surfaces • Explicit cloud of point samples • Parametric surfaces • Parametric surfaces • Implicit dynamic • Polygonal meshes • Polygonal meshes + Sharp features surface model • Subdivision surfaces • Subdivision surfaces + Efficient rendering •NURBS •Nurbs + Intuitive Editing
Point-Based Computer Graphics Mark Pauly 3 Point-Based Computer Graphics Mark Pauly 4
Motivation Interactive Modeling
• Point cloud representation • Interactive design and editing of • Minimal consistency requirements for extreme point-sampled models deformations (dynamic re-sampling) • Shape Modeling • Fast inside/outside classification for boolean operations and collision detection • Boolean operations • Explicit modeling and rendering of sharp feature • Free-form deformation curves • Appearance Modeling • Integrated, intuitive editing of shape and • Painting & texturing appearance • Embossing & engraving
Point-Based Computer Graphics Mark Pauly 5 Point-Based Computer Graphics Mark Pauly 6
1 Boolean Operations Boolean Operations
• Create new shapes by combining existing models using union, intersection, or +- - difference operations • Powerful and flexible editing paradigm mostly used in industrial design applications (CAD/CAM)
Point-Based Computer Graphics Mark Pauly 7 Point-Based Computer Graphics Mark Pauly 8
Boolean Operations Boolean Operations
• Easily performed on implicit • Point-Sampled Geometry representations • Classification • Requires simple computations on the • Inside-outside test using signed distance distance function function induced by MLS projection • Difficult for parametric surfaces •Sampling • Requires surface-surface intersection • Compute exact intersection of two MLS surfaces to sample the intersection curve • Topological complexity of resulting surface depends on geometric •Rendering complexity of input models • Accurate depiction of sharp corners and creases using point-based rendering
Point-Based Computer Graphics Mark Pauly 9 Point-Based Computer Graphics Mark Pauly 10
Boolean Operations Boolean Operations
• Classification: • Classification: • given a smooth, closed • given a smooth, closed surface S and point p. Is p S surface S and point p. Is p S inside or outside of the inside or outside of the volume V bounded by S? volume V bounded by S? p V 1.find closest point q on S p V
q
Point-Based Computer Graphics Mark Pauly 11 Point-Based Computer Graphics Mark Pauly 12
2 Boolean Operations Boolean Operations
• Classification: • Classification: • given a smooth, closed • given a smooth, closed surface S and point p. Is p S surface S and point p. Is p S inside or outside of the inside or outside of the volume V bounded by S? volume V bounded by S? 1.find closest point q on S p V 1.find closest point q on S p V n n 2.d=(p-q)·n defines signed q 2.d=(p-q)·n defines signed q distance of p to S distance of p to S 3.classify p as –inside V, if d < 0 –outside V, if d > 0
Point-Based Computer Graphics Mark Pauly 13 Point-Based Computer Graphics Mark Pauly 14
Boolean Operations Boolean Operations
• Classification: • Classification: • represent smooth surface S • represent smooth surface S by point cloud P S by point cloud P S P P m m 1.find closest point q in P V p V n q
Point-Based Computer Graphics Mark Pauly 15 Point-Based Computer Graphics Mark Pauly 16
Boolean Operations Boolean Operations
• Classification: • Classification: • represent smooth surface S • apply full MLS projection for points close by point cloud P S P to the surface m 1.find closest point q in P p V 2.classify p as n –inside V, if (p-q)·n < 0 q p –outside V, if (p-q)·n > 0 Ψ n P (p) q
Point-Based Computer Graphics Mark Pauly 17 Point-Based Computer Graphics Mark Pauly 18
3 Boolean Operations Boolean Operations
• Sampling the intersection curve • Newton scheme: 1.identify pairs of closest points
Point-Based Computer Graphics Mark Pauly 19 Point-Based Computer Graphics Mark Pauly 20
Boolean Operations Boolean Operations
• Newton scheme: • Newton scheme: 1.identify pairs of closest points 1. identify pairs of closest points 2. compute closest point on intersection of tangent spaces
r
q q 1 q2 1 q2
Point-Based Computer Graphics Mark Pauly 21 Point-Based Computer Graphics Mark Pauly 22
Boolean Operations Boolean Operations
• Newton scheme: • Newton scheme: 1. identify pairs of closest points 1. identify pairs of closest points 2. compute closest point on intersection of 2. compute closest point on intersection of tangent spaces tangent spaces 3. re-project point on both surfaces 3. re-project point on both surfaces 4. iterate r ′ q′ ′ q′ ′ 1 q2 1 q2 r
q 1 q2
Point-Based Computer Graphics Mark Pauly 23 Point-Based Computer Graphics Mark Pauly 24
4 Boolean Operations Boolean Operations
• Rendering sharp creases • Rendering sharp creases • represent points on intersection curve with two surfels that mutually clip each other
Point-Based Computer Graphics Mark Pauly 25 Point-Based Computer Graphics Mark Pauly 26
Boolean Operations Boolean Operations
• Rendering sharp creases • Boolean operations can create intricate • easily extended to handle corners by shapes with complex topology allowing multiple clipping
A + B A ⋅ B
A − B B − A
Point-Based Computer Graphics Mark Pauly 27 Point-Based Computer Graphics Mark Pauly 28
Boolean Operations Boolean Operations
• Singularities lead to numerical instabilities • Sharp creases can be blended using (intersection of almost parallel planes) oriented particles (Szeliski, Tonnesen)
Point-Based Computer Graphics Mark Pauly 29 Point-Based Computer Graphics Mark Pauly 30
5 Free-form Deformation Free-form Deformation
• Smooth deformation field F:R3→R3 that warps 3D space • Can be applied directly to point samples
Point-Based Computer Graphics Mark Pauly 31 Point-Based Computer Graphics Mark Pauly 32
Free-form Deformation Free-form Deformation
• How to define the deformation field? • Intuitive editing paradigm using Ö Painting metaphor painting metaphor • How to detect and handle self- • Define rigid surface part (zero-region) intersections? and handle (one-region) using interactive painting tool Ö Point-based collision detection, boolean • Displace handle using combination of union, particle-based blending translation and rotation • How the handle strong distortions? • Create smooth blend towards zero-region Ö Dynamic re-sampling
Point-Based Computer Graphics Mark Pauly 33 Point-Based Computer Graphics Mark Pauly 34
Free-form Deformation Free-form Deformation
• Definition of deformation field:
• Continuous scale parameter tx zero-region β d • tx = (d0 / (d0 + d1)) 0 x • d0 : distance of x to zero-region d1 • d1 : distance of x to one-region one-region • Blending function • β : [0,1] → [0,1] • β ∈C0, β (0) = 0, β (1) = 1 •t = 0 if x in zero-region original deformed x surface surface •tx = 1 if x in one-region
Point-Based Computer Graphics Mark Pauly 35 Point-Based Computer Graphics Mark Pauly 36
6 Free-form Deformation Free-form Deformation
• Definition of deformation field: • Translation for three different blending functions • Deformation function d • F (x) = F (x) +F (x) 0 T R x
d1 •Translation deformed • F (x) = x + t ·v blending T x function surface • Rotation
• FR (x) = M(tx) ·x
Point-Based Computer Graphics Mark Pauly 37 Point-Based Computer Graphics Mark Pauly 38
Free-form Deformation Free-form Deformation
• Rotational deformation along two different •Embossing effect rotation axes bitmap image
deformed zero- and original color-coded deformed surface one-regions surface scale parameter surface
Point-Based Computer Graphics Mark Pauly 39 Point-Based Computer Graphics Mark Pauly 40
Collision Detection Collision Detection
• Deformations can lead to self- • Exploiting temporal coherence intersections • Apply boolean inside/outside classification to detect collisions • Restricted to collisions between deformable region and zero-region to ensure efficient computations
Point-Based Computer Graphics Mark Pauly 41 Point-Based Computer Graphics Mark Pauly 42
7 Collision Detection Dynamic Sampling
• Interactive modeling session • Large model deformations can lead to strong surface distortions boolean union performed • Requires adaptation of the sampling density • Dynamic insertion and deletion of particle-based blending point samples collision detected
Point-Based Computer Graphics Mark Pauly 43 Point-Based Computer Graphics Mark Pauly 44
Dynamic Sampling Dynamic Sampling
• Surface distortion varies locally 1. Measure local surface stretch from first fundamental form 2. Split samples that exceed stretch threshold 3. Regularize distribution by relaxation 4. Interpolate scalar attributes
color-coded surface after surface stretch dynamic re-sampling
Point-Based Computer Graphics Mark Pauly 45 Point-Based Computer Graphics Mark Pauly 46
Dynamic Sampling Free-form Deformation
•2D illustration • Interactive modeling session with dynamic sampling
original surface with zero- and one-regions
final surface intermediate steps of deformation
Point-Based Computer Graphics Mark Pauly 47 Point-Based Computer Graphics Mark Pauly 48
8 Results Results
• 3D shape modeling functionality has been •Ab-initio design of an Octopus integrated into Pointshop3D to create a • Free-form deformation with dynamic sampling complete system for point-based shape and from 69,706 to 295,222 points appearance modeling • Boolean operations • Free-form deformation • Painting & texturing •Sculpting •Filtering •Etc.
Point-Based Computer Graphics Mark Pauly 49 Point-Based Computer Graphics Mark Pauly 50
Results Results
• Modeling with synthetic and scanned data • Boolean operations on scanned data • Combination of free-form deformation with • Irregular sampling pattern, low resolution collision detection, boolean operations, models particle-based blending, embossing and texturing
Point-Based Computer Graphics Mark Pauly 51 Point-Based Computer Graphics Mark Pauly 52
Results Conclusion
• Interactive modeling with scanned data • Points are a versatile shape modeling • noise removal, free-form deformation, cut-and- primitive paste editing, interactive texture mapping • Combines advantages of implicit and parametric surfaces • Integrates boolean operations and free- form deformation • Dynamic restructuring • Time and space efficient implementations
Point-Based Computer Graphics Mark Pauly 53 Point-Based Computer Graphics Mark Pauly 54
9 Conclusion References
• Pauly: Point Primitives for Interactive Modeling and • Complete and versatile point-based Processing of 3D Geometry, PhD Thesis, ETH Zurich, 2003 3D shape and appearance modeling • Pauly, Keiser, Kobbelt, Gross: Shape Modeling with Point- sampled Geometry, SIGGRAPH 03 system • Pauly, Kobbelt, Gross: Multiresolution Modeling with Point- • Directly applicable to scanned data sampled Geometry, ETH Technical Report, 2002 • Zwicker, Pauly, Knoll, Gross: Pointshop3D: An Interactive • Suitable for low-cost 3D content creation System for Point-based Surface Editing, SIGGRAPH 02 and rapid proto-typing • Adams, Dutre: Boolean Operations on Surfel-Bounded Solids, SIGGRAPH 03 • Szeliski, Tonnesen: Surface Modeling with Oriented Particle Systems, SIGGRAPH 92 • www.pointshop3d.com
Point-Based Computer Graphics Mark Pauly 55 Point-Based Computer Graphics Mark Pauly 56
10