Interaction With 3D Geometry
Stan Melax Graphics Software Engineer, Intel 3D Interaction Happens with Geometric Objects
Rigid Body Skinned Characters Soft Body { { { Mesh geometry; Mesh geometry; Mesh geometry; Vec3 position; Vec3 position[]; Springs connectivity; Quat orientation; Quat orientation[]; }; }; }; Agenda – Interacting with 3D Geometry
Practical topics among: ● Core Geometry Concepts ● Convex Polyhedra ● Spatial and Mass Properties ● Soft Geometry and Springs
With examples and implementation issues. Warning: Some Math Ahead
푚 푚 푚 푣 v 푢 × 푣 00 01 02 푥 푀푣 = 푚10 푚11 푚12 푣푦 v 푚 푚 푚 푣 u 20 21 22 푧 u v 푢푥푣푥 푢푥푣푦 푢푥푣푧 푢 ∙ 푣 > 0 푢푣푇 = 푢푦푣푥 푢푦푣푦 푢푦푣푧 u u 푢푧푣푥 푢푧푣푦 푢푧푣푧
v 휕푓푥 휕푓푥 휕푓푥 v 휕푣푥 휕푣푦 휕푣푧 u v 푑푓 휕푓푦 휕푓푦 휕푓푦 = 푑푣 휕푣 휕푣 휕푣 u 푥 푦 푧 b 휕푓푧 휕푓푧 휕푓푧 푢 ∙ 푣 < 0 a 푣 × 푢 휕푣푥 휕푣푦 휕푣푧 Vector Arithmetic Dot Product Cross Product Matrix Stuff What’s an “outer product”? Familiar dot or inner product:
푣푥 푇 푢 푢 푢 푢 ∙ 푣 = 푢 푣 = 푥 푦 푧 푣푦 = 푢푥푣푥 + 푢푦푣푦 + 푢푧푣푧 푣푧
Outer product:
푢푥 푢푥푣푥 푢푥푣푦 푢푥푣푧 푇 푢⨂푣 = 푢푣 = 푢푦 푣푥 푣푦 푣푧 = 푢푦푣푥 푢푦푣푦 푢푦푣푧 푢푧 푢푧푣푥 푢푧푣푦 푢푧푣푧 Outer Product - Geometric View
v v v
u u u 푢 = 1 푢 ∙ 푣 푢(푢 ∙ 푣) Distance of v along u (scalar) Projection of v along u (vector)
Outer product 풖 ⊗ 풖 of a 푢 푢 ∙ 푣 ≠ 푢 ∙ 푢 푣 unit vector 풖 projects any vector along that direction. 푢 푢 ∙ 푣 = 푢 ⊗ 푢 푣 Outer Product for Plane Projection
−푢 푢 ∙ 푣 −푢 푢 ∙ 푣 v v 푣 − 푢 푢 ∙ 푣 v u u 푢 푢 ∙ 푣 u Remove portion of v or (푢 ⊗ 푢)푣 that runs along u
푣 − 푢 푢 ∙ 푣 = 퐼푣 − 푢 ⊗ 푢 푣 = (퐼 − 푢 ⊗ 푢 )푣
푰 − 풖 ⊗ 풖 projects any vector onto plane with normal 풖. Numerical Precision Issues
3 “collinear” points Triangles and Planes Could use: ● Specify Front and Back 퐴푥 + 퐵푦 + 퐶푧 == 퐷 Prefer convention: 퐴푥 + 퐵푦 + 퐶푧 + 퐷 == 0 c 0 0 0 푝푤 Given a point [xyz] its 푝푥 푝푦 푝푧 distance above plane is: a b 푝푥 푥 + 푝푦 푦 + 푝푧 푧 + 푝푤
• <0 below Triangle (푎 푏 푐) Plane 푝 = 푝 푝 푝 푝 푥 푦 푧 푤 • =0 on plane CCW winding • >0 above Intersection of 3 planes p0 p2
푝0푥 푣푥 + 푝0푦 푣푦 + 푝0푧 푣푧 + 푝0푤 == 0 p1 푝1푥 푣푥 + 푝1푦 푣푦 + 푝1푧 푣푧 + 푝1푤 == 0
푝2푥 푣푥 + 푝2푦 푣푦 + 푝2푧 푣푧 + 푝2푤 == 0 p0
푝0푥 푝0푦 푝0푧 푝0푤 푣푥 푃 = 푝1푥 푝1푦 푝1푧 , 푏 = 푝1푤 , 푣 = 푣푦 푝2 푝2 푝2 푝2 푣푧 푥 푦 푧 푤 v 푃푣 = −푏 p1 p2 푣 = −푃−1푏 What if we have 4 planes? Example:
Floor Only Walls Roof Planes Roof Planes overhead view As house grows bottom up, how will the 4 roof planes come together at the top of this house? Determining How 4 Planes Meet Note: these are top down views p3 of a convex region p0 p2 ?
p1
푝0 푝0 If 4 planes meet at a point xyz 푥 푦 푝0푧 푝0푤 푥 0 then we’ve found a solution to: 푝1푥 푝1푦 푝1푧 푝1푤 푦 0 푧 = 푝2푥 푝2푦 푝2푧 푝2푤 0 1 푝3푥 푝3푦 푝3푧 푝3푤 0 Only possible if matrix singular. Determining How 4 Planes Meet p3 p0 p2 ?
p1
d<0 d==0 d>0
푝0푥 푝1푥 푝2푥 푝3푥 푝0 푝1 푝2 푝3 푝0 푝1 푝2 푝3 푦 푦 푦 푦 Notes: these are top down views of a convex region. d = = Planes well “behaved” all point same way. 푝0푧 푝1푧 푝2푧 푝3푧 Planes in CCW order. 푝0푤 푝1푤 푝2푤 푝3푤 Det of any 3x3 subblock from first 3 xyz rows is >0.
Intersect Line Plane 푝푥푦푧 ∙ 푣푡 + 푝푤
푝푥푦푧 v1 푑1 = 푝푥푦푧 ∙ 푣1 +푝푤 풅ퟏ
p time 0 1 푝 푝 푝 t=? 푝푥푦푧 = 푥 푦 푧 v0 푑0 = 푝푥푦푧 ∙ 푣0 +푝푤 풅ퟎ
푑푖푠푡 = 푝푤 distance above plane distance above
−푑 푝푥푦푧 ∙ 푣0 + 푝푤 푖푚푝푎푐푡 = 푣 + 푣 − 푣 푡 푡 = 0 = − 0 1 0 (푝푥푦푧 ∙ 푣1 + 푝푤) − (푝푥푦푧 ∙ 푣0 + 푝푤) 푑1 − 푑0 Simple Ray Triangle Intersection Test
● Intersect ray with plane and get a point p. p ● For each edge va to vb ● Cross product with p-va ● Dot result with tri normal n ● <0 means outside ● Backside hit if dot(n,ray)>0 Ray Mesh Intersection
● Check Against Every Polygon ● (spatial structures can rule out many quickly) Multiple impact points – take closest. ● Detecting a “hit” - might not be closest ● Numerical Robustness – don’t slip between two adjacent triangles. ● Mesh “intact” – t-intersections, holes Numeric precision can result in plane intersection point landing just caused by missing triangles. outside each time, thus missing both neighbors. Solid Geometry
● “Water-Tight” borderless manifold mesh: ● Every edge has 1 adjacency edge going other way ● Consistent winding. Polygon normals all face to exterior. ● The mesh is the boundary representation - the infinitely thin surface that separates solid matter from empty space. Inside/Outside Test of a point
● Cast a long ray from point ray ● point is inside if it first hits the backside of a polygon. ● point is outside the object if it first hits a front side or p nothing at all. Convex Polyhedra Convex Mesh 풂 풃 Math textbook: 푲 퐾 푐표푛푣푒푥 ↔ ∀푎, 푏 ∈ 퐾 → 푎푏 ⊆ 퐾
● Neighboring face normals tilt away. ● Volume bounded by a number of planes. ● Every vertex lies at or below every face.
Many algorithms much easier when using convex shapes instead of general meshes.
Convex In/Out test
A point is inside a convex volume if it lies under every plane boundary. p
w No testing with vertices or edges. q
Convex Ray Intersection v0 v0 v0 v1 v0
● Crop Ray with all front facing planes: 푛 ∙ 푣1 − 푣0 < 0 ● Impact at v0 if under all backside planes Convex Line-Segment Intersection v0
v0 v1 v1 v0 v1
● Trim v0 for Front facing planes 푛 ∙ 푣1 − 푣0 < 0 ● Trim v1 for Back facing planes 푛 ∙ 푣1 − 푣0 > 0 Convex-Convex Contact
● Separating Axis Theorem – Two non touching convex polyhedra are separated by a plane. ● The contact between two touching convex polyhedra will be either ● A point ● A line segment ● A convex polygon
Physics engines like convex objects Convex Hull from points
Convex Hull - smallest convex polyhedron that contains all the points. ● Convex hull of a mesh will contain the mesh. ● Often a sufficient proxy for interaction and collision Techniques ● Expanding outward: QuickHull [Eddy ’77] ● Reducing inward: Gift Wrapping [Chand ’70]
Compute 3D Convex Hull
● Start with 2 back to back triangles. (or a tetrahedron) ● Find a vertex outside current volume (above faces). ● Find edge loop silhouette (around all faces below point) ● Replace with new fan of polys ● Remove Folds (if any) ● Rinse and Repeat Hull Numerical Robustness
Grow Hull Generated skinny Flip edge to fix triangle with bad normal. Hull Tri-Tet Implementation
● Simple Triangle-mesh based approach ● When point d above any [abc] add tetrahedron [abcd] (triangles [acb][dab][dbc][dca]) to mesh. ● Prune any back-to-back tris such as [abc] and [bac]
d d d
a c a c a c
b b b Hull Tri-Tet Numeric Robustness a a a a
c c e c e c e e b d d b d b d b
● 5 points {a,b,c,d,e} are coplanar-ish at one end of the point cloud ● But next point e tests above triangle [abc] but below [adb]. ● Silhouette is {abc} for which we extrude a new tetrahedron up to e ● This produces triangles [eca],[ebc] (blue) facing the right way and [eab] (red) facing the wrong way – based on known interior point. ● This provides the hindsight to see that [adb] should also be extruded Simplified Convex Hull
● Off-the-shelf solutions typically generate the complete hull. ● All hull vertices may not be needed and may be inefficient for usage. ● Instead use greedy incremental algorithm picking next vertex that increases hull size the most. ● Stop when vertex count or error Full Hull Simplified threshold reached
Minimize Number of Planes vs Points
Minimize Planes: ● Compute full hull
푝푥 푝푦 푝푧 ● Dual points 푝푤 푝푤 푝푤 ● Compute simplified hull 12 Vertices 20 Vertices ● Take Dual 20 Planes 12 Planes Convex Decomposition
3DS Max Screenshot
Manual Creation of Automatic Mesh Use skinned mesh Collision Proxy Decomposition bone weights to [Ratcliff], [Mamou] create collision hulls Example: Convex Bones For Collisions
Ragdolls on Stairs
Hand catching coins Constructive Solid Geometry Boolean Operations
Solid Geometry Boolean Operations Applications for Booleans
Art tools (already have CSG) ● Modeling and level design
Game Usages (less fidelity required): ● Destruction ● Geomod (tunneling) A convex based approach may suffice. Convex Cropping and Intersection
● Like general mesh cropping, but only 1 open loop that is itself a convex polygon.
● Intersecting two convexes can be done by cropping one with all the planes of the other
● Non-convex operands can be implemented as a union Convex Cropping Robustness
● Floating point issues can occur even in this most basic of mesh operations. ● One solution is to utilize plane equations to determine face/plane connectivity.
Bad Slice Correct Slice Pre Slice Robustness with Quantum Planes Let all planes p of CSG Boolean operands satisfy: 푠 푝 푠 푝 ∀푝∃푠 ∶ 푠푝 = 푥 푦 푠 푝푧 푠 푝푤 , 푠 푝{푥,푦,푧,푤} ∈ ℤ, 푠푝{푥,푦,푧} ≤ 푘
푠 푝0 푠 푝0 푠 푝0 0 푥 0 푦 0 푧 푠0푝0푤 Recall how vertices are the −1 푀 = 푠1푝1푥 푠1푝1푦 푠1푝1푧 푣 = −푀 푠1푝1푤 intersection of 3 or more planes 푠2푝2푥 푠2푝2푦 푠2푝2푧 푠2푝2푤
푎 ∃푎, 푏: 푣 = , 푎, 푏 ∈ ℤ , 푏 ≤ det 푀 ≤ 6푘3 Generated points have {푥,푦,푧} 푏 Finite denominator
∴ ∃휀∀푢, 푣: 푢 − 푣 < 휀 ↔ 푢 = 푣 A fixed epsilon can now be used to indicate if two points are the same. Destruction – geometry modification
Texturing break Objects in Motion Spatial Properties
Spatial and Mass Properties
● With a solid mesh we can correctly derive properties that affect motion: ● Volume ● Center of mass ● Covariance (and 3x3 Inertia Tensor) Triangle Center and Area
c n b c
a b a a b c 푎 + 푏 + 푐 (푏 − 푎) × (푐 − 푏) center = abs area = Given a pre set 3 2 normal n, result is signed. i.e. Center of mass. 푏 − 푎 × 푐 − 푏 ∙ 푛 So area could be There are other ways area = negative. to define “center”. 2 Cross Product - Edge Choice Irrelevant
c c c
a b a b a b
푏 − 푎 × 푐 − 푏 = 푐 − 푏 × 푎 − 푐 = 푎 − 푐 × 푏 − 푎
All the same Area of Polygon (2D) Triangle Summation: ? ● Pick a reference point p. Normal n known. ● For each edge (a,b) sum area of triangle 1 (p,a,b): 푎 − 푝 × 푏 − 푎 ∙ 푛 2 p ● Signed Area - upside down triangles cancel out extra area if p outside. - =
p p Solid’s Area Weighted Normals Cancel
Sum of cross w products for each w-u v tetrahedron face is zero. u
푢 × 푤 + 푤 × 푣 + 푣 × 푢 + 푣 − 푢 × 푤 − 푢 = 0 Polygon Normal ? ● Pick reference point p (origin) ● For each edge (ab) in polygon, sum cross product:
푎 − 푝 × (푏 − 푎) p 퐸푑𝑔푒 (푎,푏)
+ = For a trapezoid explanation search for “Newell Normal” p p Tetrahedron Volume and Center
d d
a b b c a c
(푐 − 푎) × 푏 − 푎 ∙ (푑 − 푎) center 푎 + 푏 + 푐 + 푑 volume = = 6 of mass 4
= 푐 − 푎 푏 − 푎 푑 − 푎 ÷ 6
Triple product in determinant form Tetrahedron Integration d General Form: w 훼 훼 훽 푢 푣(1 − ) 푤(1− − ) v b 푢 푢 푣 푓 훼 + 훽 + 훾 푑훾 푑훽 푑훼 a c 0 0 0 u Letting f(X)=1, we get our volume: Edges: u,v,w 훼 훼 훽 a=[0 0 0] (origin) 푢 푣(1 − ) 푤(1− − ) 푢 푢 푣 푢푣푤 1 푑훾 푑훽 푑훼 = 0 0 0 6 Letting f(X)=X/vol for center of mass (relative to vertex a): 훼 훼 훽 푢 푣(1 − 푢) 푤(1− 푢 − 푣) 훼 + 훽 + 훾 푢 + 푣 + 푤 푑훾 푑훽 푑훼 = 0 0 0 푢푣푤 6 4 Tetrahedral Summation (3D)
origin
푣표푙푢푚푒 = 푢 푣 푤 /6 (푢,푣,푤)∈푚푒푠ℎ
푐푒푛푡푒푟 푢 + 푣 + 푤 푢 푣 푤 = /푣표푙(푚푒푠ℎ) 표푓 푚푎푠푠 4 6 (푢,푣,푤)∈푚푒푠ℎ Center of Mass Affects Gameplay Catapult geometry: Inertia Calculation
푚표푚푒푛푡 표푓 = 푟2 = 푥2 + 푦2 = 푥2 + 푦2 2D: 푖푛푒푟푡푖푎
Variance in x Variance in y Fixed Axis
3D: 3x3 Inertia Tensor 푦푦 + 푧푧 −푥푦 −푥푧 −푥푦 푥푥 + 푧푧 −푦푧 Related to covariance −푥푧 −푦푧 푥푥 + 푦푦 Covariance (origin = center of mass)
● If object was a collection of point masses v
2 푣푥 푣푥푣푦 푣푥푣푧 푇 2 푣푣 = 푣푦푣푥 푣푦 푣푦푣푧 2 푣푧푣푥 푣푧푣푦 푣푧
● Single Tetrahedron (0,u,v,w)…
1 1−푎 1−푎−푏 푎푢 + 푏푣 + 푐푤 ⨂(푎푢 + 푏푣 + 푐푤) 푑푐 푑푏 푑푎 ∗ 푣표푙_푎푑푗 0 0 0 Tetrahedron (0,u,v,w) Covariance u,v,w are vectors from center of mass to triangle on mesh Inertia Tetrahedral Summation
푦푦 + 푧푧 −푥푦 −푥푧 푖푛푡푒푟푡푖푎: 푇 = −푥푦 푥푥 + 푧푧 −푦푧 −푥푧 −푦푧 푥푥 + 푦푦 Center Of Mass
Sum for each triangle uvw, Covariance matrix C:
2푢푖푢푗 + 2푣푖푣푗 + 2푤푖푤푗 + 푢푖푣푗 + 푣푖푤푗 + 푤푖푢푗 + 푢푖푤푗 + 푣푖푢푗 + 푤푖푣푗 푢 푣 푤 푐 = 푖,푗 120 6 푢,푣,푤 ∈푚푒푠ℎ−푐표푚
푖푛푒푟푡푖푎: 푇 = 퐼푑푒푛푡푖푡푦 (푐0,0 + 푐1,1 + 푐2,2) − 퐶 Inertia Tensor and Object Motion
휔 = 푞 푇−1 푞−1 ℎ
휔 → 푠푝푖푛 ℎ → 푚표푚푒푛푢푚 푞 → 표푟푖푒푛푡푎푡푖표푛 푇 → 퐼푛푒푟푡푖푎 Proof it’s the same: Time Integration 휔 lim 푠 → 푑푡, 1 Updating state to the next time step. ( 휔 푑푡)→0 2 휔 푠 ∗ 푞 = 0001 ∗ 푞 + 푑푡, 0 ∗ 푞 ● Position: 푝푡+푑푡 = 푝푡 + 푣푒푙표푐푖푡푦 ∗ 푑푡 푡 푡 2 푡 휔 ● Orientation: 푠 ∗ 푞 = 푞 + , 0 ∗ 푞 푑푡 푡 푡 2 푡
Build a Quat for Multiplication Add Derivative
휔 휔 푑푡 휔 푑푡 푑푞 푠 = sin ( ), cos ( ) 푞푡+푑푡 = 푞푡 + 푑푡 휔 2 2 푑푡 휔 푞 = 푠 ∗ 푞 푞 = 푞 + 푞 푑푡 푡+푑푡 푡 푡+푑푡 푡 2 푡 Time Integration without Numerical Drift
휔(푞 ) 푘 = 푡 ∗ 푞 1 2 푡 휔(푞 + 푘 ∗ 푑푡/2) 푑푡 푘 = 푡 1 ∗ (푞 + 푘 ∗ ) 2 2 푡 1 2 휔(푞 + 푘 ∗ 푑푡/2) 푑푡 푘 = 푡 2 ∗ (푞 + 푘 ∗ ) 3 2 푡 2 2 휔(푞 + 푘 ∗ 푑푡) 푘 = 푡 3 ∗ (푞 + 푘 ∗ 푑푡) 4 2 푡 3
휔(푞푡) 푑푡 푑푡 푑푡 푑푡 푞 = 푞 + ∗ 푞 ∗ 푑푡 푞푡+푑푡 = 푞푡 + 푘1 ∗ + 푘2 ∗ + 푘3 ∗ + 푘4 ∗ 푡+푑푡 푡 2 푡 6 3 3 6 Forward Euler Runge Kutta Time Integration Euler vs RK4
Forward Euler: • Spin drifts toward principle axis • Energy gained
Runge Kutta • Spin orbits as expected • Energy stays constant Soft Body Objects Soft Body Meshes
● Every vertex has its own position and velocity ● Vertices are connected via springs or constraints.
Object Construction
● Connection topology can differ from visual mesh ● Stiffness can vary to simulate different objects and materials
Cloth Cube Lattice Table Time Integration – Simulating Soft Body
Kinematic Dynamic ● Connections as constraints ● Connections as springs ● Iterative position projection ● Forward Euler can only handle ● Easy to Implement weak “springy” forces ● Numerically Robust (will ● Implicit Integration required never explode) for stiff springs ● Most common system used ● Damped behavior ● May not converge under ● Converges to force-correct stress (compression or state stretch) ● Harder to Implement
Kinematic Solver
Realistic Behavior
Easy To Implement
Algorithm: repeat a few times for each constraint move endpoints toward rest-length Implicit Integration Spring Network
● Forward Euler −1 푣푡+푑푡 = 푣푡 + 푚 푓표푟푐푒푡 푑푡
● Implicit Euler −1 푣푡+푑푡 = 푣푡 + 푚 푓표푟푐푒푡+푑푡 푑푡
휕푓 휕푓 푝표푠푖푡푖표푛 푝 푓표푟푐푒 = 푓 = 푓 + Δ푝 + Δ푣 , 푡+푑푡 푡+푑푡 푡 휕푝 휕푣 푣푒푙표푐푖푡푦 푣
Derivatives of Force 휕푓푥 휕푓푥 휕푓푥 휕푣푥 휕푣푦 휕푣푧 Force at spring 푝 푑푓 휕푓푦 휕푓푦 휕푓푦 푓 = −푘 푝 − 푟 = endpoint 푝 푑푣 휕푣푥 휕푣푦 휕푣푧 휕푓푧 휕푓푧 휕푓푧 f 휕푣푥 휕푣푦 휕푣푧 Compressed Spring: Jacobian 0 p r f
Stretched Spring: 0 p
How force changes end start Δ푓 = - = Δ푓 as endpoint moves = = −푘 along spring direction: Δ푝 Δ푝 Derivatives of Force – Endpoint moves orthogonal
Compressed: Stretched: f p f 푝푠푡푎푟푡 0 0 Δ푝 Δ푝 p 푝푒푛푑 f f Δ푓 = - = Δ푓 = - =
How force changes Δ푓 = or Δ푓 푟 as endpoint moves = −푘 (1 − ) lateral to spring dir: Δ푝 = Δ푝 푝 Derivatives of Force – 3x3 Jacobian General change in Force for a given change in position:
휕푓 푝 ⊗ 푝 푝 ⊗ 푝 푟 = −푘 + 퐼 − 1 − 휕푝 푝 ∙ 푝 푝 ∙ 푝 푝 Avoid compression singularity by clamping r/||p|| .
Solve for Δ푣 in linear system: 휕푓 휕푓 휕푓 퐴 Δ푣 = 푏 , 퐴 = 퐼 − 푑푡 − 푑푡2, 푏 = 푓 푑푡 + 푣 푑푡2 휕푣 휕푝 휕푝
(note: slightly oversimplified) Implicit Integration Force Based
Stiff Springs
Realistic Behavior
Responsive
Convergence (Jitter Free) Interacting with 3D Geometry
Summary 휇푚푚훼푟훾
Variety of topics: Moving beyond triangles, dot and cross product: ● Objects as 3D Solids ● Convex parts Volume ● Spatial props Integration ● “2nd Year” Maths ● Object Motion Time ● Stiff systems Integration
Q&A
Stan Melax [email protected]
Hand Tracking – if someone asks….
Tracking model Depth data Fit model