Reflections and Caustics : Catadioptric Imaging + Refractions/Reflections and Caustics: Photon Mapping Lecture #16 Thanks to Henrik Jensen, John Hart, Ron Fedkiw, Pat Hanrahan, Rahul Swaminathan, Ko Nishino Examples of Imaging Systems New Imaging Applications The application at hand must drive the design of imaging systems. Eyes in Nature Mosquito http://ebiomedia.com/gall/eyes/octopus-insect.html Perspective Imaging Viewpoint Image Detector Single Viewpoint Imaging Viewpoint Image Detector Wide Angle Imaging Fish Eye Lens Catadioptric Imaging Examples: Rees 70, Charles 87, Nayar 88, Examples: Wood 1906, Miyamoto 64, Hall 87, Yagi 90, Hong 91, Yamazawa 95, Bogner 95, Zimmerman 93 , Poelstra 96 , Kuban et al. 94 Nalwa 96, Nayar 97, Chahl & Srinivasan 97 Single Viewpoint Omnidirectional Imaging perspective panorama What’s the Mirror’s Shape ? (with Simon Baker 98) camera scene pinhole normal z mirror z(r) Complete Class of Mirrors c 2 k c2 k2 z r2 1 (k viewpoint r 2 2 4 k (k > 0) c2 c2 2kc2 z r21 (k > 2)(k 2 2k 4 OneShot 360 by RemoteReality (Nayar 97) 4 Megapixel (2000 x 2000) 360 degree still camera Curved Mirrors in Scallop Eyes Telescopic Eye (by Mike Land, Sussex) Non-Single Viewpoint Imaging Detector Viewpoint locus Non-Central Catadioptric Systems Locus of Viewpoints: Caustics Adapted from [ Jensen, 1997 ] Envelope of a Family of Curves Implicit Form of Equation: Parameteric Form: Surface Area of Envelope is Zero: Set Determinant of Jacobian to zero. Vector Product and Surface Area Changing Variables in Calculus Determinant of Jacobian Cardioid’s Caustic = Nephroid Cardioid’s Caustic = Two Cardioids Circle and its Caustics Depends on the Radiant Point Computing the Caustic Burkhard & Shealy, 1973 At the caustic, there exists a Entry Pupil singularity : Det (J(Sr rVr)) 0 Vl(t) Sc(t,r)Sr(t)rVr(t) Sr(t) r Vr(t) Caustic Reflector Conic Catadioptrics Imaging plane Reflector surface: z(t) t d 2 2 2 Vr(t) (t) (t) (e 1)t 2 pt p O Ellipse : e < 1 Parabola : e = 1 p Hyperbola : e > 1 Vi(t) Reflector z(t) Parameters: ( e, p, d ) Solution of Viewpoint Loci Swaminathan, Grossberg & Nayar ICCV’01 Reflector surface: z(t)t (t) (e2 1)t2 2pt p2 Caustic Surface: 3 parameter family of curves (e, p, d) 2p3(d p) 6(d(e2 1) p)p2(d p)t 3p(d p)(d(23e2 e4) 2p)t2 (d2(e2 2)(e2 1)2 d(47e2 3e4)p 2(e4 e2 1)p2)t3 zc(t) e2(2(d p)p2(d p) 6p2(d p)t 3(e2 1)p(d p)t2 (e2 1)(d(e2 1) p)t3) 2(d p)(d de2 p e2p) ((e2 1)t2 2pt p2)3 c(t) e2(2(d p)p2(d p) 6p2(d p)t 3(e2 1)p(d p)t2 (e2 1)(d(e2 1) p)t3) Caustics Example: Elliptic Reflector Entrance Pupil Elliptical Reflector : d (t) e =.75 , p = 1.0 , d = 0.5 p Caustic Elliptic Reflector z(t) Caustic Example: Parabolic Lens Pinhole Parabolic Reflector : e =1.0 , p = 1.0 , d = 1.0 d (t) p Parabolic Reflector Caustic z(t) 3D Caustic Surfaces Caustic for a parabola, source Caustic surface for a source outside the reflector and along placed off-axis with respect to the the axis of symmetry. reflector. ReflectorCaustic All caustics are computed in closed form. Calibration of Non-Central Systems Perspective lens based camera Spherical reflector (ball bearing) Self-Calibration of Caustic Scene point Tx Two views: Point Correspondences Unknown scene points imaged in two views. Estimated Caustic Swaminathan, Grossberg & Nayar,ICCV’01 Z (mm) Ground truth Caustic. Estimated Caustic (mm) The World in an Eye Ko Nishino Shree K. Nayar Columbia University Supported by NSF Ophthalmology Computer Vision •Anatomy, physiology •Iris recognition [Helmholtz 09;…] [Daugman 93;…] HCI Computer Graphics • Gaze as an interface • Vision‐realistic Rendering [Bolt 82;…] [Barsky et al. 02;…] Geometric Model of the Cornea Iris Pupil Sclera Cornea R=7.6mm tb 2.18mm rL 5.5mm eccentricity = 0.5 Finding the Limbus limbus parameters e: radii rx,ry center cx,cy tilt max grx * Ix,y ds g ry * Ix,y ds e rx e ry e Gaussian intensity value Self‐calibration: 3D Coordinates, 3D Orientation How does the World Appear in an Eye? Imaging Characteristics: Viewpoint Locus P Camera Pupil Vr X N r V c i S Z Vt,,r St, rVit, det JVt,,rc 0 [Burkhard and Shealy 73; Swaminathan et al. 01] Viewpoint Loci camera pupil cornea What does the Eye Reveal? Environment Map from an Eye What Exactly You are Looking At Eye Image: Computed Retinal Image: Eyes Reveal … –Where the person is –What the person is looking at –The structure of objects Implications Human Affect Studies: Social Networks Security: Human Localization Advanced Interfaces: Robots, Computers Computer Graphics: Relighting [SIGGRAPH 04] VisualEyesTM http://www.cs.columbia.edu/CAVE/ with Akira Yanagawa Rendering Caustics using Photon Mapping Thanks to Henrik Wann Jensen, Michael Kaiser, Christian Finger, John Hart • What is wrong with this rendering? • With caustics Looking at Water •Light is reflected and refracted at the same time •There is a light pattern on the ground (Caustics) • Shafts of light and caustics: – http://nis‐lab.is.s.u‐tokyo.ac.jp/~nis/cdrom/pg/pg2001_iwa.pdf Caustics by Refraction: Dia‐Caustics •Light is refracted by the water surface •Some spots are stronger illuminated then others Light rays through a water surface dark Bright areas caused by bunching of refracted rays ( 1800s ) Water‐filled glass spheres to focus or condense candlelight onto small areas Caustics by Reflections: Cata‐Caustics Light Transport Notation L Lightsource E Eye S Specular reflection D Diffuse reflection (k)+ one or more k events (k)* zero or more of k events (k)? zero or one k event (k|k’) a k or k’ event Caustics Caustics are formed when light reflected from or transmitted through one or more specular surfaces strikes a diffuse surface. In Light transport notation: LS+DE Comparisons Radiosity Handles only diffuse reflections LD*E. Not suitable for rendering caustics. Photon Mapping Handles all kinds of reflections. It has the following possibilities: L(S|D)*E Good for Caustics. 2-Pass Algorithm First Pass: Generating a Global Photon Map and Caustic Map. Second Pass: Rendering the Photon Map via Distribution Ray Tracing. Storing Photons • Uses a kd-tree – a sequence of axis- aligned partitions – 2-D partitions are lines – 3-D partitions are planes • Axis of partitions alternates wrt depth of the tree • Average access time is O(log n) • Worst case O(n) when tree is severely lopsided • Need to maintain a balanced tree, which can be done in O(n log n) • Can find k nearest neighbors in O(k + log n) time using a heap Photon Mapping + = Direct Illumination Indirect Illumination + = Specular Part Photon Mapping + Caustics = Caustics in Cornell Box Results NEXT WEEK Light Polarization and its applications for Vision Lectures #17 and #18.