Reflections and Caustics : Catadioptric Imaging + Refractions/Reflections and Caustics: Photon Mapping

Lecture #16

Thanks to Henrik Jensen, John Hart, Ron Fedkiw, Pat Hanrahan, Rahul Swaminathan, Ko Nishino Examples of Imaging Systems New Imaging Applications The application at hand must drive the design of imaging systems. Eyes in Nature

Mosquito

http://ebiomedia.com/gall/eyes/octopus-insect.html Perspective Imaging

Viewpoint

Image Detector Single Viewpoint Imaging

Viewpoint

Image Detector Wide Angle Imaging

Fish Eye Lens Catadioptric Imaging

Examples: Rees 70, Charles 87, Nayar 88, Examples: Wood 1906, Miyamoto 64, Hall 87, Yagi 90, Hong 91, Yamazawa 95, Bogner 95, Zimmerman 93 , Poelstra 96 , Kuban et al. 94 Nalwa 96, Nayar 97, Chahl & Srinivasan 97 Single Viewpoint Omnidirectional Imaging

perspective

panorama What’s the Mirror’s Shape ?

(with Simon Baker 98) camera

scene

pinhole

normal   z mirror z(r) Complete Class of Mirrors 2 2 2 2 1 (  c 2 k 2 c k4  viewpoint r z   r       (k > k0)       k 

 2  2   22      2   1    (k > 2)(   c 2 c 2 k c 4 z r     k k OneShot 360 by RemoteReality

(Nayar 97)

4 Megapixel (2000 x 2000) 360 degree still camera

Curved Mirrors in Scallop Eyes

Telescopic Eye (by Mike Land, Sussex)

Non-Single Viewpoint Imaging

Detector

Viewpoint locus Non-Central Catadioptric Systems Locus of Viewpoints: Caustics

Adapted from [ Jensen, 1997 ] Envelope of a Family of Curves

Implicit Form of Equation:

Parameteric Form:

Surface Area of Envelope is Zero: Set Determinant of Jacobian to zero. Vector Product and Surface Area

Changing Variables in Calculus

Determinant of Jacobian Cardioid’s Caustic = Nephroid Cardioid’s Caustic = Two Cardioids Circle and its Caustics

Depends on the Radiant Point Computing the Caustic

Burkhard & Shealy, 1973 At the caustic, there exists a Entry Pupil singularity :

Det (J r(S rVr ))  0 Vl (t)

Sc (t,r) Sr (t) rVr (t) Sr (t) r Vr (t)

Caustic

Reflector Conic Catadioptrics

Imaging plane Reflector surface: (t) z(t)  t d 2 2 2 Vr (t)   (t)  (e 1)t  2 pt  p O Ellipse : e < 1 Parabola : e = 1 p Hyperbola : e > 1 Vi (t)

Reflector z(t)

Parameters: ( e, p, d ) Solution of Viewpoint Loci

Swaminathan, Grossberg & Nayar ICCV’01 Reflectorz(t) surface:t  (t) (e 21)t 22pt p 2 2 2 2 4 4 2 2 3 2)( 1) (4 7 3 ) 2( 1) ) Caustic Surface:3 3 2parameter2 family of2 4curves (e, p, d) 2 ( ) 6( ( 1) ) ( 2) 3 ( 2)(d(2 3e 2e ) 2 2 2 2 3 p d  p  d(2e(  ) p(p d) p6t ( p d ) p 3(  1) ( ) ( 1)( ( 1) ) ) 2 2 2 2p)t (d (e e d e e p e e p t ( )           z t  e d p p cd p p d p t e p d p t e d e 2 p2t 2 2 2 3     2( )(   ) (( 1)  2 ) ( )  2 2 2 2 2 2 2 3 (2( ) ( ) 6 ( ) 3( 1) ( ) ( 1)( ( 1) ) ) d p d de p e p e t pt p        c t e d p p d p p d p t e p d p t e d e p t            Caustics Example: Elliptic Reflector (t)

Entrance Pupil Elliptical Reflector : d  e =.75 , p = 1.0 , d = 0.5

p

Caustic

Elliptic Reflector z(t) Caustic Example: Parabolic Lens

Pinhole Parabolic Reflector : e =1.0 , p = 1.0 , d = 1.0 d  (t)

p Parabolic Reflector

Caustic z(t) 3D Caustic Surfaces

Caustic for a parabola, source Caustic surface for a source outside the reflector and along placed off-axis with respect to the the axis of symmetry. reflector.

ReflectorCaustic

All caustics are computed in closed form. Calibration of Non-Central Systems

Perspective lens based camera

Spherical reflector (ball bearing) Self-Calibration of Caustic

Scene point

Tx Two views: Point Correspondences

Unknown scene points imaged in two views. Estimated Caustic

Swaminathan, Grossberg & Nayar,ICCV’01

Z (mm)

Ground truth Caustic.

Estimated Caustic

 (mm) The World in an Eye

Ko Nishino Shree K. Nayar

Columbia University

Supported by NSF Ophthalmology Computer Vision •Anatomy, physiology •Iris recognition [Helmholtz 09;…] [Daugman 93;…]

HCI Computer Graphics • Gaze as an interface • Vision‐realistic Rendering [Bolt 82;…] [Barsky et al. 02;…] Geometric Model of the Cornea

Iris Pupil Sclera

Cornea R=7.6mm

tb 2.18mmrL 5.5mm eccentricity = 0.5 Finding the Limbus

 limbus parameters e : radii rx,ry  center cx,cy  tilt      maxxg r * Ix,y ds yg r * I x,y ds e  e   e Gaussian intensity valuex r y r Self‐calibration: 3D Coordinates, 3D Orientation How does the World Appear in an Eye? Imaging Characteristics: Viewpoint Locus

P Camera Pupil

Vr X N r V c i S Z V t, ,rSt,  Vr i t,  detJVt, r,c  0 [Burkhard and Shealy 73; Swaminathan et al. 01] Viewpoint Loci

camera pupil

cornea What does the Eye Reveal?

Environment Map from an Eye What Exactly You are Looking At Eye Image:

Computed Retinal Image:

Eyes Reveal …

–Where the person is

–What the person is looking at

–The structure of objects Implications

Human Affect Studies: Social Networks

Security: Human Localization

Advanced Interfaces: Robots, Computers

Computer Graphics: Relighting [SIGGRAPH 04] VisualEyesTM http://www.cs.columbia.edu/CAVE/

with Akira Yanagawa Rendering Caustics using Photon Mapping

Thanks to Henrik Wann Jensen, Michael Kaiser, Christian Finger, John Hart • What is wrong with this rendering? • With caustics

Looking at Water

•Light is reflected and refracted at the same time •There is a light pattern on the ground (Caustics) • Shafts of light and caustics: – http://nis‐lab.is.s.u‐tokyo.ac.jp/~nis/cdrom/pg/pg2001_iwa.pdf Caustics by Refraction: Dia‐Caustics

•Light is refracted by the water surface •Some spots are stronger illuminated then others Light rays through a water surface

dark Bright areas caused by bunching of refracted rays ( 1800s )

Water‐filled glass spheres to focus or condense candlelight onto small areas Caustics by Reflections: Cata‐Caustics Light Transport Notation

 L Lightsource

 E Eye

 S Specular reflection

 D Diffuse reflection

 (k)+ one or more k events

 (k)* zero or more of k events

 (k)? zero or one k event

 (k|k’) a k or k’ event Caustics

 Caustics are formed when light reflected from or transmitted through one or more specular surfaces strikes a diffuse surface.

In Light transport notation: LS+DE Comparisons

 Radiosity  Handles only diffuse reflections LD*E. Not suitable for rendering caustics.

 Photon Mapping  Handles all kinds of reflections.  It has the following possibilities: L(S|D)*E Good for Caustics. 2-Pass Algorithm

 First Pass:

 Generating a Global Photon Map and Caustic Map.

 Second Pass:

 Rendering the Photon Map via Distribution Ray Tracing. Storing Photons

• Uses a kd-tree – a sequence of axis- aligned partitions – 2-D partitions are lines – 3-D partitions are planes • Axis of partitions alternates wrt depth of the tree • Average access time is O(log n) • Worst case O(n) when tree is severely lopsided • Need to maintain a balanced tree, which can be done in O(n log n) • Can find k nearest neighbors in O(k + log n) time using a heap Photon Mapping

+ =

Direct Illumination Indirect Illumination

+ =

Specular Part Photon Mapping

+

Caustics

= Caustics in Cornell Box Results NEXT WEEK

Light Polarization and its applications for Vision

Lectures #17 and #18