Cats + mirrors + face filters [reddit – juicysox] [Isa Milefchik (1430 HTA Spring 2020)] [Madelyn Adams (student Spring 2019)] Zoom protocol Please: • Cameras on (it really helps me to see you) • Real names • Mics muted • Raise hands in Zoom for questions, unmute when I call • I will ask more often for questions Project 4 – due Friday • Both parts – Written – Code Project 5 • Questions and code due Friday April 10th Final group project • Groups of four • Groups of one are discouraged – you need a good reason. • Group by timezone where possible; use Piazza • We’ll go over possible projects at a later date Questions • What else did I miss? By Suren Manvelyan, http://www.surenmanvelyan.com/gallery/7116 By Suren Manvelyan, http://www.surenmanvelyan.com/gallery/7116 By Suren Manvelyan, http://www.surenmanvelyan.com/gallery/7116 What is a camera? Camera obscura: dark room Known during classical period in China and Greece (e.g., Mo-Ti, China, 470BC to 390BC) Illustration of Camera Obscura Freestanding camera obscura at UNC Chapel Hill Photo by Seth Ilys James Hays James, San Francisco, Aug. 2017 Camera obscura / lucida used for tracing Lens Based Camera Obscura, 1568 Camera lucida drawingchamber.wordpress.com Tim’s Vermeer Vermeer, The Music Lesson, 1665 Tim Jenison (Lightwave 3D, Video Toaster) Tim’s Vermeer – video still First Photograph Oldest surviving photograph Photograph of the first photograph – Took 8 hours on pewter plate Joseph Niepce, 1826 Stored at UT Austin Niepce later teamed up with Daguerre, who eventually created Daguerrotypes Dimensionality Reduction Machine (3D to 2D) 3D world 2D image Point of observation Figures © Stephen E. Palmer, 2002 Holbein’s The Ambassadors - 1533 Holbein’s The Ambassadors – Memento Mori 2D IMAGE TRANSFORMS Parametric (global) transformations T p = (x,y) p’ = (x’,y’) Transformation T is a coordinate-changing machine: p’ = T(p) What does it mean that T is global? – T is the same for any point p T can be described by just a few numbers (parameters) For linear transformations, we can represent T as a matrix p’ = Tp x' x = T y' y Common transformations Original Transformed Translation Rotation Scaling Perspective Affine Slide credit (next few slides): A. Efros and/or S. Seitz Scaling • Scaling a coordinate means multiplying each of its components by a scalar • Uniform scaling means this scalar is the same for all components: 2 Scaling • Non-uniform scaling: different scalars per component: X 2, Y 0.5 Scaling • Scaling operation: x'= ax y'= by • Or, in matrix form: x' a 0x = y' 0 b y scaling matrix S 2-D Rotation y (x’, y’) (x, y) x’ = x cos() - y sin() y’ = x sin() + y cos() x 2-D Rotation This is easy to capture in matrix form: x' cos( ) − sin ( )x = y' sin ( ) cos( ) y R Even though sin() and cos() are nonlinear functions of , – x’ is a linear combination of x and y – y’ is a linear combination of x and y What is the inverse transformation? – Rotation by – −1 T – For rotation matrices R = R Basic 2D transformations s 0 x' x x x' 1 x = = x y' 0 s y y y' y 1 y Scale Shear x x' cos − sin x x 1 0 t = = x y y' sin cos y y 0 1 t y 1 Rotate Translate x x a b c Affine is any combination of = y translation, scale, rotation, and shear y d e f 1 Affine Affine Transformations x x a b c Affine transformations are combinations of = y • Linear transformations, and y d e f 1 • Translations or Properties of affine transformations: • Lines map to lines x' a b c x • Parallel lines remain parallel y' = d e f y • Ratios are preserved 1 0 0 1 1 • Closed under composition 2D image transformations (reference table) ‘Homography’ Szeliski 2.1 Projective Transformations Projective transformations are combos of x' a b c x y' = d e f y • Affine transformations, and • Projective warps w' g h i w Properties of projective transformations: • Lines map to lines • Parallel lines do not necessarily remain parallel • Ratios are not preserved • Closed under composition • Models change of basis • Projective matrix is defined up to a scale (8 degrees of freedom) Cameras and World Geometry How tall is this woman? How high is the camera? What is the camera rotation wrt. world? Which ball is closer? James Hays Let’s design a camera Idea 1: Put a sensor in front of an object Do we get a reasonable image? sensor Slide source: Seitz Let’s design a camera Idea 2: Add a barrier to block most rays – Pinhole in barrier – Only sense light from one direction. • Reduces blurring. – In most cameras, this aperture can vary in size. sensor Slide source: Seitz Pinhole camera model f c Real object f = Focal length c = Optical center of the camera Figure from Forsyth Projection: world coordinates→image coordinates X . P = Y Image center Z (u , v ) .0 0 f . Z Y V Camera Center .U (0, 0, 0) U p = f f V U = − X * V = −Y * Z Z p = distance from image center What is the effect if f and Z are equal? Projective Geometry Length (and so area) is lost. Who is taller? Which is closer? Length and area are not preserved A’ C’ B’ Figure by David Forsyth Projective Geometry Angles are lost. Parallel? Perpendicular? Projective Geometry What is preserved? • Straight lines are still straight. Vanishing points and lines Parallel lines in the world intersect in the projected image at a “vanishing point”. Parallel lines on the same plane in the world converge to vanishing points on a “vanishing line”. E.G., the horizon. Vanishing Point Vanishing Point Vanishing Line Vanishing points and lines Vertical vanishing point (at infinity) Vanishing Vanishing point point Slide from Efros, Photo from Criminisi Pinhole camera model f c Real object f = Focal length c = Optical center of the camera Forsyth Projection: world coordinates→image coordinates X . P = Y Image center Z (u , v ) .0 0 f . Z Y V Camera Center .U (0, 0, 0) U p = f f V U = − X * V = −Y * Z Z p = distance from image center What is the effect if f and Z are equal? Projective geometry • 2D point in cartesian = (x,y) coordinates • 2D point in projective = (x,y,w) coordinates Y Projector X Idea from www.tomdalling.com Projective geometry • 2D point in cartesian = (x,y) coordinates • 2D point in projective = (x,y,w) coordinates Y Projector W X Idea from www.tomdalling.com Varying w w1 w2 < w1 Y Y Projector Projector X X Projected image becomes smaller. Projective geometry • 2D point in projective = (x,y,w) coordinates – w defines the scale of the projected image. – Each x,y point becomes a ray! Y Projector W X Projective geometry • In 3D, point (x,y,z) becomes (x,y,z,w) • Perspective is w varying with z: – Objects far away are appear smaller C’ B’ Homogeneous coordinates Converting to homogeneous coordinates 2D (image) coordinates 3D (scene) coordinates Converting from homogeneous coordinates 2D (image) coordinates 3D (scene) coordinates Homogeneous coordinates Scale invariance in projection space x kx kx x k y = ky kw = w ky y kw w w kw Homogeneous Cartesian Coordinates Coordinates E.G., we can uniformly scale the projective space, and it will still produce the same image -> scale ambiguity Homogeneous coordinates • Projective • Point becomes a line To homogeneous From homogeneous Song Ho Ahn Lenses Real cameras aren’t pinhole cameras. Home-made pinhole camera Why so blurry? http://www.debevec.org/Pinhole/ Shrinking the aperture Integrate over fewer angles Less light gets through [Steve Seitz] Shrinking the aperture Less light gets through Why not make the aperture as small as possible? • Less light gets through • Diffraction effects… [Steve Seitz] Shrinking the aperture - diffraction Light diffracts as wavelength of aperture equals wavelength of light The reason for lenses Slide by Steve Seitz Let’s design a camera Idea 2: Add a barrier to block most rays • Pinhole in barrier • Only sense light from one direction. – Reduces blurring. • In most cameras, this aperture can vary in size. world barrier sensor Slide source: Seitz Focus and Defocus world lens sensor “circle of confusion” or coma A lens focuses light onto the film • There is a specific distance at which objects are “in focus” – other points project to a “circle of confusion” in the image • Changing the shape of the lens changes this distance Slide by Steve Seitz Thin lenses Thin lens equation: 1 1 1 − = 푓 푑표 푑푖 Any object point satisfying this equation is in focus What is the shape of the focus region? How can we change the focus region? Thin lens applet: https://sites.google.com/site/marclevoylectures/applets/operation-of-a-thin-lens (by Andrew Adams, Nora Willett, Marc Levoy) Slide by Steve Seitz Beyond Pinholes: Real apertures Bokeh: [Rushif – Wikipedia] Depth Of Field Depth of Field Depth of Field http://www.cambridgeincolour.com/tutorials/depth-of-field.htm Changing the aperture size affects depth of field Depth of field Wide aperture Depth of field Narrow aperture A narrower aperture increases the range in which the object is approximately in focus. Narrower aperture reduces amount of light – need to increase exposure. Varying the aperture Large aperture = small DOF Small aperture = large DOF Accidental Cameras Accidental Pinhole and Pinspeck Cameras Revealing the scene outside the picture. Antonio Torralba, William T. Freeman Accidental Cameras James Hays DSLR – Digital Single Lens Reflex Camera DSLR – Digital Single Lens Reflex Camera “See what the main lens sees” Your eye The world 1. Objective (main) lens 2. Mirror 3. Shutter 4.