Computational Imaging Advanced Computer Graphics (Spring 2013) §. Digital cameras now commonplace §. Can we use computation for better images CS 283, Lecture 24 §. Many novel capabilities relative to film Computational Imaging and Photography §. And new ways of processing images Ravi Ramamoorthi §. Is this computer graphics, optics, or image proc? http://inst.eecs.berkeley.edu/~cs283/sp13 §. All of the above; many rendering ideas apply §. Application shift. Computer aided design to movies/games to photography (big market) §. Brief lecture. Subject of whole conference ICCP §. Industry: Light Field cameras, Google glass, … Many slides courtesy Ramesh Raskar, SIGGRAPH 2008 course How do we see the world? Outline §. Image formation, basic lens-based camera §. Light Field camera §. Coded aperture depth of field §. Flutter shutter (coded aperture shutter) §. Many many more old, new innovations Let’s design a camera D Idea 1: put a piece of film in front of an object D Do we get a reasonable image? Slide by Steve Seitz Pinhole camera Pinhole camera model Add a barrier to block off most of the rays Pinhole model: D This reduces blurring D Captures pencil of rays – all rays through a single point D The opening known as the aperture D The point is called Center of Projection (COP) D How does this transform the image? D The image is formed on the Image Plane D Effective focal length f is distance from COP to Image Plane Slide by Steve Seitz Slide by Steve Seitz 1 Dimensionality Reduction Machine (3D to 2D) Funny things happen… 3D world 2D image What have we lost? D Angles D Distances (lengths) Figures © Stephen E. Palmer, 2002 Parallel lines aren’t… Lengths can’t be trusted... A’ C’ B ’ Figure by David Forsyth Figure by David Forsyth …but humans adapt! Camera Obscura Camera Obscura, Gemma Frisius, 1558� Müller-Lyer Illusion The first camera ’ We don t make measurements in the image plane D Known to Aristotle http://www.michaelbach.de/ot/sze_muelue/index.html D Depth of the room is the effective focal length 2 From Pinhole to Lenses Home-made pinhole camera Computer graphics assumes pinhole model But making aperture narrow limits light Making aperture large causes blurriness Real cameras have lenses to collect more light, and focus it on the image plane Why so blurry? (Kolb et al. 95 simulates lens effects rendering) http://www.debevec.org/Pinhole/ Shrinking the aperture The reason for lenses Less light gets through Why not make the aperture as small as possible? D Less light gets through D Diffraction effects… Slide by Steve Seitz Slide by Steve Seitz Focus and Defocus Thin lenses “circle of confusion” A lens focuses light onto the film Thin lens equation: D There is a specific distance at which objects are “in focus” – other points project to a “circle of confusion” in the image D Any object point satisfying this equation is in focus D Changing the shape/separation of lens changes this distance D What is the shape of the focus region? D How can we change the focus region? D Thin lens applet: http://www.phy.ntnu.edu.tw/java/Lens/lens_e.html (by Fu-Kwun Hwang ) Slide by Steve Seitz Slide by Steve Seitz 3 Depth of Field Outline §. Image formation, basic lens-based camera §. Light Field camera §. Coded aperture depth of field §. Flutter shutter (coded aperture shutter) §. Many many more old, new innovations http://www.cambridgeincolour.com/tutorials/depth-of-field.htm Light Field Inside a Camera Light Field Inside a Camera Lenslet-based Light Field camera [Adelson and Wang, 1992, Ng et al. 2005 ] Stanford Plenoptic Camera [Ng et al 2005] Digital Refocusing Contax medium format camera Kodak 16-megapixel sensor [Ng et al 2005] Adaptive Optics microlens array 125µ square-sided microlenses 4000 × 4000 pixels ÷ 292 × 292 lenses = 14 × 14 pixels per lens 4 Mask based Light Field Camera Cosine Mask Used Sensor Mask Mask Tile 1/f0 [Veeraraghavan, Raskar, Agrawal, Tumblin, Mohan, Siggraph 2007 ] Captured 2D Photo 2D F F T Traditional Camera Photo Magnitude of 2D FFT Encoding due to Mask 2D F F T Heterodyne Camera Photo Magnitude of 2D FFT [Veeraraghavan, Raskar, Agrawal, Tumblin, Mohan, Siggraph 2007 ] Outline Engineering the PSF when you cannot §. Image formation, basic lens-based camera capture Lightfield §. Light Field camera §. Coded aperture depth of field §. Flutter shutter (coded aperture shutter) §. Many many more old, new innovations Out of Focus Photo: Coded Aperture 5 2D Photo LED In Focus Photo Out of Focus Photo: Open Aperture Captured Blurred Photo Out of Focus Photo: Coded Aperture [Veeraraghavan, Raskar, Agrawal, Tumblin, Mohan, Siggraph 2007 ] Refocused on Person Outline §. Image formation, basic lens-based camera §. Light Field camera §. Coded aperture depth of field §. Flutter shutter (coded aperture shutter) Increase DoF + large aperture §. Many many more old, new innovations 6 Traditional Camera Our Camera Shutter is OPEN Flutter Shutter Lab Setup Shutter is OPEN and CLOSED Sync Function Blurring == Convolution Preserves High Frequencies!!! Traditional Camera: Box Filter Flutter Shutter: Coded Filter 7 Comparison Inverse Filter stable Inverse Filter Unstable Input Image Rectified Crop Deblurred Result 8 .