Computational Imaging Advanced Computer Graphics (Spring 2013) §. Digital cameras now commonplace §. Can we use computation for better images CS 283, Lecture 24 §. Many novel capabilities relative to film Computational Imaging and Photography §. And new ways of processing images Ravi Ramamoorthi §. Is this computer graphics, optics, or image proc? http://inst.eecs.berkeley.edu/~cs283/sp13 §. All of the above; many rendering ideas apply §. Application shift. Computer aided design to movies/games to photography (big market) §. Brief lecture. Subject of whole conference ICCP §. Industry: Light Field cameras, Google glass, … Many slides courtesy Ramesh Raskar, SIGGRAPH 2008 course
How do we see the world? Outline
§. Image formation, basic lens-based camera §. Light Field camera §. Coded aperture depth of field §. Flutter shutter (coded aperture shutter)
§. Many many more old, new innovations Let’s design a camera D Idea 1: put a piece of film in front of an object D Do we get a reasonable image?
Slide by Steve Seitz
Pinhole camera Pinhole camera model
Add a barrier to block off most of the rays Pinhole model: D This reduces blurring D Captures pencil of rays – all rays through a single point D The opening known as the aperture D The point is called Center of Projection (COP) D How does this transform the image? D The image is formed on the Image Plane D Effective focal length f is distance from COP to Image Plane
Slide by Steve Seitz Slide by Steve Seitz
1 Dimensionality Reduction Machine (3D to 2D) Funny things happen…
3D world 2D image
What have we lost? D Angles D Distances (lengths)
Figures © Stephen E. Palmer, 2002
Parallel lines aren’t… Lengths can’t be trusted...
A’ C’
B ’
Figure by David Forsyth Figure by David Forsyth
…but humans adapt! Camera Obscura
Camera Obscura, Gemma Frisius, 1558�
Müller-Lyer Illusion
The first camera ’ We don t make measurements in the image plane D Known to Aristotle http://www.michaelbach.de/ot/sze_muelue/index.html D Depth of the room is the effective focal length
2 From Pinhole to Lenses Home-made pinhole camera
Computer graphics assumes pinhole model
But making aperture narrow limits light
Making aperture large causes blurriness
Real cameras have lenses to collect more light, and focus it on the image plane Why so blurry? (Kolb et al. 95 simulates lens effects rendering)
http://www.debevec.org/Pinhole/
Shrinking the aperture The reason for lenses
Less light gets through
Why not make the aperture as small as possible? D Less light gets through D Diffraction effects…
Slide by Steve Seitz Slide by Steve Seitz
Focus and Defocus Thin lenses
A lens focuses light onto the film Thin lens equation:
D There is a specific distance at which objects are “in focus” – other points project to a “circle of confusion” in the image D Any object point satisfying this equation is in focus D Changing the shape/separation of lens changes this distance D What is the shape of the focus region? D How can we change the focus region? D Thin lens applet: http://www.phy.ntnu.edu.tw/java/Lens/lens_e.html (by Fu-Kwun Hwang ) Slide by Steve Seitz Slide by Steve Seitz
3 Depth of Field Outline
§. Image formation, basic lens-based camera §. Light Field camera §. Coded aperture depth of field §. Flutter shutter (coded aperture shutter)
§. Many many more old, new innovations
http://www.cambridgeincolour.com/tutorials/depth-of-field.htm
Light Field Inside a Camera Light Field Inside a Camera
Lenslet-based Light Field camera
[Adelson and Wang, 1992, Ng et al. 2005 ]
Stanford Plenoptic Camera [Ng et al 2005] Digital Refocusing
Contax medium format camera Kodak 16-megapixel sensor
[Ng et al 2005]
Adaptive Optics microlens array 125µ square-sided microlenses
4000 × 4000 pixels ÷ 292 × 292 lenses = 14 × 14 pixels per lens
4 Mask based Light Field Camera Cosine Mask Used Sensor Mask Mask Tile
1/f0
[Veeraraghavan, Raskar, Agrawal, Tumblin, Mohan, Siggraph 2007 ]
Captured 2D Photo
2D F F T
Traditional Camera Photo Magnitude of 2D FFT
Encoding due to Mask 2D F F T
Heterodyne Camera Photo Magnitude of 2D FFT [Veeraraghavan, Raskar, Agrawal, Tumblin, Mohan, Siggraph 2007 ]
Outline Engineering the PSF when you cannot §. Image formation, basic lens-based camera capture Lightfield §. Light Field camera §. Coded aperture depth of field §. Flutter shutter (coded aperture shutter)
§. Many many more old, new innovations
Out of Focus Photo: Coded Aperture
5 2D Photo
LED
In Focus Photo Out of Focus Photo: Open Aperture
Captured Blurred Photo
Out of Focus Photo: Coded Aperture [Veeraraghavan, Raskar, Agrawal, Tumblin, Mohan, Siggraph 2007 ]
Refocused on Person Outline
§. Image formation, basic lens-based camera §. Light Field camera §. Coded aperture depth of field §. Flutter shutter (coded aperture shutter)
Increase DoF + large aperture §. Many many more old, new innovations
6 Traditional Camera Our Camera
Shutter is OPEN Flutter Shutter
Lab Setup Shutter is OPEN and CLOSED
Sync Function Blurring == Convolution Preserves High Frequencies!!!
Traditional Camera: Box Filter Flutter Shutter: Coded Filter
7 Comparison
Inverse Filter stable
Inverse Filter Unstable
Input Image
Rectified Crop
Deblurred Result
8