3D Photography
Total Page:16
File Type:pdf, Size:1020Kb
Load more
Recommended publications
-
Bibliography.Pdf
Bibliography Thomas Annen, Jan Kautz, Frédo Durand, and Hans-Peter Seidel. Spherical harmonic gradients for mid-range illumination. In Alexander Keller and Henrik Wann Jensen, editors, Eurographics Symposium on Rendering, pages 331–336, Norrköping, Sweden, 2004. Eurographics Association. ISBN 3-905673-12-6. 35,72 Arthur Appel. Some techniques for shading machine renderings of solids. In AFIPS 1968 Spring Joint Computer Conference, volume 32, pages 37–45, 1968. 20 Okan Arikan, David A. Forsyth, and James F. O’Brien. Fast and detailed approximate global illumination by irradiance decomposition. In ACM Transactions on Graphics (Proceedings of SIGGRAPH 2005), pages 1108–1114. ACM Press, July 2005. doi: 10.1145/1073204.1073319. URL http://doi.acm.org/10.1145/1073204.1073319. 47, 48, 52 James Arvo. The irradiance jacobian for partially occluded polyhedral sources. In Computer Graphics (Proceedings of SIGGRAPH 94), pages 343–350. ACM Press, 1994. ISBN 0-89791-667-0. doi: 10.1145/192161.192250. URL http://dx.doi.org/10.1145/192161.192250. 72, 102 Neeta Bhate. Application of rapid hierarchical radiosity to participating media. In Proceedings of ATARV-93: Advanced Techniques in Animation, Rendering, and Visualization, pages 43–53, Ankara, Turkey, July 1993. Bilkent University. 69 Neeta Bhate and A. Tokuta. Photorealistic volume rendering of media with directional scattering. In Alan Chalmers, Derek Paddon, and François Sillion, editors, Rendering Techniques ’92, Eurographics, pages 227–246. Consolidation Express Bristol, 1992. 69 Miguel A. Blanco, M. Florez, and M. Bermejo. Evaluation of the rotation matrices in the basis of real spherical harmonics. Journal Molecular Structure (Theochem), 419:19–27, December 1997. -
CEO & Co-Founder, Pinscreen, Inc. Distinguished Fellow, University Of
HaoLi CEO & Co-Founder, Pinscreen, Inc. Distinguished Fellow, University of California, Berkeley Pinscreen, Inc. Email [email protected] 11766 Wilshire Blvd, Suite 1150 Home page http://www.hao-li.com/ Los Angeles, CA 90025, USA Facebook http://www.facebook.com/li.hao/ PROFILE Date of birth 17/01/1981 Place of birth Saarbrücken, Germany Citizenship German Languages German, French, English, and Mandarin Chinese (all fluent and no accents) BIO I am CEO and Co-Founder of Pinscreen, an LA-based Startup that builds the world’s most advanced AI-driven virtual avatars, as well as a Distinguished Fellow of the Computer Vision Group at the University of California, Berkeley. Before that, I was an Associate Professor (with tenure) in Computer Science at the University of Southern California, as well as the director of the Vision & Graphics Lab at the USC Institute for Creative Technologies. I work at the intersection between Computer Graphics, Computer Vision, and AI, with focus on photorealistic human digitization using deep learning and data-driven techniques. I’m known for my work on dynamic geometry processing, virtual avatar creation, facial performance capture, AI-driven 3D shape digitization, and deep fakes. My research has led to the Animoji technology in Apple’s iPhone X, I worked on the digital reenactment of Paul Walker in the movie Furious 7, and my algorithms on deformable shape alignment have improved the radiation treatment for cancer patients all over the world. I have been keynote speaker at numerous major events, conferences, festivals, and universities, including the World Economic Forum (WEF) in Davos 2020 and Web Summit 2020. -
Efficient Ray Tracing of Volume Data
Efficient Ray Tracing of Volume Data TR88-029 June 1988 Marc Levay The University of North Carolina at Chapel Hill Department of Computer Science CB#3175, Sitterson Hall Chapel Hill, NC 27599-3175 UNC is an Equal Opportunity/ Affirmative Action Institution. 1 Efficient Ray Tracing of Volume Data Marc Levoy June, 1988 Computer Science Department University of North Carolina Chapel Hill, NC 27599 Abstract Volume rendering is a technique for visualizing sampled scalar functions of three spatial dimen sions without fitting geometric primitives to the data. Images are generated by computing 2-D pro jections of a colored semi-transparent volume, where the color and opacity at each point is derived from the data using local operators. Since all voxels participate in the generation of each image, rendering time grows linearly with the size of the dataset. This paper presents a front-to-back image-order volume rendering algorithm and discusses two methods for improving its performance. The first method employs a pyramid of binary volumes to encode coherence present in the data, and the second method uses an opacity threshold to adaptively terminate ray tracing. These methods exhibit a lower complexity than brute-force algorithms, although the actual time saved depends on the data. Examples from two applications are given: medical imaging and molecular graphics. 1. Introduction The increasing availability of powerful graphics workstations in the scientific and computing communities has catalyzed the development of new methods for visualizing discrete multidimen sional data. In this paper, we address the problem of visualizing sampled scalar functions of three spatial dimensions, henceforth referred to as volume data. -
Volume Rendering on Scalable Shared-Memory MIMD Architectures
Volume Rendering on Scalable Shared-Memory MIMD Architectures Jason Nieh and Marc Levoy Computer Systems Laboratory Stanford University Abstract time. Examples of these optimizations are hierarchical opacity enumeration, early ray termination, and spatially adaptive image Volume rendering is a useful visualization technique for under- sampling [13, 14, 16, 181. Optimized ray tracing is the fastest standing the large amounts of data generated in a variety of scien- known sequential algorithm for volume rendering. Nevertheless, tific disciplines. Routine use of this technique is currently limited the computational requirements of the fastest sequential algorithm by its computational expense. We have designed a parallel volume still preclude real-time volume rendering on the fastest computers rendering algorithm for MIMD architectures based on ray tracing today. and a novel task queue image partitioning technique. The combi- Parallel machines offer the computational power to achieve real- nation of ray tracing and MIMD architectures allows us to employ time volume rendering on usefully large datasets. This paper con- algorithmic optimizations such as hierarchical opacity enumera- siders how parallel computing can be brought to bear in reducing tion, early ray termination, and adaptive image sampling. The use the computation time of volume rendering. We have designed of task queue image partitioning makes these optimizations effi- and implemented an optimized volume ray tracing algorithm that cient in aparallel framework. We have implemented our algorithm employs a novel task queue image partitioning technique on the on the Stanford DASH Multiprocessor, a scalable shared-memory Stanford DASH Multiprocessor [lo], a scalable shared-memory MIMD machine. Its single address-space and coherent caches pro- MIMD (Multiple Instruction, Multiple Data) machine consisting vide programming ease and good performance for our algorithm. -
Computational Photography
Computational Photography George Legrady © 2020 Experimental Visualization Lab Media Arts & Technology University of California, Santa Barbara Definition of Computational Photography • An R & D development in the mid 2000s • Computational photography refers to digital image-capture cameras where computers are integrated into the image-capture process within the camera • Examples of computational photography results include in-camera computation of digital panoramas,[6] high-dynamic-range images, light- field cameras, and other unconventional optics • Correlating one’s image with others through the internet (Photosynth) • Sensing in other electromagnetic spectrum • 3 Leading labs: Marc Levoy, Stanford (LightField, FrankenCamera, etc.) Ramesh Raskar, Camera Culture, MIT Media Lab Shree Nayar, CV Lab, Columbia University https://en.wikipedia.org/wiki/Computational_photography Definition of Computational Photography Sensor System Opticcs Software Processing Image From Shree K. Nayar, Columbia University From Shree K. Nayar, Columbia University Shree K. Nayar, Director (lab description video) • Computer Vision Laboratory • Lab focuses on vision sensors; physics based models for vision; algorithms for scene interpretation • Digital imaging, machine vision, robotics, human- machine interfaces, computer graphics and displays [URL] From Shree K. Nayar, Columbia University • Refraction (lens / dioptrics) and reflection (mirror / catoptrics) are combined in an optical system, (Used in search lights, headlamps, optical telescopes, microscopes, and telephoto lenses.) • Catadioptric Camera for 360 degree Imaging • Reflector | Recorded Image | Unwrapped [dance video] • 360 degree cameras [Various projects] Marc Levoy, Computer Science Lab, Stanford George Legrady Director, Experimental Visualization Lab Media Arts & Technology University of California, Santa Barbara http://vislab.ucsb.edu http://georgelegrady.com Marc Levoy, Computer Science Lab, Stanford • Light fields were introduced into computer graphics in 1996 by Marc Levoy and Pat Hanrahan. -
Computational Photography Research at Stanford
Computational photography research at Stanford January 2012 Marc Levoy Computer Science Department Stanford University Outline ! Light field photography ! FCam and the Stanford Frankencamera ! User interfaces for computational photography ! Ongoing projects • FrankenSLR • burst-mode photography • languages and architectures for programmable cameras • computational videography and cinematography • multi-bucket pixels 2 ! Marc Levoy Light field photography using a handheld plenoptic camera Ren Ng, Marc Levoy, Mathieu Brédif, Gene Duval, Mark Horowitz and Pat Hanrahan (Proc. SIGGRAPH 2005 and TR 2005-02) Light field photography [Ng SIGGRAPH 2005] ! Marc Levoy Prototype camera Contax medium format camera Kodak 16-megapixel sensor Adaptive Optics microlens array 125µ square-sided microlenses 4000 ! 4000 pixels ÷ 292 ! 292 lenses = 14 ! 14 pixels per lens Digital refocusing ! ! ! 2010 Marc Levoy Example of digital refocusing ! 2010 Marc Levoy Refocusing portraits ! 2010 Marc Levoy Light field photography (Flash demo) ! 2010 Marc Levoy Commercialization • trades off (excess) spatial resolution for ability to refocus and adjust the perspective • sensor pixels should be made even smaller, subject to the diffraction limit, for example: 36mm ! 24mm ÷ 2.5µ pixels = 266 Mpix 20K ! 13K pixels 4000 ! 2666 pixels ! 20 ! 20 rays per pixel or 2000 ! 1500 pixels ! 3 ! 3 rays per pixel = 27 Mpix ! Marc Levoy Camera 2.0 Marc Levoy Computer Science Department Stanford University The Stanford Frankencameras Frankencamera F2 Nokia N900 “F” ! facilitate research in experimental computational photography ! for students in computational photography courses worldwide ! proving ground for plugins and apps for future cameras 14 ! 2010 Marc Levoy CS 448A - Computational Photography 15 ! 2010 Marc Levoy CS 448 projects M. Culbreth, A. Davis!! ! ! A phone-to-phone 4D light field fax machine J. -
Capturing and Simulating Physically Accurate Illumination in Computer Graphics
Capturing and Simulating Physically Accurate Illumination in Computer Graphics PAUL DEBEVEC Institute for Creative Technologies University of Southern California Marina del Rey, California Anyone who has seen a recent summer blockbuster has witnessed the dramatic increases in computer-generated realism in recent years. Visual effects supervisors now report that bringing even the most challenging visions of film directors to the screen is no longer a question of whats possible; with todays techniques it is only a matter of time and cost. Driving this increase in realism have been computer graphics (CG) techniques for simulating how light travels within a scene and for simulating how light reflects off of and through surfaces. These techniquessome developed recently, and some originating in the 1980sare being applied to the visual effects process by computer graphics artists who have found ways to channel the power of these new tools. RADIOSITY AND GLOBAL ILLUMINATION One of the most important aspects of computer graphics is simulating the illumination within a scene. Computer-generated images are two-dimensional arrays of computed pixel values, with each pixel coordinate having three numbers indicating the amount of red, green, and blue light coming from the corresponding direction within the scene. Figuring out what these numbers should be for a scene is not trivial, since each pixels color is based both on how the object at that pixel reflects light and also the light that illuminates it. Furthermore, the illumination does not only come directly from the lights within the scene, but also indirectly Capturing and Simulating Physically Accurate Illumination in Computer Graphics from all of the surrounding surfaces in the form of bounce light. -
Image-Based Lighting
Tutorial Image-Based Paul Debevec Lighting USC Institute for Creative Technologies mage-based lighting (IBL) is the process of 2. mapping the illumination onto a representation of Iilluminating scenes and objects (real or syn- the environment; thetic) with images of light from the real world. It 3. placing the 3D object inside the environment; and evolved from the reflection-mapping technique1,2 in 4. simulating the light from the environment illumi- which we use panoramic images as nating the computer graphics object. texture maps on computer graphics This tutorial shows how models to show shiny objects reflect- Figure 1 shows an example of an object illuminated ing real and synthetic environments. entirely using IBL. Gary Butcher created the models in image-based lighting can IBL is analogous to image-based 3D Studio Max, and the renderer used was the Arnold modeling, in which we derive a 3D global illumination system written by Marcos Fajardo. illuminate synthetic objects scene’s geometric structure from I captured the light in a kitchen so it includes light from images, and to image-based render- a ceiling fixture; the blue sky from the windows; and with measurements of real ing, in which we produce the ren- the indirect light from the room’s walls, ceiling, and dered appearance of a scene from its cabinets. Gary mapped the light from this room onto a light, making objects appear appearance in images. When used large sphere and placed the model of the microscope effectively, IBL can produce realistic on the table in the middle of the sphere. -
Computational Photography & Cinematography
Computational photography & cinematography (for a lecture specifically devoted to the Stanford Frankencamera, see http://graphics.stanford.edu/talks/camera20-public-may10-150dpi.pdf) Marc Levoy Computer Science Department Stanford University The future of digital photography ! the megapixel wars are over (and it’s about time) ! computational photography is the next battleground in the camera industry (it’s already starting) 2 ! Marc Levoy Premise: available computing power in cameras is rising faster than megapixels 12 70 9 52.5 6 35 megapixels 3 17.5 billions of pixels / second of pixels billions 0 0 2005 2006 2007 2008 2009 2010 Avg megapixels in new cameras, CAGR = 1.2 NVIDIA GTX texture fill rate, CAGR = 1.8 (CAGR for Moore’s law = 1.5) ! this “headroom” permits more computation per pixel, 3 or more frames per second, or less custom hardware ! Marc Levoy The future of digital photography ! the megapixel wars are over (long overdue) ! computational photography is the next battleground in the camera industry (it’s already starting) ! how will these features appear to consumers? • standard and invisible • standard and visible (and disable-able) • aftermarket plugins and apps for your camera 4 ! Marc Levoy The future of digital photography ! the megapixel wars are over (long overdue) ! computational photography is the next battleground in the camera industry (it’s already starting) ! how will these features appear to consumers? • standard and invisible • standard and visible (and disable-able) • aftermarket plugins and apps for your camera -
Uva-DARE (Digital Academic Repository)
UvA-DARE (Digital Academic Repository) Interactive Exploration in Virtual Environments Belleman, R.G. Publication date 2003 Link to publication Citation for published version (APA): Belleman, R. G. (2003). Interactive Exploration in Virtual Environments. General rights It is not permitted to download or to forward/distribute the text or part of it without the consent of the author(s) and/or copyright holder(s), other than for strictly personal, individual use, unless the work is under an open content license (like Creative Commons). Disclaimer/Complaints regulations If you believe that digital publication of certain material infringes any of your rights or (privacy) interests, please let the Library know, stating your reasons. In case of a legitimate complaint, the Library will make the material inaccessible and/or remove it from the website. Please Ask the Library: https://uba.uva.nl/en/contact, or a letter to: Library of the University of Amsterdam, Secretariat, Singel 425, 1012 WP Amsterdam, The Netherlands. You will be contacted as soon as possible. UvA-DARE is a service provided by the library of the University of Amsterdam (https://dare.uva.nl) Download date:30 Sep 2021 References s [1]] 3rdTech webpage, http://www.3rdtech.com/. [2]] M.J. Ackerman. Accessing the visible human project. D-Lib Magazine: The MagazineMagazine of the Digital Library Forum, October 1995. [3]] H. Afsarmanesh, R.G. Belleman, A.S.Z. Belloum, A. Benabdelkader, J.F.J, vann den Brand, G.B. Eijkel, A. Frenkel, C. Garita, D.L. Groep, R.M.A. Heeren, Z.W.. Hendrikse, L.0. Hertzberger, J.A. Kaandorp, E.C. -
SIGGRAPH 2008 Course: Computational Photography: Advanced Topics
SIGGRAPH 2008 Course: Computational Photography: Advanced Topics http://computationalphotography.org Speakers Paul Debevec (USC, USA) (debevec (at) ict.usc.edu) Ramesh RASKAR (MIT Media Lab, USA) (raskar (at) media.mit.edu) Jack TUMBLIN (Northwestern University, USA) (jet (at) cs.northwestern.edu) Course Abstract Computational photography combines plentiful computing, digital sensors, modern optics, many varieties of actuators, probes and smart lights to escape the limitations of traditional film cameras and enables novel imaging applications. Unbounded dynamic range, variable focus, resolution, and depth of field, hints about shape, reflectance, and lighting, and new interactive forms of photos that are partly snapshots and partly videos, performance capture and interchangeably relighting real and virtual characters are just some of the new applications emerging in Computational Photography. The computational techniques encompass methods from modification of imaging parameters during capture to sophisticated reconstructions from indirect measurements. We will bypass basic and introductory material presented in earlier versions of this course (Computational Photography 2005,6,7) and expand coverage of more recent topics. Emphasizing more recent work in computational photography and related fields (2006 or later) this course will give more attention to advanced topics only briefly touched before, including tomography, heterodyning and Fourier Slice applications, inverse problems, gradient illumination, novel optics, emerging sensors -
Multi-View Image Fusion
Multi-view Image Fusion Marc Comino Trinidad1 Ricardo Martin Brualla2 Florian Kainz2 Janne Kontkanen2 1Polytechnic University of Catalonia, 2Google Inputs Outputs Color Transfer Mono Color High-def mono Color High-def color HDR Fusion Color EV+2 Color EV-1 Color EV+2 Color EV-1 Denoised Detail Transfer DSLR Stereo DSLR Low-def stereo High-def stereo Figure 1: We present a method for multi-view image fusion that is a applicable to a variety of scenarios: a higher resolution monochrome image is colorized with a second color image (top row), two color images with different exposures are fused into an HDR lower-noise image (middle row), and a high quality DSLR image is warped to the lower quality stereo views captured by a VR-camera (bottom row). Abstract 1. Introduction In this paper we focus on the problem of fusing multiple We present an end-to-end learned system for fusing mul- misaligned photographs into a chosen target view. Multi- tiple misaligned photographs of the same scene into a cho- view image fusion has become increasingly relevant with sen target view. We demonstrate three use cases: 1) color the recent influx of multi-camera mobile devices. transfer for inferring color for a monochrome view, 2) HDR The form factor of these devices constrains the size of fusion for merging misaligned bracketed exposures, and 3) lenses and sensors, and this limits their light capturing abil- detail transfer for reprojecting a high definition image to ity. Cameras with larger lens apertures and larger pixels the point of view of an affordable VR180-camera.