DOCSLIB.ORG

NHI-NSF Visualization Research Challenges Report

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
NHI-NSF Visualization Research Challenges Report MITSUBISHI ELECTRIC RESEARCH LABORATORIES http://www.merl.com NHI-NSF Visualization Research Challenges Report Chris Johnson, Robert Moorhead, Tamera Munzner, Hanspeter Pfister, Penny Rheingans, Terry S. Yoo TR2006-118 July 2006 Abstract In this report we describe some of the remarkable achievements visualization enables and discuss the major obstacles blocking the disciplines´ advancement. Our findings and recommendations reflect not only information gathered from visualization and applications scientists during two workshops on Visualization Research Challenges but also input from the larger visualization community. Computing in Science and Engineering This work may not be copied or reproduced in whole or in part for any commercial purpose. Permission to copy in whole or in part without payment of fee is granted for nonprofit educational and research purposes provided that all such whole or partial copies include the following: a notice that such copying is by permission of Mitsubishi Electric Research Laboratories, Inc.; an acknowledgment of the authors and individual contributions to the work; and all applicable portions of the copyright notice. Copying, reproduction, or republishing for any other purpose shall require a license with payment of fee to Mitsubishi Electric Research Laboratories, Inc. All rights reserved. Copyright c Mitsubishi Electric Research Laboratories, Inc., 2006 201 Broadway, Cambridge, Massachusetts 02139 MERLCoverPageSide2 NIH-NSF Visualization Research Challenges Report Chris Johnson1, Robert Moorhead2, Tamara Munzner3, Hanspeter Pfister4, Penny Rheingans5, and Terry S. Yoo6 1 Scientific Computing and Imaging Institute School of Computing University of Utah Salt Lake City, UT 84112 [email protected] 2 Department of Electrical and Computer Engineering Mississippi State University Mississippi State, MS 39762 [email protected] 3 Department of Computer Science University of British Columbia 2366 Main Mall Vancouver, BC V6T 1Z4 Canada [email protected] 4 MERL - Mitsubishi Electric Research Laboratories 201 Broadway Cambridge, MA 02139 [email protected] 5 Department of Computer Science and Electrical Engineering University of Maryland Baltimore County Baltimore, MD 21250 [email protected] 6 Office of High Performance Computing and Communications National Library of Medicine, U.S. National Institutes of Health Bethesda, MD 20894 [email protected] October 20, 2005 Contents 1 Executive Summary . 3 2 The Value of Visualization . 5 3 The Process of Visualization . 7 3.1 Moving Beyond Moore’s Law. 7 3.2 Determining Success . 9 3.3 Supporting Repositories and Open Standards . 10 3.4 Achieving Our Goals . 10 4 The Power of Visualization . 13 4.1 Transforming Health Care . 13 4.2 Transforming Science and Engineering . 16 4.3 Transforming Life . 19 5 Roadmap . 23 6 State of the Field . 25 6.1 Other Reports . 25 6.2 National Infrastructure . 25 6.2.1 Visualization Hardware . 26 6.2.2 Networking . 27 6.2.3 Visualization Software . 27 7 Workshop Participants . 31 Acknowledgments . 32 2 EXECUTIVE SUMMARY 1 In this report we describe some of the remarkable security, and competitiveness, support for research and achievements visualization enables and discuss the major development in this critical, multidisciplinary field has been obstacles blocking the discipline’s advancement. Our findings inadequate. Unless we recommit ourselves to substantial and recommendations reflect not only information gathered support for visualization research and development, we will from visualization and applications scientists during two see a decline in the progress of discovery in other important workshops on Visualization Research Challenges but also input disciplines dependent on visualization. As these disciplines from the larger visualization community. lose their ability to harness and make sense of information, the rate of discovery itself will decline. In the inevitable chain Advances in computing science and technology have resulted reaction, we will lose our competitive edge in business and in unprecedented improvements in medicine, engineering, industry and experience a weakening in our ability to address science, and economics. The great benefits of these advances numerous social ills. are only starting to be realized. In order to promote and perpetuate similar advances in the future, we need to increase Principal Finding: Visualization is indispensable to the our ability to understand large amounts of data and information solution of complex problems in every sector, from traditional arising from a multitude of sources. One of the greatest science and engineering domains to such key areas as financial scientific challenges of the 21st century is to effectively markets, national security, and public health. Advances in understand and make use of the vast amount of information visualization enable researchers to analyze and understand being produced. We must create new theories, techniques, unprecedented amounts of experimental, simulated, and and methods to enable discoveries in science, engineering, observational data, and, through this understanding, to address medicine, the arts, and the humanities. problems previously deemed intractable or beyond imagination. Yet, despite the great opportunities created by and needs As we work to tame the accelerating information explosion fulfilled by visualization, NSF and NIH (and other Federal and employ it to advance scientific, biomedical, and engineering government agencies) have not effectively recognized the research, defense and national security, and industrial strategic significance and importance of visualization in either innovation, visualization will be among our most important their organizational structures or their research and educational tools. Visualization is fundamental to understanding models planning. These inadequacies compromise the future of U.S. of complex phenomena, such as multi-level models of human scientific leadership and economic competitiveness. physiology from DNA to whole organs, multi-century climate shifts, international financial markets, or multidimensional Principal Recommendations: NSF and NIH must make simulations of airflow past a jet wing. Visualization reduces coordinated investments in visualization to address the 21st and refines data streams rapidly and economically, enabling century’s most important problems, which are predominantly us to winnow huge volumes of data in applications such as collaborative, crossing disciplines, agencies, and sectors. Both the surveillance of public health at regional and national levels NSF and NIH can and should provide leadership to other in order to track the spread of infectious diseases. Federal funding partners, as well as to the research Visualizations of such application problems as hurricane communities they support, by modifying their programmatic dynamics and biomedical imaging are generating new practices to better engage visualization capabilities across knowledge that crosses traditional disciplinary boundaries. And disciplines important to scientific and social progress and to visualization provides industry with a competitive edge by encourage and reward interdisciplinary research, open transforming business and engineering practices. practices, and reproducibility in technical and scientific developments. Such agency leadership is critical if we are to While visualization is itself a discipline, advances in meet the needs of our nation in critical areas and to maintain visualization lead inevitably to advances in other disciplines. U.S. competitiveness in a global environment. Agency However, despite the importance of visualization to discovery, leadership should include: 3 • Realigning Visualization Policy: We recommend that community and to demonstrate the reproducibility of their review panels for proposals, papers, and promotion technical and scientific developments. acknowledge and accommodate the interdisciplinary nature and enabling capacity of visualization. In both basic and • National Research Investment: In order for the U.S. to application research, we can advance the use and impact of remain competitive in a global research and development visualization with relatively limited new investment by changing community with increasing resources, we must establish a support and review policies. A small shift in the funding support national infrastructure of data repositories, validation centers, for the basic sciences will encourage researchers to include technology development programs, and research initiatives more visualization capabilities in their programs and will lead for interaction, abstraction, modeling, and portrayal of complex to an increased emphasis on collaborative research of information. We must also make a coordinated national visualization researchers with domain specialists and, as a investment, directed toward a spectrum of centralized and result, will help to match research needs with already distributed programs, to use visualization to support, accelerate, developing resources. New sponsored research in all fields and enhance progress in science, engineering, medicine, and should actively incorporate modern visual presentation and other fields that promote social welfare and economic address the users’ need to interact with models, data, and advancement. And, to meet the future needs of our nation in abstractions. In order to ensure this goal, review protocols critical areas affecting science and society, we will need new for research papers and applications
Load more

Recommended publications
  • Ways of Seeing Data: a Survey of Fields of Visualization
    • Ways of seeing data: • A survey of fields of visualization • Gordon Kindlmann • [email protected] Part of the talk series “Show and Tell: Visualizing the Life of the Mind” • Nov 19, 2012 http://rcc.uchicago.edu/news/show_and_tell_abstracts.html 2012 Presidential Election REPUBLICAN DEMOCRAT http://www.npr.org/blogs/itsallpolitics/2012/11/01/163632378/a-campaign-map-morphed-by-money 2012 Presidential Election http://gizmodo.com/5960290/this-is-the-real-political-map-of-america-hint-we-are-not-that-divided 2012 Presidential Election http://www.npr.org/blogs/itsallpolitics/2012/11/01/163632378/a-campaign-map-morphed-by-money 2012 Presidential Election http://www.npr.org/blogs/itsallpolitics/2012/11/01/163632378/a-campaign-map-morphed-by-money 2012 Presidential Election http://www.npr.org/blogs/itsallpolitics/2012/11/01/163632378/a-campaign-map-morphed-by-money Clarifying distortions Tube map from 1908 http://www.20thcenturylondon.org.uk/beck-henry-harry http://briankerr.wordpress.com/2009/06/08/connections/ http://en.wikipedia.org/wiki/Harry_Beck Clarifying distortions http://www.20thcenturylondon.org.uk/beck-henry-harry Harry Beck 1933 http://briankerr.wordpress.com/2009/06/08/connections/ http://en.wikipedia.org/wiki/Harry_Beck Clarifying distortions Clarifying distortions Joachim Böttger, Ulrik Brandes, Oliver Deussen, Hendrik Ziezold, “Map Warping for the Annotation of Metro Maps” IEEE Computer Graphics and Applications, 28(5):56-65, 2008 Maps reflect conventions, choices, and priorities “A single map is but one of an indefinitely large
    [Show full text]
  • Domain-Specific Programming Systems
    Lecture 22: Domain-Specific Programming Systems Parallel Computer Architecture and Programming CMU 15-418/15-618, Spring 2020 Slide acknowledgments: Pat Hanrahan, Zach Devito (Stanford University) Jonathan Ragan-Kelley (MIT, Berkeley) Course themes: Designing computer systems that scale (running faster given more resources) Designing computer systems that are efficient (running faster under constraints on resources) Techniques discussed: Exploiting parallelism in applications Exploiting locality in applications Leveraging hardware specialization (earlier lecture) CMU 15-418/618, Spring 2020 Claim: most software uses modern hardware resources inefficiently ▪ Consider a piece of sequential C code - Call the performance of this code our “baseline performance” ▪ Well-written sequential C code: ~ 5-10x faster ▪ Assembly language program: maybe another small constant factor faster ▪ Java, Python, PHP, etc. ?? Credit: Pat Hanrahan CMU 15-418/618, Spring 2020 Code performance: relative to C (single core) GCC -O3 (no manual vector optimizations) 51 40/57/53 47 44/114x 40 = NBody 35 = Mandlebrot = Tree Alloc/Delloc 30 = Power method (compute eigenvalue) 25 20 15 10 5 Slowdown (Compared to C++) Slowdown (Compared no data no 0 data no Java Scala C# Haskell Go Javascript Lua PHP Python 3 Ruby (Mono) (V8) (JRuby) Data from: The Computer Language Benchmarks Game: CMU 15-418/618, http://shootout.alioth.debian.org Spring 2020 Even good C code is inefficient Recall Assignment 1’s Mandelbrot program Consider execution on a high-end laptop: quad-core, Intel Core i7, AVX instructions... Single core, with AVX vector instructions: 5.8x speedup over C implementation Multi-core + hyper-threading + AVX instructions: 21.7x speedup Conclusion: basic C implementation compiled with -O3 leaves a lot of performance on the table CMU 15-418/618, Spring 2020 Making efficient use of modern machines is challenging (proof by assignments 2, 3, and 4) In our assignments, you only programmed homogeneous parallel computers.
    [Show full text]
  • Point-Based Computer Graphics
    Point-Based Computer Graphics Eurographics 2002 Tutorial T6 Organizers Markus Gross ETH Zürich Hanspeter Pfister MERL, Cambridge Presenters Marc Alexa TU Darmstadt Markus Gross ETH Zürich Mark Pauly ETH Zürich Hanspeter Pfister MERL, Cambridge Marc Stamminger Bauhaus-Universität Weimar Matthias Zwicker ETH Zürich Contents Tutorial Schedule................................................................................................2 Presenters Biographies........................................................................................3 Presenters Contact Information ..........................................................................4 References...........................................................................................................5 Project Pages.......................................................................................................6 Tutorial Schedule 8:30-8:45 Introduction (M. Gross) 8:45-9:45 Point Rendering (M. Zwicker) 9:45-10:00 Acquisition of Point-Sampled Geometry and Appearance I (H. Pfister) 10:00-10:30 Coffee Break 10:30-11:15 Acquisition of Point-Sampled Geometry and Appearance II (H. Pfister) 11:15-12:00 Dynamic Point Sampling (M. Stamminger) 12:00-14:00 Lunch 14:00-15:00 Point-Based Surface Representations (M. Alexa) 15:00-15:30 Spectral Processing of Point-Sampled Geometry (M. Gross) 15:30-16:00 Coffee Break 16:00-16:30 Efficient Simplification of Point-Sampled Geometry (M. Pauly) 16:30-17:15 Pointshop3D: An Interactive System for Point-Based Surface Editing (M. Pauly) 17:15-17:30 Discussion (all) 2 Presenters Biographies Dr. Markus Gross is a professor of computer science and the director of the computer graphics laboratory of the Swiss Federal Institute of Technology (ETH) in Zürich. He received a degree in electrical and computer engineering and a Ph.D. on computer graphics and image analysis, both from the University of Saarbrucken, Germany. From 1990 to 1994 Dr. Gross was with the Computer Graphics Center in Darmstadt, where he established and directed the Visual Computing Group.
    [Show full text]
  • Mike Roberts Curriculum Vitae
    Mike Roberts +1 (650) 924–7168 [email protected] mikeroberts3000.github.io Research Using photorealistic synthetic data for computer vision; motion planning, trajectory optimization, Interests and control methods for robotics; reconstructing 3D scenes from images; continuous and discrete optimization; submodular optimization; software tools and algorithms for creativity support Education Stanford University Stanford, California Ph.D. Computer Science 2012–2019 Advisor: Pat Hanrahan Dissertation: Trajectory Optimization Methods for Drone Cameras Harvard University Cambridge, Massachusetts Visiting Research Fellow Summer 2013 Advisor: Hanspeter Pfister University of Calgary Calgary, Canada M.S. Computer Science 2010 University of Calgary Calgary, Canada B.S. Computer Science 2007 Employment Intel Labs Seattle, Washington Research Scientist 2021– Mentor: Vladlen Koltun Apple Seattle, Washington Research Scientist 2018–2021 Microsoft Research Redmond, Washington Research Intern Summer 2016, 2017 Mentors: Neel Joshi, Sudipta Sinha Skydio Redwood City, California Research Intern Spring 2016 Mentors: Adam Bry, Frank Dellaert Udacity Mountain View, California Course Developer, Introduction to Parallel Computing 2012–2013 Instructors: John Owens, David Luebke Harvard University Cambridge, Massachusetts Research Fellow 2010–2012 Advisor: Hanspeter Pfister NVIDIA Austin, Texas Developer Tools Programmer Intern Summer 2009 Radical Entertainment Vancouver, Canada Graphics Programmer Intern 2005–2006 Honors And Featured in the Highlights of
    [Show full text]
  • Mike Roberts Curriculum Vitae
    Mike Roberts Union Street, Suite , Seattle, Washington, + () / [email protected] http://mikeroberts.github.io Research Using photorealistic synthetic data for computer vision; motion planning, trajectory optimization, Interests and control methods for robotics; reconstructing D scenes from images; continuous and discrete optimization; submodular optimization; software tools and algorithms for creativity support. Education Stanford University Stanford, California Ph.D. Computer Science – Advisor: Pat Hanrahan Dissertation: Trajectory Optimization Methods for Drone Cameras Harvard University Cambridge, Massachusetts Visiting Research Fellow Summer John A. Paulson School of Engineering and Applied Sciences Advisor: Hanspeter Pfister University of Calgary Calgary, Canada M.S. Computer Science University of Calgary Calgary, Canada B.S. Computer Science Employment Apple Seattle, Washington Research Scientist – Microsoft Research Redmond, Washington Research Intern Summer , Advisors: Neel Joshi, Sudipta Sinha Skydio Redwood City, California Research Intern Spring Mentors: Adam Bry, Frank Dellaert Udacity Mountain View, California Course Developer, Introduction to Parallel Computing – Instructors: John Owens, David Luebke Harvard University Cambridge, Massachusetts Research Fellow, John A. Paulson School of Engineering and Applied Sciences – Advisor: Hanspeter Pfister NVIDIA Austin, Texas Developer Tools Programmer Intern Summer Radical Entertainment Vancouver, Canada Graphics Programmer Intern – Honors And Featured in the Highlights
    [Show full text]
  • Monocular Neural Image Based Rendering with Continuous View Control
    Monocular Neural Image Based Rendering with Continuous View Control Xu Chen∗ Jie Song∗ Otmar Hilliges AIT Lab, ETH Zurich {xuchen, jsong, otmar.hilliges}@inf.ethz.ch ... ... Figure 1: Interactive novel view synthesis: Given a single source view our approach can generate a continuous sequence of geometrically accurate novel views under fine-grained control. Top: Given a single street-view like input, a user may specify a continuous camera trajectory and our system generates the corresponding views in real-time. Bottom: An unseen hi-res internet image is used to synthesize novel views, while the camera is controlled interactively. Please refer to our project homepagey. Abstract like to view products interactively in 3D rather than from We propose a method to produce a continuous stream discrete view angles. Likewise in map applications it is of novel views under fine-grained (e.g., 1◦ step-size) cam- desirable to explore the vicinity of street-view like images era control at interactive rates. A novel learning pipeline beyond the position at which the photograph was taken. This determines the output pixels directly from the source color. is often not possible because either only 2D imagery exists, Injecting geometric transformations, including perspective or because storing and rendering of full 3D information does projection, 3D rotation and translation into the network not scale. To overcome this limitation we study the problem forces implicit reasoning about the underlying geometry. of interactive view synthesis with 6-DoF view control, taking The latent 3D geometry representation is compact and mean- only a single image as input. We propose a method that ingful under 3D transformation, being able to produce geo- can produce a continuous stream of novel views under fine- grained (e.g., 1◦ step-size) camera control (see Fig.1).
    [Show full text]
  • Thouis R. Jones [email protected]
    Thouis R. Jones [email protected] Summary Computational biologist with 20+ years' experience, including software engineering, algorithm development, image analysis, and solving challenging biomedical research problems. Research Experience Connectomics and Dense Reconstruction of Neural Connectivity Harvard University School of Engineering and Applied Sciences (2012-Present) My research is on software for automatic dense reconstruction of neural connectivity. Using using high-resolution serial sectioning and electron microscopy, biologists are able to image nanometer-scale details of neural structure. This data is extremely large (1 petabyte / cubic mm). The tools I develop in collaboration with the members of the Connectomics project allow biologists to analyze and explore this data. Image analysis, high-throughput screening, & large-scale data analysis Curie Institute, Department of Translational Research (2010 - 2012) Broad Institute of MIT and Harvard, Imaging Platform (2007 - 2010) My research focused on image and data analysis for high-content screening (HCS) and large-scale machine learning applied to HCS data. My work emphasized making powerful analysis methods available to the biological community in user-friendly tools. As part of this effort, I cofounded the open-source CellProfiler project with Dr. Anne Carpenter (director of the Imaging Platform at the Broad Institute). PhD Research: High-throughput microscopy and RNAi to reveal gene function. My doctoral research focused on methods for determining gene function using high- throughput microscopy, living cell microarrays, and RNA interference. To accomplish this goal, my collaborators and I designed and released the first open-source high- throughput cell image analysis software, CellProfiler (www.cellprofiler.org). We also developed advanced data mining methods for high-content screens.
    [Show full text]
  • Human-AI Collaboration for Natural Language Generation with Interpretable Neural Networks
    Human-AI Collaboration for Natural Language Generation with Interpretable Neural Networks a dissertation presented by Sebastian Gehrmann to The John A. Paulson School of Engineering and Applied Sciences in partial fulfillment of the requirements for the degree of Doctor of Philosophy in the subject of Computer Science Harvard University Cambridge, Massachusetts May 2020 ©2019 – Sebastian Gehrmann Creative Commons Attribution License 4.0. You are free to share and adapt these materials for any purpose if you give appropriate credit and indicate changes. Thesis advisors: Barbara J. Grosz, Alexander M. Rush Sebastian Gehrmann Human-AI Collaboration for Natural Language Generation with Interpretable Neural Networks Abstract Using computers to generate natural language from information (NLG) requires approaches that plan the content and structure of the text and actualize it in fuent and error-free language. The typical approaches to NLG are data-driven, which means that they aim to solve the problem by learning from annotated data. Deep learning, a class of machine learning models based on neural networks, has become the standard data-driven NLG approach. While deep learning approaches lead to increased performance, they replicate undesired biases from the training data and make inexplicable mistakes. As a result, the outputs of deep learning NLG models cannot be trusted. Wethus need to develop ways in which humans can provide oversight over model outputs and retain their agency over an otherwise automated writing task. This dissertation argues that to retain agency over deep learning NLG models, we need to design them as team members instead of autonomous agents. We can achieve these team member models by considering the interaction design as an integral part of the machine learning model development.
    [Show full text]
  • F I N a L P R O G R a M 1998
    1998 FINAL PROGRAM October 18 • October 23, 1998 Sheraton Imperial Hotel Research Triangle Park, NC THE INSTITUTE OF ELECTRICAL IEEE IEEE VISUALIZATION 1998 & ELECTRONICS ENGINEERS, INC. COMPUTER IEEE SOCIETY Sponsored by IEEE Computer Society Technical Committee on Computer Graphics In Cooperation with ACM/SIGGRAPH Sessions include Real-time Volume Rendering, Terrain Visualization, Flow Visualization, Surfaces & Level-of-Detail Techniques, Feature Detection & Visualization, Medical Visualization, Multi-Dimensional Visualization, Flow & Streamlines, Isosurface Extraction, Information Visualization, 3D Modeling & Visualization, Multi-Source Data Analysis Challenges, Interactive Visualization/VR/Animation, Terrain & Large Data Visualization, Isosurface & Volume Rendering, Simplification, Marine Data Visualization, Tensor/Flow, Key Problems & Thorny Issues, Image-based Techniques and Volume Analysis, Engineering & Design, Texturing and Rendering, Art & Visualization Get complete, up-to-date listings of program information from URL: http://www.erc.msstate.edu/vis98 http://davinci.informatik.uni-kl.de/Vis98 Volvis URL: http://www.erc.msstate.edu/volvis98 InfoVis URL: http://www.erc.msstate.edu/infovis98 or contact: Theresa-Marie Rhyne, Lockheed Martin/U.S. EPA Sci Vis Center, 919-541-0207, [email protected] Robert Moorhead, Mississippi State University, 601-325-2850, [email protected] Direct Vehicle Loading Access S Dock Salon Salon Sheraton Imperial VII VI Imperial Convention Center HOTEL & CONVENTION CENTER Convention Center Phones
    [Show full text]
  • The Mission of IUPUI Is to Provide for Its Constituents Excellence in  Teaching and Learning;  Research, Scholarship, and Creative Activity; and  Civic Engagement
    INFO H517 Visualization Design, Analysis, and Evaluation Department of Human-Centered Computing Indiana University School of Informatics and Computing, Indianapolis Fall 2016 Section No.: 35557 Credit Hours: 3 Time: Wednesdays 12:00 – 2:40 PM Location: IT 257, Informatics & Communications Technology Complex 535 West Michigan Street, Indianapolis, IN 46202 [map] First Class: August 24, 2016 Website: http://vis.ninja/teaching/h590/ Instructor: Khairi Reda, Ph.D. in Computer Science (University of Illinois, Chicago) Assistant Professor, Human–Centered Computing Office Hours: Mondays, 1:00-2:30PM, or by Appointment Office: IT 581, Informatics & Communications Technology Complex 535 West Michigan Street, Indianapolis, IN 46202 [map] Phone: (317) 274-5788 (Office) Email: [email protected] Website: http://vis.ninja/ Prerequisites: Prior programming experience in a high-level language (e.g., Java, JavaScript, Python, C/C++, C#). COURSE DESCRIPTION This is an introductory course in design and evaluation of interactive visualizations for data analysis. Topics include human visual perception, visualization design, interaction techniques, and evaluation methods. Students develop projects to create their own web- based visualizations and develop competence to undertake independent research in visualization and visual analytics. EXTENDED COURSE DESCRIPTION This course introduces students to interactive data visualization from a human-centered perspective. Students learn how to apply principles from perceptual psychology, cognitive science, and graphics design to create effective visualizations for a variety of data types and analytical tasks. Topics include fundamentals of human visual perception and cognition, 1 2 graphical data encoding, visual representations (including statistical plots, maps, graphs, small-multiples), task abstraction, interaction techniques, data analysis methods (e.g., clustering and dimensionality reduction), and evaluation methods.
    [Show full text]
  • Computational Photography
    Computational Photography George Legrady © 2020 Experimental Visualization Lab Media Arts & Technology University of California, Santa Barbara Definition of Computational Photography • An R & D development in the mid 2000s • Computational photography refers to digital image-capture cameras where computers are integrated into the image-capture process within the camera • Examples of computational photography results include in-camera computation of digital panoramas,[6] high-dynamic-range images, light- field cameras, and other unconventional optics • Correlating one’s image with others through the internet (Photosynth) • Sensing in other electromagnetic spectrum • 3 Leading labs: Marc Levoy, Stanford (LightField, FrankenCamera, etc.) Ramesh Raskar, Camera Culture, MIT Media Lab Shree Nayar, CV Lab, Columbia University https://en.wikipedia.org/wiki/Computational_photography Definition of Computational Photography Sensor System Opticcs Software Processing Image From Shree K. Nayar, Columbia University From Shree K. Nayar, Columbia University Shree K. Nayar, Director (lab description video) • Computer Vision Laboratory • Lab focuses on vision sensors; physics based models for vision; algorithms for scene interpretation • Digital imaging, machine vision, robotics, human- machine interfaces, computer graphics and displays [URL] From Shree K. Nayar, Columbia University • Refraction (lens / dioptrics) and reflection (mirror / catoptrics) are combined in an optical system, (Used in search lights, headlamps, optical telescopes, microscopes, and telephoto lenses.) • Catadioptric Camera for 360 degree Imaging • Reflector | Recorded Image | Unwrapped [dance video] • 360 degree cameras [Various projects] Marc Levoy, Computer Science Lab, Stanford George Legrady Director, Experimental Visualization Lab Media Arts & Technology University of California, Santa Barbara http://vislab.ucsb.edu http://georgelegrady.com Marc Levoy, Computer Science Lab, Stanford • Light fields were introduced into computer graphics in 1996 by Marc Levoy and Pat Hanrahan.
    [Show full text]
  • Compositional Gan: Learning Conditional Image Composition
    Under review as a conference paper at ICLR 2019 COMPOSITIONAL GAN: LEARNING CONDITIONAL IMAGE COMPOSITION Anonymous authors Paper under double-blind review ABSTRACT Generative Adversarial Networks (GANs) can produce images of surprising com- plexity and realism, but are generally structured to sample from a single latent source ignoring the explicit spatial interaction between multiple entities that could be present in a scene. Capturing such complex interactions between different objects in the world, including their relative scaling, spatial layout, occlusion, or viewpoint transformation is a challenging problem. In this work, we propose to model object composition in a GAN framework as a self-consistent composition- decomposition network. Our model is conditioned on the object images from their marginal distributions and can generate a realistic image from their joint distribu- tion. We evaluate our model through qualitative experiments and user evaluations in scenarios when either paired or unpaired examples for the individual object images and the joint scenes are given during training. Our results reveal that the learned model captures potential interactions between the two object domains given as input to output new instances of composed scene at test time in a reasonable fashion. 1I NTRODUCTION Generative Adversarial Networks (GANs) have emerged as a powerful method for generating images conditioned on a given input. The input cue could be in the form of an image (Isola et al., 2017; Zhu et al., 2017; Liu et al., 2017; Azadi et al., 2017; Wang et al., 2017; Pathak et al., 2016), a text phrase (Zhang et al., 2017; Reed et al., 2016b;a; Johnson et al., 2018) or a class label layout (Mirza & Osindero, 2014; Odena et al., 2016; Antoniou et al., 2017).
    [Show full text]