CEO & Co-Founder, Pinscreen, Inc. Distinguished Fellow, University Of
Total Page：16
File Type：pdf, Size：1020Kb
Load more
Recommended publications
-
Learning to Infer Implicit Surfaces Without 3D Supervision
Learning to Infer Implicit Surfaces without 3D Supervision , , , , Shichen Liuy x, Shunsuke Saitoy x, Weikai Chen (B)y, and Hao Liy x z yUSC Institute for Creative Technologies xUniversity of Southern California zPinscreen {liushichen95, shunsuke.saito16, chenwk891}@gmail.com [email protected] Abstract Recent advances in 3D deep learning have shown that it is possible to train highly effective deep models for 3D shape generation, directly from 2D images. This is particularly interesting since the availability of 3D models is still limited compared to the massive amount of accessible 2D images, which is invaluable for training. The representation of 3D surfaces itself is a key factor for the quality and resolution of the 3D output. While explicit representations, such as point clouds and voxels, can span a wide range of shape variations, their resolutions are often limited. Mesh-based representations are more efficient but are limited by their ability to handle varying topologies. Implicit surfaces, however, can robustly handle complex shapes, topologies, and also provide flexible resolution control. We address the fundamental problem of learning implicit surfaces for shape inference without the need of 3D supervision. Despite their advantages, it remains nontrivial to (1) formulate a differentiable connection between implicit surfaces and their 2D renderings, which is needed for image-based supervision; and (2) ensure precise geometric properties and control, such as local smoothness. In particular, sampling implicit surfaces densely is also known to be a computationally demanding and very slow operation. To this end, we propose a novel ray-based field probing technique for efficient image-to-field supervision, as well as a general geometric regularizer for implicit surfaces, which provides natural shape priors in unconstrained regions. -
Innovations Dialogue. DEEPFAKES, TRUST & INTERNATIONAL SECURITY 25 AUGUST 2021 | 09:00-17:30 CEST
the 2021 innovations dialogue. DEEPFAKES, TRUST & INTERNATIONAL SECURITY 25 AUGUST 2021 | 09:00-17:30 CEST WELCOME AND OPENING REMARKS 09:00 Robin Geiss United Nations Institute for Disarmament Research KEYNOTE ADDRESS: TRUST AND INTERNATIONAL SECURITY IN THE ERA 09:10 OF DEEPFAKES Nina Schick Tamang Ventures The scene-setting keynote address will explore the importance of trust for international security and stability and shed light on the extent to which the growing deepfake phenomenon could undermine this trust. UNPACKING DEEPFAKES – CREATION AND DISSEMINATION OF DEEPFAKES 09:40 MODERATED BY: Giacomo Persi Paoli United Nations Institute for Disarmament Research FEATURING: Ashish Jaiman Microsoft Hao Li Pinscreen and University of California, Berkeley Katerina Sedova Center for Security and Emerging Technology, Georgetown University This panel will provide a technical overview of how visual, audio and textual deepfakes are generated and disseminated. The key questions it will seek to address include: what are the underlying technologies of deepfakes? How are deepfakes disseminated, particularly on social media and communication platforms? What types of deepfakes currently exist and what is on the horizon? Which actors currently have the means to create and disseminate them? COFFEE BREAK 11:15 UNDERSTANDING THE IMPLICATIONS FOR INTERNATIONAL SECURITY 11:30 AND STABILITY MODERATED BY: Kerstin Vignard United Nations Institute for Disarmament Research FEATURING: Saiffudin Ahmed Nanyang Technological University Alexi Drew RAND Europe Anita Hazenberg INTERPOL Innovation Centre Moliehi Makumane United Nations Institute for Disarmament Research Carmen Valeria Solis Rivera Ministry of Foreign Affairs of Mexico Petr Topychkanov Stockholm International Peace Research Institute This panel will examine the uses of deepfakes and the extent to which they could erode trust and present novel risks for international security and stability. -
Employing Optical Flow on Convolutional Recurrent Structures for Deepfake Detection
Rochester Institute of Technology RIT Scholar Works Theses 6-13-2020 Employing optical flow on convolutional recurrent structures for deepfake detection Akash Chintha [email protected] Follow this and additional works at: https://scholarworks.rit.edu/theses Recommended Citation Chintha, Akash, "Employing optical flow on convolutional recurrent structures for deepfake detection" (2020). Thesis. Rochester Institute of Technology. Accessed from This Thesis is brought to you for free and open access by RIT Scholar Works. It has been accepted for inclusion in Theses by an authorized administrator of RIT Scholar Works. For more information, please contact [email protected]. Employing optical flow on convolutional recurrent structures for deepfake detection by Akash Chintha June 13, 2020 A Thesis Submitted in Partial Fulfillment of the Requirements for the Degree of Master of Science in Computer Engineering Department of Computer Engineering Kate Gleason College of Engineering Rochester Institute of Technology Approved By: Dr. Raymond Ptucha Date Primary Advisor - RIT Dept. of Computer Engineering Dr. Matthew Wright Date Secondary Advisor - RIT Dept. of Computing and Information Sciences Dr. Shanchieh Yang Date Secondary Advisor - RIT Dept. of Computer Engineering Akash Chintha's Masters Thesis Document 1 ACKNOWLEDGEMENTS I would like to take this opportunity to thank Dr. Raymond Ptucha and Dr. Matthew Wright for continuous guidance and support in my research. I am also thankful to Dr. Shanchieh Yang for showing interest in my work and being a part of the thesis committee. I would like to thank my coresearchers Saniat Sohrawardi, Aishwarya Rao, Bao Thai, Akhil Santha, and Kartavya Bhatt for assisting me with my thesis research. -
Capturing and Simulating Physically Accurate Illumination in Computer Graphics
Capturing and Simulating Physically Accurate Illumination in Computer Graphics PAUL DEBEVEC Institute for Creative Technologies University of Southern California Marina del Rey, California Anyone who has seen a recent summer blockbuster has witnessed the dramatic increases in computer-generated realism in recent years. Visual effects supervisors now report that bringing even the most challenging visions of film directors to the screen is no longer a question of whats possible; with todays techniques it is only a matter of time and cost. Driving this increase in realism have been computer graphics (CG) techniques for simulating how light travels within a scene and for simulating how light reflects off of and through surfaces. These techniquessome developed recently, and some originating in the 1980sare being applied to the visual effects process by computer graphics artists who have found ways to channel the power of these new tools. RADIOSITY AND GLOBAL ILLUMINATION One of the most important aspects of computer graphics is simulating the illumination within a scene. Computer-generated images are two-dimensional arrays of computed pixel values, with each pixel coordinate having three numbers indicating the amount of red, green, and blue light coming from the corresponding direction within the scene. Figuring out what these numbers should be for a scene is not trivial, since each pixels color is based both on how the object at that pixel reflects light and also the light that illuminates it. Furthermore, the illumination does not only come directly from the lights within the scene, but also indirectly Capturing and Simulating Physically Accurate Illumination in Computer Graphics from all of the surrounding surfaces in the form of bounce light. -
Image-Based Lighting
Tutorial Image-Based Paul Debevec Lighting USC Institute for Creative Technologies mage-based lighting (IBL) is the process of 2. mapping the illumination onto a representation of Iilluminating scenes and objects (real or syn- the environment; thetic) with images of light from the real world. It 3. placing the 3D object inside the environment; and evolved from the reflection-mapping technique1,2 in 4. simulating the light from the environment illumi- which we use panoramic images as nating the computer graphics object. texture maps on computer graphics This tutorial shows how models to show shiny objects reflect- Figure 1 shows an example of an object illuminated ing real and synthetic environments. entirely using IBL. Gary Butcher created the models in image-based lighting can IBL is analogous to image-based 3D Studio Max, and the renderer used was the Arnold modeling, in which we derive a 3D global illumination system written by Marcos Fajardo. illuminate synthetic objects scene’s geometric structure from I captured the light in a kitchen so it includes light from images, and to image-based render- a ceiling fixture; the blue sky from the windows; and with measurements of real ing, in which we produce the ren- the indirect light from the room’s walls, ceiling, and dered appearance of a scene from its cabinets. Gary mapped the light from this room onto a light, making objects appear appearance in images. When used large sphere and placed the model of the microscope effectively, IBL can produce realistic on the table in the middle of the sphere. -
Rapid Avatar Capture and Simulation Using Commodity Depth Sensors
Rapid Avatar Capture and Simulation using Commodity Depth Sensors Ari Shapiro∗1, Andrew Feng†1, Ruizhe Wang‡2, Hao Li§2, Mark Bolas¶1, Gerard Medionik2, and Evan Suma∗∗1 1Institute for Creative Technologies, University of Southern California 2University of Southern California Abstract objects into a virtual environment in 3D. For ex- We demonstrate a method of acquiring a 3D ample, a table, a room, or work of art can be model of a human using commodity scanning quickly scanned, modeled and displayed within hardware and then controlling that 3D figure in a virtual world with a handheld, consumer scan- a simulated environment in only a few minutes. ner. There is great value to the ability to quickly The model acquisition requires 4 static poses and inexpensively capture real-world objects taken at 90 degree angles relative to each and create their 3D counterparts. While numer- other. The 3D model is then given a skeleton ous generic 3D models are available for low- and smooth binding information necessary for or no-cost for use in 3D environments and vir- control and simulation. The 3D models that tual worlds, it is unlikely that such acquired 3D are captured are suitable for use in applications model matches the real object to a reasonable where recognition and distinction among char- extent without individually modeling the object. acters by shape, form or clothing is important, In addition, the ability to capture specific ob- such as small group or crowd simulations, or jects that vary from the generic counterparts is other socially oriented applications. Due to the valuable for recognition, interaction and com- speed at which a human figure can be captured prehension within a virtual world. -
ARCH: Animatable Reconstruction of Clothed Humans
ARCH: Animatable Reconstruction of Clothed Humans Zeng Huang1,2,∗ Yuanlu Xu1, Christoph Lassner1, Hao Li2, Tony Tung1 1Facebook Reality Labs, Sausalito, USA 2University of Southern California, USA [email protected], [email protected], [email protected], [email protected], [email protected] Canonical Reconstructed Abstract Input Reposed Reconstruction 3D Avatar In this paper, we propose ARCH (Animatable Recon- struction of Clothed Humans), a novel end-to-end frame- work for accurate reconstruction of animation-ready 3D clothed humans from a monocular image. Existing ap- proaches to digitize 3D humans struggle to handle pose variations and recover details. Also, they do not produce models that are animation ready. In contrast, ARCH is a learned pose-aware model that produces detailed 3D rigged full-body human avatars from a single unconstrained RGB image. A Semantic Space and a Semantic Deformation Field are created using a parametric 3D body estimator. They allow the transformation of 2D/3D clothed humans into a canonical space, reducing ambiguities in geometry caused by pose variations and occlusions in training data. Figure 1. Given an image of a subject in arbitrary pose (left), Detailed surface geometry and appearance are learned us- ARCH creates an accurate and animatable avatar with detailed ing an implicit function representation with spatial local clothing (center). As rigging and albedo are estimated, the avatar can be reposed and relit in new environments (right). features. Furthermore, we propose additional per-pixel su- pervision on the 3D reconstruction using opacity-aware dif- Moreover, the lack of automatic rigging prevents animation- ferentiable rendering. Our experiments indicate that ARCH based applications. -
Headphones Needed: ☒ YES ☐ NO
Erik Amerikaner.guru Tech In The News Headphones Needed: ☒ YES ☐ NO Step One: Watch this video Step Two: Read the story at the bottom of this document Step Three: Navigate to this website and review the information Step Four: Create a Word document, and list the five W’s (Who,What,Where,When,Why) of the story. Why is this technology potentially so important to society? What are the pros/cons of this technology? Can it be used for good and/or evil? What should a ‘consumer’ of this technology be aware of? If you were showing this technology to a young person, how would you describe it? Do you think this technology should be developed? Why/why not? Submit Your PinScreen_your name Assignment : To Mr. Amerikaner Gdrive Using: 2/19/2018 Fake videos are on the rise. As they become more realistic, seeing shouldn't always be believing ADVERTISEMENT 12 WEEKS FOR 99¢ TOPICS LOG IN Sale ends 2/28 SPECIAL OFFER | 12 WEEKS FOR 99¢ Former Trump aide Richard Netanyahu in deeper peril as L.A. County's homeless problem Toyota Prius so Gates to plead guilty; agrees to more Israeli officials are arrested is worsening despite billions reduce fuel effi TECHNOLOGY BUSINESS LA TIMES testify against Manafort, sourc… on corruption charges from tax measures say Fake videos are on the rise. As they become more realistic, seeing shouldn't always be believing By DAVID PIERSON FEB 19, 2018 | 4:00 AM 12 WEEKS FOR ONLY 99¢ SPECIAL OFFER SAVE NOW Hurry, offer ends 2/28 Pinscreen, a Los Angeles start-up, uses real-time capture technology to make photorealistic avatars. -
Chloe A. Legendre E-Mail: [email protected] Cell: (443) 690-6924
Chloe A. LeGendre E-mail: [email protected] Cell: (443) 690-6924 www.chloelegendre.com RESEARCH I develop and apply computer vision and computational photography techniques to STATEMENT solve problems in computer graphics, towards the goal of creating compelling, photo- realistic content for people to enjoy. RESEARCH Computational Photography, Appearance Capture, Lighting Capture, Color Imaging & INTERESTS Measurement, Computer Vision for Computer Graphics EDUCATION University of Southern California, Los Angeles, CA August 2015 - May 2019 Ph.D., Computer Science; GPA: 3.82 Dissertation: ”Compositing Real and Virtual Objects with Realistic, Color-Accurate Illumination” Stevens Institute of Technology, Hoboken, NJ September 2012 - May 2015 M.S., Computer Science; GPA: 4.00 University of Pennsylvania, Philadelphia, PA September 2005 - May 2009 B.S. in Engineering; GPA: 3.69 PROFESSIONAL Google Research, Los Angeles, CA EXPERIENCE Senior Software Engineer - Computational Photography June 2020 - present • Core technology research and development for computational photography. • Developed machine learning based lighting estimation module for the Portrait Light feature, a hero launch for the Google Pixel 2020 phone in both Google Camera and Google Photos. Google Daydream (VR/AR), Los Angeles, CA Software Engineer - Augmented Reality June 2019 - June 2020 • Core technology research and development for mobile AR. Student Researcher - Augmented Reality December 2017 - May 2019 • ML-based lighting estimation technique was published at CVPR 2019 and shipped as the Environmental HDR Lighting Estimation module for ARCore in June 2019. • Trained ML models using distributed Tensorflow environment. Software Engineering Intern - Augmented Reality June 2017 - August 2017 • Developed mobile AR lighting estimation techniques for Google Pixel Phone’s AR Stickers camera mode (Playground). -
SIGGRAPH 2008 Course: Computational Photography: Advanced Topics
SIGGRAPH 2008 Course: Computational Photography: Advanced Topics http://computationalphotography.org Speakers Paul Debevec (USC, USA) (debevec (at) ict.usc.edu) Ramesh RASKAR (MIT Media Lab, USA) (raskar (at) media.mit.edu) Jack TUMBLIN (Northwestern University, USA) (jet (at) cs.northwestern.edu) Course Abstract Computational photography combines plentiful computing, digital sensors, modern optics, many varieties of actuators, probes and smart lights to escape the limitations of traditional film cameras and enables novel imaging applications. Unbounded dynamic range, variable focus, resolution, and depth of field, hints about shape, reflectance, and lighting, and new interactive forms of photos that are partly snapshots and partly videos, performance capture and interchangeably relighting real and virtual characters are just some of the new applications emerging in Computational Photography. The computational techniques encompass methods from modification of imaging parameters during capture to sophisticated reconstructions from indirect measurements. We will bypass basic and introductory material presented in earlier versions of this course (Computational Photography 2005,6,7) and expand coverage of more recent topics. Emphasizing more recent work in computational photography and related fields (2006 or later) this course will give more attention to advanced topics only briefly touched before, including tomography, heterodyning and Fourier Slice applications, inverse problems, gradient illumination, novel optics, emerging sensors -
Protecting World Leaders Against Deep Fakes
Protecting World Leaders Against Deep Fakes Shruti Agarwal and Hany Farid University of California, Berkeley Berkeley CA, USA {shruti agarwal, hfarid}@berkeley.edu Yuming Gu, Mingming He, Koki Nagano, and Hao Li University of Southern California / USC Institute for Creative Technologies Los Angeles CA, USA {ygu, he}@ict.usc.edu, [email protected], [email protected] Abstract peared [5], and used to create non-consensual pornography in which one person’s likeliness in an original video is re- The creation of sophisticated fake videos has been placed with another person’s likeliness [13]; (2) lip-sync, in largely relegated to Hollywood studios or state actors. Re- which a source video is modified so that the mouth region is cent advances in deep learning, however, have made it sig- consistent with an arbitrary audio recording. For instance, nificantly easier to create sophisticated and compelling fake the actor and director Jordan Peele produced a particularly videos. With relatively modest amounts of data and com- compelling example of such media where a video of Presi- puting power, the average person can, for example, create a dent Obama is altered to say things like “President Trump is video of a world leader confessing to illegal activity leading a total and complete dip-****.”; and (3) puppet-master, in to a constitutional crisis, a military leader saying something which a target person is animated (head movements, eye racially insensitive leading to civil unrest in an area of mil- movements, facial expressions) by a performer sitting in itary activity, or a corporate titan claiming that their profits front of a camera and acting out what they want their puppet are weak leading to global stock manipulation. -
![Arxiv:2012.02512V3 [Cs.CV] 20 Aug 2021 a High Level of Realism](https://docslib.b-cdn.net/cover/7838/arxiv-2012-02512v3-cs-cv-20-aug-2021-a-high-level-of-realism-3007838.webp)
Arxiv:2012.02512V3 [Cs.CV] 20 Aug 2021 a High Level of Realism
ID-Reveal: Identity-aware DeepFake Video Detection Davide Cozzolino1 Andreas Rossler¨ 2 Justus Thies2,3 Matthias Nießner2 Luisa Verdoliva1 1University Federico II of Naples 2Technical University of Munich 3Max Planck Institute for Intelligent Systems, Tubingen¨ Abstract A major challenge in DeepFake forgery detection is that state-of-the-art algorithms are mostly trained to detect a specific fake method. As a result, these approaches show poor generalization across different types of facial manip- ulations, e.g., from face swapping to facial reenactment. To this end, we introduce ID-Reveal, a new approach that learns temporal facial features, specific of how a person moves while talking, by means of metric learning coupled with an adversarial training strategy. The advantage is that Figure 1: ID-Reveal is an identity-aware DeepFake video detec- we do not need any training data of fakes, but only train on tion. Based on reference videos of a person, we estimate a tem- real videos. Moreover, we utilize high-level semantic fea- poral embedding which is used as a distance metric to detect fake tures, which enables robustness to widespread and disrup- videos. tive forms of post-processing. We perform a thorough exper- imental analysis on several publicly available benchmarks. Compared to state of the art, our method improves gener- basis. As a result, supervised detection, which requires ex- alization and is more robust to low-quality videos, that are tensive training data of a specific forgery method, cannot usually spread over social networks. In particular, we ob- immediately detect a newly-seen forgery type. tain an average improvement of more than 15% in terms of This mismatch and generalization issue has been ad- accuracy for facial reenactment on high compressed videos.