DOCSLIB.ORG

Deep Learning Based Robust Human Body Segmentation for Pose Estimation from RGB-D Sensors

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Deep Learning Based Robust Human Body Segmentation for Pose Estimation from RGB-D Sensors University of Nevada, Reno Deep Learning Based Robust Human Body Segmentation for Pose Estimation from RGB-D Sensors A thesis submitted in partial fulfillment of the requirements for the degree of Master of Science in Computer Science and Engineering by David Frank Dr. David Feil-Seifer, Thesis Advisor May, 2016 THE GRADUATE SCHOOL We recommend that the thesis prepared under our supervision by DAVID FRANK Entitled Deep Learning Based Robust Human Body Segmentation For Pose Estimation From RGB-D Sensors be accepted in partial fulfillment of the requirements for the degree of MASTER OF SCIENCE Dr. David Feil-Seifer., Advisor Dr. Monica Nicolescu, Committee Member Dr. Jacqueline Snow, Graduate School Representative David W. Zeh, Ph.D., Dean, Graduate School May, 2016 i Abstract This project focuses on creating a system for human body segmentation meant to be used for pose estimation. Recognizing a human figure in a cluttered environment is a challenging problem. Current systems for pose estimation assume that there are no objects around the person, which restricts their use in a real world scenario. This project is based on new advances in deep learning, a field of machine learning that can tackle tough vision problems. The system contains a whole pipeline for training and using a system to estimate the pose of a human. It contains a data generation module that creates the training data for the deep learning module. The deep learning module is the main contribution of this work and provides a robust method for segmenting the body parts of a human. Finally, the project includes a pose estimation module which focuses on reducing the detailed output of the deep learning module into a pose skeleton. ii Acknowledgments This material is based in part upon work supported by: NASA Space Grant: NNX10AN23H, the Nevada Governor’s Office of Economic Development (NV-GOED: OSP-1400872), and Flirtey Technology Pty Ltd., and by Cubix Corporation through use of their PCIe slot expansion hardware solutions and HostEngine. Software used in the implementation of this project include: Blender, MakeHuman, OpenEXR, HDF5, The Point Cloud Library, and Torch7. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of NASA, NV-GOED, Cubix Corporation, Blender Foundation, Open Perception, The HDF Group, MakeHuman Team, Flirtey Technology Pty Ltd., Industrial Light & Magic, Deepmind Technologies, NYU, NEC Laboratories America, Flirtey Technology Pty Ltd or IDIAP Research Institute. Advisor: Dr. David Feil-Seifer Committee members: Dr. Monica Nicolescu Dr. Jacqueline Snow Essential Guidance: Dr. Richard Kelley Computational Horsepower Courtesy of: Dr. Frederick C. Harris Network and Computer Wizard: Zachary Newell Provider of Template and Elusive Information About Graduating: Jessica Smith iii Contents Abstract i Acknowledgments ii List of Tables v List of Figures vi 1 Introduction 1 2 Background 4 2.1 Pose Estimation . .4 2.2 Deep Learning . .9 2.3 Related Work . 13 3 Data Generation 16 3.1 Creating Data in Blender . 16 3.1.1 Human Model . 17 3.1.2 Clutter and Occlusions . 18 3.2 Post Processing . 20 3.3 Data Sets . 21 4 Network Training 24 4.1 Network Structure . 24 5 Pose Estimation 27 5.1 Point Cloud Representation . 27 5.2 Pose Skeleton . 28 6 Results 31 6.1 Mask Images . 31 6.1.1 Set 1 . 31 6.1.2 Set 2 . 33 6.1.3 Set 3 . 33 6.1.4 Set 4 . 33 6.2 Pose Estimation . 35 iv 6.3 Real Data . 38 6.4 Discussion . 39 7 Conclusion 40 7.1 Summary . 40 7.2 Future Work . 40 A Supporting Software 42 A.1 Blender . 42 A.2 MakeHuman . 42 A.3 OpenEXR . 43 A.4 Numpy . 43 A.5 HDF5 . 43 A.6 Torch . 43 A.7 PCL . 44 Bibliography 45 v List of Tables 3.1 The body parts of interest and their color labels . 19 6.1 Per class accuracy for set 1 . 32 6.2 Per class accuracy for set 2 . 33 6.3 Per class accuracy for set 3 . 33 6.4 Per class accuracy for set 4 . 35 vi List of Figures 2.1 Examples of pose skeletons detected by the Microsoft Kinect. These poses record 2D position in the frame, as well as the depth . .5 2.2 The stages of the Kinect system. The left images show the depth data, the middle images show the segemations from the RFD. Finally, the right images show the 3D pose estimation . .7 2.3 A simple Random Decision Tree for predicting Titanic survivors. The numbers at the leaves give the percentage for survival followed by the amount of samples at that leaf. .7 2.4 A single neuron in a neural network . .9 2.5 A simple neural network showing the connections between neurons . 10 2.6 An example of a convolutional neural network. Each pixel in later layers is taken from a window of pixels in the previous layers . 13 3.1 The human model textured for labeling in a default pose as it appears in Blender . 17 3.2 The human model textured for labeling in a default pose as it is rendered 18 3.3 The scene after the model has been posed and clutter objects have been added as seen in Blender . 19 3.4 The scene after the model has been posed and clutter objects have been added as rendered for viewing . 20 3.5 Example labeled images from Set 1 . 21 3.6 Example labeled images from Set 2 . 22 3.7 Example labeled images from Set 3 . 22 3.8 Example labeled images from Set 4 . 23 4.1 An outline of the fully convolutional network. Convolutional layers are shown as vertical lines with the number of feature planes they contain above them. Max pooling layers are shown as rectangles (all used kernels of 2x2) . 25 5.1 A point cloud representation of a person and the corresponding pose vectors . 29 6.1 Network predictions for Set 1 data . 32 6.2 Good performance network predictions for Set 2 data . 34 6.3 The hybrid network running on an image similar to those in Set 1 . 34 vii 6.4 Reduced performance network predictions for Set 2 data . 35 6.5 Different views of the 3D reconstruction of a person, Large dots mark the point centers . 36 6.6 The orientation of the body parts shown as vectors . 37 6.7 Performance on real data. Top left: RGB image, not used in process- ing. Top right: depth image. Bottom: network predictions . 38 1 Chapter 1 Introduction Person detection and pose estimation are common needs in Human Robot Interaction (HRI). Person detection is simply recognizing that there is a person nearby. The information gained may be the location of a person. This is useful, but lacks many important clues about what a person is doing. Pose classification goes a step past simple detection and gives a more complete description of the person, such as the location of the person’s arms and legs. With this information, a robot can gain a more complete understanding of a person than it can just by knowing where the person is. For example, a waving gesture may indicate that the person is trying to gain the robot’s attention while crossed arms may indicate an unwillingness to interact. This project focuses mainly on pose estimation. For a robotic system, the ability to locate a person is essential for achieving a basic level of social interaction [12]. For example, a robotic waiter may need recognize when a patron has approached it; or it may need to see when somebody is looking in its direction and waving. Tele-presence rehabilitation is another application [16], a person can engage in rehabilitation exercises monitored by a pose tracker to ensure that they are doing the exercise correctly. Many methods exist for doing pose detection [13]. One of the most prominent methods is used by Microsoft for the Kinect [18] [19]. This method uses depth images and a two stage system for classifying poses. The two stages are as follows: First, a mask image is produced from the depth image that labels each pixel as a body part of interest. Second, this mask image is used to calculate the center of each body part. 2 By using the depth data, that center can then be placed into the scene to get the 3D location of that body part. The method in [18] and other possible pose estimation methods such as [21] do not account for objects within the vicinity of the person. The environment needs to be structured so that any environemnt or non-person objects are easy to isolate from the person data. For example; in an entertainment scenario the person can be assumed to be a certain known distance from the sensor, which eliminates objects not near this assumed location, the floor around the person can be removed with well-known plane detection techniques. The method used in this project follows a very similar structure to the one used by [18] for the Xbox Kinect. Depth images are used since they have several advantages over color or black and white images for this application. They are unaffected by changes in clothing color and texture; this helps to remove unnecessary information from the image. Depth images naturally lend themselves to creating three dimensional (3D) representations of the environment which makes getting a 3D pose simple. The first stage of the system uses deep learning techniques to produce a mask image. Recently, a method for using convolutional neural networks (CNNs) to seg- ment images was proposed called Fully Convolutional Networks (FCNs) [11].
Load more

Recommended publications
  • Procedural Content Generation for Games
    Procedural Content Generation for Games Inauguraldissertation zur Erlangung des akademischen Grades eines Doktors der Naturwissenschaften der Universit¨atMannheim vorgelegt von M.Sc. Jonas Freiknecht aus Hannover Mannheim, 2020 Dekan: Dr. Bernd L¨ubcke, Universit¨atMannheim Referent: Prof. Dr. Wolfgang Effelsberg, Universit¨atMannheim Korreferent: Prof. Dr. Colin Atkinson, Universit¨atMannheim Tag der m¨undlichen Pr¨ufung: 12. Februar 2021 Danksagungen Nach einer solchen Arbeit ist es nicht leicht, alle Menschen aufzuz¨ahlen,die mich direkt oder indirekt unterst¨utzthaben. Ich versuche es dennoch. Allen voran m¨ochte ich meinem Doktorvater Prof. Wolfgang Effelsberg danken, der mir - ohne mich vorher als Master-Studenten gekannt zu haben - die Promotion an seinem Lehrstuhl erm¨oglichte und mit Geduld, Empathie und nicht zuletzt einem mir unbegreiflichen Verst¨andnisf¨ur meine verschiedenen Ausfl¨ugein die Weiten der Informatik unterst¨utzthat. Sie werden mir nicht glauben, wie dankbar ich Ihnen bin. Weiterhin m¨ochte ich meinem damaligen Studiengangsleiter Herrn Prof. Heinz J¨urgen M¨ullerdanken, der vor acht Jahren den Kontakt zur Universit¨atMannheim herstellte und mich ¨uberhaupt erst in die richtige Richtung wies, um mein Promotionsvorhaben anzugehen. Auch Herr Prof. Peter Henning soll nicht ungenannt bleiben, der mich - auch wenn es ihm vielleicht gar nicht bewusst ist - davon ¨uberzeugt hat, dass die Erzeugung virtueller Welten ein lohnenswertes Promotionsthema ist. Ganz besonderer Dank gilt meiner Frau Sarah und meinen beiden Kindern Justus und Elisa, die viele Abende und Wochenenden zugunsten dieser Arbeit auf meine Gesellschaft verzichten mussten. Jetzt ist es geschafft, das n¨achste Projekt ist dann wohl der Garten! Ebenfalls geb¨uhrt meinen Eltern und meinen Geschwistern Dank.
    [Show full text]
  • Automatic Generation of a 3D City Model
    UNIVERSITY OF CASTILLA-LA MANCHA ESCUELA SUPERIOR DE INFORMÁTICA COMPUTER ENGINEERING DEGREE DEGREE FINAL PROJECT Automatic generation of a 3D city model David Murcia Pacheco June, 2017 AUTOMATIC GENERATION OF A 3D CITY MODEL Escuela Superior de Informática UNIVERSITY OF CASTILLA-LA MANCHA ESCUELA SUPERIOR DE INFORMÁTICA Information Technology and Systems SPECIFIC TECHNOLOGY OF COMPUTER ENGINEERING DEGREE FINAL PROJECT Automatic generation of a 3D city model Author: David Murcia Pacheco Director: Dr. Félix Jesús Villanueva Molina June, 2017 David Murcia Pacheco Ciudad Real – Spain E-mail: [email protected] Phone No.:+34 625 922 076 c 2017 David Murcia Pacheco Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.3 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the section entitled "GNU Free Documentation License". i TRIBUNAL: Presidente: Vocal: Secretario: FECHA DE DEFENSA: CALIFICACIÓN: PRESIDENTE VOCAL SECRETARIO Fdo.: Fdo.: Fdo.: ii Abstract HIS document collects all information related to the Degree Final Project (DFP) of Com- T puter Engineering Degree of the student David Murcia Pacheco, tutorized by Dr. Félix Jesús Villanueva Molina. This work has been developed during 2016 and 2017 in the Escuela Superior de Informática (ESI), in Ciudad Real, Spain. It is based in one of the proposed sub- jects by the faculty of this university for this year, called "Generación automática del modelo en 3D de una ciudad".
    [Show full text]
  • Benchmarks on WWW Performance
    The Scalability of X3D4 PointProperties: Benchmarks on WWW Performance Yanshen Sun Thesis submitted to the Faculty of the Virginia Polytechnic Institute and State University in partial fulfillment of the requirements for the degree of Master of Science in Computer Science and Application Nicholas F. Polys, Chair Doug A. Bowman Peter Sforza Aug 14, 2020 Blacksburg, Virginia Keywords: Point Cloud, WebGL, X3DOM, x3d Copyright 2020, Yanshen Sun The Scalability of X3D4 PointProperties: Benchmarks on WWW Performance Yanshen Sun (ABSTRACT) With the development of remote sensing devices, it becomes more and more convenient for individual researchers to acquire high-resolution point cloud data by themselves. There have been plenty of online tools for the researchers to exhibit their work. However, the drawback of existing tools is that they are not flexible enough for the users to create 3D scenes of a mixture of point-based and triangle-based models. X3DOM is a WebGL-based library built on Extensible 3D (X3D) standard, which enables users to create 3D scenes with only a little computer graphics knowledge. Before X3D 4.0 Specification, little attention has been paid to point cloud rendering in X3DOM. PointProperties, an appearance node newly added in X3D 4.0, provides point size attenuation and texture-color mixing effects to point geometries. In this work, we propose an X3DOM implementation of PointProperties. This implementation fulfills not only the features specified in X3D 4.0 documentation, but other shading effects comparable to the effects of triangle-based geometries in X3DOM, as well as other state-of-the-art point cloud visualization tools.
    [Show full text]
  • Comparative Analysis of Human Modeling Tools Emilie Poirson, Mathieu Delangle
    Comparative analysis of human modeling tools Emilie Poirson, Mathieu Delangle To cite this version: Emilie Poirson, Mathieu Delangle. Comparative analysis of human modeling tools. International Digital Human Modeling Symposium, Jun 2013, Ann Arbor, United States. hal-01240890 HAL Id: hal-01240890 https://hal.archives-ouvertes.fr/hal-01240890 Submitted on 24 Dec 2015 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. Comparative analysis of human modeling tools Emilie Poirson & Matthieu Delangle LUNAM, IRCCYN, Ecole Centrale de Nantes, France April 25, 2013 Abstract sometimes a multitude of functions that are not suitable for his application case. Digital Human Modeling tools simulate a task performed by a human in a virtual environment and provide useful The first step of our study consisted in listing all indicators for ergonomic, universal design and represen- the comparable software and to select the comparison tation of product in situation. The latest developments criteria. Then a list of indicators is proposed, in three in this field are in terms of appearance, behaviour and major categories: degree of realism, functions and movement. With the considerable increase of power com- environment. Based on software use, literature searches puters,some of these programs incorporate a number of [7] and technical reports ([8], [9], [10], for example), the key details that make the result closer and closer to a real table of indicator is filled and coded from text to a quinary situation.
    [Show full text]
  • Christian Otten
    CHRISTIAN OTTEN 3D GENERALIST PORTFOLIO 2018 Demo Scene - Apothecary Modelled with Maya | ngplant 2017 Rendered with Corona for C4D Compositing with Nuke demo scene - sector 51 Modelled with Maya | Cinema 4D 2017 Rendered with Corona for C4D Compositing with After Effects (Lens Flares) and Nuke demo scene - wynyard Modelled with Maya | zBrush | ngplant 2017 Rendered with Vray (Raven) and Corona Compositing with Nuke prototype Modelled with Cinema 4D 2018 Rendered with Corona Compositing with Nuke interiors Modelled with Cinema 4D | 3D Studio Max 2014-2018 Rendered with Corona | Vray | C4D Physical Renderer Compositing with Photoshop | Nuke exteriors Modelled with Cinema 4D | Maya | zbrush | ngplant 2011-2018 Rendered with Corona | Vray | C4D Physical Renderer Compositing with Photoshop | Nuke fantasy Modelled with Cinema 4D | zBrush | ngplant | makehuman 2011-2018 Rendered with Corona | C4D Physical Renderer Compositing with Photoshop | darktable | Nuke futuristic Modelled with Cinema 4D | zBrush 2012-2015 Rendered with C4D Physical Renderer Compositing with Photoshop For a more comprehensive portfolio feel free to visit: christianotten.daportfolio.com or ignisferroque.cgsociety.org A few animated experiments are available on: https://vimeo.com/christianotten All models, scenes and materials presented here where made by me, unless stated otherwise. Photo textures from cgtextures.com Thank you for watching! CHRISTIAN OTTEN Curriculum Vitae PERSONAL INFORMATION EDUCATION: Date of Birth: 09.09.1984 2016-2017 3D Animation and VFX course Place
    [Show full text]
  • Seamless Texture Mapping of 3D Point Clouds
    Seamless Texture Mapping of 3D Point Clouds Dan Goldberg Mentor: Carl Salvaggio Chester F. Carlson Center for Imaging Science, Rochester Institute of Technology Rochester, NY November 25, 2014 Abstract The two similar, quickly growing fields of computer vision and computer graphics give users the ability to immerse themselves in a realistic computer generated environment by combining the ability create a 3D scene from images and the texture mapping process of computer graphics. The output of a popular computer vision algorithm, structure from motion (obtain a 3D point cloud from images) is incomplete from a computer graphics standpoint. The final product should be a textured mesh. The goal of this project is to make the most aesthetically pleasing output scene. In order to achieve this, auxiliary information from the structure from motion process was used to texture map a meshed 3D structure. 1 Introduction The overall goal of this project is to create a textured 3D computer model from images of an object or scene. This problem combines two different yet similar areas of study. Computer graphics and computer vision are two quickly growing fields that take advantage of the ever-expanding abilities of our computer hardware. Computer vision focuses on a computer capturing and understanding the world. Computer graphics con- centrates on accurately representing and displaying scenes to a human user. In the computer vision field, constructing three-dimensional (3D) data sets from images is becoming more common. Microsoft's Photo- synth (Snavely et al., 2006) is one application which brought attention to the 3D scene reconstruction field. Many structure from motion algorithms are being applied to data sets of images in order to obtain a 3D point cloud (Koenderink and van Doorn, 1991; Mohr et al., 1993; Snavely et al., 2006; Crandall et al., 2011; Weng et al., 2012; Yu and Gallup, 2014; Agisoft, 2014).
    [Show full text]
  • John Bachofer Graphic Designer
    JOHN BACHOFER GRAPHIC DESIGNER Animation/Graphic Design/3D Modelling Marketing/Freelancer [email protected] | (760)-518-0145 http://johnkathrynjanewayba.wixsite.com/johnbachoferartist https://www.linkedin.com/in/johnny-bachofer-32888ab8/ Education Summary B.S. in Graphic Design with Meticulous and knowledgeable Graphic Design graduate known for skill in 3D modeling, rigging Emphasis on Animation and texturing. Deadline driven and results oriented artist who has exhibited exceptional talent in Grand Canyon University building highly detailed products yielding customer satisfaction. Graduated 2018 Key Skills include: graphic design, illustration, modeling/texturing, rigging and animation Software Work Experience Autodesk MAYA Source FilmMaker Real Art Daily Productions - Los Angeles, CA (Online) 11/2019 - Present Blender 3D Adobe Creative Lightwave 3D Suite: Character Animator DAZ Studio Photoshop • Mastered the use of Unreal Engine 4. Unreal Engine 4 Illustrator • Developed comprehensive 3D modelling and rigging skills. MakeHuman InDesign • Completed the company’s first animation of a quadruped character. Autodesk 3DS MAX After Effects • Met project milestones for writing, storyboard development and comencement of production. Unity Lightroom • Successful team member in a diverse and inclusive workplace. ZBrush Microsoft Office Sketchup Suite: Mogul Mommies Inc - New York, NY (Online) Feb 2017 - May 2018 AutoCAD Excel Game Development Artist Mixamo Fuse Word • Developed and released the “Toss That!” mobile app game. Poser Powerpoint
    [Show full text]
  • Easy Facial Rigging and Animation Approaches
    Pedro Tavares Barata Bastos EASY FACIAL RIGGING AND ANIMATION APPROACHES A dissertation in Computer Graphics and Human-Computer Interaction Presented to the Faculty of Engineering of the University of Porto in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy in Digital Media Supervisor: Prof. Verónica Costa Orvalho April 2015 ii This work is financially supported by Fundação para a Ciência e a Tecnologia (FCT) via grant SFRH/BD/69878/2010, by Fundo Social Europeu (FSE), by Ministério da Educação e Ciência (MEC), by Programa Operacional Potencial Humano (POPH), by the European Union (EU) and partially by the UT Austin | Portugal program. Abstract Digital artists working in character production pipelines need optimized facial animation solutions to more easily create appealing character facial expressions for off-line and real- time applications (e.g. films and videogames). But the complexity of facial animation has grown exponentially since it first emerged during the production of Toy Story (Pixar, 1995), due to the increasing demand of audiences for better quality character facial animation. Over the last 15 to 20 years, companies and artists developed various character facial animation techniques in terms of deformation and control, which represent a fragmented state of the art in character facial rigging. Facial rigging is the act of planning and building the mechanical and control structures to animate a character's face. These structures are the articulations built by riggers and used by animators to bring life to a character. Due to the increasing demand of audiences for better quality facial animation in films and videogames, rigging faces became a complex field of expertise within character production pipelines.
    [Show full text]
  • AUTOR: Trabajo De Titulación Previo a La
    FACULTAD DE ESPECIALIDADES EMPRESARIALES CARRERA DE EMPRENDIMIENTO TEMA: “Propuesta para la creación de una empresa productora y comercializadora de réplicas de figuras a tamaño escala en 3D en la ciudad de Santiago de Guayaquil” AUTOR: García Ruiz, Andrés Alexander. Trabajo de titulación previo a la obtención del título de INGENIERO EN DESARROLLO DE NEGOCIOS BILINGüE. TUTOR: Ing. Rolando Farfán Vera, MAE Guayaquil, Ecuador. 18 de febrero del 2019. FACULTAD DE ESPECIALIDADES EMPRESARIALES CARRERA DE EMPRENDIMIENTO CERTIFICACIÓN Certificamos que el presente trabajo de titulación fue realizado en su totalidad por García Ruiz Andrés Alexander, como requerimiento para la obtención del título de Ingeniero en Desarrollo de Negocios Bilingüe. TUTOR f. ______________________ Ing. Rolando Farfán Vera, MAE. DIRECTOR DE LA CARRERA f. ______________________ CPA. Cecilia Vélez Barros, Mgs. Guayaquil, 18 de febrero del 2019 FACULTAD DE ESPECIALIDADE EMPRESARIALES CARRERA DE EMPRENDIMIENTO DECLARACIÓN DE RESPONSABILIDAD Yo, García Ruiz Andrés Alexander. DECLARO QUE: El Trabajo de Titulación, “Propuesta para la creación de una empresa productora y comercializadora de réplicas de figuras a tamaño escala en 3D en la ciudad de Santiago de Guayaquil”, previo a la obtención del título de Ingeniero en Desarrollo de Negocios Bilingüe, ha sido desarrollado respetando derechos intelectuales de terceros conforme las citas que constan en el documento, cuyas fuentes se incorporan en las referencias o bibliografías. Consecuentemente este trabajo es de mi total autoría. En virtud de esta declaración, me responsabilizo del contenido, veracidad y alcance del Trabajo de Titulación referido. Guayaquil, 18 de febrero del 2019 EL AUTOR f. ______________________________ García Ruiz Andrés Alexander. FACULTAD DE ESPECIALIDADES EMPRESARIALES CARRERA DE EMPRENDIMIENTO AUTORIZACIÓN Yo, García Ruiz Andrés Alexander.
    [Show full text]
  • Surrealist Aesthetics
    SURREALIST AESTHETICS Using Surrealist Aesthetics to Explore a Personal Visual Narrative about Air Pollution Peggy Li Auckland University of Technology Master of Design 2020 A thesis submitted to Auckland University of Technology in partial fulfilment of the requirements for the degree of Master of Design, Digital Design P a g e | 1 Abstract “Using Surrealist Aesthetics to Explore a Personal Visual Narrative about Air Pollution”, is a practice-based research project focusing on the production of a poetic short film that incorporates surrealist aesthetics with motion capture and digital simulation effects. The project explores surrealist aesthetics using visual effects combined with motion capture techniques to portray the creator’s personal experiences of air pollution within a poetic short film form. This research explicitly portrays this narrative through the filter of the filmmaker’s personal experience, deploying an autoethnographic methodological approach in the process of its creation. The primary thematic contexts situated within this research are surrealist aesthetics, personal experiences and air pollution. The approach adopted for this research was inspired by the author’s personal experiences of feeling trapped in an air-polluted environment, and converting these unpleasant memories using a range of materials, memories, imagination, and the subconscious mind to portray the negative effects of air pollution. The overall aim of this process was to express my experiences poetically using a surrealist aesthetic, applied through
    [Show full text]
  • Objekttracking Für Dynamisches Videoprojektions Mapping
    Bachelorthesis Iwer Petersen Using object tracking for dynamic video projection mapping Fakultät Technik und Informatik Faculty of Engineering and Computer Science Studiendepartment Informatik Department of Computer Science Iwer Petersen Using object tracking for dynamic video projection mapping Bachelorthesis submitted within the scope of the Bachelor examination in the degree programme Bachelor of Science Technical Computer Science at the Department of Computer Science of the Faculty of Engineering and Computer Science of the Hamburg University of Applied Science Mentoring examiner: Prof. Dr. Ing. Birgit Wendholt Second expert: Prof. Dr.-Ing. Andreas Meisel Submitted at: January 31, 2013 Iwer Petersen Title of the paper Using object tracking for dynamic video projection mapping Keywords video projection mapping, object tracking, point cloud, 3D Abstract This document presents a way to realize video projection mapping onto moving objects. Therefore a visual 3D tracking method is used to determine the position and orientation of a known object. Via a calibrated projector-camera system, the real object then is augmented with a virtual texture. Iwer Petersen Thema der Arbeit Objekttracking für dynamisches Videoprojektions Mapping Stichworte Video Projektions Mapping, Objektverfolgung, Punktwolke, 3D Kurzzusammenfassung Dieses Dokument präsentiert einen Weg um Video Projektions Mapping auf sich bewegende Objekte zu realisieren. Dafür wird ein visuelles 3D Trackingverfahren verwendet um die Position und Lage eines bekannten Objekts zu bestimmen. Über ein kalibriertes Projektor- Kamera System wird das reale Objekt dann mit einer virtuellen Textur erweitert. Contents 1 Introduction1 1.1 Motivation . .1 1.2 Structure . .3 2 Related Work4 2.1 Corresponding projects . .4 2.1.1 Spatial Augmented Reality . .4 2.1.2 2D Mapping onto people .
    [Show full text]
  • Room Layout Estimation on Mobile Devices
    En vue de l'obtention du DOCTORAT DE L'UNIVERSITÉ DE TOULOUSE Délivré par : Institut National Polytechnique de Toulouse (Toulouse INP) Discipline ou spécialité : Image, Information et Hypermédia Présentée et soutenue par : M. VINCENT ANGLADON le vendredi 27 avril 2018 Titre : Room layout estimation on mobile devices Ecole doctorale : Mathématiques, Informatique, Télécommunications de Toulouse (MITT) Unité de recherche : Institut de Recherche en Informatique de Toulouse (I.R.I.T.) Directeur(s) de Thèse : M. VINCENT CHARVILLAT M. SIMONE GASPARINI Rapporteurs : M. CARSTEN GRIWODZ, UNIVERSITE D'OSLO M. DAVID FOFI, UNIVERSITE DE BOURGOGNE Membre(s) du jury : Mme LUCE MORIN, INSA DE RENNES, Président M. PASCAL BERTOLINO, UNIVERSITE GRENOBLE ALPES, Membre M. SIMONE GASPARINI, INP TOULOUSE, Membre M. TOMISLAV PRIBANIC, UNIVERSITE DE ZAGREB, Membre M. VINCENT CHARVILLAT, INP TOULOUSE, Membre Acknowledgments I would like to thank my thesis committee for giving me the opportunity to work on an exciting topic, and for the trust they placed in me. First, Simone Gasparini, I had the pleasure to have as advisor, who kept a cautious eye on my scientific and technical productions. I am certain this great attention to the details played an important role in the quality of my publications and the absence of rejection notice. Then, my thesis director, Vincent Charvillat, who was always generous in original ideas and positive energy. His advice helped me to put a more flattering light on my works and feel more optimistic. Finally, Telequid, which funded my works, with a special thought to Frédéric Bruel and Benjamin Ahsan for their great patience. I would also like to thank my referees: Prof.
    [Show full text]