DOCSLIB.ORG

Layered Depth Images for Multi-View Coding Vincent Jantet

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Layered Depth Images for Multi-View Coding Vincent Jantet Layered Depth Images for Multi-View Coding Vincent Jantet To cite this version: Vincent Jantet. Layered Depth Images for Multi-View Coding. Multimedia [cs.MM]. Université Rennes 1, 2012. English. tel-00758301v2 HAL Id: tel-00758301 https://tel.archives-ouvertes.fr/tel-00758301v2 Submitted on 14 Feb 2013 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. ANNÉE 2012 THÈSE / UNIVERSITÉ DE RENNES 1 sous le sceau de l'Université Européenne de Bretagne pour le grade de DOCTEUR DE L'UNIVERSITÉ DE RENNES 1 Mention : Informatique École doctorale Matisse présentée par Vincent Jantet préparée à l'unité de recherche TEMICS UMR 6074 IRISA / INRIA TraitemEnt, Modélisation d'Images & CommunicationS Université de Rennes 1 Thèse soutenue à Rennes le 23 novembre 2012 devant le jury composé de : Layered Depth Images Béatrice PESQUET Professeur, Télécom ParisTech / rapporteur for Multi-View Coding Ferran MARQUÉS Professeur, Université Polytechnique de Catalogne, Espagne / rapporteur Edmond BOYER Directeur de Recherche, INRIA Grenoble / examina- teur Aljoscha SMOLIC Ingénieur de Recherche, Disney Research, Zurich, Suisse / examinateur Christine GUILLEMOT Directeur de Recherche, INRIA Rennes / directrice de thèse Luce MORIN Professeur, INSA-Rennes / co-directrice de thèse 2 Contents Glossary 7 Résumé en français: Compression multi-vues par représentation LDI 9 1 Construction des LDI . 10 1.1 Équations de projection . 11 1.2 Construction naïve des LDI . 11 1.3 Construction incrémentale des LDI . 13 1.4 Segmentation par couche-object . 14 2 Compression des LDI . 15 2.1 Codeur MVC . 16 2.2 Codage des LDI par MVC . 16 3 Synthèse de vues . 16 3.1 Projection Ordonnée JPF (pour Joint Projection Filling)......... 17 3.2 Prise en compte de la profondeur lors du remplissage . 20 3.3 Qualité du rendu . 20 4 Résultats . 22 5 Conclusions . 23 1 General introduction 26 1.1 Historical context . 26 1.2 Present challenges . 28 1.3 Objectives . 32 1.4 Contributions . 33 1.5 Scope of the thesis . 34 2 3D content representations and compression format: a state-of-the-art 36 2.1 3D video sequences representation . 36 2.1.1 Rendering-optimized representations . 37 Multi-video with additional geometry . 37 The plenoptic function . 37 Microfacet billboarding . 38 Conclusion . 39 2.1.2 Transmission-optimized representations . 39 3 Image-based representations . 39 Model-based representations . 42 2.2 3D video coding . 43 2.2.1 Introduction . 43 2.2.2 Concepts of image and video compression . 44 Image prediction . 44 Distortion measures . 45 2D Video compression . 46 2.2.3 Multi-view video compression . 48 Simulcast Coding scheme . 49 Multi-View Coding (MVC) . 50 Multi-View plus Depth coding (MVD) . 52 I Virtual view synthesis 55 3 Background work on DIBR: State-of-the-art 57 3.1 Perspectives on projective geometry . 58 3.1.1 Homogeneous coordinates . 58 3.1.2 Perspective projection . 59 From R0 to RC ................................. 59 From RC to image plane . 60 From image plane to pixel coordinate system IC .............. 61 Summary . 61 3.1.3 Reverse equations . 62 3.1.4 Optimization . 62 3.2 Forward projection . 63 3.3 Backward projection for cracks handling . 64 3.4 Ghosting artifacts removal through boundaries detection . 65 3.5 Disocclusion filling . 65 3.5.1 Image inpainting . 66 Exemplar-based inpainting approach . 67 Inpainting methods for disocclusions filling . 68 Depth-aided inpainting method . 70 3.6 Conclusion . 71 4 Joint Projection Filling method for new DIBR 73 4.1 Joint Projection Filling method for disocclusion handling . 74 4.1.1 Disocclusion detection . 74 4.1.2 Disocclusion filling . 77 4.1.3 Results . 78 4.2 View extrapolation with full-Z depth-aided inpainting . 79 4.2.1 Full-Z depth-aided inpainting method . 81 4.2.2 Inpainting Results . 81 4 4.3 Intermediate view interpolation with Floating Texture . 85 4.4 Conclusion . 87 II Layered Depth Image (LDI) for compact 3D representation 88 5 Background work on LDI: State-of-the-art 90 5.1 Classical LDI construction . 91 5.1.1 LDI from 3D models . 91 5.1.2 LDI from real images . 92 5.1.3 LDI from multiple video plus depth sequences . 92 5.2 LDI compression schemes . 93 5.2.1 Aggregation . 94 5.2.2 Filling . 95 5.3 Conclusion . 95 6 Layer Depth Image representation 97 6.1 Compression of LDI sequences with AVC/MVC . 97 6.1.1 LDI coding with AVC/MVC . 98 6.1.2 Experimental results . 98 6.2 Incremental-LDI (I-LDI) . 100 6.2.1 Incremental scheme for LDI construction . 101 6.2.2 Experimental results . 103 6.2.3 Conclusion . 105 6.3 Object-based LDI . 106 6.3.1 Object-based classification . 107 Classification with global threshold . 108 Classification by region growing . 108 6.3.2 Background filling by inpainting . 110 6.3.3 Results . 110 6.3.4 Conclusion . 112 6.4 LDI Rendering . 112 6.4.1 Mesh-based LDI rendering . 113 6.4.2 JPF-based view synthesis . 114 6.5 Conclusion . 115 7 Conclusion and perspectives 117 7.1 Summary of contributions . 117 7.1.1 Virtual view synthesis . 117 Joint Projection Filling method . 118 Pixel confidence measure for ghosting removal . 118 Full-Z depth-aided inpainting . 118 Texture registration with texture misalignments handling . 119 Conclusion . 119 5 7.1.2 LDI processing . 119 LDI compression . 120 Incremental LDI . ..
Load more

Recommended publications
  • Lenticular Printing - Wikipedia, the Free Encyclopedia
    Lenticular printing - Wikipedia, the free encyclopedia http://en.wikipedia.org/wiki/Lenticular_printing Lenticular printing From Wikipedia, the free encyclopedia Lenticular printing is a technology in which a lenticular lens is used to produce images with an illusion of depth, or the ability to change or move as the image is viewed from different angles. Examples of lenticular printing include prizes given in Cracker Jack snack boxes that showed flip and animation effects such as winking eyes, and modern airport advertising graphics that change their message depending on the viewing angle. This technology was created in the 1940s but has evolved in recent years to show more motion and increased depth. Originally used mostly in novelty items, lenticular prints are now being used as a marketing tool to show products in motion. Recent advances in large- format presses have allowed for oversized lenses to be used in lithographic lenticular printing.[1] Contents 1 Process 2 How it works 3 Types of lenticular prints 4 History of lenticular image technology 5 Manufacturing process 5.1 Printing 5.1.1 Ink composition 5.1.1.1 Pigments 5.1.1.2 Prepolymers 5.1.1.3 Photoinitiators 5.1.1.4 Monomers (diluents) 5.1.1.5 Additives and inhibitors 5.1.2 Water wetting 5.1.3 UV curing 5.2 Defects 5.2.1 Design defects 5.2.2 Prepress defects 5.2.3 Printing defects 5.2.4 Cutting defects 6 See also 7 Notes and references 8 External links Process 1 of 10 12/28/08 6:43 PM Lenticular printing - Wikipedia, the free encyclopedia http://en.wikipedia.org/wiki/Lenticular_printing Lenticular printing is a multi-step process consisting of creating a lenticular image from at least two existing images, and combining it with a lenticular lens.
    [Show full text]
  • Optimal Sub-Pixel Arrangements and Coding for Ultra-High Resolution Three-Dimensional OLED Displays
    Optimal Sub-Pixel arrangements and coding for ultra-high resolution three-dimensional OLED displays by Zhivko Yordanov A doctoral dissertation submitted as partial fulfilment for the award of the academic degree of Doctor of Engineering Sciences (Dr.-Ing.) Faculty of Electrical and Computer Engineering of the University Kassel 2007 Supervisors: 1. Prof. Dr.-Ing. Siegbert Hentschke 2. Prof. Dr. Ivo W. Rangelow 3. Prof. Assen Nedev Examination Board: 1. Prof. Dr.-Ing. Siegbert Hentschke 2. Prof. Dr. Ivo W. Rangelow 3. Prof. Dr.-Ing. Josef Börcsök 4. Prof. Dr.-Ing. Wolf-Jürgen Becker Day of the Oral Examination: 27 July 2007 To my wife and my daughter, to whom I promised this a few years ago. They fully supported me to conclude it during weekends and holidays. I promise I will not do anything like this again! To my parents, who would have liked to see it finished. 6 Table of Contents 1. INTRODUCTION ....................................................................................................................... 9 1.1 GENESIS............................................................................................................................. 9 1.2 AIMS AND OBJECTIVES ...................................................................................................... 11 1.3 SUMMARIES AND LAYOUT OF THESIS .................................................................................. 13 1.4 SUMMARY ......................................................................................................................... 13 2.
    [Show full text]
  • First Novel Invention of In- Line Web Fed Roll Print
    FIRST NOVEL INVENTION OF IN- LINE WEB FED ROLL PRINT MANUFACTURING PRODUCTION OF ANIMATED / THREE DIMENSIONAL IMAGED PRINT PRODUCTS INCORPORATING ADVANCED LENTICULAR TRANSPARENT SUBSTRATE…ITS ADVANTAGES AND THE COMPARISON / CONTRAST ORDER ANALYSIS TO PRIOR U.S.P.T.O. PATENTED ART A Dissertation Presented to the Faculty of the School of Engineering Kennedy-Western University In Partial Fulfillment Of the Requirements for the Degree of Doctor of Philosophy in Engineering Management By Gary A. Jacobsen Itasca, Illinois USA 2004 Abstract of Dissertation FIRST NOVEL INVENTION OF IN- LINE WEB FED ROLL PRINT MANUFACTURING PRODUCTION OF ANIMATED / THREE DIMENSIONAL IMAGED PRINT PRODUCTS INCORPORATING ADVANCED LENTICULAR TRANSPARENT SUBSTRATE…ITS ADVANTAGES AND THE COMPARISON / CONTRAST ORDER ANALYSIS TO PRIOR U.S.P.T.O. PATENTED ART By Gary A. Jacobsen Itasca, IL USA 2004 Kennedy-Western University Cheyenne, Wyoming USA ii THE PROBLEM Prior to the lenticular printing industries first of its kind invention of Jacobsen Patents # 5,560,799, (1996) and # 5,753,344, (1998) granted within the United States Patent and Trademark Office (USPTO), there were only seven other (USPTO) categories of prior art records indicating: 1) Lenticular prepatory (prepress) interlacing 2) Inefficient multi-step lenticular (printing processes), 3) Related consumer products that have been developed to fabricate (photographically) created lenticular images, 4) Sequentially stepped non-lenticular (printed layers of transparent material) to simulate dimension and distance of printed images, 5) (Multiple step process) single roll lenticular web fed offset printing requiring numerous and separate multiple off-line post production finishing procedures, iii 6) (Sheet fed lenticular printed products) manufactured at one sheet at a time that display a virtual illustrative printed image which appears to be three dimensional (3-D) or animated (moving sequential images) when viewed by the naked human eye.
    [Show full text]
  • Glassless 3D Technology a 3D Solution Without Glasses - the Future of 3D Technology
    Glassless 3D Technology A 3D Solution without glasses - the future of 3D Technology Mr. M. Venkatesan Founder / CEO, SAI3D (a unit of SAI Media Productions) India–Canada [email protected] Today it's hard to find a person who's never heard about 3D technology. With the popularity of 3D tablets, 3D TV sets, 3D laptops and 3D smartphones on the rise it's no wonder that more and more people are getting interested in the nature of three dimensional technology. 3D vision has started with the invention of stereoscopic 3D concept and has gradually evolved to many modern methods of 3D technology. So if you can tell the difference between anaglyph red cyan, polarized and shutter 3D glasses, you definitely have a clear perception of what 3D is and how it works . Most of us associate 3D technology with the use of special glasses or virtual reality headgear to filter what image each of our eyes sees. But what do we know about 3D without glasses? Is it just a futuristic concept or already a reality? How 3D without glasses works? The technology of 3D without glasses is called auto-stereoscopy. Because this kind of three dimensional technology doesn't utilize special spectacles or headgear it became also known as glasses-less 3D or glasses free 3D. Auto-stereoscopy is a method of displaying stereoscopic images (adding binocular perception of 3D depth) without the use of special headgear or glasses on the part of the viewer. Because headgear is not required, it is also called "glasses-free 3D" or "glasses less 3D".
    [Show full text]
  • Télécom Bretagne
    N° d'ordre : 2016telb0389 Sous le sceau de l' Université Bretagne Loire Télécom Bretagne En accréditation conjointe avec l'Ecole Doctorale Sicma Impact of 3D technologies and environment stimuli on stereo vision perception Thèse de Doctorat Mention : Sciences et Technologies Information Communication Présentée par Di ZHANG Département : Optique Directeur de thèse : Jean-Louis de Bougrenet de la Tocnaye Soutenue le 13 janvier 2016 Jury : M. Pascal Mamassian, Directeur de Recherche, Ecole Normale Supérieure (Rapporteur) Mme Emma Gowen, Senior Lecturer, Université de Manchester (Rapporteur) M. Jean-Louis de Bougrenet de la Tocnaye, Professeur, Télécom Bretagne (Directeur de thèse) Mme Béatrice Cochener, Professeur, CHRU Brest (Examinateur) M Xinzhu Sang, Professeur, BUPT Pékin, (Examinateur) M. Christian Roux, Professeur, Institut Mines Télécom (Examinateur) 1 Acknowledgements I have been thinking about the acknowledgements since I just started to write the dissertation. The PhD thesis is not only a research project, it is also my three years life. Every chapter I have written, there are stories and experience in it, with the people who were working with me or who had offered their disinterested help to me. My first and the deepest gratitude belongs to my supervisor Professor Jean-Louis De Bougrenet De La Tocnaye, for the continuous guidance and support of my study and research, for his patience, encouragement, trust and immense knowledge. The characteristic I appreciate the most is his inspiration, it has been always a great pleasure to discuss with him, each time I feel better understanding for my research field and gain more confidence. My sincere thanks also goes to my team members, specifically for Miss Yulia Fattakhova, Dr.
    [Show full text]
  • Photography Techniques Advanced Skills
    Photography Techniques Advanced Skills PDF generated using the open source mwlib toolkit. See http://code.pediapress.com/ for more information. PDF generated at: Wed, 21 Aug 2013 16:46:53 UTC Contents Articles Zone System 1 High-dynamic-range imaging 8 Contre-jour 18 Night photography 20 Multiple exposure 25 Camera obscura 28 Pinhole camera 33 Stereoscopy 38 References Article Sources and Contributors 60 Image Sources, Licenses and Contributors 62 Article Licenses License 65 Zone System 1 Zone System The Zone System is a photographic technique for determining optimal film exposure and development, formulated by Ansel Adams and Fred Archer.[1] Adams described how the Zone System as "[...] not an invention of mine; it is a codification of the principles of sensitometry, worked out by Fred Archer and myself at the Art Center School in Los Angeles, around 1939-40."[2] The technique is based on the late 19th century sensitometry studies of Hurter and Driffield. The Zone System provides photographers with a systematic method of precisely defining the relationship between the way they visualize the photographic subject and the final results. Although it originated with black-and-white sheet film, the Zone System is also applicable to roll film, both black-and-white and color, negative and reversal, and to digital photography. Principles Visualization An expressive image involves the arrangement and rendering of various scene elements according to photographer’s desire. Achieving the desired image involves image management (placement of the camera, choice of lens, and possibly the use of camera movements) and control of image values. The Zone System is concerned with control of image values, ensuring that light and dark values are rendered as desired.
    [Show full text]
  • 3D Lenticular Imaging for Art
    Proceedings of Bridges 2015: Mathematics, Music, Art, Architecture, Culture 3D Lenticular Imaging for Art Yitzhak Weissman Pop3DArt P. O. Box 12767 Herzliya 4673324, Israel Email: [email protected] Abstract 3D lenticular printing technology can be used to display three-dimensional art objects on a flat medium. Such pictures can be viewed naturally without any aids or impediments, and be easily duplicated. This makes them a very attractive alternative to making physical sculptures in displaying three-dimensional art objects. In principle, any object, real or virtual, can be displayed in a 3D lenticular picture. This paper outlines how artwork can be submitted for lenticular print and describes the main characteristics of 3D lenticular depth imaging. Introduction Practitioners of two-dimensional (2D) visual arts have many readily available and inexpensive options to present their art. This is not the case with practitioners of three-dimensional (3D) visual arts. Traditionally, art objects with 3D presence are displayed as sculptures. To make a sculpture one needs a studio, tools, and materials. The sculpture itself may be heavy and bulky; it requires a big storage space, and cannot be hung on a wall. Nowadays, there is an option to create sculptures with a 3D printing machine. However, such sculptures are limited in size, and suffer from inadequate surface finish and coloring. 3D lenticular technology offers another alternative, which is particularly attractive for artistic applications. It uses a flat medium which can be easily stored, shipped, and hung on a wall. The artistic part of 3D lenticular picture making requires only a PC and suitable software.
    [Show full text]