DOCSLIB.ORG

Download File

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Download File Improvements in the robustness and accuracy of bioluminescence tomographic reconstructions of distributed sources within small animals Bradley J. Beattie Submitted in partial fulfillment of the requirements for the degree of Doctor of Engineering Science in the Fu Foundation School of Engineering and Applied Science COLUMBIA UNIVERSITY 2018 © 2018 Bradley J. Beattie All rights reserved ABSTRACT Improvements in the robustness and accuracy of bioluminescence tomographic reconstructions of distributed sources within small animals Bradley J. Beattie High quality three-dimensional bioluminescence tomographic (BLT) images, if available, would constitute a major advance and provide much more useful information than the two-dimensional bioluminescence images that are frequently used today. To-date, high quality BLT images have not been available, largely because of the poor quality of the data being input into the reconstruction process. Many significant confounds are not routinely corrected for and the noise in this data is unnecessarily large and poorly distributed. Moreover, many of the design choices affecting image quality are not well considered, including choices regarding the number and type of filters used when making multispectral measurements and choices regarding the frequency and uniformity of the sampling of both the range and domain of the BLT inverse problem. Finally, progress in BLT image quality is difficult to gauge owing to a lack of realistic gold-standard references that engage the full complexity and uncertainty within a small animal BLT imaging experiment. Within this dissertation, I address all of these issues. I develop a Cerenkov-based gold- standard wherein a Positron Emission Tomography (PET) image can be used to gauge improvements in the accuracy of BLT reconstruction algorithms. In the process of creating this reference, I discover and describe corrections for several confounds that if left uncorrected would introduce artifacts into the BLT images. This includes corrections for the angle of the animal’s skin surface relative to the camera, for the height of each point on the skin surface relative to the focal plane, and for the variation in bioluminescence intensity as a function of luciferin concentration over time. Once applied, I go on to derive equations and algorithms that when employed are able to minimize the noise in the final images under the constraints of a multispectral BLT data acquisition. These equations and algorithms allow for an optimal choice of filters to be made and for the acquisition time to be optimally distributed among those filtered measurements. These optimizations make use of Barrett’s and Moore-Penrose pseudoinverse matrices which also come into play in a paradigm I describe that can be used to guide choices regarding sampling of the domain and range. Table of Contents List of Figures v List of Tables vi 1. INTRODUCTION 1 1.1. Background and motivation 1 1.2. The BLT inverse problem 5 1.3. Multispectral vs. hyperspectral 6 1.4. Overall Goals and Specific Aims 12 2. SPECIFIC AIM 1: MULTIMODALITY REGISTRATION 14 2.1. Overview 14 2.2. Scanners. 18 2.3. Overview of the registration procedure. 18 2.4. Registration of a 2D image to a 3D image set. 21 2.5. BLI camera model. 22 2.6. BLI camera calibration. 23 i 2.7. Registration accuracy. 24 2.8. Animals and imaging procedures. 26 2.9. Imaging protocols. 28 2.10. Registration accuracy. 29 2.11. Experiments 30 2.12. Light Fall-off Correction. 34 2.13. Discussion 36 3. SPECIFIC AIM 2: CERENKOV QUANTITATION 38 3.1. Overview 38 3.2. Modeling. 41 3.3. Experimental Cerenkov measurements. 49 3.4. Spectral measurements. 52 3.5. Cerenkov efficiency as a function of refractive index. 53 3.6. Cerenkov efficiency as a function of volume. 53 3.7. Region of interest measurements and profiles. 54 ii 3.8. Corrections 55 3.9. Comparisons 60 3.10. Modeled Cerenkov production efficiencies as a function of refractive index. 67 3.11. Modeled Cerenkov point spread functions. 69 3.12. Discussion 69 4. SPECIFIC AIM 3: OPTIMIZED ACQUISITION PROTOCOL 73 4.1. Overview 73 4.2. Camera noise model 75 4.3. Digital mouse phantom 76 4.4. Solving the forward model 77 4.5 Image Reconstruction Methods 80 4.6. Derivation of the optimization expression 84 4.7. Algorithm to select optimal filters 90 4.8. Testing the optimization expression 92 4.9. Two-wavelength LSQ simulation results 94 iii 4.10. Three-wavelength LSQ simulation 97 4.11. Two-wavelength MLEM simulation 98 4.12. Mouse simulations 99 4.13. Guidance for improved conditioning and SNR through optimized sampling 103 4.14. Discussion 107 5. OVERALL SUMMARY AND CONCLUSIONS 109 REFERENCES 112 APPENDIX 121 iv List of Figures 1.1 Artifacts in BLT ..................................................................................................... 1 2.1 Mouse Bed ......................................................................................................... 19 2.2 Transgenic Mouse ............................................................................................. 26 2.3 Light Fall-off with Angle in Animal ...................................................................... 31 2.4 Registered CT, Reflectance and BLI Images .................................................... 32 2.5 Light Flux versus Time Curve ............................................................................ 33 2.6 Parameter Defining Diagram ............................................................................. 34 2.7 Light Fall-off with Angle in Phantom .................................................................. 35 3.1 Cerenkov versus Wavelength, Radioactivity and Height ................................... 56 3.2 Experimental Setup and Radionuclide Cerenkov Results ................................. 61 3.3 Cerenkov Point Spread Function ....................................................................... 65 3.4 Secondary Electron Point Spread Function ....................................................... 66 3.5 Cerenkov versus Refractive Index ..................................................................... 68 4.1 Pathlength Histogram ........................................................................................ 78 4.2 Log-mean and Log-standard-deviation of Pathlength Distribution ..................... 79 4.3 Importance of Politte’s Correction ...................................................................... 84 4.4 Optimal Time Distribution for 2-Wavelength Moore-Penrose Solution ................ 95 4.5 Uncertainty for 2-Wavelength Moore-Penrose Solution ..................................... 96 4.6 Optimal Time Distribution for 3-Wavelengths Moore-Penrose Solution ............. 97 4.7 Uncertainty for 2-Wavelength MLEM Solution ................................................... 98 4.8 Optimal Time Distribution for 2-Wavelength MLEM Solution ............................. 99 v 4.9 Reference Image, Source Spectra and Attenuation ........................................ 100 4.10 Firefly and Cerenkov Optimal Time Distributions ............................................. 101 4.11 Improved MLEM Images from Optimally Sampled Data .................................. 102 4.12 Improved Moore-Penrose Images from Optimally Sampled Data .................... 103 4.13 Optimized Domain Sampling ........................................................................... 106 List of Tables 2.1 Registration Error .............................................................................................. 30 3.1 Radionuclide Abundances ................................................................................. 43 3.2 Ac-225 Daughter Abundances ........................................................................... 44 3.3 Experiment Overview ........................................................................................ 50 3.4 Cerenkov Efficiencies ........................................................................................ 68 3.5 Cerenkov Point Spread Function Widths ........................................................... 69 vi 1. INTRODUCTION 1.1. Background and motivation Bioluminescence tomography (BLT) reconstruction algorithms have been available to researchers since Wang first described their theoretical basis in 2004 [1]. Today, BLT reconstruction capabilities are included as standard software accompanying the ubiquitous IVIS bioluminescence imaging systems (Perkin Elmer). However, these algorithms have seen relatively little use, as evidenced in the literature by the relative dearth of articles published that actually make use of BLT. BLT has not been well embraced, I believe, because of the generally poor quality of the BLT reconstructed images (suffering from clear artifacts, see figure 1.1, and because they are often noisy and unstable) and because the accuracy of these images has not been validated under realistic conditions (i.e. researchers don’t trust them to be accurate). It is well recognized that the BLT image reconstruction problem is ill-posed, requiring multispectral measurements to avoid degeneracy and often remains ill-conditioned even when multispectral measurements are made [2-4]. Thus it is typical for even small Figure 1.1 Although the bioluminescent amounts of noise in the measured data to be tumor implanted in the brain of this mouse is roughly spherical, the source distribution greatly amplified during the reconstruction shown here, reconstructed by Chaudhari et al. clearly is not. 1 process, producing images with a large degree
Load more

Recommended publications
  • Depth Assisted Composition of Synthetic and Real 3D Scenes
    ©2016 Society for Imaging Science and Technology DOI: 10.2352/ISSN.2470-1173.2016.21.3DIPM-399 Depth Assisted Composition of Synthetic and Real 3D Scenes Santiago Cortes, Olli Suominen, Atanas Gotchev; Department of Signal Processing; Tampere University of Technology; Tampere, Finland Abstract In media production, previsualization is an important step. It Prior work allows the director and the production crew to see an estimate of An extensive description and formalization of MR systems the final product during the filmmaking process. This work focuses can be found in [1]. After the release of the ARToolkit [2] by the in a previsualization system for composite shots, which involves Nara Institute of Technology, the research community got much real and virtual content. The system visualizes a correct better access to real time blend of real and virtual data. Most of the perspective view of how the real objects in front of the camera AR applications running on web browsers produced during the operator look placed in a virtual space. The aim is to simplify the early 2000s were developed using the ARToolkit. MR has evolved workflow, reduce production time and allow more direct control of with the technology used to produce it, and the uses for it have the end result. The real scene is shot with a time-of-flight depth multiplied. Nowadays, in the embedded system era, a single device camera, whose pose is tracked using a motion capture system. can have most of, if not all, the sensors and computing power to Depth-based segmentation is applied to remove the background run a good MR application.
    [Show full text]
  • (12) United States Patent (10) Patent No.: US 9,729,765 B2 Balakrishnan Et Al
    USOO9729765B2 (12) United States Patent (10) Patent No.: US 9,729,765 B2 Balakrishnan et al. (45) Date of Patent: Aug. 8, 2017 (54) MOBILE VIRTUAL CINEMATOGRAPHY A63F 13/70 (2014.09); G06T 15/20 (2013.01); SYSTEM H04N 5/44504 (2013.01); A63F2300/1093 (71) Applicant: Drexel University, Philadelphia, PA (2013.01) (58) Field of Classification Search (US) None (72) Inventors: Girish Balakrishnan, Santa Monica, See application file for complete search history. CA (US); Paul Diefenbach, Collingswood, NJ (US) (56) References Cited (73) Assignee: Drexel University, Philadelphia, PA (US) PUBLICATIONS (*) Notice: Subject to any disclaimer, the term of this Lino, C. et al. (2011) The Director's Lens: An Intelligent Assistant patent is extended or adjusted under 35 for Virtual Cinematography. ACM Multimedia, ACM 978-1-4503 U.S.C. 154(b) by 651 days. 0616-Apr. 11, 2011. Elson, D.K. and Riedl, M.O (2007) A Lightweight Intelligent (21) Appl. No.: 14/309,833 Virtual Cinematography System for Machinima Production. Asso ciation for the Advancement of Artificial Intelligence. Available (22) Filed: Jun. 19, 2014 from www.aaai.org. (Continued) (65) Prior Publication Data US 2014/O378222 A1 Dec. 25, 2014 Primary Examiner — Maurice L. McDowell, Jr. (74) Attorney, Agent, or Firm — Saul Ewing LLP: Related U.S. Application Data Kathryn Doyle; Brian R. Landry (60) Provisional application No. 61/836,829, filed on Jun. 19, 2013. (57) ABSTRACT A virtual cinematography system (SmartWCS) is disclosed, (51) Int. Cl. including a mobile tablet device, wherein the mobile tablet H04N 5/222 (2006.01) device includes a touch-sensor Screen, a first hand control, a A63F 3/65 (2014.01) second hand control, and a motion sensor.
    [Show full text]
  • A Cost Effective, Accurate Virtual Camera System for Games, Media Production and Interactive Visualisation Using Game Motion Controllers
    EG UK Theory and Practice of Computer Graphics (2013) Silvester Czanner and Wen Tang (Editors) A Cost Effective, Accurate Virtual Camera System for Games, Media Production and Interactive Visualisation Using Game Motion Controllers Matthew Bett1, Erin Michno2 and Dr Keneth B. McAlpine1 1University of Abertay Dundee 2Quartic Llama ltd, Dundee Abstract Virtual cameras and virtual production techniques are an indispensable tool in blockbuster filmmaking but due to their integration into commercial motion-capture solutions, they are currently out-of-reach to low-budget and amateur users. We examine the potential of a low budget high-accuracy solution to create a simple motion capture system using controller hardware designed for video games. With this as a basis, a functional virtual camera system was developed which has proven usable and robust for commercial testing. Categories and Subject Descriptors (according to ACM CCS): I.3.6 [Computer Graphics]: Methodology and Techniques—Interaction Techniques I.3.7 [Information Interfaces and Presentations]: Three-Dimensional Graphics and Realism —Virtual reality 1. Introduction sult is a loss of truly ’organic’ camera shots being created with ease in the production process. In recent years, advances in computer graphics, motion cap- Virtual camera systems allow the user to apply conven- ture hardware and a need to give greater directorial control tional camera-craft within the CGI filmmaking process. A over the digital filmmaking process have given rise to the physical device similar to a conventional camera is used by world of virtual production as a tool in film and media pro- the camera operator with a monitor acting as the view-finder duction [AUT09].
    [Show full text]
  • NOAO/NSO Newsletter Issue 86 June 2006
    NOAO/NSO Newsletter Issue 86 June 2006 Science Highlights Cerro Tololo Inter-American Observatory Using the ‘NOAO System’ of Small and Large Telescopes Conociendo al Pueblo Atacameño ............................................26 to Investigate Young Brown Dwarfs.......................................3 The El Peñón DIMM ...................................................................28 The First Complete Solar Cycle of GONG Observations SOAR Telescope Update .............................................................28 of Solar Convection Zone Dynamics .....................................5 Robert Blum Moves from NOAO South to North ................ 29 Discovery of the First Radio-Loud Quasar at z > 6 ..................6 Reversed Granulation and Gravity Waves Kitt Peak National Observatory in the Mid-Photosphere............................................................8 The Tohono O’odham Nation, the NSF, VERITAS, and The Volume-Averaged Properties Kitt Peak National Observatory ............................................30 of Luminous Galaxies at z < 3....................................................9 The WIYN One-Degree Imager and its Precursor QUOTA ..................................................................32 Director’s Office Thank You, Judy Prosser, for 18 Years of Service! ..................34 TMT Project Passes Conceptual Design Review ....................12 Taft Armandroff moves from NGSC Director National Solar Observatory/GONG to Keck Director .....................................................................
    [Show full text]
  • An Extensible System for Physically-Based Virtual Camera Control Using Rotational Motion Capture
    AN EXTENSIBLE SYSTEM FOR PHYSICALLY-BASED VIRTUAL CAMERA CONTROL USING ROTATIONAL MOTION CAPTURE A Thesis by ROBERT SHELBY HUEBEL Submitted to the Office of Graduate and Professional Studies of Texas A&M University in partial fulfillment of the requirements for the degree of MASTER OF SCIENCE Chair of Committee, Philip Galanter Committee Members, Tim McLaughlin John Keyser Head of Department, Tim McLaughlin May 2014 Major Subject: Visualization Copyright 2014 Robert Shelby Huebel ABSTRACT An important characteristic of any well-designed system is user interface design. Even in extremely complex systems with high learning curves, a designer's goal is to create an interface that is simple to use and that feels natural, while at the same time allowing the user full access to all of the system's functionality. In the field of visual effects and computer animation, effective manipulation of the virtual camera is essential to any successful production. In most cases, the virtual camera is manipulated with standard input devices such as the mouse and keyboard. This thesis presents an alternative user interface design for virtual cameras using physical controls. This thesis describes an extensible system which offers physically-based, real-time virtual camera control in a commercial 3D software package by means of rotational motion capture. The system described is composed of both programmed physical hardware and a software application. The physical hardware consists of multiple analog sensors programmed to work in conjunction to detect rotation about the de- vice's three axes. The software application interprets the data sent from the physical hardware into a form a 3D software package can use to manipulate a virtual cam- era, while also providing additional functionality for the motion capture and camera manipulation processes.
    [Show full text]
  • Eye Space: an Analytical Framework for the Screen-Mediated Relationship in Video Games
    Art and Design Review, 2017, 5, 84-101 http://www.scirp.org/journal/adr ISSN Online: 2332-2004 ISSN Print: 2332-1997 Eye Space: An Analytical Framework for the Screen-Mediated Relationship in Video Games Yu-Ching Chang, Chi-Min Hsieh Institute of Applied Arts, National Chiao Tung University, Taiwan How to cite this paper: Chang, Y.-C., & Abstract Hsieh, C.-M. (2017). Eye Space: An Ana- lytical Framework for the Screen-Mediated This article explores the connections between players and game worlds Relationship in Video Games. Art and through the screen-mediated space in games—i.e., eye space. Eye space is the Design Review, 5, 84-101. crucial link between players and the game world; it’s the decisive area where https://doi.org/10.4236/adr.2017.51007 the gameplay takes place. However, the concept of eye space is not attention Received: January 14, 2017 to and is frequently confused with game space because the two are closely Accepted: February 25, 2017 connected and sometimes they can even interchange with each other. Thus, Published: February 28, 2017 the study focuses on the basic building blocks and the structure of eye space in Copyright © 2017 by authors and games. An analytical framework based on the existing literature and practice Scientific Research Publishing Inc. is proposed with special attention on the interactive nature of video games in This work is licensed under the Creative order to examine the interplay between players and game spaces. The frame- Commons Attribution International work encompasses three aspects: the visual elements within the eye space, the License (CC BY 4.0).
    [Show full text]
  • Technology Title: Smart Virtual Camera System Tech ID: 13-1563D Technology Summary: Applications: Advantage(S): IP Status: Pate
    Technology Title: Smart Virtual Camera System Tech ID: 13-1563D Technology Summary: In the current video production environment, the ability to previsualize shots utilizing a virtual camera system requires expensive hardware and large motion capture spaces making this technology accessible to a limited audience. To overcome these limitations, a student/faculty Drexel team has developed an innovative low-cost alternative called the Smart VCS that offers the potential to make virtual cinematograhy accessible to the consumer market while offering an array of novel features. The Smart VCS is able to achieve both advantages by integrating consumer technologies such a multi-touch tablet (iPad) and video game motion controller (PS3 Move) with openly accessible game engines. The Smart VCS system is designed for directors, both amateur and professional, who wish to embrace the notion of Virtual Production for films and game cinematics without a big studio budget. The Smart VCS enables a director to compose and record camera motions in freespace and manipulate scene elements, such as characters & environments, through a real-time intuitive touch interface that is guided by system intelligence and based on cinematic principles. Applications: Virtual Cinematography Game level design Real-time compositing & post-production Architectural visualization Advantage(s): Lower equipment costs Lower production costs – supports iterative production pipeline allowing directors and cinematographers to experiment with their shot compositions throughout the production process instead of simply at the previsualization stage Tablet (iPad) integration - provides opportunity to build intelligence into the tool that may open up a new market of amateur content makers while also enabling higher quality content Smaller/Lighter Form Factor IP Status: Patent Pending Inventors: Girish Balakrishnan, Paul Diefenbach Relevant Publications: SIGGRAPH 2013 Conference (SIGGRAPH Talk Site) Multi-media Links: Video Demonstration Production Blog Unity Awards 2013 .
    [Show full text]
  • Computer Generation of Integral Images Using Interpolative Shading Techniques
    Computer Generation of Integral Images using Interpolative Shading Techniques Graham E. Milnthorpe A thesis submitted for the degree of Doctor of Philosophy School of Engineering and Manufacture, De Montfort University, Leicester December 2003 IMAGING SERVICES NORTH Boston Spa, Wetherby West Yorkshire, LS23 7BQ www.bl.uk The following has been excluded at the request of the university Pages 151 -164 Summary i Research to produce artificial 3D images that duplicates the human stereovision has been ongoing for hundreds of years. What has taken millions of years to evolve in humans is proving elusive even for present day technological advancements. The difficulties are compounded when real-time generation is contemplated. The problem is one of depth. When perceiving the world around us it has been shown that the sense of depth is the result of many different factors. These can be described as monocular and binocular. Monocular depth cues include overlapping or occlusion, shading and shadows, texture etc. Another monocular cue is accommodation (and binocular to some extent) where the focal length of the crystalline lens is adjusted to view an image. The important binocular cues are convergence and parallax. Convergence allows the observer to judge distance by the difference in angle between the viewing axes of left and right eyes when both are focussing on a point. Parallax relates to the fact that each eye sees a slightly shifted view of the image. If a system can be produced that requires the observer to use all of these cues, as when viewing the real world, then the transition to and from viewing a 3D display will be seamless.
    [Show full text]
  • 3GPP TR 26.928 V. 1.0.0
    ETSI TR 126 928 V16.0.0 (2020-11) TECHNICAL REPORT 5G; Extended Reality (XR) in 5G (3GPP TR 26.928 version 16.0.0 Release 16) 3GPP TR 26.928 version 16.0.0 Release 16 1 ETSI TR 126 928 V16.0.0 (2020-11) Reference DTR/TSGS-0426928vg00 Keywords 5G ETSI 650 Route des Lucioles F-06921 Sophia Antipolis Cedex - FRANCE Tel.: +33 4 92 94 42 00 Fax: +33 4 93 65 47 16 Siret N° 348 623 562 00017 - NAF 742 C Association à but non lucratif enregistrée à la Sous-Préfecture de Grasse (06) N° 7803/88 Important notice The present document can be downloaded from: http://www.etsi.org/standards-search The present document may be made available in electronic versions and/or in print. The content of any electronic and/or print versions of the present document shall not be modified without the prior written authorization of ETSI. In case of any existing or perceived difference in contents between such versions and/or in print, the prevailing version of an ETSI deliverable is the one made publicly available in PDF format at www.etsi.org/deliver. Users of the present document should be aware that the document may be subject to revision or change of status. Information on the current status of this and other ETSI documents is available at https://portal.etsi.org/TB/ETSIDeliverableStatus.aspx If you find errors in the present document, please send your comment to one of the following services: https://portal.etsi.org/People/CommiteeSupportStaff.aspx Copyright Notification No part may be reproduced or utilized in any form or by any means, electronic or mechanical, including photocopying and microfilm except as authorized by written permission of ETSI.
    [Show full text]
  • Previsualization in Computer Animated Filmmaking THESIS
    Previsualization in Computer Animated Filmmaking THESIS Presented in Partial Fulfillment of the Requirements for the Degree Master of Fine Arts in the Graduate School of The Ohio State University By Nicole Lemon Graduate Program in Industrial, Interior and Visual Communication Design The Ohio State University 2012 Master's Examination Committee: Maria Palazzi, Advisor Alan Price Dan Shellenbarger Copyright by Nicole Lemon 2012 Abstract Previsualization (previs) is a pre-production process that uses 3D animation tools to generate preliminary versions of shot or sequences. This process is quickly gaining popularity in live action film, and is beginning to be used in animation production as well. This is because it fosters creativity by allowing for designers and artists to experiment more freely and intuitively with visual design choices, and insures efficiency in production and post-production. Previs is also able to provide a means to communicate and test plans visually in the pre-production stage which enhances clarity and understanding. The intention of this thesis is to make available information about previs that is, for the most part, unpublished or unknown by all but those already deeply involved in the process, and to explore and document the application of a previs process of my own in the production my first short film. To begin I will describe the previs process from several perspectives. Previs will be presented in historical context in order to provide insight into its development. Next I will present the results of an industry professionals survey conducted in late 2011 and early 2012 as a way of revealing an insider’s viewpoint on the use of previs in commercial computer animation production.
    [Show full text]
  • JUN-JULY- EDUSUP2011.Pdf
    houdiniad_education.pdf 1 11-07-12 3:45 PM C M Y CM MY CY CMY K Light a Fire for your Students with Film-Quality VFX Skills Demand for Houdini talent is at an all-time high and top film, broadcast, and gaming production studios are looking for graduates with strong VFX skills and a creative mind-set. Putting Houdini in your school curriculum will give your students the edge they deserve - a deeper understanding of computer graphics and the problem-solving skills essential for the reality of industry production. Schools that teach Houdini’s node-based procedural workflow find that they produce smarter and more efficient visual effects and animation graduates - graduates that studios would love to meet. Visit us at Booth #712 at SIGGRAPH 2011 to learn how your institution can benefit from going procedural with Houdini. www.sidefx.com/education houdiniad_education.pdf 1 11-07-12 3:45 PM Beyond the Basics Faculty and industry combine efforts and resources to deliver novel, nontraditional tools for an advanced educational experience By Courtney E. Howard A basic, or general, education is no longer enough. As those in the industry to provide unique opportunities and the computer graphics industry and its various market invest in future generations of digital artists. segments—from visual effects, animation and its many “We provide computer graphics training aimed at giv- nuances, and game development, to computer-aided de- ing aspiring CG artists the skills and tools they need to C sign, digital art, motion graphics, and more—continue to get straight to work in the VFX and CG industries,” explains evolve, educational facilities are keeping pace through the Dominic Davenport, CEO and founder of Escape Studios.
    [Show full text]
  • Monomr: Synthesizing Pseudo-2.5D Mixed Reality Content from Monocular Videos
    applied sciences Article MonoMR: Synthesizing Pseudo-2.5D Mixed Reality Content from Monocular Videos Dong-Hyun Hwang and Hideki Koike * Department of Computer Science, School of Computing, Tokyo Institute of Technology, Tokyo 152-8550, Japan; [email protected] * Correspondence: [email protected] Abstract: MonoMR is a system that synthesizes pseudo-2.5D content from monocular videos for mixed reality (MR) head-mounted displays (HMDs). Unlike conventional systems that require multiple cameras, the MonoMR system can be used by casual end-users to generate MR content from a single camera only. In order to synthesize the content, the system detects people in the video sequence via a deep neural network, and then the detected person’s pseudo-3D position is estimated by our proposed novel algorithm through a homography matrix. Finally, the person’s texture is extracted using a background subtraction algorithm and is placed on an estimated 3D position. The synthesized content can be played in MR HMD, and users can freely change their viewpoint and the content’s position. In order to evaluate the efficiency and interactive potential of MonoMR, we conducted performance evaluations and a user study with 12 participants. Moreover, we demonstrated the feasibility and usability of the MonoMR system to generate pseudo-2.5D content using three example application scenarios. Keywords: augmented reality; computer vision; human-computer interaction Citation: Hwang, D.-H.; Koike, H. MonoMR: Synthesizing Pseudo-2.5D Mixed Reality Content from 1. Introduction Monocular Videos. Appl. Sci. 2021, 11, Mixed reality (MR) head-mounted devices (HMDs) are display devices that can 7946.
    [Show full text]