DOCSLIB.ORG

The Pennsylvania State University the Graduate School OPTIMIZING

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
The Pennsylvania State University the Graduate School OPTIMIZING The Pennsylvania State University The Graduate School OPTIMIZING VIDEO PROCESSING FOR NEXT-GENERATION MOBILE PLATFORMS A Dissertation in Computer Science and Engineering by Haibo Zhang © 2020 Haibo Zhang Submitted in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy May 2020 The dissertation of Haibo Zhang was reviewed and approved by the following: Mahmut T. Kandemir Professor of Computer Science and Engineering Dissertation Co-Advisor, Co-Chair of Committee Anand Sivasubramaniam Distinguished Professor of Computer Science and Engineering Dissertation Co-Advisor, Co-Chair of Committee Chita R. Das Distinguished Professor of Computer Science and Engineering Department Head of Computer Science and Engineering Dinghao Wu Associate Professor of Information Sciences and Technology ii Abstract Video has become one of the most dominant applications in the mobile and Internet- of-Things (IoT) platforms. Video applications cover numerous kinds of emerging applications, such as live broadcasts, video posts, video conferences, smart camera- based object tracking, augmented reality (AR), and virtual reality (VR), expanding the boundaries of communication and entertainment. Video-oriented applications, e.g., Netflix, TikTok, Snapchat, are among the most popular applications in the mobile application stores. Smart mobile devices, commonly equipped with camera(s) and display, serve as the platform for video applications and, therefore, contribute to a significant volume of video traffic. It is thus essential that better video quality and low energy consumption are the keys for continued growth and further acceptance of mobile devices. While previous research has mainly studied how to transmit video data efficiently over a network, this dissertation provides a systematic study of the inefficiencies within the mobile platforms. This dissertation reveals that, even with highly optimized video codecs and video hardware accelerators, today’s mobile platforms for video processing are still suffering from various energy and Quality of Service (QoS) inefficiency issues, which lie within current system and hardware stack, and warrant a revisit. To further improve QoS and energy efficiency of video processing on mobile platforms, the optimization scope needs to expand to the interaction between multiple video processing stages as well as their memory system. With the concept of considering multiple stages as an entire optimization entity, this dissertation tackles above inefficiencies in the system/hardware stacks of video processing, by 1) intelligently scheduling video processing requests regarding their QoS requirements, 2) designing a new power management policy, 3) saving energy by leveraging the value locality inherited in video memory traffic. Overall, this dissertation aims at systematically optimizing video processing for next-generation mobile platforms. iii Table of Contents List of Figures vii List of Tables xi Acknowledgments xii Chapter 1 Introduction 1 Chapter 2 Background 9 2.1 Video Basics . 9 2.2 Display-based Video Processing in Mobile Systems . 11 2.3 Camera-based Video Processing in Mobile Systems . 12 2.4 Application Example: Video Streaming . 14 Chapter 3 Flow Sensitive Scheduling for Multi-Videos on Mobile Platforms 16 3.1 Introduction . 16 3.2 Motivation . 18 3.3 Solving Memory Bottleneck . 21 3.3.1 A Flow-Sensitive Scheduler: FLOSS . 21 3.3.2 Implementation Details . 25 3.4 Experimental Evaluation . 25 3.4.1 Experiment Setup . 25 3.4.2 Results . 27 3.5 Related Work . 29 3.6 Conclusions . 29 iv Chapter 4 Optimizing QoS and Energy for Display-based Video Processing 30 4.1 Introduction . 30 4.2 Background and Motivation . 33 4.2.1 Insomnia: Inefficiencies of Current Video Processing . 34 4.3 Race-to-Sleep . 36 4.3.1 Batch Decoding . 37 4.3.2 Race Decoding . 39 4.3.3 Race-to-Sleep . 41 4.4 Content Caching . 42 4.4.1 Address Locality vs. Value Locality . 43 4.4.2 MACH: Macroblock Cache . 44 4.4.3 Gradient Blocks . 46 4.4.4 Implementation of MACH . 47 4.4.5 Discussion . 49 4.5 Display Caching . 50 4.5.1 Display Cache and MACH Buffer . 51 4.5.2 Memory Access Savings in Display . 53 4.6 Experimental Evaluation . 53 4.6.1 Experimental Setup and Workloads . 53 4.6.2 Results . 54 4.6.3 Sensitivity Study and Overhead . 58 4.6.4 Other Potential Applications of MACH . 59 4.7 Related Work . 60 4.8 Conclusions . 62 Chapter 5 Reducing Memory Usage and Energy for Camera-based Video Processing 63 5.1 Introduction . 63 5.2 Background and Motivation . 67 5.2.1 Value Locality in Raw Videos . 68 5.2.2 Reducing Video Buffer Bandwidth . 69 5.3 Isolating the Essential Bits . 72 5.3.1 Base Selection . 73 5.3.2 Evaluation of MinVB and MidVB . 78 5.4 Distilling the Essential Bits . 79 5.4.1 Approximation through Distilling . 81 5.4.2 Implementation Details of Distill . 87 5.5 Evaluation . 89 v 5.5.1 Workloads and Experimental Setup . 89 5.5.2 Results . 90 5.6 Related Work . 94 5.7 Conclusions . 96 Chapter 6 Future Work 97 6.1 Smart Vision Pipeline . 98 6.2 Smart Display Pipeline . 101 Chapter 7 Concluding Remarks 102 Bibliography 105 vi List of Figures 1.1 Overview of my dissertation research. 4 2.1 Illustration of display-based video processing. 11 2.2 Video playback application invoking IPs in parallel. ......... 14 3.1 FPS improved due to PIP and SoC utilization with and without PIP. 18 3.2 Examples showing the advantages and drawbacks of different QoS- aware memory schedulers. Timeline is measured as units of memory requests, and all time slots are normalized to the time taken to process one memory request. 20 3.3 FLOSS: Rate calculation under various scenarios. 24 3.4 Different memory scheduling schemes on FPS and energy efficiency per frame. The FPS and energy efficiency results are w.r.t. the non-PIP/FR-FCFS baseline. 27 3.5 Per frame memory performance and utilization improvement . 27 4.1 (a) Performance and energy breakdown of video streaming; (b) Control flow and data flow of video streaming (YouTube). 33 vii 4.2 (a) State diagram of power state transitions. (b) and (c), CDF plots of frame execution time/energy consumption for 5000 frames sorted in increasing order.Per frame execution time is fixed at 16.6ms and per frame energy is fixed at 5mJ (decoder power 300mW * frame time). The plots show the time/energy spent in five different states (S3, S1, transition, short slack, and execution). (d) and (e) depict time/energy CDFs with batching. ........................... 34 4.3 Pictorial representation of different decoding schemes. tSlack is the time taken to process the next frame. P1 and P0 are Pstates. The transition energy between P and S states is shown in red. 37 4.4 Effects of various schemes. (a) and (b) show the effect of Batch Decoding. (c) and (d) plot the time/energy CDFs with Race Decoding and Race-to-Sleep for processing 5000 frames (as similar to Fig. 4.2b and Fig. 4.2c). Race Decoding increases transition a lot while Race- to-Sleep reduces transition mostly. 38 4.5 (a) Memory access pattern with low and high frequency settings. DRAM needs more Act/Pre for a low frequency setting. (b) showing energy saving of using a high-frequency VD. 39 4.6 Impact of Race-to-Sleep. Left: VD at low frequency; Right: VD at high frequency (Batching up to 16 frames.) . 41 4.7 Access locality vs. Value locality. (a) Conventional cache can hardly exploit address locality in writing a frame back to memory. (b) More than half of the current frame are already stored in memory. 44 4.8 Examples illustrating how MACH works. 45 4.9 Effect of content caching. (a) mab saves 13%, and gab saves 34% memory bandwidth and memory space. (b) gab always captures more matches than mab. (c) i. Memory layout of mab/gab; ii. layout of the pointers in Sec.4.4; and iii. layout of the pointers/digests in Sec.4.5. 47 4.10 Architecture and effects of display cache and MACH buffers. (a) Illustration of display cache; (b) Illustration of MACH buffers; (c) Sensitivity to size of display cache; and (d) Distribution of types of gab; (e) Fraction (%) of the memory accesses. 50 viii 4.11 Normalized energy results with our recipes shown across 16 videos. L:Baseline; B:batching; R:Racing; S:Race-to-Sleep; M:MAB; G:GAB. Normalized to baseline. (the lower the better). 54 4.12 Sensitivities to (a) the number of extra frame buffers when using gab; (b) the number of MACH buffer entries; (c) size of mab; and (d) different hashing mechanism. 57 5.1 Overview of a video processing application at edge devices. 64 5.2 Spatial value similarity in pixel values. 68 5.3 (a) Number of unique tiles of raw video and encoded video inputs. (b) Pixel intensity (tile 1x1) of a sequence of tiles (in total 1.8 billion tiles of 1199 frames). 34% of the pixels require 8 bits to fully represent their values. Partitioning frames into tiles: each tile consists of 4x4, or 16x16 pixels. A tile’s bit-width is determined by the largest bit-width of the pixels in the tile. 70 5.4 (a) Distribution of the pixel deltas in a tile. Pixel deltas are the differences from the first pixel value in the tile. Around 20% of the pixel deltas are 0, indicating that they are the same as the first pixel value. Only 0.01% of the pixels still require 8 bit-width. (b) The bit-width needed for a tile is now determined by the number of bits needed to represent the maximal (minimal) of deltas of the tile.
Load more

Recommended publications
  • Tracking and Automation of Images by Colour Based
    Vol 11, Issue 8,August/ 2020 ISSN NO: 0377-9254 TRACKING AND AUTOMATION OF IMAGES BY COLOUR BASED PROCESSING N Alekhya 1, K Venkanna Naidu 2 and M.SunilKumar 3 1PG student, D.N.R College of Engineering, ECE, JNTUK, INDIA 2 Associate Professor D.N.R College of Engineering, ECE, JNTUK, INDIA 3Assistant Professor Sir CRR College of Engineering , EEE, JNTUK, INDIA [email protected], [email protected] ,[email protected] Abstract— Now a day all application sectors are mostly image analysis involves maneuver the moving for the automation processing and image data to conclude exactly the information sensing . for example image processing in compulsory to help to answer a computer imaging medical field ,in industrial process lines , object problem. detection and Ranging application, satellite Digital image processing methods stems from two imaging Processing ,Military imaging etc, In principal application areas: improvement of each and every application area the raw images pictorial information for human interpretation, and are to be captured and to be processed for processing of image data for tasks such as storage, human visual inspection or digital image transmission, and extraction of pictorial processing systems. Automation applications In information this proposed system the video is converted into The remaining paper is structured as follows. frames and then it is get divided into sub bands Section 2 deals with the existing method of Image and then background is get subtracted, then the Processing. Section 3 deals with the proposed object is get identified and then it is tracked in method of Image Processing. Section 4 deals the the framed from the video .This work presents a results and discussions.
    [Show full text]
  • Color Spaces YCH and Ysch for Color Specification and Image Processing in Multi-Core Computing and Mobile Systems
    Programación Matemática y Software (2012) Vol. 4. No 2. ISSN: 2007-3283 Recibido: 14 de septiembre del 2011 Aceptado: 3 de enero del 2012 Publicado en línea: 8 de enero del 2013 Color spaces YCH and YScH for color specification and image processing in multi-core computing and mobile systems Yuriy Kotsarenko, Fernando Ramos Tecnológico de Monterrey, Campus Cuernavaca [email protected], [email protected] Resumen. En este trabajo dos nuevos espacios de color se describen para especificación de colores y procesamiento de imágenes utilizando la forma cilíndrica del espacio de color YIQ. Los espacios de colores clásicos tales como HSL y HSV no toman en cuenta la visión humana y son perceptualmente inexactos. Los espacios de colores perceptualmente uniformes como CIELAB y CIELUV son muy costosos computacionalmente para aplicaciones interactivas de tiempo real y son difíciles de implementar. Las alternativas propuestas, por otro lado, tienen un balance entre uniformidad perceptual, desempeño y simplicidad de cálculo. Estos espacios modelan colores de forma más exacta y son rápidos de calcular. Los resultados experimentales en este trabajo comparan espacios de colores clásicos con los propuestos en términos de uniformidad, riqueza de colores y desempeño, incluyendo numerosas pruebas de rapidez en procesadores de varios núcleos y sistemas móviles tales como ultra portátiles y los tablets tipo iPad. Los resultados evidencian que los espacios de colores propuestos son mejores alternativas para la industria de computación donde actualmente se utilicen los espacios de colores clásicos. Abstract. Two novel color spaces are described for color specification and image processing using cylindrical variants of YIQ color space.
    [Show full text]
  • Fiery Color Reference C9800 OKI Americas Inc
    2Copyright Copyright ES3640e MFP Color Reference Guide P/N 59377001, Revision 1.0 June, 2005 Every effort has been made to ensure that the information in this document is complete, accurate, and up-to-date. Oki assumes no responsibility for the results of errors beyond its control. Oki also cannot guarantee that changes in software and equipment made by other manufacturers and referred to in this guide will not affect the applicability of the information in it. Mention of software products manufactured by other companies does not necessarily constitute endorsement by Oki. While all reasonable efforts have been made to make this document as accurate and helpful as possible, we make no warranty of any kind, expressed or implied, as to the accuracy or completeness of the information contained herein. The most up-to-date drivers and manuals are available from the Oki web site: http://my.okidata.com Copyright © 2005 Oki Data Americas, Inc. and Electronics for Imaging, Inc. All rights reserved. This publication is protected by copyright, and all rights are reserved. No part of it may be reproduced or transmitted in any form or by any means for any purpose without express prior written consent from Electronics for Imaging, Inc. Information in this document is subject to change without notice and does not represent a commitment on the part of Electronics for Imaging, Inc. This publication is provided in conjunction with an EFI product (the “Product”) which contains EFI software (the “Software”). The Software is furnished under license and may only be used or copied in accordance with the terms of the Software license set forth below.
    [Show full text]
  • Color-Proposal.Pdf
    Colors in Snap! -bh proposal, draft, do not distribute Your computer monitor can display millions of colors, but you probably can’t distinguish that many. For example, here’s red 57, green 180, blue 200: And here’s red 57, green 182, blue 200: You might be able to tell them apart if you see them side by side: … but maybe not even then. Color space—the collection of all possible colors—is three-dimensional, but there are many ways to choose the dimensions. RGB (red-green-blue), the one most commonly used, matches the way TVs and displays produce color. Behind every dot on the screen are three tiny lights: a red one, a green one, and a blue one. But if you want to print colors on paper, your printer probably uses a different set of three colors: CMY (cyan-magenta-yellow). You may have seen the abbreviation CMYK, which represents the common technique of adding black ink to the collection. (Mixing cyan, magenta, and yellow in equal amounts is supposed to result in black ink, but typically it comes out a not-very-intense gray instead.) Other systems that try to mimic human perception are HSL (hue-saturation-lightness) and HSV (hue-saturation-value). If you are a color professional—a printer, a web designer, a graphic designer, an artist—then you need to understand all this. It can also be interesting to learn about. For example, there are colors that you can see but your computer display can’t generate. If that intrigues you, look up color theory in Wikipedia.
    [Show full text]
  • Color Control of LED Luminaires by Robert Bell
    Color control of LED luminaires BY ROBERT BELL Why it is not as easy as you might think. Another description is by hue, saturation, not create every color your eye can see. Below A bit about additive and luminance, HSL. (Some say “intensity” is a hypothetical locus of an RGB system color mixing or “lightness” instead of “luminance.”) rendered on the entire visible light spectrum. WITH RECENT MASS ACCEPTANCE Equally valid is hue, saturation, and value, of solid-state LED lighting, it’s time HSV. Value is sometimes referred to as for an explanation of this technology’s brightness and is similar to luminance. complexities and ways in which it can be However, saturation in HSL and HSV differ tamed. LED luminaires use the output of dramatically. For simplicity, I define hue multiple sources to achieve different colors as color and saturation as the amount of and intensities. Additive color mixing is color. I also try to remember if “L” is set to nothing new to our industry. We’ve done 100%, that is white, 0% is black, and 50% it for years on cycloramas with gelled is pure color when saturation is 100%. As luminaires hitting the same surface, but for “V”, 0% is black and 100% is pure color, control can be tricky. The first intelligent and the saturation value has to make up the luminaire I used was a spotlight that had difference. That over simplifies it, but let’s three MR16 lamps, fitted with red, green, carry on, as we’re not done yet. and blue filters.
    [Show full text]
  • Color Image Segmentation Using Perceptual Spaces Through Applets for Determining and Preventing Diseases in Chili Peppers
    African Journal of Biotechnology Vol. 12(7), pp. 679-688, 13 February, 2013 Available online at http://www.academicjournals.org/AJB DOI: 10.5897/AJB12.1198 ISSN 1684–5315 ©2013 Academic Journals Full Length Research Paper Color image segmentation using perceptual spaces through applets for determining and preventing diseases in chili peppers J. L. González-Pérez1 , M. C. Espino-Gudiño2, J. Gudiño-Bazaldúa3, J. L. Rojas-Rentería4, V. Rodríguez-Hernández5 and V.M. Castaño6 1Computer and Biotechnology Applications, Faculty of Engineering, Autonomous University of Queretaro, Cerro de las Campanas s/n, C.P. 76010, Querétaro, Qro., México. 2Faculty of Psychology, Autonomous University of Queretaro, Cerro de las Campanas s/n, C.P. 76010, Querétaro, Qro., México. 3Faculty of Languages Letters, Autonomous University of Queretaro, Cerro de las Campanas s/n, C.P. 76010, Querétaro, Qro., México. 4New Information and Communication Technologies, Faculty of Engineering, Department of Intelligent Buildings, Autonomous University of Queretaro, Cerro de las Campanas s/n, C.P. 76010, Querétaro, Qro., México. 5New Information and Communication Technologies, Faculty of Computer Science, Autonomous University of Queretaro, Cerro de las Campanas s/n, C.P. 76010, Querétaro, Qro., México. 6Center for Applied Physics and Advanced Technology, National Autonomous University of Mexico. Accepted 5 December, 2012 Plant pathogens cause disease in plants. Chili peppers are one of the most important crops in the world. There are currently disease detection techniques classified as: biochemical, microscopy, immunology, nucleic acid hybridization, identification by visual inspection in vitro or in situ but these have the following disadvantages: they require several days, their implementation is costly and highly trained.
    [Show full text]
  • Transforming Thermal Images to Visible Spectrum Images Using Deep Learning
    Master of Science Thesis in Electrical Engineering Department of Electrical Engineering, Linköping University, 2018 Transforming Thermal Images to Visible Spectrum Images using Deep Learning Adam Nyberg Master of Science Thesis in Electrical Engineering Transforming Thermal Images to Visible Spectrum Images using Deep Learning Adam Nyberg LiTH-ISY-EX–18/5167–SE Supervisor: Abdelrahman Eldesokey isy, Linköpings universitet David Gustafsson FOI Examiner: Per-Erik Forssén isy, Linköpings universitet Computer Vision Laboratory Department of Electrical Engineering Linköping University SE-581 83 Linköping, Sweden Copyright © 2018 Adam Nyberg Abstract Thermal spectrum cameras are gaining interest in many applications due to their long wavelength which allows them to operate under low light and harsh weather conditions. One disadvantage of thermal cameras is their limited visual inter- pretability for humans, which limits the scope of their applications. In this the- sis, we try to address this problem by investigating the possibility of transforming thermal infrared (TIR) images to perceptually realistic visible spectrum (VIS) im- ages by using Convolutional Neural Networks (CNNs). Existing state-of-the-art colorization CNNs fail to provide the desired output as they were trained to map grayscale VIS images to color VIS images. Instead, we utilize an auto-encoder ar- chitecture to perform cross-spectral transformation between TIR and VIS images. This architecture was shown to quantitatively perform very well on the problem while producing perceptually realistic images. We show that the quantitative differences are insignificant when training this architecture using different color spaces, while there exist clear qualitative differences depending on the choice of color space. Finally, we found that a CNN trained from day time examples generalizes well on tests from night time.
    [Show full text]
  • Fiery Color Reference
    Fiery® Color Server SERVER & CONTROLLER SOLUTIONS Fiery Color Reference © 2004 Electronics for Imaging, Inc. The information in this publication is covered under Legal Notices for this product. 45046197 24 September 2004 CONTENTS 3 CONTENTS INTRODUCTION 7 About this manual 7 For additional information 8 OVERVIEW OF COLOR MANAGEMENT CONCEPTS 9 Understanding color management systems 9 How color management works 10 Using ColorWise and application color management 11 Using ColorWise color management tools 12 USING COLOR MANAGEMENT WORKFLOWS 13 Understanding workflows 13 Standard recommended workflow 15 Choosing colors 16 Understanding color models 17 Optimizing for output type 18 Maintaining color accuracy 19 MANAGING COLOR IN OFFICE APPLICATIONS 20 Using office applications 20 Using color matching tools with office applications 21 Working with office applications 22 Defining colors 22 Working with imported files 22 Selecting options when printing 23 Output profiles 23 Ensuring color accuracy when you save a file 23 CONTENTS 4 MANAGING COLOR IN POSTSCRIPT APPLICATIONS 24 Working with PostScript applications 24 Using color matching tools with PostScript applications 25 Using swatch color matching tools 25 Using the CMYK Color Reference 25 Using the PANTONE reference 26 Defining colors 27 Working with imported images 29 Using CMYK simulations 30 Using application-defined halftone screens 31 Ensuring color accuracy when you save a file 32 MANAGING COLOR IN ADOBE PHOTOSHOP 33 Loading monitor settings files and ICC device profiles in Photoshop 6.x/7.x 33 Specifying
    [Show full text]
  • Analysis of Binarization Techniques and Tsetlin Machine Architectures Targeting Image Classification
    Analysis of binarization techniques and Tsetlin machine architectures targeting image classification ERIK M(T)ISE* )(LVOR S. S'ØRVIK SUPERVISORS Ole-Christo!er "ranmo# Jivitesh Sharma University of Agder, 2020 Faculty of Engineering and Science Department of Information and Communication Technology Abstract The Tsetlin Machine is a constantly evolving and developing machine learning technique with ever-increasing success. However, for every success, the Tsetlin Machine achieves, a new set of challenges are put ahead. To sufficiently bring the Tsetlin Machine to a broadly used standard, these challenges must be completed. This thesis focuses on the challenge of doing color image classification and will provide an introductory description of how this is possible through the usage of an older technique, namely binarization. A comparison with the various Tsetlin Machine adaptations made public in recent times is also present after the achieved color image classification. The results of both the initial color image classification experiment and the comparison between the varying adaptations show that the Tsetlin Machine, with a little extra work, can achieve high accuracy color image classification without image augmentation or pre-training. Preface This thesis summarizes the culmination education received at the University of Agder, Grimstad. The thesis encompasses work done over the last semester. The task was pro- posed by Prof. Ole-Christoffer Granmo in early January 2020. Both the authors of this thesis have a great interest in the Tsetlin Machine and its potential capabilities. Therefore when Granmo proposed working on a proof of concept towards color image classification along with a likely comparison of the various architectures, which had yet to be done, we were highly interested.
    [Show full text]
  • A Colour Segmentation Method for Detection of Based NZ Speed Signs
    A Colour Segmentation Method for Detection of New Zealand Speed Signs Abhishek Bedi A thesis submitted to Auckland University of Technology in fulfilment of the requirements for the degree of Master of Engineering August 2011 School of Engineering Primary Supervisor: Dr John Collins Acknowledgements This dissertation is a part of the Masters of Engineering at Auckland University of Technology New Zealand. This piece of work is a result of hard work, patience, sacrifice and unconditional support of many people. I wish to thank everyone who has supported and helped me in completing this research. Firstly, I wish express my gratitude to Geosmart NZ Limited for inspiring me to carry out this work. I am thankful to my supervisor Dr. John Collins for keeping patience, being a motivating force and taking care of me during the process. I would like to thank them for the guidance they have given me throughout the period. I express my honest thanks to the AUT library for their overall support for the entire period of the study at AUT. At the end, I am grateful to my parents and family members, especially my wife for her extreme sacrifices and providing moral support. i Statement of Originality ‘I hereby declare that this submission is my own work and that, to the best of my knowledge and belief, it contains no material previously published or written by another person nor material which to a substantial extent has been accepted for qualification of any other degree or diploma of a university or other institution of higher learning, except where due acknowledgement is made in the acknowledgements.’ Abhishek Bedi ii Abstract New Zealand Speed signs provide safe travelling speed limit or guidance information to drivers on roads.
    [Show full text]
  • Real-Time Uncertainty Visualization for B-Mode Ultrasound
    Personal reprint with permission; the definite version of this article is available at http://ieeexplore.ieee.org/ Copyright c 2015, IEEE, DOI: 10.1109/SciVis.2015.7429489 Real-time Uncertainty Visualization for B-Mode Ultrasound Christian Schulte zu Berge, Denis Declara, Christoph Hennersperger, Maximilian Baust, and Nassir Navab Figure 1: Uncertainty visualization for B-mode abdominal ultrasound. Given the original B-mode ultrasound image (a), we first com- pute the corresponding Confidence Map (b), which maps regions of low attenuation to high confidence. Using different schemes, we then compute the proposed fused uncertainty visualizations. For educational purposes, we introduce an uncertainty color overlay (c). For clinical applications, we further propose mapping to chroma (d) and to fuzziness (e) as perceptional visualization schemes maintaining the original diagnostic value. ABSTRACT adjust to the transducer positioning in real-time, provides also ex- pert clinicians with a strong visual feedback on their actions. This B-mode ultrasound is a very well established imaging modality and helps them to optimize the acoustic window and can improve the is widely used in many of today’s clinical routines. However, ac- general clinical value of ultrasound. quiring good images and interpreting them correctly is a challeng- ing task due to the complex ultrasound image formation process de- Keywords: Ultrasound, Uncertainty Visualization, Confidence pending on a large number of parameters. To facilitate ultrasound Maps, Real-time. acquisitions, we introduce a novel framework for real-time uncer- tainty visualization in B-mode images. We compute real-time per- Index Terms: Computer Graphics [I.3.m]: Miscellaneous Com- pixel ultrasound Confidence Maps, which we fuse with the original puter Applications [J.3]: Life and Medical Sciences—Health.
    [Show full text]
  • 2.2 Background Subtraction 14 2.3 Combined Effect 16 2.4 Additional Considerations 17
    DEPARTMENT OF ELECTRICAL ENGINEERING AND COMPUTER SCIENCE MASTER´S THESIS CONTROL AND PROGRAMMING OF AN INDUSTRIAL ROBOT USING MICROSOFT KINECT BY: PABLO PÉREZ TEJEDA MAT. NO.: PROFESSOR: Prof. Dr. Ing. Konrad Wöllhaf SUBMISSION DATE: 31. August 2013 Acknowledgement I would like to include in this short acknowledgement a gratitude to my Professor, Dr. Wöllhaf, for giving me hints in the analysis of the problem and providing my facilities for finishing my thesis; also mention Prof. Matthias Stark, for his help with the hardware, and finally, to my family and friends, for giving me support and understanding in the long days spent in front of three computer screens. ii Abstract Industrial robot programming relies on a suitable interface between a human operator and the robot hardware. This interface has evolved through the years to facilitate the task of programming, making possible, for example, to position a robot at real time with a handheld unit, or designing ‘offline’ the layout and operation of a complex industrial process in the GUI of a computer application. The different approaches to robot programming aim to increasing rates of efficiency without impairing already assumed capacities of the programming environment, like stability control, precision or safety. Some of these approaches have found their way in computer vision, and the last generation of image sensors boosts today many applications, inside and outside the automation industry, featuring extended capabilities in a new low-cost market. The Kinect™ sensor from Microsoft® emerged in the market of video game consoles in 2010, based around a webcam-style add-on peripheral for the Xbox 360™ console, enabling gamers to control and interact with it through a natural user interface using gestures and spoken commands.
    [Show full text]