DOCSLIB.ORG

Use of Hyperspectral/Multispectral Imaging in Gastroenterology

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Use of Hyperspectral/Multispectral Imaging in Gastroenterology Journal of Clinical Medicine Review Use of Hyperspectral/Multispectral Imaging in Gastroenterology. Shedding Some–Different–Light into the Dark Samuel Ortega 1,* , Himar Fabelo 1 , Dimitris K. Iakovidis 2, Anastasios Koulaouzidis 3 and Gustavo M. Callico 1 1 Institute for Applied Microelectronics (IUMA), University of Las Palmas de Gran Canaria (ULPGC), Las Palmas de Gran Canaria 35017, Spain; [email protected] (H.F.); [email protected] (G.M.C.) 2 Dept. of Computer Science and Biomedical Informatics, University of Thessaly, 35131 Lamia, Greece; [email protected] 3 Endoscopy Unit, The Royal Infirmary of Edinburgh, Edinburgh EH16 4SA, UK; [email protected] * Correspondence: [email protected]; Tel.: +34-928-451-220 Received: 26 November 2018; Accepted: 26 December 2018; Published: 1 January 2019 Abstract: Hyperspectral/Multispectral imaging (HSI/MSI) technologies are able to sample from tens to hundreds of spectral channels within the electromagnetic spectrum, exceeding the capabilities of human vision. These spectral techniques are based on the principle that every material has a different response (reflection and absorption) to different wavelengths. Thereby, this technology facilitates the discrimination between different materials. HSI has demonstrated good discrimination capabilities for materials in fields, for instance, remote sensing, pollution monitoring, field surveillance, food quality, agriculture, astronomy, geological mapping, and currently, also in medicine. HSI technology allows tissue observation beyond the limitations of the human eye. Moreover, many researchers are using HSI as a new diagnosis tool to analyze optical properties of tissue. Recently, HSI has shown good performance in identifying human diseases in a non-invasive manner. In this paper, we show the potential use of these technologies in the medical domain, with emphasis in the current advances in gastroenterology. The main aim of this review is to provide an overview of contemporary concepts regarding HSI technology together with state-of-art systems and applications in gastroenterology. Finally, we discuss the current limitations and upcoming trends of HSI in gastroenterology. Keywords: hyperspectral imaging; multispectral imaging; clinical diagnosis; biomedical optical imaging; gastroenterology; medical diagnostic imaging 1. Introduction Hyperspectral/Multispectral (HS/MS) imaging (HSI/MSI), also known as Imaging Spectroscopy, is a technology capable of overcoming the imaging limitations of the human vision based in white light (WL). In fact, HSI combines the features provided by two technologies that have been, for decades now, used separately i.e., digital imaging and spectroscopy. Digital imaging allows recording of the morphological features of a given scene, extracting information of different objects in regards to shape and textures. Spectroscopy deals with the interaction between the electromagnetic (EM) radiation and matter. While the capabilities of the human vision are restricted to a certain region of the EM spectrum (EMS), the visible spectrum that spans from 400 to 700 nm, most common HS commercial systems expand this spectral range from 400 to 2500 nm. Although there are HS cameras able to cover the EM up to 12 microns, such systems are restricted to certain applications that are out of the scope of this manuscript. HSI provides information in regions of the EMS that the human eye cannot see, revealing therefore substance properties that are normally unavailable to human beings. Furthermore, while the J. Clin. Med. 2019, 8, 36; doi:10.3390/jcm8010036 www.mdpi.com/journal/jcm J. Clin. Med. 2019, 8, x FOR PEER REVIEW 2 of 21 J.applications Clin. Med. 2019 that, 8, 36 are out of the scope of this manuscript. HSI provides information in regions of2 ofthe 21 EMS that the human eye cannot see, revealing therefore substance properties that are normally unavailable to human beings. Furthermore, while the human eye is only capable of distinguishing humanthree different eye is only wavelengths capable of associated distinguishing with threethe opsi differentns of the wavelengths retina (Cianopsin associated sensitive with the to opsins 430 nm of the-blue retina light-; (Cianopsin Cloropsin sensitive to 430530 nm -bluegreen light light-;-; Cloropsin and, Eritropsin sensitive sensitive to 530 to nm 650 -green nm light-red light-; and,-), EritropsinHS cameras sensitive can capture to 650 the nm EMS -red in light hundreds-), HS cameras of different can capture narrow the wavelengths, EMS in hundreds largely of increasing different narrowthe resolution wavelengths, to over largelywhat humans increasing can see. the resolutionOn the other to overhand, what MSI humansis based canon the see. same On theprinciple other hand,of HSI MSI with is basedthe main on thedifference same principle being that of HSI MSI with is generally the main differencecharacterized being by that a lower MSI isnumber generally of characterizedspectral channels by a [1]. lower number of spectral channels [1]. An HS image is recorded in a datadata structurestructure calledcalled HS cube,cube, which contains both spatial and spectral information from a given image. The inform informationation inside the HS cube can be visualized in several different ways. If If a a single pixel from an HS image is selected, the spectrum related to this pixel can be examined. Likewise, it is possible to visualize the entire spatial information for a given wavelength. The observed information at various wa wavelengthsvelengths represents diffe differentrent properties of the matter. Figure1 1 shows shows anan HSHS cube,cube, where where both both types types of of representations representations can can be be observed. observed. Figure 1. Example ofof anan HSHS cube cube from from in-vivo in-vivo human human brain brain surface surface and and spectrum spectrum from from the the pixel pixel in red in (redleft ().left Several). Several images images at different at different wavelengths wavelengths obtained obtained from from the HSthe datacubeHS datacube (right (right). ). The spectral signature (also called spectral fingerprint)fingerprint) is the curve that links the EM radiation with a certaincertain material.material. The key point of this concept is that each material has its own interaction with the EMS, hence the spectral signature of any givengiven material is unique. By analyzing the spectral signatures contained in an HS image, it is possiblepossible toto distinguishdistinguish between the different substancessubstances that are are present present in in the the captured captured image. image. Nevertheless, Nevertheless, to toproperly properly differentiate differentiate materials materials by using by using the thespectral spectral signature signature information, information, some some issues issues have have to to be be addressed. addressed. First, First, the the measured spectral signature from the same material can present subtle variations, i.e., inter-sample variability. variability. Second, there are materialsmaterials thatthat presentpresent spectralspectral similaritiessimilarities amongamong them,them, beingbeing extremelyextremely challengingchallenging to perform an automatic differentiation ofof suchsuch materialsmaterials basedbased onlyonly onon theirtheir spectralspectral signatures.signatures. Many researchers have employed HSI technology for different applications [1] [1] such as non-invasive foodfood quality quality inspection inspection [2,3 ],[2,3], improving improving recycling recycling processes processes [4], or examining[4], or examining paintings forpaintings accurate for identification accurate identification of the pigments of the used pigments in order used to refine in order their to restoration refine their [5,6 restoration]. Geologists [5,6]. use HSIGeologists to identify use HSI the locationto identify of differentthe location minerals of different [7]. Furthermore, minerals [7]. in Furthermore, agriculture this in agriculture technology this has beentechnology used to has quantitatively been used to characterize quantitatively the characterize soil [8] or to identifythe soil [8] the or stress to identify levels of the plants stress [9 levels]. of plantsFigure [9]. 2 presents an example of the spectral signatures of different tissues [ 10], where the differencesFigure between2 presents the spectralan example signatures of the of spectral primary signatures (glioblastoma of different and oligodendroglioma tissues [10], where grade the III) anddifferences secondary between brain tumorsthe spectral (metastatic signatures lung, of renal primary and breast)(glioblastoma are evident. and oligodendroglioma Just a visual inspection grade of the shape of the reflectance curve reveals that it is possible to identify the type of tissue present at each J. Clin. Med. 2019, 8, x FOR PEER REVIEW 3 of 21 J. Clin. Med. 2019, 8, 36 3 of 21 III) and secondary brain tumors (metastatic lung, renal and breast) are evident. Just a visual inspection of the shape of the reflectance curve reveals that it is possible to identify the type of tissue pixelpresent of theat image.each pixel The measuredof the image. spectral The signatures measured are spectral mainly affectedsignatures by illumination,are mainly theaffected mixture by ofillumination, different substances, the mixture and/or of differ byent noise. substances, and/or by noise. FigureFigure 2.2. SpectralSpectral signaturessignatures ofof differentdifferent brainbrain tumortumor tissuestissues
Load more

Recommended publications
  • Multispectral Imaging for Medical and Industrial Machine Vision Systems
    1 | Tech Guide: Multispectral imaging for medical and industrial machine vision systems Tech Guide: Multispectral Imaging Multispectral imaging for medical and industrial machine vision systems 2 | Tech Guide: Multispectral imaging for medical and industrial machine vision systems Table of contents Introduction Chapter 1: What is multispectral imaging? Chapter 2: Multispectral imaging applications Chapter 3: Multispectral camera technologies Chapter 4: Key considerations when selecting camera technology for multispectral imaging Chapter 5: Hyperspectral and the future of multispectral imaging 3 | Tech Guide: Multispectral imaging for medical and industrial machine vision systems Introduction Just as machine vision systems have evolved from traditional monochrome cameras to many systems that now utilize full color imaging information, there has also been an evolution from systems that only captured broadband images in the visible spectrum, to those that can utilize targeted spectral bands in both visible and non- visible spectral regions to perform more sophisticated inspection and analysis. The color output of the cameras used in the machine vision industry today is largely based on Bayer-pattern or trilinear sensor technology. But imaging is moving well beyond conventional color where standard RGB is not enough to carry out inspection tasks. Some applications demand unconventional RGB wavelength bands while others demand a combination of visible and non-visible wavelengths. Others require exclusively non-visible wavelengths such as UV, NIR or SWIR, with no wavebands in the visible spectrum. Complex metrology and imaging applications are beginning to demand higher numbers of spectral channels or possibilities to select application-specific spectral filtering at high inspection throughputs. With the traditional machine vision industry merging with intricate measurement technologies, consistent, reliable, high-fidelity color and multispectral imaging are playing key roles in industrial quality control.
    [Show full text]
  • User's Manual for Nuance 3.0.2 Notice
    Nuance MULTISPECTRAL IMAGING SYSTEM User's Manual for Nuance 3.0.2 Notice The information in this document is subject to change without notice and should not be construed as a commitment by PerkinElmer, Inc. PerkinElmer assumes no responsibility for any errors that may appear in this document. This manual is believed to be complete and accurate at the time of publication. In no event shall PerkinElmer be liable for incidental or consequential damages in connection with or arising from the use of this manual. This manual describes system operation using Nuance version 3.0.2 software. For more information contact: PerknElmer, Inc. 68 Elm Street, Hopkinton, MA, 01748, USA Phone: 800-762-4000 or +1 203-925-4602 Fax: +1 203-944-4904 Email: [email protected] Web site: http://www.perkinelmer.com US Patent 5,892,612; 5,953,087; 7,655,898; and patents pending. Document Part No. 130765 Rev. 01 Table of Contents Part I Welcome To Nuance 6 Part II Introduction to Multispectral Imaging 7 1 Light...................................................................................................................................... 7 2 Human...................................................................................................................................... Perception of Light Intensity and of Color 8 3 Light ......................................................................................................................................Absorbance and Reflection 9 4 Fluorescence.....................................................................................................................................
    [Show full text]
  • Classification of Compressed Remote Sensing Multispectral Images Via
    Journal of Imaging Article Classification of Compressed Remote Sensing Multispectral Images via Convolutional Neural Networks Michalis Giannopoulos 1,2,*,† , Anastasia Aidini 1,2,† , Anastasia Pentari 1,2 , Konstantina Fotiadou 1,2 and Panagiotis Tsakalides 1,2 1 Signal Processing Lab (SPL), Institute of Computer Science, Foundation for Research and Technology-Hellas (FORTH), 70013 Crete, Greece; [email protected] (A.A.); [email protected] (A.P.); [email protected] (K.F.); [email protected] (P.T.) 2 Computer Science Department, University of Crete, 70013 Crete, Greece * Correspondence: [email protected] † These authors contributed equally to this work. Received: 17 January 2020; Accepted: 15 April 2020; Published: 18 April 2020 Abstract: Multispectral sensors constitute a core Earth observation image technology generating massive high-dimensional observations. To address the communication and storage constraints of remote sensing platforms, lossy data compression becomes necessary, but it unavoidably introduces unwanted artifacts. In this work, we consider the encoding of multispectral observations into high-order tensor structures which can naturally capture multi-dimensional dependencies and correlations, and we propose a resource-efficient compression scheme based on quantized low-rank tensor completion. The proposed method is also applicable to the case of missing observations due to environmental conditions, such as cloud cover. To quantify the performance of compression, we consider both typical image quality metrics as well as the impact on state-of-the-art deep learning-based land-cover classification schemes. Experimental analysis on observations from the ESA Sentinel-2 satellite reveals that even minimal compression can have negative effects on classification performance which can be efficiently addressed by our proposed recovery scheme.
    [Show full text]
  • Post-Processing of VIS, NIR, and SWIR Multispectral Images of Paintings
    heritage Article Post-Processing of VIS, NIR, and SWIR Multispectral Images of Paintings. New Discovery on the The Drunkenness of Noah, Painted by Andrea Sacchi, Stored at Palazzo Chigi (Ariccia, Rome) 1, , 1, , 2, , 3, , Lucilla Pronti * y , Martina Romani * y, Gianluca Verona-Rinati * y, Ombretta Tarquini * y, 4, 3, 3, 1, Francesco Colao y, Marcello Colapietro y, Augusto Pifferi y , Mariangela Cestelli-Guidi y 2, and Marco Marinelli y 1 INFN-Laboratori Nazionali di Frascati, Via E. Fermi 40, 00044 Frascati, Italy 2 INFN-Dipartimento di Ingegneria Industriale, Università degli Studi di Roma Tor Vergata, Department of Industrial Engineering, Via del Politecnico 1, 00133 Roma, Italy 3 C.N.R. Istituto di Cristallografia - Area della Ricerca Roma 1, Via Salaria Km 29300, 00015 Monterotondo (RM), Italy 4 ENEA, Italian National Agency for New Technologies, Energy and Sustainable Economic Development, Via E. Fermi 45, 00044 Frascati (RM), Italy * Correspondence: [email protected] (L.P.); [email protected] (M.R.); [email protected] (G.V.-R.); [email protected] (O.T.) These authors contributed equally to this work. y Received: 30 June 2019; Accepted: 30 July 2019; Published: 2 August 2019 Abstract: IR Reflectography applied to the identification of hidden details of paintings is extremely useful for authentication purposes and for revealing technical hidden features. Recently, multispectral imaging has replaced traditional imaging techniques thanks to the possibility to select specific spectral ranges bringing out interesting details of the paintings. VIS–NIR–SWIR images of one of the The Drunkenness of Noah versions painted by Andrea Sacchi, acquired with a modified reflex and InGaAs cameras, are presented in this research.
    [Show full text]
  • Fundamentals of Remote Sensing
    Fundamentals of Remote Sensing A Canada Centre for Remote Sensing Remote Sensing Tutorial Natural Resources Ressources naturelles Canada Canada Fundamentals of Remote Sensing - Table of Contents Page 2 Table of Contents 1. Introduction 1.1 What is Remote Sensing? 5 1.2 Electromagnetic Radiation 7 1.3 Electromagnetic Spectrum 9 1.4 Interactions with the Atmosphere 12 1.5 Radiation - Target 16 1.6 Passive vs. Active Sensing 19 1.7 Characteristics of Images 20 1.8 Endnotes 22 Did You Know 23 Whiz Quiz and Answers 27 2. Sensors 2.1 On the Ground, In the Air, In Space 34 2.2 Satellite Characteristics 36 2.3 Pixel Size, and Scale 39 2.4 Spectral Resolution 41 2.5 Radiometric Resolution 43 2.6 Temporal Resolution 44 2.7 Cameras and Aerial Photography 45 2.8 Multispectral Scanning 48 2.9 Thermal Imaging 50 2.10 Geometric Distortion 52 2.11 Weather Satellites 54 2.12 Land Observation Satellites 60 2.13 Marine Observation Satellites 67 2.14 Other Sensors 70 2.15 Data Reception 72 2.16 Endnotes 74 Did You Know 75 Whiz Quiz and Answers 83 Canada Centre for Remote Sensing Fundamentals of Remote Sensing - Table of Contents Page 3 3. Microwaves 3.1 Introduction 92 3.2 Radar Basic 96 3.3 Viewing Geometry & Spatial Resolution 99 3.4 Image distortion 102 3.5 Target interaction 106 3.6 Image Properties 110 3.7 Advanced Applications 114 3.8 Polarimetry 117 3.9 Airborne vs Spaceborne 123 3.10 Airborne & Spaceborne Systems 125 3.11 Endnotes 129 Did You Know 131 Whiz Quiz and Answers 135 4.
    [Show full text]
  • Multispectral Image Feature Points
    Sensors 2012, 12, 12661-12672; doi:10.3390/s120912661 OPEN ACCESS sensors ISSN 1424-8220 www.mdpi.com/journal/sensors Article Multispectral Image Feature Points Cristhian Aguilera 1,*, Fernando Barrera 2, Felipe Lumbreras 2,3, Angel D. Sappa 2 and Ricardo Toledo 2,3 1 Department of Electrical and Electronics Engineering, Collao 1202, University of Bío-Bío, 4051381 Concepción, Chile 2 Computer Vision Center, Edifici O, Campus UAB, 08193 Bellaterra, Barcelona, Spain; E-Mails: [email protected] (F.B.); [email protected] (F.L.); [email protected] (A.D.S.); [email protected] (R.T.) 3 Computer Science Department, Edifici O, Campus UAB, 08193 Bellaterra, Barcelona, Spain * Author to whom correspondence should be addressed; E-Mail: [email protected]; Tel.: +56-41-311-1287; Fax: +56-41-311-1013. Received: 6 July 2012; in revised form: 24 August 2012 / Accepted: 30 August 2012 / Published: 17 September 2012 Abstract: This paper presents a novel feature point descriptor for the multispectral image case: Far-Infrared and Visible Spectrum images. It allows matching interest points on images of the same scene but acquired in different spectral bands. Initially, points of interest are detected on both images through a SIFT-like based scale space representation. Then, these points are characterized using an Edge Oriented Histogram (EOH) descriptor. Finally, points of interest from multispectral images are matched by finding nearest couples using the information from the descriptor. The provided experimental results and comparisons with similar methods show both the validity of the proposed approach as well as the improvements it offers with respect to the current state-of-the-art.
    [Show full text]
  • Less Is More When Applying Transfer Learning to Multi-Spectral Data
    Less is More when Applying Transfer Learning to Multi-Spectral Data Yuvraj Sharma1 and Robert Ross2 1 Adapt Centre, Technological University Dublin, Ireland [email protected] 2 Adapt Centre, Technological University Dublin, Ireland [email protected] Abstract. Transfer Learning is widely recognized as providing incred- ible benefits to many image processing tasks. But not all problems in the computer vision field are driven by traditional Red, Green, and Blue (RGB) imagery as tend to be assumed by most large pre-trained mod- els for Transfer Learning. Satellite based remote sensing applications for example typically use multispectral bands of light. While transferring RGB features to this non-RGB domain has been shown to generally give higher accuracy than training from scratch, the question remains whether a more suitable fine tuned method can be found. Given this challenge, this paper presents a study in multispectral image analysis using multiple methods to achieve feature transfer. Specifically, we train and compare two pre-trained models based on the Resnet50 architec- ture and apply them to a multi-spectral image processing task. The key difference between the two models is that one was pre-trained in a con- ventional way against RGB data, while the other was trained against a single band greyscale variant of the same data. Our results demonstrate an improved performance on the greyscale pre-trained model relative to the more traditional RGB model. Keywords: Deep learning · Transfer learning · Image Analysis · Resnet · CNN · Multispectral images · ImageNet · Satellite imagery · EuroSat. 1 Introduction The rapid advancement of artificial intelligence (AI) in the computer vision field has increased the demand of large-scale labelled data.
    [Show full text]
  • High Resolution Multispectral and Thermal Remote Sensing-Based Water Stress Assessment in Subsurface Irrigated Grapevines
    Article High Resolution Multispectral and Thermal Remote Sensing-Based Water Stress Assessment in Subsurface Irrigated Grapevines Carlos Zúñiga Espinoza 1, Lav R. Khot 1,2,*, Sindhuja Sankaran 1,2, Pete W. Jacoby 2,3 1 Department of Biological System Engineering, Washington State University, Pullman, WA 99164-6120, USA; [email protected] (C.Z.E); [email protected] (S.S.) 2 Center for Precision and Automated Agricultural Systems, Washington State University, Prosser, WA 99350-8694, USA; [email protected] 3 Department of Crop and Soil Science, Washington State University, Pullman, WA 99164-6420, USA * Correspondence: [email protected]; Tel.: +1-509-786-9302 Received: 8 June 2017; Accepted: 5 September 2017; Published: 16 September 2017 Abstract: Precision irrigation management is based on the accuracy and feasibility of sensor data assessing the plant water status. Multispectral and thermal infrared images acquired from an unmanned aerial vehicle (UAV) were analyzed to evaluate the applicability of the data in the assessment of variants of subsurface irrigation configurations. The study was carried out in a Cabernet Sauvignon orchard located near Benton City, Washington. Plants were subsurface irrigated at a 30, 60, and 90 cm depth, with 15%, 30%, and 60% irrigation of the standard irrigation level as determined by the grower in commercial production management. Half of the plots were irrigated using pulse irrigation and the other half using continuous irrigation techniques. The treatments were compared to the control plots that received standard surface irrigation at a continuous rate. The results showed differences in fruit yield when the control was compared to deficit irrigated treatments (15%, 30%, 60% of standard irrigation), while no differences were found for comparisons of the techniques (pulse, continuous) or depths of irrigation (30, 60, 90 cm).
    [Show full text]
  • Reconstruction of Multispatial, Multispectral Image Data Using Spatial Frequency Content
    R. A. SCHOWENGERDT* Office of Arid Lands Studies and Systems and Zndustrial Engineering Department University of Arizona Tucson, AZ 8571 9 Reconstruction of Multispatial, MuItispectraI Image Data Using Spatial Frequency Content Multispatial, multispectral imagery is reconstructed by computer processing to enhance information extraction. recent interest and success with simpler ap- proaches (Hung, 1979). HERE HAS BEEN considerable interest in tech- It has been suggested (Colvocoresses, 1977) that Tniques for reducing the quantity of image data a mixture of high spatial resolution spectral bands transmitted from spacecraft to ground receiving and lower resolution bands may be an acceptable stations. The concern about excessive data way to reduce data rates without sacrificing image acquisition/transmission rates is particularly im- information content. The basis for this suggestion portant for the next generation of high spatial and is that only one or two spectral bands are required radiometric resolution sensors, such as the the- to define the majority of edges in a scene, and matic mapper (TM) on Landsat D, the multispectral hence only these bands need to be of high resolu- resource sampler (MRS),SPOT (Chevrel, 1979), and tion. A significant advantage of mixed resolution, Mapsat (Colvocoresses, 1979). or what may be termed multispatial, compression Techniques that reduce data quantity must also is that it can be accomplished by appropriate ABSTRACT:A data compression technique that utilizes a mixture of spatial res- olutions (multispatial) for a multispectral scanner is described. The com- plementary reconstruction procedure that extrapolates edge information from the high resolution banqs) to the low resolution bands is also discussed. Exam- ples of Landsat MSS imagery that have been compressed and reconstructed to the original resolution are presented.
    [Show full text]
  • Hyperspectral and Multispectral Image Fusion Under Spectrally Varying
    Hyperspectral and Multispectral Image Fusion Under Spectrally Varying Spatial Blurs – Application to High Dimensional Infrared Astronomical Imaging Claire Guilloteau, Thomas Oberlin, Olivier Berné, Nicolas Dobigeon To cite this version: Claire Guilloteau, Thomas Oberlin, Olivier Berné, Nicolas Dobigeon. Hyperspectral and Multispectral Image Fusion Under Spectrally Varying Spatial Blurs – Application to High Dimensional Infrared Astronomical Imaging. IEEE Transactions on Computational Imaging, IEEE, 2020, 6, pp.1362-1374. 10.1109/TCI.2020.3022825. hal-02949174 HAL Id: hal-02949174 https://hal.archives-ouvertes.fr/hal-02949174 Submitted on 28 Sep 2020 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. IEEE TRANSACTIONS ON COMPUTATIONAL IMAGING 1 Hyperspectral and multispectral image fusion under spectrally varying spatial blurs – Application to high dimensional infrared astronomical imaging Claire Guilloteau, Thomas Oberlin, Olivier Berne´ and Nicolas Dobigeon Abstract—Hyperspectral imaging has become a significant spatial and spectral resolution is indeed of particular interest to source of valuable data for astronomers over the past decades. study key mechanisms in astrophysics and cosmology. More Current instrumental and observing time constraints allow direct specifically, this concerns for instance the combined sensing acquisition of multispectral images, with high spatial but low spectral resolution, and hyperspectral images, with low spatial of the morphology or spectral signatures of protoplanetary but high spectral resolution.
    [Show full text]
  • Multispectral Photogrammetric Data Acquisition and Processing for Wall
    Multispectral photogrammetric data acquisition and processing for wall paintings studies Anthony Pamart, Odile Guillon, Stéphan Faraci, Eloi Gattet, Michel Genevois, Jean-Marc Vallet, Livio de Luca To cite this version: Anthony Pamart, Odile Guillon, Stéphan Faraci, Eloi Gattet, Michel Genevois, et al.. Multispectral photogrammetric data acquisition and processing for wall paintings studies. ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2017, Nafplio, Greece. pp.559–566, 10.5194/isprs-archives-XLII-2-W3-559-2017. hal-01480562 HAL Id: hal-01480562 https://hal.archives-ouvertes.fr/hal-01480562 Submitted on 1 Mar 2017 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLII-2/W3, 2017 3D Virtual Reconstruction and Visualization of Complex Architectures, 1–3 March 2017, Nafplio, Greece MULTISPECTRAL PHOTOGRAMMETRIC DATA ACQUISITION AND PROCESSING FOR WALL PAINTINGS STUDIES A. Pamarta,∗ O. Guillonb,
    [Show full text]
  • Computationally Efficient Target Classification in Multispectral Image Data with Deep Neural Networks
    Computationally Efficient Target Classification in Multispectral Image Data with Deep Neural Networks Lukas Cavigellia,*, Dominic Bernatha, Michele Magnoa,b, Luca Beninia,b aETH Zurich, Integrated Systems Laboratory, Gloriastr. 35, CH-8092 Zurich, Switzerland bUniversity of Bologna, DEI, Viale Risorgimento 2, I-40126 Bologna, Italy ABSTRACT Detecting and classifying targets in video streams from surveillance cameras is a cumbersome, error-prone and expensive task. Often, the incurred costs are prohibitive for real-time monitoring. This leads to data being stored locally or transmitted to a central storage site for post-incident examination. The required communication links and archiving of the video data are still expensive and this setup excludes preemptive actions to respond to imminent threats. An effective way to overcome these limitations is to build a smart camera that analyzes the data on-site, close to the sensor, and transmits alerts when relevant video sequences are detected. Deep neural networks (DNNs) have come to outperform humans in visual classifications tasks and are also performing exceptionally well on other computer vision tasks. The concept of DNNs and Convolutional Networks (ConvNets) can easily be extended to make use of higher-dimensional input data such as multispectral data. We explore this opportunity in terms of achievable accuracy and required computational effort. To analyze the precision of DNNs for scene labeling in an urban surveillance scenario we have created a dataset with 8 classes obtained in a field experiment. We combine an RGB camera with a 25-channel VIS-NIR snapshot sensor to assess the potential of multispectral image data for target classification. We evaluate several new DNNs, showing that the spectral information fused together with the RGB frames can be used to improve the accuracy of the system or to achieve similar accuracy with a 3x smaller computation effort.
    [Show full text]