Digital Image Processing (In the Ai Era)
Total Page:16
File Type:pdf, Size:1020Kb
DIGITAL IMAGE PROCESSING (IN THE AI ERA) Xiangyu Yu School of Electronic and Information Engineering, South China University of Technology, P. R. China [email protected] LECTURE 2 THE HUMAN VISUAL SYSTEM “Seeing is believing” 3/10/2020 DIGITAL IMAGE PROCESSING 3 CONTENTS The human visual system Image Formation in the Eye Illusions 3/10/2020 DIGITAL IMAGE PROCESSING 4 INTRODUCTION In many image processing applications, the objective is to help a human observer perceive the visual information in an image. Therefore, it is important to understand the human visual system. The percentage of information that flows through visual pathways has been estimated at 90–95% for a typical human without any sensory impairment. The human visual system consists mainly of the eye (image sensor or camera), optic nerve (transmission path), and brain (image information processing unit or computer). It is one of the most sophisticated image processing and analysis systems. 3/10/2020 DIGITAL IMAGE PROCESSING 5 Images taken from Scott E Umbaugh, Digital Image Processing and Analysis: Human and Computer Vision Applications with CVIPtools(3e) (2017) THE HUMAN VISUAL SYSTEM 3/10/2020 Figure The Human Visual System DIGITAL IMAGE PROCESSING IMAGE DIGITAL Umbaugh3E 6 - P.360 HUMAN VISUAL SYSTEM The best vision model we have! Knowledge of how images form in the eye can help us with processing digital images We will take just a whirlwind tour of the human visual system 3/10/2020 DIGITAL IMAGE PROCESSING 7 STRUCTURE OF THE HUMAN EYE 2008) Cornea and sclera keep everything in! The choroid contains all of the blood vessels that serve as nutrition to the eye. The iris contains the pigment that gives our eyes their colour The iris diaphragm controls amount of light that enters the eye. Figure Simplified diagram of a cross section of the human eye. 3/10/2020 DIGITAL IMAGE PROCESSING G3C-P.20-21 Images taken from Gonzalez & Woods, Digital Image Processing ( Processing Image & Woods, Digital from Gonzalez taken Images G3E-P.58-59 STRUCTURE OF THE HUMAN EYE(2) The lens and the ciliary muscle focus the reflected lights from objects into the retina to form an image of the objects. The lens contains 60-70% water, 6% of fat. The lens focuses light from objects onto the retina 3/10/2020 DIGITAL IMAGE PROCESSING 9 G3C-P.21 G3E-P.58-59 RETINA Retina: consist of receptors Pattern vision is afforded by the distribution of discrete light receptors over the surface of the retina. There are two classes of receptors: cones (light sensors) and rods. Rods for general vision, cones for details. 3/10/2020 DIGITAL IMAGE PROCESSING 10 G3C-P.21 G3E-P.59 LIGHT RECEPTORS IN THE RETINA About 6-7 millions cones for bright light vision called photopic - Density of cones is about 150,000 elements/mm2. - Cones involve in color vision. - Cones are concentrated in fovea about 1.5x1.5 mm2. About 75-150 millions rods for dim light vision called scotopic - Rods are sensitive to low level of light and are not involved color vision. 3/10/2020 DIGITAL IMAGE PROCESSING 11 CONES There are 6 to 7 million cones in each eye. Concentrated in the central portion of the retina called the fovea. Highly sensitive to color. Women have extra cones which means they can see more colors Each cone is connected to its own nerve end, so human can resolve fine details. Cone vision is called photopic or BRIGHT-LIGHT VISION Photopic (bright-light) vision: vision with cones color receptors, high resolution in the fovea, less sensitive to light 3/10/2020 DIGITAL IMAGE PROCESSING 12 G3C-P.21 G3E-P.59 RODS There are 70 to 150 million cones in each eye. Rods are more spread out and are sensitive to low levels of illumination Distributed over the retina surface. Several rods are connected to a single nerve end. Rods don’t discern fine details. Rods give a general picture of the field of view. Give overall picture with reduced detail. Not involve in color vision and sensitive to low levels of illumination. Rod vision is called scotopic or DIM-LIGHT VISION. Scotopic (dim-light) vision: vision with rods color blind, much more sensitive to light (night vision), lower resolution 3/10/2020 DIGITAL IMAGE PROCESSING 13 G3C-P.21 G3E-P.59 2002) ( Inside(3e) Inside(3e) Figure Sensitivity of rods and cones based on measurements by Wald. 3/10/2020 DIGITAL IMAGE PROCESSING 14 Images taken from William K. Pratt , Digital Image Processing: PIKS PIKS K. Image Processing: Pratt from , William taken Digital Images Pratt3E-P.27 Vision 2017) ( (3e) (3e) CVIPtools : Human and Computer Computer and Human : Analysis Applications with Applications Digital Image Processing and Processing Image Digital , Umbaugh Figure Relative Response of Rods and Cones. (a) Rods react in low light levels, scotopic from Scott E Scott from vision, but respond to only a single spectral band, so cannot distinguish colors, (b) Cones react only to high light intensities, photopic vision, and since there are three different types that respond to different spectral bands, they enable us to see color. The response Images taken taken Images functions3/10/2020 of the cones are the called the tristimulus curves, for “three stimuliDIGITAL IMAGE.” PROCESSINGUmbaugh3E15 -P.362 HEXAGONAL PIXEL Cone distribution on the fovea (200,000 cones/mm2) •Models human visual system more precisely •The distance between a given pixel and its immediate neighbors is the same •Hexagonal sampling requires 13% fewer samples than rectangular sampling •ANN can be trained with less errors 3/10/2020 DIGITAL IMAGE PROCESSING 16 MORE ON THE CONE MOSAIC The cone mosaic of fish retina http://www.nibb.ac.jp/annual_report/2003/03ann502.html Lythgoe, Ecology of Vision (1979) Human retina mosaic The mosaic array of -Irregularity reduces visual acuity for high-frequency signals most vertebrates is -Introduce random noise regular 3/10/2020 DIGITAL IMAGE PROCESSING A MOSAICKED MULTISPECTRAL CAMERA 3/10/2020 DIGITAL IMAGE PROCESSING 18 RETINAL PHOTORECEPTORS •Fovea : Circular indentation in center of retina, about 1.5mm diameter, dense with cones. •Photoreceptors around fovea responsible for spatial vision (still images). •Photoreceptors around the periphery responsible for detecting motion. •Blind spot: Point on retina where optic nerve emerges, devoid of photoreceptors. •Blind spot is the region of emergence of the optic nerve from the eye. •In the blind spot there are no receptors. •1.5 mm 1.5 mm square contain 337000 cones 5mm 5mm CCD imaging chip G3C-P.20-21 3/10/2020 DIGITAL IMAGE PROCESSING G3E19 -P.59-60 CONES VS. RODS 3/10/2020 Figure Distribution of rods and cones in the retina. DIGITAL IMAGE PROCESSING 20 DISTRIBUTION OF RODS AND CONES IN THE RETINA Degrees from visual axis (center of fovea) Except for the blind spot, the distribution of receptors is radially symmetric about the fovea. G3C-P.21 3/10/2020 DIGITAL IMAGE PROCESSING G3E-P.59-60 BLIND-SPOT EXPERIMENT Draw an image similar to that below on a piece of paper (the dot and cross are about 15 cm apart) Close your right eye and focus on the cross with your left eye Hold the image about 50 cm away from your face and move it slowly towards you The dot should disappear! 3/10/2020 DIGITAL IMAGE PROCESSING 22 EYE OF OTHER ANIMALS 2011) ( Spider eyes consist of multiple sets with many individual lenses and sensors, producing comparatively low resolution but broad coverage. Bird eyes have high acuity and resolution. The chameleon can swivel its eyes independently to track different objects in left and right visual Processing Handbook(6e) Handbook(6e) Processing fields. The adult flounder has both eyes on one side of its head. The eyes of the Image octopus have very good color sensitivity but evolved with the neural circuitry on the opposite side of the retina from mammals. Primates are well adapted for Russ, The The Russ, stereo vision and have greater sensitivity to red colors than most other mammals. R6C-P.56 Images taken from taken Images 3/10/2020 DIGITAL IMAGE PROCESSING 23 R6E-P.87 WHAT THE EYE TELLS THE BRAIN 2011) ( Processing Handbook(6e) Handbook(6e) Processing Image Russ, The The Russ, Figure The principal layers in the retina. Light passes through several layers of processing neurons to reach the light-sensitive rod and cone cells. The horizontal, bipolar, and amacrine cells combine the signals from various size regions, compare them to locate R6C-P.66 Images taken from taken Images interesting details, and pass that information on to higher levels in the visual cortex. 3/10/2020 DIGITAL IMAGE PROCESSING R6E24 -P.101 THE HUBEL AND WIESEL EXPERIMENTS 3/10/2020 DIGITAL IMAGE PROCESSING 25 CULTURAL DIFFERENCES ON WHAT WE SEE Cultural differences strongly affect what we see in an image. Many westerners fix their attention on one (or a few) objects that are in the foreground and/or brightly colored, and ignore the surroundings. Many Asians pay more attention to the overall scene and the background details, noting the presence of objects in the foreground but not devoting any special attention to studying or remembering their characteristics. And, of course, recognition of something in a scene that appears familiar to the observer strongly influences where attention is focused. R6C-P.57 3/10/2020 DIGITAL IMAGE PROCESSING R6E26 -P.86-88 IS THERE ANY ANIMALS DOES NOT HAVE OR DOES NOT DEPEND ON VISUAL ORGAN? Bats and dolphins use echolocation or sonar to probe the world about them. Pit vipers sense infrared radiation. Moles, living underground, trade sight for sensitive touch organs around their nose.