Practical Computer Vision: Theory & Applications Carmen Alonso Montes 23rd-27th November 2015 Contents Course Structure Basic concepts of Computer Vision Pixels Colour vs. Black&White Noise and linear filtering Image enhancement Some practical examples My first programme with images Image denoising Image enhancement Basic commands matlab Carmen Alonso Montes 2 Course Structure This course is an introductory course for students with very few practical knowledge on computer vision Resources available for students through BCAM website Course material: slides, exercises and examples What do you need for this course? The most important thing is your self-motivation! Programming: examples will be done in Matlab, and also, if needed, I will explain how to do it in OpenCV (C++) For those C++ or Phyton programmers, I suggest to use a proper code editor (eclipse, netbeans, …) Some examples of useful tools (not needed) but good to know Versioning: git, SVN Automatic documentation generation: Doxygen Project management: Redmine Carmen Alonso Montes 3 Course Structure: Contents Each class will last 2 hours and a half 50 min Theory (9:30-10:20) 10 min break 50 min Theory (10:30-11:20) 10 min break 30 hour practice (11:30-12:00) Each slide, will contain a blue box with the command in matlab, in case you want to try it during the lessons imread At the end of the slides, you will have a table with the commands used during the lessons, for a quick guide. Carmen Alonso Montes 4 Some good books Digital Image Processing, 3rd Ed. (DIP/3e) by Gonzalez and Woods, (2008) This is the bible in this domain Learning OpenCV, Computer Vision in C++ with the OpenCV Library, by Adrian Kaehler, Gary Bradski. Publisher: O’Reilly Media (2013) This is a really good book if you want to start learning with OpenCV. OpenCV offers also dedicated tutorials online, check its website Carmen Alonso Montes 5 What is Computer Vision? Computer vision is a scientific discipline that aims to extract information from images through artificial systems to produce numerical or symbolic information, e.g. in the forms of decisions. Acquisition, processing, analyzing, or understanding techniques are usually involved The image data can take many forms, photography, video frame sequences, views from multiple cameras, or multi-dimensional data from a medical scanner, etc. Carmen Alonso Montes 6 Optical illusions Your vision can be cheated There are plenty of examples on this, and big studies This only shows the complexity of the field Carmen Alonso Montes 7 Computer vision name conventions Image Processing: set of mathematical operations by using any form of signal processing to process the image: Emphasize Segment Resize Smooth, etc) Computer graphics: set of techniques to manually made the images from physical models, environments and "Kent K. Barnes / kentkb" www.kentkb.com. lighting (e.g. animated movies). Computer Vision: high-level image processing where the system is intended to figure out the contents of an image or a video (e.g. object recognition) Machine Vision: is the application of image processing within industrial environments, dealing with issues like: dust, fat, poor illumination conditions, real-time constraints, physical restrictions within the machines, etc. Carmen Alonso Montes 8 Computer Vision World Carmen Alonso Montes 9 Some Examples Carmen Alonso Montes 10 Computer Vision System: Let's start! Image acquisition → capturing the image or video with a physical device (video, medical device, camera, mobile, etc) Image processing (low level processing) → segmentation, tracking, enhancement, feature detection. Image understanding → interpretation of those regions/objects to figure out what is actually happening in the image (e.g. object recognition, spatial relationship, etc) Image recognition → face recognition, biometrics, optical character recognition Carmen Alonso Montes 11 Computer Vision Some Basic Concepts before start Carmen Alonso Montes 12 ● How does a computer “see” an image? A 2D image (I) is seen as a set of pixels I = f(x,y) A gray-scale image in matlab and in openCV is considered as a matrix. The typical pixel value range is from [0 − 255], being 0 black and 255 white. If we are in 3D, then instead of pixels we have voxels! Images extracted from: http://what-is-this.net/fr/definition/tomography; http://oivdoc95.vsg3d.com/content/111-what-volumeviz Carmen Alonso Montes 13 Colour in images In this course, we are going to focus only in grayscale images Any colour image can be easily transformed into grayscale. 2D images + RGB (Red-Green- Blue).colormap → usually green channel is selected (less noise) But, take care since they are not actually equal! rgb2gray Carmen Alonso Montes 14 Pixel Neighbourhood Neighbourhood 4-neighbours 4-diagonal neighbours 8-neighbours Adjacency for pixels p and q 4-adjacent if they have the same value and q is in the set N_4 (p). 8-adjacent if they have the same value and q is in the set N_8 (p). m-adjacent if they have the same value and q is in the set N_4 (p) OR q is in the set N_D(p) AND the set N_4(p) ∪ N_4(q) is empty Connectivity: Two pixels p and q are connected if they are neighbours and they gray levels satisfy a specific condition (e.g. in binary images, they must have the same value). So, two pixels are m-connected if a m-path can be drawn between them. Carmen Alonso Montes 15 Pixel Distances Pixel distances Euclidean distance: straight line distance City-block (Manhattan) distance: assumes that in going from one pixel to the other it is only possible to travel directly along pixel grid lines. Diagonal moves are not allowed Chessboard distance: assumes that you can make moves on the pixel grid as if you were a King making moves in chess The last ones are faster to compute than Euclidean distance D_m distance between two pixels is defined as the shortest m−path between the points. Chessboard Carmen Alonso Montes 16 First steps with your images Any computer vision challenge starts in an image Image quality is far away of your ideal You will need to do some prior work to adjust the quality of the image to your needs Some challenges you usually face: Poor/too bright illumination conditions Camera without right position Imaging distortion Dust, pollution, … Image artifacts Carmen Alonso Montes 17 Image Preprocessing The image usually does not meet your system requirements Computer Vision Engineers will need to process the image to remove, enhance or highlight all the elements they need. We refer to “image preprocessing” to all those basic image processing operations performed over the image to make it suitable for our system Image preprocessing usually refers to the following aspects Geometric operations Noise removal Image enhancement This step shall provide a clean/homogeneous/normalized image to the other steps of our system, regardless of the input device or the conditions when the image was taken. Thus, minimizing errors or issues generated during the acquisition stage Carmen Alonso Montes 18 Image Processing Are these two images similar? Geometric transformations I cannot see the image very well Noise removal Linear Filtering Smoothing techniques The quality of the image is not the expected one. What can I do? Image Enhancement I need to identify the object's edges Thresholding What can I do with the edges? Image Morphology Carmen Alonso Montes 19 Noise Carmen Alonso Montes 20 Variety of images: Noise the common factor Image noise is random variation of brightness or color information in images, and is usually an aspect of electronic noise that adds spurious and extraneous information. It can be produced by the sensor and circuitry of a scanner or digital camera. The signal to noise ratio (SNR) is a useful and universal way of comparing the relative amounts of signal and noise for any electronic system; high ratios will have very little visible noise whereas the opposite is true for low ratios Types of noise Gaussian noise Salt-and-pepper noise Shot noise or Poisson noise Film grain Carmen Alonso Montes 21 Gaussian Noise Gaussian noise is statistical noise which values are Gaussian-distributed. The probability density function p of a Gaussian random variable z is given by: where z represents the grey level White Gaussian noise: A special case where the values at any pair of times are identically distributed and statistically independent (and hence uncorrelated). Principal sources arise during acquisition e.g. sensor noise caused by poor illumination and/or high temperature, and/or transmission e.g. electronic circuit noise. It can be reduced using a spatial filter: mean (convolution) filtering, median filtering and Gaussian smoothing imnoise Carmen Alonso Montes 22 Salt & Pepper Salt-and-pepper noise is a form of noise presents as sparsely occurring white and black pixels. This noise can, e.g., be used to model defects in the CCD or in the transmission of the image. An effective noise reduction method for this type of noise is a median filter morphological filter imnoise Carmen Alonso Montes 23 Noise: Other Shot noise or Poisson noise is a . type of electronic noise which can be modeled by a Poisson process. Many images, such as those from radiography, contain noise that satisfies a Poisson distribution. The magnitude of Poisson noise varies across the image, as it depends on the image intensity. This makes removing such noise very difficult. Film grain or granularity is the random optical texture of processed photographic film due to the presence of small particles of a metallic silver, or dye clouds, developed from silver halide that have received enough photons Carmen Alonso Montes 24 Image denoising Image denoising Carmen Alonso Montes 25 Image Averaging Image averaging works on the assumption that the noise in your image is truly random.