Chronolab Vision, Chronolab A.G. Zug Switzerland, 1994.

Introduction to Digital Image Processing and Analysis

Prof.dr.sc. Darko Stipaničev Faculty of Electrical Engineering, Naval Architecture UNIVERSITY OF SPLIT R.Boskovica bb, 21000 SPLIT, Croatia e-mail: [email protected]

Contents

1. What you can get from Digital Image Processing and Analysis 2. Digital Image Acquisition 3. Digital Image Fundamentals 4. Image Processing 4.1. Image Optimization during Acquisition 4.2. Digital Image Processing 5. Digital Image Analysis 6. Typical Applications of Digital Image Processing and Analysis 7. Further Readings 2 Darko Stipaničev

Preface

This short Introduction is designed to provide fundamental knowledge necessary to understand elementary principles of digital image processing and analyses and to give some remarks about where and how this technology be used, particularly in the field of light microscopy. Main principles of digital image acquisition, processing and analyses are given and illustrated with examples. The last Chapter provides an information resource for those who wish to obtain additional insight in this topic.

Introduction to Digital Image Processing and Analysis 3

1. What you can get from Digital Image Processing and Analysis

Digital image technology is rapidly developing, multidisciplinary technology which uses the knowledge from different fields to produce a system, which could be used in different areas of human activities, for example in medicine, biology, chemistry, engineering, industrial automatization and inspection, security, criminology. Digital image technology includes image acquisition, image digitalization which means its transformation into digital image, digital image processing and digital image analyses. This technology could be used in all those cases where decisions are based on qualitative or quantitative information extracted from the image.

One typical example is a particle classification, where particles on the image have to be classified based on an individual particle's morphological parameter (area, perimeter, circularity index or something similar). Input information to such system is image of various particles taken by video camera through some appropriate camera lens. Output information are distributions of particle's parameters. Moreover, the system could recognize each individual 4 Darko Stipaničev particle, if some knowledge about particle's typical parameters values are implemented into system's knowledge base. For material classification and recognition the information about texture could be used, pathological cells could be detected and recognized using information about color and shape, object irregularities on the production line could be identified comparing it with the regular one, in Introduction to Digital Image Processing and Analysis 5 order to exclude false products from the production line. All these tasks could be carried out today using sophisticated hardware and software, with a higher degree of speed and accuracy. It is important to know that the cost of such sophisticated and complex equipment is not any more so high as it was until recently. Couple of years ago those equipment were affordable only to well founded research laboratories, but today also small routine laboratories can afford and use all benefits of digital image processing and analyses.

The whole process of digital image application could be subdivided into three main activities:

• Image Acquisition which includes image detection and digitalization, • Image Processing which includes image restoration and optimization, and • Image Analysis which extracts information from the image, evaluates it and uses for various tasks.

Typical sequence of those functions are shown in Fig.1.

This sequence of functions will normally be used when carrying out a typical image analysis procedure. A lot of specific tasks could be "programmed" based on this sequence of functions. For example, it could be used for monitoring the pathological changes in organs by evaluating biological slices, differentiation of white blood cells in blood smear analysis, material analysis by detecting and measuring percentage of inclusions in metal specimen, fertility analysis by analyzing kinetics of sperms, angiography analyses from the sequence of x-ray images after a contrast medium has been injected, macroscopic analyses of bacterial colonies in Petri dishes, and the like.

Various individual functions could be linked together to form a complete image analyses task which could be then used for appropriate decision making, or which could be used for automatic formation of appropriate action. But to use optimally this new and amazing technology, the principles of image formation and acquisition, image processing and image analyses must be understood and known. The rest of this Introduction deals with some elementary information about these topics. For profound studying, a lot of information could be found in references mentioned in Further Readings.

6 Darko Stipaničev

Fig.1. Sequence of functions in digital image analyses procedure

Digital Image Analysis can:

• visually enhance and optimize an image, • prepare an image for quantitative analysis and measurement, • eliminate artifacts in an image, • automatically evaluate image analyses results; • it gives objective, quantitative image evaluation and measurements, • it increases accuracy of the measurements, • it introduces standardization of measurement procedures, • it increases speed at which the measurements can be carried out, • it allows that all measurements could be repeated and checked, because analyzed image could be easily stored, and • final measurement results could be automatically and easily evaluated, for example, using statistics and graphical representation.

Introduction to Digital Image Processing and Analysis 7

2. Digital Image Acquisition

Image acquisition procedure transforms the visual image of a physical object and its intrinsic characteristics into a set of numeric data which can be processed and analyzed by the processing unit of the system. The acquisition consists of five steps:

• Illumination, • Image formation and focusing, • Image detection or sensing, • Formatting camera analog electric signal, • Digitalization or transformation of analog electric signal into a set of numeric data suitable for processing in digital computer.

The block diagram of the image acquisition system is shown in Fig.2.

Illumination is a key parameter affecting the quality of digita image formation. The object whose image has to be analyzed has to be illuminated by its own source of illumination without influence of environmental illumination which mostly manifests as noise into the image data. When using with macro zoom lenses or light microscopes a stable source of cold halogen light is essential. Sometimes for some digital image applications in industry automation special methods of lighting are used, for example back 8 Darko Stipaničev lighting which enhances the object boundaries, strobe lighting which eliminates influence of ambient light, or structure lighting with a special pattern or grid used to facilitate object recognition.

Illumination

Microscope Video camera

Lens Video recorder Auxiliary (2nd) Macro lens Monitor Digitalisation unit Long distance Physical Image (Frame grabber) microscope Processing unit (computer) Computer Monitor

Fig.2. Block diagram of the image acquisition system

Image Formation and Focusing. The image of the object is focused on the sensing element of the video camera through an appropriate optic element. This could be ordinary video lens, macro or close-up zoom lens, long distance microscope or ordinary light microscope. The most important factors associated with optic system and video camera are magnification and depth of field. Magnification is a measure of the relative size of visual image of the object in a physical world to the size of image formatted on a sensor located in the detector plane in the camera. This magnification is usually called primary magnification and it is important for measurement calibration. Magnification of the video system is a measure of the relative size of object on the detector plane and object shown on the monitor, and overall magnification is a product of both. Depth of field is the space above and below the object plane where the optic system maintains the focus of the image within acceptable limits. Adjustable aperture of the camera is usually used to vary this depth of field. The smaller the lens opening the greater the depth of field, but the lower the amount of light transmitted to sensor.

Image Detection. Camera has an electro-optical device which converts electromagnetic radiation from the image of the physical object into an electric signal used by the vision digitalization unit.. The image is focused by lenses on the sensor in the camera. The sensor element, which could be video tube or solid Introduction to Digital Image Processing and Analysis 9 state sensor element, located at the sensor plane produces an analog electric signal representing the visible image. Today in most cases CCD (charge coupling device) solid state sensor are used. They are characterized by their diagonal size (1/3", 1/2" or 2/3"), number of pixel elements (typically 500 x 582 pixel elements for standard PAL cameras), sensitivity (usually between 2 and 25 Lux), and resolution (from 300 to 600 horizontal lines).This analog signal generated by sensor is then formatted according to one of video standards.

Formatting camera analog electric signal. Color cameras could have one of the following video outputs: a) RGB video output , b) Y-C (super VHS) video output, or c) Composite video output (NTSC or PAL). RGB output video signal is a fundamental output signal, because it is originally produced by pick-up device. Red (R), green (G), and blue (B) video signals are given separately on their own connector pins. Synchronization signal which determines the end of the line and the end of one image could be superimposed on one of color channels or given separately as horizontal and vertical synchronization. Y-C (S-VHS) output video signal has two outputs: luminescence signal Y and chrome signal C. Luminescence signal could be used as ordinary monochromatic, black and white signal and chroma signal carries information about color. Composite video output has only one output signal which carries information about luminescence, chroma and synchronization (VBS signal). American (NTSC) and European (PAL) standard differs considerably. American NTSC (National Television System Committee) is older (1953). It has 15.734 kHz horizontal synchronization frequency and 59.94 Hz vertical synchronization frequency. European PAL (Phase Alternation Line) has different synchronization frequencies (horizontal 15.625 kHz, vertical 50 Hz), but also it differs in a way how VBS output video signal is composed.

Digitalization. Digitalization means conversion of analog video signal to digital video signal suitable for processing digital computer. This process is usually performed by special unit called frame grabber. In PC (personal computer) based systems frame grabber is in most cases plugged in card which has to be installed in one of computers free bus slots. It accepts analog video signal, samples it, equalizes and converts in digital signal. Usually frame grabbers have their own video memory where this digital image is stored. 10 Darko Stipaničev

Processing of this digital image could be by main computer processing unit or special graphic processor sometimes built in on the frame grabber to enhance the processing power of the whole system. Digital image from the video memory could be displayed on computer monitor and/or on the additional 2nd auxiliary monitor through the special video display unit of the frame grabber.

During image acquisition some corrections and optimization could be performed. Primarily that is tuning of brightens, contrast and intensity, color correction by primarily colors red, green and blue or by hue, saturation and intensity. As these functions could be applied also in the phase of image processing we will explain them latter, because before their description some explanations of digital image fundamentals must be given. The only difference between image optimization during acquisition and during image processing is that in the first case analog electric video signal is changed, and in the second case digital image stored in video memory is transformed.

Introduction to Digital Image Processing and Analysis 11

3. Digital Image Fundamentals

An image can be a photograph, a painting, or even a dream, but in the world of computers, it is a collection of dots called "pixels". Such image is usually known as digital image and formally defined as an array of pixels whose values indicate the light intensity of the flux on the picture element represented by that pixel. Each pixel has its horizontal and vertical position and its value. Positions and value are nonnegative scalars whose range depends on the digitalization unit characteristics. For example, frame grabber supplied with CHRONOLAB Color Vision software produces digital image with 512 x 512 pixels (in ordinary mode), so x and y coordinates could be nonnegative scalars between 0 and 511. Also each frame grabber channel has 8-bit analog to digital conversion, so each pixel value could be between 0 and 255, 0 representing the darkest intensity, and 255 the lightest intensity.

Three different kinds of digital images could be distinguished: • Black and white image, • Gray value image, and • Color image.

Each pixel of black & white image, or sometimes called binary image, could have one of two possible values, for example for 0 or 255, but usually it 12 Darko Stipaničev is expressed by logical 0 and 1 values. This is the most simple image where objects are represented by their areas. Most of object measurements could be performed only on this kind of digital image. Binary image could be obtained from gray value image or color image by process called segmentation (see Image Analysis).

Pixel at location (0,0) Pixel at locaton (x,y)

y Intensity 255 Intensity 64 Intensity 125

Intensity 0

Input image Gray digital image x y

x Binary (b & w) digital image Col. dig. image Red plane Green plane Blue plane

Fig.3. Relation of input image elements and digital images elements

The pixel values of gray value digital image could be a nonnegative scalar from the range determined by analog-to-digital conversion unit. Typical range is from 0 to 255 (256 gray values), where 0 means black and 255 means white. Gray value digital images could be obtained from monochrom cameras or by color-mono conversion from color digital images. Subjective perception of gray values is not linear, and correspond to a curve known as gamma correction (for details see section Image Processing) . Color digital images have three image planes, each of them corresponding to one primary color: red (R), green (G), and blue (B). Each image pixel of color image is conceived of three color pixels: red pixel, green pixel and blue pixel, each of them having appropriate intensity value. Red, green or blue color planes have the same structure as gray value image plane (see Fig.3), Introduction to Digital Image Processing and Analysis 13 typically 256 intensities for each color. Combining various intensities of RGB pixels each image pixel could give one of 256 x 256 x 256 = 16,777.216 different color values. When defining value of color image pixel at location (x,y) a triplet must be given in the form (red, green, blue), for example (0, 255, 255), which means minimum intensity for red and maximal intensities for green and blue, and that corresponds to pure yellow hue. Color image pixel will seem as gray value pixel if all primary colors have the same intensity value: Black correspond to (R,G,B) = (0,0,0), white to (R,G,B) = (255, 255, 255), and for example (R,G,B) = (127, 127,127) will give some medium gray intensity.

Sometimes, it is more convenient to express the image pixel values not by primary colors red, green and blue, but by transformed values hue (H), saturation (S) and intensity (I). Hue corresponds to pure color information. The whole spectrum is covered by the hue values from 0 to 360 (red 0, green 120, blue 240). Saturation determines the color closeness to gray values. The pure color has saturation equal to 1 and the gray value has the saturation equal to 0. The intensity or luminance measures the overall light intensity. It could be between 0 and 255, or in relative intensity system between 0 and 1. The intensity of an object is independent of the intensity of the surrounding objects. The intensity is the objective measure of the light distribution in the image. The subjective measure of the light distribution in the image is the brightness or sometimes called apparent brightness. The brightness of an object is the perceived luminance and depends on the luminance of the surround. Two objects with different surroundings could have identical luminance but different brightness.

The RGB color space could be schematically shown by color cube (see Fig.4). Cube corners correspond to elementary colors (red, green, blue), their complements (cyan, magenta, yellow) and black and white. Gray values are located on main diagonal. The HSI color space could be schematically shown by double cones (see Fig.4). Bottom and top vertexes correspond to black and white, and gray values are located on line between them (saturation = 0). Pure colors are located on outer circles.

Each color image pixel value could be found inside RGB cube or HSI double cone as Fig.7. shows for one typical color image. If the image is gray value one, all pixel values lie on gray value lines. Pure colors are located on outer surfaces of RGB cube or HSI double cone.

14 Darko Stipaničev

One of the weakness of the RGB model, and partly of HSI model for specifying color image is its nonuniform nature; equal distance in the RGB color space does not generally correspond to equal difference in color perception. Because of that some other color models are purposed. One of them is YIQ color model of American NTSC standard for composite video signal, which is essentially a linear transformation of the RGB model with luminance information coded into the Y component and chrominance into I and Q.

White Saturatin lines Yellow White Hue circles Blue Magenta Green 0 1 Blue Red

Green Cyan

Black Red Gray value lines Intensity line RGB cube HSI double cone Black

Fig.4. Schematic representation of RGB and HSI space

This allows color video signal to be shown on black-and-white monitors by using Y component. The I parameter carries orange-cyan hue information, while the Q parameter carries green-magenta hues. Quite similar is a color model of European PAL color video standard.

Video memory for storing color digital images must be three times bigger than video memory for storing gray value digital images with the same number of intensity levels. For example, one gray value digital image 512 x 512 pixels with 256 levels of gray needs approximately 262 Kbytes of video memory, and color digital image needs 786 Kbytes of video memory. Also processing time is three times longer for color images, and because of that color digital video systems are more complicated and more expensive.

Color images described previously are known as true-color images. They need a lot of memory space for storing. To reduce storing memory space, Introduction to Digital Image Processing and Analysis 15 another kind of color images are proposed, and named color-mapped images. They have reduced color palette from 16,777.216 different colors to 256 different colors. This means that for storing each pixel value color-mapped images need only 8 bits of memory, and true-color images need three times more 24 bits. Color-mapped images need the same memory space as gray level images. For example, given before that is 262 Kbytes for 512 x 512 pixel image. Fig. A5. shows the principle of converting true-color images to color-mapped images. In true-color images each pixel value is given by red, green, blue triplet, and in color-mapped images by palette index (or sometimes called palette number). Each palette index has its corresponding color value which is shown on a display screen. When we have defined color palette and want to convert true-color image to color-mapped image, a special procedure looks for palette color which is the closest to true-image color. For example, on Fig.5 that is palette color with palette index 3.

True-Color Image Palette Color-Mapped Image y R G B

x Palette Index Pallete Index RGB value 3 (30, 32, 92) 3 Palette Value (31, 31, 95) Fig.5. Conversion of true-color image to color-mapped image

An example of color image, gray level image, and binary image is shown on color plates at the end of this introduction.

A subregion of the digital image is called a window or region of interest (ROI) and it is defined by four corners (x1, y1), (x1, y2), (x2, y1) and (x2, y2), as Fig.6. shows. All processing and analysis functions are performed only inside the ROI, if it is defined. This functions is often used in digital image processing and analysis, if it could be applied, what means, if the interesting image part is smaller than the whole image. Limiting the image processing and analysis to ROI 16 Darko Stipaničev will higher their speed, because all pixels outside the ROI are excluded from processing.

Another important term connected with digital image is digital image histogram. It is used a lot for various image processing and analysis tasks. Histogram is a statistical description of an image.

(x1, y1) (x1, y2) Region of interest (ROI)

(x2, y1) (x2, y2)

Fig.6. Definition of Region of Interest (ROI)

Mathematically it is a graphical presentation of the frequency count of the occurrence of each intensity in an image. The abscissa or x-axies of an histogram is the value of intensity levels and the ordinate or y-axis is the overall number of pixels in image having that intensity value. Histogram could be calculated from the whole image or from its region of interest (ROI). Gray value images have one histogram with gray levels on x-axis, and color images have three histograms, one for each color plane having that color intensity levels on x-axis. For 8-bit analog to digital conversion gray value range or each primary color range is from 0 to 255.

Instead of RGB histograms, color images could be specified with HSI histograms, too. Hue histogram has hue values on x-axis and number of pixels in image having appropriate hue value on y-axis .

Saturation histogram has saturation levels on x-axis, and number of pixels in image having appropriate saturation. Intensity histogram is similar to gray value histogram. Intensity levels are on x-axis, and number of pixels having appropriate intensity on y-axis. Fig.7. shows an example of RGB and HSI Introduction to Digital Image Processing and Analysis 17 histograms for the same image, together with pixels positions in RGB cube and HSI double cone.

Fig.7. Examples of RGB and HSI histograms with pixel locations in RGB cube and HSI double cone.

18 Darko Stipaničev

4. Image Processing

The term "image processing" refers to the procedures whereby the information contained in an image is altered, changed, usually to visually restore or optimize the image. Typical examples are correction of image sharpening caused by poor focusing, correction of lenses optical errors, correction of contrast, intensity or brightness, color correction, image structure enhancement to emphasize elements which are not easily seen in original image, subtraction of background noise and similar. Image processing prepare images for image analyze, both manual or automatic. This means that images could be analyzed even in cases this was previously impossible due to poor quality or to its complexity.

Today’s hardware allows implementation of a lot of sophisticated image processing functions, but it is much more important to get the original image as better as possible. The main emphasis must be on adjustment of illumination and image optimization during acquisition, and than to make, if it is necessary the final, small adjustments by digital image processing functions.

Introduction to Digital Image Processing and Analysis 19

4.1. Image optimization during acquisition

When images are recorded, optimization and enhancement procedure can be carried out while the image is being loaded into image memory. Main optimization procedures are: primary colors, brightness, contrast, saturation and sometimes hue correction. This image processing is performed on analog video signal in comparison with digital image processing which is performed on digital images stored in digital video memories.

By primary colors correction red, green or blue intensities could be emphasized or augmented. The effect is the same as the application of color filters in front of camera lens. In CHRONOLAB Color Vision software primary colors are corrected using range and offset function. They change analog video signal of appropriate primary color channel. By range function the range of analog video signal values is limited, so only a portion of that signal corresponding only to some color intensities is allowed to pass. The offset function determines which part of analog video signal is allowed to pass. Schematically range and offset are shown in Fig. 8.

Value Maximal value

Upper range limit

Offset Range

Lower range limit

Analog video signal

Minimal value

Time

Fig.8. Range and offset definition in color correction during acquisition

20 Darko Stipaničev

By carefully adjustment of color's range and offset it is possible to make image color segmentation and extract only those image parts which have appropriate color values.

Brightness correction makes image more lighter or more darker. It is used to correct incorrect illumination, but it is better to try to improve illumination system than to use this function. The lowest brightness correction gives completely black image, and the highest brightens correction gives the complete white image.

Contrast correction enhances or augments the difference in light intensities of two regions. Image is contrastive if this difference is bigger and less contrastive if it is smaller. Black and white (binary) image is a result of applying maximal contrast correction, and completely uniform gray image is a result of applying minimal contrast correction.

Saturation correction is used for color images only. High saturated image has high intensity colors, and low saturated image is quite close to gray value image. This function could be used if we have color camera, but we want to obtain gray value image .

Hue correction is sometimes applied and used to change hue palette of color image. By this adjustment red becomes green, green becomes blue and similar. The similar effect could be obtained by color range/offset correction.

Another possible image optimizations during acquisition are image averaging and image integration. Image averaging is used in cases when an image is corrupted by noise. Noise in an image is recognized by the intensity values of random image pixels that deviate from the real value. By image averaging random noise could be considerably reduced. Image integration is used for weak signals (e.g. dark field or florescent microscopy). In this procedure a number of individual images are summed, so the intensity of final image is much higher than it would be for the single image frame.

In some cases frame grabber is equipped with a device called look-up- table (LUT) which is used for image modification. Input LUT is located between analog to digital converter and video memory and it is basically an electronic switching device that enables the intensity of the image pixel to be converted into new value before storing in image memory. LUT is a Introduction to Digital Image Processing and Analysis 21 transformation table used to transform one intensity value into another. Color systems have three LUTs, one for each primary color. Output LUT is located between video memory and displaying unit. It is used for changing intensities before displaying it on auxiliary monitor. On such a way it does not affect values in video memory. Input and output LUTs are hardware devices, but they could be in the form of software, too. Monadic image processing operations or point-by-point operations are software LUTs. All point-by-point operations described below in chapter Digital Image Processing could be applied on hardware LUT, too.

4.2. Digital Image Processing

Digital image processing alters values of digital images stored in video memory in order to prepare them for image analysis. The processing of digital image data could be categorized into two main categories:

• Monadic operations which act on one image pixel value in one time moment, and • Dyadic operations which act on multiple pixel values of one image or of a few images in one time moment.

Monadic point-by-point operations

Monadic point by point operations, or sometimes called software LUT, involve the generation of the new digital image by modifying the pixel value at a single location based on a global rule applied to every location in the original image. The process involves taking the pixel value p(x,y) of a given (x,y) location, modifying it by simple linear or nonlinear operator F, and placing the new pixel value q(x,y) on the same (x,y) location. The process is repeted on the pixel value of the next location and continued over the whole image or its region of interest (ROI), which is now called processing ROI. The procedure is schematically shown in Fig.9.

Monadic point by point operations are the simplest or most elementary image processing operations. General rule applied to all image pixel intensities is:

22 Darko Stipaničev

Change the image pixel value from p to q, regardless to its location in image.

y y

Level = p(x,y) New level = q(x,y) x x q(x,y) = F [ p(x,y)]

Input image Output image

Fig.9. Monadic point by point operation

Point by point operation could be graphically represented by x-y diagram, where x-axis corresponds to input image intensities p, and y-axis to output image intensities q. Fig.10. shows one typical point by point operation called contrast enhancement.

255

Output intensities q

0 0 p1 p2 255 Input intensities - p

Fig.10. Contrast enhancement operator function and input image histogram

Contrast enhancement is a linear point by point operation, used to improve image quality when the image contrast is too weak. In that case Introduction to Digital Image Processing and Analysis 23 intensity values are not uniformly spread through all intensity range. They are limited to a portion of possible range. That could be easily noticed from input image histograms as Fig.10. shows. Input image histogram is used for calculation of clipping values p1 and p2. p1 corresponds to the darkest value in the image and p2 to the brightest value. Applying contrast enhancement operator, input image intensities between p1 and p2 are spread in output image over all intensity range. The dynamic range in output image that results from this calculation is distributed over the maximum possible intensity values of the system. For color images contrast enhancement with different clipping values for each primary color could be performed separately or the same clipping values calculated from the intensity histogram could be applied for all primary color planes. The effect of applying gray value and color contrast enhancement operator is shown on plates at the end of this Introduction.

Contrast enhancement is more powerful operation than simple contrast adjustment which can be performed by using similar operator function. By contrast adjustment image pixels value range is stretched or squeezed around midpoint. Fig.11. shows the difference between contrast enhancement and contrast adjustment. Clipping values of contrast enhancement could be any value, taking into account that p1 < p2. In contrast adjustment p1 and p2, are symmetrically positioned around midpoint of input intensities . For example, if we have 256 different input intensities (8-bit gray value input image) than midpoint intensity is 128. If p1 is chosen as 50, than p2 must be 128 + (128 - 50) = 206. Contrast adjustment increases contrast around midtowns, and contrast enhancement around any input intensity value. Also contrast adjustment has the possibility of contrast reduction. The operator function for that case is shown in Fig.11 b. Contrast increase (stretching) is used when an image looks too flat, without strong light or dark colors. Contrast reduction (squeezing) is used when lights and darks of an image are too extreme.

24 Darko Stipaničev

Contrast Output int. Output int.reduction

Contrast increase

Input intensities Input intensities a) b)

Fig.11. Difference between contrast enhancement a) and contrast adjustment b). Intensity and brightness adjustments are next quite often used point- by-point operations. They are applied when image is too dark or too light. Both of them are realized by shifting the original operator function. In the case of intensity adjustment the operator function is shifted up (to brighten an image) or down (to darken an image). Mathematically intensity adjustment is realized by adding or subtracting a constant positive value to each pixel input intensity. The brightness adjustment is also realized by shifting the operator function, but this time to the left (to brighten an image) or to the left (to darken an image). Of course value range is in both cases limited, for example to interval [0,255]. If original operator function is identity function, the result of applying intensity adjustment or brightness adjustment is the same, as Fig.12. shows. Plates at the end of this introduction show effects in such a case. If original operator function is not identity function then the effects of applying intensity or brightness adjustment will be different. Intensity adjustment will shift it in vertical direction and brightness adjustment will shift it in horizontal direction. Typical example of such operation is contrast adjustment. It is important to emphasize that after applying contrast, intensity or brightness adjustments, some input image intensities will be lost, which is not the case if they are adjusted during image acquisition. Therefore, if image needs intensity, brightness or contrast correction, it is better to apply them during image acquisition, then during image processing, but the best solution, specially for intensity or brightness correction, is to carefully adjust illumination.

Contrast enhancement is one of the most offend used image processing operation, but it is only a special case of the more general operation called clipping. Fig.12. shows clipping operator function. The difference between Introduction to Digital Image Processing and Analysis 25 contrast enhancement and clipping is that in the first case input image intensities lower than p1 and higher than p2, are deleted in output image. For clipping operation they are not deleted, but linearly transformed. Clipping is used when some intensity ranges have to be augmented, and some enhanced.

Third linear point by point operation quite often used is inversion. It inverts input image intensities according to operator function shown in Fig.12. Inversion is used to notice some image details, which could be easily noticed in original image. Inverted image looks like print negative. Double inversion gives original image.

Level reduction operation results in an output image which has a smaller number of intensity values than the number of intensity values of the input image. For example if we suppose that input image could have 256 different levels, output image, after applying level reduction operator from Fig.12., will have only 4 levels or four different intensity values. Special kind of level reduction operator is a threshold operator which gives binary output image with only two boundary intensity levels, black and white. Threshold interval operator extracts from input image only those intensities which belong to certain interval of intensities. Pixels with intensities belonging to interval are in output image displayed as white, and all other pixels as black. This operator is used in segmentation by thresholding (see Fig.13).

26 Darko Stipaničev

Fig.12. Typical point by point operators functions Midtone adjustment allows adjustment of midtones (middle gray scale or color values) discarding information at the end of brightness range (the shadow and highlights areas). Midtones enhancement (full line in Fig.12), lights the image midtones, expanding the details in the darken part of the image (the contrast of the shadow area is increased and the contrast of the highlight area is decreased at the same time). Midtoness suppression (dashed line in Fig.12), darkens the image midtones and expands the details in the highlights part of the image (the contrast of the highlight area is increased and the contrast of the shadow area is decreased at the same time).

Introduction to Digital Image Processing and Analysis 27

Shadows and highlights adjustments have similar effects as midtones adjustment, but their influence is limited to shadows (low input intensities, dark parts of input image) or highlights (high input intensities, light parts of input image). Shadows or highlights could be enhanced (full line in Fig.12) or suppressed (dashed line in Fig.12). Enhancement means that output image intensities in shadows will be lower (darker) and in highlights ranges will be higher (lighter) than corresponding input intensities. Darken means that output image intensities in appropriate ranges will be higher (lighter) or lower (darker) than corresponding input values. Enhancement of image shadows or highlights means that shadows or highlights will be more easily visible and bigger than in input image. Suppression of image shadows or highlights has the opposite effect.

Monadic point-by-point operation which has quite special importance is gamma correction. It is quite similar to midtones adjustment, but it refers to completely another matter. Gamma refers to the nonlinear transfer function of display screen. This means that the actual brightness on the screen is not in direct linear proportion to the values stored in video memory. Typical transfer function (relation between color values and screen brightness) is shown in Fig.13 a). In comparison with linear relationship, which is marked as gamma γ = 1, it is evident that this relationship is nonlinear. As the color becomes brighter, the display elements become saturated and output proportionally less light. The gamma value of an image is a measure of the nonlinearity of this color value-to-real brightness curve. Common screens have gamma values between 1.6 and 2.0 (typically 1.8). If we want to correct this nonlinearity and to obtain linear relationship between input image color values and display brightness, the input image must be changed by applying point-by-point operation inverse to display nonlinearity, as Fig.13 b) shows. Gamma value of their function must be less than 1 (typically from 0.4 to 0.8). Gamma alters the shape of image histogram. For γ > 1 it enhances midtones and for γ < 1 it suppress midtones, similar to midtones adjustment. Ordinary video cameras have usually been built in this correction, so in most cases, it is not necessary to apply gamma correction by software. Some other types of input devices, for example image scanners, have not been built in gamma correction. On display screen their image looks too dark and contrast. By applying appropriate gamma by software (typically 0.6) image becomes brighter and more pleasant for viewing.

28 Darko Stipaničev

Output Display ! >1 Brightness Values (Color ! Values =1 in ! =1 Video Memory ) ! <1 Color Values in Video Memory InputValues a) b)

Fig.13. a) Transfer function of typical video monitor. b) Gamma function which has to be applied to correct video monitor nonlinearity.

For some linear point-by-point operation, like brightness and contrast, theoretically it is better to have image which has not been gamma corrected. As most cameras have been build in gamma correction less than one, than in principle it is necessary to undo it by applying gamma bigger than 1 (typically 1.8), apply linear point-by-point operation and replay old gamma less than (typically 0.6). In practice, however, this procedure is too long, so all image processing operations, including linear point-by-point operations, are applied on image directly captured and frozen from video camera.

Dyadic point-by-point operations

Dyadic point-by-point operations use essentially the same procedure except that a new image is generated taking two or more input pixels and F is a function of multiple variables: q(x,y) = F[ p1(x1, y1), p2(x2, y2), p3(x3,y3),...]

Pixels p1, p2, p3, ... could either belong to two or more different images or to the same image.

In the first case the procedure is known as image arithmetic. Pixel positions of input images are the same for all input images (x=x1=x2=x3=..., y=y1=y2=y3=...), and F could be any mathematical operation: addition, subtraction, multiplication, division, maximum, minimum, logical AND, or, Introduction to Digital Image Processing and Analysis 29

XOR or any other function that can be devised. Care must be taken that the function contains an appropriate scaling factor to keep the magnitude of the output value within the intensity range to avoid an overflow, negative value or non-integer value. Image addition can be used to reduce the effect of noise in the data, because it can average the data in two input images. If one of the input images is constant, the result image will be lighter overall image which will appear as a shift in the image histogram. Image subtraction can be used to filter out differences in images, to detect changes between two images, to eliminate background influence and alike. If two images are taken in different times, than subtraction can be used to detect movement. Image multiplication is used to correct the sensor nonlinearities or to extract specific areas of an image by region of interest window. Image pixels are multiplied by window pixels which are 0 outside the window and 1 inside the window. Logical operations are particularly used for manipulating binary images.

As specially important applications of image arithmetic let us mention background subtraction and flat field correction. For a given temperature, all CCD cameras exhibit an offset, consisting of preamplifier bias and dark charge. The preamplifier bias appears as a constant level for a given readout rate. The dark charge generally has some structure due to small variations in pixel response. the combined effect of these offsets is a fixed background for the image. Sometimes it is necessary to remove this unwanted background from the image. To do this, the user must first to acquire background image using the same camera temperature, readout rate and exposure time covering the camera objective or switching off the microscope illumination. This background image must be saved in one of the video memories and then subtracted from the original image. Flatfielding is another powerful tool to correct for system response variations. System response nonuniformity is caused both by variations in the incident light source, which may not be uniform across the field of view, and by the response of the camera, which normally has small variations from pixel to pixel. To correct for these nonuniformites, the user first stores the image from a uniform light source. This image, minus the background, is the Flatfield image. Subsequent images are first background subtracted and then divided by the flatfield image to obtain flatfield corrected images.

In the second case of dyadic point by point operation, pixels of input images belong to the different location of the same image, and the procedure is accomplished by the convolution process. It is sometimes called spatial 30 Darko Stipaničev transformation or digital image filtering. The output image pixel value is basically obtained as a result of calculation with respect to its neighboring pixels values combined with values of appropriate convolution matrix. The most common convolution matrix size is 3x3 pixels. Its values determine the filter nature which could be highpass, lowpass, enhancing, suppressing and alike. Typical 3x3 convolution matrix has the following structure

!c c c $ 1 11 12 13 #c c c & k # 21 22 23 & c c c "# 31 32 33 %&

where k is a reduction constant and cij are convolution matrix values. The output image pixel value at location (x,y) is obtained as

1 3 3 q(x, y) = # # cij! p(x + i " 2, y + j " 2) k i =1 J =1 where locations (x-1,y-1) to (x+1,y+1) are neighboring locations to pixel at location (x,y)

". . . . .% $. p(x ! 1, y ! 1) p(x ! 1, y) p(x ! 1, y + 1) .' $ ' $. p(x, y ! 1) p(x, y) p(x, y + 1) .' $. p(x + 1, y ! 1) p(x + 1, y) p(x + 1, y + 1) .' $ ' #. . . . .&

Typical example is lowpass and highpass filtering which is used to reduce noise. Their convolution matrices is

!1 1 1$ " 0 !1 0 % 1 Lowpass = #1 1 1& Highpass = $!1 5 !1' 9 # & $ ' 1 1 1 0 1 0 "# %& #$ ! &' and, for example, output image pixel value at location (3,3) is

1 q(3,3) = 1! p(2,2) +1! p(2,3) +1! p(2,4) +1! p(3,2) +1! p(3,3) +1! p(3,4) +1! p(4,2) +1! p(4,3) +1! p(4,4) 9 [ ] Introduction to Digital Image Processing and Analysis 31

Fig.14. illustrates this procedure.

10 14 14 17 56 78 ...... 27 34 17 13 67 43 34 21 17 86 12 156 . . 43 . . . 5 98 32 65 32 87 ...... 21 65 87 32 43 34 ......

43 = (34 + 17 + 13 + 21 + 17 + 86 + 98 + 32 + 65) / 9

Fig.14. Illustration of convolution for lowpas filtering convolution matrix

The same procedure is applied for all input image pixel locations. It should be noted that 3x3 convolution or filtering could not be applied for pixel values of input image located on the image boundaries. Usually they maintain the same value on output image as they have on input image. Convolution filtering is important digital image processing procedure used quite a lot in image optimization, but also for image analyses. Some edge enhancement filters are used in edge extracting procedures and image segmentation. CHRONOLAB Color Vision in Image Processing option Users Filter offers 30 most important convolution matrices.

Low pass filters are usually known as smoothing filters. They average the image according to 3 x 3 neighborhood. It smoothes the image and blurs intricate details, somewhat like a photographer's soft-focus lens. Smoothing is used to blend contrast details and give to image a soft, blurred look. High pass or sharpening filter is more often used. Its effect is sharpening edges, but unfortunately it also increases the amount of noise in the image, making it appear more grainy. Fig.14. and plates at the end of Introduction show effects of applying smoothing and sharpening filters. Special mention should be made of the rank order operators, of which the minimum, maximum and median filters are particularly important. These filters are used for suppressing noise, eliminate artifacts and dilate or erode objects in gray or color images. Rank order operator principle is as follows: An input image array of, let us say 3x3 pixels, is used for processing on a way that the central element is replaced with the minimum, maximum or median value of all 9 pixels belonging to that array.

32 Darko Stipaničev

In quantitative image analysis the binary image is of great importance, because it is usually the input image for a morphometric evaluation of image objects. The segmentation process, described in details in Chapter 5 (Digital Image Analysis), is used to transform color or gray image in binary image. Sometimes after transformation, the binary image needs some kind optimization procedure which is known as binary image processing. The most important binary image operators are erosion and dilation.

In an erosion operation, a defined number of image points on the edges of the objects are erased, so the object which is in binary image white, becomes smaller. Dilation is the inverse of erosion, the edges of the objects expand and the white object becomes bigger. As it is the inverse operation of erosion, the dilation of the object results in an erosion of the background .

Erosion and dilation are usually combined in more complex binary image processing operations. Typical example is operation open, which carry out the erosion operation followed by a dilation operator. This procedure eliminates from the image all small white objects. The operation close is carried out in the opposite sequence, a dilation operation is followed by an erosion operation. By this operation all small black holes in objects are eliminated from the image. Open is used to clean the noisy binary image, and close to connect structures which have become separate, or to close small gaps and holes. By specific combination of erosion and dilation, edges of binary object could be extracted with various edges thickness.

Another important function is automatic find and eliminate specific binary structures, for example objects or holes smaller than a specific size. These structures are automatically filled in. Lot of these operations are based on spatial transformations or digital image filtering procedures with convolution matrices composed of 0 and 1 elements only. By specific 0-1 pattern, a particular binary operation could be performed, for example: eliminate isolated points, eliminate triple points and alike.

The binary image functions described up to now belong to the binary image optimization group of functions. The next group is binary image combination functions. Logical operators AND, OR or XOR (exclusive or) in combination with inversion (complement) are used to link binary images. For example, by masking the original image a specific area could be suppressed from Introduction to Digital Image Processing and Analysis 33 further processing. In CHRONOLAB Color Vision these operations are used for image restoration after background subtraction.

Rank order operators, which are actually color and gray value operators, could also be suitable for binary image processing, particularly for binary image cleaning (noise suppression). Comparing to erosion and dilation these methods have the advantage that the size and the shape of the main objects are not greatly affected.

Examples of applying some dyadic point-by-point operations are given on plates at the end of this Introduction.

As a result of image processing is a digital image prepared and tuned for digital image analysis, which is the most important part, because it gives the information contained in the image.

34 Darko Stipaničev 5. Digital Image Analysis

There is a great difference between processing of digital image itself, described in details in Image Processing chapter and digital processing of image information, which will be described in this chapter. The latter is known as Digital Image Analysis. An image is analyzed when the information contained in the image is evaluated.

The final results of digital image analysis are normally numeric, such as image topological characteristics (number of separate objects in the image), geometrical characteristics of image objects (area, perimeter, circularity), densitometric characteristic and texture of image objects or image itself, but the result of digital image analysis could be also another image with identified image structures (edges, boundaries, regions, objects). This second kind of digital image analysis is usually performed before quantitative image evaluation. Digital image analysis usually consists of a number of procedures which first prepare the image for quantitative evaluation, and after that the image is quantitatively analyzed. For example, for certain geometric or topological image analyses, color or gray scale image is first converted in binary image, using one of segmentation methods, and after that, when objects are marked as white areas in comparison with background which is black, the image is quantitatively analyzed and evaluated. Some other image measurements, like measurement of distances and angles or single object characteristics, when its boundaries are manually or automatically traced, do not need image transformation into binary one. They can be performed on original, color or gray scale image. The same situation is with measurement of image luminescence features, like luminescence profiles. They don't need any kind of transformation because in that case original luminescence information will be lost.

Digital image analysis could be interactive manual, interactive automatic or fully automatic. The interactive, manual or semi-automatic digital image analysis is used in applications which are not labor or time intensive, or when fully automatic image analysis procedures are so complex that the effort in preparing them is not economically viable. Interactive image analysis takes the advantage of the user’s experience in recognizing even the most complex structures. The user applies all analysis procedures one by one, for example edge extraction - segmentation by edges - binary image optimization - objects measurements. The time required for these measurements depends on the skill and knowledge of the operator. In interactive manual analysis measurement are Introduction to Digital Image Processing and Analysis 35 performed by tracking contours of objects with a cursor on the screen. The measurement and evaluation are supported by quick measuring algorithms, data processing and data management. In interactive automatic analysis user interactively apply all image processing procedures: image optimization, image segmentation, binary image optimization, but when the image is ready for measurement, the whole image field is automatically measured and evaluated. Typical examples are automatic object counting, object recognition, image field measurement and similar. For more rapid routine analysis, fully automatic image analysis could be applied. All steps, from image acquisition to data evaluation, are automatic. User only creates conclusions. Due to the complexity of the image this can be in lot of situations achieved only with great difficulty using economically realistic equipment. Therefore, a modular system offers an ideal solution to users who need to analyze images under various conditions. The basic configuration offers interactive automatic image acquisition and analysis, and if necessary it could be upgraded to a fully automatic image analysis system. CHRONOLAB Digital Video Microscopy is an example of such a system. In its basic configuration which includes basic CHRONOLAB Color Vision Ver 1.0 software, it offers all processing and analysis functions for, interactive manual or automatic application, but in its advanced version, a special macro language will be included. Using them a sequence of operations could be created for a particular image analysis tasks, and then performed automatically.

Before any measurements the calibration procedure have to be done. There are two kind of calibrations: geometric calibration and densitometric calibration. In geometric calibration a correspondence between original scene linear dimensions and digital image linear dimensions are defined. The elementary geometric element of a digital image is a pixel. Typical digital image has 512 pixels in horizontal and 512 pixels in vertical dimension. All digital image linear dimensions are primarily measured in pixels and each pixel of digital image corresponds to certain linear dimension of original image. The ratio between them is a function of overall system magnification, and it is determined during the calibration procedure. Calibration is usually performed separately for horizontal and vertical dimension, because digital image could be distorted if cameras pixel elements do not correspond to digital image pixel elements. Calibration consists of defining how many digital image pixels have horizontal or vertical line whose dimension is well known. Fig.A15. shows that schematically. A horizontal ruler is incorporated in original image, its real length is defined and connected with length of a line manually drawn over the ruler in 36 Darko Stipaničev digital image. If it is necessary, the same procedure could be done for vertical dimension. Transformation ratios could be previously defined if the system and its magnification is well known, for example in the case of light microscopy workstations.

Line whose real length is 5 mm and digital image length 120 pixels. Calibration ratio 5/120 = 0.042

5 0 5

0

Fig.15. The geometric calibration procedure. Calibration ratio 0.042 means that each pixel in horizontal dimension corresponds to 0.042 mm of real horizontal line. After geometric calibration, image linear features, angles and features of objects whose boundaries are traced and marked could be measured in real dimension.

Densitometric calibration is not used so often as geometric calibration. Usually it is used for gray level images when image texture properties described by image luminescence are in direct relationship with certain object features. Typical example is analysis of transparent object which is illuminated by transillumination. This calibration could be either linear or nonlinear. In linear case it is enough to assign to one intensity level a corresponding calibrated value, because the intensity range is defined by hardware (for example 256 intensity levels for 8 bit analog to digital conversion). In nonlinear case for each intensity level a corresponding calibrated value has to be assigned. At the end of this chapter more details about object texture properties and its measurement are given. It has been mentioned that before measuring object geometric properties the analyzed object has to be marked, either by tracing its boundaries, or by separating object from its background by appropriate segmentation procedure. So, segmentation could be defined as a process of separation of the image into objects (to be measured) and background. It is the most important step between "digital image processing" and "digital image analysis", specially in the Introduction to Digital Image Processing and Analysis 37 automatic image analyze. The result of a segmentation is usually a binary, two- valued, black and white images in which the objects are normally displayed in white and background black.

There is no single approach to segmentation and there are many different ways of identifying objects in the image. Thresholding or color/gray value discrimination is the most common procedure used. One or more threshold levels are used to determine in which intensity range the object to be measured lies. Everything else is defined as background. If the image is gray level image, pixels are separated by different gray level values. In the case of the color image the situation is more complex. Pixels could be separated according to their primary color values: red, green and blue, or according to their hue, saturation and intensity values. Input color image has three planes, and output image after segmentation one black-and-white plane. Segmentation is performed in each input color plane, and final results have to be connected. As connection operator either AND or OR could be used. For example, using “AND” and one threshold for each color, only those pixels whose red value is higher than red threshold AND green value is higher than green threshold AND blue value is higher than blue threshold are separated as "white objects". Fig.16 shows that schematically. For connective OR pixel at location (x,y) belongs to object if, at least one color value at that location is higher that defined threshold.

Black/white binary image Color digital image Red plane Green plane This pixel is white if pixel values of all three Blue plane prinary colors are higher then threshold.

Fig.16. Segmentation of color image by thresholding of primary colors and connection AND

Segmentation by thresholding is actually performed by point-by-point operation called “threshold interval”. For each color image plane two threshold levels are defined: low threshold and high threshold. The principle is that input 38 Darko Stipaničev image pixels whose values lie between thresholds are considered as "white objects", and "black background" are all other pixels whose value is lower than the low threshold or higher than the higher threshold. Fig.17. shows “threshold interval” operator function. For final binary image all three color planes have to be combined by appropriate connection.

Fig.17. Point-by-point operator “threshold interval” which is used in segmentation by thresholding Color segmentation by hue, saturation and intensity has some advantage in comparison with red, green, blue segmentation. It is easier to identify object hue or intensity than object primary colors. If someone wants to extract objects by their hues, connection AND must be used, thresholds of hue plane have to be adjusted according to operator needs, low thresholds of saturation and intensity have to be set to zero and high thresholds of saturation and intensity have to be set to maximal value (usually 255). This means that all image pixels according to saturation and intensities are included in "white objects", but only those pixels whose hues lie between defined low and high threshold. An example of blood smear image segmentation could be seen on color plates. Gray image segmentation by thresholding is not so complicated and corresponds to color segmentation by intensities only.

Introduction to Digital Image Processing and Analysis 39

Input Image Threshold Interval Output Binary Image

Output

p1 Input p2

Output

p1 Input p2

Output

p1 p2 Input Fig.18. Threshold interval adjustment for image with dark and light background and for extraction middle gray level

A prerequisite for segmentation by thresholding is that the objects can be distinguished from the background. For example, if segmentation is performed according to objects intensities or brightness, they could be extracted only if they are either light objects on a dark background or vice versa, dark objects on the light background. In the first case, when objects are lighter than background, the low threshold is used to extract objects from its surrounding. In the second case, when objects are darker than background, the high threshold is used to extract them. Fig. 18. illustrates that schematically. It also shows the third possible case when certain middle gray intensities have to be extracted. Plate AXXI shows an image segmented by thresholding.

If global threshold value procedure is insufficient for separate objects from its surrounding, one of special segmentation procedures could be used. For example, when the intensity does not indicate whether an area contains objects or background, by detecting edges, objects contours or boundaries could be extracted and image segmented. This segmentation is known as segmentation based on edges and boundaries. Third useful segmentation procedure is segmentation based on regions. 40 Darko Stipaničev

An edge is a boundary between two regions of different gray level or color. The edges could be enhanced using some high pass convolution filter, for example Laplace edge enhancement filters whose convolution matrices are given in Fig.A19.

Laplacian_ convolution_ filters " 0 !1 0 % " 1 !2 1 % $ 1 4 1'...$ 2 4 2' $! ! ' $! ! ' #$ 0 !1 0 &' #$ 1 !2 1 &' Pr ewitt_ and _ Sobel_ horizontal_ filters "!1 0 1% "!1 0 1% $ 1 0 1'...$ 2 0 2' $! ' $! ' #$!1 0 1&' #$!1 0 1&' Pr ewitt_ and _ Sobel_ vertical_ filters "!1 !1 !1% "!1 !2 !1% $ 0 0 0 '...$ 0 0 0 ' $ ' $ ' 1 1 1 1 2 1 #$ &' #$ &'

Fig.19. Convolution filters for edge enhancement and edge extraction

For image segmentation edges have to be not only enhanced, but extracted. Appropriate convolution filters for edge extraction, mostly used in practice, are Sobel and Prewitt filters for horizontal and vertical edge extraction. Their convolution matrices are also given in Fig.19. In edge extraction procedure horizontal and vertical filters are combined together to extract all edges. By defining appropriate threshold level operator can choose what kind of edges he wants to display. For low threshold level, all edges, strong and weak will be displayed, and for high threshold level only strong edges will be displayed. Plate AXVIII shows edge extraction by Sobel filter for two different threshold levels. Segmentation by edges and boundaries gives as a result black and white (binary) image where white areas correspond to object edges, and black areas, both to objects exterior and objects interior. Because of that, they are not so appropriate for extracting objects which have to be measured. They are more appropriate for qualitative estimation of image content. Plate AXXII shows an image segmented by edges and boundaries.

Introduction to Digital Image Processing and Analysis 41

Regions are image parts which have similar gray or color value. Similar means that inside one region gray or color values could differ a little. This difference could be usually defined as a gradient threshold. Region extraction procedure equalizes gray or color value inside one regions. The result is the image with regions with uniform gray or color values. Plate AXIX shows an image with region extraction.

Segmentation by regions is usually performed on gray level images, so color images have to be converted into gray level images first. After that a threshold gray level intensity is defined. All regions whose gray value is below defined threshold are treated as "background" and transformed in black, and all regions whose gray value is behind defined threshold are treated as "objects" and transformed in white. This procedure could be directly applied to extract objects which are lighter than background. But if objects of interest are darker than background, after or before the image segmentation by regions image inversion has to be done. Plate AXIII shows an image segmented by regions.

Let us mention also some advanced segmentation procedures. Adaptive thresholding segmentation is one of them. It is combination of segmentation by thresholding and edge extraction. In ordinary thresholding segmentation the same threshold value is applied for the whole image (global threshold level). In adaptive threshold segmentation a new threshold value is calculated for each pixel which is supposed to belong to the object edges are detected by some edge extraction procedure.

If there is no possibility to extract objects from their background, by any of segmentation procedure, manual or automatic tracing could be used. Manual tracing is simple procedure where the operator marks objects boundaries using visual feedback. Automatic tracing is more complex. It consists of automatic searching for closed contours whose pixel has the same intensity value. That value is called threshold for tracing. Its determination is the most difficult part of automatic tracing, and usually needs some experimentation. Problem with automatic tracing is that it usually detects not only objects boundaries, but also a lot of regions inside or outside those objects. Because of that automatic tracing must be carefully used, specially if image analysis is completely automated.

A binary image, obtained as a result of original image segmentation, usually needs some binary image optimization using binary image processing 42 Darko Stipaničev functions described in Chapter 4. Before the actual quantitative image analyses and measurements can be carried out, the user must be sure that the image objects and structures have been correctly identified. Procedures of binary image processing usually have special option for interactive and automatic object deleting. Interactive deleting is performed by point-and-click procedure, which means that the operator clicks on object to be deleted. Automatic deleting is usually specified by maximal area of objects which have to be deleted, so all objects whose area is smaller than specified one, are automatically deleted. These functions are often used to eliminate small objects which appear on the binary image as a result of nonideal segmentation procedure.

Once a binary image has been optimized, the object measurement functions can be carried out. There are two main approaches to measuring objects geometric properties. According to one, the whole image field is analyzed and described with one set of parameters. Typical example is objects statistical characterization and counting. The output result is a number of objects in the image and their statistical properties, for example min, max and mean object's area, its distribution, standard deviation and alike. Fig.20. shows typical result for binary image from the Plate AXXIV. By point-and-click operation histogram of each parameter could be shown, as Fig.30. shows.

The second approach for object measurement is when each object is separately measured. The specific object could be separated from the others by - tracing its boundaries on the original, color or gray level image, - separating the object on binary image by region-of-interest, or - automatically scanning the whole binary image and labeling each object by its number for later analyses of each image object.

Introduction to Digital Image Processing and Analysis 43

Fig.20. Objects group parameters for image from the Plate AXXIV

Typical objects geometric parameters interesting for measurement are: - area, - perimeter, - horizontal width (Feret 0), - vertical height (Feret 90), - Ferets in various directions, typically 22.5°, 45°, 67.5°, 112.5°, 135° and 157.5°, - x and y centroids, - minimal radii from centroid to object's boundaries (radii of inner circle), - maximal radii from centroid to object's boundaries (radii of outer circle), - average radii from centroid to object's boundaries, - radial boundary - gradient representation of radii according to its orientation, - radii of equivalent circle derived from area measurement, - measure of circularity (ratio between squared perimeter and area multiplied by 4Π, circle has minimal value equal to 1, square 1.27, triangle 1.65), - measure of eccentricity (ratio between max and min radii, circle has min value 1), 44 Darko Stipaničev

-object’s length (maximal Feret measured in 8,32 or 64 directions), -object’s breadth (minimal Feret measured in 8,32 or 64 directions), - object’s orthogonal length (Feret orthogonal to object’s length).

maximal radi vertical height major axis direction minimal radi Feret 22,5º centroid minor axis direction area Feret 112,5º hole area

Feret 90º

hole centroid

orientation Feret 0º horizontal width object's ortogonal length object's length

Radi

Max

Min Angle

0º 360º Radial Boundary

Fig.21. Definition of object's geometric parameters

Object’s length and orthogonal length define forming rectangle, - object’s orientation (angle of object’s length), - orientation of object’s breadth, - aspect ratio (object’s length/object’s breath), - number of holes in object, - total hole area, - ratio between hole area and object area. Introduction to Digital Image Processing and Analysis 45

Equivalent circle has the same area as object. Forming rectangle is object's outer rectangle whose one pair of sides is object’s length and another pair is object’s orthogonal length. Centroide's coordinates x and y are usual defined according to upper left image corner. Feret in certain direction is object’s maximal width in that direction.

Some of object's measurement parameters are shown in Fig.A21. All of them have not the same importance. Some are more important than the others, for example area and perimeter, but generally each application has its parameters of importance.

The main computer measuring unit is pixel, but if system was previously calibrated the measuring results could be in any defined units. Fig.22. shows detail measuring results of object No.12 from the Plate AXXV.

Fig.22. Typical object's geometric parameters values for object No.12 from Plate.AXXV. Measuring units in this example were pixels. That means that object covers 2432 pixels and its perimeter has 190 pixels and 0.953 parts of one pixel. 46 Darko Stipaničev

The perimeter length is in fraction of pixel because some neighborhood pixels on object perimeter are located diagonally and then it is necessary to make diagonal correction. Fig.23. shows that schematically.

background

7 6 4 5 3 2 1

perimeter pixels object

Fig.23. Principle of measuring object's perimeter

Neighbor pixels on perimeter could be located vertically, horizontally or diagonally. If they are located vertically or horizontally each pixel is count as 1, but for diagonal located pixel instead of 1, square root of 2 is used. For example, length of perimeter section shown in Fig.23. is:

1 + 1 + 1 + 2 + 1 + 2 + 2 = 8.2426 ! pixels

Another complex image analysis procedure, directly connected with measurement of object geometric parameters, is object recognition based on evaluation of its geometric features . Plate AXXVI shows one typical example. Reference object is marked with "R", All objects, marked with "O" are "objects recognized by area and perimeter, because their areas and parameters are inside the interval [90 % reference value, 110 % reference value]. Upper and lower limit are defined by adjustable tolerance which is in this case 10 %. Fig.24. gives statistic results of 12 "recognized objects" including reference object.

In addition to geometric parameters, densitometric parameters or texture properties could also be defined. They have sense only for gray level and color images. In the case of color images, HSI color space is more appropriate than RGB one, particularly intensity part of HSI. Densitometric Introduction to Digital Image Processing and Analysis 47 parameters could be measured for a specific area defined by region-of-interest (ROI), or for specified object marked by its boundaries. Typical densitometric parameters are minimal, maximal and mean gray level or color intensity, their standard deviation and high level moments. Mean gray value or mean intensity could be treated as a measure of overall lightness or darkness, standard deviation as a measure of contrast and some higher order moments as a measure of asymmetry.

Fig.24. Statistical results of objects geometrical parameters for 12 recognized objects.

Luminance profiles have special position in densitometric measurements. They could be point, line or area profiles. Point profile is actual values of one pixel: red, green, blue, hue, saturation and particularly intensity. Line profile is graph of intensity or color values for a line defined in image. Fig.25. shows line profile of intensity values for a line drawn in Plate AXXVII.

48 Darko Stipaničev

Fig.25. Line profile of intensity values for a line drawn in Plate AXXVII

Projection signature or cumulative profile is similar to line profile, but it is connected with area defined by region-of-interest (ROI), and not with a line. Projection signature could be horizontal or vertical. Horizontal projection signature gives for each horizontal pixel of ROI, the average intensity (or color value) of all pixels in vertical column corresponding to that horizontal pixel. Vertical projection signature is similar, but it gives average intensity (or color value) in each horizontal line. Fig.A26. shows that schematically. Image matrix gives gray value intensities, and vertical column and horizontal row are projection signatures for specified ROI.

Fig.26. Example of projection signatures calculation Introduction to Digital Image Processing and Analysis 49

Projection signatures could be calculated for each color plane (red, green and blue), or what is more appropriate, only for image intensity. Fig.A27. shows one horizontal projection signature of intensity values for area specified by ROI on the Plate AXXVII.

Fig.27. Horizontal projection signature of intensity values for area specified by ROI No.1 on the Plate AXXVII

Area profile is also connected with an area defined by region-of- interest. It gives a three dimensional representation of pixel's intensity values. x- y plane correspond to image plane and z ax corresponds to image intensities, as Fig.A28. schematically shows

Image pixel at location (x,y) whose intensity value is 195 White 255 z

195 Intensity

y

Black 0 x 50 Darko Stipaničev

Fig.28. Principle of area profile drawing Fig.29. shows one real area profile of gray level image part specified by ROI on the Plate AXXVII.

Fig.29. Area profile of intensity values for area specified by ROI on the Plate AXXVII

When an image is greatly corrupted by noise, it is more convenient to extract regions before drawing line or area profiles.

Fig.30. Area histogram of objects from the Plate AXXIV

Introduction to Digital Image Processing and Analysis 51

One of advantages of using digital image analysis system is possibility of later measurement results evaluation. One simple evaluation task is statistical analysis of measurement results. Drawing histograms of geometric object features is typical example. Fig.A30. shows area histogram for objets from Plate AXXIV whose minimal and maximal values are given in Fig.A20.

More complex evaluation task is classification. Classification could be supervised or unsupervised. In supervised classification objects identified in image are separated and classified in known groups or clusters, previously defined. Cluster is a group of objects having similar pattern or patterns, for example area, perimeter, circularity, mean gray level, certain color value, or similar. One such example is object recognition, discussed in connection with Fig.24 and Plate AXXVI. In that case, only one cluster was formed. It was defined by area and perimeter of reference object. In unsupervised classification clusters are not known at the beginning of classification.

30

25

20

15

10

5

0 0 10 20 30 40 50

25

20

15

10

5

0 0 10 20 30 40 50

Fig.31. Position of particular objects in two dimensional feature space. Left: Two clusters could be easily formed. Right: Results are too scattered and clusters could not be formed 52 Darko Stipaničev

They are formed during classification, dividing objects in groups with characteristic features. Fig.31. shows two examples. The feature space was two dimensional.Typically area and object perimeter, or object mean gray level, and object area. In the first case two clusters could be easily formed, and in the second one it is impossible to do that, because results are too scattered. At the end let us emphasize that user’s conclusions in the form of the reduced statements like "pathological change has been noticed", or "grain sizes have fulfilled relevant standard" are derived directly from this measurement results or their evaluations.

Introduction to Digital Image Processing and Analysis 53

PLATE I - Original gray level image of blood smear and its intensity histogram. Intensity values between 0 and 255 are on x - axis and number of pixels on y - axis.

PLATE II - Contrast enhancement. Resulting image, input image histogram with applied point-by-point function and resulting image histogram. 54 Darko Stipaničev

PLATE III - Brightness increasing. Resulting image, input image histogram with applied point-by-point function and resulting image histogram.

PLATE IV - Brightness decreasing. Resulting image, input image histogram with applied point-by-point function and resulting image histogram. Introduction to Digital Image Processing and Analysis 55

PLATE V - Contrast increasing. Resulting image, input image histogram with applied point-by-point function and resulting image histogram.

PLATE VI - Contrast decreasing. Resulting image, input image histogram with applied point-by-point function and resulting image histogram. 56 Darko Stipaničev

PLATE VII - Shadows enhancement. Resulting image, input image histogram with applied point_by_point operation and resulting image histogram.

PLATE VIII - Shadows suppression. Resulting image, input image histogram with applied point_by_point operation and resulting image histogram. Introduction to Digital Image Processing and Analysis 57

PLATE IX - Middtones enhancement. Resulting image, input image histogram with applied point_by_point operation and resulting image histogram.

PLATE X - Middtones suppression. Resulting image, input image histogram with applied point_by_point operation and resulting image histogram. 58 Darko Stipaničev

PLATE XI - Highlights enhancement. Resulting image, input image histogram with applied point_by_point operation and resulting image histogram.

PLATE XII - Highlights suppression. Resulting image, input image histogram with applied point_by_point operation and resulting image histogram.

Introduction to Digital Image Processing and Analysis 59

PLATE XIII - Level reduction. Resulting image, input image histogram with applied point-by-point function and resulting image histogram.

PLATE XIV - Inverse. Resulting image, input image histogram with applied point-by-point function and resulting image histogram. 60 Darko Stipaničev

PLATE XV - Sharpening by convolution filter.

PLATE XVI - Smoothing by convolution filter. Introduction to Digital Image Processing and Analysis 61

PLATE XVII - Edge extraction by Prewitt filter with threshold level 23.

PLATE XVIII - Regions extraction. 62 Darko Stipaničev

PLATE XIX - Background subtraction.

PLATE XX - Automatic tracing with threshold 112. Introduction to Digital Image Processing and Analysis 63

PLATE XXI - Segmentation by tresholding. Resulting binary image, input image histogram and applied tresholding point-by-point function.

PLATE XXII - Resulting binary image of segmentation by regions.

64 Darko Stipaničev

PLATE XXIII - Resulting binary image of segmentation by edges with low treshold ( treshold = 50).

PLATE XXIV - Resulting binary image of segmentation by edges with high treshold ( treshold = 92). Introduction to Digital Image Processing and Analysis 65

PLATE XXV - Binary operation OPEN applied on binary black & white image from plate XXI.

PLATE XXVI - Binary operation CLOSE applied on binary black & white image from plate XXI. 66 Darko Stipaničev

PLATE XXVII - Automatic objects caunting : resulting image and measured data for calibration factor 1 (one pixel one unit).

PLATE XXVIII - Automatic objects recognition : resulting image and measured data for calibration factor 1 (one pixel one unit). Introduction to Digital Image Processing and Analysis 67

PLATE XXIX - Automatic field objects measurement : resulting image and measured data for calibration factor 1 (one pixel one unit).

68 Darko Stipaničev

PLATE XXX - Automatic single object measurement : resulting image and measured data for calibration factor 1 (one pixel one unit).

Introduction to Digital Image Processing and Analysis 69

PLATE XXXI - Line profile, area profile and projection signature.

70 Darko Stipaničev

PLATE XXXII - Superposition of text and graphics on image.

Introduction to Digital Image Processing and Analysis 71

PLATE XXXIII - Measurements of dental arcs: a) input a image, b) calibration data, c) distance measurements and d) b d output data. c

72 Darko Stipaničev

PLATE XXXIV - Analysis of electrophoresis: a) input image a with defined region-of-interest (ROI), b) cumulative b luminescence profile, c) output data after profile analysis.

c

Introduction to Digital Image Processing and Analysis 73

a PLATE XXXV - Starch grains image enhancement and b analysis : a) input image, b) image after image enhancement, c c) binary image after segmentation and automatic counting and

d d) area histogram of starch grains.

74 Darko Stipaničev

a PLATE XXXVI - Area histogram of blood cells: a) input image, b) binary image after segmentation, binary image b processing and automatic counting, c) measured data and area c histogram before separation d) measured data and area d histogram after separation. Introduction to Digital Image Processing and Analysis 75

c PLATE XXXVII - Unsupervised classification of grains: a) d a input image, b) image after segmentation and field b measurements, c) measure image d) calibration data and e e)correlation diagram between minimal and maximal radii from object's centroid. 76 Darko Stipaničev

PLATE XXXVIII - Detailed analysis of grains shape: a) a binary image of grains after automatic single object c b d measurements, b) measured data, c) radial boundary and d) f e Feret diagram of white paper grain, e) measured date, f) radial g boundary and g) Feret diagram of rice grain. Introduction to Digital Image Processing and Analysis 77

d e a PLATE XXXIX - Object recognition and analysis: a) input image, b) binary image after segmentation and object b recognition, c) measured data of recognised casts d) image of c measure and e) calibration data.

78 Darko Stipaničev

d a PLATE XL - Measurements of white blood cells nuclei: a) input image, b)image after color slicing in order to extract b white blood cells nuclei and measurement of one nucleus c e defined by ROI, c) image histogram and boundaries for color slicing, d) calibration data and e) measurement data.