Internet Engineering Dr. Marek Woda Multimedia and Computer Visualisation Part 2

Digital image

Lecture Overview

• , definition, acquisition • Image description • Fundamentals of image processing 1. Digital image A digital image is a two dimensional array: f = f ( x, y ) x = 0,1,2,..,N - 1; y = 0,1,2,..,M - 1 where f(x,y) - element of image (), N, M - image width and hight,

Element f(x,y) can have different sense e. g. - gray level, f ( x , y ) Î {0 ,1,...,S } - , f ( x , y ) = [r( x , y ) g( x , y ) b( x , y )] r , g ,b Î {0 ,1,...,S}

- color index, f ( x , y ) Î {0 ,1,...,S } 1.1. Digital

CCD sensor ADC lens converter CFA filter

memory DSP RAM , HD processor 1.2.1. Lens and Parameters: f = 20 - 300 mm, F = 2.0 - 8.0 shutter 30 s. - 1/16000 s. Additional functions: ZOOM AF (Auto-) Viewfinder: optical LCD display 1.2.2. CCD and CFA filter

CCD (Charge Coupled Device) •CCD is an analog converter •Energy of light is held as electrical charge in each photo sensor and converted to voltage

1969 - W. Boyle i G. Smith (Bell Labs) - designed the first CCD (data storage) 1974 - Fairchild Electronics – first CCD application to image aquisition (100 x 100 px) 2015 - Canon – 250 MP sensor (19 580 x 12 600 px) How does a CCD sensor work ?

Clock (serial timing)

Serial Shift Register data

Clock (parallel timing) light

Parallel Shifting CFA filter ()

A Color Filter Array (CFA), is a mosaic of color filters, placed over the sensors. Most popular mosaic scheme (G-R-G-B)

Bayer Filter RGBE filter CMYY filter CYGM filter CCD sensor main parameters - an example:

Digital camera Canon PowerShot G2

• producer - unknown • total number of - 4,1mln. • efective number of pixels - 3,9 mln. • dimensins - 1/1.8” • CFA filter- G-R-G-B • sensitivity (ISO rating) - 50, 100, 200, 400 Sensor size (matters!):

•Full Frame – 36 x 24mm •APS-H – 28.1 x 18.7mm •APS-C – 23.6 x 15.8 mm (varies) •Four Thirds – 17.3 x 13 mm •One Inch – 9 x 12 mm •1/1.7” - 7.6 x 5.7 mm •1/2.5” - 5.76 x 4.29 mm Other solutions of image sensors

• Super CCD array (Fujifilm 1999)

Sensor shape - octagon

• high sensitivity and dynamics, • better color reproduction, • miniaturisation • CMOS sensors

Feature CCD CMOS

Signal out of chip analog (voltage ) digital (bits)

Fill factor high moderate

Noise Level low moderate

Dynamic Range high moderate

Speed moderate to high higher

System Complexity high low • Foveon X3 sensor

Standard sensor,

Foveon X3 sensor 1.2.3. ADC (Analog to Digital Converter)

ADC is an electronic device that converts an input analog signal (voltage or current) to a digital number proportional to the magnitude of the voltage or current.

ADC resolution:

• 8 - bit, (256 intensity levels of pixel) • 10 - bit, (1024 levels) • 12 - bit, (4096 levels) 1.2.4. DSP (Digital Signal Processing) A digital signal processor (DSP) is a specialized microprocesor with an optimized architecture. DSP processor realizes following functions:

• color interpolation, • image enhancement and color correction, • image compression (JPEG), • computational photography algorithms. 1.2.5. Memory Card Secure Digital (SD)/ MultiMediaCard • small size • small energy consumption • capacity: 1MB to 4GB (SDXC: 32 GB to 2 TB )

Sony Memory Stick (MS)

• relatively slow (< 50MB/s) • capacity: 128 MB (Original), 32 GB (PRO), 2 TB (XC)

CompactFlash • two types: 43×36×3.3 mm (Type I), 43×36×5 mm (Type II) • capacity: 2MB to 512GB (CF5.0: up to 128 PB !!!) XQD • Dimensions: 38,5×29,8×3,8 • capacity: over 2TB • PCI Express as a data transfer interface • 500 MB/s read /write speed Kodak EasyShare C122 Price: 178 PLN

• sensor: CCD, 8.1 Mpixel (1/2.5-type) • max resolution: 3304 x 2480 • sensitivity: ISO 80, 100, 200, 400, 800, 1000 • lens: f/4.5; 35 mm equivalent: 38 mm • digital zoom : 5x (continuous) • optical zoom: n/a • : 1/8–1/1400 seconds • long time : 0.5–8 seconds • memory: 16 MB (internal), optional SD/SDHC card (maximum card size - 32 GB) • image file format: EXIF 2.21 (JPEG compression) • video capture: 4 GB maximum; VGA (640 x 480 @ 30 fps) • electronic flash: built-in • communication with computer: USB 2.0 full speed • dimensions: 89.9 x 61.8 x 28.6 mm • weight: 145g Canon EOS-1D X II Nikon D5 Price [USD] 5999 6499 Sensor [MP] 20,2 20,8 ISO range 100 - 51 200 100 - 102400 (expanded) (50 - 409 600) (50 - 3 280 000) Viewfinder spec 0,76x mag 0.72 x mag 100 % coverage 100 % coverage 20 mm eyepoint 17 mm eyepoint AF points 61 (41 cross-type) 153 (99 cross-type) Live view/video AF 'Dual Pixel' Contrast detection phase detection AF working range [EV] -3 - 18 -4 - 20 RGB metering sensor 360k pixels 180k pixels resolution LCD 3.2" 1.62M-dot touch- 3.2" 2.36M-dot touch- enabled enabled Burst rate [FPS] 14 • 12 (16 with mirror up) • (14 with mirror up) Buffer • Unlimited • Unlimited JPEG / RAW / • 170 • 200 RAW+JPEG • 81 • 200 Video DCI 4K/60p UHD 4K/30p HDMI Out 1080 8-bit 4:2:2 4K/30 8-bit 4:2:2 Headphone socket? Yes Yes Card format Compact Flash / CFast Compact Flash / XQD Battery life (CIPA) 1210 shots 3780 shots Dimensions [mm] 158 x 168 x 83 160 x 159 x 92 Weight [g] 1530 1405 2. Digital image description

- histogram function - discrete Fourier transform (DFT) 2.1. Histogram function

source image 256 x 256 x 256 Overexposure image, considerable number of pixels on level 255.

Underexposure image, a lot of pixels on level 6 and 7. 2.2. Discrete Fourier transform (DFT)

f ( x, y ) ® F( u,v )

x, y = 0,1,...,N - 1; u,v = 0,1,...,N - 1

f( x, y ) - an image,

F( u, v ) - discrete Fourier transform of f( x, y ) image.

1 N -1 N -1 F( u,v ) = å å f ( x, y )e -i 2p ( xu+ yv ) / N N x=0 y=0

u,v = 0,1,..., N - 1 inverse Fourier transform

1 N -1 N -1 f ( x, y ) = å å F( u,v )e i 2p ( xu+ yv ) / N N u=0 v=0 x, y = 0,1,..., N - 1

• F( u, v ) - complex function.

• N2 integer numbers (image), is transformed in N 2 complex numbers, that is 2*N 2 real numbers. Example 1: (entirely flat image)

an image (8 x 8 pixels)

F( u,v )

FFT amplitude (module) Examle 2: (two-dimensional rectangular pulse)

an image (8 x 8 pixels)

F( u,v )

FFT amplitude (module) Example 3: (a part of photographic image)

an image (8 x 8 pixels)

F( u,v )

FFT amplitude (module) 3. Fundamentals of image processing

Input image Processing Output image f(x,y) algorithm g(x,y)

Typical image processing operations:

• improvement of quality, • geometrical deformation, • color correction, • image segmentation, • HDR (High dynamic range) imaging, 3.1. Algorithm 1 – linear filter (Gaussian filter )

the pixel of input image f(x,y) the pixel of output image g(x,y)

y y

x x

2 4 2 4 8 4 1 2 4 2 g( x, y ) = åå m( x, y )× f ( x, y ) åå m( x, y ) x y mask of filter m(x,y) x y (example) Linear filter – result of filtration

input image f(x,y) output image g(x,y)

A Gaussian blur (Gaussian smoothing) 3.2. Algorithm 2 – nonlinear filter (median filter)

Algorithm:

The median is calculated by sorting the values of input image pixel and his surrounding pixels, from low to high, and then taking the value in the center.

median Median filter – result of filtration

input image f(x,y) output image g(x,y)

In the effect black points disappeared.