Image Compression Compression Because of data sizes and perceptual issues, compression is typically applied to media data Compression may be lossless or lossy Lossless compression ◦ Data bits can be recovered exactly in the compressed version ◦ Decompressed file has identical bits in identical order to original file before any compression ◦ Example: zip files 2 Lossy Compression Some data bits cannot be recovered after compression ◦ Decompressed file has lost some bits or bytes compared to original file before any compression Goal: Discard data that doesn’t typically affect perception; ◦ Human perception of rendered decompressed data should be similar to perception of rendered data before compression It is best to apply lossy compression only once, at end of production of media, and if possible, work with uncompressed data throughout the production process ◦ i.e., Avoid compressing to lossy format, editing that format, then compressing again to lossy format, etc. 3 4 Raster File Formats Extension Name Notes .jpg Joint Photographic Experts Group Lossy compression format well suited for photographic images .png Portable Network Graphics Lossless compression image, supporting 16bit sample depth, and Alpha channel .gif Graphics Interchange Format 8bit indexed bitmap format, is superceded by PNG on all accounts but animation .tiff Tagged Image Flexible Format Lossless compression format (Lempel-Ziv-Welch – LZW) good for high-res images .exr EXR HDR, High Dynamic Range format, used by movie industry.. Direct memory dump from a digital camera, contains the direct imprint from the imaging .raw, .raw Raw image file sensor without processing with whitepoint and gamma corrections. Different cameras use different extensions, many of them derivatives of TIFF, examples are .nef, .raf and .crw A subset/clarification of TIFF, created by Adobe to provide a standard for storing RAW files, as .dgn Digital Negative well as exchanging RAW image data between applications. .psd Photoshop Document Native format of Adobe Photoshop, allows layers and other structural elements .xcf Gimp Project File GIMP's native image format. Lossless Compression Image Sampling Earlier we saw what sampling was in the context of audio Sampling In analog to digital conversion of an image what is a sample? ◦ Pixels: Small, often square, dots of color or grayscale which merge optically when viewed at a suitable distance to produce the impression of continuous tones 7 Image Resolution Resolution: ◦ Pixel dimensions of image; also: number of pixels that a device can display (render) per unit of length Examples ◦ My laptop display is about 13” (wide) by 8 ¼” (high), with a resolution of 1440x900 pixels (maximum), giving a resolution of about 108 pixels/inch Referred to as 108 dpi (dots per inch) or ppi (pixels per inch) 8 iPhone and iPad resolution Device Native Resolution (Pixels) UIKit Size (Points) iPhone X 1125 x 2436 375 x 812 iPhone 8 Plus 1080 x 1920 414 x 736 iPhone 8 750 x 1334 375 x 667 iPhone 7 Plus 1080 x 1920 414 x 736 iPhone 6s Plus 1080 x 1920 375 x 667 iPhone 6 Plus 1080 x 1920 375 x 667 iPhone 7 750 x 1334 375 x 667 iPhone 6s 750 x 1334 375 x 667 iPhone 6 750 x 1334 375 x 667 iPhone SE 640 x 1136 320 x 568 iPad Pro 12.9-inch (2nd generation) 2048 x 2732 1024 x 1366 iPad Pro 10.5-inch 2224 x 1668 1112 x 834 iPad Pro (12.9-inch) 2048 x 2732 1024 x 1366 iPad Pro (9.7-inch) 1536 x 2048 768 x 1024 iPad Air 2 1536 x 2048 768 x 1024 iPad Mini 4 1536 x 2048 768 x 1024 Typical Resolution Dimensions Megapixels Name Comment 640x480 0.3 VGA VGA Dimensions used for 720x576 0.4 CCIR 601 DV PAL PAL DV, and PAL DVDs PAL with square 768x576 0.4 CCIR 601 PAL full sampling grid ratio 800x600 0.4 SVGA The currently (2004) most common 1024x768 0.8 XGA computer screen dimensions. 1280x960 1.2 1600x1200 2.1 UXGA interlaced, high 1920x1080 2.1 1080i HDTV resolution digital TV format. Typically used for digital 2048x1536 3.1 2K effects in feature films. 3008x1960 5.3 3088x2056 6.3 4064x2704 11.1 Sampling When measuring the value for a pixel, one takes the average color of an area around the location of the pixel. A simplistic model is sampling a square, this is called a box filter, a more physically accurate measurement is to calculate a weighted Gaussian average When perceiving a bitmap image the human eye blends the pixel values together, recreating an illusion of the continuous image it represents. Sampling Grid png Sampling Depth (Quantization) 8bit ◦ A common sample format, 8 bit integers can only represent 256 discrete values (28 = 256), thus brightness levels are quantized into these levels. 12bit ◦ For high dynamic range images (images with detail both in shadows and highlights) 8bits 256 discrete values does not provide enough precision to store an accurate image. Some digital cameras operate with more than 8bit samples internally, higher end cameras (mostly SLRs) also provide RAW images that often are 12bit (212bit = 4096). 16bit ◦ The PNG and TIF image formats supports 16bit samples, many image processing and manipulation programs perform their operations in 16bit when working on 8bit images to avoid quality loss in processing. Sampling Depth (quantization) Image Compression Bitmap images take up a lot of memory, image compression reduces the amount of memory needed to store an image. For instance a 2.1 megapixel, 8bit RGB image (1600x1200) occupies 1600x1200x3 bytes = 5760000 bytes = 5.5 megabytes, this is the uncompressed size of the image. Compression ratio is the ratio between the compressed image and the uncompressed image, if the example image mentioned above was stored as a 512kb jpeg file the compression ratio would be 5.5mb: 0.5mb = 11:1. Typical Format Compression Description Ratios Lossless for images <=256 colors. Works best for flat color, sharp-edged art. Horizontally oriented GIF 4:1 - 10:1 bands of color compress better than vertically lossless oriented bands. Lossless for high res images. Uses LZW TIFF 2:1 compression. Not great choice for web due to large lossless file sizes. High quality - has little or no loss in image quality JPEG 10:1 - 20:1 with continuous tone originals. Worse results for (High) lossy flat color and sharp-edge art. JPEG Moderate quality - usually the best choice for the 30:1 - 50:1 (Medium) lossy Web. JPEG Poor quality - suitable for thumbnails and previews. 60:1 - 100:1 (Low) lossy Visible blockiness (pixelation). PNG's behave similarly to GIFs only better; they 10-30% smaller work best with flat-color, sharp-edged art. PNGs PNG than GIFs compress both horizontally and vertically, so solid lossless blocks of color generally compress best. Lossless Image Compression When an image is losslessly compressed, repetition and predictability are used to represent all the information using less memory. The original image can be restored. One of the simplest lossless image compression methods is run-length encoding. Run-length encoding encodes consecutive similar values as one token in a data stream. Digital Coding How would you encode this image? run length encoding RLE (monochrome) 20, 3, 1, 1, 11, 1, 1, 4, 10, 1, 1, 2, 1, 2, 9, 1, 1, 1, 3, 2, 8, 2, 5, 2, 6, 2, 7, 2, 4, 2, 9, 2, 2, 3, 9, 3, 3, 1, 1, 3, 1, 3, 1, 1, 2, 4, 1, 1, 1, 1, 1, 1, 1, 1, 7, 1, 1, 1, 1, 1, 3, 1, 7, 1, 1, 1, 1, 5, 7, 1, 1, 1, 1, 5, 35 run length encoding Run Length 20 RLE (monochrome) 3 1 There are 75 runs of length 1 or more 1 11 These could be coded 1 as integers 1 File size = 75 x 4 bytes 4 = 300 bytes 10 1 1 2 … run length encoding RLE (by row) There are now 78 values 16, 4, 3, 1, 1, 11, 1, 1, 4, 10, 1, 1, 2, 1, 2, 9, 1, 1, 1, 3, 2, 8, 2, 5, 2, 6, 2, 7, 2, 4, 2, 9, 2, 2, 3, 9, 3, 3, 1, 1, 3, 1, 3, 1, 1, 2, 4, 1, 1, 1, 1, 1, 1, 1, 1, 7, 1, 1, 1, 1, 1, 3, 1, 7, 1, 1, 1, 1, 5, 7, 1, 1, 1, 1, 5, 3, 16 16 run length encoding Run Length 16 RLE (monochrome) 4 3 There are 78 runs of length 1 or more 1 1 These could be coded 11 as integers 1 File size = 78 x 4 bytes 1 = 312 bytes 4 10 1 1 … Advantages of using smaller runs Storing a standard integer takes 4 bytes ◦ To store the integer 20 you need 4 bytes Storing a value from 1-16 can be done using only 4 bits! Value Bit configuration Value Bit configuration 1 0001 9 1001 2 0010 10 1010 3 0011 11 1011 4 0100 12 1100 5 0101 13 1101 6 0110 14 1110 7 0111 15 1111 8 1000 16 0000 Compression 1 integer per run = 75x4 = 300 bytes RLE (by row) – 4 bit/run = 39 bytes 0 4 20, 20 16*, 3 1 3, 1, 1, 11, 3 4, 3, 1, 1, 11, 1, 1, 4, 10, 1 11 1, 1, 4, 10, 1 1, 1, 2, 1, 2, 9, 1 1 1, 1, 2, 1, 2, 9, 1 1, 1, 1, 3, 2, 8, 1, 1, 1, 3, 2, 8, 2, 5, 2, 6, 2, 5, 2, 6, 2, 7, 2, 4, 2, 7, 2, 4, 2, 9, 2, 2, 2, 9, 2, 2, 3, 9, 3, 3, 9, 3, 3, 1, 1, 3, 1, 3, 1, 1, 2, 3, 1, 1, 3, 1, 3, 1, 1, 2, 4, 1, 1, 1, 1, 1, 1, 1, 1, 7, 4, 1, 1, 1, 1, 1, 1, 1, 1, 7, 1, 1, 1, 1, 1, 3, 1, 7, 1, 1, 1, 1, 1, 3, 1, 7, 1, 1, 1, 1, 5, 7, 1, 1, 1, 1, 5, 7, 1, 1, 1, 1, 5, 35 1, 1, 1, 1, 5, 3, 16 *16 coded as 0, since 16 0 is not otherwise valid Compression – example 2 If we assume that run lengths cannot exceed 256 (1 byte) RLE (by row) – 4bit/run = 39 bytes then1 byte per run = 75 bytes 0 4 20 16*, 20, 3 1 3 4, 3, 1, 1, 11, 3, 1, 1, 11, 1, 1, 4, 10, 1 11 1, 1, 4, 10, 1 1, 1, 2, 1, 2, 9, 1 1 1, 1, 2, 1, 2, 9, 1 1, 1, 1, 3, 2, 8, 1, 1, 1, 3, 2, 8, 2, 5, 2, 6, 2, 5, 2, 6, 2, 7, 2, 4, 2, 7, 2, 4, 2, 9, 2, 2, 2, 9, 2, 2, 3, 9, 3, 3, 9, 3, 3, 1, 1, 3, 1, 3, 1, 1, 2, 3, 1, 1, 3, 1, 3, 1, 1, 2, 4, 1, 1, 1, 1, 1, 1, 1, 1, 7, 4, 1, 1, 1, 1, 1, 1, 1, 1, 7, 1, 1, 1, 1, 1, 3, 1, 7, 1, 1, 1, 1, 1, 3, 1, 7, 1, 1, 1, 1, 5, 7, 1, 1, 1, 1, 5, 7, 1, 1, 1, 1, 5, 3, 1, 1, 1, 1, 5, 35 16 *16 coded as 0, since 16 0 is not otherwise valid Compression and Color Values RLE (by row) – 4bit/run = 39 bytes If you want to include color and 4 bits for color = 78 bytes total values you can easily do that.