Introduction to Graphics

Computer Science Tripos Part 1A/1B Michaelmas Term 2018/2019

Department of Computer Science and Technology The Computer Laboratory

William Gates Building 15 JJ Thomson Avenue Cambridge CB3 0FD

www.cst.cam.ac.uk

This handout includes copies of the slides that will be used in lectures. These notes do not constitute a complete transcript of all the lectures and they are not a substitute for text books. They are intended to give a reasonable synopsis of the subjects discussed, but they give neither complete descriptions nor all the background material.

Introduction to Graphics Michaelmas Term 2018/2019

2 What are Computer Graphics & Image Processing?

Scene Introduction to Computer Graphics description Peter Robinson & Rafał Mantiuk Computer Image analysis & www.cl.cam.ac.uk/~pr & ~rkm38 graphics computer vision Digital image Eight lectures & two practical tasks for Part IA CST Image Image capture display Two supervisions suggested Two exam questions on Paper 3 Image processing

3 4 Why bother with CG & IP? Course Structure All visual computer output depends on CG  Background ◆ What is an image? Human vision. Resolution and quantisation. Storage ◆ printed output (laser/ink jet/phototypesetter) of images in memory. [1 lecture] ◆ monitor (CRT/LCD/plasma/DMD)  Rendering ◆ Perspective. Reflection of light from surfaces and shading. Geometric ◆ all visual computer output consists of real images generated models. Ray tracing. [2 lectures] by the computer from some internal digital image  Graphics pipeline ◆ Polygonal mesh models. Transformations using matrices in 2D and 3D. Much other visual imagery depends on CG & IP Homogeneous coordinates. Projection: orthographic and perspective. ◆ TV & movie special effects & post-production Rasterisation. [2 lectures]  Graphics hardware and modern OpenGL ◆ most books, magazines, ◆ GPU APIs. Vertex processing. Fragment processing. Working with catalogues, brochures, meshes and textures. [1 lectures] junk mail, newspapers,  Colour and tone mapping packaging, posters, flyers ◆ Colour perception. Colour spaces. Tone mapping [2 lectures]

5 6 Course books Computer Graphics & Image Processing Fundamentals of Computer Graphics ◆ Shirley & Marschner Background th CRC Press 2015 (4 edition) ◆ What is an image? Computer Graphics: Principles & Practice ◆ Human vision ◆ Hughes, van Dam, McGuire, Sklar et al. ◆ Resolution and quantisation rd Addison-Wesley 2013 (3 edition) ◆ Storage of images in memory OpenGL Programming Guide: Rendering The Official Guide to Learning OpenGL Version 4.5 with SPIR-V Graphics pipeline ◆ Kessenich, Sellers & Shreiner Rasterization Addison Wesley 2016 (7th edition and later) Graphics hardware and modern OpenGL Colour and tone mapping

7 8 What is required for vision? The spectrum visible light is only a tiny illumination part of the whole electromagnetic spectrum ◼ some source of light objects ◼ which reflect (or transmit) the light eyes ◼ to capture the light as an image

the short wavelength the long wavelength end of the spectrum end of the spectrum direct viewing transmission reflection is violet is red

violet blue green yellow red

9 10 What is an image? The workings of the human visual system two dimensional function to understand the requirements of displays value at any point is an intensity or colour (resolution, quantisation and colour) we need to not digital! know how the human eye works... The lens of the eye forms an image of the world on the retina: the back surface of the eye

11 12 Structure of the human eye Light detectors in the retina

 the retina is an array of light two classes detection cells ◆ rods  the fovea is the high ◆ cones resolution area of the retina cones come in three types  the optic nerve takes signals ◆ sensitive to short, medium and long wavelengths from the retina to the visual ◆ allow you to see in colour cortex in the brain the cones are concentrated in the macula, at the centre of the retina the fovea is a densely packed region in the centre of the macula ◆ contains the highest density of cones ◆ provides the highest resolution vision

Fig. 2.1 from Gonzalez & Woods

13 14 Colour signals sent to the brain Mixing coloured lights

◆ the signal that is sent to the brain is pre-processed by the retina by mixing different amounts of red, green, long +medium + short = luminance and blue lights we can generate a wide range of responses in the human eye long – medium = red-green

long + medium – short = yellow-blue

◆ green this theory explains: green ◼ colour-blindness effects ◼ why red, yellow, green and blue are perceptually important colours blue red light light ◼ why you can see e.g. a yellowish red off fully on but not a greenish red red blue not all colours can be created in this way

15 16 What is a digital image? Sampling a contradiction in terms a digital image is a rectangular array of intensity ◆ if you can see it, it’s not digital values ◆ if it’s digital, it’s just a collection of numbers each value is called a pixel a sampled and quantised version of a real image ◆ “picture element” a rectangular array of intensity or colour values sampling resolution is normally measured in pixels per inch (ppi) or dots per inch (dpi) ◆ computer monitors have a resolution around 100-200 ppi ◆ laser and ink jet printers have resolutions between 300 and 1200 ppi ◆ typesetters have resolutions between 1000 and 3000 ppi

17 18 Image capture Image capture example  a variety of devices can be used 103 59 12 80 56 12 34 30 1 78 79 21 145 156 52 136 143 65 115 129 41 128 143 50 85 106 11 74 96 14 85 97 23 66 74 23 73 82 29 67 76 21 40 48 7 33 39 9 94 54 19 42 27 6 19 10 3 59 60 28 102 107 41 208 88 63 204 75 54 197 82 63 179 63 46 158 62 ◆ scanners 46 146 49 40 52 65 21 60 68 11 40 51 17 35 37 0 28 29 0 83 50 15 2 0 1 13 14 8 243 173 161 231 140 69 239 142 89 230 143 90 210 126 79 184 88 48 152 69 35 123 51 27 104 41 23 55 45 9 36 27 0 28 28 2 29 28 7 40 28 16 13 13 1 224 167 112 240 ◼ line CCD (charge coupled device) in a flatbed scanner 174 80 227 174 78 227 176 87 233 177 94 213 149 78 196 123 57 141 72 31 108 53 22 121 62 22 126 50 24 101 49 35 16 21 1 12 5 0 14 16 11 3 0 0 237 176 83 244 206 123 241 236 144 238 222 147 221 190 108 215 170 77 190 135 52 136 93 38 76 35 7 113 56 26 ◼ spot detector in a drum scanner 156 83 38 107 52 21 31 14 7 9 6 0 20 14 12 255 214 112 242 215 108 246 227 133 239 232 152 229 209 123 232 193 98 208 162 64 179 133 47 142 90 32 29 19 27 89 53 21 171 116 49 114 64 29 75 49 24 10 9 5 11 16 9 237 190 82 249 221 122 241 225 129 240 219 126 240 199 93 218 173 69 188 135 33 219 186 79 189 184 93 136 104 65 112 69 37 191 153 ◆ cameras 80 122 74 28 80 51 19 19 37 47 16 37 32 223 177 83 235 208 105 243 218 125 238 206 103 221 188 83 228 204 98 224 220 123 210 194 109 192 159 62 150 98 40 116 73 28 146 104 46 109 59 24 75 48 18 27 33 33 47 100 118 216 177 98 223 189 91 239 209 111 236 213 ◼ area CCD Heidelberg 117 217 200 108 218 200 100 218 206 104 207 175 76 177 131 54 142 88 41 108 65 22 103 59 22 93 53 18 76 50 17 9 10 2 54 76 74 108 111 102 218 194 108 228 203 102 228 200 drum scanner 100 212 180 79 220 182 85 198 158 62 180 138 54 155 106 37 132 82 33 95 51 14 87 48 ◼ CMOS camera chips 15 81 46 14 16 15 0 11 6 0 64 90 91 54 80 93 220 186 97 212 190 105 214 177 86 208 165 71 196 150 64 175 127 42 170 117 49 139 89 30 102 53 12 84 43 13 79 46 15 72 42 14 10 13 4 12 8 0 69 104 110 58 96 109 130 128 115 196 154 82 196 148 66 183 138 70 area CCD flatbed scanner 174 125 56 169 120 54 146 97 41 118 67 24 90 52 16 75 46 16 58 42 19 13 7 9 10 5 0 18 11 3 66 111 116 70 100 102 78 103 99 57 71 82 162 111 66 141 96 37 152 102 51 www.hll.mpg.de www.nuggetlab.com 130 80 31 110 63 21 83 44 11 69 42 12 28 8 0 7 5 10 18 4 0 17 10 2 30 20 10 58 88 96 53 88 94 59 91 102 69 99 110 54 80 79 23 69 85 31 34 25 53 41 25 21 2 0 8 0 0 17 10 4 11 0 0 34 21 13 47 35 23 38 26 14 47 35 23

The image of the Heidelberg drum A real image A digital image scanner and many other images in this section come from “Handbook of Print Media”, by Helmutt Kipphan, Springer-Verlag, 2001

19 20 Sampling resolution Quantisation 256256 128128 6464 3232 each intensity value is a number for digital storage the intensity values must be quantised ◼ limits the number of different intensities that can be stored ◼ limits the brightest intensity that can be stored how many intensity levels are needed for human consumption ◼ 8 bits often sufficient ◼ some applications use 10 or 12 or 16 bits ◼ more detail later in the course colour is stored as a set of numbers ◼ usually as 3 numbers of 5–16 bits each ◼ more detail later in the course 22 44 88 1616

21 22 Quantisation levels Storing images in memory 8 bits 7 bits 6 bits 5 bits (256 levels) (128 levels) (64 levels) (32 levels) 8 bits became a de facto standard for greyscale images ◆ 8 bits = 1 byte ◆ 16 bits is now being used more widely, 16 bits = 2 bytes ◆ an 8 bit image of size W  H can be stored in a block of W  H bytes ◆ one way to do this is to store pixel[x][y] at memory location base + x + W  y ◼ memory is 1D, images are 2D 5 base 4 3 2 5  1 base + 1 + 5  2 0 1 bit 2 bits 3 bits 4 bits 0 1 2 3 4 (2 levels) (4 levels) (8 levels) (16 levels)

23 24 Colour images The frame buffer

◆ tend to be 24 bits per pixel most computers have a special piece of memory ◼ 3 bytes: one red, one green, one blue reserved for storage of the current image being ◼ increasing use of 48 bits per pixel, 2 bytes per colour plane displayed ◆ can be stored as a contiguous block of memory ◼ of size W  H  3 B frame output U ◆ more common to store each colour in a separate “plane” buffer stage display S ◼ each plane contains just W  H values (e.g. DAC) ◆ the idea of planes can be extended to other attributes associated with each pixel the frame buffer normally consists of dual-ported ◼ alpha plane (transparency), z-buffer (depth value), A-buffer (pointer to a data Dynamic RAM (DRAM) structure containing depth and coverage information), overlay planes (e.g. for ◆ sometimes referred to as Video RAM (VRAM) displaying pop-up menus) — see later in the course for details

25 26 Computer Graphics & Image Processing Depth cues Background Rendering ◆ Perspective ◆ Reflection of light from surfaces and shading ◆ Geometric models ◆ Ray tracing Graphics pipeline Graphics hardware and modern OpenGL Technology

27 28 Rendering depth Perspective in photographs

Gates Building – the rounded version Gates Building – the rectilinear version (Stanford) (Cambridge)

29 30 Early perspective Wrong perspective Presentation at the Adoring saints Temple Lorenzo Monaco Ambrogio Lorenzetti 1342 1407-09 Uffizi Gallery National Gallery Florence London

31 32 Renaissance perspective More perspective Geometrical perspective The Annunciation Filippo Brunelleschi 1413 with Saint Emidius Holy Trinity fresco Carlo Crivelli 1486 Masaccio (Tommaso di Ser Giovanni National Gallery London di Simone) 1425 Santa Maria Novella Florence De pictura (On painting) textbook by Leon Battista Alberti 1435

33 Calculating 34 False perspective perspective

35 Ray tracing: examples 36 Ray tracing Identify point on surface and calculate illumination Given a set of 3D objects, shoot a ray from the eye through the centre of every pixel and see what surfaces it hits

shoot a ray through each pixel whatever the ray hits determines the colour of ray tracing easily handles reflection, refraction, that pixel shadows and blur ray tracing is computationally expensive

37 38 Ray tracing algorithm Intersection of a ray with an object 1

◆ plane select an eye point and a screen plane D N FOR every pixel in the screen plane O determine the ray from the eye through the pixel’s centre FOR each object in the scene ray:,PO= +sD s  0 IF the object is intersected by the ray IF the intersection is the closest (so far) to the eye plane: P N + d = 0 record intersection point and object d+ N  O END IF ; s = − END IF ; ND END FOR ; set pixel’s colour to that of the object at the closest intersection point ◆ polygon or disc END FOR ; ◼ intersection the ray with the plane of the polygon ⚫ as above ◼ then check to see whether the intersection point lies inside the polygon ⚫ a 2D geometry problem (which is simple for a disc)

39 40 Intersection of a ray with an object 2 Ray tracing: shading

◆ sphere = DDa ◆ once you have the intersection of a light 2 ray with the nearest object you can D C b = 2 ()− COD O r light 1 also: 2 ◼ calculate the normal to the object at :ray OP += sD s  0, c = ()()CO −− − rCO that intersection point 2 2 sphere: ( () ) rCPCP =−−− 0 d = b − 4ac ◼ shoot rays from that point to all of the +− db light sources, and calculate the diffuse and specular reflections off the object s1 = N 2a at that point d imaginary −− db ⚫ this (plus ambient illumination) d real s = D C 2 O r gives the colour of the object (at ◆ cylinder, cone, torus 2a that point) ◼ all similar to sphere ◼ try them as an exercise

41 42 Ray tracing: shadows Ray tracing: reflection

◆ because you are tracing ◆ if a surface is totally or light 2 rays from the intersection partially reflective then light 1 point to the light, you can new rays can be check whether another spawned to find the object is between the contribution to the

light 3 intersection and the light N 2 pixel’s colour given by and is hence casting a light the reflection N N shadow 1 ◼ this is perfect (mirror) ◼ also need to watch for self- reflection D C shadowing O r O

43 44 Ray tracing: transparency & refraction Illumination and shading

◆ objects can be totally or Dürer’s method allows us to calculate what part of partially transparent the scene is visible in any pixel ◼ this allows objects behind the current one to be seen through But what colour should it be? D it 2 Depends on: ◆ transparent objects can have light D ◆ lighting 1 refractive indices ◆ shadows ◼ bending the rays as they pass through the objects ◆ properties of surface material D'1 D0 O ◆ transparency + reflection D '2 means that a ray can split into two parts

45 46 How do surfaces reflect light? Comments on reflection

◆ the surface can absorb some wavelengths of light      ◼ e.g. shiny gold or shiny copper

perfect specular Imperfect specular diffuse reflection ◆ specular reflection has “interesting” properties at glancing angles reflection reflection (Lambertian reflection) owing to occlusion of micro-facets by one another (mirror)

the surface of a specular reflector is facetted, each facet reflects perfectly but in a slightly different direction to the other facets ◆ plastics are good examples of surfaces with: ◼ specular reflection in the light’s colour ◼ diffuse reflection in the plastic’s colour

Johann Lambert, 18th century German mathematician

47 48 Calculating the shading of a surface Diffuse shading calculation ◆ gross assumptions: ◼ there is only diffuse (Lambertian) reflection N L is a normalised vector pointing in the direction of the light source ◼ all light falling on a surface comes directly from a light source L ⚫ there is no interaction between objects  N is the normal to the surface ◼ no object casts shadows on any other Il is the intensity of the light source ⚫ so can treat each surface as if it were the only object in the scene kd is the proportion of light which is ◼ light sources are considered to be infinitely distant from the object I= Il k d cos diffusely reflected by the surface ⚫ the vector to the light is the same across the whole surface I is the intensity of the light reflected =Il k d () N  L ◆ observation: by the surface ◼ the colour of a flat surface will be uniform across it, dependent only on the colour & position of the object and the colour & position of the light sources use this equation to calculate the colour of a pixel

49 50   Diffuse shading: comments Specular reflection  Phong developed an easy-to- L is a normalised vector pointing in the ◆ can have different Il and different kd for different wavelengths direction of the light source (colours) calculate approximation to specular reflection R is the vector of perfect reflection ◆ watch out for cos < 0 N is the normal to the surface ◼ implies that the light is behind the polygon and so it cannot illuminate this N side of the polygon L R V is a normalised vector pointing at the ◆ do you use one-sided or two-sided surfaces? viewer ◼ one sided: only the side in the direction of the normal vector can be   V Il is the intensity of the light source illuminated  k is the proportion of light which is ⚫ if cos < 0 then both sides are black s specularly reflected by the surface ◼ two sided: the sign of cos determines which side of the polygon is illuminated n n is Phong’s ad hoc “roughness” coefficient I= Il k s cos  ⚫ need to invert the sign of the intensity for the back side n I is the intensity of the specularly reflected =Il k s () R  V ◆ this is essentially a simple one-parameter () BRDF light

Phong Bui-Tuong, “Illumination for computer n=1 n=3 n=7 n=20 n=40 generated pictures”, CACM, 18(6), 1975, 311–7

51 52

Examples 100% Shading: overall equation

◆ the overall shading equation can thus be considered to be the 75% ambient illumination plus the diffuse and specular reflections from each light source

N L R 50% i i specular n IIk=a a + IkLN i d()() i  + IkRV i s i    reflection    V i i 25%

◼ the more lights there are in the scene, the longer this calculation will take 0%

100% 75% 50% 25% 0% diffuse reflection

53 54 The gross assumptions revisited Sampling

◆ diffuse reflection ◆ we have assumed so far that each ray ◆ approximate specular reflection passes through the centre of a pixel ◆ no shadows ◼ i.e. the value for each pixel is the colour of the object which happens to lie exactly ◼ need to do ray tracing or shadow mapping to get shadows under the centre of the pixel ◆ lights at infinity ◆ this leads to: ◼ can add local lights at the expense of more calculation ◼ stair step (jagged) edges to objects ⚫ need to interpolate the L vector ◼ small objects being missed completely ◆ no interaction between surfaces ◼ thin objects being missed completely or ◼ cheat! split into small pieces ⚫ assume that all light reflected off all other surfaces onto a given surface can be amalgamated into a single constant term: “ambient illumination”, add this onto the diffuse and specular illumination

55 56 Anti-aliasing Sampling in ray tracing

◆ these artefacts (and others) are jointly known as aliasing ◆ single point ◼ shoot a single ray through the pixel’s ◆ methods of ameliorating the effects of aliasing are known as centre anti-aliasing ◆ super-sampling for anti-aliasing ◼ shoot multiple rays through the pixel ◼ in signal processing aliasing is a precisely defined technical term for a and average the result particular kind of artefact ◼ regular grid, random, jittered, Poisson ◼ in computer graphics its meaning has expanded to include most disc undesirable effects that can occur in the image ⚫ this is because the same anti-aliasing techniques which ameliorate ◆ adaptive super-sampling true aliasing artefacts also ameliorate most of the other artefacts ◼ shoot a few rays through the pixel, check the variance of the resulting values, if similar enough stop, otherwise shoot some more rays

57 58 Types of super-sampling 1 Types of super-sampling 2

◆ regular grid ◆ Poisson disc ◼ divide the pixel into a number of sub-pixels and ◼ shoot N rays at random points in shoot a ray through the centre of each the pixel with the proviso that no ◼ problem: can still lead to noticable aliasing unless two rays shall pass through the a very high resolution sub-pixel grid is used pixel closer than e to one another ◼ for N rays this produces a better ◆ random 12 8 4 looking image than pure random ◼ shoot N rays at random points in the pixel sampling ◼ replaces aliasing artefacts with noise artefacts ◼ very hard to implement properly ⚫ the eye is far less sensitive to noise than to aliasing

Poisson disc pure random

59 More reasons for wanting to take 60 Types of super-sampling 3 multiple samples per pixel ◆ jittered ◆ super-sampling is only one reason why we might want to take ◼ divide pixel into N sub-pixels and multiple samples per pixel shoot one ray at a random point in ◆ many effects can be achieved by distributing the multiple samples each sub-pixel over some range ◼ an approximation to Poisson disc ◼ called distributed ray tracing sampling ⚫ N.B. distributed means distributed over a range of values ◼ for N rays it is better than pure random sampling ◆ can work in two ways ◼ easy to implement each of the multiple rays shot through a pixel is allocated a random value from the relevant distribution(s) ⚫ all effects can be achieved this way with sufficient rays per pixel each ray spawns multiple rays when it hits an object jitteredPoisson disc pure random ⚫ this alternative can be used, for example, for area lights

61 62 Examples of distributed ray tracing Anti-aliasing

◼ distribute the samples for a pixel over the pixel area ⚫ get random (or jittered) super-sampling ⚫ used for anti-aliasing ◼ distribute the rays going to a light source over some area ⚫ allows area light sources in addition to point and directional light sources ⚫ produces soft shadows with penumbrae ◼ distribute the camera position over some area ⚫ allows simulation of a camera with a finite aperture lens ⚫ produces depth of field effects ◼ distribute the samples in time ⚫ produces motion blur effects on any moving objects

one sample per pixel multiple samples per pixel

63 64 Finite aperture Area vs point light source left, a pinhole camera below, a finite aperture camera below left, 12 samples per pixel below right, 120 samples per pixel note the depth of field blur: only objects at the correct distance are in focus

an area light source produces soft shadows a point light source produces hard shadows

1, 120

65 66 Computer Graphics & Image Processing Unfortunately… Background Ray tracing is computationally expensive ◆ used for super-high visual quality Rendering Video games and user interfaces need something faster Graphics pipeline Most real-time applications rely on rasterization ◆ Polygonal mesh models ◆ Model surfaces as polyhedra – meshes of polygons ◆ Use composition to build scenes ◆ Transformations using matrices in 2D and 3D ◆ Apply perspective transformation ◆ Homogeneous coordinates and project into plane of screen ◆ Work out which surface was closest ◆ Projection: orthographic and perspective ◆ Fill pixels with colour of nearest visible polygon Rasterization Modern graphics cards have hardware to support this Graphics hardware and modern OpenGL Ray tracing starts to appear in real-time rendering ◆ The latest generation of GPUs offers accelerated ray-tracing Colour and tone mapping ◆ But it still not as efficient as rasterization

67 68 Three-dimensional objects Surfaces in 3D: polygons

◆ Polyhedral surfaces are made up from Easier to consider planar polygons meshes of multiple connected polygons ◆ 3 vertices (triangle) must be planar ◆ > 3 vertices, not necessarily planar

a non-planar ◆ Polygonal meshes “polygon” rotate the polygon ◼ open or closed about the vertical axis ◼ manifold or non-manifold should the result be this or this?

◆ Curved surfaces this vertex is in ◼ must be converted to polygons to be drawn front of the other three, which are all in the same plane

69 70 Splitting polygons into triangles 2D transformations

◆ Most Graphics Processing Units (GPUs) are optimised to  scale  why? draw triangles ◆ it is extremely useful to be able to transform predefined ◆ Split polygons with more than three vertices into triangles objects to an arbitrary  rotate location, orientation, and size ◆ any reasonable graphics package will include  translate transforms which is preferable? ◼ 2D ➔ Postscript ◼ 3D ➔ OpenGL  (shear) ?

71 72 Basic 2D transformations Matrix representation of transformations ◆ scale x'= mx  scale  rotate ◼ about origin y'= my ◆ about origin, factor m ◆ about origin, angle  ◼ by factor m ◆ rotate x' m 0x x' cos− sin x x'= x cos − y sin    =      =    ◼ about origin  y' 0 m y  y' sin cos   y ◼ by angle  y'= x sin + y cos  ◆ translate x'= x + xo  do nothing  shear ◼ along vector (xo,yo) ◆ identity ◆ parallel to x axis, factor a y'= y + yo x' 1 0x x' 1 a x ◆ shear = = x'= x + ay           ◼ parallel to x axis  y' 0 1 y  y' 0 1 y ◼ by factor a y'= y

73 74 Homogeneous 2D co-ordinates Matrices in homogeneous co-ordinates  scale  rotate ◆ translations cannot be represented using simple 2D matrix multiplication on 2D vectors, so we switch to ◆ about origin, factor m ◆ about origin, angle  homogeneous co-ordinates  x' m 0 0 x   x' cos− sin  0 x  x y  y' = 0m 0 y  y' = sin cos  0 y (,,),x y w  ( w w)           w' 0 0 1w w'  0 0 1w ◆ an infinite number of homogeneous co-ordinates map to           every 2D point  do nothing  shear ◆ w=0 represents a point at infinity ◆ identity ◆ parallel to x axis, factor a ◆ usually take the inverse transform to be:  x' 1 0 0 x   x' 1a 0 x  (,)(,,)x y x y 1  y' = 0 1 0 y  y' = 0 1 0 y           w' 0 0 1w w' 0 0 1w

75 76 Translation by matrix algebra Concatenating transformations

◆ often necessary to perform more than one transformation on the x x x  ' 1 0 o   same object  y' = 0 1 y  y    0   ◆ can concatenate transformations by multiplying their matrices w' 0 0 1 w e.g. a shear followed by a scaling:

In homogeneous coordinates scale shear x'' m 0 0 x' x' 1a 0 x x'= x + wx y'= y + wy           o o w'= w  y'' = 0m 0 y'  y' = 0 1 0 y           w'' 0 0 1w' w' 0 0 1w In conventional coordinates scale shear both x' x y' y  x'' m 0 01a 0 x  m ma 0 x  = + x0 += y0  y'' = 0m 00 1 0 y = 0m 0 y w' w w' w          w'' 0 0 10 0 1w 0 0 1w

77 78 Transformation are not commutative Scaling about an arbitrary point

be careful of the order in which you concatenate ◆ scale by a factor m about point (xo,yo)

translate point (xo,yo) to the origin transformations (x ,y ) scale by a factor m about the origin o o rotate then scale scale (0,0) 2 −2 translate the origin to (xo,yo)  2 2 0 2 0 0  1 1     x' 1 0 − x  x   x'' m 0 0 x'  x''' 1 0 x  x'' 2 2 0 0 1 0 o   o rotate by 45° scale by 2                    along x axis y' = 0 1 − y y y'' = 0m 0 y' y''' = 0 1 y y''  0 0 1 0 0 1    o           o   w' 0 0 1 w w'' 0 0 1w' w''' 0 0 1 w'' 2 −1 1 −1  2 2 0  2 2 0

 2 1   1 1   x''' 1 0 xo m 0 01 0 − xo  x  2 2 0 2 2 0      y''' = 0 1 y 0m 00 1 − y  y Exercise: show how to scale by 2 rotate by 45°    o   o   along x axis  0 0 1  0 0 1 perform rotation about w''' 0 0 1 0 0 10 0 1 w scale then rotate rotate an arbitrary point

79 80 3D transformations 3D transformations are not commutative

◆ 3D homogeneous co-ordinates x y z 90° rotation 90° rotation (,,,)(,,)x y z w → w w w about z-axis about x-axis ◆ 3D transformation matrices y opposite z z faces translation identity rotation about x-axis x x  1 0 0 tx  1 0 0 0 1 0 0 0   0 1 0 t  0 1 0 0 0cos− sin  0  y       0 0 1 tz  0 0 1 0 0sin cos  0       0 0 0 1  0 0 0 1 0 0 0 1 y z z x  scale rotation about z-axis rotation about y-axis x mx 0 0 0 cos− sin  0 0  cos0 sin  0       90° rotation 90° rotation 0my 0 0 sin cos  0 0 0 1 0 0       about x-axis about z-axis  0 0mz 0  0 0 1 0 − sin0 cos  0        0 0 0 1  0 0 0 1  0 0 0 1

81 82 Model transformation 1 y Model transformation 2 y ◼ the graphics package Open Inventor defines a cylinder to be: ⚫ centre at the origin, (0,0,0) 2 x order is important: 2 x ⚫ radius 1 unit ◆ scale first ⚫ height 2 units, aligned along the y-axis ◆ rotate 2 2 ◼ this is the only cylinder that can be drawn, ◆ translate last but the package has a complete set of 3D transformations S ◼ we want to draw a cylinder of: scaling and translation are straightforward ⚫ radius 2 units y  0002   5.1001  ⚫ the centres of its two ends located at (1,2,3) and (2,4,5)  005.10   3010  ❖ its length is thus 3 units S =   T =   ◼ what transforms are required?  0200   4100  3 x and in what order should they be applied?      1000   1000  scale from translate centre of size (2,2,2) cylinder from (0,0,0) to to size (4,3,4) halfway between (1,2,3) and (2,4,5) 4

83 84 Model transformation 3 Model transformation 4

rotation is a multi-step process ◆ desired axis: (2,4,5)–(1,2,3) = (1,2,2) ◆ break the rotation into steps, each of which is rotation ◆ original axis: y-axis = (0,1,0) about a principal axis ◆ work these out by taking the desired orientation back to ◆ zero the z-coordinate by rotating about the x-axis the original axis-aligned position z 1 0 0 0   0 cosθ − sin θ 0 )2,2,1( ⚫ the centres of its two ends located at (1,2,3) and (2,4,5) R =   1 0 sin θ cosθ 0    y ◆ desired axis: (2,4,5)–(1,2,3) = (1,2,2) 0 0 0 1 + 22 0,22,1 2 ( ) θ −= arcsin 22 = )0,8,1( ◆ original axis: y-axis = (0,1,0) + 22

85 86 Model transformation 5 Model transformation 6 the overall transformation is: ◆ then zero the x-coordinate by rotating about the z-axis ◆ first scale ◆ we now have the object’s axis pointing along the y-axis ◆ then take the inverse of the rotation we just calculated ◆ finally translate to the correct position y cos φ − sin φ 00   2  x' x    2 + 0,81,0  )0,8,1(     sin φ cos φ 00   y' −1 −1 y R =   = )0,3,0(    = T R  R S   2  0 0 01   z' 1 2  z    w' w  0 0 10  x     1 φ = arcsin 2 12 + 8

87 88 Application: display multiple instances 3D  2D projection

◆ transformations allow you to define an object at one to make a picture location and then place multiple instances in your scene ◆ 3D world is projected to a 2D image ◼ like a camera taking a photograph ◼ the three dimensional world is projected onto a plane

The 3D world is described as a set of (mathematical) objects e.g. sphere radius (3.4) centre (0,2,9) e.g. box size (2,4,3) centre (7, 2, 9) orientation (27º, 156º)

89 90 Types of projection Geometry of perspective projection parallel ◆ e.g. (,,)(,)x y z→ x y y (',',)x y d ◆ useful in CAD, architecture, etc d (,,)x y z x'= x ◆ looks unrealistic z perspective ( , , 0 )00 z (,,)(,)x y z → x y ◆ e.g. z z d y'= y ◆ things get smaller as they get farther away z ◆ looks realistic d ◼ this is how cameras work

91 92 Perspective projection Projection as a matrix operation with an arbitrary camera

◆ we have assumed that: ◼ screen centre at (0,0,d) d ◼ screen parallel to xy-plane  x  1 0 0 0 x x'= x ◼ z-axis into screen z  y   10 0 0 y ◼ y-axis up and x-axis to the right   =    ◼ eye (camera) at origin (0,0,0)  /1 d  0 0 0 /1 dz d ◆ for an arbitrary camera we can either:      y'= y z ◼ work out equations for projecting objects about an arbitrary point  / dz  0 /10 d 0 1 onto an arbitrary plane ◼ transform all objects into our standard co-ordinate system (viewing This is useful in the z-buffer 1 co-ordinates) and use the above assumptions  x algorithm where we need to    / wx  z'= y   interpolate 1/z values rather remember   → / wy z  z    than z values.    / wz  w

93 94 A variety of transformations Model, View, Projection matrices

object in object in object in object in object world viewing 2D screen co-ordinates co-ordinates co-ordinates co-ordinates modelling viewing projection transform transform

◼ the modelling transform and viewing transform can be multiplied together to produce a single matrix taking an object directly from object co-ordinates into viewing co-ordinates Model ◼ either or both of the modelling transform and viewing transform matrices can matrix be the identity matrix ⚫ e.g. objects can be specified directly in viewing co-ordinates, or directly in To position each world co-ordinates object in the scene. ◼ this is a useful set of transforms, not a hard and fast model of how things Object coordinates Could be different World coordinates should be done for each object. Object centred at the origin

95 96 Model, View, Projection matrices Model, View, Projection matrices

The default OpenGL coordinate system is right-handed

Projection View matrix matrix

To position all objects To project 3D relative to the camera coordinates to a 2D World coordinates plane. Note that z View (camera) View (camera) coordinate is retained Screen coordinates coordinates coordinates for depth testing. x and y must be in the range Camera at the origin, -1 and 1 pointing at -z 95 96

97 98 All together Viewing transformation 1

world viewing 3D world Screen vertex co-ordinates viewing co-ordinates coordinates coordinates transform

xs/ws and 푠 the problem: ys/ws must be 푥 푥 between 푠 ◆ to transform an arbitrary co-ordinate system to the -1 and 1 푦 푦 default viewing co-ordinate system 푠 = 푃 ∙ 푉 ∙ 푀 ∙ 푧 푧 camera specification in world co-ordinates 푠 ◆ eye (camera) at (e ,e ,e ) 푤 푤 x y z l Projection, view and ◆ look point (centre of screen) at (lx,ly,lz) u model matrices ◆ up along vector (u ,u ,u ) x y z e ◼ perpendicular to el

99 100 Viewing transformation 2 Viewing transformation 3

◆ translate eye point, (ex,ey,ez), to origin, (0,0,0) ◆ need to align line el with z-axis

1 0 0 − ex  ◼ first transform e and l into new co-ordinate system 0 1 0 − e  e''= S  T  e = 0 l '' = S  T  l T =  y  ◼ '' '' 0 0 1 − ez  then rotate e l into yz-plane, rotating about y-axis   0 0 0 1  el z ◆ scale so that eye point to look point distance, , is distance cosθ 0 − sin θ 0 2 2 0,'',''''l l+ l from origin to screen centre, d   ( y x z )  0 1 0 0 R1 = ('','','')l l l d 0 0 0   x y z  el  sin θ 0 cosθ 0   0d 0 0   el =()()()l − e2 + l − e 2 + l − e 2 S =  el   0 0 0 1 x x x y y z z d  0 0el 0   l ''  0 0 0 1 θ = arccos z 22 +ll '''' zx

101 102 Viewing transformation 4 Viewing transformation 5

◆ having rotated the viewing vector onto the yz plane, rotate it ◆ the final step is to ensure that the up vector actually points up, about the x-axis so that it aligns with the z-axis i.e. along the positive y-axis l'''= R  l '' ◼ actually need to rotate the up vector about the z-axis so that it lies in the 1 positive y half of the yz plane u'''' = R  R  u why don’t we need to 2 1 u S T z multiply by or ? 1 0 0 0 0,,'''''' 0 l2+ l 2 ( y z )   cos ψ − sin ψ 00  u is a vector rather than 0 cosφ − sin φ 0 = (,,)0 0 d   a point, vectors do not R2 = (,''',''')0 l l sin ψ cos ψ 00 y z R   get translated 0 sin φ cosφ 0  3 =    0 0 01  0 0 0 1 y      0 0 10  scaling u by a uniform l ''' scaling matrix would z u '''' y φ = arccos ψ = arccos make no difference to the l +l '''''' 22 2 2 zy u x +u '''''''' y direction in which it points

103 104 Viewing transformation 6 Transforming normal vectors

world viewing  Transformation by co-ordinates viewing co-ordinates a nonorthogonal matrix transform does not preserve angles ◆ we can now transform any point in world co-ordinates to the 1 0 0 equivalent point in viewing co-ordinate 0 2 0  Since: 0 0 1  x'  x  Normal transform y' y   =RRRST3  2  1       z'   z  w' w 푁 ∙ 푇 = 0 e→(,,)(,,)0 0 0 l → 0 0 d ′ ′ ◆ in particular: Vertex position Transformed푁 ∙ 푇 = normal퐺푁 ∙ MT =transform 0 ◆ the matrices depend only on e, l, and u, so they can be pre- and tangent vector multiplied together MRRRST=     3 2 1  We can find that: ◆ Derivation shown on the visualizer−1 푇 퐺 = 푀 104

Introduction to Computer Graphics Rasterization

Background  Efficiently draw (thousands of) triangles  Interpolate vertex attributes inside the triangle Rendering  Barycentric Graphics pipeline coordinates Rasterization are used to Graphics hardware and modern OpenGL interpolate Colour colours, normals, texture coordinates and other attributes inside the triangle

105 106

Barycentric coordinates Triangle rasterization

 Find barycentric for y=ymin to ymax do

coordinates of the point for x=xmin to xmax do (x,y)  Given the coordinates of 훼 = 푓푐푏(푥, 푦)/푓푐푏(푥푎, 푦푎) the vertices 훽 = 푓푎푐(푥, 푦)/푓푎푐(푥푏, 푦푏) if ( and and ) then  Derivation in the lecture 훾 = 1 − 훼 − 훽 훼 > 0 훽 > 0 훾 > 0 draw pixels (x,y) with colour c 푐 = 훼푐푎 + 훽푐푏 + 훾푐푐 푓푐푏(푥,푦) 푓푎푐(푥,푦)  Optimization: the barycentric coordinates will change by the 훼 = 푓푐푏(푥푎,푦푎) 훽 = 푓푎푐(푥푏,푦푏) is the implicit line same amount when moving one pixel right (or one pixel down) equation: regardless of the position 푎푏 푓 푥, 푦  Precompute increments and use them instead of 푎푏 푎 푏 푏 푎 푎 푏 푏 푎 푓 푥, 푦 = 푦 − 푦 푥 + 푥 − 푥 푦 + 푥 푦 − 푥 푦 computing barycentric coordinates when drawing pixels sequentially Δ훼, Δ훽, Δ훾 107 108

Illumination & shading Gouraud shading

 Drawing polygons with uniform colours gives poor results  for a polygonal model, calculate the diffuse illumination at each  Interpolate colours across polygons vertex  calculate the normal at the vertex, and use this to calculate the diffuse illumination at that point  normal can be calculated directly if the polygonal model was derived from a curved surface

[(x1',' y 1), z 1,(,, r 1 g 1 b 1 )]  interpolate the colour between the vertices across the polygon

 surface will look smoothly curved [(x2',' y 2), z 2 , (,,r2 g 2 b 2 )]  rather than looking like a set of polygons  surface outline will still look polygonal

[(x3',' y 3), z 3,(,, r 3 g 3 b 3 )]

109 110 Henri Gouraud, “Continuous Shading of Curved Surfaces”, IEEE Trans Computers, 20(6), 1971

Flat vs Gouraud shading Phong shading

 similar to Gouraud shading, but calculate the specular  note how the interior is smoothly shaded but the outline remains component in addition to the diffuse component polygonal  therefore need to interpolate the normal across the polygon in order to be able to calculate the reflection vector

[(x',' y), z,(,, r g b ), N ]  N.B. Phong’s approximation to 1 1 1 1 1 1 1 specular reflection ignores (amongst other things) the [(x',' y), z , effects of glancing incidence 2 2 2 (,,r2 g 2 b 2), N 2 ] (the Fresnel term)

[(x3',' y 3), z 3,(,, r 3 g 3 b 3), N 3 ]

111 112 http://computer.howstuffworks.com/question484.htm

Occlusions (hidden surfaces) Z-Buffer - algorithm

Colour Depth Simple case buffer buffer

 Initialize the depth buffer and image buffer for all pixels colour(x, y) = Background_colour,

More difficult cases depth(x, y) = zmax // position of the far clipping plane  For every triangle in a scene do  For every fragment (x, y) representing this triangle do  Calculate z for current (x, y)

 if (z < depth(x, y)) and (z > zmin) then  depth (x, y) = z  colour (x, y) = Polygon_Colour (x, y)

113 114

View frustum Introduction to Computer Graphics

 Controlled by camera parameters: near-, far-clipping Background planes and field-of-view Z-buffer must store all Rendering these values Graphics pipeline Rasterization Graphics hardware and modern OpenGL FOV ◆ GPU & APIs ◆ OpenGL Rendering pipeline Near-clipping plane ◆ Example OpenGL code Far-clipping plane ◆ GLSL ◆ Textures ◆ Raster buffers Colour & Tone mapping

115 116

What is a GPU? CPU vs. GPU

 Graphics Processing Unit  Like CPU (Central Processing Unit) but for processing graphics  Optimized for floating point operations on large arrays of data AMD Epyc Nvidia Turing Quadro RTX 8000  Vertices, normals, pixels, etc. 32 CPU cores 4608 unified shaders 2.2-3.2 GHz 288 texture mapping units 14nm 96 render output units 19,200,000,000 transistors 72 streaming multiprocessors 576 tensor cores 12nm 18,600,000,000 transistors

117 118

What does a GPU do What makes GPU so fast?

 Performs all low-level tasks & a lot of high-level tasks  3D rendering can be very efficiently parallelized  Clipping, rasterisation, hidden surface removal, …  Millions of pixels  Essentially draws millions of triangles very efficiently  Millions of triangles  Procedural shading, texturing, animation, simulation, …  Many operations executed independently at the same time  Video rendering, de- and encoding, deinterlacing, ...  This is why modern GPUs  Physics engines  Contain between hundreds and thousands of SIMD processors  Full programmability at several pipeline stages  Single Instruction Multiple Data – operate on large arrays of data  fully programmable  >>400 GB/s memory access  but optimized for massively parallel operations  This is much higher bandwidth than CPU  But peak performance can be expected for very specific operations

119 120

GPU APIs (Application Programming Interfaces) One more API OpenGL DirectX  Vulkan – cross platform, open standard  Multi-platform  Microsoft Windows / Xbox  Low-overhead API for high performance 3D graphics  Open standard API  Proprietary API  Compared to OpenGL / DirectX  Focus on general 3D  Focus on games  Reduces CPU load applications  Application manages  Better support of multi-CPU-core architectures  Open GL driver manages resources  Finer control of GPU the resources  But  The code for drawing a few primitives can take 1000s line of code  Intended for game engines and code that must be very well optimized

121 122

And one more GPGPU - general purpose computing

 Metal (Apple iOS8)  OpenGL and DirectX are not meant to be used for  low-level, low-overhead 3D GFX and compute shaders API general purpose computing  Support for Apple A7, Intel HD and Iris, AMD, Nvidia  Example: physical simulation, machine learning  Similar design as modern APIs, such as Vulcan  CUDA – Nvidia’s architecture for parallel computing  Swift or Objective-C API  C-like programming language  Used mostly on iOS  With special API for parallel instructions  Requires Nvidia GPU  OpenCL – Similar to CUDA, but open standard  Can run on both GPU and CPU  Supported by AMD, Intel and NVidia, Qualcomm, Apple, …

123 124

GPU and mobile devices WebGL

 OpenGL ES 1.0-3.2  JavaScript library for 3D rendering in a web browser  Stripped version of OpenGL  WebGL 1.0 - based on OpenGL ES 2.0  Removed functionality that is not strictly necessary on mobile  WebGL 2.0 – based on OpenGL ES 3.0 devices  Chrome and Firefox (2017)  Devices  Most modern browsers  iOS: iPhone, iPad support WebGL  Android phones  Potentially could be used to create  PlayStation 3 3D games in a browser  Nintendo 3DS  and replace Adobe Flash  and many more http://zygotebody.com/ OpenGL ES 2.0 rendering (iOS)

125 126

OpenGL in Java OpenGL History

 Standard Java API does not include OpenGL interface  Proprietary library IRIS GL by SGI  Geometry shaders  OpenGL 1.0 (1992)  But several wrapper libraries exist  OpenGL 4.0 (2010)  OpenGL 1.2 (1998)  Catching up with Direct3D 11  Java OpenGL – JOGL  OpenGL 2.0 (2004)  OpenGL 4.5 (2014)  Lightweight Java Game Library - LWJGL  GLSL  OpenGL 4.6 (2017)  We will use LWJGL 3  Non-power-of-two (NPOT)  SPIR-V shaders  Seems to be better maintained textures  OpenGL 3.0 (2008)  Access to other APIs (OpenCL, OpenAL, …)  Major overhaul of the API  Many features from previous  We also need a linear algebra library versions depreciated  JOML – Java OpenGL Math Library  OpenGL 3.2 (2009)  Operations on 2, 3, 4-dimensional vectors and matrices  Core and Compatibility profiles

127 128

How to learn OpenGL?  Lectures – algorithms behind OpenGL, general principles  Tick 2 – detailed tutorial, learning by doing  References  OpenGL Programming Guide: The Official Guide to Learning OpenGL, Version 4.5 with SPIR-V by John Kessenich, Graham Sellers, Dave Shreiner ISBN-10:0134495497 OpenGL rendering pipeline  OpenGL quick reference guide https://www.opengl.org/documentation/glsl/  Google search: „man gl...... ”

129

OpenGL programming model OpenGL rendering pipeline

CPU code GPU code Vertex Vertex Tessellation Tessellation  gl* functions that  Fragment shaders data shader control shader evaluation shader  Create OpenGL objects  Vertex shaders  Copy data CPU<->GPU Primitive Geometry  and other shaders Rasterization Clipping  Modify OpenGL state assembly shader  Written in GLSL  Enqueue operations  Similar to C  Synchronize CPU & GPU Fragment Screen  From OpenGL 4.6 could be shader buffer  C99 library written in other language  Wrappers in most and compiled to SPIR-V Programmable programming language Fixed stages stages

131 132

OpenGL rendering pipeline OpenGL rendering pipeline

Vertex Vertex Tessellation Tessellation Vertex Vertex Tessellation Tessellation data shader control shader evaluation shader data shader control shader evaluation shader

Primitive Geometry Primitive Geometry Rasterization Clipping Rasterization Clipping assembly shader [Optional]setup Create new shader Processing of vertices, normals, primitives by tessellating existing uv texture coordinates. primitives (patches). Fragment Screen Fragment Screen shader buffer shader buffer

Programmable Programmable Fixed stages Fixed stages stages stages

133 134

OpenGL rendering pipeline OpenGL rendering pipeline Organizes vertices into primitives and prepares them for [Optional] Operate on tessellated rendering. geometry. Can create new primitives. Vertex Vertex Tessellation Tessellation Vertex Vertex Tessellation Tessellation data shader control shader evaluation shader data shader control shader evaluation shader

Primitive Geometry Primitive Geometry Rasterization Clipping Rasterization Clipping assembly shader assembly shader fur shadow volumes

Fragment Screen Fragment Screen shader buffer shader buffer

Programmable Programmable Fixed stages Fixed stages stages stages

135 136

Generates fragments (pixels) to OpenGL rendering pipeline OpenGLbe drawn rendering for each primitive. pipeline Remove or modify vertices so Interpolates vertex attributes. that they all lie within the scanlines viewport (view frustum).

Vertex Vertex Tessellation Tessellation Vertex Vertex Tessellation Tessellation data shader control shader evaluation shader data shader control shader evaluation shader

Primitive Geometry Primitive Geometry Rasterization Clipping Rasterization Clipping assembly shader assembly shader

Fragment Screen Fragment Screen shader buffer shader buffer

Programmable Programmable Fixed stages Fixed stages stages stages

137 138

Computes colour per each fragment (pixel). Can lookup Example: OpenGL renderingcolour in the texture. pipeline Can modify pixels’ depth value. preparing vertex data for a cube

Vertex Tessellation Tessellation Vertex shader control shader evaluation shader data

Primitive Geometry Primitives (triangles) PhysicallyClipping accurate Non-Photorealistic-Rendering Rasterization assembly shader materials shader Indices 0, 1, 2 Also used for tone mapping. … Fragment Screen shader buffer Vertex attributes Ind Positions Normals 0 0, 0, 0 0, 0, -1 Programmable Fixed stages stages … … …

139 140

Geometry objects in OpenGL (OO view) Simple OpenGL application - flow

 Initialize rendering window & OpenGL Initialize OpenGL context  Send the geometry (vertices, triangles, normals) to the GPU  Load and compile Shaders Set up inputs

 Clear the screen buffer  Set the model-view-projection matrix Draw a frame  Render geometry  Flip the screen buffers

Free resources

141 142

Shaders

 Shaders are small programs executed on a GPU  Executed for each vertex, each pixel (fragment), etc.  They are written in GLSL (OpenGL Shading Language)  Similar to C and Java  Primitive (int, float) and aggregate data types (ivec3, vec3) GLSL - fundamentals  Structures and arrays  Arithmetic operations on scalars, vectors and matrices  Flow control: if, switch, for, while  Functions

144

Example of a vertex shader Data types

#version 330  Basic types in vec3 position; // vertex position in local space in vec3 normal; // vertex normal in local space  float, double, int, uint, bool out vec3 frag_normal; // fragment normal in world space  Aggregate types uniform mat4 mvp_matrix; // model-view-projection matrix  float: vec2, vec3, vec4; mat2, mat3, mat4

void main()  double: dvec2, dvec3, dvec4; dmat2, dmat3, dmat4 {  int: ivec2, ivec3, ivec4 // Typicaly normal is transformed by the model matrix  uint: uvec2, uvec3, uvec4 // Since the model matrix is identity in our case, we do not modify normals  frag_normal = normal; bool: bvec2, bvec3, bvec4 vec3 V = vec3( 1.0, 2.0, 3.0 ); mat3 M = mat3( 1.0, 2.0, 3.0, // The position is projected to the screen coordinates using mvp_matrix gl_Position = mvp_matrix * vec4(position, 1.0); 4.0, 5.0, 6.0, } 7.0, 8.0, 9.0 ); Why is this piece of code needed? 145 146

Indexing components in aggregate types Swizzling

 Subscripts: rgba, xyzw, stpq (work exactly the same) You can select the elements of the aggregate type:  float red = color.r; vec4 rgba_color( 1.0, 1.0, 0.0, 1.0 );  float v_y = velocity.y; vec3 rgb_color = rgba_color.rgb; but also vec3 bgr_color = rgba_color.bgr; vec3 luma = rgba_color.ggg;  float red = color.x;  float v_y = velocity.g;  With 0-base index:  float red = color[0];  float m22 = M[1][1]; // second row and column // of matrix M

147 148

Arrays Storage qualifiers

 Similar to C  const – read-only, fixed at compile time float lut[5] = float[5]( 1.0, 1.42, 1.73, 2.0, 2.23 );  in – input to the shader  out – output from the shader  Size can be checked with “length()”  uniform – parameter passed from the application (Java), for( int i = 0; i < lut.length(); i++ ) { constant for the primitive lut[i] *= 2;  buffer – shared with the application }  shared – shared with local work group (compute shaders only)

 Example: const float pi=3.14;

149 150

Shader inputs and outputs GLSL Operators

glGetAttribLocation  Arithmetic: + - ++ -- glBindBuffer ArrayBuffer (normals) glVertexAttribPointer  Multiplication: glEnableVertexAttribArray ArrayBuffer (vertices)  vec3 * vec3 – element-wise  mat4 * vec4 – matrix multiplication (with a column vector) out vec3  Bitwise (integer): <<, >>, &, |, ^ in vec3 position frag_normal Vertex  Logical (bool): &&, ||, ^^ shader Vertex attribute out vec3 colour in vec3 normal interpolation Fragment  Assignment: shader float a=0; in vec3 frag_normal a += 2.0; // Equivalent to a = a + 2.0

[optional] FrameBuffer (pixels) glBindFragDataLocation or  See the quick reference guide at: layout(location=?) in GLSL https://www.opengl.org/documentation/glsl/ 151 152

GLSL Math GLSL flow control

 Trigonometric: if( bool ) { } // true for( int i = 0; i<10; i++ ) {  radians( deg ), degrees( rad ), sin, cos, tan, } else { ... asin, acos, atan, sinh, cosh, tanh, asinh, // false } acosh, atanh }  Exponential: while( n < 10 ) {  pow, exp, log, exp2, log2, sqrt, inversesqrt switch( int_value ) { ...  Common functions: case n: } // statements  abs, round, floor, ceil, min, max, clamp, … break; do {  And many more case m: ... // statements } while ( n < 10 ) break;  See the quick reference guide at: default: https://www.opengl.org/documentation/glsl/ 153 154

(Most important) OpenGL texture types

0 s 1 Texture can have any size but the 1D sizes that are powers of two (POT, 2n) 0 s 1 may give better performance. 0 Texel 2D

t Textures CUBE_MAP Used for environment mapping 1

3D 0

1 p 1 s 0 0 1 156

Texture mapping Texture mapping

 1. Define your texture  2. Define the correspondence 1 function (image) T(u,v) between the vertices on the  (u,v) are texture 3D object and the texture coordinates coordinates

0 0 1

157 158

Texture mapping Sampling

Texture  3. When rendering, for every surface point compute v texture coordinates. Use the texture function to get texture value. Use as color or reflectance. Up-sampling More pixels than texels Values need to be interpolated

u Down-sampling Fewer pixels than texels Values need to be averaged over an area of the texture (usually using a mipmap)

159 160

Nearest neighbor vs. v bilinear interpolation (upsampling) Texture mapping examples

u AB Texel AB

X X

CD CD

interpolation

Bilinear Nearest neighbour nearest- Pick the nearest Interpolate first along neighbour texel: D x-axis between AB and CD, then along y-axis between the interpolated points. bilinear

161 162

v Up-sampling Down-sampling  if one pixel in the texture map  if the pixel covers quite a large area u covers several pixels in the final of the texture, then it will be nearest- image, you get visible artefacts necessary to average the texture neighbour across that area, not just take a blocky  only practical way to prevent this sample in the middle of the area artefacts is to ensure that texture map is of sufficiently high resolution that it does not happen bilinear blurry artefacts

163 164

Mipmap Down-sampling  Textures are often stored at multiple without area averaging with area averaging resolutions as a mipmap  Each level of the pyramid is half the size of the lower level  Mipmap resolution is always power-of-two (1024, 512, 256, 128, ...)  It provides pre-filtered texture (area- averaged) when screen pixels are larger than the full resolution texels  Mipmap requires just an additional 1/3 of the original texture size to store  OpenGL can generate a mipmap with glGenerateMipmap(GL_TEXTURE_2D) This image is an illustration showing only 1/3 increase in storeage. Mipmaps are stored differently in the GPU memory.

165 166

Texture tiling Multi-surface UV maps

 Repetitive patterns can be represented as texture tiles.  A single texture is often used for multiple surfaces and  The texture folds over, so that objects  T(u=1.1, v=0) = T(u=0.1, v=0)

Example from: http://awshub.com/blog/blog/2011/11/01/hi-poly-vs- low-poly/ 167 168

Bump (normal) mapping Displacement mapping

 Special kind of texture that  Texture that modifies surface modifies surface normal  Better results than bump  Surface normal is a vector mapping since the surface is that is perpendicular to a not flat surface  Requires geometry shaders  The surface is still flat but shading appears as on an uneven surface  Easily done in fragment shaders

169 170

Environment mapping Environment mapping

 To show environment  Environment cube reflected by an object  Each face captures environment in that direction

171 172

Texture objects in OpenGL Setting up a texture // Create a new texture object in memory and bind it int texId = glGenTextures(); glActiveTexture(textureUnit); glBindTexture(GL_TEXTURE_2D, texId);

// All RGB bytes are aligned to each other and each component is 1 byte glPixelStorei(GL_UNPACK_ALIGNMENT, 1);

// Upload the texture data and generate mipmaps glTexImage2D(GL_TEXTURE_2D, 0, GL_RGB, tWidth, tHeight, 0, GL_RGBA, GL_UNSIGNED_BYTE, buf); glGenerateMipmap(GL_TEXTURE_2D);

173 174

Texture parameters //Setup filtering, i.e. how OpenGL will interpolate the pixels when scaling up or down glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR); glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR_MIPMAP_NEAREST);

How to How to interpolate interpolate in between mipmap Raster buffers (colour, depth, stencil) 2D levels //Setup wrap mode, i.e. how OpenGL will handle pixels outside of the expected range glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE); glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);

175

Render buffers in OpenGL Double buffering

Four components:  To avoid flicker, tearing Colour: GL_FRONT GL_BACK RGBA  Use two buffers (rasters): Typically 8 bits per  Front buffer – what is shown on the screen component GL_FRONT_LEFT GL_FRONT_RIGHT  Back buffer – not shown, GPU draws into that buffer In stereo:  When drawing is finished, swap front- and back-buffers

GL_BACK_LEFT GL_BACK_RIGHT Front buffer – display time Back buffer - draw 1st buffer 2nd buffer Depth: DEPTH To resolve occlusions (see Z-buffer algorithm) Single component, usually >8 bits

To block rendering selected pixels STENCIL Stencil: Single component, usually 8 bits.

177 178

Triple buffering Vertical Synchronization: V-Sync

 Do not wait for swapping to start drawing the next  Pixels are copied from colour buffer to monitor row-by-row frame  If front & back buffer are swapped during this process: Double buffering  Upper part of the screen contains previous frame Front buffer display time –  Lower part of the screen contains current frame Back buffer - draw  Result: tearing artefact Get rid of these gaps  Solution: When V-Sync is enabled  glwfSwapInterval(1); Triple buffering Front buffer – display time glSwapBuffers() waits until Back buffer - draw the last raw is copied to the display. 1st buffer  Shortcomings 2nd buffer  More memory needed 3rd buffer  Higher delay between drawing and displaying a frame

179 180

No V-Sync vs. V-Sync FreeSync (AMD) & G-Sync (Nvidia)

 Adaptive sync  Graphics card controls timing of the frames on the display  Can save power for 30fps video of when the screen is static

 Can reduce lag for real-time graphics

No No V-Sync V-Sync

181 182

Electromagnetic spectrum

 Visible light  Electromagnetic waves of wavelength in the range 380nm to 730nm  Earth’s atmosphere lets through Colour and colour spaces a lot of light in this wavelength band  Higher in energy than thermal infrared, so heat does not interfere with vision

183 184

Colour Black body radiation  There is no physical definition of colour – colour is the  Electromagnetic radiation emitted by a perfect absorber result of our perception at a given temperature  Graphite is a good approximation of a black body  For emissive displays / objects

colour = perception( spectral_emission )

 For reflective displays / objects

colour = perception( illumination * reflectance )

185 186

Correlated colour temperature Standard illuminant D65

 The temperature of a black body radiator that produces  Mid-day sun in Western Europe / Northern Europe light most closely matching the particular source  Colour temperature approx. 6500 K  Examples:  Typical north-sky light: 7500 K  Typical average daylight: 6500 K  Domestic tungsten lamp (100 to 200 W): 2800 K  Domestic tungsten lamp (40 to 60 W): 2700 K  Sunlight at sunset: 2000 K  Useful to describe colour of the illumination (source of light)

187 188

Reflectance Reflected light  Most of the light we see is reflected from objects L )( = I  R )()(  These objects absorb a certain part of the light spectrum  Reflected light = illumination * reflectance Spectral reflectance of ceramic tiles

Why not red?

The same object may appear to have different color under different illumination.

189 190

Fluorescence Colour vision

 Cones are the photreceptors responsible for color vision

From: http://en.wikipedia.org/wiki/Fluorescence  Only daylight, we see no colors when there is not enough light  Three types of cones  S – sensitive to short Sensitivity curves – probability that a photon of that wavelengths will be wavelengths absorbed by a photoreceptor. S,M  M – sensitive to medium and L curves are normalized in this wavelengths plot.  L – sensitive to long wavelengths 191 192

Perceived light Metamers

 cone response = sum( sensitivity * reflected light )  Even if two light spectra are different, they may appear to have the same colour  The light spectra that appear to have the same colour are called metamers  Example:

* = [L1,M1,S1] = Formally Although there is an infinite number of wavelengths, we have only three 730 photoreceptor types to sense R = S   L )()( d * = [L2,M2,S2] S  S differences between light spectra 380

193 Index S for S-cones 194

Practical application of metamerism Tristimulus Colour Representation

 Displays do not emit the same light spectra as real-world  Observation objects  Any colour can be matched  Yet, the colours on a display look almost identical using three linear independent reference colours On the display  May require “negative” contribution to test colour  Matching curves describe the * = [L ,M ,S ] 1 1 1 value for matching mono-

= chromatic spectral colours of equal intensity  With respect to a certain * = [L2,M2,S2] set of primary colours

In real world 195 196

Standard Colour Space CIE-XYZ Standard Colour Space CIE-XYZ

 CIE Experiments [Guild and Wright, 1931]  Standardized imaginary primaries  Colour matching experiments CIE XYZ (1931)  Group ~12 people with „normal“ colour vision  Could match all physically realizable  2 degree visual field (fovea only) colour stimuli  CIE 2006 XYZ  Y is roughly equivalent to luminance  Derived from LMS color matching functions by Stockman & Sharpe  Shape similar to luminous efficiency  S-cone response differs the most from CIE 1931 curve  CIE-XYZ Colour Space  Monochromatic spectral colours  Goals form a curve in 3D XYZ-space  Abstract from concrete primaries used in experiment  All matching functions are positive Cone sensitivity curves can be  Primary is roughly proportionally to light intensity (luminance) „Y” obtained by a linear transformation of CIE XYZ

197 198

Luminance CIE chromaticity diagram  Luminance – how bright the surface will appear  chromaticity values are defined in terms of x, y, z regardless of its colour. Units: cd/m2 X Y Z x = ,y = ,z = x + y + z = 1 XYZ+ + XYZ+ + XYZ+ + 700 Luminance  ignores luminance 1 520n FvDFH Fig 13.24 푉 න 푘 =  can be plotted as a 2D function m 퐿 = 푘퐿 휆 푉 휆 푑휆 683.002 540n Colour plate 2 350 510n m Light spectrum (radiance) Luminous efficiency function  pure colours (single wavelength) m (weighting) lie along the outer curve 560n m 500n  all other colours are a mix of m 580n m pure colours and hence lie 600n inside the curve m  490n points outside the curve do not m 700n exist as colours m 480n m 460n m 410n 199 200 m

Visible vs. displayable colours Standard vs. High Dynamic Range

 HDR cameras/formats/displays attempt  All physically possible and visible capture/represent/reproduce (almost) colours form a solid in XYZ space all visible colours  Each display device can reproduce a  They represent scene colours and subspace of that space therefore we often call this representation scene-referred  A chromacity diagram is a slice taken from a 3D solid in XYZ space  SDR cameras/formats/devices attempt to capture/represent/reproduce only  Colour Gamut – the solid in a colours of a standard sRGB colour colour space gamut, mimicking the capabilities of  Usually defined in XYZ to be device- CRTs monitors independent  They represent display colours and therefore we often call this representation display-referred 201  202

From rendering to display From rendering to display

203 204

Display encoding for SDR: From rendering to display gamma correction

 Gamma correction is often used to encode luminance or tri-stimulus color values (RGB) in imaging systems (displays, printers, cameras, etc.)

Gamma Gain (usually =2.2)

훾 (relative) Luminanceout 푖푛 Luma PhysicalV signal = a ∙ 푉Digital signal (0-1) Colour: the same equation applied to red, green and blue Inverse: 1 colour channels. 훾 in 1 표푢푡 205 206 V = ∙ 푉 푎

Why is gamma needed? Luma – gray-scale pixel value  Luma - pixel brightness in gamma corrected units <- Pixel value (luma) <- Luminance  , ′ are gamma-corrected′ colour′ values′ 퐿 = 0.2126푅 + 0.7152퐺 + 0.0722퐵  Prime′ ′ symbol′ denotes gamma corrected 푅 퐺 and 퐵  Gamma-corrected pixel values give a scale of brightness  Used in image/video coding levels that is more perceptually uniform  At least 12 bits (instead of 8) would be needed to encode  Note that relative luminance if often approximated with each color channel without gamma correction  And accidentally it was also the response of the CRT gun  , and 퐿 =are 0.2126푅linear colour′ +0.7152퐺훾 values +0.′ 훾0722퐵 ′ 훾 = 0.2126(푅 ) +0.7152(퐺 ) +0.0722(퐵 )  Luma and luminace are different quantities despite similar formulas푅, 퐺 퐵

207 208

How to transform between Standards for display encoding RGB colour spaces?

Display type Colour space EOTF Bit depth RGB RGB XYZ Standard Dynamic Range ITU-R 709 2.2 gamma / sRGB 8 to 10 ITU-R 709 ITU-R 2020 High Dynamic Range ITU-R 2020 ITU-R 2100 (PQ/HLG) 10 to 12 SDR Device-independent HDR Colour space Electro-Optical Transfer Function What is the colour of “pure” red, How to efficiently encode each primary  From ITU-R 709 RGB to XYZ: green and blue colour

푋 0.4124 0.3576 0.1805 푅 푌 = 0.2126 0.7152 0.0722 ∙ 퐺 푅709푡표푋푌푍 푅709 푍Relative 0.XYZ0193 Relative0.1192 XYZ 0.Relative9505 XYZ 퐵 of the red of the green of the blue primary primary primary

209 210

How to transform between RGB colour spaces? Representing colour

 From ITU-R 709 RGB to ITU-R 2020 RGB:  We need a mechanism which allows us to represent colour in the computer by some set of numbers  A) preferably a small set of numbers which can be quantised to 푅 푅 a fairly small number of bits each 퐺 = 푀푋푌푍푡표푅2020 ∙ 푀푅709푡표푋푌푍 ∙ 퐺  From ITU-R 2020 RGB to ITU-R 709 RGB:  Linear and gamma corrected RGB, sRGB 퐵 푅2020 퐵 푅709  B) a set of numbers that are easy to interpret 푅 푅  Munsell’s artists’ scheme = 푀푋푌푍푡표푅709 ∙ 푀푅2020푡표푋푌푍 ∙  HSV, HLS  Where:퐺 퐺 퐵 푅709 퐵 푅2020  C) a set of numbers in a 3D space so that the (Euclidean) and distance in that space corresponds to approximately 0.4124 0.3576 0.1805 −1 perceptually uniform colour differences 푅709푡표푋푌푍 푋푌푍푡표푅709 푅709푡표푋푌푍 푀 = 0.2126 0.7152 0.0722 푀 = 푀  CIE Lab, CIE Luv 0.0193 0.1192 0.9505 and 0.6370 0.1446 0.1689 −1 푀푅2020푡표푋푌푍 = 0.2627 0.6780 0.0593 푀푋푌푍푡표푅2020 = 푀푅2020푡표푋푌푍 211 212 0.0000 0.0281 1.0610

RGB space RGB in XYZ space

 Most display devices that output light mix red, green and  CRTs and LCDs mix red, green, and blue to make all blue lights to make colour other colours  televisions, CRT monitors, LCD screens  the red, green, and blue primaries each map to a point  Nominally, RGB space is a cube in CIE xy space  The device puts physical limitations on:  any colour within the resulting  the range of colours which can be displayed triangle can be displayed  the brightest colour which can be displayed  any colour outside the triangle cannot be displayed  the darkest colour which can be displayed  for example: CRTs cannot display very saturated purple, turquoise, or yellow FvDFH Figs 13.26, 13.27

213 214

CMY space CMYK space

 printers make colour by mixing coloured inks  in real printing we use black  the important difference between inks (CMY) and lights (key) as well as CMY (RGB) is that, while lights emit light, inks absorb light  why use black?  cyan absorbs red, reflects blue and green  inks are not perfect absorbers  magenta absorbs green, reflects red and blue  mixing C + M + Y gives a muddy  yellow absorbs blue, reflects green and red grey, not black  CMY is, at its simplest, the inverse of RGB  lots of text is printed in black:  CMY space is nominally a cube trying to align C, M and Y perfectly for black text would be a nightmare

215 216

Munsell’s colour classification system Munsell’s colour classification system  three axes  any two adjacent colours are a standard “perceptual”  hue ➢ the dominant colour distance apart  value ➢ bright colours/dark colours  worked out by testing it on people  ➢ chroma vivid colours/dull colours  a highly irregular space  can represent this as a 3D graph  e.g. vivid yellow is much brighter than vivid blue

217 218 invented by Albert H. Munsell, an American artist, in 1905 in an attempt to systematically classify colours

Colour spaces for user-interfaces HSV: hue saturation value

 RGB and CMY are based on the physical devices which  three axes, as with Munsell produce the coloured output  hue and value have same meaning  RGB and CMY are difficult for humans to use for  the term “saturation” replaces selecting colours the term “chroma”  Munsell’s colour system is much more intuitive:  hue — what is the principal colour?  value — how light or dark is it?  chroma — how vivid or dull is it?  computer interface designers have developed basic ◆ designed by Alvy Ray Smith in transformations of RGB which resemble Munsell’s human- 1978 friendly system ◆ algorithm to convert HSV to RGB and back can be found in Foley et

219 220 al., Figs 13.33 and 13.34

HLS: hue lightness saturation Perceptually uniformity

a simple variation of HSV  MacAdam ellipses & visually indistinguishable colours ◆ hue and saturation have same meaning ◆ the term “lightness” replaces the term “value” designed to address the complaint that HSV has all pure colours having the same lightness/value as white ◆ designed by Metrick in 1979 ◆ algorithm to convert HLS to RGB and back can be found in Foley et al., Figs 13.36 and 13.37 In CIE xy chromatic coordinates In CIE u’v’ chromatic coordinates

221 222

CIE L*u*v* and u’v’ CIE L*a*b* colour space  Approximately perceptually uniform  Another approximately perceptually uniform colour space  u’v’ chromacity Trichromatic values of the white point, e.g.

 CIE LUV sRGB in CIE L*u*v*

Lightness

Chromacity Colours less coordinates distinguishable when dark  Chroma and hue  Hue and chroma

223 224

Lab space Colour - references  Chapters „Light” and „Colour” in  this visualization shows those colours in Lab space  Shirley, P. & Marschner, S., Fundamentals of Computer Graphics which a human can perceive  Textbook on colour appearance  Fairchild, M. D. (2005). Color Appearance Models (second.). John  again we see that human Wiley & Sons. perception of colour is not uniform  perception of colour diminishes at the white and black ends of the L axis  the maximum perceivable chroma differs for different hues

225 226

Tone-mapping problem Why do we need tone mapping?

 To reduce dynamic range  To customize the look (colour grading)  To simulate human vision (for example night vision)  To simulate a camera (for example motion blur) luminance range [cd/m2]  To adapt displayed images to a display and viewing human vision simultaneously conditions adapted  To make rendered images look more realistic Tone mapping  To map from scene- to display-referred colours

conventional display

 Different tone mapping operators achieve different goals 227 228

From scene- to display-referred colours Tone mapping and display encoding

 The primary purpose of tone mapping is to transform an  Tone mapping is often combined with display encoding image from scene-referred to display-referred colours scene-referred, display-referred, display-referred, linear, float linear, float encoded, int

Display encoding Rendered HDR SDR raster Tone mapping (inverse display image buffer model)

Different for SDR and HDR displays

 Display encoding can model the display and account for  Display contrast (dynamic range), brightness and ambient light levels

229 230

sRGB textures and display coding Basic tone-mapping and display coding

 OpenGL offers sRGB textures to automate  The simplest form of tone-mapping is the RGB to/from sRGB conversion exposure/brightness adjustment: Scene-referred  sRGB textures store data in gamma-corrected space Display-referred red value Scene-referred  sRGB convered to (linear) RGB on texture look-up (and 푠 푅 luminance of white filtering)  R for red, the same for green푑 and blue 푅 = 푤ℎ푖푡푒  Inverse display coding  No contrast compression, only퐿 for a moderate dynamic range  RGB to sRGB conversion when writing to sRGB texture  The simplest form of display coding is the “gamma”  with glEnable(GL_FRAMEBUFFER_SRGB) Prime denotes a  Forward display coding (‘) gamma-corrected value 1 Typically =2.2 ′ 훾 푑 훾  For SDR displays only 푅 = (푅 )

231 232

Tone-curve Tone-curve

The „best” tone- mapping is the one which does not do anything, i.e. slope of the tone-mapping curves is equal to 1. But in practice contrast (slope) must be limited due to display limitations. Image histogram

233 234

Tone-curve Sigmoidal tone-curves

 Very common in digital cameras  Mimic the response of analog film  Analog film has been engineered over many years to produce Global tone- good tone-reproduction mapping is a compromise  Fast to compute between clipping and contrast compression.

235 236

Sigmoidal tone mapping Sigmoidal tone mapping example

 Simple formula for a sigmoidal tone-curve:

푏 a=0.25 푅(푥, 푦) 푅′(푥, 푦) = 푏 퐿푚 푏 + 푅(푥, 푦) where is the geometric mean푎 (or mean of logarithms):

푚 퐿 1 퐿푚 = 푒푥푝 ෍ ln(퐿 푥, 푦 ) a=1 and is the luminance of푁 the(푥,푦) pixel .

퐿 푥, 푦 푥, 푦

a=4

b=0.5 b=1 b=2 237 238

Glare Illusion Glare Illusion

Photography Painting

239 240 Computer Graphics HDR rendering in games

Scattering of the light in the eye Point Spread Function of the eye

Green – daytime (photopic)  What portion of Red – night time (scotopic) the light is scattered towards a certain visual angle  To simulate:  construct a digital filter  convolve the image with that filter

From: Spencer, G. et al. 1995. Proc. of From: Sekuler, R., and Blake, R. Perception, second ed. McGraw- Hill, New York, 1990 SIGGRAPH. (1995)

241 242

Selective application of glare Selective application of glare

 A) Glare applied to the A) Glare applied to entire image Glare kernel the entire image (PSF)

 Reduces퐼푔 =image 퐼 ∗ 퐺 contrast and sharpness

B) Glare applied only to the clipped pixels Original image B) Glare applied to 푔 푐푙푖푝푒푑 푐푙푖푝푒푑 where퐼 = 퐼 + 퐼 ∗ 퐺 − 퐼 clipped pixels only 퐼 푓표푟 퐼 > 1 퐼푐푙푖푝푒푑 = ቊ Better image quality0 표푡ℎ푒푟푤푖푠푒

243 244

Glare (or bloom) in games References: Tone-mapping

 Convolution with large, non-separable filters is too slow  Tone-mapping  REINHARD, E., HEIDRICH, W., DEBEVEC, P., PATTANAIK, S., WARD, G., AND  The effect is approximated by a combination of Gaussian MYSZKOWSKI, K. 2010. High Dynamic Range Imaging: Acquisition, Display, and Image- filters Based Lighting. Morgan Kaufmann.  MANTIUK, R.K., MYSZKOWSKI, K., AND SEIDEL, H. 2015. High Dynamic Range  Each filter with different “sigma” Imaging. In: Wiley Encyclopedia of Electrical and Electronics Engineering. John Wiley &  The effect is meant to look good, not be be accurate Sons, Inc., Hoboken, NJ, USA, 1–42. model of light scattering  http://www.cl.cam.ac.uk/~rkm38/pdfs/mantiuk15hdri.pdf (Chapter 5)  Some games simulate a camera rather than the eye

245 246