An Introduction to Computer Graphics

Joaquim Madeira

Beatriz Sousa Santos

Universidade de Aveiro

1 Topics

 What is CG  Brief history  Main applications  CG Main Tasks  Simple Graphics system  CG APIs  2D and 3D  Illumination and Computer Graphics

 The technology with which pictures, in the broadest sense of the word, are  Captured or generated, and presented  Manipulated and / or processed  Merged with other, non-graphical application data

 It includes:  Integration with other kinds of data – Multimedia  Advanced dialogue and interactive technologies

3 Computer Graphics

 Computer Graphics deals with all aspects of creating images with a computer

 Hardware

 Software

 Applications

[Angel]

4 Computer Graphics: 1950 – 1960

 Earliest days of computing

 Pen plotters

 Simple calligraphic displays

 Issues

 Cost of display refresh

 Slow, unreliable, expensive computers

5 Computer Graphics: 1960 – 1970

 Wireframe graphics

 Draw only lines !

[Angel]

6 Computer Graphics: 1960 – 1970

 Ivan Sutherland’s Sketchpad

 PhD thesis at MIT (1963)

 Man-machine interaction

 Processing loop  Display something  Wait for user input  Generate new display

[http://history-computer.com]

7 Sketchpad (Ivan Sutherland, 1963)

http://www.youtube.com/watch?feature=endscreen&NR=1&v=USyoT_Ha_bA 8

Computer Graphics: 1970 – 1980

 Raster graphics

 Allows drawing polygons

 First graphics standards

 Workstations and PCs

9 Raster graphics

 Image produced as an array (the raster) of picture elements (pixels) in the frame buffer

[Angel]

10 Raster graphics

 Drawing polygons

 Illumination models

 Shading methods

[Angel]

11 – 1971

[Wikipedia]

12 Gouraud shading

Poor highlight Very high polygon count

http://en.wikipedia.org/wiki/Gouraud_shading

13 Phong shading– 1973

[Wikipedia]

14 – 1973

[Wikipedia]

15 Can you see the differences ?

[Foley , Van Dam]

16 Computer Graphics: 1980 – 1990

 The quest for realism

Smooth shading Environment mapping Bump mapping

[Angel]

17 Ray-Tracing example

http://radsite.lbl.gov/radiance/book/img/plate10.jpg

19 https://en.wikipedia.org/wiki/Ray_tracing_(graphics)

“Vermeer’s Studio”

Wallace & Cohen, 1987: Radiosity and Ray-Tracing

22 Radiosity

Radiosity

https://en.wikipedia.org/wiki/Radiosity_(computer_graphics)

Radiosity

Without radiosity With radiosity

[Burdea]

24

Smooth shading Environment mapping Bump mapping

[Angel]

25 Computer Graphics: 1980 – 1990

 Special purpose hardware

 Industry-based standards

 PHIGS

 RenderMan (https://renderman.pixar.com/)

 Human-Computer Interaction

27 Computer Graphics: 1990 – 2000

 OpenGL API

 First successful computer-generated feature- length film: Toy Story

 New hardware capabilities

28 Oscar winner 2017- Piper

https://renderman.pixar.com/stories https://www.youtube.com/watch?v=lkQTe0Wdo2k http://oscar.go.com/news/winners/piper-is-the-2017-oscar-winner-for-short-film-animated

Computer Graphics: 2000 – …

 Photorealism

 Graphics cards for PCs dominate the market

 Nvidia

 ATI

 Game boxes / players determine the market

 CG is routine in the film industry

30 The first “two waves” of CG research

 Product design and manufacturing improvements drove research from the 1960s through the 1980s

 Arts and entertainment became the dominant driver in the mid-1980s, but has been decreasing

 And now?

Kasik, D. J. (2011). The third wave in Computer Graphics and Interactive Techniques. IEEE Computer Graphics And Applications, 31(4), 89–93. CG – Application areas

 Entertainment  Computer games  Animation films  Special effects  Engineering / Architecture  Computer-Aided Design (CAD)  Data Visualization  Medicine  Visualization  …

32 Games – Lara Croft

1996 2007 2013

[Wikipedia] 33 Animation films – Pixar

Toy Story – 1995 Ratatouille – 2007 Toy Story – 2014

[www.pixar.com]

34 Special effects – ILM

1991 1994 1999

[Wikipedia]

36 Special effects – ILM

2005 2009 2015 [Wikipedia]

37 Computer-Aided Design

CAD mockup

[Wikipedia]

38 CAD – Simulation

[Wikipedia]

39 Virtual and Augmented Reality

Virtual Reality – examples Industry

Jaguar/ Land Rover

http://www.youtube.com/watch?v=j3qcnvgVlNk 41

Medicine

http://www.youtube.com/ watch?v=zJmrcEM-uvA

43 Entertainment

Oculus Rift 2014; ~300 USD

http://www.oculusvr.com/

http://www.youtube.com/watch?v=N8uuDT5AYts44

CG is not alone… Visualization

 Core areas: IP CG HCI  CG, IP, CV and HCI CV  Satellite areas: Geometric  Geometric Modeling Modeling

 Data and Information Visualization  What is common?

 CG, IP : image file formats, color models, …

 CG, CV : 3D model representations, …

 IP, CV : noise removal, filters, …

45 Example – Medical Imaging

 Processing pipeline

 Noise removal

 Segmentation

 Generating 2D / 3D models

 Data visualization

 User interaction

 …

[www.mevislab.de]

RVA - 2014/2015 47 CG Main Tasks

 Modeling

 Construct individual models / objects

 Assemble them into a 2D or 3D scene

 Animation

 Static vs. dynamic scenes

 Movement and / or deformation

 Rendering

 Generate final images

 Where is the observer?

 How is he / she looking at the scene?

48 Basic Graphics System

[Angel]

 Image formed in the frame buffer

49 Computer Graphics APIs

 Create 2D / 3D scenes from simple primitives

 OpenGL  Rendering  No modeling or interaction facilities

 Direct 3D – Microsoft

 VTK  3D CG + Image processing + Visualization

 Three.js …

50 API contents

 Functions for specifying / instantiating

 Geometric primitives

 Materials

 Light sources

 Viewer / Camera

 …

 Functions for simple user interaction

 Input from devices: mouse, keyboard, etc.

51 Geometric Primitives

 Simple primitives

 Points

 Line segments

 Polygons

 Geometric primitives

 Parametric curves / surfaces

 Cubes, spheres, cylinders, etc.

52 Lights and materials

 Types of light sources  Point vs distributed light sources  Spot lights  Near and far sources  Color properties

 Material properties  Absorption: color properties  Scattering: diffuse and specular  Transparency

53 Camera specification

 Position and orientation

 Lens

 image size

 Orientation of image plane

[Angel]

54 OpenGL

 Multi-platform API for rendering 2D and

 Interaction with the GPU to achieve hardware-accelerated rendering

 Application areas  CAD   Scientific and Information Visualization  …

55 OpenGL

 OpenGL ES

 Subset for use in embedded systems and portable devices

 WebGL

 JavaScript API based on OpenGL ES 2.0

 Rendering interactive 2D and 3D graphics on any compatible browser, without the use of plug-ins

56 Three.js

 Cross-browser JavaScript library/API used to create and display animated 3D computer graphics in a web browser.  Uses WebGL

https://threejs.org/

Thee.js first example 1. Defining the scene, the camera and where the scene is rendered var scene = new THREE.Scene(); var camera = new THREE.PerspectiveCamera( 75, window.innerWidth / window.innerHeight, 0.1, 1000 ); var renderer = new THREE.WebGLRenderer(); renderer.setSize( window.innerWidth, window.innerHeight ); document.body.appendChild(

renderer.domElement ); 58 2.Creating an object and camera position

var geometry = new THREE.BoxGeometry(1,1,1); var material = new THREE.MeshBasicMaterial( { color: 0x00ff00 } ); var cube = new THREE.Mesh( geometry, material ); scene.add( cube ); camera.position.z = 5;

59 3. Scene rendering function render() { requestAnimationFrame(render); renderer.render(scene, camera); } render();

4. Scene animation

cube.rotation.x += 0.1; cube.rotation.y += 0.1;

60 Adding lights and shading

var material = new THREE.MeshPhongMaterial({ ambient: '#006063', color: '#00abb1', specular: '#a9fcff', shininess: 100 });

61 2D Visualization

 Define a 2D scene in the world coordinate system

 Select a clipping window in the XOY plane

 The window contents will be displayed

 Select a viewport in the display

 The viewport displays the contents of the clipping window

62 World -> display y

y

Display Coordinates x

World Coordinates x Coordinate mapping

World Coordinates Screen coordinates

64 Coordinate mapping

If the aspect ration is not the same in both situations the result is distortion

65 World -> screen y

y

Screen Coordinates x

The aspect ration is not the same in both situations: distortion!

World Coordinates x 3D Viewing

68 3D Viewing

 Where is the observer / the camera ?

 Position ?

 Close to the 3D scene ?

 Far away ?

 How is the observer looking at the scene ?

 Orientation ?

 How to represent as a 2D image ?

 Projection ?

69  Obtaining an image of the scene using perspective View frustum

image [Interactive 3D Graphics, Udacity] Light and Rendering

 In the real world the light emits rays that are reflected by objects and seen by the eye

 Computing this is too time consuming

{Interactive 3D Graphics, Udacity] Reversing the process in CG

 In CG simplifying assumptions may be made

 Start from the camera

 No shadows

 Only the rays that matter are processed

3D visualization pipeline

 Instantiate models of the scene

 Position, orientation, size

 Establish viewing parameters

 Camera position and orientation

 Compute illumination and shade polygons

 Perform clipping

 Project into 2D

 Rasterize 73 3D visualization pipeline

 Each object is processed separately

 Typically 3D triangles (e.g. a cube or a sphere are made of triangles)

 Triangles are modified by the camera view of the world

 Is the object inside the view frustum? (No -> next object!)

 Yes -> compute the location on the screen of each triangle (rasterization)

 Compute the color of each pixel

74

3D scene

Geometry

Material

Light

(animation)

+

Camera

{Interactive 3D Graphics, Udacity] Projection

Parallel Projection Perspective Projection (allows measures) (more realistic images)

76 Projections

Parallel Projection Perspective Projection

77 Parallel Projections

Ortographic / Axonometric projection

Oblique projection

Ortographic/ Multiview projection

78 Perspective Projections

79 Perspective Projections

80 How to limit what is observed ?

 Clipping window on the projection plane

 View volume (frustum) in 3D

Clipping plane

Clipping window

Clipping plane

82 Lighting

 Compute surface color based on

 Type and number of light sources

 Illumination model  Phong: ambient + diffuse + specular components

 Reflective surface properties

 Atmospheric effects  Fog, smoke  Polygons making up a model surface are shaded

 Realistic representation

85 Phong reflection model

Empirical model of the local illumination of points on a surface

It describes the way a surface reflects light as a combination of the diffuse reflection of rough surfaces with the specular reflection of shiny surfaces and a component of ambient light

86 [Wikipedia] Phong Model – Ambient illumination

 Constant illumination component for each model

 Independent from viewer position or object orientation !

 Take only material properties into account !

87 Phong Model – Ambient illumination

88 Phong Model – Diffuse reflection

 Model surface is an ideal diffuse reflector

 What does that mean ?

 Independence from viewer position !

 Unit vectors !!

89 Phong Model – Specular reflection

Normal To light Reflection

To viewer

 Important for shiny model surfaces

 How to model shininess ?

 Take into account viewer position !

 Unit vectors !

91 Phong Model – Specular reflection

92 Phong Model – Specular reflection

and

or

93 More than one light source

94 Illumination and shading

 How to optimize?

 Fewer light sources

 Simple shading method

 BUT, less computations mean less realism

 Wireframe representation

 Flat-shading

 Gouraud shading

 Phong shading

96 Flat shading

 For each polygon:

 Applies the illumination model just once  All pixels have the same color

 smooth objects seem “blocky”

 It is fast Apply the Gouraud shading illumination model at vertices

 For each triangle:

 Applies the illumination model at each vertex  Interpolates color to shade each pixel

 It provides better results than flat shading

 But highlights are not rendered correctly

highlight Gouraud shading

[Wikipedia] Same object with a much higher n. of faces

99 Phong shading

 Interpolates normals across rasterized polygons  computes pixel colors based on the interpolated normals

 It provides better results than Gouraud shading  But is more time consuming

highlight Flat shading vs Phong shading

[Wikipedia]

101 Test these examples in three.js: Material and light properties

https://threejs.org/examples/#webgl_geometry_teapot

Wire frame Flat shading

Gouraud shading Phong shading

https://threejs.org/examples/#webgl_geometry_teapot https://threejs.org/examples/?q=phong #webgl_materials_variations_phong

Physically accurate lighting

https://threejs.org/examples/#webgl_lights_physical

Examples in three.js (with code) Orthogonal camera; Ambient light + direction light; Lambertian objects

https://threejs.org/examples/#canvas_camera_orthographic

…

… (uses Gouraud shading)

Clipping

https://threejs.org/examples/#webgl_clipping

…

… Basic 2D Transformations x P = p = (x, y)  original point y

p’ = (x’, y’)  transformed point x’ P’ = y’

• Basic transformations:

- Translation Vector notation

- Scaling

- Rotation

114 Translation

• It is a rigid body transformation (it does not deform the object)

• To apply a translation to a line segment we need only to transform the end points

• To apply a translation to a polygon we need only to transform the vertices

115 Translation

• It is necessary to specify translations in x and y

x’ = x + tx y’ = y + ty

x x’ tx P = P’ = T = y y’ ty

P´ = P + T

transformation matrix Rotation

• To apply a rotation we need to specify:

- a point (center of rotation)

(xr,yr)

- A rotation angle θ (positive - counter-clockwise)

Positive rotation

117 Rotation around the origin

• The simplest case:

x’= r cos (Φ + θ) = r cos Φ cos θ – r sin Φ sin θ y’= r sin (Φ + θ) = r cos Φ sin θ + r sin Φ cos θ

Polar coordenates of the original point: x = r cos Φ r sin (Φ + θ) y = r sin Φ

Replacing:

x’ = x cos θ – y sin θ y´ = x sin θ + y cos θ r cos (Φ + θ)

118

2D Rotation r sin (Φ + θ)

r cos (Φ + θ)

119 Scaling

• Modifies the size of an object; we need to specify scaling

factors: sx and sy

x’ = x . sx y’ = y . sy

Trasformation matrix Transforming a square into a larger square applying a P’ = S . P scaling sx=2, sy=2

120 2D Transformations

 Matrix representation

 Homogeneous coordinates !!

 Concatenation = Matrix products

 Complex transformations ?

 Decompose into a sequence of basic transformations

121 Homogeneous coordinates

• Most applications involve sequences of transformations

• For instance:

- visualization transformations involve a sequence of translations and rotations to render an image of a scene

- may imply that an object is rotated and translated between two consecutive frames

• Homogeneous coordinates provide an efficient way to represent and apply sequences of transformations

122  It is possible to combine in a matrix the multiplying and additive terms if we use 3x3 matrices

 All transformations may be represented by multiplying matrices

 Each point is now represented by 3 coordinates

( x, y )  (xh, yh, h), h = 0

x = xh / h y = yh / h

( x.h, y.h, h)

123 2D Translation

124 2D Rotation r sin (Φ + θ)

r cos (Φ + θ)

125 2D Scaling

126 Concatenation

127 Concatenation

128 Arbitrary Rotation

Translation + Rotation + Inverse Translation

129 Arbitrary Scaling

Translation + Scaling + Inverse Translation

130 Order is important !

131 3D Transformations

 Translation

 Scaling

132 3D Rotation

 Rotation around each one of the coordinate axis

 Positive rotations are CCW (counter clock wise)!!

133 Rotation around ZZ’

134 Rotation around XX’

135 Rotation around YY’

136 Other useful 3D Transformations

• • Shears

• Reflections

Shear in ZZ’

137 How to apply Projections?

• • Also by matrix multiplication

Example: Matrix of the orthographic projection on the xy plane in homogeneous coordinates:

zz coordinates are discarded

y

z 138 x Transformations in three.js

https://threejs.org/docs/#api/math/Matrix4

Some reference books

 D. Hearn and M. P. Baker, Computer Graphics with OpenGL, 3rd Ed., Addison-Wesley, 2004

 E. Angel and D. Shreiner, Introduction to Computer Graphics, 6th Ed., Pearson Education, 2012

 J. Foley et al., Introduction to Computer Graphics, Addison-Wesley, 1993

 Hughes, J., A. Van Dam, et al., Computer Graphics, Principles and Practice, 3rd Ed., Addison Wesley, 2013

149 Acknowledgments

 Thanks are due to

 Paulo Dias

 Samuel Silva

150