History of Technology in Games Where We've Been… … Today

Where We’ve Been…

History of Technology in Games ! Processing ! Storage 3D Graphics ! Graphics ! CRT CMPUT 250 ! Vector Graphics Fall 2007 ! Raster Graphics ! Bitmaps Tuesday, October 9 ! Resolution, Colour Depth, Frame Rate ! Early Arcade Games Raster Wins

CMPUT 250: Fall 2007 Lecture #10: History 3

… Today

! 2D Graphics ! 3D Graphics ! Camera ! Sprites ! Meshes ! Layers ! Triangles ! Collision Detection ! Models ! Specialization and Generalization ! Culling ! Volume Partitioning ! History of Home Consoles ! Advances in 3D ! Shading and Texturing ! Special Effects

CMPUT 250: Fall 2007 Lecture #10: History 3 CMPUT 250: Fall 2007 Lecture #10: History 3 3D Graphics Types of Rendering

! This is a very complex subject ! Non Real-Time (e.g., raytracing) ! Video screens are 2D; World is 3D ! Trace paths of beams of light in the world and compute ! Produce a 2D image of a 3D world their colour when they hit the camera’s field of view ! Virtually build our 3D world ! Slow (minutes or hours for one frame) but high quality ! Point a virtual camera at it ! Used in movies like Pixar’s Toy Story, etc. ! How do we describe the objects in the world? ! Sometimes used to pre-render scenes in games Scene ! These images can also be used as a background ! What does our camera do? ! Real-Time Rendering ! Draw image in real-time (quickly enough to animate) ! Lower quality but world can change with player actions

CMPUT 250: Fall 2007 Lecture #10: History 3 CMPUT 250: Fall 2007 Lecture #10: History 3

The Camera The Camera: Frustum

! A camera is a view of the world ! Volume visible to camera: view frustum ! Occupies a position in 3D space (x,y,z) and points in a direction ! Viewing angle (horizontal and vertical) ! Real cameras also have: ! Focal length ! Lens Distortion ! Focus ! Depth of Field

! Fixed cameras ! Free cameras ! Chase camera (3rd person, follows behind) ! Finite position and direction (Myst, 7th Guest) ! Free-look (1st person, looks wherever you point) ! Fixed position (Myst 3) ! Look-spring (1st or 3rd camera can be moved but ! Fixed direction (some RTS, isometric view) automatically returns to its default when released) ! Rear-view (look behind you) ! Fixed path (rail-shooters like Virtua Cop) ! Droppable (move and then leave it there) ! Fixed path with free direction (rail-shooters like ! Switching from 1st to 3rd or vice versa Panzer Dragoon)

CMPUT 250: Fall 2007 Lecture #10: History 3 CMPUT 250: Fall 2007 Lecture #10: History 3

The Camera: Friend or Foe? Simple “3D” Tricks

! Obstructions in 3rd person view ! Scaling: making 2D sprites bigger or smaller to simulate moving toward or away ! Where does the chase camera go when you ! Boot Hill (Midway, 1977) back into a wall? ! Isometric view: 2D artistic style showing ! Camera won’t allow the angle you want perspective from a fixed angle ! Camera too slow ! Zaxxon (Sega, 1982) ! These aren’t really rendering ! Camera too erratic or has jerky transitions ! We want a world with real 3D objects (i.e., they occupy ! Camera too automatic or too manual some volume of space… not just an area of the screen)

CMPUT 250: Fall 2007 Lecture #10: History 3 CMPUT 250: Fall 2007 Lecture #10: History 3 The Scene Polygons

! Need to represent things in the world ! Polygons are 2D objects described by three ! Irrelevant what they look like inside or more points connected by line segments. ! Let’s make “hollow” objects ! They are closed (all line segments ! Here’s the plan: connected) and line segments do not cross ! Objects defined by surfaces (or meshes) each other. ! Surfaces built out of polygons ! A point used to describe a polygon is called ! Polygons built out of triangles a vertex (plural: vertices). ! Triangles defined by their vertices

CMPUT 250: Fall 2007 Lecture #10: History 3 CMPUT 250: Fall 2007 Lecture #10: History 3

Example of Polygons Examples of Non-Polygons

Not closed

Not line segments

Line segments cross

CMPUT 250: Fall 2007 Lecture #10: History 3 CMPUT 250: Fall 2007 Lecture #10: History 3 Triangles Coordinates

! Simplest polygon ! Two dimensional points can be described using 2D coordinates or vertices ! Nice mathematical properties ! Each coordinate is two numbers (x and y), referring to ! Can divide up more complex polygons into horizontal and vertical position triangles (tessellation) ! For our surfaces, we must use 3D coordinates (x, y, z) referring to horizontal, vertical, and depth ! Our polygons (triangles) will be described by 3D coordinates

CMPUT 250: Fall 2007 Lecture #10: History 3 CMPUT 250: Fall 2007 Lecture #10: History 3

Meshes Multiple Meshes

! A collection of triangles for a single ! Figures can be composed of multiple “object” is called a mesh meshes. The pieces (meshes) can separate. ! In the end, a mesh is just a big list of vertex coordinates

CMPUT 250: Fall 2007 Lecture #10: History 3 CMPUT 250: Fall 2007 Lecture #10: History 3 Normal Vectors How to Draw a Scene ! World is built out of many, many triangles ! Recall our objects are hollow and we only ! Place the camera in the world care about the “outside” ! We can calculate what a triangle looks like from the camera’s position ! Which side is the outside? ! But some triangles obscure others (?) ! Normal vectors point out from a triangle’s ! One idea: visible side ! sort triangles by how far away they are ! draw them from furthest to closest ! closer triangles will be drawn over top of further triangles

CMPUT 250: Fall 2007 Lecture #10: History 3 CMPUT 250: Fall 2007 Lecture #10: History 3

Problem Culling

! Too many triangles ! Reduce the number of triangles we need to draw ! Limited number can be drawn per frame ! Culling is the process of eliminating triangles ! Essentially the same problem we had in vector from the list of those we need to draw. graphics and sprite graphics ! Can obviously ignore ! Always need to consider scene complexity ! triangles behind the camera ! Many factors to complexity ! triangles out of the forward field of view up-down and ! Number of potentially visible triangles right-left ! Effects in rendering those triangles ! triangles beyond the camera’s range ! How can we draw complex worlds? ! all this is called frustrum culling ! only consider triangles inside the camera’s frustrum

CMPUT 250: Fall 2007 Lecture #10: History 3 CMPUT 250: Fall 2007 Lecture #10: History 3 The Camera: Frustum Clipping ! Volume visible to camera: frustum ! Only considering triangles inside the frustum now ! Triangle may only be partially inside ! Cut off part of triangle that is outside ! This process is called clipping

Still have problems… Philosophical Question

! With complex worlds, even computing what triangles are inside the frustum may be too slow! Ever wonder why you can’t just ! Camera’s max depth may cover most of the world blow any and every wall up? ! Need a fast way of deciding what triangles might be visible before we cull ! Let’s break the world up into manageable spaces

CMPUT 250: Fall 2007 Lecture #10: History 3 CMPUT 250: Fall 2007 Lecture #10: History 3 Static vs. Dynamic Objects Volume Partitioning ! Static Objects, e.g., walls, statues, trees ! Use static objects to divide up (partition) ! Precompute (calculate ahead of time) and store the world into separate spaces (volumes) information about the visibility of its triangles ! Use the information to speed up rendering ! Rooms are a natural way to do this ! If most of the world is static… life is good ! We can manually specify the division when ! … but boring we design the levels ! Static objects can still help ! There are more automatic methods, but we start with a portal system.

CMPUT 250: Fall 2007 Lecture #10: History 3 CMPUT 250: Fall 2007 Lecture #10: History 3

Portals Portals

! Divide world into regions connected by portals

A A B C ! Camera is limited to triangles in the same B region as it (or adjoining regions)

D C ! Allow portals to be open or closed to keep D the number of regions small

CMPUT 250: Fall 2007 Lecture #10: History 3 CMPUT 250: Fall 2007 Lecture #10: History 3 Automated Volume Partitioning

! Automated methods split the world’s volume up into pieces using geometry. ! Examples include ! Binary space partitioning ! Quadtrees ! Octrees ! Many games use a mixture of manual and automatic methods

CMPUT 250: Fall 2007 Lecture #10: History 3 CMPUT 250: Fall 2007 Lecture #10: History 3

Early 3D Early 3D: 2.5D

! Made possible by volume partitioning ! One “floor”: possibly variable height ! 2D graphics (sprites) in a 3D environment ! No rooms above rooms (except by trickery) ! The Games ! History ! Ultima Underworld (Looking Glass, 1992) ! Ultima Underworld allowed a complex environment ! Wolfenstein 3D (id Software, 1992) (angled walls, inclined surfaces) but was slow ! Doom (id Software, 1993) ! Wolfenstein 3D made simplifying assumptions (e.g. all- ! Heretic (Raven Software, 1994) right angles, all floors the same level) and was fast ! Marathon (Bungie, 1994) ! Doom relaxed restrictions (some angled walls, some ! System Shock (Looking Glass, 1994) variable height floors) while keeping the speed ! Duke Nukem 3D (3D Realms, 1996) ! Heretic allowed angling viewpoint up and down

CMPUT 250: Fall 2007 Lecture #10: History 3 CMPUT 250: Fall 2007 Lecture #10: History 3 Early 3D: Beyond 2.5D System Shock (Looking Glass, 1994)

! More History: ! System Shock broke 2.5D and allowed floors above other floors and transparency (although slow) ! This functionality might have been around in Ultima Underworld ! But, the id games were ultimately more successful ! Simpler ! Faster

Doom (id Software, 1993) Models

! Sprites look weird in a 3D world! ! We need 3D objects ! Game creatures, power-ups, doors ! Destructible parts of environment ! These are sometimes called models to distinguish them from static background

CMPUT 250: Fall 2007 Lecture #10: History 3 CMPUT 250: Fall 2007 Lecture #10: History 3 Models Quake (id Software, 1996)

! Meshes! ! Lots of triangles per model, though ! Can’t precompute anything ! Limit number of visible models ! Keep track of where model is in volume partitioning ! Ever notice that sprite-based 3D games had many, many enemies. In polygon-based, not so many

CMPUT 250: Fall 2007 Lecture #10: History 3 CMPUT 250: Fall 2007 Lecture #10: History 3

Later 3D: Sprites Replaced Modern 3D: Graphics Hardware

! Descent (Parallax, 1995) ! Early 3D games used software rendering ! Quake (id Software, 1996) ! Up to around Quake II (1997) ! Quake II (id Software, 1997) ! Computed entire process on the general-purpose CPU Sprite hardware only semi-useful ! GoldenEye 007 (Rare, 1997) ! 3D hardware rendering was very expensive ! Half-Life (Valve, 1998) ! Developed by Silicon Graphics (SGI) throughout the ! Thief (Looking Glass, 1998) late 80’s and 90’s ! Unreal (Epic, 1998) ! Commercially viable for home gaming starting ! System Shock 2 (Irrational Games/Looking Glass,1999) with the Voodoo card from 3dfx in 1996

CMPUT 250: Fall 2007 Lecture #10: History 3 CMPUT 250: Fall 2007 Lecture #10: History 3 Modern 3D: GPU Modern 3D: GPU

! Much of 3D is repetitive calculations ! More and more features added to GPUs ! Custom hardware can do this more ! Hardware transform and lighting efficiently than the general-purpose CPU ! Complex texturing operations ! Graphics Processing Unit (GPU) ! Completely programmable “pixel shaders” – ! Initially, vertex information computed by CPU more like a traditional CPU! and sent to GPU to draw with textures ! Sound familiar? ! Expect to see physics and collision detection soon!

CMPUT 250: Fall 2007 Lecture #10: History 3 CMPUT 250: Fall 2007 Lecture #10: History 3

Rendering Triangles Rendering: Wireframe

! Now we know what triangles to draw ! Just draw the lines: wireframe ! How do we draw them? ! A little math figures out what the triangle’s shape is from the camera’s perspective (transform) ! What do we draw?

! Fill triangles with a single, solid colour ! Draw filled triangles shaded according to light ! Soecify colour for each triangle with RGB ! Flat-shading values ! Specify colours for triangles ! Problem: adjacent faces of ! Specify light(s) ! Compute effect of light the same colour indistinguishable on filled triangles ! Single (but different) colour for each triangle ! Gives a proper 3D impression, but crude

Rendering: Smooth-Shading Rendering: Texturing

! Shade each triangle according to lights ! Fill triangles with an image ! Texturing ! Smooth-shading ! Specify entire image for surface ! Colour varies over triangle (i.e., a colour for every pixel) ! Smoother transitions between ! Texture pixels called texels ! Can shade texels according adjacent triangles to light as well ! Texture mapping: stretch image over an entire mesh, instead of just a single triangle

! Texturing adds a huge amount of realism ! If you get very close to a texture, it may look ! Can add surface detail without adding triangles “blocky” because you can see individual texels ! Textures can even be animated (i.e., change the ! Far away objects don’t need much detail image over time) ! Standard solution: mipmapping ! However, requires much more storage than ! copies of same texture at multiple resolutions shaded triangles ! switch copies as camera moves further/closer ! A huge part of modern gaming technology is ! Switching resolutions can cause odd problems concerned with allowing more textures, with ! partially solved by filtering: bilinear, trilinear, higher resolution, and moving texture information anisotropic around efficiently

CMPUT 250: Fall 2007 Lecture #10: History 3 CMPUT 250: Fall 2007 Lecture #10: History 3

Level of Detail Advanced 3D

! Mipmapping is an example of a level of detail ! It’s all about the effects (LOD) approach ! What about windows? ! Basic idea is that far away things need less detail ! What about fog or mist? ! What about smooth versus rough surfaces? ! Other LOD approaches, include ! What about shiny surfaces? ! Changing meshes at different distances (fewer triangles ! What about mirrors? when far away) (e.g., “Messiah” by Shiny, 2000) ! What about water or fire? ! Dropping lights, shadows, and any other special ! What about realistic clothing or hair or fire or smoke? features when objects are far away ! What about shadows?

CMPUT 250: Fall 2007 Lecture #10: History 3 CMPUT 250: Fall 2007 Lecture #10: History 3 Transparency Lights

! Colour the light that passes through translucent objects ! Another very complicated subject ! Do this by adding an extra value to every pixel (texel) ! Red, green, blue, and alpha ! Basic properties ! Alpha specifies how transparent the pixel is ! ! When drawing the scene, transparent pixels modify the Intensity (brightness) value of stuff behind instead of overwriting it ! Colour ! Keep in mind that transparency means more visible ! Source triangles! ! Advanced properties ! Diffusion, specular highlights, reflection

CMPUT 250: Fall 2007 Lecture #10: History 3 CMPUT 250: Fall 2007 Lecture #10: History 3

Kinds of Light Diffuse vs. Specular Lighting ! Diffuse lighting refers to surfaces absorbing and ! Ambient light then emitting light uniformly over the surface ! Artificial concept of light that is “everywhere” in the ! Angle of incoming light matters scene – all surfaces are lit by it equally (affects quantity of emitted light) ! Everything is visible to some minimum extent ! But viewer’s angle does not ! Point-source lights ! Specular lighting “bounces” off the surface ! Placed in the world (like the camera) ! light comes in at an angle and leaves at an angle ! Shine in all directions ! viewer’s angle matters ! Directional lights ! responsible for “shiny spots” ! Placed in the world (like the camera) or “glare” on objects ! Shine in a cone

CMPUT 250: Fall 2007 Lecture #10: History 3 CMPUT 250: Fall 2007 Lecture #10: History 3 Specular Highlights Reflectivity

! The reflectivity of a material determines how much light is absorbed versus reflected. ! Highly reflective surfaces create reflections ! Parts of the scene appearing in shiny surfaces ! Expensive to do properly, so faked instead

CMPUT 250: Fall 2007 Lecture #10: History 3 CMPUT 250: Fall 2007 Lecture #10: History 3

Environment Mapping Environment Mapping Cube Map Cube Map

! If lights are static, we can precompute their Texture Lightmap Rendered surface effect on a surface and store it ! The stored information is called a lightmap ! Combined with the texture of a surface + = during rendering to change its look

! Reuse textures with different lightmaps ! Saves space

Bump Mapping Bump Mapping

! A per pixel lighting effect ! Each pixel specifies its own surface normal ! Light is bounced off each pixel relative its individual surface normal ! Great for adding roughness to surfaces

CMPUT 250: Fall 2007 Lecture #10: History 3 CMPUT 250: Fall 2007 Lecture #10: History 3 Halo (Bungie Studios, 2003) Halo (Bungie Studios, 2003)

CMPUT 250: Fall 2007 Lecture #10: History 3 © www.earthenrecords.com CMPUT 250: Fall 2007 Lecture #10: History 3 © www.earthenrecords.com

Bump Mapping: Angle Sensitive Lighting: Finer than Triangles

! So far, lighting behaviour specified for each triangle ! Contemporary lighting doesn’t stop at triangles ! Lighting can be per vertex ! Lighting specified for each vertex and “blended” over the triangles ! Lighting can be per pixel ! Each pixel on the surface specifies its own lighting behaviour

CMPUT 250: Fall 2007 Lecture #10: History 3 CMPUT 250: Fall 2007 Lecture #10: History 3 Materials More, more, more

! We seem to be collecting a lot of information to describe triangles ! Pixel shaders ! Information about drawing triangles is often called ! Shadows material information ! Colour/texture ! Particle effects ! Transparency (alpha) ! Reflectivity ! Cel-shading ! Special effects (e.g. bump mapping info) ! Precomputed effects (e.g. light maps, shadow maps) ! 3D Collisions ! Games are using more and more detailed material ! More realism… more hardware information – it all costs storage!

CMPUT 250: Fall 2007 Lecture #10: History 3 CMPUT 250: Fall 2007 Lecture #10: History 3

In Summary… Questions?

! 3D graphics is a very complex subject ! Video screens are 2D; World is 3D ! Produce a 2D image of a 3D world ! Virtually build our 3D world ! Point a virtual camera at it ! How do we describe the objects in the world? Scene ! What does our camera do? Rendering

CMPUT 250: Fall 2007 Lecture #10: History 3 CMPUT 250: Fall 2007 Lecture #10: History 3 Post Mortems Why Post Mortems?

! A critical examination of the game design ! The process is good for the designers. process and the resulting final product. ! Critically analyze the design process. ! What went right? ! Identify mistakes to be avoided in the future. ! What went wrong? ! By the designers! ! Can also benefit other designers. ! They know their design goals. ! Inside track on their design goals and process. ! They know the process and decisions that went ! Learn from others mistakes. into their game.

CMPUT 250: Fall 2007 Lecture #10: History 3 CMPUT 250: Fall 2007 Lecture #10: History 3

Katamari Damacy (2004)

CMPUT 250: Fall 2007 Lecture #10: History 3