Graphics and Genres the Early Days Hardware Vs. Software

The Early Days

! The very earliest video games (pre-1975) Graphics and Genres were custom built machines. ! Designed/built by engineers (like a TV). Tennis for Two CMPUT 299 (Brookhaven Labs,1958)

Finnegan Southey

XBox Live GamerTag: Alea CMPUT 299 - Fall 2005 Graphics, Genres, and Design Pong (Atari, 1972)

Hardware vs. Software Development of Early Games

! Later games (1975 to present) were built ! Programs written by one individual. using new-fangled microprocessors ! Graphics, sound, controls, rules, AI… ! General-purpose hardware that runs software. …all by one person. ! No need to engineer from scratch for every game.

Gunfight (Taito, 1975)

CMPUT 299 - Fall 2005 Graphics, Genres, and Design CMPUT 299 - Fall 2005 Graphics, Genres, and Design Development of Early Games State vs. Dynamics

! Games were simple. ! Fundamental distinction in computing (and ! The machines were still quite simple many other things) ! Very limited storage ! State ! no “pictures” or recorded music ! All information that describes the game at a given ! Limited speed moment ! focus on moving small things around on the screen ! Dynamics ! Only so much one could do ! The way one state turns into another ! more people would be a waste of effort

CMPUT 299 - Fall 2005 Graphics, Genres, and Design CMPUT 299 - Fall 2005 Graphics, Genres, and Design

State Dynamics

! positions of all game entities ! agents moving due to player controls ! walls ! agents moving due to artificial intelligence ! resources: health, magic points, money, fuel, etc. ! agents moving due to “physics” ! points: score, tokens collected, goals achieved ! agents not moving due to “collisions” ! inventories: weapons, magic items, gadgets, food ! updating view of the world ! switches: doors locked/unlocked, levers pulled ! health changes from damage/healing ! velocities of game entities ! special powers/action executed ! character names, stats, description ! much, much more… ! everything that changes one state into another…

CMPUT 299 - Fall 2005 Graphics, Genres, and Design CMPUT 299 - Fall 2005 Graphics, Genres, and Design Memory and Processors Memory Size

! Two main parts of a computer ! Memories have a size ! Memory stores state ! in the 70’s, measured in kB (kilobytes) ! in the late 80’s and 90’s, measured in MB (megabytes) ! Processors access and change memory (1 MB = 1000 kB) (dynamics) ! late 90’s to present day, measure in GB (gigabytes) (1 ! The main processor in a computer is called GB = 1000 MB) the central processing unit (CPU). ! Early games had 4 kB ! Games today will cheerfully use several GB CPU Memory ! a factor of one million (1 DVD holds over 4 GB)

CMPUT 299 - Fall 2005 Graphics, Genres, and Design CMPUT 299 - Fall 2005 Graphics, Genres, and Design

Processors Memory Types

! Processors are rated by how quickly they can ! Memories can be perform calculations on things in memory ! fast or slow ! A program is a set of instructions that tells the ! memory for showing graphics and doing processor what to do with the information in calculations is fast (it can be accessed quickly) memory. ! harddisks and DVDs are much slower but hold ! Programmers write these instructions. much more ! Early microprocessor games were a program + a ! read-only or read/write tiny amount of data ! read-only memory (like a game’s DVD) cannot be changed ! A contemporary game is a program + lots and lots ! read/write memory can be changed of data

CMPUT 299 - Fall 2005 Graphics, Genres, and Design CMPUT 299 - Fall 2005 Graphics, Genres, and Design Memory Types Storage for Games

! Another distinction ! RAM chips: volatile, read-write ! volatile: erased when power turned off ! microchips for running game ! non-volatile: retains information unpowered ! part of game machine ! And another ! ROM chips: non-volatile, read-only ! moving parts vs. no moving parts ! expensive per amount of memory ! can affect reliability/portability ! microchips

CMPUT 299 - Fall 2005 Graphics, Genres, and Design CMPUT 299 - Fall 2005 Graphics, Genres, and Design

Storage for Games Storage for Games

! cartridges ! cassette (magnetic tape): non-volatile, read-write ! ROM chips in a plastic case ! moving parts ! connectors attach chip to game machine ! early personal computers (and some arcade?) ! durable ! not very durable ! cartridges with save memory ! sequential access (have to rewind/fast forward to ! ROM chips + non-volatile, read/write memory chips reach different information) ! allows storage of saved games/high scores ! floppy disk: non-volatile, read-write ! memory cards ! moving parts ! small cartridges with non-volatile, read/write memory ! early personal computers chips ! not very durable ! used to save games when cartridges went out of style ! “random” access (can quickly access any piece of CMPUT 299 - Fall 2005 Graphics, Genres, and Design CMPUTinformation) 299 - Fall 2005 Graphics, Genres, and Design Storage for Games Storage for Games

! hard-disk: non-volatile, read-write ! magnetic strip cards: non-volatile, read-write ! moving parts ! arcade for saving stats/games (Initial D, F-Zero AX, ! later personal computers and XBox Tekken 5) ! much larger storage and faster access than floppy ! very limited storage ! expensive ! very cheap ! laser disc: non-volatile, read-only ! CD-ROM: non-volatile, read-only ! moving parts, tons of storage ! moving parts, lots of storage ! able to store full screen video ! cheap to mass produce ! expensive (videophile technology) ! arcade (Killer Instinct) and later personal computers ! arcade (Dragon’s Lair, Mach 3, Space Ace) ! consoles (3DO, Saturn, Playstation) (N64 still cartridge!)

CMPUT 299 - Fall 2005 Graphics, Genres, and Design CMPUT 299 - Fall 2005 Graphics, Genres, and Design

Storage for Games Software-Based Games

! DVD-ROM: non-volatile, read-only ! Once microprocessors were used for games, ! moving parts, tons of storage programmers took control. ! cheap to mass produce ! personal computers ! The advent of personal ! consoles (XBox, Playstation2) computers (~1976) opened ! Custom disc formats (Dreamcast, Gamecube) up the field to “amateurs”. Gunfight (Taito, 1975) ! Coming up: DVD-HD and BluRay (PS3) ! bigger DVD ! Basement game programmers ! Piracy! (arrrr…)

CMPUT 299 - Fall 2005 Graphics, Genres, and Design CMPUT 299 - Fall 2005 Graphics, Genres, and Design Cathode Ray Tubes (CRT) Diagram of CRT

! Braun (1897) - CRT oscilloscope ! Zworykin (1929) kinescope (early TV) ! Du Mont (1931) first commercial TV tube Tennis for Two (Brookhaven Labs,1958)

Diagram courtesy of Wikipedia CMPUT 299 - Fall 2005 Graphics, Genres, and Design CMPUT 299 - Fall 2005 Graphics, Genres, and Design

Very pretty, but where’s the gamepad?

Video courtesy of the University of Illinois, Dept. of Chemistry Diagram courtesy of Williamson Labs CMPUT 299 - Fall 2005 Graphics, Genres, and Design CMPUT 299 - Fall 2005 Graphics, Genres, and Design Games with Vector Graphics Vector Graphics (no more after 1985) ! Use CRT’s beam like a pen. ! Asteroids (Atari, 1979) ! Turn beam on and off to draw lines. ! Lunar Lander (Atari, 1979) ! Use magnets to guide it. ! Battlezone (Atari, 1980) ! How many lines you can draw depends on ! Red Baron (Atari, 1980) ! how fast you can move the beam around ! how long the image stays on the fluorescent ! Tempest (Atari, 1980) screen ! Space Fury (Sega, 1981) ! Draw all lines for one frame… repeat. ! Star Wars (Atari, 1983)

CMPUT 299 - Fall 2005 Graphics, Genres, and Design CMPUT 299 - Fall 2005 Graphics, Genres, and Design

Cool Things About Vector Issues with Vector Graphics Graphics ! Time to draw frame depends on complexity ! Wireframe (3D) of frame ! Smooth lines (even diagonals) ! Beam moves in arbitrary pattern ! It’s all green and glowy and stuff… ! different patterns possible for same picture ! Makes me feel like I’m in a submarine or ! what’s the best (fastest) pattern? something… ! Directly control beam with game

CMPUT 299 - Fall 2005 Graphics, Genres, and Design CMPUT 299 - Fall 2005 Graphics, Genres, and Design Raster Graphics Raster (aka Bitmap) Images

! Basis of TV and almost all graphics today ! Treat screen like a “grid” ! Move beam in fixed pattern lighting up the screen in little dots ! These dots are called pixels 0 0 0 0 ! Change beam intensity to make pixels 0 1 1 0 brighter or darker 0 1 1 0 0 1 1 0 CMPUT 299 - Fall 2005 Graphics, Genres, and Design CMPUT 299 - Fall 2005 Graphics, Genres, and Design

Rendering the Screen Grayscale 0 0 0 0 Video Hardware

0 1 1 0 Screen Image 0 1 1 0 0 0 0 0 0 1 1 0 0 100100 0 Image Buffer 0 50 50 0 0 10 10 0 Game Software

CMPUT 299 - Fall 2005 Graphics, Genres, and Design CMPUT 299 - Fall 2005 Graphics, Genres, and Design CRT Screen Refresh Colour Pixels

0 0 0 0 0 1 1 0 0 1 1 0 0 1 1 0

CMPUT 299 - Fall 2005 Graphics, Genres, and Design CMPUT 299 - Fall 2005 Graphics, Genres, and Design

Diagram of Colour CRT RGB Images

0 0 0 0 0 0 0 0 0 0 0 0 0 100100 0 0 100100 0 0 100100 0 0 50 50 0 0 50 50 0 0 50 50 0 0 10 10 0 0 10 10 0 0 10 10 0

CMPUT 299 - Fall 2005 Graphics, Genres, and Design CMPUT 299 - Fall 2005 Graphics, Genres, and Design How many pixels? How many colours? ! If we make pixels smaller, we can fit more! ! Fineness of grid called resolution (width x height) ! Suppose we’re using grayscale. ! Typical television resolution ! If we have 10 different values for ! grid of 648 x 486 (~300,000 pixels) brightness, then we get black, white, and 8 ! Computer screens, HDTV shades of gray. ! 1024 x 768 (~780,000 pixels) ! ! 1280 x 1024 (~1.3 million pixels) If we have 10 different values for each of ! 1600 x 1200 (~1.9 million pixels) red, green, and blue, then we have ! Old games ! 10 x 10 x 10 colours (1000) ! Space Invaders - 224 x 240 (53,760 pixels)

CMPUT 299 - Fall 2005 Graphics, Genres, and Design CMPUT 299 - Fall 2005 Graphics, Genres, and Design

Colour Depth Framerate ! How long does it take to “draw” the next frame? ! The number of values we have for colours is ! If it takes a tenth of a second often called colour depth and is usually ! we can draw 10 frames per second k measured in bits (k bits = 2 values) ! Common framerates ! 4 bits = 16 x 16 x 16 = 4096 colours ! Minimum for “smooth” motion ~ 15 fps ! 8 bits = 256 x 256 x 256 = ~ 16m colours ! Film ~ 24 fps ! TV ~ 30 fps ! 24 bits = 16m x 16m x 16 m = ! Games ~4700000000000000000000 ! RTS ~ 8 fps ! 32 bits = 4b x 4b x 4b = 7.9 x 10^28 ! platformers ~ 30 fps ! first person shooters ~60 fps

CMPUT 299 - Fall 2005 Graphics, Genres, and Design CMPUT 299 - Fall 2005 Graphics, Genres, and Design Raster Side Effect: Aliasing

a.k.a “the jaggies”

CMPUT 299 - Fall 2005 Graphics, Genres, and Design CMPUT 299 - Fall 2005 Graphics, Genres, and Design

Early Raster Games Vector vs. Raster ! Space Invaders (Taito, 1978) ! Super Breakout (Atari, 1978) Vector Raster ! Galaxians (Midway, 1979) ! Pacman (Namco/Midway), 1980) ! clean lines ! aliasing ! Berzerk (Stern, 1980) ! outlines only ! filled areas ! Centipede (Atari, 1980) ! Defender (Williams, 1980) ! screen refresh ! screen refreshes at ! Cheeky Mouse (Universal Corp., 1980) depends on frame a fixed rate ! Missile Command (Atari, 1980) ! direct control ! image buffer ! Warlords (Atari, 1980) ! Donkey Kong (Nintendo, 1981) ! special monitors ! home TV’s

CMPUT 299 - Fall 2005 Graphics, Genres, and Design CMPUT 299 - Fall 2005 Graphics, Genres, and Design Vectrex Last Time…

! Vector did make it to the home! ! Computational Model ! Storage for Games ! Graphics ! CRT ! Vector Graphics ! Raster Graphics ! Bitmaps ! Resolution, Colour Depth, Frame Rate

CMPUT 299 - Fall 2005 Graphics, Genres, and Design CMPUT 299 - Fall 2005 Graphics, Genres, and Design

This time…

! Raster wins Background ! More 2-D graphics ! Sprites, Collisions, Layers, Scrolling ! Specialization in Hardware ! Trends in specialization/generalizations ! 3-D Graphics

CMPUT 299 - Fall 2005 Graphics, Genres, and Design CMPUT 299 - Fall 2005 Graphics, Genres, and Design CRT Screen Refresh

Background 0 0 0 0 0 1 1 0 0 1 1 0 0 1 1 0

CMPUT 299 - Fall 2005 Graphics, Genres, and Design CMPUT 299 - Fall 2005 Graphics, Genres, and Design

Moving Things Around

Background

0 0 0 0 0 0 0 0 0 0 0 0 0 100100 0 0 100100100 0 0 100100 0 50 50 0 0 50 50 50 0 0 50 50 0 10 10 0 0 10 10 10 0 0 10 10

CMPUT 299 - Fall 2005 Graphics, Genres, and Design CMPUT 299 - Fall 2005 Graphics, Genres, and Design Background Background

Sprite

CMPUT 299 - Fall 2005 Graphics, Genres, and Design CMPUT 299 - Fall 2005 Graphics, Genres, and Design

Sprites

! Special purpose hardware for moving things Background around. ! A sprite is a small image that can be quickly drawn against a background. It has its own small Sprite image buffer. ! Includes transparency. ! Hardware sprites somewhere before 1980 Sprite ! Pacman (Namco, 1980) – eight hardware sprites ! 8-bit personal computers (Atari 400/800, Commodore 64) Sprite

CMPUT 299 - Fall 2005 Graphics, Genres, and Design CMPUT 299 - Fall 2005 Graphics, Genres, and Design Collisions Collisions

Background Background

CMPUT 299 - Fall 2005 Graphics, Genres, and Design CMPUT 299 - Fall 2005 Graphics, Genres, and Design

Collisions Collisions and Sprites

Background ! Collisions a huge part of determining action in a game. collision ! Have to check whether moving one graphic causes it to intersect with another. ! Tedious and slow checking of numbers. ! Sprites to the rescue again. ! Hardware support for collision-detection.

CMPUT 299 - Fall 2005 Graphics, Genres, and Design CMPUT 299 - Fall 2005 Graphics, Genres, and Design Parallax Scrolling Layers

! Sprites change independent of background ! Now we can change the background at will without worrying about sprites ! Good for motion effects, commonly called scrolling ! Can give even better effects with parallax scrolling ! Moon Patrol (Irem, 1982)

CMPUT 299 - Fall 2005 Graphics, Genres, and Design CMPUT 299 - Fall 2005 Graphics, Genres, and Design

Layers Layers

CMPUT 299 - Fall 2005 Graphics, Genres, and Design CMPUT 299 - Fall 2005 Graphics, Genres, and Design Hardware Support for Scrolling Last Generation Sprite Hardware

! Again, special purpose hardware allows ! Playstation multiple backgrounds, all scrolling at ! 4,000 8x8 sprites at 60 fps different speeds. ! scaling, rotation, transparency, and more ! So special 2-D graphics hardware allows ! Saturn ! sprites ! around 8,000 sprites (flat-shaded polygons) ! multiple, layered scrolling backgrounds ! collision-detection Sega fought the wrong battle.

CMPUT 299 - Fall 2005 Graphics, Genres, and Design CMPUT 299 - Fall 2005 Graphics, Genres, and Design

So what?

! Early move from custom built machines to ! Have you seen this man? microprocessors. ! Later move to custom hardware for sprites and layered backgrounds. ! A move toward more general-purpose equipment to allow innovation… ! … and a move to specialized equipment to improve performance.

CMPUT 299 - Fall 2005 Graphics, Genres, and Design CMPUT 299 - Fall 2005 Graphics, Genres, and Design The Rise of Teams Specialization

! Better hardware allowed more ambitious ! Larger games ! Larger teams (4 to 8) games. ! Stable companies producing multiple games ! A competitive, fast-moving market ! Specialization begins increased the number of programmers ! someone does graphics ! someone does sound ! Split the game up into two or three pieces, ! someone does music each programmed (somewhat) separately ! someone does gameplay ! Still required a broad cross-section of skills ! someone does levels ! albeit not necessarily deep skills in many areas ! etc…

CMPUT 299 - Fall 2005 Graphics, Genres, and Design CMPUT 299 - Fall 2005 Graphics, Genres, and Design

Contemporary Development The Management

! Multiple teams totaling around 100 people ! Large projects require stability ! Communications becomes a real problem ! money and management ! Some teams rarely (if ever) interact ! Small publishers giving way to larger ! Others must interact all the time publishers with more stable management ! More like a movie production ! Still a lot of flux ! teams often put together for a single project It’s all about teams

CMPUT 299 - Fall 2005 Graphics, Genres, and Design CMPUT 299 - Fall 2005 Graphics, Genres, and Design The Game Designer Specialization " Generality

! Someone has to coordinate all of these ! Industry has gone through many phases and cycles activities to ensure a coherent product. ! workforce ! small teams of generalists, specialists, designers as generalists ! Need to understand something of ! special hardware vs. general purpose everything. ! graphics, sound, soon physics ! custom software vs. reusable software ! The game designer is a generalist. ! reusablility trend recently, partly necessary because games are becoming very expensive (> $50 million for some AAA titles) ! licensed game engines (Quake, Unreal, Lithtech) ! Havok physics engine ! Middleware: Renderware ! game design too (genre refinement & cross-genre) CMPUT 299 - Fall 2005 Graphics, Genres, and Design CMPUT 299 - Fall 2005 Graphics, Genres, and Design

Content over Container Revolution vs. Evolution

! Early games didn’t separate content from the rest ! hardware sprites vs. general purpose graphics of the game ! many more objects on screen managed by custom hardware ! graphics/sound drawn/played directly by the program ! personal computers vs. arcade ! Natural separation into game engines and the data ! opened up development to “amateurs” for different parts of the game ! larger storage and recorded music/graphics ! level data, graphics textures, music, sound effects, ! laser disc (Dragon’s Lair), hard disc (Killer Instinct), and CD character stats, etc. (everything now) ! Content is now the largest part of games and the ! 2-D ! 3-D most expensive part. ! Alone in the Dark, Ultima Underworld, Wolfenstein 3-D, Super Mario 64 ! Where from? Artists, but increasingly, the real world. CMPUT 299 - Fall 2005 Graphics, Genres, and Design CMPUT 299 - Fall 2005 Graphics, Genres, and Design 3-D Graphics Types of Rendering ! Complex subject ! Non Real-Time (e.g., raytracing) ! Screens are 2-D ! compute paths of beams of light in world and figure out their colour when they hit the camera’s field of view ! Produce a 2-D image of a 3-D world ! slow (a single frame may take minutes or hours) but ! Build our 3-D world high quality ! Point a virtual camera at it ! used in movies like Pixar’s Toy Story, etc. ! How do we describe the objects in the world? ! sometimes used to pre-render scenes in games ! What does our camera do? ! these images can be used as fixed background of high- quality ! Process of producing a 2-D image is called rendering. ! Real-Time ! draw image in real-time (quickly enough to animate) ! Often call the part of the world we’re rendering a scene. ! lower quality but world can change with player actions CMPUT 299 - Fall 2005 Graphics, Genres, and Design CMPUT 299 - Fall 2005 Graphics, Genres, and Design

The Camera The Camera: Frustum ! A camera is a view into the world ! Volume visible to camera: frustum ! It occupies a position in 3-D space (x,y,z) ! It points in a direction ! It has viewing angles that determine how much it can see right-left and up-down ! May have a maximum depth of view (max distance) ! We can have special effects related to the camera ! lens flares, fish eye ! hand-held, out-of-focus, motion blur st rd ! 1 person vs. 3 person © George Otto, Penn State University CMPUT 299 - Fall 2005 Graphics, Genres, and Design CMPUT 299 - Fall 2005 Graphics, Genres, and Design The Camera: Styles The Camera: Styles

! Fixed cameras ! Free cameras ! chase camera (3rd person, follows behind) ! finite position and direction (Myst, 7th Guest) ! chase with right/left (3rd person, follows but allows ! fixed position and direction (Resident Evil) right/left glancing) ! fixed position (Myst 3) ! free-look (1st person, looks wherever you point) ! look-spring (1st or 3rd camera can be moved but ! fixed direction (some RTS, isometric view) automatically returns to its default when released) ! fixed path (rail-shooters like Virtua Cop) ! rear-view (look behind you) ! fixed path with free direction (rail-shooters like ! arbitrary view (e.g. side view) Panzer Dragoon) ! droppable (move and then leave it there) ! switching from 1st to 3rd or vice versa

CMPUT 299 - Fall 2005 Graphics, Genres, and Design CMPUT 299 - Fall 2005 Graphics, Genres, and Design

The Camera: Friend or Foe? Simple “3-D” Tricks

! Obstructions in 3rd person view ! Scaling ! Where does the chase camera go when you back ! making 2-D sprites (or bitmaps) bigger or into a wall? smaller to simulate moving toward or away ! Camera won’t allow angle you want ! Isometric view ! Camera too slow ! 2-D artistic style showing perspective ! Camera too erratic or has jerky transitions ! But we want to have a world with real 3-D ! Camera too automatic or too manual objects (i.e., they occupy some volume of ! Camera really just wants to kill you (any 3-D space) Sonic game) ! Can follow crazy trajectories, though! CMPUT 299 - Fall 2005 Graphics, Genres, and Design CMPUT 299 - Fall 2005 Graphics, Genres, and Design 3-D Objects Polygons

! Need to represent objects in the world ! Polygons are 2-D objects described by three ! Don’t necessarily care what they look like or more points connected by line segments. inside ! They are closed (all line segments ! Let’s make “hollow” objects connected) and line segments do not cross ! Defined by surfaces each other. ! We can build surfaces out of polygons ! A point used to describe a polygon is called a vertex (plural: vertices).

CMPUT 299 - Fall 2005 Graphics, Genres, and Design CMPUT 299 - Fall 2005 Graphics, Genres, and Design

Example of Polygons Examples of Non-Polygons

Not closed

Not line segments

Line segments cross

CMPUT 299 - Fall 2005 Graphics, Genres, and Design CMPUT 299 - Fall 2005 Graphics, Genres, and Design Triangles Coordinates

! Simplest polygon ! Two dimensional points can be described using 2- D coordinates. ! Nice mathematical properties ! Each coordinate is two numbers (x and y), ! Can divide up more complex polygons into referring to horizontal and vertical position. triangles (tessellation) ! For our surfaces, we must describe points in 3-D space using 3-D coordinates (x, y, z) referring to horizontal, vertical, and depth. ! Our polygons (triangles) will be described by 3-D coordinates.

CMPUT 299 - Fall 2005 Graphics, Genres, and Design CMPUT 299 - Fall 2005 Graphics, Genres, and Design

Lots of Triangles Meshes

! A collection of triangle for a single “object” is called a mesh. ! In the end, a mesh is just a big list of vertex coordinates

CMPUT 299 - Fall 2005 Graphics, Genres, and Design CMPUT 299 - Fall 2005 Graphics, Genres, and Design Multiple Meshes 2-D Object in 3-D Space

! Figures can be composed of multiple (0,5,0) (5,5,0) (2,2,3) (3,2,3) meshes. The pieces (meshes) can separate.

(0,0,0) (5,0,0) (0,0,0) (5,0,0)

(These numbers are a filthy lie)

CMPUT 299 - Fall 2005 Graphics, Genres, and Design CMPUT 299 - Fall 2005 Graphics, Genres, and Design

Normal Vectors Last time…

! The idea was to make “hollow” figures from ! Finished 2-D graphics surfaces built out of polygons. ! Specialization and Generalization ! Only care about the “outside” of the figure. ! Triangles (often) have only one visible face ! Started 3-D graphics ! Look at it from one face, see the triangle ! Types of rendering ! Flip it over and it’s transparent ! Cameras ! Conceptually, there is an arrow called the normal ! Polygons, Triangles, Vertices, and Coordinates vector pointing straight out of one face to indicate that it’s the visible face. ! Meshes, Normal vectors

CMPUT 299 - Fall 2005 Graphics, Genres, and Design CMPUT 299 - Fall 2005 Graphics, Genres, and Design This time… How to Draw a Scene ! World is built out of many, many triangles ! How to render 3-D scenes ! Place the camera in the world ! Models ! Determine what triangles can seen. ! Culling ! We can calculate what a triangle looks like from the ! Volume Partitioning camera’s position ! From a particular viewpoint, some triangles obscure ! Advances in 3-D others. ! Shading and Texturing ! One idea: ! Special Effects ! sort triangles by how far away they are ! Genres as time permits ! draw them from furthest to closest ! closer triangles will be drawn over top of further triangles CMPUT 299 - Fall 2005 Graphics, Genres, and Design CMPUT 299 - Fall 2005 Graphics, Genres, and Design

Problem Culling ! In a large world, way too many triangles ! Need to reduce the number of triangles we need to ! Limited in the number of triangles we can draw draw per frame ! Culling is the process of eliminating triangles ! Essentially the same problem we had in vector from the list of those we need to draw. graphics (and 2-D raster too). ! Can obviously ignore ! Always need to consider scene complexity ! triangles behind the camera ! many factors to complexity ! triangles out of the forward field of view up-down and ! number of potentially visible triangles right-left ! effects ! triangles beyond the camera’s range ! How can we have complex worlds but still draw ! all this is called frustrum culling them? ! only consider triangles inside the camera’s frustrum CMPUT 299 - Fall 2005 Graphics, Genres, and Design CMPUT 299 - Fall 2005 Graphics, Genres, and Design The Camera: Frustum Clipping ! Volume visible to camera: frustum ! Only considering triangles inside the frustum now ! Triangle may only be partially inside ! Cut off part of triangle that is outside ! This process is called clipping

Still have problems… Philosophical Question

! With complex worlds, even computing what triangles are inside the frustum may be too slow! ! Camera’s max depth may cover most of the world ! Ever wonder why you can’t just blow it all ! So, even if we can do frustum culling, we may have way too many triangles to draw up? ! Need a fast way of deciding what triangles might be visible before we cull ! Let’s break the world up into manageable spaces

CMPUT 299 - Fall 2005 Graphics, Genres, and Design CMPUT 299 - Fall 2005 Graphics, Genres, and Design Static vs. Dynamic Objects Volume Partitioning ! Suppose an object in the world is static (doesn’t move) (e.g., a wall, a non-killing-you statue) ! One way to exploit static environments is to ! We can precompute (calculate ahead of time) and divide up (partition) the world into separate store information about the visibility of its spaces (volumes). triangles (do this when developing the game) ! Rooms are a natural way to do this. ! We can use the information to speed up rendering ! We can manually specify the division when we ! If an object moves (is dynamic), then we can’t design the levels always do this ! There are more automatic methods, but the ideas ! If most of the world is static, life will be good are easiest to explain in a portal system. ! Sadly, this means we can’t change much of the world CMPUT 299 - Fall 2005 Graphics, Genres, and Design CMPUT 299 - Fall 2005 Graphics, Genres, and Design

Portals Portals

! Divide world into regions connected by portals. ! Camera is limited to triangles in the same A A B C B region as it. C ! Can extend this by allowing view through D portals into other regions. D ! Extend further by allowing portals to be open or closed during the game.

CMPUT 299 - Fall 2005 Graphics, Genres, and Design CMPUT 299 - Fall 2005 Graphics, Genres, and Design Automated Volume Partitioning

! Automated methods split the world’s volume up into pieces using geometry. ! Examples include ! binary space partitioning ! quadtrees ! octrees ! Many games use a mixture of manual and automatic methods

CMPUT 299 - Fall 2005 Graphics, Genres, and Design CMPUT 299 - Fall 2005 Graphics, Genres, and Design

Early 3-D Early 3-D: 2.5D or “Raycasting” ! Early games allowed only one “floor”, at most ! Many early 3-D games were made possible by allowing variations in the height of that floor. No volume partitioning rooms above rooms (except by trickery). ! 2-D graphics (sprites) in a 3-D environment ! Ultima Underworld allowed complex environment ! Ultima Underworld (Looking Glass, 1992) (angled walls, inclined surfaces) but was slow ! Wolfenstein 3D (id Software, 1992) ! Wolfenstein 3D made some simplifying ! Doom (id Software, 1993) assumptions (e.g. all-right angles, all floors the ! Heretic (Raven Software, 1994) same level) and was fast ! ! Marathon (Bungie, 1994) Doom relaxed restrictions (angled walls (right angled to floor though), variable height floors (no ! System Shock (Looking Glass, 1994) slopes)) while keeping the speed ! Duke Nukem 3D (3D Realms, 1996) ! Heretic allowed angling viewpoint up and down CMPUT 299 - Fall 2005 Graphics, Genres, and Design CMPUT 299 - Fall 2005 Graphics, Genres, and Design Beyond 2.5… Models ! Not everything is background… ! System Shock (Looking Glass, 1994) broke 2.5-D ! And sprites look weird in a 3-D world… and allowed floors above other floors and ! So we need some dynamic 3-D objects transparency (a bit slow) (this functionality might have been around in Ultima Underworld… 1992!) ! These are sometimes called models to distinguish them from static background ! But, the id games were ultimately more successful ! simpler ! Used for ! faster ! game creatures, power-ups, doors ! destructible parts of environment

CMPUT 299 - Fall 2005 Graphics, Genres, and Design CMPUT 299 - Fall 2005 Graphics, Genres, and Design

Models Later 3-D ! Made of meshes ! Games eventually introduced 3-D models in place ! Can be animated by changing the vertex of sprites coordinates in the mesh ! Descent (Parallax, 1995) ! Limited numbers of (visible) models because we ! Quake (id Software, 1996) can’t precompute ! Quake II (id Software, 1997) ! Need to keep track of where model is in volume ! GoldenEye 007 (Rare, 1997) partitioning ! Half-Life (Valve, 1998) ! This is why back in sprite-based 3-D games, there ! Thief (Looking Glass, 1998) were many, many enemies. In polygon enemy ! Unreal (Epic, 1998) games, we can’t do so many. ! System Shock 2 (Irrational Games/Looking Glass, 1999) CMPUT 299 - Fall 2005 Graphics, Genres, and Design CMPUT 299 - Fall 2005 Graphics, Genres, and Design 3-D Graphics Hardware 3-D Hardware

! Early 3-D games used software rendering (up to ! Much of 3-D is repetitive calculations around Quake II, 1997) ! Custom hardware can do this more efficiently than ! Computed entire process on the general-purpose the general-purpose CPU CPU – sprite hardware only semi-useful. ! Initially, vertex information computed by CPU ! 3-D hardware rendering was very expensive stuff and sent to GPU to draw with textures developed mainly by Silicon Graphics (SGI) ! More and more features added to GPUs throughout the late 80’s and 90’s. ! hardware transform and lighting ! Commercially viable for home gaming starting ! complex texturing operations with the Voodoo card from 3dfx in 1996 ! most recently, completely programmable “pixel ! Now main processor feeds graphics information to shaders” – more like a traditional CPU! the graphics processing unit (GPU) ! expect to see physics and collision detection soon! CMPUT 299 - Fall 2005 Graphics, Genres, and Design CMPUT 299 - Fall 2005 Graphics, Genres, and Design

Rendering Triangles Rendering: Wireframe

! Now we know what triangles to draw ! Just draw the lines: wireframe ! How do we draw them? ! A little math figures out what the triangle’s shape is like from the camera’s perspective (transform) ! What do we draw?

! Fill triangles with a single, solid colour ! Draw filled triangles shaded according to light ! filled ! flat-shading ! specify colours for triangles ! specify colour for each triangle ! specify light(s) with RGB values ! compute effect of light ! Problem: adjacent faces of on filled triangles same colour indistinguishable ! single (but different) colour for each triangle ! gives a proper 3-D impression, but crude Need to introduce the concept of lighting

Rendering: Smooth-Shading Rendering: Texturing ! Fill triangles with an image ! Shade each triangle according to lights ! texturing ! smooth-shading ! specify entire image for surface ! colour varies over triangle (i.e. a colour for every pixel) ! can shade textured triangles ! smoother transitions between according to light as well adjacent triangles ! often call pixels in a texture image texels ! texture mapping: stretch image over an entire mesh, instead of just a single triangle © www.realityfactory.ca © www.realityfactory.ca CMPUT 299 - Fall 2005 Graphics, Genres, and Design CMPUT 299 - Fall 2005 Graphics, Genres, and Design Texturing Issues Texture Resolution ! Texturing adds a huge amount of realism ! An image has a resolution ! Can add surface detail without adding triangles to ! If you get very close to a texture, it may look the mesh “blocky” because you can see individual texels ! Textures can even be animated (change image ! Far away objects don’t need much detail over time) ! Standard solution: mipmapping ! However, requires much more storage than ! copies of same texture at multiple resolutions shaded triangles ! switch copies as camera moves further/closer ! A huge part of modern gaming technology is ! Switching resolutions can cause odd problems concerned with allowing more textures, with ! partially solved by filtering: bilinear, trilinear, higher resolution, and moving texture information anisotropic around efficiently

CMPUT 299 - Fall 2005 Graphics, Genres, and Design CMPUT 299 - Fall 2005 Graphics, Genres, and Design

Level of Detail Transparency ! Nice to have transparent/translucent surfaces ! mipmapping is an example of a level of detail (or distance) (LOD) approach ! Can colour the light that passes through them ! basic idea is that far away things need less detail ! Do this by adding an extra value to every pixel (texel) ! we can save computation by changing what we ! red, green, blue, and alpha draw when depending on distances ! alpha specifies how transparent the pixel is ! other LOD approaches, include ! When drawing the scene, transparent pixels ! changing meshes at different distances (fewer triangles modify the value of stuff behind instead of when far away) (e.g. “Messiah” by Shiny, 2000) overwriting it ! dropping lights, shadows, and any other special features when stuff is far away ! Keep in mind that transparency means more visible triangles! CMPUT 299 - Fall 2005 Graphics, Genres, and Design CMPUT 299 - Fall 2005 Graphics, Genres, and Design Materials Lights

! Seem to be collecting a lot of information to ! A very complicated subject describe triangles ! Basic properties ! Information about drawing triangles is often called ! intensity (brightness) material information ! colour ! colour/texture ! transparency (alpha) ! A few kinds of lights ! reflectivity ! ambient light ! special effects (e.g. bump mapping info) ! artificial concept of light that is “everywhere” in the ! precomputed effects (e.g. light maps, shadow maps) scene – all surfaces are lit by it equally ! useful so that everything is visible to some minimum ! Games are using more and more detailed material extent information – it all costs storage!

CMPUT 299 - Fall 2005 Graphics, Genres, and Design CMPUT 299 - Fall 2005 Graphics, Genres, and Design

Lights Diffuse vs. Specular Lighting

! point-source lights ! diffuse lighting refers to surfaces absorbing and ! placed in the world (like the camera) then emitting light uniformly over the surface ! position ! angle of incoming light matters ! shine in all directions (affects quantity of emitted light) ! directional lights ! but viewer’s angle does not ! placed in the world (like the camera) ! specular lighting refers to light “bouncing” off the ! position and direction surface ! shine in a cone ! light comes in at an angle and leaves at an angle ! dynamic vs. static ! viewer’s angle matters ! responsible for “shiny spots” ! static lights stay where they are placed or “glare” on objects ! dynamic lights move CMPUT 299 - Fall 2005 Graphics, Genres, and Design CMPUT 299 - Fall 2005 Graphics, Genres, and Design Specular Highlights Reflectivity

! The reflectivity of a material determines how much light is absorbs vs. reflects. ! Can also talk about actual reflections: parts of the scene reflected in shiny or mirror-like surfaces ! Expensive to do properly, so faked one way or another ! render scene from another camera and paste the image onto the “mirror” surface (also do video cameras, etc. this way) – good but slow, have to do a whole separate render! ! environment maps: precompute images for surroundings and blend with reflective surface – fast but inaccurate and no dynamic stuff

CMPUT 299 - Fall 2005 Graphics, Genres, and Design CMPUT 299 - Fall 2005 Graphics, Genres, and Design

Enviroment Mapping: Cube Map Enviroment Mapping: Cube Map

© Stephen Chenney, U. Wisconsin CMPUT 299 - Fall 2005 Graphics, Genres, and Design © www.developer.com CMPUT 299 - Fall 2005 Graphics, Genres, and Design Lighting: Finer than triangles Precomputing Lighting

! So far, lighting behaviour specified for each ! If lights are static, we can precompute their triangle effect on a surface and store it ! Contemporary lighting doesn’t stop at triangles ! The stored information is called a lightmap ! lighting can be per vertex ! Combined with the texture of a surface ! lighting behaviour specified for each vertex in the mesh and then “blended” over the triangles during rendering to change its look ! lighting can be per pixel ! each pixel on the surface specifies its own lighting behaviour

CMPUT 299 - Fall 2005 Graphics, Genres, and Design CMPUT 299 - Fall 2005 Graphics, Genres, and Design

Lightmaps Lightmaps

! Textures often reused and they are large Texture Lightmap Rendered surface ! Lightmaps are small and can be used to make textures look different in different places ! Can animate lightmaps (multiple lightmaps for a + = surface) – flickering lights! ! Can even generate lightmaps dynamically for dynamic lighting – slow but still good for saving storage

! A per pixel lighting effect ! Each pixel specifies its own surface normal ! Light is bounced off each pixel relative its individual surface normal ! Great for adding roughness to surfaces

CMPUT 299 - Fall 2005 Graphics, Genres, and Design CMPUT 299 - Fall 2005 Graphics, Genres, and Design

CMPUT 299 - Fall 2005 Graphics, Genres, and Design © www.earthenrecords.com CMPUT 299 - Fall 2005 Graphics, Genres, and Design © www.earthenrecords.com Bump Mapping: Angle Sensitive Pixel Shaders

! General-purpose hardware in GPUs that can run a small program to decide what every pixel looks like! ! Used to create a huge range of effects. ! Much of graphics today is coming up with novel shader techiques ! Graphics hardware is increasing the flexibility of these shaders with each generation

CMPUT 299 - Fall 2005 Graphics, Genres, and Design © www.earthenrecords.com CMPUT 299 - Fall 2005 Graphics, Genres, and Design

Particle Effects and Procedural Shadows Animation ! Received a lot of attention lately ! Effects like smoke, fire, fountains are often ! Simplest: draw a dark oval under the object achieved with particles: many, many tiny dots ! More complex shadows depend on both models moved around in patterns and lights ! Often, particle effects are a kind of procedural ! Dynamic shadows, from moving objects and animation. lights, obviously more costly ! Most animation is pre-recorded and just played ! Shadows can be hard (sharp edges/uniformly back. Comes from animators and/or motion dark) or soft (fuzzy edges) capture data. ! Very complicated subject with huge artistic value ! Procedural animation uses a program to decide how objects move. Pretty good for natural phenomena that can be described by simple laws. CMPUT 299 - Fall 2005 Graphics, Genres, and Design CMPUT 299 - Fall 2005 Graphics, Genres, and Design Non-Photorealism 3-D Collisions

! Most graphics concerned with increasing realism ! Need to decide when two models (or meshes) ! Some efforts towards new artistic looks collide. ! A popular example: cel-shading ! Exact collisions expensive to compute. ! named for the acetate cels used in hand-drawn ! Approximations: animation ! bounding box: find smallest box that contains each ! renders a 3-D scene to look “flatter” and more like model and check whether boxes intersect hand-drawn ! bounding sphere: same idea but with spheres instead of ! reduces colours and outlines in black boxes ! Fear Effect (Kronos, 2000), Jet Set Radio (Sega, 2000), ! Errors mean bits of models moving through each Legend of Zelda: Wind Waker (Nintendo, 2003), other or not actually touching when they stop. XIII (Ubisoft, 2003)

CMPUT 299 - Fall 2005 Graphics, Genres, and Design CMPUT 299 - Fall 2005 Graphics, Genres, and Design

Stencils

! Arbitrary 2-D cutouts ! Very efficiently implemented in hardware

CMPUT 299 - Fall 2005 Graphics, Genres, and Design