sign in sign up
<<
Home , 3D rendering, PowerVR

and Imaging Architectures

Kayvon Fatahalian http://www.cs.cmu.edu/afs/cs/academic/class/15869-f11/www/ About Kayvon ▪ New faculty, just arrived from Stanford

▪ Dissertation: Evolving real-time for micropolygon rendering (micropolygon meshes = high-resolution meshes used in o!-line rendering for "lm)

▪ In addition to rendering: involved in many research projects related to computing on GPUs - Brook (precursor to CUDA/OpenCL) - Sequoia (locality-aware parallel programming language) - GRAMPS (stream computing for heterogeneous HW)

Kayvon Fatahalian CMU 15-869, Fall 2011 COMPUTING IS BECOMING HIGHLY VISUAL!

Visually rich user interfaces (that require 3D graphics!)

Innovative new input modalities (touch, gyro, cameras)

Ubiquitous, high-resolution cameras (emergence of computational photography)

Games, entertainment (continuous push towards higher visual realism)

Kayvon Fatahalian CMU 15-869, Fall 2011 COMPUTING EFFICIENTLY IS INCREASINGLY CRITICAL!

Ubiquitous parallelism

Heterogeneous parallelism Core i7 (Nahalem)

Power constraints

Mobile, mobile, mobile

Real-time 3D graphics has always found a way to consume all available compute: GPUs are e#cient parallel, heterogeneous systems (that are relatively easy to program) Kayvon Fatahalian CMU 15-869, Fall 2011 My Macbook Pro 2011 (two GPUs) AMD HD GPU

Quad-core Core i7 CPU (Contains integrated Intel GPU)

From i"xit.com teardown Kayvon Fatahalian CMU 15-869, Fall 2011 iPad 2 main board SoC

Dual-core ARM CPU

Video Image processing DSP GPU Encode/ Decode

Flash memory

From i"xit.com teardown Kayvon Fatahalian CMU 15-869, Fall 2011 Touchscreen controller (integrated DSP) ~ $1

From US Patent Application 2006/0097991 Kayvon Fatahalian CMU 15-869, Fall 2011 My Nikon D7000 camera

16.2 MP sensor 14 bits per 6 fps burst

Kayvon Fatahalian CMU 15-869, Fall 2011 What this course is This is a course about how real-time graphics systems work

GRAPHICS mapping/scheduling MACHINE ALGORITHMS ORGANIZATION (the workload)

Parallelism rasterization Locality Communication anti-aliasing The design of throughput processing cores The role of "xed-function HW

Kayvon Fatahalian CMU 15-869, Fall 2011 What this course is This is a course about how real-time graphics systems work

GRAPHICS mapping/scheduling MACHINE ALGORITHMS ORGANIZATION (the workload)

ABSTRACTIONS (the real-time graphics pipeline)

choice of primitives level of abstraction

Kayvon Fatahalian CMU 15-869, Fall 2011 What this course is not

▪ This is not an [OpenGL, CUDA, OpenCL] programming course - But we will be discussing the design of these abstractions (and their future) in great detail!

Kayvon Fatahalian CMU 15-869, Fall 2011 Logistics 1: schedule ▪ Expect it to be $uid - it’s my "rst time doing this - students in this class have wide variation in background - http://www.cs.cmu.edu/afs/cs/academic/class/15869-f11/www/ ▪ First half of class: focused on real-time 3D graphics - 3D graphics abstractions and the implementation of the real-time 3D graphics pipeline (e.g., OpenGL/) on modern GPUs

▪ Second half of class: broader sampling of applications and systems - future 3D graphics pipelines, GPU programming model issues (OpenCL/ CUDA), hybrid CPU/GPUs, image processing architectures, cameras

▪ Several external speakers (AMD, Intel, )

Kayvon Fatahalian CMU 15-869, Fall 2011 Logistics 2: expectations This is a reading/project course

▪ (40%) Paper readings - Weekly: you will be asked to submit brief reviews of selected papers (or answer a speci"c questions) - Participate in class discussions

▪ (10%) Two quick (2-3 day) C++ programming assignments - Meant to concretize concepts, not stress you out

▪ (50%) Semester-long, self-selected research project (teams of 1-2) - Aim for publishable results

Kayvon Fatahalian CMU 15-869, Fall 2011

Image credit: Henrik Wann Jensen Model of a scene: Image 3D surface geometry (e.g., triangle mesh) surface materials lights camera How does each triangle contribute to each pixel in the image?

Kayvon Fatahalian CMU 15-869, Fall 2011 What is an architecture? (white board)

Kayvon Fatahalian CMU 15-869, Fall 2011 Real-time graphics pipeline (entities)

3 1 4

2 Vertices Primitives (triangles, points, lines)

Fragments

Kayvon Fatahalian CMU 15-869, Fall 2011 Real-time graphics pipeline (operations)

3 1 Vertex Generation 4 Vertices in 3D space 2 Vertices Vertex stream Vertex Processing Vertex stream Vertices in positioned on screen Primitive Generation Primitive stream Primitives Primitive Processing Primitive stream Triangles positioned on screen Fragment Generation (Rasterization) Fragment stream Fragments Fragments (one per each covered pixel) Fragment Processing Fragment stream Shaded fragments

Pixels Pixel Operations

Output image (pixels)

Kayvon Fatahalian CMU 15-869, Fall 2011 Real-time graphics pipeline (state) Memory Bu!ers (system state) 3 1 Vertex Generation 4 Vertex Data bu!ers 2 Vertices Vertex stream Vertex Processing Texture bu!ers Vertex stream Vertex transform matrices Primitive Generation Primitive stream Primitives Primitive Processing Texture bu!ers Primitive stream Fragment Generation (Rasterization) Fragment stream Fragments Fragment Processing Texture bu!ers Fragment stream

Pixels Pixel Operations

Output image bu!er

Kayvon Fatahalian CMU 15-869, Fall 2011 History: Evolution of interactive 3D graphics

Following slides borrowed from Kurt Akeley and Pat Hanrahan http://graphics.stanford.edu/wikis/cs448-07-spring

Kayvon Fatahalian CMU 15-869, Fall 2011 Slide credit Kurt Akeley and Pat Hanrahan Kayvon Fatahalian CMU 15-869, Fall 2011 Early years ▪ Evans and Sutherland built machines for $ight simulators throughout the 70s ▪ Early 80s: Custom ASIC for geometry processing - The Geometry Engine: a VLSI Geometry System for Graphics (Jim Clark, SIGGRAPH 82) - Work started at Stanford (led to Silicon Graphics, 1982)

Kayvon Fatahalian CMU 15-869, Fall 2011 Slide credit Kurt Akeley and Pat Hanrahan Kayvon Fatahalian CMU 15-869, Fall 2011 Slide credit Kurt Akeley and Pat Hanrahan Kayvon Fatahalian CMU 15-869, Fall 2011 Slide credit Kurt Akeley and Pat Hanrahan Kayvon Fatahalian CMU 15-869, Fall 2011 Slide credit Kurt Akeley and Pat Hanrahan Kayvon Fatahalian CMU 15-869, Fall 2011 Slide credit Kurt Akeley and Pat Hanrahan Kayvon Fatahalian CMU 15-869, Fall 2011 O!-line rendering for "lm

Image credit: Pixar, Toy Story 1 , 1995 ▪ O!-line graphics advancing as well - Pixar created in 1986 - Initially thought they were going to build hardware (Pixar image ) ▪ Reyes graphics system at Pixar (Renderman) - Feature rich: smooth/complex surfaces, programmable , anti-aliasing, motion blur - Implemented as a software application on CPUs - Hours/frame

Kayvon Fatahalian CMU 15-869, Fall 2011 Early PC graphics

Wolfenstein 3D, 1992 Doom, 1993

Software applications running on CPU Extreme amount of software optimization, algorithm development (and hacks)

Kayvon Fatahalian CMU 15-869, Fall 2011 PC 3D graphics ▪ 3D graphics acceleration for PC motivated by games ▪ add-in boards made by 3DFX, NVIDIA, ATI, PowerVR, and more - 3DFX founders were from SGI - 3DFX Voodoo (1997) ▪ standardized around accelerating OpenGL/Direct3D - o%oaded rasterization, texture mapping, frame-bu!er processing - initially only subset of GL implemented by Quake

Quake 1 Kayvon Fatahalian CMU 15-869, Fall 2011 Slide credit Kurt Akeley and Pat Hanrahan Kayvon Fatahalian CMU 15-869, Fall 2011 Fourth generation ▪ Programmable shading

Kayvon Fatahalian CMU 15-869, Fall 2011 Current generation ▪ Tessellation (smooth/complex surfaces)

Image credit: NVIDIA/Unigine

Kayvon Fatahalian CMU 15-869, Fall 2011 Current generation ▪ Some GPU computations now driven by alternative programming interface (not 3D graphics pipeline) - to augment 3D rendering (game physics, global lighting) - for non-graphics applications (scienti"c computing)

Image credit: NVIDIA Image credit: Folding @ Home

Kayvon Fatahalian CMU 15-869, Fall 2011 Today: high versatility, high peak compute

Intel Core i7 (quad-core CPU) AMD Radeon HD 5870 GPU ~100 GFLOPS ~2.7 TFLOPS 730 million transistors 2.2 billion transistors More compute power than Pixar’s entire render farm for Toy Story 1!

Kayvon Fatahalian CMU 15-869, Fall 2011 Readings ▪ D. Blythe, The Rise of the Graphics Processor Proceedings of the IEEE, 2008

▪ Real-Time Rendering, Ch 2. The Graphics Rendering Pipeline (handout)

Kayvon Fatahalian CMU 15-869, Fall 2011