IMGD 4000: Hot Topics in Graphics

Prof Emmanuel Agu

Computer Science Dept. Worcester Polytechnic Institute (WPI) Graphics Processing Unit (GPU)

 Entire graphics library (OpenGL, DirectX) in hardware => FAST!!  Speed: GPUs renders graphics faster CPUs  Programmable: in last 8 years (Shaders)  Located either on PC motherboard (Intel) or Separate graphics card (Nvidia or ATI)

On PC motherboard On separate PCI express card GPU Evolution

 High throughput computation  GeForce GTX 280: 933 GFLOP/s  High bandwidth memory  GeForce GTX 280: 140 GB/s  High availability to all “Fermi”  180+ million shader‐capable GPUs in the wild 3B xtors  Numbers doubling every 6 months (Moore’s law cubed) GeForce 8800 681M xtors GeForce FX 125M xtors GeForce 3 GeForce® 256 60M xtors RIVA 128 23M xtors 3M xtors

1995 2000 2005 2010 Slide from Stanford course 193G 2010 DesignTradeoff makes GPU Fast

 Graphics has many calculations, few conditional (if/else, loops)  Arithmetic Logic Unit (ALU) is part of GPU for calculations  Tradeoff: Assign  More transistors for ALU: faster rendering calculations, math  Fewer transistors for control, branches, loops (graphics rarely uses them)  ALU chip area: CPU (7%) vs GPU ( > 50%)  Result: GPU runs graphics much faster than CPU GFLOP/sec growth

GPU!!

CPU!! GPU Memory Bandwidth

GPU!!

CPU!!

Graph from: CUDA Programming Guide 4.2 Aside: GPGPU: GPUs to speedup Non‐Graphics Applications

146X 36X 18X 50X 100X

Medical Molecular Video Matlab Astrophysics Imaging Dynamics Transcoding Computing RIKEN U of Utah U of Illinois, Elemental Tech AccelerEyes Urbana

149X 47X 20X 130X 30X

Financial Linear Algebra 3D Quantum Gene simulation Universidad Ultrasound Chemistry Sequencing Oxford Jaime Techniscan U of Illinois, U of Maryland Urbana Trend 1: Mobile GPUs  GPUs on Phones, tablets now powerful  Mobile GPUs support OpenGL ES  OpenGL ES: Subset of OpenGL, shader‐based Trend 1: Mobile GPUs

 Many game titles now ship only smartphones only, no consoles Unity: Cross Platform Shader Support

 Artist specifies high‐level shader on the Material  ex: "Bumped Specular", "Tree Leaves", "Unlit“  Run‐time picks specific platform shader depending on supported feature set Testing Mobile Apps: Challenging

 Animoca, android mobile developer has to test each app on over 400 different smartphones and tablets  Screens  Aspect ratios  Form factors  Input controls  OS versions  CPU/GPU  OpenGL/DirectX versions……. etc Recap: What is Graphics?

 Trend 2: Capture light, materials, geometry, for rendering data from real world

4. Light/Material Interaction 1. Light (Lighting model)

2. Geometry Next: capturing light 3. Materials Capturing Light: Light Probes

 Photograph silver ball in environment  High dynamic range Using Light Probes Example Trend 2: Capture rendering data

4. Light/Material Interaction 1. Light (Lighting model)

2. Geometry 3. Materials Next: capturing geometry Trend 2: Capture Geometry

 Digitize real object geometry, billions of faces  High precision databases of models: shared by many  Better realism than packages such as maya  Important for sports games?

Mesh Trend 2: Capture rendering data

4. Light/Material Interaction 1. Light (Lighting model)

2. Geometry 3. Materials Next: capturing materials Measuring BRDFs: Gonioreflectometer

Murray‐Coleman and Smith Gonioreflectometer. ( Copied and Modified from [Ward92] ). Measured BRDF Samples

 BRDF: How materials reflect light  Examples: cloth, wood, velvet, etc  Mitsubishi Electric Research Lab (MERL) http://www.merl.com/brdf/  Wojciech Matusik  MIT PhD Thesis  100 Samples Capture Time‐Varying BRDFs

 Time varying?: surface reflectance changes over time  Examples: weathering, ripening fruits, rust, etc Exactly What Can We Capture?

1. Appearance (volume, scattering, transparency, translucency, etc)

2. Geometry

3. Reflectance & Illumination 4. Motion Trend 3: Real‐Time LoD Geometry Management tesselation

Far = Less detailed Near = More detailed mesh mesh

Simplification

 Previously: Pre‐generate mesh versions offline  Geometry shader unit added to GPU in DirectX 10 (2007)  generate new vertices, primitives from mesh  Mesh simplification/tesselation on GPU = Real time LoD  Tesselation: Demo Textures in Games

 Most games loaded with textures  Computationally cheap way to look pretty Trend 5: New Directions in Texture Synthesis

 Texture = picture or algorithmically rendered  Internet (e.g. Flickr) has billions of pictures  Example‐Based Texture Synthesis: New images from samples  Input: small (photographic) textures  Output: produce arbitrarily large texture  Approach: discover pattern, extrapolate algorithm, math

Synthesis

Algorithm output

input Pixel‐based Algorithm

Random Oriented

Regular Semi-regular Texture Synthesis on Surfaces

Sample Multi‐Level Texture Synthesis

Sample:

Coarse Medium Fine 1,000 vertices 4,000 vertices 16,000 vertices Example‐Based Model Synthesis Paul Merrell, I3D 2007 Input: Polygon models Approach:  Partition models into pieces  Place pieces in new arrangement subject to constraints

Random Consistent Partitions Placement Placement Example‐Based Model Synthesis Example‐Based Model Synthesis

Result

Input Texture Replacement

 Lots of new work around algorithmically altering images (found on web?)  Goal: Alter, remove or replace part of a image algorithmically Artifact Removal Detail Hallucination

 Alcatraz Scene Completion using Millions of Photographs Seam Carving Local Illumination

 Consider 1 bounce only  OpenGL does this  Looks fake  E.g. Torque 3D game engine

 Global Illumination

 Global illumination: model interaction of light from all surfaces in scene (track multiple bounces) shadow

multiple reflection translucent surface Trend 5: New Real time Global Illumination Algos

 Dynamic scenes: Need real‐time GI, on‐the‐fly  Real Time Global Illumination: state‐of‐the art  Calculate GI equations on GPU at run‐time  Improves:  Shadows  Ambient Occlusion  Reflections  Transmittance  Refractions  Caustics  What does RT‐GI look like? Real‐time Lighting in Games Better Sky Model

 [BrunetonNeyret2008]  Single and multiple scattering  Light shafts  Participating media!! Different Atmospheres Time Of Day Better Shadows

Courtesy Hellgate:London, Variance shadow mapping flagship studios inc Courtesy Nvidia SDK 10 Better Caustics and Refraction

Courtesy Chris Wyman, Univ Iowa Sub‐Surface Scattering

Crysis skin demo

Marble

Human Skin Sub‐Surface Scattering Light Propagation Volumes (LPV)

 CryEngine: new physically‐based game engine by CryTek  LPV: new real time Global Illumination algorithm in CryEngine  Runs faster than ray tracing on GPU Crytek Crisis Engine Screenshots Demo

 Light Propagation Volumes Demo LPV Idea Main idea: represent light propagation as Virtual Point Lights (VPL) Re‐project VPL into adjacent cells References

 Akenine Moller et al, Real‐Time Rendering, 3rd edition  Advances in Real‐Time Rendering in 3D graphics and games, SIGGRAPH course notes 2009  Anton Kaplanyan and Carsten Dachbacher, Cascaded light propagation volumes for real‐time indirect illumination, in Proc. Si3D 2010  Hao Chen and Natalya Tatarchuk, Lighting Research at Bungie, Advances in Real‐Time Rendering in 3D Graphics and Games SIGGRAPH 2009 Course notes  Advances in Real‐Time Rendering in 3D Graphics and Games, SIGGRAPH 2010 Course Notes  CS 543/563 slides