GDCGDCGDC Tutorial:Tutorial:Tutorial: AdvancedAdvancedAdvanced OpenGLOpenGLOpenGL GameGameGame DevelopmentDevelopmentDevelopment A Practical and Robust Bump-mapping Technique for Today’s GPUs

MarchMarchMarch 8,8,8, 200020002000 Mark J. Kilgard Graphics Software Engineer NVIDIA Corporation “Hardware Bump Mapping” mostly hype so far

PreviousPreviousPrevious techniquestechniquestechniques •••ProneProne Prone tototo aliasingaliasingaliasing artifactsartifactsartifacts •••DoDo Do notnotnot correctlycorrectlycorrectly handlehandlehandle surfacesurfacesurface selfselfself-shadowing--shadowingshadowing •••CutCut Cut tootootoo manymanymany corners,corners,corners, i.e.i.e.i.e. failfailfail tototo renormalize,renormalize,renormalize, etc.etc.etc. •••TooToo Too expensive:expensive:expensive: manymanymany passespassespasses (OIM),(OIM),(OIM), needsneedsneeds accumulationaccumulationaccumulation bufferbufferbuffer ororor glCopyPixelsglCopyPixelsglCopyPixels •••OnlyOnly Only handlehandlehandle infinite,infinite,infinite, nonnonnon-attenuated,--attenuated,attenuated, nonnonnon-spotlight--spotlightspotlight lightslightslights •••AssumeAssume Assume locallocallocal viewerviewerviewer •••CorrectCorrect Correct onlyonlyonly forforfor essentiallyessentiallyessentially flatflatflat surfacessurfacessurfaces •••DifficultDifficult Difficult tototo implementimplementimplement andandand authorauthorauthor Demo! Overview of the Technique

UsesUsesUses newnewnew NVIDIANVIDIANVIDIA GPUGPUGPU featuresfeaturesfeatures •••CubeCube Cube mapsmapsmaps ---forfor for perperper-pixel--pixelpixel normalizationnormalizationnormalization –––EXT_texture_cube_mapEXT_texture_cube_map EXT_texture_cube_map extensionextensionextension •••RegisterRegister Register combinerscombinerscombiners ---forfor for perperper-pixel--pixelpixel dotdotdot productsproductsproducts andandand additionaladditionaladditional mathmathmath –––NV_register_combinersNV_register_combiners NV_register_combiners •••AndAnd And dualdualdual-textured--texturedtextured ---forfor for normalizationnormalizationnormalization cubecubecube mapmapmap andandand 2D2D2D normalnormalnormal mapmapmap –––ARB_multitextureARB_multitexture ARB_multitexture Features of the technique

CorrectlyCorrectlyCorrectly handleshandleshandles •••SurfaceSurface Surface selfselfself-shadowing--shadowingshadowing forforfor bothbothboth diffusediffusediffuse andandand specularspecularspecular contributionscontributionscontributions •••DiffuseDiffuse Diffuse minificationminificationminification sososo thatthatthat bumpedbumpedbumped objectsobjects inin thethe distancedistance looklooklook dimmerdimmerdimmer thanthanthan equivalentequivalentequivalent smoothsmoothsmooth distantdistantdistant objectsobjectsobjects •••MipmapMipmap Mipmap filteringfilteringfiltering ofofof normalnormalnormal mapsmapsmaps minimizesminimizesminimizes aliasingaliasingaliasing •••Local,Local, Local, attenuated,attenuated,attenuated, and/orand/orand/or spotspotspot-light--lightlight lightlightlight sourcessources withwithwith locallocallocal viewerviewerviewer •••AmbientAmbient Ambient andandand surfacesurfacesurface decalsdecalsdecals Anatomy of the technique

ThreeThreeThree passespassespasses (or(or(or justjustjust twotwotwo ififif nonono specular)specular)specular)

××× +++

diffusediffusediffuse decaldecaldecal specularspecularspecular High-level view

TangentTangentTangent spacespacespace perperper-pixel--pixelpixel lightinglightinglighting [Peercy[Peercy[Peercy 97]97]97] •••CPUCPU CPU suppliessuppliessupplies tangenttangenttangent spacespacespace lightlightlight vectorvectorvector ororor halfhalfhalf-angle--angleangle vectorvectorvector asasas (s,t,r)(s,t,r)(s,t,r) texturetexturetexture coordinatescoordinatescoordinates –––linearlylinearly linearly interpolatedinterpolatedinterpolated –––normalizednormalized normalized bybyby ananan RGB8RGB8RGB8 cubecubecube mapmapmap texturetexturetexture •••NormalNormal Normal mapmapmap isisis RGBA8RGBA8RGBA8 2D2D2D texturetexturetexture –––RGBRGB RGB containscontainscontains “compressed”“compressed”“compressed” normalizednormalizednormalized perturbedperturbed normalnormal –––AlphaAlpha Alpha containscontainscontains averagedaveragedaveraged normalnormalnormal shorteningshorteningshortening •••ButBut But nonono hardhardhard-wired--wiredwired perperper-pixel--pixelpixel “lighting“lighting“lighting engine”!engine”!engine”! Per-pixel lighting building blocks

CubeCubeCube mapsmapsmaps enableenableenable perperper-pixel--pixelpixel vectorvectorvector normalizationnormalizationnormalization •••CubeCube Cube mapmapmap normalizesnormalizesnormalizes interpolatedinterpolatedinterpolated (s,t,r)(s,t,r)(s,t,r) intointointo RGBRGBRGB –––(s,t,r)(s,t,r) (s,t,r) isisis interpolatedinterpolatedinterpolated perperper-vertex--vertexvertex lightlightlight ororor halfhalfhalf-angle--angleangle vectorvectorvector •••CombinersCombiners Combiners expandexpandexpand RGBRGBRGB fromfromfrom [0,1][0,1][0,1] rangerangerange tototo [[[-1,1]--1,1]1,1] rangerangerange –––expansionexpansion expansion donedonedone bybyby registerregisterregister combinerscombinerscombiners ••• OptimizationOptimizationOptimization::: IfIfIf infiniteinfiniteinfinite viewerviewerviewer ororor infiniteinfiniteinfinite lightlightlight used,used,used, halfhalfhalf-angle--angleangle cancancan bebebe computedcomputedcomputed withinwithinwithin thethethe cubecubecube map!map!map! •••32x32x632x32x6 32x32x6 RGB8RGB8RGB8 cubecubecube mapmapmap texturetexturetexture isisis sufficientsufficientsufficient Per-pixel lighting building blocks

RegisterRegisterRegister combinerscombinerscombiners performperformperform perperper-pixel--pixelpixel mathmathmath •••SignedSigned Signed mathmathmath forforfor dotdotdot productsproductsproducts –––LL L dotdotdot N’N’N’ forforfor diffuse,diffuse,diffuse, HHH dotdotdot N’N’N’ forforfor BlinnBlinnBlinn specularspecularspecular –––LL L andandand HHH properlyproperlyproperly normalizednormalizednormalized bybyby cubecubecube mapmapmap –––PrePre Pre-normalized--normalizednormalized N’N’N’ suppliedsuppliedsupplied bybyby normalnormalnormal mapmapmap

•••UsesUses Uses tangenttangenttangent spacespacespace LLLzzz forforfor geometricgeometricgeometric surfacesurfacesurface selfselfself-shadowing--shadowingshadowing •••ConstantConstant Constant colorcolorcolor forforfor addingaddingadding ininin ambientambientambient contributioncontributioncontribution •••SuccessiveSuccessive Successive squaringsquaringsquaring forforfor specularspecularspecular exponentexponent Surface self-shadowing

TwoTwoTwo kindskindskinds ofofof selfselfself-shadowing--shadowingshadowing •••max(0,max(0, max(0, LLL dotdotdot N’)N’)N’) basedbasedbased ononon thethethe perturbedperturbedperturbed normalnormalnormal •••AlsoAlso Also shouldshouldshould clampclampclamp whenwhenwhen LLL dotdotdot NNN goesgoesgoes negative!negative!negative!

N’N’N’ NNN NNN N’N’N’

LLL LLL SurfaceSurface shouldshould selfself-shadow-shadow SurfaceSurface shouldshould selfself-shadow-shadow duedue toto perturbedperturbed normal,normal, duedue toto unperturbedunperturbed normal,normal, I.e.,I.e., LL dotdot N’<0N’<0 I.e.,I.e., LL dotdot N<0N<0 Surface self-shadowing

SelfSelfSelf-shadowing--shadowingshadowing computationcomputationcomputation •••TechniqueTechnique Technique supportssupportssupports bumpedbumpedbumped selfselfself-shadowing--shadowingshadowing bybyby computingcomputingcomputing min(8*max(0,Lz),min(8*max(0,Lz),min(8*max(0,Lz), 1)1)1) *** max(0,max(0,max(0, LLL dotdotdot N’)N’)N’) •••AlsoAlso Also stashesstashesstashes thisthisthis selfselfself-shadowing--shadowingshadowing termtermterm ininin destinationdestinationdestination alphaalphaalpha min(8*max(0,Lz),min(8*max(0,Lz),min(8*max(0,Lz), 1)1)1) •••SpecularSpecular Specular passpasspass blendsblendsblends withwithwith DST_ALPHA,ONEDST_ALPHA,ONEDST_ALPHA,ONE computingcomputingcomputing min(8*max(0,Lz),min(8*max(0,Lz),min(8*max(0,Lz), 1)1)1) *** max(0,max(0,max(0, HHH dotdotdot N’)^8N’)^8N’)^8 •••IlluminationIllumination Illumination doesdoesdoes notnotnot appearappearappear ononon geometricgeometricgeometric “back“back“back side”side”side” –––SteepSteep Steep ramprampramp avoidsavoidsavoids winkingwinkingwinking andandand poppingpoppingpopping atatat selfselfself-shadow--shadowshadow boundaryboundaryboundary Complete lighting model

MoreMoreMore perperper-pixel--pixelpixel mathmathmath thanthanthan youyouyou mightmightmight think!think!think! •••AmbientAmbient Ambient === constantconstantconstant •••DiffuseDiffuse Diffuse === min(8*max(0,Lz),min(8*max(0,Lz),min(8*max(0,Lz), 1)1)1) *** max(0,max(0,max(0, LLL dotdotdot N’)N’)N’) •••SpecularSpecular Specular === min(8*max(0,Lz),min(8*max(0,Lz),min(8*max(0,Lz), 1)1)1) *** max(0,max(0,max(0, HHH dotdotdot N’)^8N’)^8N’)^8 •••FinalFinal Final === (Diffuse(Diffuse(Diffuse +++ Ambient)Ambient)Ambient) *** DecalDecalDecal +++ SpecularSpecularSpecular AttenuationAttenuationAttenuation and/orand/orand/or spotlightspotlightspotlight •••FinalFinal Final === (Attenuate*Diffuse(Attenuate*Diffuse(Attenuate*Diffuse +++ Ambient)Ambient)Ambient) *** DecalDecalDecal +++ Attenuate*SpecularAttenuate*SpecularAttenuate*Specular Self-shadowing example

LightLight leaksleaks ontoonto torustorus “back“back side”side” ProperProper geometricgeometric duedue toto badbad geometricgeometric selfself-shadowing-shadowing selfself-shadowing!-shadowing! Some CPU work required

PerPerPer-vertex--vertexvertex workworkwork •••ModelModel Model requiresrequiresrequires perperper-vertex--vertexvertex tangenttangenttangent spacespacespace basisbasisbasis –––anyany any bumpbumpbump mappingmappingmapping schemeschemescheme needsneedsneeds surfacesurfacesurface normalsnormalsnormals (N),(N),(N), tangentstangentstangents (T),(T),(T), andandand binormalsbinormalsbinormals (B)(B)(B) •••IfIf If lightlightlight ororor viewerviewerviewer (in(in(in thethethe specularspecularspecular case)case)case) changeschangeschanges relativerelativerelative tototo thththeee object,object,object, CPUCPUCPU rerere-compute--computecompute tangenttangenttangent spacespacespace lightlightlight andandand halfhalfhalf-angle--angleangle –––TransformTransform Transform objectobjectobject-space--spacespace vectorvectorvector bybyby [[[ TTT BBB NNN ]]] 3x33x33x3 matrixmatrixmatrix ••• NiceNiceNice split:split:split: CPUCPUCPU doesdoesdoes perperper-vertex--vertexvertex work,work,work, GPUGPUGPU doesdoesdoes transformtransformtransform andandand perperper-pixel--pixelpixel workworkwork Issues

TruthTruthTruth ininin advertisingadvertisingadvertising •••SpecularSpecular Specular exponentexponentexponent onlyonlyonly 888 sososo notnotnot tootootoo shinyshinyshiny –––bandingbanding banding evidentevidentevident ininin specularspecularspecular highlights,highlights,highlights, thoughthoughthough easyeasyeasy tototo hidehidehide artifactsartifactsartifacts ininin surfacesurfacesurface decaldecaldecal andandand diffusediffusediffuse contributioncontributioncontribution •••PerturbedPerturbed Perturbed normalnormalnormal slightlyslightlyslightly dedede-normalized--normalizednormalized bybyby filteringfilteringfiltering –––causescauses causes veryveryvery slightslightslight dimming,dimming,dimming, alternativealternativealternative isisis aliasingaliasingaliasing –––mostmost most objectionableobjectionableobjectionable whenwhenwhen magnifying,magnifying,magnifying, “grated”“grated”“grated” looklooklook •••NotNot Not “bumped“bumped“bumped environmentenvironmentenvironment mapping”mapping”mapping” –––butbut but DX6DX6DX6 bumpbumpbump-env--envenv supportsupportsupport isisis aaa jokejokejoke •••RequiresRequires Requires GeForceGeForceGeForce 256,256,256, Quadro,Quadro,Quadro, ororor anyanyany futurefuturefuture NVIDIANVIDIANVIDIA GPUGPUGPU Claim: practical

JustificationJustificationJustification forforfor thisthisthis claimclaimclaim •••RealReal Real-time--timetime frameframeframe ratesratesrates ononon currentcurrentcurrent generationgenerationgeneration GPUsGPUsGPUs –––tweakable:tweakable: tweakable: eliminateeliminateeliminate specularspecularspecular ororor decaldecaldecal pass,pass,pass, etc.etc.etc. •••MixableMixable Mixable withwithwith otherotherother multimultimulti-pass--passpass renderingrenderingrendering techniquestechniquestechniques –––stenciledstenciled stenciled shadows,shadows,shadows, multiplemultiplemultiple lights,lights,lights, fog,fog,fog, etc.etc.etc. •••BumpBump Bump mapmapmap easyeasyeasy tototo authorauthorauthor asasas ananan 888-bit--bitbit graygraygray-scale--scalescale imageimageimage –––examples:examples: examples: Claim: robust

JustificationJustificationJustification forforfor thisthisthis claimclaimclaim •••AnimatesAnimates Animates withwithwith minimalminimalminimal temporaltemporaltemporal aliasingaliasingaliasing •••ModelsModels Models bothbothboth selfselfself-shadowing--shadowingshadowing terms,terms,terms, includingincludingincluding forforfor specularspecularspecular •••DiffuseDiffuse Diffuse illuminationilluminationillumination filteredfilteredfiltered reasonablyreasonablyreasonably •••LightLight Light andandand halfhalfhalf-angle--angleangle vectorsvectorsvectors renormalizedrenormalizedrenormalized perperper-pixel--pixelpixel •••Local,Local, Local, attenuated,attenuated,attenuated, andandand spotspotspot-light--lightlight lightlightlight sourcessourcessources •••LocalLocal Local viewerviewerviewer •••Overall,Overall, Overall, veryveryvery reasonablereasonablereasonable fidelityfidelityfidelity tototo Blinn’sBlinn’sBlinn’s originaloriginaloriginal formulatioformulatioformulationnn Source available on the web already

BumpdemoBumpdemoBumpdemo sourcesourcesource codecodecode •••GoGo Go tototo www.nvidia.comwww.nvidia.comwww.nvidia.com DeveloperDeveloperDeveloper SectionSectionSection ––http://www.nvidia.com/developer.nsf/htmlmedia/devMainFR_top.html http://www.nvidia.com/developer.nsf/htmlmedia/devMainFR_top.html •••FindFind Find andandand downloaddownloaddownload bumpdemo.zipbumpdemo.zipbumpdemo.zip –––requiresrequires requires GeForceGeForceGeForce 256256256 ororor QuadroQuadroQuadro tototo runrunrun •••AlsoAlso Also downloaddownloaddownload “NVIDIA“NVIDIA“NVIDIA OpenGLOpenGLOpenGL ExtensionExtensionExtension Specifications”Specifications”Specifications” –––nvOpenGLspecs.pdfnvOpenGLspecs.pdf nvOpenGLspecs.pdf –––AdobeAdobe Adobe Acrobat,Acrobat,Acrobat, 197197197 pagespagespages –––documentsdocuments documents OpenGLOpenGLOpenGL extensionsextensionsextensions usedusedused bybyby bumpdemobumpdemobumpdemo Other novel uses for cube mapping

StableStableStable specularspecularspecular highlightshighlightshighlights •••EncodeEncode Encode specularspecularspecular &&& diffusediffusediffuse lightinglightinglighting solutionsolutionsolution ––Diffuse Diffuse cubecube mapmap usingusing normalnormal mapmap texgentexgen ––Specular Specular cubecube mapmap usingusing reflectionreflection mapmap texgentexgen ––Encode Encode unlimitedunlimited numbernumber ofof directionaldirectional lightslights •••ResultResult Result isisis stablestablestable specularspecularspecular highlightshighlightshighlights •••LessLess Less significantsignificantsignificant forforfor diffusediffusediffuse lightinglightinglighting ––Average Average ofof dotdot productsproducts ≅≅ dotdot productproduct ofof averagesaverages ––Excepting Excepting clampingclamping Example for stable specular highlights Cube map Standard lighting per-vertex lighting

Bright, stable Poor per-vertex specular highlight, sampling of the highlight. even at low tessellation. Wobbles during animation. Another example of stable specular highlights

HighHighHigh tessellationtessellationtessellation doesdoesdoes notnotnot completelycompletelycompletely fixfixfix perperper--- vertexvertexvertex artifactsartifactsartifacts

Still bright and stable. Better sampled, but still has streaky artifacts. Still more novel uses for cube maps & combiners

PerPerPer-pixel--pixelpixel specularspecularspecular normalnormalnormal mappingmappingmapping •••AssumesAssumes Assumes fancyfancyfancy perperper-pixel--pixelpixel dotdotdot productproductproduct operations!operations!operations! •••CubeCube Cube mapmapmap encodesencodesencodes normalizednormalizednormalized halfhalfhalf-angle--angleangle –––TexgenTexgen Texgen suppliessuppliessupplies viewviewview vectorvectorvector –––CubeCube Cube mapmapmap generatesgeneratesgenerates normalize(V+L)normalize(V+L)normalize(V+L) vectorvectorvector •••AnotherAnother Another 2D2D2D texturetexturetexture suppliessuppliessupplies perperper-pixel--pixelpixel surfacesurfacesurface normalsnormalsnormals •••PerPer Per-pixel--pixelpixel specularspecularspecular lightinglightinglighting •••PerPer Per-pixel--pixelpixel “normalized(V+L)“normalized(V+L)“normalized(V+L) dotdotdot N”N”N” !!! One pass per-pixel diffuse & specular example!

OpenGL per-vertex lighting with mesh of thousands of vertices (note displacement).

Per-pixel specular normal mapping with cube maps (flat, but lighting all there). Example in wireframe

Dense, dense mesh.

One polygon! Object-space per-pixel lighting example Object-space per-pixel example in wire-frame

TheTheThe detaildetaildetail isisis perperper-pixel!--pixel!pixel! Register combiners overview

••overrides overrides texturetexture stages/environment,stages/environment, colorcolor sum,sum, andand fogfog inin currentcurrent APIsAPIs ••signed signed mathmath (negative(negative oneone toto positivepositive range)range) ––extended extended rangerange throughthrough scalingscaling ••dot dot productsproducts forfor lightinglighting andand imageimage processingprocessing applicationsapplications ––designed designed forfor specular,specular, diffuse,diffuse, andand ambientambient perper-pixel-pixel lightinglighting ––object object spacespace bumpbump mapmap lightinglighting ––tangent tangent spacespace bumpbump mapmap lightinglighting ––post post-filtering-filtering 3x33x3 colorcolor matrixmatrix forfor colorcolor spacespace conversionsconversions ••register register modelmodel supportssupports nonnon-linear-linear datadata flowsflows ––superior superior toto linearlinear chainchain inin currentcurrent APIsAPIs ••effectively, effectively, aa VLIWVLIW instructioninstruction setset forfor fragmentfragment coloringcoloring ••very very efficientefficient hardwarehardware implementationimplementation Register combiners operational overview

initialize RGB Portion Alpha Portion registers

stage #0 AB+CD, AB, CD AB+CD, AB, CD

general combiner stages

stage #1 (optional) AB+CD, AB, CD AB+CD, AB, CD

final combiner AB+(1-A)C+D, EF, G RGBA fragment General combiners RGB operation

input registers output registers RGB A RGB A input input input input primary color map map map map primary color

secondary color secondary color AB C D texture 0 texture 0 A B + C D texture 1 -or- texture 1 A B mux C D spare 0 spare 0

spare 1 spare 1 A B scale fog -or- and fog bias A • B constant color 0 constant color 0

constant color 1 C D constant color 1 zero -or- zero C • D not readable not writeable computations General combiners Alpha operation

input registers output registers RGB A RGB A input input input input primary color map map map map primary color secondary color secondary color AB C D texture 0 texture 0 A B + C D texture 1 -or- texture 1 A B mux C D spare 0 spare 0

spare 1 spare 1 scale fog A B and fog bias constant color 0 constant color 0 constant color 1 constant color 1 C D zero zero

not writeable not readable computations Final combiner operation

input registers input input map map RGB A

primary color EF

secondary color E F texture 0 spare 0 + texture 1 secondary color

input input input input input spare 0 map map map map map

spare 1 AB C D G fog fragment RGB out constant color 0 A B + ( 1 - A) C + D

constant color 1 fragment Alpha out G zero

computations