Alpha Alpha Compositing • alpha compositing is the process of combining an image with a background to create the appearance of partial . and • In order to correctly combine these image elements, it is Render Passes necessary to keep an associated for each element. • This matte contains the coverage information - the shape of the geometry being drawn • This allows us to distinguish between parts of the image where the geometry was actually drawn and other parts of the image which are empty.

Tuesday, 2 February 2010 Tuesday, 2 February 2010

The Alpha The Alpha Channel “A separate component is needed to retain the matte information, the extent of coverage of an element at a pixel. • To store this matte information, the concept of an alpha In a full colour rendering of an element, the RGB components retain only the channel was introduced by A. R. Smith (1970s) colour. In order to place the element over an arbitrary background, a mixing factor is required at every pixel to control the linear interpolation of foreground and • Porter and Duff then expanded this to give us the basic background colours. algebra of Compositing in the paper "Compositing Digital In general, there is no way to encode this component as part of the colour Images" in 1984. information. For anti-aliasing purposes, this mixing factor needs to be of comparable resolution to the colour channels. Let us call this an alpha channel, and let us treat an alpha of 0 to indicate no coverage, 1 to mean full coverage, with fractions corresponding to partial coverage.”

Porter & Duff 84

Tuesday, 2 February 2010 Tuesday, 2 February 2010 Alpha Channel Pre multiplied alpha

• In a 2D image element which stores a colour for each pixel, an additional value is “What is the meaning of the quadruple (r,g,b,a) at a pixel? stored in the alpha channel containing a value ranging from 0 to 1. How do we express that a pixel is half covered by a full red object? One obvious suggestion is to assign (1,0,0,.5} to that pixel: the .5 indicates the • 0 means that the pixel does not have any coverage information; i.e. there was no coverage and the (1,0,0) is the colour. colour contribution from any geometry because the geometry did not overlap There are a few reasons to dismiss this proposal, the most severe being that all this pixel. compositing operations will involve multiplying the 1 in the red channel by • 1 means that the pixel is fully opaque because the geometry completely the .5 in the alpha channel to compute the red contribution of this object at this overlapped the pixel. pixel. The desire to avoid this multiplication points up a better solution, storing the • It is important to distinguish between the following pre-multiplied value in the colour component, so that (.5,0,0,.5) will indicate a full red object half covering a pixel”. black = (0,0,0,1) clear = (0,0,0,0) Porter & Duff 84

Tuesday, 2 February 2010 Tuesday, 2 February 2010

Pre multiplied alpha Compositing Algebra

• If an alpha channel is used in an image, it is common to also multiply • Porter and Duff proposed a number the colour by the alpha value, in order to save on additional of basic operations, which are multiplication during the Compositing process. performed on a per pixel, per RGB • This is usually referred to as pre-multiplied alpha. channel basis • Thus, assuming that the pixel colour is expressed using RGB triples, • Most of these operations pre- suppose that the RGB channels have • a pixel value of (0.0, 0.5, 0.0, 0.5) implies a pixel which is fully green already been pre-multiplied by the and has 50% coverage. alpha value

Tuesday, 2 February 2010 Tuesday, 2 February 2010 A over B • The basic over operator is similar to the painters algorithm where the A element is placed over the B element. • For each Pixel we do the following

C = C + C (1 α ) o a b × − a α = α + α (1 α ) o a b × − a

Tuesday, 2 February 2010 Tuesday, 2 February 2010

Other Functions Unary operators

To assist in dissolving and colour balancing Porter and Duff Function Use Maths • also suggested the following operations Atop Add effects to foreground Co = Ca α +(C (1 αa)) × b b × − ADD Add mattes together Co = Ca + Cb darken(A, φ) (φr , φg , φb , α ) ≡ A A A A DIV Co = Ca Cb ÷ dissolve(A, δ) (δrA, δgA, δbA, δαA) Mult Mask Elements Co = Ca C ≡ × b Inside Mask Elements Co = Ca α Normally 0 φ, δ 1 although none of the theory requires it × b ≤ ≤ Sub Subtract Co = Ca Cb − As φ varies from 1 to 0, the element will change from normal to complete blackness. outside Mask Elements Co = Ca (1 αb) × − If φ > 1 the element will be brightened. xor Co = Ca (1 αb)+Cb (1 αa) × − × − As δ goes from 1 to 0 the element will gradually fade from view

Tuesday, 2 February 2010 Tuesday, 2 February 2010 Luminescent objects The Plus Operator • Luminescent objects, which add colour information without • The expression A plus B holds no notion of precedence in any obscuring the background, can be handled with the introduction of a area covered by both pictures; the components are simply added. opaqueness factor ω, 0 ω 1 ≤ ≤ • This allows us to dissolve from one picture to another by specifying opaque(A, ω) (rA,gA,bA, ωαA) ≡ dissolve(A, α) plus dissolve(B,1 α) As ω varies from 1 to 0, the element will change from normal coverage over the background to no obscuration. − • In terms of the binary operators above, plus allows both pictures to survive in the subpixel area AB.

Tuesday, 2 February 2010 Tuesday, 2 February 2010

The OpenEXR File Format OpenEXR Channels OpenEXR is a high dynamic-range (HDR) image file format • An OpenEXR image can have any number of channels with arbitrary developed by Industrial Light & Magic for use in computer imaging • names. applications. The specialized RGBA image interface assumes that channels with OpenEXR's features include: • • the names "R", "G", "B" and "A" mean red, green, blue and alpha. Higher dynamic range and color precision than existing 8- and • No predefined meaning has been assigned to any other channels. 10-bit image file formats. • This means that 1 exr file can actually contain multiple image planes • Support for 16-bit floating-point, 32-bit floating-point, and 32-bit • integer pixels. The 16-bit floating-point format, called "half", is • At present we can do this with Renderman and Mantra by use of compatible with the half data type in NVIDIA's Cg graphics Auxiliary Output Variables (Mental ray at present does not let us do language and is supported natively on their new GeForce FX this) and Quadro FX 3D graphics solutions.

Tuesday, 2 February 2010 Tuesday, 2 February 2010 1 2 AllPasses.exr: 3 4 file format version: 2, flags 0x0 5 capDate (type string): "2009:01:27 17:33:25" 6 channels (type chlist): 7 Oambient.b, 32-bit floating-point, sampling 1 1 Rendering in Passes 8 Oambient.g, 32-bit floating-point, sampling 1 1 9 Oambient.r, 32-bit floating-point, sampling 1 1 exrinfo 10 Obounce.b, 32-bit floating-point, sampling 1 1 11 Obounce.g, 32-bit floating-point, sampling 1 1 Rendering in Passes is not a new technique for 3D graphics 12 Obounce.r, 32-bit floating-point, sampling 1 1 • 13 Odiffuse.b, 32-bit floating-point, sampling 1 1 14 Odiffuse.g, 32-bit floating-point, sampling 1 1 15 Odiffuse.r, 32-bit floating-point, sampling 1 1 16 Oopacity.b, 32-bit floating-point, sampling 1 1 The word pass originated in motion-control photography used for 17 Oopacity.g, 32-bit floating-point, sampling 1 1 • 18 Oopacity.r, 32-bit floating-point, sampling 1 1 19 Oreflect.b, 32-bit floating-point, sampling 1 1 miniatures. 20 Oreflect.g, 32-bit floating-point, sampling 1 1 21 Oreflect.r, 32-bit floating-point, sampling 1 1 22 Ospecular.b, 32-bit floating-point, sampling 1 1 23 Ospecular.g, 32-bit floating-point, sampling 1 1 24 Ospecular.r, 32-bit floating-point, sampling 1 1 Each time a motion is repeated it is called a “pass” 25 OwsNormal.b, 32-bit floating-point, sampling 1 1 • 26 OwsNormal.g, 32-bit floating-point, sampling 1 1 27 OwsNormal.r, 32-bit floating-point, sampling 1 1 28 OwsPtPos.b, 32-bit floating-point, sampling 1 1 29 OwsPtPos.g, 32-bit floating-point, sampling 1 1 Usually the first pass was fully lit and called the “beauty pass” 30 OwsPtPos.r, 32-bit floating-point, sampling 1 1 • 31 clipFar (type float): 20 32 clipNear (type float): 0.1 33 comments (type string): "Created using: Pixar PhotoRealistic RenderMan 14.2 osxTiger_x 86_gcc40icc101_external_release Next the lighting and film would be changed for a High contrast “matte” 34 Created on: neuromancer" • 35 compression (type compression): zip, multi-scanline blocks 36 dataWindow (type box2i): (0 0) - (719 575) pass. 37 displayWindow (type box2i): (0 0) - (719 575) 38 lineOrder (type lineOrder): increasing y 39 pixelAspectRatio (type float): 1 40 screenWindowCenter (type v2f): (-2.08616e-07 -1.49012e-08) 41 screenWindowWidth (type float): 1.16577 For a third pass a light may be turned on in the model. 42 utcOffset (type float): 0 43 worldToCamera (type m44f): • 44 (0.819152 -0.0996005 -0.564863 0 45 0 0.984808 -0.173648 0 46 0.573576 0.142244 0.806707 0 47 0 0 3 1) Finally all the passes used to be printed together optically for the final shot 48 worldToNDC (type m44f): • 49 (1.40534 -0.213594 -0.567701 -0.564863 50 0 2.11193 -0.174521 -0.173648 (before the day of systems) 51 0.984031 0.305044 0.810761 0.806707 52 0 0 2.91457 3)

Tuesday, 2 February 2010 Tuesday, 2 February 2010

Render Passes Development

It is usually preferable to split our renders up into multiple passes • Typically when developing a shader we combine in the shader a • number of different elements layered together. • These can either be separate files for each pass or combined into one exr file. • Usually for a simple pass we would have a basic colour element and then several elements which take into account the position of • This will allow us to make adjustments to overall colour values in the the viewer and the lights in the scene. image, and also add extra effects or even place elements in different positions • Next week we will start to look at the mathematics behind this next week • It is also not uncommon to output other non colour values which give us information about elements of the scene such as depth from • This week we will introduce the basic terms and techniques used camera, surface normals, motion vectors etc. in rendering.

Tuesday, 2 February 2010 Tuesday, 2 February 2010 Ambient Pass Diffuse Pass • The diffuse pass usually has the basic shading model applied to the objects • This pass takes into account Position of the lights in the scene as well as the orientation of the surface (based on the surface Normal) • The top pass has no shadows • The 2nd pass has the shadows included as by default the renderman ray trace • The ambient pass on the right is the basic colour of the object shadows are in the diffuse pass multiplied by the ambient light values set for the scene. • The colour on the left is just the colour assigned to the object

Tuesday, 2 February 2010 Tuesday, 2 February 2010

Specular Pass Combining Basic Passes • Usually when using a basic Blinn / Plastic / Phong Shader the output of the render will be a combination of the Ambient, Diffuse and Specular • The specular or Highlight pass takes contributions into account the direction of the light • If we have separate passes we can combine them to get the same effect • Specular highlights are important in using the following compositing structure 3D , as they provide a strong visual cue for the shape of an object and its location with respect to light sources in the scene.

Tuesday, 2 February 2010 Tuesday, 2 February 2010 Ray tracing Reflection pass • The previous images can be produced very easily using a process called scan line rendering • Usually we can determine the • This process works on a row-by-row basis rather than a polygon- number of times the rays bounce by-polygon or pixel-by-pixel basis. for the reflection pass • However it doesn’t take into account rays bouncing off of object • In this case 2 but increasing to 3 and reflections / refraction in the scene or 4 will increase the render times logarithmically • To do this we need to invoke ray-tracing which will bounce rays around the scene to gather more information about the lights in • It is best to keep the bounce the scene. levels as low as possible • This increases the processing required and will slow down renders

Tuesday, 2 February 2010 Tuesday, 2 February 2010

indirect diffuse Advanced Passes • So far we have dealt with passes that contain colour information Indirect diffuse illumination • However we can save out other information instead of rgb values. can add different elements • such as colour bleeding • Typically we still save to the rgb channels but we store other data in the same format. • There are a number of methods of calculating this • For example we may want to store the surface normal values which including, Photon mapping are usually in the form of a Normalized vector N [x,y,z] so the r channel becomes x, g=y and b=z. • Renderman has a build in function called indirectdiffuse • We can view these images but they are typically used in the compositor for advanced effects such as re-lighting or Zdepth DOF again this process can be slow • effects

Tuesday, 2 February 2010 Tuesday, 2 February 2010 zDepth Normals

• This zDepth pass was created in renderman using the built in depth • The surface normal of the object in function view is stored in the image • This allows us to pass the current • This is usually normalized and point being shaded and get the forced to face towards the camera / depth relative to the camera viewer to ensure it is in the correct direction 1 Zdepth = 1-depth(P);

Tuesday, 2 February 2010 Tuesday, 2 February 2010

Points Point re-lighting

• This pass contains the visible point transformed into world co- ordinates. • These values can be used for re- lighting / additional lighting in the scene

Tuesday, 2 February 2010 Tuesday, 2 February 2010 Spot Light Point Clouds

Tuesday, 2 February 2010 Tuesday, 2 February 2010

References Further Reading • Thomas Porter and Tom Duff, Compositing Digital Images, Computer Graphics, 18(3), July 1984, • http://en.wikipedia.org/wiki/Alpha_transparency • http://www.denofgeek.com/movies/139783/ the_den_of_geek_interview_john_dykstra.html#dykstraflex • Apple Shake Ref Guide • http://www.openexr.com/documentation.html • Digital Lighting and Rendering Jeremy Birn

Tuesday, 2 February 2010 Tuesday, 2 February 2010