Alpha Compositing Alpha Compositing • alpha compositing is the process of combining an image with a background to create the appearance of partial transparency. and • In order to correctly combine these image elements, it is Render Passes necessary to keep an associated matte for each element. • This matte contains the coverage information - the shape of the geometry being drawn • This allows us to distinguish between parts of the image where the geometry was actually drawn and other parts of the image which are empty. Tuesday, 2 February 2010 Tuesday, 2 February 2010 The Alpha Channel The Alpha Channel “A separate component is needed to retain the matte information, the extent of coverage of an element at a pixel. • To store this matte information, the concept of an alpha In a full colour rendering of an element, the RGB components retain only the channel was introduced by A. R. Smith (1970s) colour. In order to place the element over an arbitrary background, a mixing factor is required at every pixel to control the linear interpolation of foreground and • Porter and Duff then expanded this to give us the basic background colours. algebra of Compositing in the paper "Compositing Digital In general, there is no way to encode this component as part of the colour Images" in 1984. information. For anti-aliasing purposes, this mixing factor needs to be of comparable resolution to the colour channels. Let us call this an alpha channel, and let us treat an alpha of 0 to indicate no coverage, 1 to mean full coverage, with fractions corresponding to partial coverage.” Porter & Duff 84 Tuesday, 2 February 2010 Tuesday, 2 February 2010 Alpha Channel Pre multiplied alpha • In a 2D image element which stores a colour for each pixel, an additional value is “What is the meaning of the quadruple (r,g,b,a) at a pixel? stored in the alpha channel containing a value ranging from 0 to 1. How do we express that a pixel is half covered by a full red object? One obvious suggestion is to assign (1,0,0,.5} to that pixel: the .5 indicates the • 0 means that the pixel does not have any coverage information; i.e. there was no coverage and the (1,0,0) is the colour. colour contribution from any geometry because the geometry did not overlap There are a few reasons to dismiss this proposal, the most severe being that all this pixel. compositing operations will involve multiplying the 1 in the red channel by • 1 means that the pixel is fully opaque because the geometry completely the .5 in the alpha channel to compute the red contribution of this object at this overlapped the pixel. pixel. The desire to avoid this multiplication points up a better solution, storing the • It is important to distinguish between the following pre-multiplied value in the colour component, so that (.5,0,0,.5) will indicate a full red object half covering a pixel”. black = (0,0,0,1) clear = (0,0,0,0) Porter & Duff 84 Tuesday, 2 February 2010 Tuesday, 2 February 2010 Pre multiplied alpha Compositing Algebra • If an alpha channel is used in an image, it is common to also multiply • Porter and Duff proposed a number the colour by the alpha value, in order to save on additional of basic operations, which are multiplication during the Compositing process. performed on a per pixel, per RGB • This is usually referred to as pre-multiplied alpha. channel basis • Thus, assuming that the pixel colour is expressed using RGB triples, • Most of these operations pre- suppose that the RGB channels have • a pixel value of (0.0, 0.5, 0.0, 0.5) implies a pixel which is fully green already been pre-multiplied by the and has 50% coverage. alpha value Tuesday, 2 February 2010 Tuesday, 2 February 2010 A over B • The basic over operator is similar to the painters algorithm where the A element is placed over the B element. • For each Pixel we do the following C = C + C (1 α ) o a b × − a α = α + α (1 α ) o a b × − a Tuesday, 2 February 2010 Tuesday, 2 February 2010 Other Functions Unary operators To assist in dissolving and colour balancing Porter and Duff Function Use Maths • also suggested the following operations Atop Add effects to foreground Co = Ca α +(C (1 αa)) × b b × − ADD Add mattes together Co = Ca + Cb darken(A, φ) (φr , φg , φb , α ) ≡ A A A A DIV Co = Ca Cb ÷ dissolve(A, δ) (δrA, δgA, δbA, δαA) Mult Mask Elements Co = Ca C ≡ × b Inside Mask Elements Co = Ca α Normally 0 φ, δ 1 although none of the theory requires it × b ≤ ≤ Sub Subtract Co = Ca Cb − As φ varies from 1 to 0, the element will change from normal to complete blackness. outside Mask Elements Co = Ca (1 αb) × − If φ > 1 the element will be brightened. xor Co = Ca (1 αb)+Cb (1 αa) × − × − As δ goes from 1 to 0 the element will gradually fade from view Tuesday, 2 February 2010 Tuesday, 2 February 2010 Luminescent objects The Plus Operator • Luminescent objects, which add colour information without • The expression A plus B holds no notion of precedence in any obscuring the background, can be handled with the introduction of a area covered by both pictures; the components are simply added. opaqueness factor ω, 0 ω 1 ≤ ≤ • This allows us to dissolve from one picture to another by specifying opaque(A, ω) (rA,gA,bA, ωαA) ≡ dissolve(A, α) plus dissolve(B,1 α) As ω varies from 1 to 0, the element will change from normal coverage over the background to no obscuration. − • In terms of the binary operators above, plus allows both pictures to survive in the subpixel area AB. Tuesday, 2 February 2010 Tuesday, 2 February 2010 The OpenEXR File Format OpenEXR Channels OpenEXR is a high dynamic-range (HDR) image file format • An OpenEXR image can have any number of channels with arbitrary developed by Industrial Light & Magic for use in computer imaging • names. applications. The specialized RGBA image interface assumes that channels with OpenEXR's features include: • • the names "R", "G", "B" and "A" mean red, green, blue and alpha. Higher dynamic range and color precision than existing 8- and • No predefined meaning has been assigned to any other channels. 10-bit image file formats. • This means that 1 exr file can actually contain multiple image planes • Support for 16-bit floating-point, 32-bit floating-point, and 32-bit • integer pixels. The 16-bit floating-point format, called "half", is • At present we can do this with Renderman and Mantra by use of compatible with the half data type in NVIDIA's Cg graphics Auxiliary Output Variables (Mental ray at present does not let us do language and is supported natively on their new GeForce FX this) and Quadro FX 3D graphics solutions. Tuesday, 2 February 2010 Tuesday, 2 February 2010 1 2 AllPasses.exr: 3 4 file format version: 2, flags 0x0 5 capDate (type string): "2009:01:27 17:33:25" 6 channels (type chlist): 7 Oambient.b, 32-bit floating-point, sampling 1 1 Rendering in Passes 8 Oambient.g, 32-bit floating-point, sampling 1 1 9 Oambient.r, 32-bit floating-point, sampling 1 1 exrinfo 10 Obounce.b, 32-bit floating-point, sampling 1 1 11 Obounce.g, 32-bit floating-point, sampling 1 1 Rendering in Passes is not a new technique for 3D graphics 12 Obounce.r, 32-bit floating-point, sampling 1 1 • 13 Odiffuse.b, 32-bit floating-point, sampling 1 1 14 Odiffuse.g, 32-bit floating-point, sampling 1 1 15 Odiffuse.r, 32-bit floating-point, sampling 1 1 16 Oopacity.b, 32-bit floating-point, sampling 1 1 The word pass originated in motion-control photography used for 17 Oopacity.g, 32-bit floating-point, sampling 1 1 • 18 Oopacity.r, 32-bit floating-point, sampling 1 1 19 Oreflect.b, 32-bit floating-point, sampling 1 1 miniatures. 20 Oreflect.g, 32-bit floating-point, sampling 1 1 21 Oreflect.r, 32-bit floating-point, sampling 1 1 22 Ospecular.b, 32-bit floating-point, sampling 1 1 23 Ospecular.g, 32-bit floating-point, sampling 1 1 24 Ospecular.r, 32-bit floating-point, sampling 1 1 Each time a motion is repeated it is called a “pass” 25 OwsNormal.b, 32-bit floating-point, sampling 1 1 • 26 OwsNormal.g, 32-bit floating-point, sampling 1 1 27 OwsNormal.r, 32-bit floating-point, sampling 1 1 28 OwsPtPos.b, 32-bit floating-point, sampling 1 1 29 OwsPtPos.g, 32-bit floating-point, sampling 1 1 Usually the first pass was fully lit and called the “beauty pass” 30 OwsPtPos.r, 32-bit floating-point, sampling 1 1 • 31 clipFar (type float): 20 32 clipNear (type float): 0.1 33 comments (type string): "Created using: Pixar PhotoRealistic RenderMan 14.2 osxTiger_x 86_gcc40icc101_external_release Next the lighting and film would be changed for a High contrast “matte” 34 Created on: neuromancer" • 35 compression (type compression): zip, multi-scanline blocks 36 dataWindow (type box2i): (0 0) - (719 575) pass. 37 displayWindow (type box2i): (0 0) - (719 575) 38 lineOrder (type lineOrder): increasing y 39 pixelAspectRatio (type float): 1 40 screenWindowCenter (type v2f): (-2.08616e-07 -1.49012e-08) 41 screenWindowWidth (type float): 1.16577 For a third pass a light may be turned on in the model. 42 utcOffset (type float): 0 43 worldToCamera (type m44f): • 44 (0.819152 -0.0996005 -0.564863 0 45 0 0.984808 -0.173648 0 46 0.573576 0.142244 0.806707 0 47 0 0 3 1) Finally all the passes used to be printed together optically for the final shot 48 worldToNDC (type m44f): • 49 (1.40534 -0.213594 -0.567701 -0.564863 50 0 2.11193 -0.174521 -0.173648 (before the day of digital compositing systems) 51 0.984031 0.305044 0.810761 0.806707 52 0 0 2.91457 3) Tuesday, 2 February 2010 Tuesday, 2 February 2010 Render Passes Shader Development It is usually preferable to split our renders up into multiple passes • Typically when developing a shader we combine in the shader a • number of different elements layered together.