Objectives
Programming with WebGL •Development of the OpenGL API Part 1: Background •OpenGL Architecture - OpenGL as a state machine CS 432 Interactive Computer Graphics - WebGL as a data flow machine Prof. David E. Breen •Functions Department of Computer Science - Types - Formats •Simple program
E. Angel and D. Shreiner: Interactive Computer Graphics 6E © Addison-Wesley 2012 1 E. Angel and D. Shreiner: Interactive Computer Graphics 6E © Addison-Wesley 2012 2
1 2
Retained vs. Immediate Mode Early History of APIs Graphics • Immediate •IFIPS (1973) formed two committees to - Geometry is drawn when CPU sends it to GPU come up with a standard graphics API - All data needs to be resent even if little changes - Graphical Kernel System (GKS) - Once drawn, geometry on GPU is discarded • 2D but contained good workstation model - Requires major bandwidth between CPU and GPU - Core - Minimizes memory requirements on GPU • Both 2D and 3D • Retained - GKS adopted as IS0 and later ANSI standard - Geometry is sent to GPU and stored (1980s) - It is displayed when directed by CPU •GKS not easily extended to 3D (GKS-3D) - CPU may send transformations to move geometry - Far behind hardware development - Minimizes data transfers, but GPU now needs enough memory to store geometry 3 E. Angel and D. Shreiner: Interactive Computer Graphics 6E © Addison-Wesley 2012 4 3 4
PHIGS and X SGI and GL
•Programmers Hierarchical Graphics •Silicon Graphics (SGI) revolutionized the System (PHIGS) graphics workstation by implementing the - Arose from CAD community graphics pipeline in hardware (1982) - Database model with retained graphics (structures) •To access the system, application •X Window System programmers used a library called GL - DEC/MIT effort •With GL, it was relatively simple to - Client-server architecture with graphics program three dimensional interactive •PEX combined the two applications - Not easy to use (all the defects of each)
E. Angel and D. Shreiner: Interactive Computer Graphics 6E © Addison-Wesley 2012 5 E. Angel and D. Shreiner: Interactive Computer Graphics 6E © Addison-Wesley 2012 6
5 6
1 OpenGL OpenGL Evolution
The success of GL lead to OpenGL (1992), • Originally controlled by an Architectural Review a platform-independent API that was Board (ARB) - Easy to use - Members included SGI, Microsoft, Nvidia, HP, 3DLabs, IBM,……. - Close enough to the hardware to get excellent performance - Now Khronos Group - Focused on rendering - Was relatively stable (through version 2.5) • Backward compatible - Omitted windowing and input to avoid window • Evolution reflected new hardware capabilities system dependencies – 3D texture mapping and texture objects - An immediate mode system, that later added – Vertex and fragment programs retained mode functionality - Allows platform specific features through extensions
E. Angel and D. Shreiner: Interactive Computer Graphics 6E © Addison-Wesley 2012 7 E. Angel and D. Shreiner: Interactive Computer Graphics 6E © Addison-Wesley 2012 8
7 8
Modern OpenGL OpenGL 3.1 (2009)
•Performance is achieved by using GPU •Totally shader-based rather than CPU - No default shaders •Control GPU through programs called - Each application must provide both a vertex shaders and a fragment shader •Application’s job is to send data to GPU •No immediate mode •GPU does all rendering •Few state variables •Most 2.5 functions deprecated - deprecate in CS - To mark (a component of a software standard) as obsolete to warn against its use in the future, so that it may be phased out. •Backward compatibility not required
E. Angel and D. Shreiner: Interactive Computer Graphics 6E © Addison-Wesley 2012 9 E. Angel and D. Shreiner: Interactive Computer Graphics 6E © Addison-Wesley 2012 10
9 10
What About Other Low-Level Other Versions Graphics Libraries? • OpenGL ES •Direct3D - Embedded systems - Part of DirectX, Windows-only - Version 1.0 simplified OpenGL 2.1 - Version 2.0 simplified OpenGL 3.1 •Mantle • Shader based - Developed by AMD - Version 3.0 simplified OpenGL 4.3 •Metal • WebGL 1.0 - Developed by Apple - Javascript implementation of ES 2.0 •Vulkan - Supported on newer browsers - “next-gen” OpenGL • OpenGL 4.1 è 4.5 - Added geometry & compute shaders and tessellator - Derived from Mantle
E. Angel and D. Shreiner: Interactive Computer Graphics 6E © Addison-Wesley 2012 11 E. Angel and D. Shreiner: Interactive Computer Graphics 6E © Addison-Wesley 2012 12
11 12
2 OpenGL Architecture A Simple Program (?)
Generate a square on a solid background
E. Angel and D. Shreiner: Interactive Computer Graphics 6E © Addison-Wesley 2012 13 E. Angel and D. Shreiner: Interactive Computer Graphics 6E © Addison-Wesley 2012 15
13 15
It used to be easy What happened
#include
E. Angel and D. Shreiner: Interactive Computer Graphics 6E © Addison-Wesley 2012 16 E. Angel and D. Shreiner: Interactive Computer Graphics 6E © Addison-Wesley 2012 17
16 17
Execution in Browser Event Loop
URL Browser Web Server •Remember that the sample program Web Page HTML specifies a render function which is a JS files JS Engine event listener or callback function - Every program should have a render callback - For a static application we need only execute Application CPU/GPUprogram the render function once - In a dynamic application, the render function can call itself recursively but each redrawing of Framebuffer Canvas the display must be triggered by an event
Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015 18 Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015 19
18 19
3 Lack of Object Orientation WebGL function format
•All versions of OpenGL are not object function name oriented so that there are multiple functions dimension for a given logical function gl.uniform3f(x,y,z) •Example: sending values to shaders -glUniform3f belongs to WebGL canvas x,y,z are float variables -glUniform2i -glUniform3dv •Underlying storage mode is the same gl.uniform3fv(p) p is an array
E. Angel and D. Shreiner: Interactive Computer Graphics 6E © Addison-Wesley 2012 21 Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015 22
21 22
WebGL constants WebGL and GLSL
•Most constants are defined in the canvas •WebGL requires shaders and is based object less on a state machine model than a - In desktop OpenGL, they were in #include files data flow model such as gl.h •Most state variables, attributes and •Examples related pre-3.1 OpenGL functions have -desktop OpenGL been deprecated • glEnable(GL_DEPTH_TEST); •Action happens in shaders -WebGL • gl.enable(gl.DEPTH_TEST) •Job of application is to get data to GPU -gl.clear(gl.COLOR_BUFFER_BIT)
Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015 23 Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015 24
23 24
GLSL
•OpenGL Shading Language Programming with OpenGL •C-like with Part 2: Complete Programs - Matrix and vector types (2, 3, 4 dimensional) - Overloaded operators - C++ like constructors •Similar to Nvidia’s Cg and Microsoft HLSL •Code sent to shaders as source code •WebGL functions compile, link and get information to shaders
E. Angel and D. Shreiner: Interactive Computer Graphics 6E © Addison-Wesley 2012 25 E. Angel and D. Shreiner: Interactive Computer Graphics 6E © Addison-Wesley 2012 26
25 26
4 Objectives Coding in WebGL
•Build a complete first program •Example: Draw a square - Introduce shaders - Each application consists of (at least) two files - Introduce a standard program structure - HTML file and a JavaScript file •Simple viewing •HTML - Two-dimensional viewing as a special case of - describes page three-dimensional viewing - includes utilities •Initialization steps and program structure - includes shaders •JavaScript - contains the graphics code
E. Angel and D. Shreiner: Interactive Computer Graphics 6E © Addison-Wesley 2012 27 Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015 28
27 28
Coding in WebGL Square Program
•Can run WebGL on any recent browser - Chrome - Firefox - Safari - Edge •Code written in JavaScript •JS runs within browser - Use local resources
Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015 29 Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015
29 30
WebGL square.html
•Five steps Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015
31 32
5 Old square.html Shaders
• We assign names to the shaders that we - one required built-in variable (gl_Position) in the
• Must set precision in fragment shader Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015 Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015
33 34
square.html (cont) Files
•../Common/initShaders.js: contains JS and WebGL code for reading, compiling and linking the shaders
: our matrix-vector package •square.js: the application fileAngel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015 Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015
35 36
square.js Notes
var canvas; : determines where to start var gl; •onload
window.onload = function init(){ execution when all code is loaded canvas = document.getElementById( "gl-canvas" ); •canvas gets WebGL context from HTML file gl = canvas.getContext('webgl2'); if ( !gl ) { alert( "WebGL 2.0 isn't available" ); } •vertices use vec2 type in MV.js // Four Vertices •JS array is not the same as a C or Java var vertices = [ array vec2( -0.5, -0.5 ), vec2( -0.5, 0.5 ), - object with methods vec2( 0.5, 0.5 ), vec2( 0.5, -0.5) - vertices.length // 4 ]; •Values in clip coordinates
Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015 Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015
37 38
6 square.js (cont) Notes
// Configure WebGL •initShaders used to load, compile and gl.viewport( 0, 0, canvas.width, canvas.height ); gl.clearColor( 0.0, 0.0, 0.0, 1.0 ); link shaders to form a program object // Load shaders and initialize attribute buffers • Load data onto GPU by creating a vertex var program = initShaders( gl, "vertex-shader", "fragment-shader" ); gl.useProgram( program ); buffer object on the GPU
// Load the data into the GPU - Note use of flatten() to convert JS array to an array of float32’s var bufferId = gl.createBuffer(); gl.bindBuffer( gl.ARRAY_BUFFER, bufferId ); gl.bufferData( gl.ARRAY_BUFFER, flatten(vertices), gl.STATIC_DRAW ); •Finally we must connect variable in
// Associate our shader variables with our data buffer program with variable in shader
var aPosition = gl.getAttribLocation( program, ”aPosition" ); - need name, type, location in buffer gl.vertexAttribPointer( aPosition, 2, gl.FLOAT, false, 0, 0 ); gl.enableVertexAttribArray( aPosition ); Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015 Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015
39 40
square.js (cont) Triangles, Fans or Strips
render(); }; gl.drawArrays( gl.TRIANGLE_FAN, 0, 4 ); // 0, 1 , 2, 3
function render() { gl.drawArrays( gl.TRIANGLES, 0, 6 ); // 0, 1, 2, 0, 2, 3 gl.clear( gl.COLOR_BUFFER_BIT ); gl.drawArrays( gl.TRIANGLE_FAN, 0, 4 ); } 1 2 1 2 1 2
0 3 0 3 3 0 gl.drawArrays( gl.TRIANGLE_STRIP, 0, 4 ); // 0, 1, 3, 2
Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015 Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015
41 42
JavaScript Notes JS Notes
•JavaScript (JS) is the language of the •Is JS slow? Web - JS engines in browsers are getting much faster - All browsers will execute JS code - Not a key issues for graphics since once we get - JavaScript is an interpreted object-oriented the data to the GPU it doesn’t matter how we language got the data there •References •JS is a (too) big language - Flanagan, JavaScript: The Definitive Guide, - We don’t need to use it all O’Reilly - Choose parts we want to use - Crockford, JavaScript, The Good Parts, O’Reilly - Don’t try to make your code look like C or Java - Many Web tutorials
Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015 43 Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015 44
43 44
7 JS Notes Scoping
•Very few native types: •Different from other languages - numbers •Function scope - strings •variables are hoisted within a function - booleans - can use a variable before it is declared •Only one numerical type: 64 bit float •Note functions are first class objects in JS - var x = 1; - var x = 1.0; // same - potential issue in loops - two operators for equality == and === •Dynamic typing Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015 45 Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015 46
45 46
JS Arrays Typed Arrays
•JS arrays are objects JS has typed arrays that are like C arrays - inherit methods - var a = [1, 2, 3]; var a = new Float32Array(3) is not the same as in C++ or Java var b = new Uint8Array(3) - a.length // 3 - a.push(4); // length now 4 - a.pop(); // 4 Generally, we prefer to work with standard - avoids use of many loops and indexing JS arrays and convert to typed arrays only - Problem for WebGL which expects C-style arrays when we need to send data to the GPU with the flatten function in MV.js Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015 47 Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015 48
47 48
A Minimalist Approach Buffer Object
•We will use only core JS and HTML •Buffers objects allow us to transfer large - no extras or variants amounts of data to the GPU •No additional packages •Need to create, bind (make current) and - CSS identify/specify data - JQuery var buffer_id; buffer_id = gl.createBuffer(); •Focus on graphics gl.bindBuffer(gl.ARRAY_BUFFER, buffer_id); - examples may lack beauty gl.bufferData(gl.ARRAY_BUFFER, data, gl.STATIC_DRAW);
•Data in current buffer is sent to GPU Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015 49 E. Angel and D. Shreiner: Interactive Computer Graphics 6E © Addison-Wesley 2012 50
49 50
8 Why use Buffer Objects? gl.createBuffer()
Only Advantages •gl.createBuffer() • The memory manager in the buffer object will - creates a buffer object and returns the buffer put the data into the best memory locations object based on user's hints • Memory manager can optimize the buffers by WebGLBuffer gl.createBuffer() balancing between 3 kinds of memory: - system, GPU and video memory • Returns a WebGLBuffer for storing data such as vertices or colors.
51 53
gl.bindBuffer() gl.bufferData()
- Once the buffer object has been created, we need to - You can initialize and copy the data into the buffer bind it to a target. object with gl.bufferData(). - Also makes the buffer “current” void gl.bufferData(GLenum target, GLsizeiptr size, void gl.bindBuffer(GLenum target, WebGLBuffer buffer) GLenum usage) void gl.bufferData(GLenum target, ArrayBuffer data, - Target can be GLenum usage) • gl.ARRAY_BUFFER: Any vertex attribute, such as vertex • target is either GL_ARRAY_BUFFER or coordinates, texture coordinates, normals and color component arrays GL_ELEMENT_ARRAY_BUFFER. • gl.ELEMENT_ARRAY_BUFFER: Index array which is used for • size is the number of bytes of data to transfer. glDraw[Range]Elements() • Data is the array holding the data to be copied. - Once first called, the buffer is initialized with a zero- • "usage" flag is a performance hint to provide how the buffer sized memory buffer and sets the initial states object is going to be used: static, dynamic or stream, and read, copy or draw. 54 55
Usage Flags gl.bufferSubData()
- gl.STATIC_DRAW • Contents of the buffer are likely to be used often and void gl.bufferSubData(GLenum target, not change often. GLintptr offset, ArrayBuffer data) - gl.DYNAMIC_DRAW • Contents of the buffer are likely to be used often and change often. - Like gl.bufferData(), - gl.STREAM_DRAW • used to copy data into BO • Contents of the buffer are likely to not be used often. - It only replaces a range of data into the existing buffer, starting from the given offset. - The total size of the buffer must be set by - All contents are written to the buffer, but not gl.bufferData() before using gl.bufferSubData(). read.
56 58
9 gl.deleteBuffer() Program Execution
•WebGL runs within the browser void gl.deleteBuffers(WebGLBuffer buffer) - complex interaction among the operating system, the window system, the browser and - You can delete a BO with gl.deleteBuffer(), if it your code (HTML and JS) is no longer needed. After a buffer object is •Simple model deleted, its contents will be lost. - Start with HTML file - files read in asynchronously - start with onload function • event driven input
Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015
59 61
Coordinate Systems Coordinate Systems and Shaders
• The units in vertices are determined by the •Vertex shader must output vertices in clip application and are called object, world, model or problem coordinates coordinates • Viewing specifications usually are also in object •Input to fragment shader from rasterizer is coordinates in window coordinates (pixels) •GL_Positions are passed to clipping volume •Application can provide vertex data in any - Most important is clip coordinates coordinate system, but vertex shader • Eventually pixels will be produced in window coordinates must eventually produce gl_Positions • WebGL also uses some internal representations in clip coordinates that usually are not visible to the application but •Simple example uses clip coordinates are important in the shaders Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015 Angel and Shreiner: Interactive Computer Graphics 7E © Addison-Wesley 2015
62 63
WebGL Camera Orthographic Viewing
•WebGL places a camera at the origin in In the default orthographic (parallel) view, all camera space pointing in the negative z points in the view volume are projected along direction the z axis onto the plane z =0. •The view/clipping volume is a box z=0 centered at the origin with sides of length 2 z=0 •(-1,-1,-1) è (1,1,1)
E. Angel and D. Shreiner: Interactive Computer Graphics 6E © Addison-Wesley 2012 64 E. Angel and D. Shreiner: Interactive Computer Graphics 6E © Addison-Wesley 2012 65
64 65
10 Viewing Rectangle Maps to WebGL View Volume Viewport •Only geometry (gl_Positions) inside of Objects in the Viewing Rectangle are mapped view volume will be rendered! into the Viewport Note the window’s coordinate frame! •Doesn’t matter if they are in front of or behind camera (origin) •The viewing/clipping volume is a box centered at the origin (canvas) (0,0) with sides of length 2 •(-1,-1,-1) è (1,1,1)
E. Angel and D. Shreiner: Interactive Computer Graphics 6E © Addison-Wesley 2012 66 E. Angel and D. Shreiner: Interactive Computer Graphics 6E © Addison-Wesley 2012 67
66 67
Viewports Transformations and Viewing
•Do not have to use the entire canvas for • In WebGL, projection is carried out by a the image: gl.viewport(x,y,w,h) projection matrix (transformation) •Values in pixels (window coordinates) • Transformation functions are also used for changes in coordinate systems and should and and are •w h x y • Pre 3.0 OpenGL had a set of transformation recommended to functions which have been deprecated be non-negative • Three choices •Specified in - Application code square.js - GLSL functions - MV.js
E. Angel and D. Shreiner: Interactive Computer Graphics 6E © Addison-Wesley 2012 68 E. Angel and D. Shreiner: Interactive Computer Graphics 6E © Addison-Wesley 2012 69
68 69
First Programming First Programming Assignment Assignment •Get example code from HW1 web •Modify square.js page - Draw a red pentagon, rather than a white square •Get test code running - Change viewport •Make minor modifications to it •Add vertex/vertices to define another •Change color in square.html triangle •Modify gl.drawArrays()
70 71
70 71
11