/INFOMOV/ Optimization & Vectorization J. Bikker - Sep-Nov 2015 - Lecture 10: “Practical”
Welcome! Today’s Agenda:
. DotCloud: profiling & high-level (1) . DotCloud: low-level and blind stupidity . DotCloud: high-level (2) . Billiards: high level . Digest INFOMOV – Lecture 10 – “Practical” 3 Practical Matters
Re-introducting DotCloud
Application breakdown:
Tick
Sort
Transform
Render
DrawScaled INFOMOV – Lecture 10 – “Practical” 4 Practical Matters
Performance Analysis & Scalability
ms per frame 256 1024 4096 16384 Transform 0.002 0.005 0.016 0.061 Sort 0.090 1.190 21.600 480.100 Render 0.650 1.420 5.130 19.681
ms per dot 256 1024 4096 16384 Transform 0.0000 0.0000 0.0000 0.0000 Sort 0.0004 0.0011 0.0053 0.0293 Render 0.0025 0.0014 0.0013 0.0012 INFOMOV – Lecture 10 – “Practical” 5 Practical Matters
Solving the Sort Problem
Current Sort: bubblesort ( 푂(푁2) ). Alternatives*:
Quicksort Shell sort Pigeonhole sort Heapsort Binary tree sort Bucket sort Mergesort Library sort Spread sort Radixsort Smoothsort Burstsort Insertionsort Strand sort Flashsort Selectionsort Cocktail sort Postman sort Monkeysort Comb sort Bread sort Countingsort Block sort Bitonic sort Introsort Odd-even sort Stooge sort
* See e.g.: http://www.sorting-algorithms.com INFOMOV – Lecture 10 – “Practical” 6 Practical Matters
Solving the Sort Problem
Current Sort: bubblesort ( 푂(푁2) ). Best case: O(N).
Which case do we have here? Factors: How much effort should we spend on this?
. Size of set . For small sets, sorting takes far less time . Already sorted / almost sorted? than rendering . Distributed (even / uneven) . Anything that is not 푂(푁2) will probably . Type of data (just key / full records) be fine. . Key type (float / int / string) . Would be nice if we can find something . … that fits well in the current code (safe time for other optimizations). INFOMOV – Lecture 10 – “Practical” 7 Practical Matters
Solving the Sort Problem
Current Sort: bubblesort ( 푂(푁2) ). Alternative: QuickSort ( 푂( 푁 log 푁 ) ).
void Swap( vec3& a, vec3& b ) { vec3 t = a; a = b; b = t; } int Pivot( vec3 a[], int first, int last ) { int p = first; vec3 e = a[first]; for( int i = first + 1; i <= last; i++ ) if (a[i].z <= e.z) Swap( a[i], a[++p] ); Swap( a[p], a[first] ); return p; } void QuickSort( vec3 a[], int first, int last) { int pivotElement; if (first >= last) return; pivotElement = Pivot( a, first, last ); QuickSort( a, first, pivotElement - 1 ); QuickSort( a, pivotElement + 1, last ); } INFOMOV – Lecture 10 – “Practical” 8 Practical Matters
Repeated Profiling
bubblesort 256 1024 4096 16384 Transform 0.002 0.005 0.016 0.061 Sort (bubble) 0.090 1.190 21.600 480.100 Sort (quick) 0.014 0.063 0.305 1.569 Render 0.650 1.420 5.130 19.681 INFOMOV – Lecture 10 – “Practical” 9 Practical Matters
Low Level Optimization of DrawScaled
void Sprite::DrawScaled( int a_X, int a_Y, int a_Width, int a_Height, Surface* a_Target ) { if ((a_Width == 0) || (a_Height == 0)) return; for ( int x = 0; x < a_Width; x++ ) for ( int y = 0; y < a_Height; y++ ) { int u = (int)((float)x * ((float)m_Width / (float)a_Width)); int v = (int)((float)y * ((float)m_Height / (float)a_Height)); Pixel color = GetBuffer()[u + v * m_Pitch]; if (color & 0xffffff) a_Target->GetBuffer()[a_X + x + ((a_Y + y) * a_Target->GetPitch())] = color; } }
Functionality: . for every pixel of the rectangular target image, . find the corresponding source pixel, . using interpolation. INFOMOV – Lecture 10 – “Practical” 10 Practical Matters
Low Level Optimization of DrawScaled
A few basic optimizations:
void Sprite::DrawScaled( int a_X, int a_Y, int a_Width, int a_Height, Surface* a_Target ) { for ( int y = 0; y < a_Height; y++ ) { int v = (int)((float)y * ((float)m_Height / (float)a_Height)); for ( int x = 0; x < a_Width; x++ ) { int u = (int)((float)x * ((float)m_Width / (float)a_Width)); Pixel color = GetBuffer()[u + v * m_Pitch]; if (color & 0xffffff) a_Target->GetBuffer()[a_X + x + ((a_Y + y) * a_Target->GetPitch())] = color; } } }
. Loop swap (to improve cache usage) . Loop hoisting (variable v is constant inside x loop) . Removed check for zero-width sprite (doesn’t happen in our case) INFOMOV – Lecture 10 – “Practical” 11 Practical Matters
Low Level Optimization of DrawScaled
More basic optimizations:
void Sprite::DrawScaled( int a_X, int a_Y, int a_Width, int a_Height, Surface* a_Target ) { float rh = (float)m_Height / (float)a_Height, rw = (float)m_Width / (float)a_Width; for ( int y = 0; y < a_Height; y++ ) { int v = (int)((float)y * rh); Pixel* line = a_Target->GetBuffer() + a_X + (a_Y + y) * a_Target->GetPitch(); for ( int x = 0; x < a_Width; x++ ) { int u = (int)((float)x * rw); Pixel color = GetBuffer()[u + v * m_Pitch]; if (color & 0xffffff) line[x] = color; } } }
. Precalculate m_Height / a_Height, m_Width / a_Width . Calculate y component of target address once per line INFOMOV – Lecture 10 – “Practical” 12 Practical Matters
Low Level Optimization of DrawScaled
Fixed point optimization:
void Sprite::DrawScaled( int a_X, int a_Y, int a_Width, int a_Height, Surface* a_Target ) { const int rh = (m_Height << 10) / a_Height, rw = (m_Width << 10) / a_Width; Pixel* line = a_Target->GetBuffer() + a_X + a_Y * a_Target->GetPitch(); for ( int y = 0; y < a_Height; y++, line += a_Target->GetPitch() ) { const int v = (y * rh) >> 10; for ( int x = 0; x < a_Width; x++ ) { const int u = (x * rw) >> 10; const Pixel color = GetBuffer()[u + v * m_Pitch]; if (color & 0xffffff) line[x] = color; } } }
. Fixed point works really well here… but doesn’t improve performance. . Incremental calculation of line address helps a bit . Seems we reached the end here… INFOMOV – Lecture 10 – “Practical” 13 Practical Matters
Low Level Optimization of DrawScaled
Now what?
. Plot multiple pixels at a time? . …
How many different ball sizes do we encounter?
…Why don’t we simply precalculate those frames? INFOMOV – Lecture 10 – “Practical” 14
“More computing sins are committed in the name of efficiency (without necessarily achieving it) than for any other single reason – including blind stupidity.” (W.A. Wulff) INFOMOV – Lecture 10 – “Practical” 15 Practical Matters
High Level Optimization of DrawScaled
Sprite* scaled[64]; void Game::Init() { ... for( int i = 0; i < 64; i++ ) { int size = i + 1; scaled[i] = new Sprite( new Surface( size, size ), 1 ); scaled[i]->GetSurface()->Clear( 0 ); m_Dot->DrawScaled( 0, 0, size, size, scaled[i]->GetSurface() ); } }
scaled[size]->Draw( (sx - size / 2), (sy - size / 2), screen ); INFOMOV – Lecture 10 – “Practical” 16 Practical Matters
Repeated Profiling
bubblesort 256 1024 4096 16384 Transform 0.002 0.005 0.016 0.061 Sort 0.014 0.063 0.305 1.569 Render (old) 0.650 1.420 5.130 19.681 Render (new) 0.350 0.720 1.977 6.383 INFOMOV – Lecture 10 – “Practical” 17 Practical Matters
Optimization of Dense Clouds
Observation: beyond a certain dot count, a large number of particles is occluded.
Specifically, we won’t be able to see the back half.
if (m_Rotated[i].z > -0.2f) scaled[size]->Draw( (sx - size / 2), (sy - size / 2), screen );
(perhaps we could also limit rendering to the outer shell of the cloud?)
Rendering is now down to 4.8ms, and sorting is slowly becoming significant again:
At 65536 dots, we get 4.7ms for sorting, 17.3ms for rendering. INFOMOV – Lecture 10 – “Practical” 18 Practical Matters
Low Level Optimization of DrawScaled
Extreme Optimization:
. We simply generate a function that plots every pixel, without the need for a loop.
FILE* f = fopen( "drawfunc.h", "w" ); fprintf( f, "void Sprite::DrawBall( int x, int y, int size, Surface* target )\n" ); fprintf( f, "{\nuint* a = target->GetBuffer() + x + y * SCRWIDTH;\nswitch( size )\n{\n" ); for( int i = 0; i < 64; i++ ) { ... fprintf( f, "case %i:\n", size ); for( int y = 0; y < size; y++) for( int x = 0; x < size; x++ ) { int a = y * SCRWIDTH + x; if (scaled[i]->GetBuffer()[x + y * size] & 0xffffff) fprintf( f, "a[%i]=%i;\n", a, scaled[i]->GetBuffer()[x + y * size] & 0xffffff ); } fprintf( f, "break;\n" ); } fprintf( f, "}\n}\n" ); INFOMOV – Lecture 10 – “Practical” 19 Practical Matters
Low Level Optimization of DrawScaled
The last optimization worked surprisingly well, yielding a final performance of:
65536 dots @ ~7ms (render time only). Sorting is now definitely significant. INFOMOV – Lecture 10 – “Practical” 20 Practical Matters
Sorting in O(1)
For this specific situation, we can sort in O(1), e.g., independent of particle count.
Observation: dots do not move independently.
Intuition: why rotate 64k dots if you can rotate a single camera? INFOMOV – Lecture 10 – “Practical” 21 Practical Matters
Sorting in O(1) INFOMOV – Lecture 10 – “Practical” 22 Practical Matters
Sorting in O(1) INFOMOV – Lecture 10 – “Practical” 23 Practical Matters
Sorting in O(1) INFOMOV – Lecture 10 – “Practical” 24 Practical Matters
Sorting in O(1) INFOMOV – Lecture 10 – “Practical” 25 Practical Matters
Sorting in O(1) INFOMOV – Lecture 10 – “Practical” 26 Practical Matters
Sorting in O(1)
For each split:
. Process nearest half first . Then farthest half . Recurse
Where ‘nearest’ is the side that the ‘camera’ is on. Today’s Agenda:
. DotCloud: profiling & high-level (1) . DotCloud: low-level and blind stupidity . DotCloud: high-level (2) . Billiards: high level . Digest INFOMOV – Lecture 10 – “Practical” 28 Practical Matters
Introducting Billiards
Application breakdown:
Game::Tick
For each ball: . Draw ball . Update position . Bounce off of boundaries . For each other ball: . Bounce off of peer
Clearly, we have 푂(푁2) behavior here. How do we fix this efficiently? INFOMOV – Lecture 10 – “Practical” 29 Practical Matters
Profiling
Adding a timer:
timer t; t.reset();
Visualizing timing result:
screen->Bar( 10, 10, 170, 20, 0 ); char r[128]; sprintf( r, "simulation time: %5.2f ms", t.elapsed() ); screen->Print( r, 12, 11, 0xffffff );
Apparently, we can check 2k balls against 2k balls in 4.5ms. INFOMOV – Lecture 10 – “Practical” 30 Practical Matters
Low level optimization
Lots of square roots in the main loop:
. Length . Normalize (that’s actually the same square root!) . Another length (not being used, cost?) . Normalize (this one is for accuracy reasons) . Normalize (also for accuracy reasons)
Improvements:
. Use 푑2, only calculate square root if we already know balls are too close. . Do the ‘accuracy fix’ normalizations only if accuracy becomes problematic (e.g., squared length > 1 + epsilon). INFOMOV – Lecture 10 – “Practical” 31 Practical Matters
High Level Optimization
Each ball checks every other ball (with a larger index, so in fact only half of the other balls on average).
We need an efficient way to find nearby balls.
. Quadtree? . Grid? Steps:
Grid it is: we will check the balls in the cell our ball is 1. Per frame, update the grid in, plus the direct neighbors. 2. Per ball, determine relevant cells 3. Loop over cells 4. Loop over balls in cell INFOMOV – Lecture 10 – “Practical” 32 Practical Matters
High Level Optimization
Attempt 1: std::vector
// allocate grid std::vector
// fill the grid for( int y = 0; y < SCRHEIGHT / 16; y++ ) for( int x = 0; x < SCRWIDTH / 16; x++ ) grid[y][x].clear(); for( int i = 0; i < BALLCOUNT; i++ ) { int gx = CLAMP( (int)(pos[i].x / 16 ), 0, SCRWIDTH / 16 - 1 ); int gy = CLAMP( (int)(pos[i].y / 16 ), 0, SCRHEIGHT / 16 - 1 ); grid[gy][gx].push_back( i ); } INFOMOV – Lecture 10 – “Practical” 33 Practical Matters
High Level Optimization
Attempt 1: std::vector
// using the grid int gx = CLAMP( (int)(pos[i].x / 16 ), 0, SCRWIDTH / 16 - 1 ); int gy = CLAMP( (int)(pos[i].y / 16 ), 0, SCRHEIGHT / 16 - 1 ); int gx1 = MAX( 0, gx - 1 ), gx2 = MIN( SCRWIDTH / 16 - 1, gx + 1 ); int gy1 = MAX( 0, gy - 1 ), gy2 = MIN( SCRHEIGHT / 16 - 1, gy + 1 ); for( int y = gy1; y <= gy2; y++ ) for( int x = gx1; x <= gx2; x++ ) for( int k = 0; k < grid[y][x].size(); k++ ) { int j = grid[y][x][k]; if (j <= i) continue; INFOMOV – Lecture 10 – “Practical” 34 Practical Matters
High Level Optimization
Attempt 1: std::vector
Result: simulation time went down from 4.5ms to 1.17ms (although performance is very unstable). INFOMOV – Lecture 10 – “Practical” 35 Practical Matters
High Level Optimization
Attempt 2: regular arrays
Assumption: in a 16x16 grid cell, we will never have more than 8 balls.
// allocate grid int grid[SCRHEIGHT / 16][SCRWIDTH / 16][8]; // allocate array for storing grid cell ball count int nr[SCRHEIGHT / 16][SCRWIDTH / 16]
// fill the grid memset( nr, 0, SCRHEIGHT / 16 * SCRWIDTH / 16 * 4 ); for( int i = 0; i < BALLCOUNT; i++ ) { int gx = CLAMP( (int)(pos[i].x / 16 ), 0, SCRWIDTH / 16 - 1 ); int gy = CLAMP( (int)(pos[i].y / 16 ), 0, SCRHEIGHT / 16 - 1 ); grid[gy][gx][nr[gy][gx]++] = i; } INFOMOV – Lecture 10 – “Practical” 36 Practical Matters
High Level Optimization
Attempt 2: regular arrays
// using the grid int gx = CLAMP( (int)(pos[i].x / 16 ), 0, SCRWIDTH / 16 - 1 ); int gy = CLAMP( (int)(pos[i].y / 16 ), 0, SCRHEIGHT / 16 - 1 ); int gx1 = MAX( 0, gx - 1 ), gx2 = MIN( SCRWIDTH / 16 - 1, gx + 1 ); int gy1 = MAX( 0, gy - 1 ), gy2 = MIN( SCRHEIGHT / 16 - 1, gy + 1 ); for( int y = gy1; y <= gy2; y++ ) for( int x = gx1; x <= gx2; x++ ) for( int k = 0; k < nr[y][x]; k++ ) { int j = grid[y][x][k]; if (j <= i) continue; INFOMOV – Lecture 10 – “Practical” 37 Practical Matters
High Level Optimization
Attempt 2: regular arrays
Result: simulation time went down from 1.17ms to 1.07ms (but, still quite unstable).
…Why is it unstable?
After fixing this, we are now at 0.45ms, A 10x improvement. INFOMOV – Lecture 10 – “Practical” 38 Practical Matters
Data Locality
The access pattern is currently as follows:
For every ball For every cell in the neighborhood Check every ball in the cell
Each individual ball will be in a random cell. Drawing the balls will access the screen randomly.
We can improve locality by looping over cells in the outer loop. INFOMOV – Lecture 10 – “Practical” 39 Practical Matters
Currently, our bottlenecks are: More Profiling . Clear . Sprite::Draw . vec2::length
Also note that the three for loops take quite a bit of time. Today’s Agenda:
. DotCloud: profiling & high-level (1) . DotCloud: low-level and blind stupidity . DotCloud: high-level (2) . Billiards: high level . Digest INFOMOV – Lecture 10 – “Practical” 41 Digest
High Level Optimization
High level optimization: 1. reducing algorithmic complexity of bottlenecks; 2. exchanging inefficient algorithms
High level optimization almost always yields the biggest gains in performance.
Typical approach:
. Divide and conquer: spatial subdivision / object subdivision . Preventing work for a group of input elements . Use your intuition: does this for loop really need every iteration? INFOMOV – Lecture 10 – “Practical” 42 Digest
Low Level Optimization
Use that list of common opportunities! . Expensive operations . Powers of two enable bitshifts, masks (for cheap modulos), … . Lookup tables . Branching is evil . Late in / early out . Work around excessive type conversion
And: . Mind memory access patterns: strive for linear . Watch for static expressions inside a loop /INFOMOV/
END of “Practical” next lecture: “Presentations”