the gamedesigninitiative at cornell university

Lecture 10 : The Details Sizing Up Memory

Primitive Data Types Complex Data Types

— byte: basic value (8 bits) — Pointer: platform dependent — 4 bytes on 32 bit machine — char: 1 byte — 8 bytes on 64 bit machine — short: 2 bytes — Java reference is a pointer

— int: 4 bytes — Array: data size * length Not standard — long: 8 bytes May change — Strings similar (w/ trailing null) — Struct: sum of struct fields — float: 4 bytes IEEE standard Won’t change — Same rule for classes — double: 8 bytes — Structs = classes w/o methods

the gamedesigninitiative 2 Memory Details at cornell university Memory Example

class Date { short year; 2 byte byte day; 1 byte byte month; 1 bytes } 4 bytes class Student { int id; 4 bytes Date birthdate; 4 bytes Student* roommate; 4 or 8 bytes (32 or 64 bit) } 12 or 16 bytes

the gamedesigninitiative 3 Memory Details at cornell university Memory Alignment

— All data types should align

— Type starts at multiple of size class Date { — Shorts at even addresses — Ints/words at multiple of 4 short year; — Longs at multiple of 8 byte day; — Structs may require padding — Field order matters! byte month; — Pad between fields to align } — Worse on 64 bit machines — Rule: Order large to small

the gamedesigninitiative 4 Memory Details at cornell university Memory Alignment

— All data types should align Struct w/ 3 shorts and an int: — Type starts at multiple of size short short short int — Shorts at even addresses — Ints/words at multiple of 4

short int short short — Longs at multiple of 8 — Structs may require padding short short int short — Field order matters! — Pad between fields to align int short short short — Worse on 64 bit machines — Rule: Order large to small

the gamedesigninitiative 5 Memory Details at cornell university Related Topic: Cache Lines

— All CPUs have caches — A6 (iOS): 32k L1, 1Mb L2 — Snapdragon (Nexus): 4k L0, 16k L2, 2Mb L2 — Populate with cache lines — Data block of fixed size — Relative to cache size Fetch pulls in whole line Fetches the — CPU whole line — Can affect performance into cache — Accessing neighbors is fast! — Example: array scanning

the gamedesigninitiative 6 Memory Details at cornell university Data Structures and Memory

— Collection types are costly — Even null pointers use memory — Common for pointers to use as much memory as the pointees — Unbalanced trees are very bad — Even true of (pointer) arrays — Array uses additional memory — Not so in array of structs — Objects stored directly in array — But memory alignment!

the gamedesigninitiative 7 Memory Details at cornell university Data Structures and Memory

— Collection types are costly — Even null pointers use memory — Common for pointers to use as much memory as the pointees — Unbalanced trees are very bad — Even true of (pointer) arrays — Array uses additional memory — Not so in array of structs — Objects stored directly in array — But memory alignment!

the gamedesigninitiative 8 Memory Details at cornell university Two Main Concerns with Memory

— Allocating Memory — With OS support: standard allocation — Reserved memory: memory pools

— Getting rid of memory you no longer want — Doing it yourself: deallocation — Runtime support: garbage collection

the gamedesigninitiative 9 Memory Details at cornell university C/C++: Allocation Process

malloc new

— Based on memory size — Based on data type — Give it number of bytes — Give it a data type — Typecast result to assign it — If a class, calls constructor — No initialization at all — Else no default initialization — Example: — Example: char* p = (char*)malloc(4) Point* p = new Point(); Stack Heap Stack Heap sizeof

? n bytes 1

? 0 (Class) … … ? 1

the gamedesigninitiative 10 Memory Details at cornell university C/C++: Allocation Process

malloc new

— Based on memory size — Based on data type — Give it number of bytes — Give it a data type — Typecast result to assign it — If a class, calls constructor — No initialization at all — Else no default initialization Preferred in C Preferred in C++ — Example: — Example: char* p = (char*)malloc(4) Point* p = new Point(); Stack Heap Stack Heap sizeof

? n bytes 1

? 0 (Class) … … ? 1

the gamedesigninitiative 11 Memory Details at cornell university C/C++: Allocation Process

malloc new

— Based on memory size — Can emulate malloc — Give it number of bytes — Create a char (byte) array — Typecast result to assign it — Arrays not initialized — No initialization at all — Typecast after creation — Example: — Example: char* p = (char*)malloc(4) Point* p = (Point*)(new char[8]) Stack Heap Stack Heap n bytes ? ? n bytes ? ? … … ? ?

the gamedesigninitiative 12 Memory Details at cornell university Recall: Custom Allocators

Pre-allocated Array (called Object Pool)

Start Free End

— Instead of new, get object from array — Just reassign all of the fields Easy if only — Use Factory pattern for constructor one object — Delete all objects at frame end type to — Just reset free pointer to start allocate — Do not worry about freeing mid frame

the gamedesigninitiative 13 Memory Details at cornell university Anatomy of an Objective-C Constructor

NSDate* d1 = [NSDate dateWithString:"2014-03-05"];

Static Method Allocates & Initializes

the gamedesigninitiative 14 Memory Details at cornell university Anatomy of an Objective-C Constructor

NSDate* d1 = [NSDate dateWithString:"2014-03-05"];

Static Method Allocates & Initializes NSDate* d2 = [[NSDate alloc] initWithString:"2014-03-05"]];

Static Method Instance Method Allocates Initializes

the gamedesigninitiative 15 Memory Details at cornell university Custom Allocation in Objective-C

@implementation GObject

static char* mempool = malloc(sizeof(GObject)*AMOUNT); static int pointer = 0;

+ (id)alloc { if (pointer >= AMOUNT) { return Nil; } pointer += sizeof(GObject); return (id)mempool[pointer-sizeof(Gobject)]; }

the gamedesigninitiative 16 Memory Details at cornell university Custom Allocation in Objective-C

@implementation GObject

static char* mempool = malloc(sizeof(GObject)*AMOUNT); static int pointer = 0;

+ (id)alloc { if (pointer >= AMOUNT) { return Nil; } Fail gracefully pointer += sizeof(GObject); return (id)mempool[pointer-sizeof(Gobject)]; }

the gamedesigninitiative 17 Memory Details at cornell university Object Pools In Java public class GObjectFactory {

private Gobject mempool = new Gobject[AMOUNT]; private int pointer = 0;

public GObjectFactory() { for(int ii = 0; ii < AMOUNT; ii++) { mempool[ii] = new GObject(); } } Initialize Pool …

the gamedesigninitiative 18 Memory Details at cornell university Object Pools In Java public class GObjectFactory {

… public MakeGObject() { if (pointer >= AMOUNT) { return null; } GObject o = mempool[pointer++];

// Initialize object here return o; Initialization } & Allocation Combined

the gamedesigninitiative 19 Memory Details at cornell university C++: Objects vs. Allocation

Stack Based Object Heap Based Object

— Call with () after variable — Call with new syntax — Calls constructor with args — Pointer on stack — Puts object entirely on stack — Object in the heap — Deleted when stack popped — Must be manually deleted — Example: — Example: Point p(1,2); Point p* = new Point(1,2) Stack Stack Heap

1 1 2 2

the gamedesigninitiative 20 Memory Details at cornell university C++: Objects vs. Allocation

Stack Based Object Heap Based Object

— Call with () after variable — Call with new syntax — Calls constructor with args — Pointer on stack — Not in Java, C#, Obj-C — Puts object entirely on stack — Object in the heap — —But Deleted C#/Obj when-C have stack structs popped — Must be manually deleted — Classes without any methods — Example: — Example: — Can exist on the stack Point p(1,2); Point p* = new Point(1,2) Stack Stack Heap

1 1 2 2

the gamedesigninitiative 21 Memory Details at cornell university Two Main Concerns with Memory

— Allocating Memory — With OS support: standard allocation — Reserved memory: memory pools

— Getting rid of memory you no longer want — Doing it yourself: deallocation — Runtime support: garbage collection

the gamedesigninitiative 22 Memory Details at cornell university Manual Deletion in C/C++

— Depends on allocation int main() { cout << "Program started" << endl; — malloc: free int* a = new int[LENGTH]; — new: delete

— What does deletion do? delete a; — Marks memory as available for(int ii = 0; ii < LENGTH; ii++) { — Does not erase contents cout << "a[" << ii << "]=" — Does not reset pointer << a[ii] << endl; — Only crashes if pointer bad } — Pointer is currently NULL cout << "Program done" << endl; — Pointer is illegal address }

the gamedesigninitiative 23 Memory Details at cornell university Memory Leaks

— Leak: Cannot release memory — Object allocated on the heap — Only reference is moved — No way to reference object — Consumes memory fast! memoryArea = newArea; — Can even happen in Java — JNI supports native libraries — Method may allocate memory — Need another method to free — Example: dispose() in JOGL

the gamedesigninitiative 24 Memory Details at cornell university A Question of Ownership

void foo() { void foo(int key) { MyObject* o = MyObject* o = new MyObject(); table.get(key); o.doSomething(); o.doSomething(); o = null; o = null; Memory Not a return; Leak return; Leak } }

the gamedesigninitiative 25 Memory Details at cornell university A Question of Ownership

void foo() { void foo(int key) { MyObject* o = MyObject* o = table.get(key); table.get(key); table.remove(key); table.remove(key); ntable.put(key,o); o = null; Memory o = null; Leak? return; Not a return; Leak } }

the gamedesigninitiative 26 Memory Details at cornell university A Question of Ownership

Thread 1 Thread 2

“Owners” of obj

void run() { void run() { o.doSomething1(); o.doSomething2(); } }

Who deletes obj?

the gamedesigninitiative 27 Memory Details at cornell university Understanding Ownership

Function-Based Object-Based

— Object owned by a function — Owned by another object — Function allocated object — Referenced by a field — Can delete when function done — Stored in a data structure — Ownership never transferred — Allows multiple ownership — No guaranteed relationship — May pass to other functions between owning objects — But always returns to owner — Call each owner a reference — Really a stack-based object — When can we deallocate? — Active as long as allocator is — No more references — But allocated on heap (why?) — References “unimportant”

the gamedesigninitiative 28 Memory Details at cornell university Understanding Ownership

Function-Based Object-Based

— Object owned by a function — Owned by another object — Function allocated object — Referenced by a field — Can delete when function done — Stored in a data structure — Ownership never transferred — Allows multiple ownership — No guaranteed relationship — MayEasy pass: to Will other ignorefunctions between owning objects — But always returns to owner — Call each owner a reference — Really a stack-based object — When can we deallocate? — Active as long as allocator is — No more references — But allocated on heap (why?) — References “unimportant”

the gamedesigninitiative 29 Memory Details at cornell university Reference Strength

Strong Reference Weak Reference

— Reference asserts ownership — Reference != ownership — Cannot delete referred object — Object can be deleted anytime — Assign to NULL to release — Often for performance caching — Else assign to another object — Only use indirect references — Can use reference directly — Copy to local variable first — Compute on local variable — No need to copy reference — Treat like a normal object — Be prepared for NULL — Reconstruct the object? Standard type of reference — — Abort the computation?

the gamedesigninitiative 30 Memory Details at cornell university Weak Reference Example in Java public class CombatComponent { WeakReference target; Class in package java.lang.ref … public void attackTarget(AIController ai) { NPC theTarget = target.get(); if (theTarget == null) { // Be prepared for NULL theTarget = ai.pickCombatTargetFor(this); target = new WeakReference(theTarget); } // Do stuff with theTarget … } }

the gamedesigninitiative 31 Memory Details at cornell university Weak Reference Example in Java public class CombatComponent { WeakReference target; Class in package java.lang.ref … public void attackTarget(AIController ai) { Reference Managers in Java NPC theTarget = target.get(); if (theTarget == null) { // Be prepared for— NULL WeakReference theTarget = ai.pickCombatTargetFor(this); — GC if no standard references left target = new WeakReference(theTarget—); Encourages eager GC policies } — SoftReference // Do stuff with theTarget — GC possible if no references left … — But only if space is needed } }

the gamedesigninitiative 32 Memory Details at cornell university

— Every object has a counter — Tracks number of “owners” — No owners = Ref 1 Ref 2 — Increment when assign reference — Can be explicit method call

— …can we do it automatically? Object Count = 12 — Decrement when remove reference — Can be explicit or automatic — If makes count 0, delete it

the gamedesigninitiative 33 Memory Details at cornell university When to Adjust the Count?

— On object allocation public class Container { RCObject object; — Initial allocator is an owner public Container() { — Even if in a local variable // Initial allocation; ownership — When added to an object Object = new RCObject(); } — Often handled by setter … — Part of class invariant public void setObject(RCObject o) { — When removed from object if (object != null) { object.decrement(); — Also handled by the setter } — Release before reassign o.increment(); object = o; — Any other time? } }

the gamedesigninitiative 34 Memory Details at cornell university Reference Counting in Obj-C

// create a new instance Fraction *frac = [[Fraction alloc] init]; Reference count 1 [frac retain]; Reference count 2 // the values and print [frac setNumerator: 1]; [frac setDenominator: 3]; printf( "The fraction is: " ); [frac print]; printf( "\n" ); // free memory [frac release]; Reference count 1 [frac release]; frac is deleted Reference count 0

the gamedesigninitiative 35 Memory Details at cornell university Reference Counting in Classic Obj-C

// create a new instance Fraction *frac = [[Fraction alloc] init]; Reference count 1 [frac retain]; Reference count 2 // set the values and print [frac setNumerator: 1]; [frac setDenominator: 3]; printf( "The fraction is: " ); [frac print]; printf( "\n" ); // free memory [frac release]; Reference count 1 Memory Leak!

the gamedesigninitiative 36 Memory Details at cornell university Which Is Correct?

Fraction* foo(int n, int d) { Fraction* foo(int n, int d) { // create a new instance // create a new instance Fraction *frac = Fraction *frac = [[Fraction alloc] init]; [[Fraction alloc] init]; // set the values // set the values [frac setNumerator: n]; [frac setNumerator: n]; [frac setDenominator: d]; [frac setDenominator: d]; // free memory // Do nothing [frac release]; // return it // return it return frac; return frac; } }

the gamedesigninitiative 37 Memory Details at cornell university Which Is Correct?

Fraction* foo(int n, int d) { Fraction* foo(int n, int d) { // create a new instance // create a new instance Fraction *frac = Fraction *frac = [[Fraction alloc] init]; [[Fraction alloc] init]; // set the values // set the values [frac setNumerator: n]; [frac setNumerator: n]; [frac setDenominator: d]; [frac setDenominator: d]; // free memory Trick Question! // Do nothing [frac release]; // return it // return it return frac; return frac; } }

the gamedesigninitiative 38 Memory Details at cornell university Neither is Optimal

Fraction* foo(int n, int d) { Fraction* foo(int n, int d) { // create a new instance // create a new instance Fraction *frac = FractionOne *frac possibility = : [[Fraction alloc] init]; make [[Fraction ownership alloc] init]; // set the values // settransfer the values part of [frac setNumerator: n]; [frac thesetNumerator specification: n]; [frac setDenominator: d]; [frac setDenominator: d]; // free memory // Do nothing [frac release]; Reference kept. Object freed. // return it // return it Who will release Nothing left return frac; return frac; this reference? to return. } }

the gamedesigninitiative 39 Memory Details at cornell university Objective-C: An Alternate Solution

Fraction* foo(int n, int d) { Autorelease // create a new instance Fraction *frac = — Places the object in a pool [[Fraction alloc] init]; — OS releases all in the pool // set the values — At the end of handler [frac setNumerator: n]; [frac setDenominator: d]; — Games are not event driven! // free memory — Game loop runs continuously [frac autorelease]; — Pool is not released Delay release // return it until later. — You can release it manually return frac; Handled by the OS. — At end of loop iteration? }

the gamedesigninitiative 40 Memory Details at cornell university Objective-C: An Alternate Solution

Fraction* foo(int n, int d) { Autorelease // create a new instance Al Demers says: Fraction *frac = — Places theThis object is a hack. in a pool [[Fraction alloc] init]; — OS releasesBad Apple, all in thebad. pool // set the values — At the end of event handler [frac setNumerator: n]; [frac setDenominator: d]; — Games are not event driven! // free memory — Game loop runs continuously [frac autorelease]; — Pool is not released Delay release // return it until later. — You can release it manually return frac; Handled by the OS. — At end of loop iteration? }

the gamedesigninitiative 41 Memory Details at cornell university Reference Counting in iOS 5

// create a new instance Fraction *frac = [[Fraction alloc] init]; [frac retain]; No-op (does nothing) // set the values and print Automated Reference Counting [frac setNumerator: 1]; — Handled by the compiler [frac setDenominator: 3]; — Inserts retain/release for you printf( "The fraction is: " ); — Still reference counting, not GC [frac print]; printf( "\n" ); — Old methods are deprecated — Backwards compatibility only // free memory — No-ops if ARC is turned on [frac release]; No-op (does nothing)

the gamedesigninitiative 42 Memory Details at cornell university C++ Analogue: Smart Pointers

— C++ can override anything — Assignment operator = void foo(){ — Dereference operator -> // Create — Use special object as pointer smart_ptr p(); — A field to reference object // Allocate & assign it — Also a reference counter p = new MyObject(); — Assignment increments p->doSomething(); — What about decrementing? // NO LEAK — When smart pointer deleted } — Delete object if count is 0

the gamedesigninitiative 43 Memory Details at cornell university C++ Analogue: Smart Pointers

— C++ can override anything — Assignment operator = void foo(){ — Dereference operator -> // Create smart pointer smart_ptr p(); — Use special object asStack pointer object; not in heap. — A field to reference object // Allocate & assign it — Also a reference counter p = new MyObject(); — Assignment increments p->doSomething(); — What about decrementing? // NO LEAK

— When smart pointer deleted } Stack released; — Delete object if count is 0 Smart pointer is disposed.

the gamedesigninitiative 44 Memory Details at cornell university Where Does the Count Go?

Non-Intruisive Pointers Intruisive Pointers

— Count inside smart pointer — Count inside referred object — Advantage: — Advantage: — Works with any class — Easy to mix with raw pointers — Disadvantage: — Disadvantage: — Combining with raw pointers — Requires custom base object (and hence any stdlib code)

[Images courtesy of Kosmas Karadimitriou] the gamedesigninitiative 45 Memory Details at cornell university References vs. Garbage Collectors

Reference Counting Mark-and-Sweep

— Advantages — Advantages — Deallocation is immediate — No assignment overhead — Works on non-memory objects — Can handle reference cycles — Ideal for real-time systems — No specialized training to use — Disadvantages — Disadvantages — Overhead on every assignment — Collection can be expensive — Cannot easily handle cycles — Hurts performance when runs (e.g. object points to itself) — Usually triggered whenever — Requires training to use the memory is close to full

the Memory Details gamedesigninitiative 46 at cornell university Incremental Mark-and-Sweep

— Objects have multiple “colors” — Indicate where in GC process — At GC, change some colors — Free objects of certain color — Natural for game loops — Give GC a time budget — Do at end of game loop — Stop when done or time up Lua 3.0: Quad-Color M&S — See online reading for more

the gamedesigninitiative 47 Memory Details at cornell university Incremental Mark-and-Sweep

— Objects have multiple “colors” Al Demers says: — Indicate where in GC process Unless memory is always — At GC, change some colors close to full, incremental — Free objects of certain color mark and sweep is better than all other options — Natural for game loops (even manual deallocation) — Give GC a time budget — Do at end of game loop — Stop when done or time up Lua 3.0: Quad-Color M&S — See online reading for more

the gamedesigninitiative 48 Memory Details at cornell university Incremental Mark-and-Sweep

— Problem is availability — Not available in Java — Objective-C uses ARC — No good C/C++ libraries — You need to implement! — Create a singleton allocator — Has (weak) references to all objects that it allocates Lua 3.0: Quad-Color M&S — Only for complex games

the gamedesigninitiative 49 Memory Details at cornell university Summary

— Memory management a major challenge — Manual deallocation is difficult — Garbage collection is better on paper — But not all techniques easily available

— Custom allocators common in AAA games — Organize memory in pools and budget use — Could theoretically support GC as well

the Memory Details gamedesigninitiative 50 at cornell university