Virtual Address Space Physical Memory Allocation

4/12/2017

5A. Memory Management and Address Spaces 1. allocate/assign physical memory to processes 5B. Simple Memory Allocation – explicit requests: malloc (sbrk) 5C. Dynamic Allocation Algorithms – implicit: program loading, stack extension 5D. Advanced Allocation Techniques 2. manage the virtual address space 5G. Errors and Diagnostic Free Lists – instantiate virtual address space on context switch – extend or reduce it on demand 5F. Garbage Collection 3. manage migration to/from secondary storage – optimize use of main storage – minimize overhead (waste, migrations)

Memory Management 1 Memory Management 2

Memory Management Goals Linux Process – virtual address space 1. transparency

– process sees only its own virtual address space 0x00000000 0x0100000 0x0110000

– process is unaware memory is being shared shared code private data DLL 1 DLL 2 2. efficiency – high effective memory utilization DLL 3 private stack

– low run-time cost for allocation/relocation 0x0120000 0xFFFFFFFF 3. protection and isolation All of these segments appear to be present in memory – private data will not be corrupted whenever the process runs. – private data cannot be seen by other processes

Memory Management 3 Memory Management 4

Physical Memory Allocation (code segments) • program code process 1 shared lib data and stack X – allocated when program loaded OS kernel shared code process 3 – initialized with contents of load module segment A data and stack

process 2 • shared and Dynamically Loadable Libraries data and stack – mapped in at exec time or when needed shared code segment B • all are read-only and fixed size – somehow shared by multiple processes Physical memory is divided between the OS kernel, process private data, and shared code segments. – shared code must be read only

Memory Management 5 Memory Management 6

1 4/12/2017

(implementing: code segments) (data/stack segments)

• program loader • they are process-private, read/write – ask for memory (size and virtual location) • initialized data – copy code from load module into memory – allocated when program loaded • run-time loader – initialized from load module – request DLL be mapped (location and size) • data segment expansion/contraction – edit PLT pointers from program to DLL – requested via system calls (e.g. sbrk) • memory manager – only added/truncated part is affected – allocates memory, maps into process • process stack – allocated and grown automatically on demand

Memory Management 7 Memory Management 8

(implementing: data/stack) Fixed Partition Memory Allocation

• program loader • pre-allocate partitions for n processes – ask for memory (location and size) – reserving space for largest possible process – copy data from load module into memory • very easy to implement – common in old batch processing systems – zero the uninitialized data • well suited to well-known job mix memory manager • – must reconfigure system for larger – invoked for allocations and stack extensions processes – allocates and deallocates memory • likely to use memory inefficiently – adjusts process address space accordingly – large internal fragmentation losses – swapping results in convoys on partitions

Memory Management 9 Memory Management 10

Fragmentation Internal Fragmentation partn #1 partn #2 partn #3 • The curse of spatially partitioned resources 8MB 4MB 4MB – wasted or unusable free space • internal fragmentation – not needed by its owner waste 2MB • external fragmentation – uselessly small pieces – all such resources, not merely memory process • inheritance of farm land A waste 1MB waste 2MB • calendar appointment packing (6 MB) process process B C • Choose your poison (3 MB) (2 MB) – static allocation … constant internal waste – dynamic allocation … progressive external loss Total waste = 2MB + 1MB + 2MB = 5/16MB = 31%

Memory Management 11 Memory Management 12

2 4/12/2017

(Internal Fragmentation) Variable Partition Allocation

• wasted space in fixed sized blocks • start with one large "heap" of memory • caused by a mis-match between • when a process requests more memory – the chosen sizes of a fixed-sized blocks – find a large enough chunk of memory – the actual sizes that programs request – carve off a piece of the requested size • average waste: 50% of each block – put the remainder back on the free list • overall waste reduced by multiple sizes • when a process frees memory – suppose blocks come in sizes S1 and S2 – put it back on the free list – average waste = ((S1/2) + (S2 - S1)/2)/2 • eliminates internal fragmentation losses

Memory Management 13 Memory Management 14

External Fragmentation (External/Global Fragmentation) each allocation creates left-over fragments P • P P F A A – over time these become smaller and smaller P P PB D D • tiny left-over fragments are useless – they are too small to satisfy any request PC PC PC – but their combined size may be significant

PE PE • there are three obvious approaches: – try to avoid creating tiny fragments – try to recombine adjacent fragments – re-pack the allocated space more densely

Memory Management 15 Memory Management 16

Fixed vs Variable Partition Stack vs Heap Allocation

• Fixed partition allocation • stack allocation – allocation and free lists are trivial – compiler manages space (locals, call info) – internal fragmentation is inevitable – data is valid until stack frame is popped • average 50% (unless we have multiple sizes) – OS automatically extends/shrinks stack segment • Variable partition allocation • heap allocation – allocation is complex and expensive – explicitly allocated by application (malloc/new) • long searches of complex free lists – data is valid until free/delete (or G.C.) – eliminates internal fragmentation – heap space managed by user-mode library – external fragmentation is inevitable data segment size adjusted by system call • can be managed by (complex) coalescing –

Memory Management 17 Memory Management 18

3 4/12/2017

sbrk(2) vs. malloc(3) managing process private data

• sbrk(2) … managing size of data segment initialized 000000000000 data 000000000000 – each address space has a private data segment – process can request that it be grown/shrunk – sbrk(2) specifies desired ending address 1. loader allocates space for, and copies initialized data from load module. – this is a coarse and expensive operation • malloc(3) … dynamic heap allocation 2. loader allocates space for, and zeroes uninitialized data from load module. 3. after it starts, program uses sbrk to extend the process data segment – sbrk(2) is called to extend/shrink the heap and then puts the newly created chunk on the free list – malloc(3) is called to carve off small pieces 4. Free space “heap” is eventually consumed – mfree(3) is called to return them to the heap program uses sbrk to further extend the process data segment and then puts the newly created chunk on the free list

Memory Management 19 Memory Management 20

Variable Partition Free List (Free lists: keeping track of it all)

F N L • fixed sized blocks are easy to track R E E head E X N E T – a bit map indicating which blocks are free U N L S E E E X N D T • variable chunks require more information List might F N L contain all R E – a linked list of descriptors, one per chunk E E X N memory E T fragments – each lists size of chunk, whether it is free U N L S E E E X N each has pointer to next chunk on list …or only D T – B1 fragments F N descriptors often at front of each chunk L – R E that are free. E E X N E T … • allocated memory may have descriptors too

Memory Management 21 Memory Management 22

Free Chunk Carving Avoid Creating Small Fragments

U N L S E • Choose carefully which piece to carve E E X N 1. find large enough free chunk D T • “There Ain’t No Such Thing As A Free Lunch”

U S – careful choices mean longer searches E 2. reduce len to requested size D

F N F N – optimization means more complex data structures L L R E R E E E E X E X N N 3. new header for residual chunk E T E T – cost of reduced fragmentation is more cycles • A few obvious choices for “smart” choices … 4. insert new chunk into list – Best fit U N L S E E E X N 5. mark carved piece as in use D T – Worst fit … – First fit – Next fit

Memory Management… 23 Memory Management 24

4 4/12/2017

Which chunk: best fit Which chunk: worst fit

• search for the "best fit" chunk • search for the "worst fit" chunk – smallest size greater/equal to requested size – largest size greater/equal to requested size • advantages: • advantages: – might find a perfect fit – tends to create very large fragments • disadvantages: … for a while at least – have to search entire list every time • disadvantages: – quickly creates very small fragments – still have to search entire list every time

Memory Management 25 Memory Management 26

Which chunk: first fit Which Chunk: next Fit

F N • take first chunk that is big enough L R E E head E X N • advantages: E T U N After each L S E E E X guess N – very short searches search, set D T guess pointer pointer F N L – creates random sized fragments to chunk after R E E E X N the one we E T • disadvantages: chose. U N L S E E E X N – the first chunks quickly fragment That is the D T

point at which F N L – searches become longer R E E we will begin E X N – ultimately it fragments as badly as best fit our next E T search. …

Memory Management 27 Memory Management 28

(next-fit ... guess pointers) Coalescing – de-fragmentation

• the best of both worlds • all dynamic algorithms have ext fragmentation – short searches (maybe shorter than first fit) – some get it faster, some spread it out more evenly – spreads out fragmentation (like worst fit) • we need a way to reassemble fragments • guess pointers are a general technique – check neighbors when ever a chunk is freed – think of them as a lazy (non-coherent) cache – recombine w/free neighbors whenever possible – if they are right, they save a lot of time – free list can be designed to make this easier – if they are wrong, the algorithm still works • e.g. address-sorted doubly linked list of all chunks – they can be used in a wide range of problems • e.g. “Buddy” allocation w/implicit neighbors • counters forces of external fragmentation

Memory Management 29 Memory Management 30

5 4/12/2017

Free Chunk Coalescing Coalescing vs. Fragmentation

F N • opposing processes operate in parallel L R E E head E X N E T – which of the two processes will dominate?

U N L S E • what fraction of space is typically allocated? E E X N D T Previous – coalescing works better with more free space F N L chunk is free, R E E E X how fast is allocated memory turned over? N • so coalesce E T backwards. – chunks held for long time cannot be coalesced UF N L SR E E E X N FREE DE T • how variable are requested chunk sizes?

F N L – high variability increases fragmentation rate R E E E X Next chunk is N E T also free, so • how long will the program execute … coalesce forwards. – fragmentation, like rust, gets worse with time

Memory Management 31 Memory Management 32

Memory Allocation Requests (memory allocation requests)

frequency Internal fragmentation results from • memory requests are not well distributed mismatches between chunk sizes and request sizes (which we have assumed to – some sizes are used much more than others be randomly distributed) • many key services use fixed-size buffers

But if we look at what actually – file systems (for disk I/O) happens, it turns out that – network protocols (for packet assembly) memory allocation requests aren’t random at all. – standard request descriptors • these account for much transient use – they are continuously allocated and freed 64 256 1K 4K

Memory Management 33 Memory Management 34

Special Buffer Pools Balancing Space for Buffer Pools • if there are popular sizes • many different special purpose pools – reserve special pools of particular size buffers – demand for each changes continuously – allocate/free matching requests from those pools – memory needs to migrate between them • benefit: improved efficiency • sounds like dynamic a equilibrium – much simpler than variable partition allocation – managed free space margins – reduces (or eliminates) external fragmentation • maximum allowable free space per service • but ... we must know how much to reserve • graceful handling of changing loads – too little: buffer pool will become a bottleneck – claw-back call-back – too much: we will have a lot of idle space • OS requests services to free all available memory • prompt handling of emergencies

Memory Management 35 Memory Management 36

6 4/12/2017

Buffer Pools – Slab Allocation Common Dynamic Memory Errors

• requests are not merely for common sizes • Memory Leaks – they are often for the same data structure – neglect to free memory after done with it – or even assemblies of data structures – often happens in (not thought out) error cases • initializing and demolition are expensive • Buffer over-run – writing past beginning or end of allocated chunk – many fields and much structure are constant – usually result of not checking parameters/limits stop destroying and reinitializing • • Continuing to use it after freeing it – recycle data structures (or assemblies) – may be result of a race condition – only reinitialize the fields that must be changed – may be result of returning pointers to locals – only disassemble to give up the space – may simply be “poor house-keeping”

Memory Management 37 Memory Management 38

Diagnostic Free List (Diagnostic Free lists can help)

A G G A F N L U U L • all chunks in list (whether allocated or free) R L E L A A L E E X O R R O E N T C D D C free space G enables us to find state of ALL memory chunks B T P T Z Z – R I H T I O O T T R M N N N E E E • record of who last allocated each chunk • standard chunk header – what routine called malloc, when – free bit, chunk length, next chunk pointer • allocation audit info – enables to identify type of data structure – tracing down the source of memory leaks E1 • guard zones • guard zones at beginning and end of chunks – detect application buffer over-runs E • zero memory when it is freed 2 – limited protection against buffer under/overrun – detect continued use after chunk is freed – enables us to detect data under/overrun E3

Memory Management 39 Memory Management 40

Memory Leaks Garbage Collection: “A New Hope” • a very common problem • garbage collection is alternative to freeing – programs often forget to free heap memory • this is why the automatic stack model is so attractive – applications allocate objects, never free them – losses can be significant in long running processes • user-mode memory manager monitors space • process grows, consuming ever-more resources when it gets low, initiate garbage collection • degrading system and application performance – • the operating system cannot help • Garbage Collection: – the heap is managed entirely by user-mode code – search data space finding every object pointer – finding and fixing the leaks is too difficult for most – note address/size of all accessible objects some advocate regular “prophylactic restarts” – – compute the compliment (what is inaccessible) – add all inaccessible memory to the free list

Memory Management 41 Memory Management 42

7 4/12/2017

Garbage Collection: TANSTAAFL Finding all accessible data

• Garbage Collection is expensive • object oriented languages often enable this – scan all possible object references – all object references are tagged – compute an active reference count for each – all object descriptors include size information – may require stopping application for a long time • it is often possible for system resources • Progressive Background Garbage Collection – where all resources and references are known – runs continuously, in parallel with application (e.g. we know who has which files open) – continuous overhead and competing data access • The more you need it, the more it costs • resources can be designed with GC in mind – more frequent garbage collection scans – but, in the general case, it may be impossible – yielding less free memory per scan

Memory Management 43 Memory Management 44

General Case GC: What’s so hard? GC vs. Reference Counting • Compiler can know static/automatic pointers • What if there are multiple pointers to object? • How do we identify pointers in the heap? – when can we safely delete it – search data segment for address-like values? • Associate a reference count w/each object a number or string could look like an address – – increment count on each reference creation • How do we know if pointers are still live? – decrement count on each reference deletion – a value doesn’t mean the code is still using it – delete object when reference count hits zero • What kind of structure does it point to? – we need to know how much memory to free • This is not the same as Garbage Collection • Only possible if all data/pointers are tagged – it requires explicit close/release operations – which is, in general, not the case – doesn’t involve searching for unreferenced objs

Memory Management 45 Memory Management 46

Assignments • Projects – start project 1B … involves server, encryption • Reading (73pp) – AD 15 (relocation) – AD 16 (segmentation) – AD 18 (paging) – AD 19 (TLBs … analogous to demand paging) – AD 21 (swapping) – AD 22 (swapping policy) – Working Set Replacment

Memory Management 47