Operating Systems (2INC0) 2019/20

Virtual Memory (10)

Dr. Tanir Ozcelebi Courtesy of Dr. I. Radovanovic, Dr. R. Mak

System Architecture and Networking Group Agenda

• Recap memory management in early systems • Principles of virtual memory • Paging • Segmentation • Paging and Segmentation

Tanir Ozcelebi 10-Dec-19 2 Agenda

• Recap memory management in early systems • Principles of virtual memory • Paging • Segmentation • Paging and Segmentation

Tanir Ozcelebi 10-Dec-19 3 Memory management: requirements

IDEALLY, memory needs to be

• Simple to use • Private (isolation) • Non-volatile / permanent (data remains in memory) • Fast (zero-time) access • Huge (unlimited) capacity • Cheap (cost-effective)

Conflicting requirements! We make use of memory hierarchy and virtualization to find a good tradeoff.

Tanir Ozcelebi 10-Dec-19 4 Memory Hierarchy

CPU Registers

L1 Cache Memory

L2 Cache \Memory

Primary “Main” Memory (Executable)

Solid State Memory

Rotating Magnetic Memory Larger storage Larger Faster access Faster

Secondary Optical Memory

Sequentially Accessed Memory

Tanir Ozcelebi 10-Dec-19 5 Early systems (properties)

• Every active process resides in its entirety in MM • Large programs can’t execute (except with an overlay structure) − Overlay: Structure a program in independent parts (e.g. function calls) which can be overlaid in memory

B C versus

D E

• Every active process is allocated a contiguous part of main memory. • Three partitioning schemes • fixed, dynamic, relocatable dynamic partitions

Tanir Ozcelebi 10-Dec-19 6 Early systems (partitioning schemes)

• Fixed partitions • limits the number and maximal size of the active processes • internal fragmentation

• Dynamic partitions • external fragmentation

• Relocatable dynamic partitions • requires dynamic binding and relocatable load modules • no fragmentation, but expensive compaction (or swapping)

Tanir Ozcelebi 10-Dec-19 7 Recap: Do we have the ideal memory properties so far?

With the exception Simple Yes of overlays

Isolation can be provided with dynamic Private Yes/No binding. No sharing

Unless the programmer enforces Permanent No this during execution.

Process-based compaction and swapping Fast No are expensive

Process size cannot exceed main Huge No memory size

Cost-effective Yes Hardware hierarchy

Tanir Ozcelebi 10-Dec-19 8 Agenda

• Recap memory management in early systems • Principles of virtual memory • Paging • Segmentation • Paging and Segmentation

Tanir Ozcelebi 10-Dec-19 9 VM: abstraction

What needs to be done such that • programs do not have to be stored contiguously, and • not the entire program, but only parts are stored in main memory?

Approach: • Split the memory into segments and try to fit parts of the program into those segments. • Terminology: • If segments are of different sizes: segmentation • If segments are of the same size: paging − Physical memory blocks: (page) frames − Logical memory blocks: pages • Paging and segmentation can be combined

Tanir Ozcelebi 10-Dec-19 10 Paging memory allocation

• Processes divided into pages (memory blocks) of equal size

• provides non-contiguous memory allocation (size: 350 lines) • works well if page size is equal to page frame size − and the disk’s section size (sectors)

• Advantages: • An empty page frame is always usable by any process • The compaction scheme is not required − No external and almost no internal fragmentation • Disadvantage: • Mechanism needed to keep track of page locations

Tanir Ozcelebi 10-Dec-19 11 Non-contiguous allocation -an example-

Page Process 1 Main Memory frame # P1 size=350 lines 1st 100 lines Operating system 0 1 page size = 100 lines Page 0 2 3 OS size = 300 lines 2nd 100 lines 4 Page 1 Process 1 – Page 2 5 6 7 3rd 100 lines Process 1 – Page 0 8 Page 2 9 Process 1 – Page 1 10 Remaining 50 lines Page 3 Process 1 – Page 3 11 Internal fragmentation Wasted space 12

• The number of free page frames left (13-3-4) = 6 • Any process of more than 600 lines has to wait until Proc1 ends. • Any process of more than 1000 lines cannot fit into memory at all!

• Problem remains (needs solving): • Entire process must be stored in memory during its execution

Tanir Ozcelebi 10-Dec-19 12 VM: Demand paging

• Bring a page into memory only when it is needed

• Takes advantage of the fact that programs are written sequentially (not all pages are necessary at once). For example: • User-written error handling modules • Mutually exclusive modules • Some parts are not always accessible • Only a fraction of table is actually used

• No restriction of having the entire process stored in memory • Gives appearance of an infinite physical memory

Tanir Ozcelebi 10-Dec-19 13 VM: implementation issues

• Address binding / translation • logical addresses translated to physical ones at run time • HW support to reduce access time & to enforce isolation • Placement strategies • simple for pages: any free page frame will do. • for segments, strategies are similar to those for dynamic partitions • Replacement strategies • that decide which page(s) or segment(s) must be swapped out in case there is not enough free space in MM • Load control policies that determine • how many pages of a process are resident in MM? • when to load pages into MM (demand paging, pre-paging)? • Sharing • possible with both paging and segmentation

Tanir Ozcelebi 10-Dec-19 14 Agenda

• Recap memory management in early systems • Principles of virtual memory • Paging • address translation − frame table, page table, TLBs • replacement strategies • load policies • Segmentation • Segmentation with paging

Tanir Ozcelebi 10-Dec-19 15 Frame table vs Page table

Process Process Page ID page Frame number Number 0 3 9 0 2 1 2 11 1 7 2 1 0 2 5 3 1 3 3 3 Process Page Index 4 3 8 Say the CPU wants to load an 5 1 2

Page Frame Index instruction of a certain process (at 6 3 12 a certain page p and an offset w). 7 1 1 How to find physical address (pa)?

Tanir Ozcelebi 10-Dec-19 16 Virtual vs physical address

|p| bits |w| bits Virtual Address (for a process id) va Page number p w 2|p| pages |w| w: offset in the corresponding page (# words) 2 words/page VM size: 2|p|+|w| words

Goal: address a word Physical Address |f | bits |w| bits 2|f| page frames pa Frame number f w 2|w| words/page (or /frame) PM size: 2|f|+|w| words

Tanir Ozcelebi 10-Dec-19 17 How to find physical address using the frame table?

address_map (id, p, w) { Virtual address: id p w · . pa = UNDEFINED; . for (f = 0; f

Page Frame Index Frame Page 4 3 8

Tanir Ozcelebi 10-Dec-19 18 How to find physical address using page tables?

Virtual address: id p w address_map (p, w) { pa = *(PTR+p) || w; return pa; . . . . }

PTR frame f

page p p w A Page Table Register (PTR) is Recall example used for fast access (hardware f support). Its content is stored in Page the process table. Frame Number Reading/writing from/to memory 0 2 requires two memory accesses: 1 7 · · One for accessing the page table, . . and one for accessing actual data. 2 5 . . 3 3 Proc. Page Proc.Index Page

Tanir Ozcelebi 10-Dec-19 19 Process table

• …maintained by OS for context switching and scheduling, etc. • Entries (rows) are process control info: name, size, state, priority, registers, a semaphore waited on, page table location etc.

Process Table (a) Process Table (b) Process Table (c) Process Size Page Table Process Size Page Table Process Size Page Table Location Location Location P1 400 3096 400 3096 400 3096 P2 200 3100 700 3100 500 3150 500 3150 500 3150 P3 (a) PT has 3 entries initially; one for each process (b) second process ends, entry in table is released and replaced by

Tanir Ozcelebi 10-Dec-19 20 Translation Look-aside Buffer

Problems with page tables • They can be huge. • It requires 2 memory accesses per reference.

Approach • use a TLB (a cache memory) • keeps track of locations of the most recently used pages in MM • does not contain actual data or instructions • most common replacement scheme: least recently used (LRU) • if the page location cannot be found in the TLB, the page table is used • if the page location still cannot be found a page fault is generated • meaning the page is currently not found in the physical memory.

Tanir Ozcelebi 10-Dec-19 21 Page faults

• Page faults happen when a part of program needs to be brought into MM

• Upon page fault: Page fault handler determines whether there are empty frames in MM • If not, it must decide which page to swap out • This depends on predefined policy for page removal

• A single memory reference may create several page faults • e.g. when the page table itself is not in MM

• Tables to be updated after swapping: • Page tables of two tasks (1 in 1 out) and table to locate free frames

• Problem with swapping: Thrashing

Tanir Ozcelebi 10-Dec-19 22 Thrashing

• A process may spend more time paging than executing

• When does it happen? • Pages in active use are replaced by other pages in active use • Happens with increased degree of multiprogramming

• Solution: • Provide the process with as many frames in MM as it needs − needs a replacement strategy

Tanir Ozcelebi 10-Dec-19 23 Thrashing -an example-

for (j=1; j<100; ++j) { Page 0 k:=j*j; Swapping between these two pages m:=a*j; printf(“\n%d %d %d”, j, k, m);

} Page 1 printf(“\n”);

Tanir Ozcelebi 10-Dec-19 24 Agenda

Tanir Ozcelebi 10-Dec-19 25 Replacement strategies

• Comparison of replacement strategies is done using reference strings • an execution trace in which only memory references are recorded • only the page number of the referenced location is mentioned − Reference string: r0r1r2…

• Goodness criteria • the number of generated page faults • the total number of pages loaded due to page faults (these two are equal under pure demand paging)

Global strategies • fixed number of page frames shared by all processes • evicted page need not be owned by the process that needs extra memory

Local strategies • each process has a set of pages called the working set • when a process runs out of memory a page from its working set is evicted

Tanir Ozcelebi 10-Dec-19 26 Global replacement strategies

• MIN replacement (looks to the future) • select the page which will not be used for the longest time in the future, this gives the minimum number of page faults • Random replacement • select a random page for replacement • FIFO replacement • select the page that has been resident in MM for the longest time • LRU replacement • select the page that is least recently used • Clock replacement (second chance) • circular list of all resident pages equipped with a use-bit u • upon each reference u is set to 1 (set to 2 for third chance) • search clockwise for u=0, while setting the use-bits to zero

Tanir Ozcelebi 10-Dec-19 27 MIN policy (looks to the future)

Page requested: A B A C A B D B A C D

Page Frame 1

Page A A A A A A D D D D D

Page Frame 2

(empty) B B C C B B B A C C

Interrupt: * * * * * * * Time: 1 2 3 4 5 6 7 8 9 10 11

• How each page requested is swapped into the 2 available page frames using MIN. • When program is ready to be processed all 4 pages are on secondary storage. • Throughout program: 11 page requests are issued. When program calls a page that isn’t already in memory, a page fault is issued (shown by *). à 7 page faults

Tanir Ozcelebi 10-Dec-19 28 FIFO policy

Page Requested: A B A C A B D B A C D

Page Frame 1

Page A A A C C B B B A A D

Page Frame 2

(empty) B B B A A D D D C C

Interrupt: * * * * * * * * * Time: 1 2 3 4 5 6 7 8 9 10 11

• à 9 page faults

Tanir Ozcelebi 10-Dec-19 29 LRU policy

Page Requested: A B A C A B D B A C D

Page Frame 1

Page A A A A A A D D A A D

Page Frame 2

(empty) B B C C B B B B C C

Interrupt: * * * * * * * * Time: 1 2 3 4 5 6 7 8 9 10 11 • Only 8 page faults • Efficiency slightly better than FIFO • The most widely used static replacement algorithm

Tanir Ozcelebi 10-Dec-19 30 Page Table Extensions Extra fields

Page Status bit Referenced bit Modified bit Page frame 0 1 1 1 5 1 1 0 0 9 2 1 0 0 7 3 1 1 0 12

• Status bit indicates whether page is currently in memory or not

• Referenced bit (use bit) indicates whether page has been referenced recently • Used by LRU to determine which pages should be swapped out

• Modified bit (dirty bit) indicates whether page contents have been altered • Used to determine if page must be rewritten to secondary storage when it is swapped out

• There may be more of these bits for other purposes, e.g., locking

Tanir Ozcelebi 10-Dec-19 31 Local Replacement strategies

• VMIN replacement • looks to the future • at each memory reference − if there is page fault, then the requested page is loaded immediately − if a page is not referenced during the next t memory references, it is removed

• Working Set (WS) model • uses recent history (past t memory references) • at each memory reference a WS is determined − only the pages that belong to the WS reside in MM − a process can run if its entire WS is in MM

• the working set is given by W (t, t) = { rj | t-t < j ≤ t }

− Reference string: r0r1r2…rT • hardware support in the form of aging registers

Tanir Ozcelebi 10-Dec-19 32 Working-set model (example)

• A set of active pages large enough to avoid thrashing • Use a parameter t to determine a working-set window

Page reference window ….2 6 1 5 7 7 7 7 5 1 6 2 3 4 1 2 3 4 4 4 3 4 3 4 4 4 1 3 2 3 4 4 4…. t = 9 t = 9

WS (t1)={1,2,5,6,7} WS (t2)={3,4}

D = åWSSi

D: the total demand for frames in memory

WSSi: the working set size for process i Thrashing will occur when WS’s are small in size and/or when D is greater than the total number of available frames!

Tanir Ozcelebi 10-Dec-19 33 Working set –an example-

-2 -1 Time t 0 1 2 3 4 5 6 7 8 9 10

e d Reference a c c d b c e c e a d string

Page a √ √ √ √ ------√ √

Page b ------√ √ √ √ ------

Page c -- √ √ √ √ √ √ √ √ √ √

Page d √ √ √ √ √ √ √ ------√

Page e √ √ ------√ √ √ √ √

INt c * b * e * a * d *

OUTt e a d b

Working set: from t-t to t; i.e. (t-t, t]; in this case t=4

Tanir Ozcelebi 10-Dec-19 34 Agenda

Tanir Ozcelebi 10-Dec-19 35 Load control

• Paging strategy • which and how many pages to load − static paging: upon activation all pages of the process are loaded − also called simple paging − dynamic paging: upon a page fault one or more pages are loaded − pure demand paging loads a single page − demand paging with pre-paging loads several pages upon (re)activation

• Degree of multiprogramming with dynamic paging? • when using local replacement (working set) à automatic • when using global replacement à a separate policy is needed to determine the number of pages per process − look at service time of a page fault & average time between page faults − a good tradeoff will help avoid thrashing

Tanir Ozcelebi 10-Dec-19 36 Load control (cnt’d)

• Global replacement with static paging: Replace the pages of which process? (depends on scheduling) • lowest priority process − follows CPU scheduling (unlikely to be immediately scheduled again) • last process activated − considered to be the least important • smallest process − least expensive to swap out • largest process − frees the largest number of page frames

Tanir Ozcelebi 10-Dec-19 37 Example: Android virtual memory

• Use of paging (~4KB per page) and memory-mapping • Memory allocated to an app cannot be paged out (except: mmaped files that are unchanged can be paged out, e.g., a code file). • Garbage collection − Process of identifying those objects of a process that are no longer accessible and reclaiming their resources. − Releasing object references of the app makes allocated memory available to the ‘garbage collector’, which adds the freed up space back to the heap. • Background apps kept in cache (killed / removed based on LRU). • Swap space inside RAM (zRAM compression 2:1) instead of flash storage. − Because swap read/write would shorten flash storage life time. − Storage space is not significantly increased by this (2:1 ratio is not consistent). CPU use!

• Objects are part of a generation (Young, Old, Permanent) • Maximum size of occupied memory by each generation is fixed. − Garbage collection within generation if its memory is filling up. • Objects can move between generations.

Example:

Tanir Ozcelebi 10-Dec-19 38 Example: Android virtual memory

• Proportional set size (PSS): private memory allocated to a process plus the proportion of shared memory with other processes. • For two processes that are allocated 100KB each, plus a shared memory of 50KB between the two, each would have PSS=100+(50/2)=125KB. • PSS in memory per app is limited (OutOfMemoryError upon trying to allocate more).

• Sharing: All app processes forked from the system process ‘Zygote’. • Child process’ address space overwritten by the app. • Children can share resources of the parent. • Kernel shares some memory regions across processes.

Example:

Tanir Ozcelebi 10-Dec-19 39 Agenda

• Recap memory management in early systems • Principles of virtual memory • Paging • Segmentation • Segmentation with paging

Tanir Ozcelebi 10-Dec-19 40 Segmented memory allocation

• Based on common practice by programmers of structuring their programs in modules (logical groupings of code) • A segment is a logical unit such as: main program, subroutine, procedure, function, local variables, global variables, common block, stack, symbol table, or an array

• Main memory is not divided into page frames because size of each segment is different • Memory is allocated dynamically

Tanir Ozcelebi 10-Dec-19 41 Segmentation

• Each process has three obvious candidates • code, data, and stack • other candidates are − stacks of the individual threads of a process − memory-mapped files • Virtual addresses consist of segment number s and offset w address_map (s, w) { pa = *(STR+s)+w; return pa; }

Note: *(STR+s) gives the address of the first word of the segment s. • Until now: Pure segmentation: contiguous segments • Next: Segmentation with paging: paged segments

Tanir Ozcelebi 10-Dec-19 42 Agenda

• Recap memory management in early systems • Principles of virtual memory • Paging • Segmentation • Segmentation with paging • address translation, segment table, TLBs

Tanir Ozcelebi 10-Dec-19 43 Segment + page tables

Virtual address: s p w

...... frame f STR page p Segment s p w Table Register

· · · ......

Tanir Ozcelebi 10-Dec-19 44 Segment + page tables (cnt’d)

• Each memory reference requires three accesses to MM address_map (s, p, w) { pa = *(*(STR+s)+p)+w; return pa; } • Use TLB to reduce memory accesses per reference to one

• Balance between |s| and |p| • older systems: |s| long and |p| short − many small segments: segment tables are large and can themselves be paged − this results in yet another memory access • multimedia applications, however, favor |s| short and |p| long − many pages of a segment: now the page tables are large and must be paged

Tanir Ozcelebi 10-Dec-19 45 Advantages of VM

• Process size is no longer restricted to MM size • (or the free space within main memory) • Memory is used more efficiently • Eliminates external fragmentation when used with paging and eliminates internal fragmentation when used with segmentation

• Allows an unlimited amount of multiprogramming

• Allows a program to be loaded multiple times occupying a different memory location each time

• Allows sharing of code and data

• Facilitates dynamic linking of program segments

Tanir Ozcelebi 10-Dec-19 46 Disadvantages of VM

• Increased processor hardware costs

• Increased overhead for handling paging interrupts

• Increased software complexity to prevent thrashing

Tanir Ozcelebi 10-Dec-19 47 Last remark: Cache memory

CPU CPU Cache Memory bus bus

memory device memory device

• a high-speed memory that speeds up the CPU’s access to data • increases performance • no need to contend for the bus

Tanir Ozcelebi 10-Dec-19 48 Cache memory -improved efficiency-

• Cache hit ratio h: number of requests found in the cache h = total number of requests

• Average memory access time:

ta = h×tc + (1- h)×tm

tc - average cache access time

tm - average main memory access time

Tanir Ozcelebi 10-Dec-19 49 Summary

Simple Yes User has a linear address space

Private Yes VM also facilitates sharing

Unless the programmer enforces Permanent No this during execution.

Management overhead for tables and Fast Moderate strategies

Huge Yes Memory size virtually unlimited

Cost-effective Yes Hardware support for VM can be expensive

Tanir Ozcelebi 10-Dec-19 50 History: memory.c

Taken from http://lxr.free-electrons.com

Tanir Ozcelebi 10-Dec-19 51