Demand Paging CSE 120: Principles of Opera�Ng Systems

Lecture 9: Demand Paging CSE 120: Principles of Operang Systems

UC San Diego: Summer Session I, 2009 Frank Uyeda Announcements

• Project 2 is due on Tonight • Homework 3 is due Monday.

• Final Exam: 3p‐6p on Saturday, August 1

2 PeerWise

• Processes are loaded and then removed from memory, creang free memory space broken into lile pieces. What type of fragmentaon is this? – Internal – External

3 PeerWise

• What is the diﬀerence between "Segmenon" and "Paging"? (Choose the one with the highest truth value) – A. Le has the answer. – B. Segmentaon is non‐connous allocaon of pages of memory, where paging is connous allocaon of pages of memory. – C. Segmentaon is the use of variable page sizes, and paging is the use of discrete page sizes. – D. C – E. D & A

• Really?! – None of the answers are correct… except, maybe A.

4 V Page Frame Recap: Paging 1 0x04 Process 1’s VAS 1 0x01 Physical Memory 1 0x05 Page 1 0x00000000 1 0x07 Page 1 Page 2 0x00000400 Process 2’s VAS Page 2 Page 3 MMU

….. Page 1 TLB Page 3

Page ... Page M Page 2 Table Page P Page 3 V Page Frame .. …..

1 0x04

1 0x01 Page N Page Q 0x00040000 5 1 0x05

1 0x07 Recap: Virtual Memory

• Memory Management Unit (MMU) – Hardware unit that translates a virtual address to a physical address • Translaon Table – Stored in main memory – Each Page Table Entry stores permission and bookkeeping bits. • Translaon Lookaside Buﬀer (TLB) – Hardware cache for the MMU’s virtual‐to‐physical translaons table

Virtual Address M R V Prot Page Frame Number

Translaon Physical CPU MMU Table Address Memory

TLB

6 Recap: Page Tables Physical Memory

0x00000000 Virtual Address Page number Oﬀset Page 1

Physical Address Page 2 Page frame Oﬀset Page Table Page 3 …..

Page table entry

Page N

0xFFFFFFFF Problem: Page Tables can be large! 7 Recap: 2 Level Page Table Physical Memory

0x00000000 Virtual Address Master Secondary Oﬀset Page 1

Physical Address Page 2 Page frame Oﬀset Page 3 …..

Page frame (PTE) Page frame (PTE) Page table entry Page N

Master Secondary 0xFFFFFFFF Page Table Page Table(s)

8 Inverted Page Table Physical Memory

0x00000000 Virtual Address PID VPN Oﬀset Page 1

Physical Address Page 2 index Oﬀset Page 3

index …..

Page N

Inverted Page Table 0xFFFFFFFF

9 Inverted Page Table

• One global page table – One page table entry per physical page. – Entries keyed by PID and virtual page number. – Physical frame number = index in page table. • Advantages – Bounded amount of memory for page table(s). • 32‐bit address space, 4K pages, 4GB RAM, 4B per PTE • 1‐level page table: _____ x # processes. • Inverted Table: ______MB • Disadvantages – Costly translaon • Using hashing can help

10 Recap: Segmentaon

Heap Stack Text

Data

11 Recap: Segmentaon

Physical Memory Segment Table

Virtual Address limit base Segment # Oﬀset

Yes? < +

No?

Protecon Fault

12 Goals for Today

• Paging Tricks – Demand Paging – Copy on Write • Page Replacement Policies – Which pages should we keep in memory? – How many pages should each process get?

13 Paged Virtual Memory • What if there isn’t enough physical memory available to load a program? – A large poron of program’s code may be unused – A large poron of program’s code may be used infrequently • Idea: load a page only if you need it Process’s Virtual Address Space Machine’s Physical Address Space (available RAM)

14 Demand Paging

• We’ve menoned before that pages can be moved between memory and disk – This process is called demand paging • OS uses main memory as a page cache of all the data allocated by processes in the system – Inially, pages are allocated from memory – When memory ﬁlls up, allocang a page in memory requires some other page to evicted from memory • Why physical memory pages are called “frames” – Evicted pages go to disk (where? The swap ﬁle paron) – The movement of pages between memory and disk is done by the OS, and is transparent to the applicaon

15 Paged Virtual Memory

• A program’s pages loaded on demand Machine’s Physical Address Space • Where are pages otherwise? (available RAM) – On disk (executable) – In OS swap ﬁle (an actual ﬁle on disk) – Shared (more later)

Process’s Virtual Address Space

Disk

~/program.coﬀ /swap

16 Demand Paging Example

Process’s Virtual Address Space Physical Memory PC PC Page Table VPN V PPN 0x00 1 0x00 0x01 0 1 0x01 ‐‐‐ 0x02 0 ‐‐‐ 0x03 0 ‐‐‐ … Disk 0x10 0 ‐‐‐ 0x11 0 ‐‐‐

~/program.coﬀ /swap … … 17 Demand Paging Example

Process’s Virtual Address Space Physical Memory 0x00

PC Page Table VPN V PPN 0x00 1 0x00 0x01 0 1 0x01 ‐‐‐ 0x09 0x02 1 0x02 0x0A 0x03 1 0x03 0x04 1 … 0x09 Disk 0x10 1 0 0x00 ?? 0x09 0x11 0 ‐‐‐

~/program.coﬀ /swap … … 18 Page Faults

• What happens when a process accesses a page that has been evicted (or is otherwise not in memory)? – 1. When it evicts a page, the OS sets the PTE as invalid and stores the locaon of the page in the swap file in the PTE – 2. When a process accesses the page, the invalid PTE will cause a trap (page fault) – 3. The trap will run the OS page fault handler – 4. Handler uses the invalid PTE to locate page in swap file – 5. Reads page into a physical frame, updates PTE to point to it – 6. Restarts process • But where does it put it? Has to evict something else – OS usually keeps a pool of free pages around so that allocaons do not always cause evicons – Replace another page • Evicted page may go in swap file. Which pages need to be wrien out to swap ?

19 Recap: Page Lookups

Virtual Address

CPU

TLB Page Table

Disk

Physical Memory Address

20 Recap: Page Lookups

Virtual Address

CPU

TLB ..... Secondary Page Table Page Table

Disk

Physical Memory Address

21 Recap: Page Lookups

Virtual Address Case 1: Virtual Address –to‐ Physical Address Mapping is in TLB

CPU VPN V R M RO PFN

0x000 1 1 0 1 0x100 TLB 0x121 1 0 0 0 0x111 Secondary Page Table 0x071 Page Table 1 1 0 0 0x212 (1)

How many memory references? Disk TLB_hit: 1 memory reference Physical Memory Address

22 Recap: Page Lookups

Virtual Address Case 2: Virtual Address –to‐ Physical Address Mapping is not in TLB but is in Memory

CPU VPN V R M RO PFN

0x000 1 1 0 1 0x100 TLB 0x121 1 0 0 0 0x111 Secondary (1) Page Table Page Table 0x071 1 1 0 0 0x212

(2) Disk How many memory references? Physical Mem_hit: 1 page table lookup Memory Address + 1 memory reference

Two‐level Page table? 23 Recap: Page Lookups

Virtual Address Case 3: Virtual Address –to‐ Physical Address Mapping is not in TLB and not in Memory

CPU VPN V R M RO PFN

0x000 1 1 0 1 0x100 TLB 0x121 0 0 0 0 0x111 Secondary (1) Page Table (3?) Page Table 0x071 1 1 0 0 0x212

(2) (4?) How many memory references? Mem_miss: 1 page table lookup + 1 page table update(s)* Disk Physical + 1 memory reference Memory Address How much me?

disk retrieval involved too 24 Memory Accesses

• TLB Hit: fast – 1 TLB access + 1 memory access • Memory Hit: medium – 1 TLB access + 2+ memory access • Memory Miss: slow – 1 TLB access + 3+ memory access + 1 disk access

• Want things fast as possible; minimize me spent resolving physical address. How do you do this? – Minimize number of memory references (TLB policy) – Minimize number of page faults (Page replacement policy)

25 Observaon

• OSes spend a lot of me copying data – System call arguments between user/kernel space – Enre address spaces to implement fork()

• How can we reduce this cost?

26 Page Sharing

Emacs buﬀer #1 Emacs buﬀer #2 VPN V PPN VPN V PPN 0x00 1 0x03 0x00 1 0x03 0x04

0x010x03 0 1 0x11 ‐‐‐ 0x010x03 0 1 0x13 ‐‐‐ 0x10 1 0x12 0x10 1 0x14 0x11 1 0x15 0x11 1 0x16

Physical Memory .. What if shared page is modiﬁed? 27 Copy‐On‐Write

• OSes spend a lot of me copying data – System call arguments between user/kernel space – Enre address spaces to implement fork() • Use Copy‐on‐Write (CoW) to defer large copies as long as possible, hoping to avoid them altogether – Instead of copying pages, create shared mappings of parent pages in child virtual address space – Shared pages are protected as read‐only in child • Reads happen as usual • Writes generate a protecon fault, trap to OS, copy page, change page mapping in client page table, restart write instrucon • How does this help fork()?

28 Page Sharing: CoW

Emacs buﬀer #1 VPN V M PPN Emacs buﬀer #2 0x00 1 0 0x03

0x010x03 0 1 0 ‐‐‐ 0x11 0x10 1 0 1 0x12 0x11 1 0 0x15 VPN V M PPN 0x00 1 0 0x03

0x010x03 0 1 0 ‐‐‐ 0x13 0x10 1 0 0x12 0x11 1 0 0x16 Physical Memory

Buﬀers share VPN 0x12 .. Process writes to page

29 Page Sharing: CoW

Emacs buﬀer #1 VPN V M PPN Emacs buﬀer #2 0x00 1 0 0x03

0x010x03 0 1 0 ‐‐‐ 0x11 0x10 1 0 0x12 0x11 1 0 0x15 VPN V M PPN 0x00 1 0 0x03

0x010x03 0 1 0 ‐‐‐ 0x13 0x10 1 0 0x14 0x11 1 0 0x16 Physical Memory

Buﬀers share VPN 0x12 .. Process writes to page New page is allocated

30 Paging Tricks Summary

• Demand Paging – Observaon: Limited available RAM, not all pages used – Idea: Load pages only when needed • Page Sharing – Observaon: Oen mes, processes have similar virtual pages – Idea: Copy‐on‐write for eﬃcient page sharing

31 Actual Memory Stascs 3.7 GB used, 4.1 GB available => 90+% memory used 2.1 GB swap allocated

32 Actual Memory Stascs ~10% ‐ 50% of memory is shared per process => ~370 MB – 1.85 GB mem saved

33 Actual Memory Stascs Which pages to evict? How much physical memory should each process get?

34 Page Replacement

• Problem: Memory is scarce – Idea: let’s use only pages that we need

• Page replacement policy: – How do we decide which pages to kick out when memory is full or near full? – How much memory do we allocate to each process?

35 Page Replacement Policy

• Want to keep most acve pages in memory – Conversely, want to kick out inacve pages

• What can we do? – Swapping: suspend an enre process and puts all of its pages to disk/swap – Paging: remove unused or infrequently used pages for any process to disk/swap

36 Goal: Evicng the Best Page

• The goal of the replacement algorithm is to reduce the page fault rate – Select the best vicm page to remove • The best page to evict is the one never touched again – Process will never ask for it, so it will never fault on it • How do we know that there a best page? • Never is a long me, so picking the page closest to “never” is the next best thing – Evicng the page that won’t be used for the longest period of me minimizes the number of page faults – Proved by Belady

37 Locality

• All paging schemes depend on locality – Processes reference pages in localized paerns • Temporal locality – Locaons referenced recently likely to be referenced again • Spaal locality – Locaons near recently referenced locaons are likely to be referenced soon • Although the cost of paging is high, if it is infrequent enough it is acceptable – Processes usually exhibit both kinds of locality during their execuon, making paging praccal

38 Goal: Evicng the Best Page

• The goal of the replacement algorithm is to reduce the page fault rate – Select the best vicm page to remove • The best page to evict is the one never touched again – Process will never ask for it, so it will never fault on it • How do we know that there a best page? Locality • Never is a long me, so picking the page closest to “never” is the next best thing – Evicng the page that won’t be used for the longest period of me minimizes the number of page faults – Proved by Belady

39 Belady’s Algorithm

• Belady’s algorithm is known as the opmal page replacement algorithm – Lowest fault rate for any page reference stream – Idea: replace the page that will not be used for the longest me in the future – Problem: have to predict the future • Why is Belady’s useful then? Use it as a yardsck Page – Compare implementaons of page replacement algorithms with 1 2 3 4 1 2 5 1 2 3 4 5 references: the opmal to gauge room for improvement – If opmal is not much beer, then algorithm is prey good – If opmal is much beer, then algorithm could use some work 1 1 1 1 1 1 1 1 2 3 4 4 2 Pages RAM • Random replacement is oen the lower bound 2 3 4 4 2 5 5 5 5 5 5 9 page faults • But random is not necessarily the worst…..what would be?

40 Belady’s Algorithm

• Belady’s algorithm is known as the opmal page replacement algorithm – Lowest fault rate for any page reference stream – Idea: replace the page that will not be used for the longest me in the future – Problem: have to predict the future • Why is Belady’s useful then? Use it as a yardsck – Compare implementaons of page replacement algorithms with the opmal to gauge room for improvement – If opmal is not much beer, then algorithm is prey good – If opmal is much beer, then algorithm could use some work • Random replacement is oen the lower bound • But random is not necessarily the worst…..what would be?

41 First‐In First‐Out (FIFO)

• FIFO is an obvious algorithm and simple to implement – Maintain a list of pages in order in which they were paged in – On replacement, evict the one brought in longest me ago • Why might this be good? – Maybe the one brought in the longest ago is not being used • Why might this be bad? – Then again, maybe it’s not • We don’t have any informaon to say one way or another

42 FIFO Page Replacement Example Page 1 2 3 4 1 2 5 1 2 3 4 5 references:

1 1 3 3 1 1 5 5 2 2 4 4 2 Pages RAM 12 page faults 2 2 4 4 2 2 1 1 3 3 5

3 Pages RAM

4 Pages RAM

43 FIFO Page Replacement Example Page 1 2 3 4 1 2 5 1 2 3 4 5 references:

1 1 3 3 1 1 5 5 2 2 4 4 2 Pages RAM 12 page faults 2 2 4 4 2 2 1 1 3 3 5

1 1 1 4 4 4 5 5 5 5 5 5 3 Pages RAM 2 2 2 1 1 1 1 1 3 3 3 9 page faults 3 3 3 2 2 2 2 2 4 4

4 Pages RAM

44 FIFO Page Replacement Example Page 1 2 3 4 1 2 5 1 2 3 4 5 references:

1 1 3 3 1 1 5 5 2 2 4 4 2 Pages RAM 12 page faults 2 2 4 4 2 2 1 1 3 3 5

1 1 1 4 4 4 5 5 5 5 5 5 3 Pages RAM 2 2 2 1 1 1 1 1 3 3 3 9 page faults 3 3 3 2 2 2 2 2 4 4

1 1 1 1 1 1 5 5 5 5 4 4 2 2 2 2 2 2 1 1 1 1 5 4 Pages RAM 3 3 3 3 3 3 2 2 2 2 10 page faults! 4 4 4 4 4 4 3 3 3 45 First‐In First‐Out (FIFO)

• FIFO suﬀers from “Belady’s Anomaly” – The fault rate might actually increase when the algorithm is given more memory (very bad)

46 Least Recently Used (LRU)

• LRU uses reference informaon to make a more informed replacement decision – Idea: We can’t predict the future, but we can make a guess based upon past experience – On replacement, evict the page that has not been used for the longest me in the past (Belady’s: future) – When does LRU do well? When does LRU do poorly? • Implementaon – To be perfect, need to me stamp every reference (or maintain a stack) – much too costly – So we need to approximate it

47 LRU Page Replacement Example Page 1 2 3 4 1 2 5 1 2 3 4 5 references:

1 1 3 3 1 1 5 5 2 2 4 4 2 Pages RAM 12 page faults 2 2 4 4 2 2 1 1 3 3 5

3 Pages RAM

4 Pages RAM

48 LRU Page Replacement Example Page 1 2 3 4 1 2 5 1 2 3 4 5 references:

1 1 3 3 1 1 5 5 2 2 4 4 2 Pages RAM 12 page faults 2 2 4 4 2 2 1 1 3 3 5

1 1 1 4 4 4 5 5 5 3 3 3 3 Pages RAM 2 2 2 1 1 1 1 1 1 4 4 10 page faults 3 3 3 2 2 2 2 2 2 5

1 1 1 1 1 1 1 1 1 1 1 5 2 2 2 2 2 2 2 2 2 2 2 4 Pages RAM 8 page faults 3 3 3 3 5 5 5 5 4 4 4 4 4 4 4 4 3 3 3 49 Least Recently Used (LRU)

50 Approximang LRU

• LRU approximaons use the PTE reference bit – Keep a counter for each page – At regular intervals, for every page do: • If ref bit = 0, increment counter • If ref bit = 1, zero the counter and the reference bit – The counter will contain the number of intervals since the last reference to the page – The page with the largest counter is the least recently used • Some architectures don’t have a reference bit – Can simulate reference bit using the valid bit to induce faults – What happens when we make a page invalid?

51 Approximang LRU Page Table VPN PFN R CNTR CNTR CNTR CNTR CNTR 0 0x00 0xA1 0 0000 0001 0002 0003 0004 10 0x01 0xA2 0 0000 0001 0000 0001 0000 10 0x02 0xA0 0 0000 0001 0000 0001 0000 0 0x03 0x10 0 0000 0001 0002 0003 0004 0 0x04 0x03 0 0000 0001 0002 0003 0004 10 0x05 0x04 0 0000 0001 0000 0001 0002

Interval 1 Interval 2 Interval 3 Interval 4

Which page(s) would be evicted next based on Approximate LRU? What does a good approximaon depend upon?

52 Approximang LRU

53 Approximang LRU: Clock

• Not Recently Used (NRU) – Used by Unix – Replace page that is “old enough” – Arrange all of physical page frames in a big circle (clock) – A clock hand is used to select a good LRU candidate • Sweep through the pages in circular order like a clock • If the ref bit is off, it hasn’t been used recently – What is the minimum “age” if reference bit is off? • If the reference bit is on, turn it off and go to next page – Arm moves quickly when pages are needed – Low overhead when plenty of memory – If memory is large, “accuracy” of informaon degrades • Use addional hands

54 Approximang LRU: Clock Page Table VPN PFN R VPN PFN R VPN PFN R

0x00 0xA1 1 0x00 0xA1 0 0x00 0xA1 0 0x01 0xA2 1 0x01 0xA2 1 0x01 0xA2 0 0x02 0xA0 1 0x02 0xA0 1 0x02 0xA0 1 0x03 0x10 1 0x03 0x10 1 0x03 0x10 1 (1) (2) (3)

VPN PFN R VPN PFN R

0x00 0xA1 0 0x00 0xA1 0 Evict page! 0x01 0xA2 0 0x01 0xA2 0 0x02 0xA0 0 0x02 0xA0 0 0x03 0x10 1 0x03 0x10 0 (4) (5) Just like FIFO, depending on clock hand? Clock Hand 55 Approximang LRU: Clock Page Table VPN PFN R VPN PFN R

0x00 0xA1 1 0x00 0xA1 0 0x01 0xA2 0 0x01 0xA2 0 Evict page! 0x02 0xA0 0 0x02 0xA0 1 0x03 0x10 1 0x03 0x10 1 (1) (2)

What do you need for this algorithm to work well? What if memory is large?

Clock Hand 56 Approximang LRU: Clock

57 Today’s lecture: Page Replacement (2)

• Problem: Memory is scarce – Idea: let’s use only pages that we need

• Page replacement policy: – How do we decide which pages to kick out when memory is full or near full? – How much memory do we allocate to each process?

58 Page Replacement Policies

• In a mulprogramming system, we need a way to allocate memory to compeng processes • Problem: How to determine how much memory to give to each process? – Fixed space algorithms • Each process is given a limit of pages it can use • When it reaches the limit, it replaces from its own pages • Local replacement – Some processes may do well while others suﬀer – Variable space algorithms • Process’ set of pages grows and shrinks dynamically • Global replacement – One process can ruin it for the rest

59 Working Set Model

• A working set of a process is used to model the dynamic locality of its memory usage – Deﬁned by Peter Denning in the 60s

• Deﬁnion – WS(t,w) = { pages P s.t. P was referenced in the time interval (t,t+w) } – t = time, w = working-set window (measured in page references)

• A page is in the working set WS only if it was referenced in the last w references

60 Working Set Size

• Working set size is the number of pages in the working set – The number of pages referenced in the interval (t,t+w)

• The working set size changes with program locality – During periods of poor locality, you reference more pages – Within that period of me, the working set size is larger

• Intuively, want the working set to be the set of pages a process needs in memory to prevent heavy faulng – Each process has a parameter w that determines a working set with few faults – Denning: Don’t run a process unless working set is in memory

61 Working Set Problems

• Problems – How do we determine w? – How do we know when the working set changes? • Too hard to answer – So, working set is not used in pracce as a page replacement algorithm • However, it is sll used as an abstracon – The intuion is sll valid – When people ask, “How much memory does iTunes need?”, they are in eﬀect asking for the size of its working set.

62 Page Fault Frequency (PFF)

• Page Fault Frequency (PFF) is a variable space algorithm that uses a more ad‐hoc approach – 1. Monitor the fault rate for each process – 2. If the fault rate is above a high threshold: give it more memory • Hope that it faults less • Not always true (FIFO + Belady’s Anomaly) – 3. Else if the fault rate is below a low threshold: take away memory • Hope that it sll doesn’t fault much • Might not fault more at all • Hard to use PFF to disnguish between changes in locality and changes in size of working set

63 PFF and Mulprogramming

• What does the page fault frequency of processes tell you?

System “goodput” (# of programs’ instrucons executed)

Can this happen?

Degree of mulprogramming (# concurrent processes) 64 Thrashing

• Page replacement algorithms avoid thrashing – When most of the me is spend by the OS in paging data back and forth from the disk – No me spent doing useful work (making progress) – In his situaon, the system is overcommied • No idea which pages should be in memory to reduce faults • Could just be that there isn’t enough physical memory for all of the processes in the system • Ex: Running Windows XP with 64 MB memory … (or Vista with 1 GB) – Possible soluons • Swapping – write out all pages of a process • Buy more memory

65 Page Replacement Summary

• Which page(s) to kick out when memory is under constraint? – Page replacement algorithms • Belady’s – opmal replacement (min # faults) • FIFO – replace page loaded furthest in past • LRU – replace page referenced furthest in past • LRU Clock – replace page that is “old enough” • Page Fault Frequency – grow/shrink page set as a funcon of fault rate

• How many pages per process? – Global vs local replacement policies • Should a process replace its own page, or that of another? – Working Set – keep the set of pages in memory that has minimal fault rate (the “working set”) • Page Fault Frequency – grow/shrink page set as a funcon of fault rate to avoid thrashing

66 Next Time

• Read Chapter 10 & 11.1‐11.8 • Homework due on Monday. • Check Web site for course announcements – hp://www.cs.ucsd.edu/classes/su09/cse120