CS 251 Fall 2019 CS 240 Principles of Programming Languages Problems with physical addressing Foundations of Computer Systems λ Ben Wood Main memory 0: 1: Physical address 2: (PA) 3: CPU 4: 4 5: 6: 7: 8: Process Abstraction, Part 2: Private Address Space ...

Motivation: why not direct physical memory access? M-1: Address translation with pages Optimizing translation: translation lookaside buffer Data Extra benefits: sharing and protection

Memory as a contiguous array of bytes is a lie! Why?

https://cs.wellesley.edu/~cs240/ Virtual Memory 1 Virtual Memory 2

Problem 1: memory management Problem 2: capacity

64-bit addresses can address Physical main memory offers Main memory several exabytes a few gigabytes (18,446,744,073,709,551,616 bytes) (e.g. 8,589,934,592 bytes)

Process 1 stack Process 2 (To scale with 64-bit address space, heap What goes Process 3 where? ? you can't see it.) × code … globals Process n …

Also: Context switches must swap out entire memory contents. 1 virtual address space per process, Isn't that expensive? Virtual Memory 3 with many processes… Virtual Memory 4 Problem 3: protection Solution: Virtual Memory (address indirection)

data Physical main memory

Process i Process 1

virtual Physical memory Process j Virtual address space addresses virtual-to-physical mapping physical addresses Problem 4: sharing Physical main memory Process n virtual

Process i Virtual address space addresses data

Process j Private virtual address Single physical address space space per process. managed by OS/hardware. Virtual Memory 5 Virtual Memory 6

Indirection "2" Tangent: indirection everywhere (it's everywhere!) 0 • Pointers 1 Direct naming "2" • Constants "2" Thing 2 3 What X • Procedural abstraction 4 currently • "x" maps to 5 Domain Name Service (DNS) Indirect naming "x" 2 6 • Dynamic Host Configuration Protocol (DHCP) "x" 7 • Phone numbers "x" • 911 What if we move Thing? • Call centers • Snail mail forwarding “Any problem in computer science can be solved by adding another level of indirection.” –David• Wheeler,… inventor of the subroutine, or Butler Lampson

Another Wheeler quote? "Compatibility means deliberately repeating other people's mistakes." Virtual Memory 7 Virtual Memory 8 fixed-size, aligned pages Virtual addressing and address translation Page-based mapping page size = power of two

Memory Management Unit Virtual Address Space translates virtual address to physical address 0 Virtual Physical Main memory Page 0: Address Space 0 0 CPU Chip 1: Virtual Physical Page Virtual address Physical address 2: Page 3: 0 (VA) (PA) 1 CPU MMU 4: Physical 4100 4 5: Virtual Page Page 6: 1 7: 2 8: Virtual ... Page Map virtual pages 3 … M-1: onto physical pages. Physical … Page m 2p - 1 Virtual 2 - 1 Data Page 2n - 1 2v - 1 Some virtual pages do not fit! Where are they stored? Physical addresses are invisible to programs.Virtual Memory 9 Virtual Memory 10

Not drawn to scale Cannot fit all virtual pages! Where are the rest stored? Virtual memory: cache for disk?

Virtual Memory virtual address space Address Space usually much larger than SRAM DRAM 0 physical address space Virtual Physical Memory ~4 MB ~8 GB ~500 GB Page Address Space 0 0 L1 Virtual Physical Page I-cache Page L2 0 1 Main 32 KB unified Disk Virtual Physical cache Memory Page Page L1 CPU Reg 2 1 D-cache Virtual Page Throughput: 16 B/cycle 8 B/cycle 2 B/cycle 1 B/30 cycles solid-state "flash" 3 … Latency: 3 cycles 14 cycles 100 cycles millions 1. On disk if used Physical or … Page spinning 2m - 1 2p - 1 magnetic platter. Virtual Cache miss penalty Page v (latency): 33x 2n - 1 2 - 1 2. Nowhere if not (yet?) used Memory miss penalty

Virtual Memory 11 Example system (latency): 10,000x Virtual Memory 12 Design for a slow disk: exploit locality Design for a slow disk: exploit locality

Virtual Memory Virtual Memory Address Space Address Space LargePage page size? size 0 0 usually 4KB, up to 2-4MB Virtual Physical Memory Virtual Physical Memory Page Address Space Page Address Space 0 0 0 0 Virtual Physical Virtual Physical Page Page Page Page 0 0 1 1 Fully associative Virtual Physical Virtual • Store anyAssociativity? virtual page in any physical page Physical Page Page Page • Large mapping function Page 2 1 2 1 Virtual Virtual Page Page 3 … 3 … on disk Physical on disk Physical … Page … Page m 2p - 1 m 2p - 1 Virtual 2 - 1 Virtual 2 - 1 Page Page SophisticatedReplacement v v 2n - 1 2 - 1 2n - 1 2 - 1 replacementpolicy? policy Write • Not just hardware Write-back Virtual Memory 13 policy?Virtual Memory 14

Address translation Page table Physical pages array of page table entries (PTEs) (Physical memory) mapping virtual page to where it is stored Main memory VP 1 PP 0 Physical Page Number 0: CPU Chip 1: Valid or disk address VP 2 Virtual address Physical address 2: PTE 0 0 null 3: (VA) (PA) 1 VP 7 CPU MMU 4: 4100 4 5: 1 VP 4 PP 3 6: 0 7: 8: 1 ... 0 null 0 M-1: PTE 7 1 Swap space Data page table (Disk) VP 3 Memory resident, managed by HW (MMU), OS VP 6

Virtual Memory 15 How many page tables are in the system? Virtual Memory 16 virtual page is in memory Address translation with a page table Page hit: Physical pages (Physical memory) Virtual Page Number Page table Virtual address (VA) VP 1 PP 0 base register Physical Page Number (PTBR) Virtual page number (VPN) Virtual page offset (VPO) Valid or disk address VP 2 PTE 0 0 null Base address of current process's Page table 1 PP 0 VP 7 page table Valid Physical page number (PPN) 1 PP 1 VP 4 PP 3 0 On disk 1 PP 3 0 null 0 On disk PTE 7 1 PP 2 Swap space Virtual page mapped (Disk) to physical page? page table VP 3 yes = page hit Physical page number (PPN) Physical page offset (PPO) VP 6 Physical address (PA)

Virtual Memory 17 Virtual Memory 18

exceptional control flow : Physical pages Page fault: (Physical memory) Virtual Page Number Process accessed virtual address in a page that is not in physical memory. VP 1 PP 0 Physical Page Number Valid or disk address VP 2 PP 1 Process PTE 0 0 null 1 PP 0 VP 7 PP 2 User Code OS exception handler 1 PP 1 VP 4 PP 3 0 On disk 1 PP 3 exception: page fault 0 null movl Load page 0 On disk into memory PTE 7 1 PP 2 Swap space return page table (Disk) VP 3

VP 6 Returns to faulting instruction: movl is executed again! Virtual Memory 19 Virtual Memory 20 "Page out" page not in memory OS evicts another page. Page fault: 1. Physical pages Page fault: 2. Physical pages (Physical memory) (Physical memory) Virtual Page Number Exception! Virtual Page Number VP 1 PP 0 VP 1 PP 0 Physical Page Number Physical Page Number Valid or disk address VP 2 Valid or disk address VP 2 PTE 0 0 null PTE 0 0 null 1 PP 0 VP 7 0 On disk VP 7 1 PP 1 1 PP 1 VP 4 PP 3 VP 4 PP 3 0 On disk 0 On disk 1 PP 3 1 PP 3 0 null 0 null 0 On disk 0 On disk PTE 7 1 PP 2 Swap space PTE 7 1 PP 2 Swap space page table (Disk) page table (Disk) VP 3 VP 3 What now? VP 6 VP 6 VP 1 OS handles fault Virtual Memory 21 Virtual Memory 22

"Page in" OS loads needed page. Page fault: 3. Physical pages Terminology (Physical memory) Virtual Page Number context switch VP 3 PP 0 Physical Page Number Switch control between processes on the same CPU. Valid or disk address VP 2 PTE 0 0 null page in 1 On disk VP 7 Move page of virtual memory from disk to physical memory. 1 PP 1 VP 4 PP 3 1 PP 0 page out 1 PP 3 Move page of virtual memory from physical memory to disk. 0 null 0 On disk thrash PTE 7 1 PP 2 Swap space Total working set size of processes is larger than physical memory. page table (Disk) Most time is spent paging in and out instead of doing useful work. VP 3 Finally: Re-execute faulting instruction. VP 6 Page hit! VP 1 Virtual Memory 23 Virtual Memory 24 Address translation: page hit Address Translation: Page Fault

Exception Page fault handler 4

2 2 CPU Chip PTEA CPU Chip PTEA Victim page 1 1 PTE 5 VA Cache/ VA PTE Cache/ CPU MMU 3 CPU MMU Disk Memory 7 3 Memory PA New page 4 6 Data 1) Processor sends virtual address to MMU 5 2-3) MMU fetches PTE from page table in cache/memory 1) Processor sends virtual address to MMU (memory management unit) 4) Valid bit is zero, so MMU triggers page fault exception 2-3) MMU fetches PTE from page table in cache/memory 5) Handler identifies victim (and, if dirty, pages it out to disk) 4) MMU sends physical address to cache/memory 6) Handler pages in new page and updates PTE in memory 5) Cache/memory sends data word to processor 7) Handler returns to original process, restarting faulting instruction

Virtual Memory 25 Virtual Memory 26

How fast is translation? TLB hit How many physical memory accesses are required to complete one virtual memory access? CPU Chip TLB 2 PTE VPN 3 Translation Lookaside Buffer (TLB) 1 VA PA CPU MMU 4 Cache/ Small hardware cache in MMU just for page table entries Memory e.g., 128 or 256 entries

Data Much faster than a page table lookup in memory. 5

In the running for "un/classiest name of a thing in CS" A TLB hit eliminates a memory access

Virtual Memory 27 Virtual Memory 28 TLB miss Memory system example (small)

Simulate accessing these virtual addresses on Addressing the system: 0x03D4, 0x0B8F, 0x0020 CPU Chip 14-bit virtual addresses TLB 4 12-bit physical address 2 PTE VPN Page size = 64 bytes

1 3 13 12 11 10 9 8 7 6 5 4 3 2 1 0 VA PTEA CPU MMU Cache/ PA Memory VPN VPO 5 Virtual Page Number Virtual Page Offset

Data 6 11 10 9 8 7 6 5 4 3 2 1 0

A TLB miss incurs an additional memory access (the PTE) PPN PPO Fortunately, TLB misses are rare. Does a TLB miss require disk access? Physical Page Number Physical Page Offset Virtual Memory 29 Virtual Memory 30

Memory system example: page table Memory system example: TLB

Only showing first 16 entries (out of 256 = 28) 16 entries virtual page #___ TLB index___ TLB tag ____ TLB Hit? __ Page Fault? __ physical page #: ____ 4-way associative TLB ignores page offset. Why?

VPN PPN Valid VPN PPN Valid 00 28 1 08 13 1 TLB tag TLB index 01 – 0 09 17 1 13 12 11 10 9 8 7 6 5 4 3 2 1 0 02 33 1 0A 09 1 03 02 1 0B – 0 virtual page number virtual page offset 04 – 0 0C – 0 05 16 1 0D 2D 1 virtual page #___ TLB index___ TLB tag ____ TLB Hit? __ Page Fault? __ physical page #: ____ 06 – 0 0E 11 1 07 – 0 0F 0D 1 Set Tag PPN Valid Tag PPN Valid Tag PPN Valid Tag PPN Valid 0 03 – 0 09 0D 1 00 – 0 07 02 1 1 03 2D 1 02 – 0 04 – 0 0A – 0 2 02 – 0 08 – 0 06 – 0 03 – 0 What about a real address space? Read more in the book… 3 07 – 0 03 0D 1 0A 34 1 02 – 0

Virtual Memory 31 Virtual Memory 32 Virtual memory benefits: Memory system example: cache Simple address space allocation

16 lines cache tag cache index cache offset Process needs private contiguous address space. 4-byte block size 11 10 9 8 7 6 5 4 3 2 1 0 Storage of virtual pages in physical pages is fully associative. Physically addressed physical page number physical page offset Direct mapped Virtual Address Spaces Physical Address Space (DRAM) Process 1: 0 0 cache offset___ cache index___ cache tag____ Hit? __ Byte: ____ VP 1 VP 2 PP 2 Idx Tag Valid B0 B1 B2 B3 Idx Tag Valid B0 B1 B2 B3 ... 0 19 1 99 11 23 11 8 24 1 3A 00 51 89 1 15 0 – – – – 9 2D 0 – – – – N-1 2 1B 1 00 02 04 08 A 2D 1 93 15 DA 3B PP 6 3 36 0 – – – – B 0B 0 – – – – 0 4 32 1 43 6D 8F 09 C 12 0 – – – – Process 2: PP 8 5 0D 1 36 72 F0 1D D 16 1 04 96 34 15 VP 1 PP 9 VP 2 6 31 0 – – – – E 13 1 83 77 1B D3 ...... 7 16 1 11 C2 DF 03 F 14 0 – – – – N-1 M-1

Virtual Memory 33 Virtual Memory 34

Virtual memory benefits: Virtual memory benefits: Simple cached access to storage > memory Protection: Good locality, or least "small" working set = mostly page hits All accesses go through translation. Impossible to access physical memory not mapped in virtual address space.

All necessary Sharing: Working set pages page table entries fit in physical memory Map virtual pages in separate address spaces to same physical page (PP 6). fit in TLB Virtual Address Spaces Physical Address Space (DRAM) Process 1: 0 0 VP 1 If combined working set > physical memory: VP 2 PP 2 Thrashing: Performance meltdown. CPU always waiting or paging. ...

N-1 (e.g., execute-only PP 6 Full indirection quote: library code: libc) 0 “Every problem in computer science can be solved by adding another Process 2: PP 8 level of indirection, but that usually will create another problem.” VP 1 VP 2 ......

Virtual Memory 35 N-1 M-1 Virtual Memory 36 Virtual memory benefits: Summary: virtual memory Memory permissions MMU checks on every access. Programmer’s view of virtual memory Exception if not allowed. Each process has its own private linear address space Cannot be corrupted by other processes permission bits Physical Process 1: Valid READ WRITE EXEC Physical Page Num Address Space VP 0: Yes No No Yes PP 6 System view of virtual memory VP 1: Yes No No Yes PP 4 PP 2 VP 2: Yes Yes Yes No PP 2 Uses memory efficiently (due to locality) by caching virtual memory pages Page Table PP 4 Simplifies memory management and sharing PP 6 Simplifies protection -- easy to interpose and check permission bits permissions Process 2: Valid READ WRITE EXEC Physical Page Num PP 8 More goodies: VP 0: Yes Yes Yes No PP 9 PP 9 • Memory-mapped files VP 1: Yes No No Yes PP 6 • Cheap fork() with copy-on-write pages (COW) VP 2: Yes Yes No No PP 11 PP 11 Page Table How would you set permissions for the stack, heap, global variables, literals, code? Virtual Memory 37 Virtual Memory 38

Summary: memory hierarchy

L1/L2/L3 Cache: Pure Hardware Purely an optimization "Invisible" to program and OS, no direct control Programmer cannot control caching, can write code that fits well Virtual Memory: Software-Hardware Co-design Supports processes, memory management Operating System (software) manages the mapping Allocates physical memory Maintains page tables, permissions, metadata Handles exceptions Memory Management Unit (hardware) does translation and checks Translates virtual addresses via page tables, enforces permissions TLB caches the mapping Programmer cannot control mapping, can control sharing/protection via OS

Virtual Memory 39