ECE4750/CS4420 Computer Architecture L7: Virtual Memory
Edward Suh Computer Systems Laboratory [email protected]
Announcements
. HW1 due today • Check the clarification and announcement on blackboard • Submit a version early – we will be strict with the deadline
. Prelim conflict should be reported by the end of this week
ECE4750/CS4420 — Computer Architecture, Fall 2008 2
1
Overview
. Previous lecture: Improving cache performance • Reduce hit time • Reduce miss rate • Reduce miss penalty
. Today: Virtual memory (VM) • Why do we need it? • What is it? • How does it affect the cache design?
. Reading: Appendix C.3, C.4, Chapter 5.4
ECE4750/CS4420 — Computer Architecture, Fall 2008 3
Imagine Computers w/o VM
Wireless sensor node (8-bit CPU, 4KB RAM) EDSAC, early 50’s What are the difficulties of programming such machines?
ECE4750/CS4420 — Computer Architecture, Fall 2008 4
2
Virtual Memory Concept
ECE4750/CS4420 — Computer Architecture, Fall 2008 5
Simple Base and Bound Translation
current Load X Virtual segment
Address Main Memory Main
Program Address Space
ECE4750/CS4420 — Computer Architecture, Fall 2008 6
3
Separate Areas for Program and Data
Data Bound Bounds Register Violation? data Effective Addr Load X Register segment Data Base Register +
Program Program Bound Bounds Address Register Violation? Memory Main Space Program program Counter segment Program Base Register +
What is an advantage of this separation?
ECE4750/CS4420 — Computer Architecture, Fall 2008 7
Memory Fragmentation
Users 4 & 5 Users 2 & 5 free arrive leave
user 1 16K user 1 16K user 1 16K user 2 24K user 2 24K 24K user 4 24K 16K user 4 16K 8K 8K user 3 32K user 3 32K user 3 32K
24K user 5 24K 24K
ECE4750/CS4420 — Computer Architecture, Fall 2008 8
4
Paged Memory Systems
. Processor generated address can be interpreted as a pair
1 0 0 1 2 3 3 Address Space of User-1 2 Physical Memory
ECE4750/CS4420 — Computer Architecture, Fall 2008 9
Private Address Space per User
OS User 1 VA1 pages
Page Table
Physical Memory
User 2 VA1
Page Table
User 3 VA1
Page Table free
ECE4750/CS4420 — Computer Architecture, Fall 2008 10
5
A Problem in Early Sixties
. There were many applications whose data could not fit in the main memory, e.g., payroll 40k bits • Paged memory system reduced fragmentation but still required the whole program to be resident in main the main memory
640k bits . Manual overlay: programmer keeps track of addresses in the main memory and initiates an drum I/O transfer when required Central Store Ferranti Mercury 1956 . Problems?
ECE4750/CS4420 — Computer Architecture, Fall 2008 11
Demand Paging in Atlas (1962)
“A page from secondary storage is brought into the primary storage whenever it is (implicitly) demanded by the processor.” Tom Kilburn Primary 32 Pages 512 words/page
Primary memory as a cache for secondary memory Secondary Central (Drum) User sees 32 x 6 x 512 words Memory 32x6 pages of storage
ECE4750/CS4420 — Computer Architecture, Fall 2008 12
6
Atlas Demand Paging Scheme
. Page Address Register (PAR, 32 entries) . On a memory access, compare the effective (virtual) page address against all 32 PAR entries • Match: normal access • No match: page fault . On a page fault: • If a free page is left – Transfer into a free page is initiated – The Page Address Register (PAR) is updated • If no free page is left, a page is selected to be replaced (based on usage) – The replaced page is written on the drum • to minimize drum latency effect, the first empty page on the drum was selected – The page table is updated to point to the new location of the page on the drum
ECE4750/CS4420 — Computer Architecture, Fall 2008 13
Caching vs. Demand Paging
secondary memory
primary primary CPU cache CPU memory memory
Caching Demand paging
ECE4750/CS4420 — Computer Architecture, Fall 2008 14
7
Modern Virtual Memory Systems Illusion of a large, private, uniform store
OS
useri
Swapping Store Primary Memory
VA mapping PA TLB ECE4750/CS4420 — Computer Architecture, Fall 2008 15
Linear Page Table
Data Pages . Page Table Entry (PTE) contains: Page Table PPN • A bit to indicate if a page exists PPN DPN • PPN (physical page number) for PPN a memory-resident page Data word • DPN (disk page number) for a Offset page on the disk • Status bits for protection and DPN PPN usage PPN DPN DPN . OS sets the Page Table Base VPN Register whenever active user DPN process changes PPN PPN
PT Base Register VPN Offset Virtual address ECE4750/CS4420 — Computer Architecture, Fall 2008 16
8
Size of Linear Page Table
. With 32-bit addresses, 4-KB pages & 4-byte PTEs:
. What about 64-bit virtual address space???
. Larger pages?
ECE4750/CS4420 — Computer Architecture, Fall 2008 17
Hierarchical Page Table
Virtual Address
offset Root of the Current Page Table p2 p1
(Processor Level 1 Register) Page Table
page in primary memory Level 2 page in secondary memory Page Tables
PTE of a nonexistent page Data Pages
ECE4750/CS4420 — Computer Architecture, Fall 2008 18
9
Where Should Page Tables Reside?
. Space required by the page tables (PT) is proportional to the address space, number of users, ...
ECE4750/CS4420 — Computer Architecture, Fall 2008 19
Page Tables in Physical Memory
PT User 1
VA1 PT User 2 User 1
VA1
User 2
ECE4750/CS4420 — Computer Architecture, Fall 2008 20
10
Address Translation & Protection
Virtual Address Virtual Page No. (VPN) offset Kernel/User Mode
Read/Write Protection Address Check Translation
Exception? Physical Address Physical Page No. (PPN) offset
• Every instruction and data access needs address translation and protection checks
A good VM design needs to be fast (~ one cycle) and space efficient
ECE4750/CS4420 — Computer Architecture, Fall 2008 21
Translation Lookaside Buffers
Address translation is very expensive!
ECE4750/CS4420 — Computer Architecture, Fall 2008 22
11
TLB Designs
. Typically 32-128 entries, usually fully associative • Each entry maps a large page, hence less spatial locality across pages more likely that two entries conflict • Sometimes larger TLBs (256-512 entries) are 4-8 way set-associative . Random or FIFO replacement policy . No process information in TLB? . TLB Reach: Size of largest virtual address space that can be simultaneously mapped by TLB
Example: 64 TLB entries, 4KB pages, one page per entry
TLB Reach = ______?
ECE4750/CS4420 — Computer Architecture, Fall 2008 23
Handling a TLB Miss
Software (MIPS, Alpha) TLB miss causes an exception and the operating system walks the page tables and reloads TLB. A privileged “untranslated” addressing mode used for walk
Hardware (SPARC v8, x86, PowerPC) A memory management unit (MMU) walks the page tables and reloads the TLB
If a missing (data or PT) page is encountered during the TLB reloading, MMU gives up and signals a Page-Fault exception for the original instruction
ECE4750/CS4420 — Computer Architecture, Fall 2008 24
12
Address Translation: putting it all together Virtual Address
TLB Lookup
ECE4750/CS4420 — Computer Architecture, Fall 2008 25
Address Translation in CPU pipeline
Inst Inst. Data Data Decode PC TLB Cache D E + M TLB Cache W
TLB miss? Page Fault? TLB miss? Page Fault? Protection violation? Protection violation? . Software handlers need a restartable exception on page fault or protection violation . How does a TLB affect the performance?
ECE4750/CS4420 — Computer Architecture, Fall 2008 26
13
Virtual Address Caches
PA VA Physical Primary CPU TLB Cache Memory
Alternative: place the cache before the TLB
Primary CPU Memory (StrongARM)
ECE4750/CS4420 — Computer Architecture, Fall 2008 27
Aliasing in Virtual-Address Caches
Page Table Tag Data
VA1 Data Pages VA1
PA VA2
VA2
Two virtual pages share one physical page
General Solution: Software (i.e., OS) solution for direct-mapped cache VAs of shared pages must agree in cache index bits; this ensures all VAs accessing same PA will conflict in direct- mapped cache (early SPARCs)
ECE4750/CS4420 — Computer Architecture, Fall 2008 28
14
Concurrent Access to TLB & Cache
VA VPN
TLB k Direct-map Cache 2L blocks 2b-byte block PA PPN Page Offset
ECE4750/CS4420 — Computer Architecture, Fall 2008 29
Concurrent Access & Large L1
Virtual Index L1 PA cache VA VPN a Page Offset b Direct-map
VA PPN Data TLB 1 a
VA2 PPNa Data
PA PPN Page Offset b = hit? Tag
ECE4750/CS4420 — Computer Architecture, Fall 2008 30
15
Virtual-Index Physical-Tag Caches: Associative Organization
Virtual 2a VA VPN a L = k-b b Index
Direct-map Direct-map k TLB 2L blocks 2L blocks Phy. PA PPN Page Offset Tag = = Tag hit? 2a
Data
ECE4750/CS4420 — Computer Architecture, Fall 2008 31
A Solution via Second Level Cache
L1 Instruction Memory Cache Unified L2 Memory CPU Cache Memory L1 Data RF Cache Memory
Usually a common L2 cache backs up both Instruction and Data L1 caches
L2 is “inclusive” of both Instruction and Data caches
ECE4750/CS4420 — Computer Architecture, Fall 2008 32
16
Anti-Aliasing Using L2: MIPS R10000
Virtual Index L1 cache: VA VPN a Page Offset b Physical tag
TLB
PA PPN Page Offset b PPN = hit? Tag
L2 cache
ECE4750/CS4420 — Computer Architecture, Fall 2008 33
Virtually-Addressed L1: Anti-Aliasing using L2
Virtual VA VPN Page Offset b Index & Tag
VA1 Data TLB VA2 Data
PA PPN Page Offset b L1 VA Cache
“Virtual Tag Physical Tag” Index & Tag
PA VA1 Data Physically-addressed L2 can also be used to avoid aliases in virtually- L2 PA Cache addressed L1 L2 “contains” L1
ECE4750/CS4420 — Computer Architecture, Fall 2008 34
17 Virtual Memory Use Today - 1
. Desktops/servers have full demand-paged virtual memory • Portability between machines with different memory sizes • Protection between multiple users or multiple tasks • Share small physical memory among active tasks • Simplifies implementation of some OS features
. Vector supercomputers have translation and protection but not demand- paging (Older Crays: base&bound, Japanese & Cray X1: pages) • Don’t waste expensive CPU time thrashing to disk (make jobs fit in memory) • Mostly run in batch mode (run set of jobs that fits in memory) • Difficult to implement restartable vector instructions
ECE4750/CS4420 — Computer Architecture, Fall 2008 35
Virtual Memory Use Today - 2
. Most embedded processors and DSPs provide physical addressing only • Can’t afford area/speed/power budget for virtual memory support • Often there is no secondary storage to swap to! • Programs custom written for particular memory configuration in product • Difficult to implement restartable instructions for exposed architectures
Given the software demands of modern embedded devices (e.g., cell phones, PDAs) all this is changing!
ECE4750/CS4420 — Computer Architecture, Fall 2008 36
18