Topic 8: Memory Management

Home , Overlay (programming), Segmentation fault

Tongping Liu

University of Massachusetts Amherst 1 Objectives • To provide various ways of organizing memory hardware

• To discuss various memory management techniques, including partitions, swapping and paging

• To provide detailed description for paging and virtual memory techniques

University of Massachusetts Amherst 2 Outline

• Background • Segmentation • Swapping • Paging System

University of Massachusetts Amherst 3 Memory Hierarchy

n CPU can directly access main CPU memory and registers only, so (Processor/ALU) programs and data must be Internal brought from disk into memory Memory n Memory access is bottleneck l Cache between memory and registers n Memory Hierarchy I/O l Cache: small, fast, expensive; SRAM; Devices (disks) l Main memory: medium-speed, not that expensive; DRAM l Disk: many gigabytes, slow, cheap, non-volatile storage

University of Massachusetts Amherst 4 Main Memory

• The ideal main memory is – Very large – Very fast – Non-volatile (doesn’t go away when power is turned off) • The real main memory is – Not very large – Not very fast – Affordable (cost) ! Þ Pick any two… • Memory management: make the real world look as much like the ideal world as possible J

Department of Computer Science @ UTSA 5 University of Massachusetts Amherst 5 Background about Memory

• Memory is an array of words containing program instructions and data • How do we execute a program? Fetch an instruction à decode à may fetch operands à execute à may store results • Memory hardware sees a stream of ADDRESSES

University of Massachusetts Amherst 6 Simple Memory Management

• Management for a single process – Memory can be divided simply

0xFFFF Device drivers Operating system (ROM) User program (ROM) (RAM) User program User program (RAM) Operating system (RAM) Operating system (RAM) 0 (RAM)

• Memory protection may not be an issue (only one program) • Flexibility may still be useful (allow OS changes, etc.)

ChapterUniversity 4 of Massachusetts Amherst 7 7 Real Memory Management

• Multiple processes need 1 CPU 0.9 – 0.8 More processes will 0.7

increase CPU utilization 0.6 – At 20% I/O wait, 3–4 0.5 0.4 processes fully utilize CPU Utilization CPU 0.3 – 0.2 At 80% I/O wait, even 0.1

10 processes aren’t 0 enough 0 1 2 3 4 5 6 7 8 9 10 Degree of Multiprogramming

• More processes => 80% I/O Wait 50% I/O Wait 20% I/O Wait memory management

University of Massachusetts Amherst 8 Purpose of Memory Management

• Keeping multiple processes in memory is essential to improve the CPU utilization • Manage and protect main memory while sharing it among multiple processes with these requirements: – Relocation – Protection – Sharing

University of Massachusetts Amherst 9 Memory Management

• Relocation: – Assign load addresses for position-dependent code and data of a program

• Protection: – Each program cannot access others’ memory, only accesses locations that have been allocated to it.

• Sharing: – Allow several processes to access the same memory

University of Massachusetts Amherst 10 Partitioning

• Partition: divide memory to partitions, and dynamically assign to different processes – Fixed Partitioning – Dynamic Partitioning – Segmentation – Paging

University of Massachusetts Amherst 11 Fixed Partitioning

• Equal-size partitions – Any process whose size is less than or equal to the partition size can be loaded into an available partition • The operating system can swap a process out of a partition

University of Massachusetts Amherst 12 Fixed Partitioning Problems

• A program may not fit in a partition. – The programmer must design the program with overlays • Main memory use is inefficient. – Any program, no matter how small, occupies an entire partition. – This results in internal fragmentation.

University of Massachusetts Amherst 13 Dynamic Partitioning

• Unequal-size partitioning • Reduce both problems – but doesn’t solve completely • In right figure, – Smaller programs can be placed in smaller partitions, reducing internal fragmentation – Programs larger than 16M cannot be accommodated without overlay

University of Massachusetts Amherst 14 Dynamic Partitioning

• Assign each process to the smallest partition – Minimize internal fragmentation • Separate input queues for each partition • Single input queue

900K 900K Partition 4 Partition 4 700K 700K Partition 3 Partition 3 600K 600K Partition 2 Partition 2 500K 500K Partition 1 Partition 1 100K 100K OS OS 0 0

University of Massachusetts Amherst 15 Memory Management Policies

• Best-fit algorithm – Chooses the block that is closest in size to the request – Worst performer overall – Since smallest block is found for process, the smallest amount of fragmentation is left – Memory compaction must be done more often

University of Massachusetts Amherst 16 Memory Management Policies

• First-fit algorithm – Scans memory form the beginning and chooses the first available block that is large enough – Fastest – May have many process loaded in the front end of memory that must be searched over when trying to find a free block

University of Massachusetts Amherst 17 Memory Management Policies

• Next-fit – Scans memory from the location of the last placement – More often allocate a block of memory at the end of memory where the largest block is found – The largest block of memory is broken up into smaller blocks – Compaction is required to obtain a large block at the end of memory

University of Massachusetts Amherst 18 Protection via Base and Limit Registers

• A pair of base and limit registers define the logic address range of a process. Every memory access is checked by hardware to ensure the correctness

University of Massachusetts Amherst 19 Dynamic Partitioning

• Typically, a process is putted into a partition – Which is not very convenient

• Segmentation aims to solve this issue

University of Massachusetts Amherst 20 Outline

• Background • Segmentation • Swapping • Paging System

University of Massachusetts Amherst 21 Segmentation

• Segmentation is a technique for breaking memory up into logical pieces • Each “piece” is a group of related information – data segments for each process – code segments for each process – data segments for the OS…… • One segment is continuous both virtually and physically • Segmentation uses logic addresses

University of Massachusetts Amherst 22 Segmentation View

P1 data P2 code print function P2 data P1 code

OS Code OS data OS stack

Logical Address Space

University of Massachusetts Amherst 23 Difference from Dynamic Partitioning

• Segmentation is similar to dynamic partitioning – Suffer from external fragmentation • Difference: – With segmentation, a program may occupy more than one partition, and these partitions need not be contiguous. – Eliminates internal fragmentation

University of Massachusetts Amherst 24 Logical vs. Physical Address

• Logical address – Generated by the CPU, and is used as a reference to access the physical memory location by CPU – Logic Address is called as virtual address on a system with virtual memory • Logical Address Space – All logical addresses generated by a program’s perspective. • Physical address – Address sent to the RAM (or ROM, or IO) for a read or write operation

University of Massachusetts Amherst 25 Address Binding

Mapping from one address space to another • Compile time: If memory of a process is known at compile time, absolute address will be generated; Must recompile if starting location changes (e.g., DOS programs) • Load time: Compiler generates relocatable code, loader translates them to absolute addresses typically by added to base address • Execution time: Binding is delayed until execution time, if a process can be moved during its execution from one memory segment to another (e.g., dynamic library)

University of Massachusetts Amherst 26 Address Binding of Linux

• Commonly referred to as symbol (or pointer) relocation – At compile time (external functions and variables are copied into a target application by a compiler or linker) – At load time (when the dynamic linker resolves symbols in a shared library) – At execution time (when the running program resolves symbols manually, e.g., using dlopen).

University of Massachusetts Amherst 27 Logical vs. Physical Address

• Logical and physical addresses are the same in compile-time address-binding schemes • Logical (virtual) and physical addresses differ in execution-time address-binding scheme – The mapping form logical address to physical address is done by a hardware called memory management unit (MMU).

Translator Physical User process Virtual (MMU) Physical memory address address

University of Massachusetts Amherst 28 Why Virtual Memory?

N • Basic idea: allow OS to allocate 2 more memory than the real Auxiliary regions • Program uses virtual addresses – Addresses local to the process Stack – Limited by # of bits in address (32/64) – 32 bits: 4G Heap • Virtual memory >> physical Text memory 0

Department of Computer Science @ UTSA 29 University of Massachusetts Amherst 29 Motivations for Virtual Memory

1. Use physical DRAM as the cache for disk 2. Simplify memory management – Multiple processes resident in main memory • Each with its own address space – Only “active” code and data is actually in memory 3. Provide protection – One process can’t interfere with another except via IPC • They operate in different address spaces – User process cannot access privileged information • Different sections of address spaces have different permissions

University of Massachusetts Amherst 30 Virtual Memory for Multiprogramming

• Virtual memory (VM) is helpful in multiprogramming – Multiple processes in memory concurrently, where each process occupies a small portion of memory – CPU runs process B while process A waits for its long I/O operations ( e.g., retrieve data from disks) • Physical Memory de/allocation – Keep recently used content in physical memory – Move less recently used stuff to disk

Translator Physical User process Virtual (MMU) Physical memory address address

Department of Computer Science @ UTSA 31 University of Massachusetts Amherst 31 Simple MMU: Relocation Register

• In segmentation, relocation register maps logical (virtual) address to physical address, where user program deals with logical addresses, but not real physical addresses

University of Massachusetts Amherst 32 Segment Management

• Logical address is divided into a segment number and an offset into the segment – Segment number: index into a segment register table, with base (physical address) of each segment – Add the offset to the base to generate the physical address – Before doing this, check the offset against the limit, which is the size of the segment

University of Massachusetts Amherst 33 Segmentation Lookup

Index to segment Segment register table register table physical memory limit base segment 0 segment # offset segment 1 virtual address

segment 2 yes

University of Massachusetts Amherst 34 Segmentation

0010000000100000 + 0000001011110000 ------0010001100010000

University of Massachusetts Amherst 35 Segmentation

Segment # Base Bound Segment 0 4000 700 Code Segment 1 0 500 Data segment Segment 2 Unused Unused Stack segment Segment 3 2000 1000 Virtual memory Physical memory (3,fff) 46ff . (Stack) . (Code segment) (3,000) 4000 … Offset … Segment # (1,4ff) 2fff . (Data) . (Stack segment) (1,000) 2000 … … (0,6ff) 4ff . (Code) . (Data segment) (0,0) 0

University of Massachusetts Amherst 36 Segmentation

Segment # Base Bound Segment 0 4000 700 Code Segment 1 0 500 Data segment Segment 2 Unused Unused Stack segment Segment 3 2000 1000 Virtual memory • Not all virtual addresses are valid (3,fff) . (Stack) • Nothing in segment 2 (3,000) … • Nothing in segment 1 above 4ff (1,4ff) . (Data) • Valid = part of process’s address space (1,000) … • Accesses to invalid addresses are illegal (0,6ff) . (Code) • Hence a “segmentation fault” (0,0)

University of Massachusetts Amherst 37 Segmentation

Segment # Base Bound Segment 0 4000 700 Code Segment 1 0 500 Data segment Segment 2 Unused Unused Stack segment Segment 3 2000 1000 Virtual memory • Segments can grow (3,fff) . (Stack) • (can move to new physical location) (3,000) … • Protection (1,4ff) . (Data) • Different protections for segments (1,000) … • E.g. read-only or read-write (0,6ff) . (Code) (0,0)

University of Massachusetts Amherst 38 Segmentation

Segment # Base Bound Segment 0 4000 700 Code Segment 1 0 500 Data segment Segment 2 Unused Unused Stack segment Segment 3 2000 1000 Virtual memory • What changes on a context switch? (3,fff) . (Stack) • Contents of segment table (3,000) … • Typically small (not many segments) (1,4ff) . (Data) (1,000) … (0,6ff) . (Code) (0,0)

University of Massachusetts Amherst 39 Segmentation sharing

Segment # Base Bound Segment 0 4000 700 Code Segment 1 0 500 Data segment Segment 2 Unused Unused Stack segment Segment 3 2000 1000

Segment # Base Bound Segment 0 4000 700 Code Segment 1 4700 500 Data segment Segment 2 Unused Unused Segment 3 3000 1000 Stack segment

Virtual info Physical info

University of Massachusetts Amherst 40 Segmentation pros and cons

• Pros – Multiple areas of address space can grow separately – Easy to share parts of address space (can share code segment) • Cons – Complex memory allocation – (still have external fragmentation)

University of Massachusetts Amherst 41 Memory Fragmentation

• External Fragmentation – total memory space exists to satisfy a request, but it is not contiguous • Internal Fragmentation – allocate memory larger than requested; the size difference is called internal fragmentation (can’t be used by others) • Reduce external fragmentation – Compaction: migrate memory segments to place all free chunks together into one larger block – Compaction is possible only if relocation is dynamic, and is done at execution time

University of Massachusetts Amherst 42 Segment Allocation

• When a process arrives, allocate memory from a hole large enough to accommodate it • Operating system maintains information about: – a) allocated partitions b) free partitions (hole)

E C C C B B B B A A A D OS OS OS OS OS

University of Massachusetts Amherst 43 . Segment Allocation Consider a multi-programming environment: • Each program must be in the memory to be executed • Processes come into memory • Leave memory when execution is completed

E C C C What should we do, B B B B reject it or accept it? A A A D OS OS OS OS OS Swapping supports more processes, by swapping out an old process to the disk, e.g. B here?

University of Massachusetts Amherst 44 Outline

• Background • Segmentation • Swapping • Paging System

University of Massachusetts Amherst 45 Swapping Definition

• It is a mechanism to swap a process temporarily out of main memory to a backing store (disk), and then bring it back into memory at a later time for continued execution – Backing store: large enough to accommodate all memory for all users – Swap out, swap in typically with a priority algorithm, a lower priority process is swapped out to make the memory available for higher-priority processes

University of Massachusetts Amherst 46 Swapping Overview

University of Massachusetts Amherst 47 Swapping Overhead

• Major part of swapping time is transfer time; – Total transfer time is directly proportional to the amount of memory swapped (e.g., 10MB process/ 40MB per sec = 0.25 sec) • Swapping often requires too much swapping time to used often • Swapping is implemented on many systems (i.e., UNIX, Linux, and Windows)

University of Massachusetts Amherst 48 Thrashing vs. Swapping

Thrashing: When frequent swapping activities occur such that it is the major consumer of the CPU time, then it is effectively thrashing.

Thrashing is a state that the CPU perform 'productive' work less and 'swapping' more

University of Massachusetts Amherst 49 Reduce Thrashing

• Thrashing occurs when active processes use too much memory • Approaches to reduce thrashing: – Running fewer programs – Writing programs that use memory more efficiently – Adding RAM to the system – Increasing the swap size

University of Massachusetts Amherst 50 Reduce Thrashing

University of Massachusetts Amherst 51 Outline

• Background • Segmentation • Swapping • Paging System

University of Massachusetts Amherst 52 Paging • Paging is a page-based memory management scheme that eliminates the need for contiguous allocation of physical memory – Both physical memory and virtual address of a process is divided into equal fixed-size chunks (frames and pages) – Operating system maintains a page table to map pages to frames dynamically

University of Massachusetts Amherst 53 54 Paging System

• Virtual address 60–64K - 56–60K - – Divided into pages 52–56K - 48–52K - 44–48K 5 • Physical memory 40–44K 1 – Divided into frames 36–40K - 32–36K - 28–32K 3 7 28–32K 24–28K - 6 24–28K 20–24K - 5 20–24K • Page vs. Frame 16–20K 0 4 16–20K 12–16K - 3 12–16K – Same size block 8–12K - 2 8–12K – 4–8K 4 1 4–8K Unit of mapping/allocation 0–4K 7 0 0–4K Virtual Space Physical Memory OS manages virtual space, physical memory, and the mapping between them

University of Massachusetts Amherst 54 Paged AddressPage Mapping Translation A process process virtual address space

CODE DATA STACK

CS 111 physical memory Lecture 11 Spring 2015 Page 6 University of Massachusetts Amherst 55 Page Table for Mapping

• Each process has one page table 15 – Map page number à physical frame number Frame 6 Frame 1 • Number of Page Table Entries (PTEs) – Number of total pages in virtual space Frame 3 – Not just the pages in use Frame 0 • Page table is checked for 1 Frame 4

every address translation 0 Frame 7 – Where to store page table? Page table

56 University of Massachusetts Amherst 56 Page Table for Mapping

• Page table will be stored in the memory • Not all pages need to be mapped to frames at the same time • Not all physical frames need to be used – Actually one process may only use part of frames

University of Massachusetts Amherst 57 Page Table for Mapping

• Page table should contain all pages of virtual space ØOtherwise, some addresses can’t be accessed.

• All page table entries should be physically continuous ØProof by contradiction. If the index is not continuous, we will need another mechanism to tell us where has a hole or not

Department of Computer Science @ UTSA 58 University of Massachusetts Amherst 58 Translate Virtual to Physical Address n Suppose logical address space is 2m and page size is 2n, so the number of pages is 2m / 2 n , which is 2m-n • Virtual Address (m bits) is divided into: – Page number (p) – used as an index into a page table which contains frame number of physical memory – Page offset (d) – combined with its base address to define the physical memory address that is sent to the memory unit

page number page offset

p d

m - n n

University of Massachusetts Amherst 59 A Translation Example

• 64 KB (216) virtual memory • 32 KB (215) physical memory • 4 KB page/frame size à 12 bits as offset (d)

Page #:4bits Offset: 12 bits Virtual address: 16 bits How many virtual pages?

Frame #:3bits Offset: 12 bits Physical address: 15 bits How many physical frames?

University of Massachusetts Amherst 60 A Translation Example

• 64 KB (216) virtual memory • 32 KB (215) physical memory • 4 KB page/frame size à 12 bits as offset (d)

Page #:4bits Offset: 12 bits Virtual address: 16 bits How many virtual pages? Translation

Frame #:3bits Offset: 12 bits Physical address: 15 bits How many physical frames?

University of Massachusetts Amherst 61 Address Translation Architecture

frame number page number CPU page offset 0 p d f d 1 Virtual address . page table physical address . 0 f-1 1 f . . f+1 p-1 f+2 p f . . . . physical memory

University of Massachusetts Amherst 62 Paging

University of Massachusetts Amherst 63 Computing Physical Address

1. Compute the page number

2. Check the page table, get physical frame number

3. Compute the starting address of physical frame

4. Compute the physical address

Page size is 128 bytes (0x80)

University of Massachusetts Amherst 64 Computing Physical Address

Virtual address 0x44, offset 0x44 1. Compute the page number Page number is #0 2. Check the page table, get physical frame number Frame number is #2 3. Compute the starting address of physical frame Start Address = 2 * 0x80 = 0x100 4. Compute the physical address PhysicalAddress = 0x100 + 0x44 = 0x144 Page size is 128 bytes (0x80)

University of Massachusetts Amherst 65 Computing Physical Address

Virtual address 0x224, offset 0x24 1. Compute the page number Page number is #4 2. Check the page table, get physical frame number Frame number is #3 3. Compute the starting address of physical frame Start Address = 3 * 0x80 = 0x180 4. Compute the physical address PhysicalAddress = 0x180 + 0x24 = 0x1A4 Page size is 128 bytes (0x80)

University of Massachusetts Amherst 66 Support of Multiple Processes

• Each process has its own page table • The total pages of multiple processes can be larger than the physical pages • What if the current process adds or removes pages? – Directly update its own page table (by OS) • What if we switch from one process to another? – Load the page directory entry ("movl %0,%%cr3”) – A reload instruction flushes previously cached entries, flush_cache_mm(mm); change_all_page_tables_of(mm); flush_tlb_mm(mm);

University of Massachusetts Amherst 67 Share Pages between Multiple Processes

• Making PTEs of different processes point to the same physical page – Can be read-only or read/write sharing – Virtual addresses in different processes can be different

University of Massachusetts Amherst 68 Logical Addresses

University of Massachusetts Amherst 69 Paging

University of Massachusetts Amherst 70 Segmentation

0010000000100000 + 0000001011110000 ------0010001100010000

University of Massachusetts Amherst 71 Similarities of Paging and Segmentation

• They separate the physical address spaces of processes: – Segmentation can assign a different linear address space to each process – Paging can map the same linear address space into different physical address spaces Linux prefers paging to segmentation – Memory management is simpler when all processes use the same segment register values – Linux is designed for portability, but RISC has limited support on segmentation

University of Massachusetts Amherst 72 Diff between Paging and Segmentation

• Paging needs more memory, one entry per page, while each segment only needs two register (base + offset) • Page is fixed-size, while segment is variable-size • Size of a page is determined by hardware, while size of a segment is determined by the user • Paging may have internal fragmentation (small) but segmentation may have external fragmentation (big)

University of Massachusetts Amherst 73 Paging Fragmentation

• Internal fragmentation – Average only ½ page (half of the last one) • External fragmentation – Completely non-existent since we never carve up pages

University of Massachusetts Amherst 74 Segment Fragmentation

• Consider this scenario: – Average requested allocation is 128K – 256K fixed size segments available • For segmentation, internal fragmentation is 50% (128K of 256K used) • For paging, only the last page of an allocation is not used, with 2K on average (2K/128K = 1.5%)

University of Massachusetts Amherst 75 Is Paging Perfect?

• Page is a nice memory allocation unit – Eliminate internal and external fragmentation – Require a simple but powerful MMU • However, pages are not natural – Programs don’t think of pages – Programs operate on segments – Segments are natural chunks, e.g., code, data, stack – Each segment contains many pages

University of Massachusetts Amherst 76 Integrating Paging with Segmentation

• Programs request code, data, stack segments • Requires two levels of memory management – A virtual address space is comprised of segments, via segment based addressing – Relocation & swapping is done on a page basis • User processes see segments, while pages are invisible

University of Massachusetts Amherst 77 Segments and Pages

• A segment is a collection of pages, with continuous virtual addresses • Operations on segments: – Create/open/destroy – Map/unmap segment to/from process – Find physical frame number of each virtual page • Connection between them – Segment mapping implemented with page mapping – Page fault uses segments to find requested page

University of Massachusetts Amherst 78 Linux Support on Segments

• But both cs (code segment) and ds(data segment) base addresses are set to zero, so the segmentation is not really used • An exception is thread local data, one of the normally unused segment registers points to thread local data. But that is mainly to avoid reserving one of the general purpose registers for this task.

University of Massachusetts Amherst 79 University of Massachusetts Amherst 80 . Page/Frame Size

• Determine the number of bits in offset

• Smaller page size – + Less internal fragmentation – - Larger page table: require more than one frame, hard to allocate!

• Larger page size – + Smaller page table and less overhead to track them – + More efficient to transfer to/from disk – - More internal fragmentation, wasting memory

University of Massachusetts Amherst 81 In-Class Exercise

• For an embedded small computer supporting up to 1k bytes physical memory, where its virtual address is 12 bits. Suppose that the size of page/frame is 128 bytes. Use a figure to illustrate the physical and virtual address that shows the number of bits for offset and frame/page number for one level page table.

• What is the number of virtual pages for each process? • How many physical frames in total? • How many entries in page table for each process?

University of Massachusetts Amherst 82 In-Class Exercise

• # of virtual pages: 2^12/128= 2^5 = 32 Page #:5bits Offset: 7 bits Virtual address: 12 bits • # of physical frames

1K/128=8 Frame #:3bits Offset: 7 bits Physical address: 10 bits • # of page table entries 32

University of Massachusetts Amherst 83 Outline

• Background • Segmentation • Swapping • Paging System

University of Massachusetts Amherst 84