TCSS 422 A – Winter 2018 2/21/2018 Institute of Technology
TCSS 422: OPERATING SYSTEMS OBJECTIVES
Optional Ungraded Quiz – Synchronized Array Homework 2 Questions Memory API, Homework 3 Address Translation, Memory Segmentation, Ch. 14 Free Space Management . Memory API Ch. 15 . Address Translation Wes J. Lloyd Ch. 16 Institute of Technology . (Memory) Segmentation Ch. 17 University of Washington - Tacoma . Free Space Management
TCSS422: Operating Systems [Winter 2018] TCSS422: Operating Systems [Winter 2018] February 21, 2018 February 21, 2018 L12.2 Institute of Technology, University of Washington - Tacoma Institute of Technology, University of Washington - Tacoma
FEEDBACK FROM 2/14 FEEDBACK - 2
What is a bounded buffer, and when is it used? Homework #2: I’m using “for_each_process(task)” but Ubuntu can’t find this function… HELP!? What is piping in operating systems? In your kernel module make file, note the required files: All target: What is a condition variable? make -C /lib/modules/$(shell uname -r)/build M=$(PWD) modules
Check to be sure you have this kernel sources: sudo apt-get install build-essential linux-headers-`uname –r`
Helpful command - list kernel modules: $lsmod
TCSS422: Operating Systems [Winter 2018] TCSS422: Operating Systems [Winter 2018] February 21, 2018 L12.3 February 21, 2018 L12.4 Institute of Technology, University of Washington - Tacoma Institute of Technology, University of Washington - Tacoma
FEEDBACK - 3
“TCSS 422 has overlap with TCSS 333 for memory maps, memory/address translation”…
Initial chapters of memory virtualization may be review
However, memory virtualization spans chapters 13, 14, 15, CHAPTER 14: THE 16, 17, 18, 19, 20, 21, 22. . . (10 chapters) MEMORY API
. Suspect this is not all review…
TCSS422: Operating Systems [Winter 2018] TCSS422: Operating Systems [Winter 2018] February 21, 2018 L12.5 February 21, 2018 Institute of Technology, University of Washington - Tacoma Institute of Technology, University of Washington - Tacoma L12.6
Slides by Wes J. Lloyd L12.1 TCSS 422 A – Winter 2018 2/21/2018 Institute of Technology
MALLOC SIZEOF()
Not safe to assume data type sizes using Allocates memory on the heap different compilers, systems size_t unsigned integer (must be +) size size of memory allocation in bytes Dynamic array of 10 ints
Returns Static array of 10 ints SUCCESS: A void * to a memory address FAIL: NULL
sizeof() often used to ask the system how large a given datatype or struct is
TCSS422: Operating Systems [Winter 2018] TCSS422: Operating Systems [Winter 2018] February 21, 2018 L12.7 February 21, 2018 L12.8 Institute of Technology, University of Washington - Tacoma Institute of Technology, University of Washington - Tacoma
#include
Free memory allocated with malloc() void set_magic_number_b() { Provide: (void *) ptr to malloc’d memory int b = 11111; } Returns: nothing int main() { int * x = NULL; x = set_magic_number_a(); printf("The magic number is=%d\n“,*x); set_magic_number_b(); printf(“The magic number is=%d\n“,*x); TCSS422: Operating Systems [Winter 2018] return 0; February 21, 2018 L12.9 Institute of Technology, University of Washington - Tacoma } 10
#include
Slides by Wes J. Lloyd L12.2 TCSS 422 A – Winter 2018 2/21/2018 Institute of Technology
DANGLING POINTER (2/2) CALLOC()
Fortunately in the case, a compiler warning is generated: Allocate “C”lear memory on the heap $ g++ -o pointer_error -std=c++0x pointer_error.cpp Calloc wipes memory in advance of use… size_t num : number of blocks to allocate pointer_error.cpp: In function ‘int* set_magic_number_a()’: size_t size : size of each block(in bytes) pointer_error.cpp:6:7: warning: address of local variable ‘a’ returned [enabled by default] Calloc() prevents… char *dest = malloc(20); This is a common mistake - - - printf("dest string=%s\n", dest); accidentally referring to addresses that have gone “out of scope” dest string=��F
TCSS422: Operating Systems [Winter 2018] TCSS422: Operating Systems [Winter 2018] February 21, 2018 L12.13 February 21, 2018 L12.14 Institute of Technology, University of Washington - Tacoma Institute of Technology, University of Washington - Tacoma
REALLOC() DOUBLE FREE
Resize an existing memory allocation Can’t deallocate twice Returned pointer may be same address, or a new address Second call core dumps . New if memory allocation must move void *ptr: Pointer to memory block allocated with malloc, calloc, or realloc size_t size: New size for the memory block(in bytes)
EXAMPLE: realloc.c EXAMPLE: nom.c
TCSS422: Operating Systems [Winter 2018] TCSS422: Operating Systems [Winter 2018] February 21, 2018 L12.15 February 21, 2018 L12.16 Institute of Technology, University of Washington - Tacoma Institute of Technology, University of Washington - Tacoma
SYSTEM CALLS
brk(), sbrk()
Used to change data segment size (the end of the heap) Don’t use these CHAPTER 15: ADDRESS Mmap(), munmap() TRANSLATION Can be used to create an extra independent “heap” of memory for a user program
See man page
TCSS422: Operating Systems [Winter 2018] TCSS422: Operating Systems [Winter 2018] February 21, 2018 L12.17 February 21, 2018 Institute of Technology, University of Washington - Tacoma Institute of Technology, University of Washington - Tacoma L12.18
Slides by Wes J. Lloyd L12.3 TCSS 422 A – Winter 2018 2/21/2018 Institute of Technology
OBJECTIVES ADDRESS TRANSLATION
Virtual mapping Address translation 64KB Address space Base and bounds example
HW and OS Support Translation: mapping virtual to Memory segments physical
Memory fragmentation
Address Space TCSS422: Operating Systems [Winter 2018] TCSS422: Operating Systems [Winter 2018] February 21, 2018 L12.19 February 21, 2018 L12.20 Institute of Technology, University of Washington - Tacoma Institute of Technology, University of Washington - Tacoma
BASE AND BOUNDS INSTRUCTION EXAMPLE
Dynamic relocation Base = 32768 Two registers base & bounds: on the CPU Bounds =16384 OS places program in memory Fetch instruction at 128 (virt addr) ↑ Sets base register . Phy addr = virt addr + base reg . 32896 = 128 + 32768 (base) Execute instruction . Load from address (var x is @ 15kb=15360) Bounds register . 48128 = 15360 + 32768 (base) -- found x… . Stores size of program address space (16KB) Bounds register: terminate process if OS verifies that every address: . ACCESS VIOLATION: Virtual address > bounds reg 0≤ 푣푖푟푡푢푎푙 푎푑푑푟푒푠푠 < 푏표푢푛푑푠 Int x
TCSS422: Operating Systems [Winter 2018] TCSS422: Operating Systems [Winter 2018] February 21, 2018 L12.21 February 21, 2018 L12.22 Institute of Technology, University of Washington - Tacoma Institute of Technology, University of Washington - Tacoma
MEMORY MANAGEMENT UNIT DYNAMIC RELOCATION OF PROGRAMS
MMU Hardware requirements: . Portion of the CPU dedicated to address translation Requirements HW support . Contains base & bounds registers Privileged mode CPU modes: kernel, user Base & Bounds Example: Base / bounds registers Registers to support address translation . Consider address translation Translate virtual addr; check if in Translation circuitry, check limits . 4 KB (4096 bytes) address space, loaded at 16 KB physical location bounds Privileged instruction(s) to Instructions for modifying base/bound Virtual Address Physical Address update base / bounds regs registers 0 16384 Privileged instruction(s) Set code pointers to OS code to handle faults 1024 17408 to register exception handlers 3000 19384 Ability to raise exceptions For out-of-bounds memory access, or attempts to access privileged instr. FAULT 4400 20784 (out of bounds)
TCSS422: Operating Systems [Winter 2018] TCSS422: Operating Systems [Winter 2018] February 21, 2018 L12.23 February 21, 2018 L12.24 Institute of Technology, University of Washington - Tacoma Institute of Technology, University of Washington - Tacoma
Slides by Wes J. Lloyd L12.4 TCSS 422 A – Winter 2018 2/21/2018 Institute of Technology
OS SUPPORT FOR MEMORY OS: WHEN PROCESS STARTS RUNNING VIRTUALIZATION
For base and bounds OS support required OS searches for free space for new process . Free list: data structure that tracks available memory slots . When process starts running . Allocate address space in physical memory
. When a process is terminated . Reclaiming memory for use
. When context switch occurs . Saving and storing the base-bounds pair
. Exception handlers . Function pointers set at OS boot time
TCSS422: Operating Systems [Winter 2018] TCSS422: Operating Systems [Winter 2018] February 21, 2018 L12.25 February 21, 2018 L12.26 Institute of Technology, University of Washington - Tacoma Institute of Technology, University of Washington - Tacoma
OS: WHEN PROCESS IS TERMINATED OS: WHEN CONTEXT SWITCH OCCURS
OS places memory back on the free list OS must save base and bounds registers . Saved to the Process Control Block PCB (task_struct in Linux)
TCSS422: Operating Systems [Winter 2018] TCSS422: Operating Systems [Winter 2018] February 21, 2018 L12.27 February 21, 2018 L12.28 Institute of Technology, University of Washington - Tacoma Institute of Technology, University of Washington - Tacoma
DYNAMIC RELOCATION
OS can move process data when not running
1. OS deschedules process from scheduler 2. OS copies address space from current to new location 3. OS updates PCB (base and bounds registers) CHAPTER 16: 4. OS reschedules process SEGMENTATION When process runs new base register is restored to CPU
Process doesn’t know it was even moved!
TCSS422: Operating Systems [Winter 2018] TCSS422: Operating Systems [Winter 2018] February 21, 2018 L12.29 February 21, 2018 Institute of Technology, University of Washington - Tacoma Institute of Technology, University of Washington - Tacoma L12.30
Slides by Wes J. Lloyd L12.5 TCSS 422 A – Winter 2018 2/21/2018 Institute of Technology
BASE AND BOUNDS INEFFICIENCIES MULTIPLE SEGMENTS
Address space Memory segmentation . Contains significant unused memory . Is relatively large Address space has (3) segments . Preallocates space to handle stack/heap growth .Contiguous portions of address space Large address spaces .Logically separate segments for: code, stack, heap . Hard to fit in memory Each segment can placed separately How can these issues be addressed? Track base and bounds for each segment (registers)
TCSS422: Operating Systems [Winter 2018] TCSS422: Operating Systems [Winter 2018] February 21, 2018 L12.31 February 21, 2018 L12.32 Institute of Technology, University of Washington - Tacoma Institute of Technology, University of Washington - Tacoma
SEGMENTS IN MEMORY ADDRESS TRANSLATION: CODE SEGMENT
Consider 3 segments:
Much smaller Code segment - physically starts at 32KB (base) Starts at “0” in virtual address space
Bounds check: Is virtual address within 2KB address space?
Virtual Address Space Physical Address Space
TCSS422: Operating Systems [Winter 2018] TCSS422: Operating Systems [Winter 2018] February 21, 2018 L12.33 February 21, 2018 L12.34 Institute of Technology, University of Washington - Tacoma Institute of Technology, University of Washington - Tacoma
ADDRESS TRANSLATION: HEAP SEGMENTATION FAULT
푽풊풓풕풖풂풍 풂풅풅풓풆풔풔 + 풃풂풔풆 is not the correct physical address. Access beyond the address space Heap starts at virtual address 4096 Heap starts at virtual address: 4096 The data is at 4200 Data pointer is to 7KB (7168) Offset= 4200 – 4096 = 104 (virt addr – virt heap start) Is data pointer valid? Physical address = 104 + 34816 (offset + heap base) Heap starts at 4096 + 2 KB seg size = 6144 Offset= 7168 > 4096 + 2048 (6144)
TCSS422: Operating Systems [Winter 2018] TCSS422: Operating Systems [Winter 2018] February 21, 2018 L12.35 February 21, 2018 L12.36 Institute of Technology, University of Washington - Tacoma Institute of Technology, University of Washington - Tacoma
Slides by Wes J. Lloyd L12.6 TCSS 422 A – Winter 2018 2/21/2018 Institute of Technology
SEGMENT REGISTERS SEGMENTATION DEREFERENCE
Used to dereference memory during translation
First two bits identify segment type Remaining bits identify memory offset VIRTUAL ADDRESS = 01000001101000 (on heap) Example: virtual heap address 4200 (01000001101000) SEG_MASK = 0x3000 (11000000000000) SEG_SHIFT = 01 heap (mask gives us segment code) OFFSET_MASK = 0xFFF (00111111111111) OFFSET = 000001101000 = 104 (isolates segment offset) OFFSET < BOUNDS : 104 < 2048
TCSS422: Operating Systems [Winter 2018] TCSS422: Operating Systems [Winter 2018] February 21, 2018 L12.37 February 21, 2018 L12.38 Institute of Technology, University of Washington - Tacoma Institute of Technology, University of Washington - Tacoma
STACK SEGMENT SHARED CODE SEGMENTS
Stack grows backwards (FILO) Code sharing: enabled with HW support Requires hardware support: Supports storing shared libraries in memory only once DLL: dynamic linked library Direction bit: tracks direction segment grows .so (linux): shraed object in Linux (under /usr/lib) Many programs can access them Protection bits: track permissions to segment
TCSS422: Operating Systems [Winter 2018] TCSS422: Operating Systems [Winter 2018] February 21, 2018 L12.39 February 21, 2018 L12.40 Institute of Technology, University of Washington - Tacoma Institute of Technology, University of Washington - Tacoma
SEGMENTATION GRANULARITY SEGMENTATION GRANULARITY - 2
Coarse-grained Fine-grained Manage memory as list of segments Manage memory as large purpose Code, heap, stack segments composed based segments: of multiple smaller segments Segment table .Code segment . On early systems .Heap segment . Stored in memory .Stack segment . Tracked large number of segments
TCSS422: Operating Systems [Winter 2018] TCSS422: Operating Systems [Winter 2018] February 21, 2018 L12.41 February 21, 2018 L12.42 Institute of Technology, University of Washington - Tacoma Institute of Technology, University of Washington - Tacoma
Slides by Wes J. Lloyd L12.7 TCSS 422 A – Winter 2018 2/21/2018 Institute of Technology
MEMORY FRAGMENTATION COMPACTION
Consider how much free space? Supports rearranging memory We’ll say about 24 KB Can we fulfil the request for 20 KB of contiguous memory? Request arrives to allocate a 20 KB heap segment Drawback: Compaction is slow . Rearranging memory is time consuming Can we fulfil the request for 20 KB of . 64KB is fast contiguous memory? . 4GB+ … slow
Algorithms: . Best fit: keep list of free spaces, allocate the most snug segment for the request . Others: worst fit, first fit… (in future chapters)
TCSS422: Operating Systems [Winter 2018] TCSS422: Operating Systems [Winter 2018] February 21, 2018 L12.43 February 21, 2018 L12.44 Institute of Technology, University of Washington - Tacoma Institute of Technology, University of Washington - Tacoma
FREE SPACE MANAGEMENT
Management of memory using
Only fixed-sized units . Easy: keep a list CHAPTER 17: FREE . Memory request return first free entry SPACE MANAGEMENT . Simple search With variable sized units . More challenging . Results from variable sized malloc requests . Leads to fragmentation
TCSS422: Operating Systems [Winter 2018] TCSS422: Operating Systems [Winter 2018] February 21, 2018 February 21, 2018 L12.46 Institute of Technology, University of Washington - Tacoma L12.45 Institute of Technology, University of Washington - Tacoma
QUESTIONS
Slides by Wes J. Lloyd L12.8