DOCSLIB.ORG

Segmentation and Paging CS 332 S20

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Segmentation and Paging CS 332 S20 Segmentation and Paging CS 332 s20 Announcements ● Lab 3 due 9pm, email me to use late days ○ No late days on lab 4 (deadline is last day of finals) ● Lab 4 available soon (you’ll be implementing some core memory functionality) ○ Design doc due May 27 ○ Peer review due June 1 (same groups) ○ Lab due June 8 ● Week 7 quiz bumped to Monday, due 9pm May 29 (Thursday) ● Mea culpa for being behind on grading Base and Bound: Simple, Limited Virtual Memory ● Pros: ○ Simple, fast, safe ○ Can relocate physical memory without changing process ● Cons: ○ Can’t prevent process from overwriting its own code (only check exceeding bound) ○ Can’t share code/data with other processes ○ Can’t grow stack/heap as needed Segmentation: Multiple Bases and Bounds to Divide Memory Into Segments ● Segment is a contiguous region of virtual memory ● Each process has a segment table (stored in hardware) ○ Each entry in the table defines a segment ● Segment can be located anywhere in physical memory ○ Each segment has: start (base), length (bound), access permissions ● Just as with base and bound, a memory access to a segment exceeding the bound triggers an exception ○ On UNIX systems, called a segmentation fault Common theme: virtual address split into index and offset Table value and offset concatenated to form physical address 00 01 Maximum segment length? 10 11 212 segment local address Use Segmentation to Avoid Unnecessary Copying for fork: Copy-on-Write ● UNIX fork ○ Makes a complete copy of a process ○ If child never uses parent’s memory (e.g., just calls exec), a lot of unnecessary copying ● Segments allow a more efficient implementation ○ Copy segment table into child ○ Mark parent and child segments read-only ○ Start child process; return to parent ○ If child or parent writes to a segment (ex: stack, heap) ■ Trap into kernel ■ Make a copy of the segment, change permissions, and resume Segmentation Facilitates Sharing ● Same base, bound, same/different access permissions ● For example, two instances of the same program sharing a code segment Use Segmentation to Efficiently Manage Dynamically Allocated Memory: Zero-on-Reference ● Need to zero out contents of memory before allocation ○ Avoid accidentally leaking information! ○ But only want to bother if memory will actually be used ● How much physical memory is needed for the stack or heap? ○ Only what is currently in use (could be anywhere from KBs to GBs) ○ Reserve larger space, but only zero the first few KB, set bound to zeroed portion ● When program uses memory beyond end of stack or expands the heap ○ Segmentation fault into OS kernel ○ Kernel allocates some memory (extends bound) ○ Zeros the memory ○ Modifies the segment table ○ Resumes the process Segmentation: Big Improvement Over Simple Translation, Introduces Complexity ● Pros? ○ Can share code/data segments between processes ○ Can protect code segment from being overwritten ○ Can transparently grow stack/heap as needed ○ Can detect if need to copy-on-write ● Cons? ○ Complex memory management ■ Need to find chunk of a particular size (e.g., some programs have small data segment, some large) ○ May need to rearrange memory from time to time to make room for new segment or growing segment ■ Compacting free memory can create a lot of processor overhead ■ External fragmentation: wasted space between chunks Paging: Divide Memory Into Fixed-Size Pages ● Manage physical memory in fixed size units, or pages ● Finding a free page is easy ○ Bitmap allocation: 0011111100000001100 ○ Each bit represents one physical page frame, indicates free or allocated ■ Bitmaps also used in file systems to represent free disk blocks (also fixed-size units) ● Each process has its own page table ○ Stored in physical memory ● Hardware registers ○ Pointer to page table start ○ Page table length Program Memory May Be Scattered Throughout Physical Memory ● Each process sees its virtual address space as a neatly ordered deck of cards ● Physical memory is all the decks from all the processes shuffled together Virtual Address Used to Look Up Physical Address ● Again, part of the virtual address is an index, and part is an offset ○ Since offset is lower-order bits, specifies location within contiguous virtual page table entry page or physical frame (PTE) ● Page table contains access permissions like segment table Virtual Page Number Frame Number 0 0 1 1 2 2 3 3 4 5 6 7 Paging and Copy on Write ● Can we share memory between processes? ○ Set entries in both page tables to point to same page frames ○ Need core map of page frames to track which processes are pointing to which page frames (e.g., reference count) ● UNIX fork with copy on write ○ Copy page table of parent into child process ○ Mark all pages (in new and old page tables) as read-only ○ Trap into kernel on write (in child or parent) ○ Copy page ○ Mark both as writeable ○ Resume execution Get a Head Start: Demand Paging ● Can I start running a program before its code is in physical memory? ● Set all page table entries to invalid ● When a page is referenced for first time, kernel trap ● Kernel brings page in from disk ○ Page in on demand ● Resume execution ● Remaining pages can be transferred in the background while program is running Paging: Simplifies Physical Memory Management, Introduces Other Overhead ● Pros: ○ Fixed-size pages avoid external fragmentation (can always use free memory) ○ Finding a free page is easy, no need to rearrange to make room ○ Flexible, enables sparse address spaces ● Cons: ○ Page table is in memory, so every memory access requires an extra memory access to first retrieve the page table entry ○ What if virtual address space is large? ■ Need a page table entry for each virtual page ■ 32-bits, 4KB pages => 500K page table entries ■ 64-bits => 4 quadrillion page table entries ○ Reduce page table size with larger pages? ■ Internal fragmentation: if we don’t need all of the space inside a fixed size chunk Reading: Introduction to Paging OSTEP Chapter 18 (p. 197–208) Provides a concrete introduction to paging with examples.
Load more

Recommended publications
  • University of California at Berkeley College of Engineering Department of Electrical Engineering and Computer Science
    University of California at Berkeley College of Engineering Department of Electrical Engineering and Computer Science EECS 61C, Fall 2003 Lab 2: Strings and pointers; the GDB debugger PRELIMINARY VERSION Goals To learn to use the gdb debugger to debug string and pointer programs in C. Reading Sections 5.1-5.5, in K&R GDB Reference Card (linked to class page under “resources.”) Optional: Complete GDB documentation (http://www.gnu.org/manual/gdb-5.1.1/gdb.html) Note: GDB currently only works on the following machines: • torus.cs.berkeley.edu • rhombus.cs.berkeley.edu • pentagon.cs.berkeley.edu Please ssh into one of these machines before starting the lab. Basic tasks in GDB There are two ways to start the debugger: 1. In EMACS, type M-x gdb, then type gdb <filename> 2. Run gdb <filename> from the command line The following are fundamental operations in gdb. Please make sure you know the gdb commands for the following operations before you proceed. 1. How do you run a program in gdb? 2. How do you pass arguments to a program when using gdb? 3. How do you set a breakpoint in a program? 4. How do you set a breakpoint which which only occurs when a set of conditions is true (eg when certain variables are a certain value)? 5. How do you execute the next line of C code in the program after a break? 1 6. If the next line is a function call, you'll execute the call in one step. How do you execute the C code, line by line, inside the function call? 7.
    [Show full text]
  • Compiling and Debugging Basics
    Compiling and Debugging Basics Service CoSiNus IMFT P. Elyakime H. Neau A. Pedrono A. Stoukov Avril 2015 Outline ● Compilers available at IMFT? (Fortran, C and C++) ● Good practices ● Debugging Why? Compilation errors and warning Run time errors and wrong results Fortran specificities C/C++ specificities ● Basic introduction to gdb, valgrind and TotalView IMFT - CoSiNus 2 Compilers on linux platforms ● Gnu compilers: gcc, g++, gfortran ● Intel compilers ( 2 licenses INPT): icc, icpc, ifort ● PGI compiler fortran only (2 licenses INPT): pgf77, pgf90 ● Wrappers mpich2 for MPI codes: mpicc, mpicxx, mpif90 IMFT - CoSiNus 3 Installation ● Gnu compilers: included in linux package (Ubuntu 12.04 LTS, gcc/gfortran version 4.6.3) ● Intel and PGI compilers installed on a centralized server (/PRODCOM), to use it: source /PRODCOM/bin/config.sh # in bash source /PRODCOM/bin/config.csh # in csh/tcsh ● Wrappers mpich2 installed on PRODCOM: FORTRAN : mympi intel # (or pgi or gnu) C/C++ : mympi intel # (or gnu) IMFT - CoSiNus 4 Good practices • Avoid too long source files! • Use a makefile if you have more than one file to compile • In Fortran : ” implicit none” mandatory at the beginning of each program, module and subroutine! • Use compiler’s check options IMFT - CoSiNus 5 Why talk about debugging ? Yesterday, my program was running well: % gfortran myprog.f90 % ./a.out % vmax= 3.3e-2 And today: % gfortran myprog.f90 % ./a.out % Segmentation fault Yet I have not changed anything… Because black magic is not the reason most often, debugging could be helpful! (If you really think that the cause of your problem is evil, no need to apply to CoSiNus, we are not God!) IMFT - CoSiNus 6 Debugging Methodical process to find and fix flows in a code.
    [Show full text]
  • Lecture 15 15.1 Paging
    CMPSCI 377 Operating Systems Fall 2009 Lecture 15 Lecturer: Emery Berger Scribe: Bruno Silva,Jim Partan 15.1 Paging In recent lectures, we have been discussing virtual memory. The valid addresses in a process' virtual address space correspond to actual data or code somewhere in the system, either in physical memory or on the disk. Since physical memory is fast and is a limited resource, we use the physical memory as a cache for the disk (another way of saying this is that the physical memory is \backed by" the disk, just as the L1 cache is \backed by" the L2 cache). Just as with any cache, we need to specify our policies for when to read a page into physical memory, when to evict a page from physical memory, and when to write a page from physical memory back to the disk. 15.1.1 Reading Pages into Physical Memory For reading, most operating systems use demand paging. This means that pages are only read from the disk into physical memory when they are needed. In the page table, there is a resident status bit, which says whether or not a valid page resides in physical memory. If the MMU tries to get a physical page number for a valid page which is not resident in physical memory, it issues a pagefault to the operating system. The OS then loads that page from disk, and then returns to the MMU to finish the translation.1 In addition, many operating systems make some use of pre-fetching, which is called pre-paging when used for pages.
    [Show full text]
  • NASM for Linux
    1 NASM for Linux Microprocessors II 2 NASM for Linux Microprocessors II NASM Package nasm package available as source or as executables Typically /usr/bin/nasm and /usr/bin/ndisasm Assembly NASM Linux requires elf format for object files ELF = Executable and Linking Format Typical header size = 330h bytes for nasm −f elf [−o <output>] <filename> Linking Linux Object files can be linked with gcc gcc [−options] <filename.o> [other_files.o] Disassembly View executable as 32-bit assembly code ndisasm −e 330h –b 32 a.out | less objdump –d a.out | less Fall 2007 Hadassah College Dr. Martin Land Fall 2007 Hadassah College Dr. Martin Land 3 NASM for Linux Microprocessors II 4 NASM for Linux Microprocessors II gcc Stages Example — 1 Stages of Gnu C compilation factorial2.c #include <math.h> main #include <stdio.h> sets j = 12 main() Source Translation Assembly Object Executable calls factorial 10,000,000 times Code Unit Code Code File { int times; prog.c prog.i prog.s prog.o a.out int i , j = 12; preprocess compile assemble link for (times = 0 ; times < 10000000 ; ++times){ i = factorial(j); gcc -E } gcc -S printf("%d\n",i); gcc -c } gcc int factorial(n) int n; factorial calculates n! by recursion { if (n == 0) return 1; else return n * factorial(n-1); } Fall 2007 Hadassah College Dr. Martin Land Fall 2007 Hadassah College Dr. Martin Land 5 NASM for Linux Microprocessors II 6 NASM for Linux Microprocessors II Example — 2 Example — 3 ~/gcc$ gcc factorial2.c Compile program as separate files produces executable a.out factorial2a.c ~/gcc$ time a.out main() { 479001600 int times; int i,j=12; for (times = 0 ; times < 10000000 ; ++times){ real 0m9.281s i = factorial(j); factorial2b.c } #include <math.h> printf("%d\n",i); user 0m8.339s #include <stdio.h> } sys 0m0.008s int factorial(n) int n; { Program a.out runs in 8.339 seconds on 300 MHz if (n == 0) Pentium II return 1; else return n * factorial(n-1); } Fall 2007 Hadassah College Dr.
    [Show full text]
  • Memory Management
    Memory management Virtual address space ● Each process in a multi-tasking OS runs in its own memory sandbox called the virtual address space. ● In 32-bit mode this is a 4GB block of memory addresses. ● These virtual addresses are mapped to physical memory by page tables, which are maintained by the operating system kernel and consulted by the processor. ● Each process has its own set of page tables. ● Once virtual addresses are enabled, they apply to all software running in the machine, including the kernel itself. ● Thus a portion of the virtual address space must be reserved to the kernel Kernel and user space ● Kernel might not use 1 GB much physical memory. ● It has that portion of address space available to map whatever physical memory it wishes. ● Kernel space is flagged in the page tables as exclusive to privileged code (ring 2 or lower), hence a page fault is triggered if user-mode programs try to touch it. ● In Linux, kernel space is constantly present and maps the same physical memory in all processes. ● Kernel code and data are always addressable, ready to handle interrupts or system calls at any time. ● By contrast, the mapping for the user-mode portion of the address space changes whenever a process switch happens Kernel virtual address space ● Kernel address space is the area above CONFIG_PAGE_OFFSET. ● For 32-bit, this is configurable at kernel build time. The kernel can be given a different amount of address space as desired. ● Two kinds of addresses in kernel virtual address space – Kernel logical address – Kernel virtual address Kernel logical address ● Allocated with kmalloc() ● Holds all the kernel data structures ● Can never be swapped out ● Virtual addresses are a fixed offset from their physical addresses.
    [Show full text]
  • ENCM 335 Fall 2018 Lab 3 for the Week of October 1
    page 1 of 11 ENCM 335 Fall 2018 Lab 3 for the Week of October 1 Steve Norman Department of Electrical & Computer Engineering University of Calgary September 2018 Lab instructions and other documents for ENCM 335 can be found at https://people.ucalgary.ca/~norman/encm335fall2018/ Administrative details Each student must hand in their own assignment Later in the course, you may be allowed to work in pairs on some assignments. Due Dates The Due Date for this assignment is 3:30pm Friday, October 5. The Late Due Date is 3:30pm Tuesday, October 9 (not Monday the 8th, because that is a holiday). The penalty for handing in an assignment after the Due Date but before the Late Due Date is 3 marks. In other words, X/Y becomes (X{3)/Y if the assignment is late. There will be no credit for assignments turned in after the Late Due Date; they will be returned unmarked. Marking scheme A 4 marks B 8 marks C unmarked D 2 marks E 8 marks F 2 marks G 4 marks total 28 marks How to package and hand in your assignments Please see the information in the Lab 1 instructions. Function interface comments, continued For Lab 2, you were asked to read a document called \Function Interface Com- ments". ENCM 335 Fall 2018 Lab 3 page 2 of 11 Figure 1: Sketch of a program with a function to find the average element value of an array of ints. The function can be given only one argument, and the function is supposed to work correctly for whatever number of elements the array has.
    [Show full text]
  • Theory of Operating Systems
    Exam Review ● booting ● I/O hardware, DMA, I/O software ● device drivers ● virtual memory 1 booting ● hardware is configured to execute a program in Read-Only Memory (ROM) or flash memory: – the BIOS, basic I/O system – UEFI is the current equivalent ● BIOS knows how to access all the disk drives, chooses one to boot (perhaps with user assistance), loads the first sector (512 bytes) into memory, and starts to execute it (jmp) – first sector often includes a partition table 2 I/O hardware and DMA ● electronics, and sometimes moving parts, e.g. for disks or printers ● status registers and control registers read and set by CPU software – registers can directly control hardware, or be read and set by the device controller ● device controller can be instructed to do Direct Memory Access to transfer data to and from the CPU's memory – DMA typically uses physical addresses 3 Structure of I/O software ● user programs request I/O: read/write, send/recv, etc – daemons and servers work autonomously ● device-independent software converts the request to a device-dependent operation, and also handles requests from device drivers – e.g file systems and protocol stacks – e.g. servers in Minix ● one device driver may manage multiple devices – and handles interrupts ● buffer management required! 4 Device Drivers ● configure the device or device controller – i.e. must know specifics about the hardware ● respond to I/O requests from higher-level software: read, write, ioctl ● respond to interrupts, usually by reading status registers, writing to control registers, and transferring data (either via DMA, or by reading and writing data registers) 5 Memory Management ● linear array of memory locations ● memory is either divided into fixed-sized units (e.g.
    [Show full text]
  • Theory of Operating Systems
    Exam Review ● booting ● I/O hardware, DMA, I/O software ● device drivers ● memory (i.e. address space) management ● virtual memory 1 booting ● hardware is configured to execute a program in Read-Only Memory (ROM) or flash memory: – the BIOS, basic I/O system – UEFI is the current equivalent ● BIOS knows how to access all the disk drives, chooses one to boot (perhaps with user assistance), loads the first sector (512 bytes) into memory, and starts to execute it (jmp) – first sector often includes a partition table 2 I/O hardware and DMA ● electronics, and sometimes (disks, printers) moving parts ● status registers and control registers read and set by CPU software – registers can directly control hardware, or be read and set by the device controller ● device controller can be instructed to do Direct Memory Access to transfer data to and from the CPU's memory – DMA typically uses physical addresses 3 Structure of I/O software ● user programs request I/O: read/write, send/recv, etc – daemons and servers work autonomously ● device-independent software converts the request to a device-dependent operation, and also handles requests from device drivers – e.g file systems and protocol stacks – e.g. servers in Minix ● one device driver may manage multiple devices – and handles interrupts ● buffer management required! 4 Device Drivers ● configure the device or device controller – i.e. must know specifics about the hardware ● respond to I/O requests from higher-level software: read, write, ioctl ● respond to interrupts, usually by reading status registers, writing to control registers, and transferring data (either via DMA, or by reading and writing data registers) 5 Memory Management ● linear array of memory locations ● memory is either divided into fixed-sized units (e.g.
    [Show full text]
  • A Study on Faults and Error Propagation in the Linux Operating System
    A Thesis for the Degree of Ph.D. in Engineering A Study on Faults and Error Propagation in the Linux Operating System March 2016 Graduate School of Science and Technology Keio University Takeshi Yoshimura Acknowledgement I would like to thank my adviser, Prof. Kenji Kono. His guidance helped me in all the time of research. I would like to express my sincere gratitude to Prof. Hiroshi Yamada. This dissertation would not have been possible without their advice and encouragement. I am also grateful to the members of my thesis committee: Prof. Shingo Takada, Prof. Hiroaki Saito, and Prof. Kenichi Kourai. This dissertation was greatly improved by their invaluable feedback. During my Ph.D., I did an internship at NEC. I enjoyed working with Dr. Masato Asahara and the opportunity had a significant impact on my research skills. He also taught me LDA, which is the core of the fault study in this dis- sertation. I am also thankful to my colleagues in the sslab. Their surprising enthusiasm and skills have always inspired me. I appreciate the financial supports from the Research Fellowships of the Japan Society for the Promotion of Science for Young Scientists and the Core Re- search for Evolutional Science and Technology of Japan Science and Technology Agency. Finally, I would like to thank my family, my parents, sister for their support all these years. Without their support and encouragement, many accomplishments in my life including this dissertation would not have been possible. 2 Abstract A Study on Faults and Error Propagation in the Linux Operating System Takeshi Yoshimura Operating systems are crucial for application reliability.
    [Show full text]
  • GDB Debugger CS 211 – Programming Practicum GDB Debugger
    GDB Debugger CS 211 – Programming Practicum GDB Debugger • Part of the GNU Software Tools • Many Debuggers in IDEs are just Wrappers for GDB • Huge amount of commands in GBD, many options available to help debug your program • Even to most basic knowledge can save you lots of time GDB Debugger • Step 1 – Compile your program using the –g flag • gcc –g mazeflawed.c • The –g flag creates additional information for the executable that gdb used to convert machine code lines back to the source code line of the original program GDB Debugger • Step 2 - Open the GDB debugger • gdb a.out • Give the name of the executable created by the compiler GDB Debugger • Step 3 – Run your program with any command line arguments • run <command-line-arguments> • If no command line arguments are needed, just type in run. • For the mazeflawed.c program, you need the datafile name, so: • run mazedata1.txt GDB Debugger • Step 4 – Let GDB tell you on which line the Segmentation Fault occurs Program received signal SIGSEGV, Segmentation fault. 0x00000000004007a9 in main (argc=2, argv=0x7fffffffca78) at mazeflawed.c:52 52 m1.arr[i][j] = '.’; • The above states the Segmentation fault occurred at line 52. • It also shows the code at line 52 GDB Debugger • Step 5 – The list command will display more lines of code • Use the help command to find out about more commands. • GDB has so many commands that no one knows them all. • Most people learn a few key commands. GDB Debugger • Step 6 – The print command will display values stored in variables 52 m1.arr[i][j] = ‘.’; print i $1 = 14 • The print i command shows that the variable i contains the value of 14 at the time of the segmentation fault.
    [Show full text]
  • Release Notes for Iplanet Unified Development Server (UDS) Version 5.0, SP1 Updated December 7, 2001
    Release Notes for iPlanet Unified Development Server (UDS) Version 5.0, SP1 Updated December 7, 2001 These release notes contain important information available at the time of the release of iPlanet Unified Development Server, Version 5.0, SP1 (UDS). Information on significant updates, known problems, fixed bugs, and other issues are addressed here. Read this document before you begin using this release of UDS. These release notes contain the following sections: • “General Information” • “Supported Systems and Software” • “Updates in UDS, Version 5.0, SP1” • “Bugs Fixed in UDS, Version 5.0, SP1” • “Known Problems” • “UDS Distribution Files” • “How to Report Problems” • “For More Information” 1 General Information General Information UDS Version 5.0, SP1 contains fixes for problems that were found with iPlanet Unified Development Server, Version 5.0. This release is available to all currently supported customers through iPlanet’s SubscribeNet service and will be shipped to new customers. New customers who want access to this release, please contact iPlanet sales. The distribution for this release includes the following: • UDS Ve rsion 5.0 , SP 1 • UDS Express Version 5.0, SP1 • UDS Web Enterprise Version 5.0, SP1 • UDS Runtime Version 5.0, SP1 • UDS Version 5.0 documentation Changes in Distribution Format The format for the distribution files has changed for this release. The section in this document, “UDS Distribution Files,” provides information on how to prepare the downloaded distribution files for installation. Note: UDS for the OS/390 platform is only available on IBM 3490 tape media. The SubscribeNet web site contains information about obtaining the UDS for the OS/390 distribution.
    [Show full text]
  • Memory Protection: Kernel and User Address Spaces Andy Wang Operating Systems COP 4610 / CGS 5765 up to This Point
    Memory Protection: Kernel and User Address Spaces Andy Wang Operating Systems COP 4610 / CGS 5765 Up to This Point . Threads provide the illusion of an infinite number of CPUs . On a single processor machine . Memory management provides a different set of illusions . Protected memory . Infinite amount of memory . Transparent sharing Physical vs. Virtual Memory Physical memory Virtual memory No protection Each process isolated from others and from OS Limited size Illusion of infinite memory Sharing visible to Each process cannot tell processes if memory is shared Memory Organizations . Simplest: uniprogramming without memory protection . Each application runs within a hardwired range of physical memory addresses . One application runs at a time . Application can use the same physical addresses every time, across reboots Uniprogramming Without Memory Protection . Applications typically use the lower memory addresses . An OS uses the higher memory addresses . An application can address any physical memory location Application Operating system 000000 ffffff Physical memory Multiprogramming Without Memory Protection . When a program is copied into memory, a linker-loader alters the code of the program (e.g., loads, stores, and jumps) . To use the address of where the program lands in memory . This is kind of what happens when you run the command gcc –o [filename]. It links object code into an executable. Multiprogramming Without Memory Protection . Bugs in any program can cause other programs to crash, even the OS Application 1 Application 2 Operating system 000000 ffffff Physical memory Multiprogrammed OS With Memory Protection . Memory protection keeps user programs from crashing one another and the OS . Two hardware-supported mechanisms .
    [Show full text]