DOCSLIB.ORG

Computer Science from the Bottom up Ian Wienand Computer Science from the Bottom up Ian Wienand a PDF Version Is Available At

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Computer Science from the Bottom up Ian Wienand Computer Science from the Bottom up Ian Wienand a PDF Version Is Available At Computer Science from the Bottom Up Ian Wienand Computer Science from the Bottom Up Ian Wienand A PDF version is available at https://www.bottomupcs.com/csbu.pdf. A EPUB version is available at https:// www.bottomupcs.com/csbu.epub The original souces are available at https://github.com/ianw/bottomupcs Copyright © 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015, 2016 Ian Wienand This work is licensed under the Creative Commons Attribution-ShareAlike License. To view a copy of this license, visit http:// creativecommons.org/licenses/by-sa/3.0/ or send a letter to Creative Commons, 559 Nathan Abbott Way, Stanford, California 94305, USA. Table of Contents Introduction ................................................................................................................. xi Welcome ............................................................................................................. xi Philosophy ................................................................................................... xi Why from the bottom up? ........................................................................... xi Enabling Technologies ................................................................................ xi 1. General Unix and Advanced C ............................................................................... 1 Everything is a file! ............................................................................................ 1 Implementing abstraction .................................................................................... 2 Implementing abstraction with C ................................................................. 2 Libraries ....................................................................................................... 4 File Descriptors .................................................................................................... 5 The Shell ...................................................................................................... 8 2. Binary and Number Representation ..................................................................... 11 Binary — the basis of computing ...................................................................... 11 Binary Theory ............................................................................................ 11 Hexadecimal .............................................................................................. 17 Practical Implications ................................................................................. 19 Types and Number Representation .................................................................. 21 C Standards ............................................................................................... 21 Types ......................................................................................................... 22 Number Representation ............................................................................. 26 3. Computer Architecture .......................................................................................... 35 The CPU ............................................................................................................ 35 Branching ................................................................................................... 36 Cycles ........................................................................................................ 36 Fetch, Decode, Execute, Store ................................................................. 36 CISC v RISC ............................................................................................. 39 Memory .............................................................................................................. 41 Memory Hierarchy ..................................................................................... 41 Cache in depth .......................................................................................... 42 Peripherals and buses ....................................................................................... 46 Peripheral Bus concepts ............................................................................ 46 DMA ........................................................................................................... 48 Other Buses ............................................................................................... 49 Small to big systems ......................................................................................... 51 Symmetric Multi-Processing ...................................................................... 51 Clusters ...................................................................................................... 53 Non-Uniform Memory Access .................................................................... 54 Memory ordering, locking and atomic operations ...................................... 57 4. The Operating System .......................................................................................... 62 The role of the operating system ...................................................................... 62 Abstraction of hardware ............................................................................. 62 Multitasking ................................................................................................ 62 Standardised Interfaces ............................................................................. 62 Security ...................................................................................................... 63 Performance ............................................................................................... 63 iii Computer Science from the Bottom Up Operating System Organisation ........................................................................ 63 The Kernel ................................................................................................. 64 Userspace .................................................................................................. 69 System Calls ...................................................................................................... 69 Overview .................................................................................................... 69 Analysing a system call ............................................................................. 70 Privileges ........................................................................................................... 76 Hardware .................................................................................................... 76 Other ways of communicating with the kernel .......................................... 81 File Systems .............................................................................................. 82 5. The Process .......................................................................................................... 83 What is a process? ........................................................................................... 83 Elements of a process ...................................................................................... 83 Process ID ................................................................................................. 83 Memory ...................................................................................................... 84 File Descriptors .......................................................................................... 89 Registers .................................................................................................... 89 Kernel State ............................................................................................... 89 Process Hierarchy ............................................................................................. 90 Fork and Exec ................................................................................................... 90 Fork ............................................................................................................ 90 Exec ........................................................................................................... 91 How Linux actually handles fork and exec ................................................ 91 The init process ......................................................................................... 93 Context Switching .............................................................................................. 95 Scheduling ......................................................................................................... 95 Preemptive v co-operative scheduling ...................................................... 95 Realtime ..................................................................................................... 96 Nice value .................................................................................................. 96 A brief look at the Linux Scheduler ........................................................... 96 The Shell ........................................................................................................... 97 Signals ............................................................................................................... 98 Example ..................................................................................................... 99 6. Virtual Memory ...................................................................................................
Load more

Recommended publications
  • VAX MACRO and Instruction Set Reference Manual
    VAX MACRO and Instruction Set Reference Manual Order Number: AA-LA89A-TE April 1988 This document describes the features of the VAX MACRO instruction set and assembler. It includes a detailed description of MACRO directives and instructions, as well as information about MACRO source program syntax. Revision/Update Information: This manual supersedes the VAX MACRO and Instruction Set Reference Manual, Version 4.0. Software Version: VMS Version 5.0 digital equipment corporation maynard, massachusetts April 1988 The information in this document is subject to change without notice and should not be construed as a commitment by Digital Equipment Corporation. Digital Equipment Corporation assumes no responsibility for any errors that may appear in this document. The software described in this document is furnished under a license and may be used or copied only in accordance with the terms of such license. No responsibility is assumed for the use or reliability of software on equipment that is not supplied by Digital Equipment Corporation or its affiliated companies. Copyright © 1988 by Digital Equipment Corporation All Rights Reserved. Printed in U.S.A. The postpaid READER'S COMMENTS form on the last page of this document requests the user's critical evaluation to assist in preparing future documentation. The following are trademarks of Digital Equipment Corporation: DEC DIBOL UNIBUS DEC/CMS EduSystem VAX DEC/MMS IAS VAXcluster DECnet MASSBUS VMS DECsystem-10 PDP VT DECSYSTEM-20 PDT DECUS RSTS DECwriter RSX t:JD~DD5lD TM ZK4515 HOW TO ORDER ADDITIONAL DOCUMENTATION DIRECT MAIL ORDERS USA 8t PUERTO Rico* CANADA INTERNATIONAL Digital Equipment Corporation Digital Equipment Digital Equipment Corporation P.O.
    [Show full text]
  • Common Object File Format (COFF)
    Application Report SPRAAO8–April 2009 Common Object File Format ..................................................................................................................................................... ABSTRACT The assembler and link step create object files in common object file format (COFF). COFF is an implementation of an object file format of the same name that was developed by AT&T for use on UNIX-based systems. This format encourages modular programming and provides powerful and flexible methods for managing code segments and target system memory. This appendix contains technical details about the Texas Instruments COFF object file structure. Much of this information pertains to the symbolic debugging information that is produced by the C compiler. The purpose of this application note is to provide supplementary information on the internal format of COFF object files. Topic .................................................................................................. Page 1 COFF File Structure .................................................................... 2 2 File Header Structure .................................................................. 4 3 Optional File Header Format ........................................................ 5 4 Section Header Structure............................................................. 5 5 Structuring Relocation Information ............................................... 7 6 Symbol Table Structure and Content........................................... 11 SPRAAO8–April 2009
    [Show full text]
  • Virtual Memory - Paging
    Virtual memory - Paging Johan Montelius KTH 2020 1 / 32 The process code heap (.text) data stack kernel 0x00000000 0xC0000000 0xffffffff Memory layout for a 32-bit Linux process 2 / 32 Segments - a could be solution Processes in virtual space Address translation by MMU (base and bounds) Physical memory 3 / 32 one problem Physical memory External fragmentation: free areas of free space that is hard to utilize. Solution: allocate larger segments ... internal fragmentation. 4 / 32 another problem virtual space used code We’re reserving physical memory that is not used. physical memory not used? 5 / 32 Let’s try again It’s easier to handle fixed size memory blocks. Can we map a process virtual space to a set of equal size blocks? An address is interpreted as a virtual page number (VPN) and an offset. 6 / 32 Remember the segmented MMU MMU exception no virtual addr. offset yes // < within bounds index + physical address segment table 7 / 32 The paging MMU MMU exception virtual addr. offset // VPN available ? + physical address page table 8 / 32 the MMU exception exception virtual address within bounds page available Segmentation Paging linear address physical address 9 / 32 a note on the x86 architecture The x86-32 architecture supports both segmentation and paging. A virtual address is translated to a linear address using a segmentation table. The linear address is then translated to a physical address by paging. Linux and Windows do not use use segmentation to separate code, data nor stack. The x86-64 (the 64-bit version of the x86 architecture) has dropped many features for segmentation.
    [Show full text]
  • Multi-Method Dispatch Using Single-Receiver Projections
    Multi-Method Dispatch Using Single-Receiver Projections Wade Holst Duane Szafron Candy Pang Yuri Leontiev g fwade,duane,candy,yuri @cs.ualberta.ca Department of Computing Science University of Alberta Edmonton, Canada Abstract A new technique for multi-method dispatch, called Single-Receiver Projections and abbre- viated SRP, is presented. It operates by performing projections onto single-receiver dispatch tables, and thus benefits from existing research into single-receiver dispatch techniques. It pro- k vides method dispatch in O(k ), where is the arity of the method. Unlike existing multi-method dispatch techniques, it also maintains the set of all applicable methods. This extra information allows SRP to support languages like CLOS that use next-method. SRP is also highly incremen- tal in nature, so it is well-suited for languages that require separate compilation and languages that allow methods and classes to be added at run-time. SRP uses less data-structure memory than all existing multi-method techniques and has the second fastest dispatch time. Furthermore, if the other techniques are extended to support all applicable methods, SRP becomes the fastest dispatch techniques. RESEARCH PAPER Subject Areas: language design and implementation, programming environments Keywords: multi-method, dispatch, object-oriented, implementation, compiler Word Count: 8722 Contact Author: Duane Szafron Department of Computing Science University of Alberta Edmonton, Canada, T6G 2H1 Phone: (780) 492-5468 Fax: (780) 492-1071 Email: [email protected] 1 1 Introduction In object-oriented languages, the method to invoke is determined not only by the method (selector) name, but also by the dynamic types of its arguments.
    [Show full text]
  • Address Translation
    CS 152 Computer Architecture and Engineering Lecture 8 - Address Translation John Wawrzynek Electrical Engineering and Computer Sciences University of California at Berkeley http://www.eecs.berkeley.edu/~johnw http://inst.eecs.berkeley.edu/~cs152 9/27/2016 CS152, Fall 2016 CS152 Administrivia § Lab 2 due Friday § PS 2 due Tuesday § Quiz 2 next Thursday! 9/27/2016 CS152, Fall 2016 2 Last time in Lecture 7 § 3 C’s of cache misses – Compulsory, Capacity, Conflict § Write policies – Write back, write-through, write-allocate, no write allocate § Multi-level cache hierarchies reduce miss penalty – 3 levels common in modern systems (some have 4!) – Can change design tradeoffs of L1 cache if known to have L2 § Prefetching: retrieve memory data before CPU request – Prefetching can waste bandwidth and cause cache pollution – Software vs hardware prefetching § Software memory hierarchy optimizations – Loop interchange, loop fusion, cache tiling 9/27/2016 CS152, Fall 2016 3 Bare Machine Physical Physical Address Inst. Address Data Decode PC Cache D E + M Cache W Physical Memory Controller Physical Address Address Physical Address Main Memory (DRAM) § In a bare machine, the only kind of address is a physical address 9/27/2016 CS152, Fall 2016 4 Absolute Addresses EDSAC, early 50’s § Only one program ran at a time, with unrestricted access to entire machine (RAM + I/O devices) § Addresses in a program depended upon where the program was to be loaded in memory § But it was more convenient for programmers to write location-independent subroutines How
    [Show full text]
  • Lecture 12: Virtual Memory Finale and Interrupts/Exceptions 1 Review And
    EE360N: Computer Architecture Lecture #12 Department of Electical and Computer Engineering The University of Texas at Austin October 9, 2008 Disclaimer: The contents of this document are scribe notes for The University of Texas at Austin EE360N Fall 2008, Computer Architecture.∗ The notes capture the class discussion and may contain erroneous and unverified information and comments. Lecture 12: Virtual Memory Finale and Interrupts/Exceptions Lecture #12: October 8, 2008 Lecturer: Derek Chiou Scribe: Michael Sullivan and Peter Tran Lecture 12 of EE360N: Computer Architecture summarizes the significance of virtual memory and introduces the concept of interrupts and exceptions. As such, this document is divided into two sections. Section 1 details the key points from the discussion in class on virtual memory. Section 2 goes on to introduce interrupts and exceptions, and shows why they are necessary. 1 Review and Summation of Virtual Memory The beginning of this lecture wraps up the review of virtual memory. A broad summary of virtual memory systems is given first, in Subsection 1.1. Next, the individual components of the hardware and OS that are required for virtual memory support are covered in 1.2. Finally, the addressing of caches and the interface between the cache and virtual memory are described in 1.3. 1.1 Summary of Virtual Memory Systems Virtual memory automatically provides the illusion of a large, private, uniform storage space for each process, even on a limited amount of physically addressed memory. The virtual memory space available to every process looks the same every time the process is run, irrespective of the amount of memory available, or the program’s actual placement in main memory.
    [Show full text]
  • CS 351: Systems Programming
    Virtual Memory CS 351: Systems Programming Michael Saelee <[email protected]> Computer Science Science registers cache (SRAM) main memory (DRAM) local hard disk drive (HDD/SSD) remote storage (networked drive / cloud) previously: SRAM ⇔ DRAM Computer Science Science registers cache (SRAM) main memory (DRAM) local hard disk drive (HDD/SSD) remote storage (networked drive / cloud) next: DRAM ⇔ HDD, SSD, etc. i.e., memory as a “cache” for disk Computer Science Science main goals: 1. maximize memory throughput 2. maximize memory utilization 3. provide address space consistency & memory protection to processes Computer Science Science throughput = # bytes per second - depends on access latencies (DRAM, HDD) and “hit rate” Computer Science Science utilization = fraction of allocated memory that contains “user” data (aka payload) - vs. metadata and other overhead required for memory management Computer Science Science address space consistency → provide a uniform “view” of memory to each process 0xffffffff Computer Science Science Kernel virtual memory Memory (code, data, heap, stack) invisible to 0xc0000000 user code User stack (created at runtime) %esp (stack pointer) Memory mapped region for shared libraries 0x40000000 brk Run-time heap (created by malloc) Read/write segment ( , ) .data .bss Loaded from the Read-only segment executable file (.init, .text, .rodata) 0x08048000 0 Unused address space consistency → provide a uniform “view” of memory to each process Computer Science Science memory protection → prevent processes from directly accessing
    [Show full text]
  • Chapter 9: Memory Management
    Chapter 9: Memory Management I Background I Swapping I Contiguous Allocation I Paging I Segmentation I Segmentation with Paging Operating System Concepts 9.1 Silberschatz, Galvin and Gagne 2002 Background I Program must be brought into memory and placed within a process for it to be run. I Input queue – collection of processes on the disk that are waiting to be brought into memory to run the program. I User programs go through several steps before being run. Operating System Concepts 9.2 Silberschatz, Galvin and Gagne 2002 Binding of Instructions and Data to Memory Address binding of instructions and data to memory addresses can happen at three different stages. I Compile time: If memory location known a priori, absolute code can be generated; must recompile code if starting location changes. I Load time: Must generate relocatable code if memory location is not known at compile time. I Execution time: Binding delayed until run time if the process can be moved during its execution from one memory segment to another. Need hardware support for address maps (e.g., base and limit registers). Operating System Concepts 9.3 Silberschatz, Galvin and Gagne 2002 Multistep Processing of a User Program Operating System Concepts 9.4 Silberschatz, Galvin and Gagne 2002 Logical vs. Physical Address Space I The concept of a logical address space that is bound to a separate physical address space is central to proper memory management. ✦ Logical address – generated by the CPU; also referred to as virtual address. ✦ Physical address – address seen by the memory unit. I Logical and physical addresses are the same in compile- time and load-time address-binding schemes; logical (virtual) and physical addresses differ in execution-time address-binding scheme.
    [Show full text]
  • One Physical Address Space Per Machine – the Size of a Physical Address Determines the Maximum Amount of Addressable Physical Memory
    Virtual Memory 1 Virtual and Physical Addresses • Physical addresses are provided directly by the machine. – one physical address space per machine – the size of a physical address determines the maximum amount of addressable physical memory • Virtual addresses (or logical addresses) are addresses provided by the OS to processes. – one virtual address space per process • Programs use virtual addresses. As a program runs, the hardware (with help from the operating system) converts each virtual address to a physical address. • the conversion of a virtual address to a physical address is called address translation On the MIPS, virtual addresses and physical addresses are 32 bits long. This limits the size of virtual and physical address spaces. CS350 Operating Systems Winter 2011 Virtual Memory 2 What is in a Virtual Address Space? 0x00400000 − 0x00401b30 text (program code) and read−only data growth 0x10000000 − 0x101200b0 stack high end of stack: 0x7fffffff data 0x00000000 0xffffffff This diagram illustrates the layout of the virtual address space for the OS/161 test application testbin/sort CS350 Operating Systems Winter 2011 Virtual Memory 3 Simple Address Translation: Dynamic Relocation • hardware provides a memory management unit which includes a relocation register • at run-time, the contents of the relocation register are added to each virtual address to determine the corresponding physical address • the OS maintains a separate relocation register value for each process, and ensures that relocation register is reset on each context
    [Show full text]
  • CS61 Section Notes Week 7 (Fall 2010) Virtual Memory We Are Given a System with the Following Properties
    CS61 Section Notes Week 7 (Fall 2010) Virtual Memory We are given a system with the following properties: • The memory is byte addressable. • Memory accesses are to 4-byte words • Physical addresses are 16 bits wide. • Virtual addresses are 20 bits wide. • The page size is 4096 bytes. • The TLB is 4-way set associative with 16 total entries. In the following tables, all numbers are given in hexadecimal. The contents of the TLB and the page table for the first 32 pages are as follows: TLB Page Table Index Tag PPN Valid VPN PPN Valid VPN PPN Valid 03 B 1 00 7 1 10 6 0 07 6 0 01 8 1 11 7 0 0 28 3 1 02 9 1 12 8 0 01 F 0 03 A 1 13 3 0 31 0 1 04 6 0 14 D 0 12 3 0 05 3 0 15 B 0 1 07 E 1 06 1 0 16 9 0 0B 1 1 07 8 0 17 6 0 2A A 0 08 2 0 18 C 1 11 1 0 09 3 0 19 4 1 2 1F 8 1 0A 1 1 1A F 0 07 5 1 0B 6 1 1B 2 1 07 3 1 0C A 1 1C 0 0 3F F 0 0D D 0 1D E 1 3 10 D 0 0E E 0 1E 5 1 32 0 0 0F D 1 1F 3 1 Question 1a: The box below shows the format of a virtual address. Indicate (by labeling the diagram) the fields that would be used to determine the following: 1 VPO The virtual page offset VPN The virtual page number TLBI The TLB index TLBT The TLB tag 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 0 VPN VPO TLBT TLBI Question 1b: The box below shows the format of a physical address.
    [Show full text]
  • MIPS SDE 6.X Programmers' Guide
    TECHNOLOGIES MIPS® SDE 6.x Programmers’ Guide Document Number: MD00428 Revision 1.17 April 4, 2007 MIPS Technologies, Inc 1225 Charleston Road Mountain View,CA94043-1353 Copyright © 1995-2007 MIPS Technologies, Inc. All rights reserved. Copyright © 1995-2007 MIPS Technologies, Inc. All rights reserved. Unpublished rights (if any) reserved under the copyright laws of the United States of America and other countries. This document contains information that is proprietary to MIPS Technologies, Inc. (‘‘MIPS Technologies’’). Any copying, reproducing, modifying or use of this information (in whole or in part) that is not expressly permitted in writing by MIPS Technologies or an authorized third party is strictly prohibited. At a minimum, this information is protected under unfair competition and copyright laws. Violations thereof may result in criminal penalties and fines. Anydocument provided in source format (i.e., in a modifiable form such as in FrameMaker or Microsoft Word format) is subject to use and distribution restrictions that are independent of and supplemental to anyand all confidentiality restrictions. UNDER NO CIRCUMSTANCES MAYADOCUMENT PROVIDED IN SOURCE FORMATBEDISTRIBUTED TOATHIRD PARTY IN SOURCE FORMATWITHOUT THE EXPRESS WRITTEN PERMISSION OF MIPS TECHNOLOGIES, INC. MIPS Technologies reserves the right to change the information contained in this document to improve function, design or otherwise. MIPS Technologies does not assume anyliability arising out of the application or use of this information, or of anyerror or omission in such information. Anywarranties, whether express, statutory,implied or otherwise, including but not limited to the implied warranties of merchantability or fitness for a particular purpose, are excluded. Except as expressly provided in anywritten license agreement from MIPS Technologies or an authorized third party,the furnishing of this document does not give recipient anylicense to anyintellectual property rights, including anypatent rights, that coverthe information in this document.
    [Show full text]
  • CS420: Operating Systems Paging and Page Tables
    CS420: Operating Systems Paging and Page Tables YORK COLLEGE OF PENNSYLVANIA YORK COLLEGE OF PENNSYLVANIA YORK COLLEGE OF PENNSYLVANIA YORK COLLEGE OF PENNSYLVANIA 'GHI<GJHK&L<MNK'GHI<GJHK&L<MNK 'GONJHK&P@JJHG 'GONJHK&P@JJHG MFIF&'<IJ@QH&'@OK MFIF&'<IJ@QH&'@OK !<GR%&'@ !<GR%&'@ James Moscola 'HGPOJ&N<F&012 'HGPOJ&N<F&012 !"#$%&'())*+,-.)/.&0123454670!"#$%&'())*+,-.)/.&0123454670 YORK COLLEGE OF PENNSYLVANIA Department of Physical Sciences 'GHI<GJHK&L<MNK 8.9:;*&<:(#.="#>&1015?26511??8.9:;*&<:(#.="#>&1015?26511?? @A9/**/")*&<B!&C(>&1015?2D50633@A9/**/")*&<B!&C(>&1015?2D50633'GONJHK&P@JJHG York College of Pennsylvania 05?3352775?30? 05?3352775?30? MFIF&'<IJ@QH&'@OK EEEF+C:F(A; EEEF+C:F(A; !<GR%&'@ 'HGPOJ&N<F&012 !""#$%%&'$#()*$&+$,-$%.$"!""#$%%&'$#()*$&+$,-$%.$" !"#$%&'())*+,-.)/.&0123454670 8.9:;*&<:(#.="#>&1015?26511?? @A9/**/")*&<B!&C(>&1015?2D50633 05?3352775?30? EEEF+C:F(A; !""#$%%&'$#()*$&+$,-$%.$" CS420: Operating Systems Based on Operating System Concepts, 9th Edition by Silberschatz, Galvin, Gagne COLLEGE CATALOG 2009–2011 COLLEGE CATALOG COLLEGE CATALOG 2009–2011 COLLEGE CATALOG COLLEGE CATALOG 2009–2011 COLLEGE CATALOG College Catalog 2009–2011College Catalog 2009–2011 College Catalog 2009–2011 !""#$%&'()*+,--.../ 012$1$"..."34#3$4.56 !""#$%&'()*+,--.../ 012$1$"..."34#3$4.56 !""#$%&'()*+,--.../ 012$1$"..."34#3$4.56 Paging • Paging is a memory-management scheme that permits the physical address space of a process to be noncontiguous - Avoids external fragmentation - Avoids the need for compaction - May still have internal fragmentation
    [Show full text]