Functional Programming for Systems Software: Implementing Baremetal Programs in Habit
Total Page:16
File Type:pdf, Size:1020Kb
Load more
Recommended publications
-
Common Object File Format (COFF)
Application Report SPRAAO8–April 2009 Common Object File Format ..................................................................................................................................................... ABSTRACT The assembler and link step create object files in common object file format (COFF). COFF is an implementation of an object file format of the same name that was developed by AT&T for use on UNIX-based systems. This format encourages modular programming and provides powerful and flexible methods for managing code segments and target system memory. This appendix contains technical details about the Texas Instruments COFF object file structure. Much of this information pertains to the symbolic debugging information that is produced by the C compiler. The purpose of this application note is to provide supplementary information on the internal format of COFF object files. Topic .................................................................................................. Page 1 COFF File Structure .................................................................... 2 2 File Header Structure .................................................................. 4 3 Optional File Header Format ........................................................ 5 4 Section Header Structure............................................................. 5 5 Structuring Relocation Information ............................................... 7 6 Symbol Table Structure and Content........................................... 11 SPRAAO8–April 2009 -
Virtual Memory - Paging
Virtual memory - Paging Johan Montelius KTH 2020 1 / 32 The process code heap (.text) data stack kernel 0x00000000 0xC0000000 0xffffffff Memory layout for a 32-bit Linux process 2 / 32 Segments - a could be solution Processes in virtual space Address translation by MMU (base and bounds) Physical memory 3 / 32 one problem Physical memory External fragmentation: free areas of free space that is hard to utilize. Solution: allocate larger segments ... internal fragmentation. 4 / 32 another problem virtual space used code We’re reserving physical memory that is not used. physical memory not used? 5 / 32 Let’s try again It’s easier to handle fixed size memory blocks. Can we map a process virtual space to a set of equal size blocks? An address is interpreted as a virtual page number (VPN) and an offset. 6 / 32 Remember the segmented MMU MMU exception no virtual addr. offset yes // < within bounds index + physical address segment table 7 / 32 The paging MMU MMU exception virtual addr. offset // VPN available ? + physical address page table 8 / 32 the MMU exception exception virtual address within bounds page available Segmentation Paging linear address physical address 9 / 32 a note on the x86 architecture The x86-32 architecture supports both segmentation and paging. A virtual address is translated to a linear address using a segmentation table. The linear address is then translated to a physical address by paging. Linux and Windows do not use use segmentation to separate code, data nor stack. The x86-64 (the 64-bit version of the x86 architecture) has dropped many features for segmentation. -
Address Translation
CS 152 Computer Architecture and Engineering Lecture 8 - Address Translation John Wawrzynek Electrical Engineering and Computer Sciences University of California at Berkeley http://www.eecs.berkeley.edu/~johnw http://inst.eecs.berkeley.edu/~cs152 9/27/2016 CS152, Fall 2016 CS152 Administrivia § Lab 2 due Friday § PS 2 due Tuesday § Quiz 2 next Thursday! 9/27/2016 CS152, Fall 2016 2 Last time in Lecture 7 § 3 C’s of cache misses – Compulsory, Capacity, Conflict § Write policies – Write back, write-through, write-allocate, no write allocate § Multi-level cache hierarchies reduce miss penalty – 3 levels common in modern systems (some have 4!) – Can change design tradeoffs of L1 cache if known to have L2 § Prefetching: retrieve memory data before CPU request – Prefetching can waste bandwidth and cause cache pollution – Software vs hardware prefetching § Software memory hierarchy optimizations – Loop interchange, loop fusion, cache tiling 9/27/2016 CS152, Fall 2016 3 Bare Machine Physical Physical Address Inst. Address Data Decode PC Cache D E + M Cache W Physical Memory Controller Physical Address Address Physical Address Main Memory (DRAM) § In a bare machine, the only kind of address is a physical address 9/27/2016 CS152, Fall 2016 4 Absolute Addresses EDSAC, early 50’s § Only one program ran at a time, with unrestricted access to entire machine (RAM + I/O devices) § Addresses in a program depended upon where the program was to be loaded in memory § But it was more convenient for programmers to write location-independent subroutines How -
Lecture 12: Virtual Memory Finale and Interrupts/Exceptions 1 Review And
EE360N: Computer Architecture Lecture #12 Department of Electical and Computer Engineering The University of Texas at Austin October 9, 2008 Disclaimer: The contents of this document are scribe notes for The University of Texas at Austin EE360N Fall 2008, Computer Architecture.∗ The notes capture the class discussion and may contain erroneous and unverified information and comments. Lecture 12: Virtual Memory Finale and Interrupts/Exceptions Lecture #12: October 8, 2008 Lecturer: Derek Chiou Scribe: Michael Sullivan and Peter Tran Lecture 12 of EE360N: Computer Architecture summarizes the significance of virtual memory and introduces the concept of interrupts and exceptions. As such, this document is divided into two sections. Section 1 details the key points from the discussion in class on virtual memory. Section 2 goes on to introduce interrupts and exceptions, and shows why they are necessary. 1 Review and Summation of Virtual Memory The beginning of this lecture wraps up the review of virtual memory. A broad summary of virtual memory systems is given first, in Subsection 1.1. Next, the individual components of the hardware and OS that are required for virtual memory support are covered in 1.2. Finally, the addressing of caches and the interface between the cache and virtual memory are described in 1.3. 1.1 Summary of Virtual Memory Systems Virtual memory automatically provides the illusion of a large, private, uniform storage space for each process, even on a limited amount of physically addressed memory. The virtual memory space available to every process looks the same every time the process is run, irrespective of the amount of memory available, or the program’s actual placement in main memory. -
Cristina Opriceana, Hajime Tazaki (IIJ Research Lab.) Linux Netdev 2.2, Seoul, Korea 08 Nov
Network stack personality in Android phone Cristina Opriceana, Hajime Tazaki (IIJ Research Lab.) Linux netdev 2.2, Seoul, Korea 08 Nov. 2017 1 Librarified Linux taLks (LLL) Userspace network stack (NUSE) in general (netdev0.1) kernel CI with libos and ns-3 (netdev1.1) Network performance improvement of LKL (netdev1.2, by Jerry Chu) How bad/good with LKL and hrtimer (BBR) (netdev2.1) Updating Android network stack (netdev2.2) 2 Android a platform of billions devices billions installed Linux kernel Questions When our upstreamed code available ? What if I come up with a great protocol ? https://developer.android.com/about/dashboards/index.html 3 Android (cont'd) When our upstreamed code available ? wait until base kernel is upgraded backport specific function What if I come up with a great protocol ? craft your own kernel and put into your image Long delivery to all billions devices 4 Approaches to alleviate the issue Virtualization (KVM on Android) Overhead isn't negligible to embedded devices Project Treble (since Android O) More modular platform implementation Fushia Rewrite OS from scratch QUIC (transport over UDP) Rewrite transport protocols on UDP https://source android com/devices/architecture/treble https://source.android.com/devices/architecture/treble An alternate approach network stack personality use own network stack implemented in userspace no need to replace host kernels but (try to) preserve the application compatibility NUSE (network stack in userspace) No delay of network stack update Application can choose a network stack if needed 56 Userspace implementations Toys, Misguided People Selfish Motivation Trying to present that a Toy is practically useful 7 Linux Kernel Library intro (again) Out-of-tree architecture (h/w-independent) Run Linux code on various ways with a reusable library h/w dependent layer on Linux/Windows /FreeBSD uspace, unikernel, on UEFI, network simulator (ns-3) Android 8 LKL: current status Sent RFC (Nov. -
CS 351: Systems Programming
Virtual Memory CS 351: Systems Programming Michael Saelee <[email protected]> Computer Science Science registers cache (SRAM) main memory (DRAM) local hard disk drive (HDD/SSD) remote storage (networked drive / cloud) previously: SRAM ⇔ DRAM Computer Science Science registers cache (SRAM) main memory (DRAM) local hard disk drive (HDD/SSD) remote storage (networked drive / cloud) next: DRAM ⇔ HDD, SSD, etc. i.e., memory as a “cache” for disk Computer Science Science main goals: 1. maximize memory throughput 2. maximize memory utilization 3. provide address space consistency & memory protection to processes Computer Science Science throughput = # bytes per second - depends on access latencies (DRAM, HDD) and “hit rate” Computer Science Science utilization = fraction of allocated memory that contains “user” data (aka payload) - vs. metadata and other overhead required for memory management Computer Science Science address space consistency → provide a uniform “view” of memory to each process 0xffffffff Computer Science Science Kernel virtual memory Memory (code, data, heap, stack) invisible to 0xc0000000 user code User stack (created at runtime) %esp (stack pointer) Memory mapped region for shared libraries 0x40000000 brk Run-time heap (created by malloc) Read/write segment ( , ) .data .bss Loaded from the Read-only segment executable file (.init, .text, .rodata) 0x08048000 0 Unused address space consistency → provide a uniform “view” of memory to each process Computer Science Science memory protection → prevent processes from directly accessing -
Chapter 9: Memory Management
Chapter 9: Memory Management I Background I Swapping I Contiguous Allocation I Paging I Segmentation I Segmentation with Paging Operating System Concepts 9.1 Silberschatz, Galvin and Gagne 2002 Background I Program must be brought into memory and placed within a process for it to be run. I Input queue – collection of processes on the disk that are waiting to be brought into memory to run the program. I User programs go through several steps before being run. Operating System Concepts 9.2 Silberschatz, Galvin and Gagne 2002 Binding of Instructions and Data to Memory Address binding of instructions and data to memory addresses can happen at three different stages. I Compile time: If memory location known a priori, absolute code can be generated; must recompile code if starting location changes. I Load time: Must generate relocatable code if memory location is not known at compile time. I Execution time: Binding delayed until run time if the process can be moved during its execution from one memory segment to another. Need hardware support for address maps (e.g., base and limit registers). Operating System Concepts 9.3 Silberschatz, Galvin and Gagne 2002 Multistep Processing of a User Program Operating System Concepts 9.4 Silberschatz, Galvin and Gagne 2002 Logical vs. Physical Address Space I The concept of a logical address space that is bound to a separate physical address space is central to proper memory management. ✦ Logical address – generated by the CPU; also referred to as virtual address. ✦ Physical address – address seen by the memory unit. I Logical and physical addresses are the same in compile- time and load-time address-binding schemes; logical (virtual) and physical addresses differ in execution-time address-binding scheme. -
Installing Management Node Remotely
Installing Management Node Remotely This chapter contains the following topics: • Overview to Installation of Management Node Remotely, on page 1 • Overview to Cisco VIM Baremetal Manager REST API, on page 5 • Installing Cisco VIM Baremetal Manager Management Node On a UCS C-series Server, on page 6 • Preparing the Cisco VIM Baremetal Manager Management Node from Cisco VIM Software Hub Server, on page 7 Overview to Installation of Management Node Remotely Cisco VIM fully automates the installation operation of the cloud. In releases prior to Cisco VIM 3.4.1, the management node installation was always manual, as the bootstrap of the cloud happens from there. Using this feature, the management node, referred to as Cisco VIM Baremetal Manager is automatically installed over a layer 3 network to accelerate the Cisco VIM installation process. Note In this chapter, the term Cisco VIM Baremetal Manager and Remote Install of Management Node (RIMN) are used interchangeably. Remote Install of Management Node Remote Install of Management Node (RIMN) software is deployed on the RIMN deployment node from where one or more management nodes are installed. Cisco VIM Baremetal Manager or RIMN supports remote installation of servers across WAN or LAN with either IPv4 or IPv6 connectivity. Cisco VIM Baremetal Manager can be installed on the Cisco VIM Baremetal Manager deployment node by using air-gapped installation. After you install the RIMN software on its management node, you must define an input file for bare-metal config (in YAML format) and use Cisco VIM Baremetal Manager CLI or Rest API to deploy the user-specified ISO into the target platform (as depicted in the figure below): Installing Management Node Remotely 1 Installing Management Node Remotely Hardware Requirements for RIMN RIMN solution is built based on the interaction of several components as depicted below: • Rest-API and CLI: Pushes the received input data into Etcd datastore. -
View the Slides
RedLeaf: Isolation and Communication in a Safe Operating System Vikram Narayanan1, Tianjiao Huang1, David Detweiler1, Dan Appel1, Zhaofeng Li1, Gerd Zellweger2, Anton Burtsev1 OSDI ’20 1University of California, Irvine 2VMware Research History of Isolation Cedar Ka�eOS Multics Pilot Scomp SPIN J-Kernel Mondrian VINO Singularity 1973 1980 1983 1995 1996 1999 2002 2005 Year • Isolation of kernel subsystems • Final report of Multics (1976) • Scomp (1983) • Systems remained monolithic • Isolation was expensive 1 Isolation mechanisms • Hardware Isolation • Segmentation (46 cycles)1 • Page table isolation (797 cycles)2 • VMFUNC (396 cycles)3 • Memory protection keys (20-26 cycles)4 • Language based isolation • Compare drivers written (DPDK-style) in a safe high-level language (C, Rust, Go, C#, etc.)5 • Managed runtime and Garbage collection (20-50% overhead on a device-driver workload) 1L4 Microkernel: Jochen Liedtke 2https://sel4.systems/About/Performance/ 3Lightweight Kernel Isolation with Virtualization and VM Functions, VEE 2020 4Hodor: Intra-process isolation for high-throughput data plane libraries 5The Case for Writing Network Drivers in High-Level Programming Languages, ANCS 2019 2 • Linear types • Enforces type and memory safety • Statically checked at compile time • Safety without runtime garbage collection overhead Rust Traditional Safe languages vs Rust Java, C# etc. A 3 • Linear types • Enforces type and memory safety • Statically checked at compile time • Safety without runtime garbage collection overhead Rust Traditional Safe languages vs Rust Java, C# etc. A Vector 3 • Linear types • Enforces type and memory safety • Statically checked at compile time • Safety without runtime garbage collection overhead Rust Traditional Safe languages vs Rust Java, C# etc. -
One Physical Address Space Per Machine – the Size of a Physical Address Determines the Maximum Amount of Addressable Physical Memory
Virtual Memory 1 Virtual and Physical Addresses • Physical addresses are provided directly by the machine. – one physical address space per machine – the size of a physical address determines the maximum amount of addressable physical memory • Virtual addresses (or logical addresses) are addresses provided by the OS to processes. – one virtual address space per process • Programs use virtual addresses. As a program runs, the hardware (with help from the operating system) converts each virtual address to a physical address. • the conversion of a virtual address to a physical address is called address translation On the MIPS, virtual addresses and physical addresses are 32 bits long. This limits the size of virtual and physical address spaces. CS350 Operating Systems Winter 2011 Virtual Memory 2 What is in a Virtual Address Space? 0x00400000 − 0x00401b30 text (program code) and read−only data growth 0x10000000 − 0x101200b0 stack high end of stack: 0x7fffffff data 0x00000000 0xffffffff This diagram illustrates the layout of the virtual address space for the OS/161 test application testbin/sort CS350 Operating Systems Winter 2011 Virtual Memory 3 Simple Address Translation: Dynamic Relocation • hardware provides a memory management unit which includes a relocation register • at run-time, the contents of the relocation register are added to each virtual address to determine the corresponding physical address • the OS maintains a separate relocation register value for each process, and ensures that relocation register is reset on each context -
CS61 Section Notes Week 7 (Fall 2010) Virtual Memory We Are Given a System with the Following Properties
CS61 Section Notes Week 7 (Fall 2010) Virtual Memory We are given a system with the following properties: • The memory is byte addressable. • Memory accesses are to 4-byte words • Physical addresses are 16 bits wide. • Virtual addresses are 20 bits wide. • The page size is 4096 bytes. • The TLB is 4-way set associative with 16 total entries. In the following tables, all numbers are given in hexadecimal. The contents of the TLB and the page table for the first 32 pages are as follows: TLB Page Table Index Tag PPN Valid VPN PPN Valid VPN PPN Valid 03 B 1 00 7 1 10 6 0 07 6 0 01 8 1 11 7 0 0 28 3 1 02 9 1 12 8 0 01 F 0 03 A 1 13 3 0 31 0 1 04 6 0 14 D 0 12 3 0 05 3 0 15 B 0 1 07 E 1 06 1 0 16 9 0 0B 1 1 07 8 0 17 6 0 2A A 0 08 2 0 18 C 1 11 1 0 09 3 0 19 4 1 2 1F 8 1 0A 1 1 1A F 0 07 5 1 0B 6 1 1B 2 1 07 3 1 0C A 1 1C 0 0 3F F 0 0D D 0 1D E 1 3 10 D 0 0E E 0 1E 5 1 32 0 0 0F D 1 1F 3 1 Question 1a: The box below shows the format of a virtual address. Indicate (by labeling the diagram) the fields that would be used to determine the following: 1 VPO The virtual page offset VPN The virtual page number TLBI The TLB index TLBT The TLB tag 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 0 VPN VPO TLBT TLBI Question 1b: The box below shows the format of a physical address. -
CS420: Operating Systems Paging and Page Tables
CS420: Operating Systems Paging and Page Tables YORK COLLEGE OF PENNSYLVANIA YORK COLLEGE OF PENNSYLVANIA YORK COLLEGE OF PENNSYLVANIA YORK COLLEGE OF PENNSYLVANIA 'GHI<GJHK&L<MNK'GHI<GJHK&L<MNK 'GONJHK&P@JJHG 'GONJHK&P@JJHG MFIF&'<IJ@QH&'@OK MFIF&'<IJ@QH&'@OK !<GR%&'@ !<GR%&'@ James Moscola 'HGPOJ&N<F&012 'HGPOJ&N<F&012 !"#$%&'())*+,-.)/.&0123454670!"#$%&'())*+,-.)/.&0123454670 YORK COLLEGE OF PENNSYLVANIA Department of Physical Sciences 'GHI<GJHK&L<MNK 8.9:;*&<:(#.="#>&1015?26511??8.9:;*&<:(#.="#>&1015?26511?? @A9/**/")*&<B!&C(>&1015?2D50633@A9/**/")*&<B!&C(>&1015?2D50633'GONJHK&P@JJHG York College of Pennsylvania 05?3352775?30? 05?3352775?30? MFIF&'<IJ@QH&'@OK EEEF+C:F(A; EEEF+C:F(A; !<GR%&'@ 'HGPOJ&N<F&012 !""#$%%&'$#()*$&+$,-$%.$"!""#$%%&'$#()*$&+$,-$%.$" !"#$%&'())*+,-.)/.&0123454670 8.9:;*&<:(#.="#>&1015?26511?? @A9/**/")*&<B!&C(>&1015?2D50633 05?3352775?30? EEEF+C:F(A; !""#$%%&'$#()*$&+$,-$%.$" CS420: Operating Systems Based on Operating System Concepts, 9th Edition by Silberschatz, Galvin, Gagne COLLEGE CATALOG 2009–2011 COLLEGE CATALOG COLLEGE CATALOG 2009–2011 COLLEGE CATALOG COLLEGE CATALOG 2009–2011 COLLEGE CATALOG College Catalog 2009–2011College Catalog 2009–2011 College Catalog 2009–2011 !""#$%&'()*+,--.../ 012$1$"..."34#3$4.56 !""#$%&'()*+,--.../ 012$1$"..."34#3$4.56 !""#$%&'()*+,--.../ 012$1$"..."34#3$4.56 Paging • Paging is a memory-management scheme that permits the physical address space of a process to be noncontiguous - Avoids external fragmentation - Avoids the need for compaction - May still have internal fragmentation