Proceedings Template

Total Page:16

File Type:pdf, Size:1020Kb

Proceedings Template OS Paradigms Adaptation to Fit New Architectures Xavier Joglar Judit Planas Marisa Gil Universitat Politècnica de Catalunya Universitat Politècnica de Catalunya Universitat Politècnica de Catalunya Jordi Girona 1-3 Jordi Girona 1-3 Jordi Girona 1-3 Barcelona, Spain Barcelona, Spain Barcelona, Spain [email protected] [email protected] [email protected] ABSTRACT In this way, they can be built from components that work Future architectures and computer systems will be better in specific engines in a more energy-saving way. heterogeneous multi-core models, which will improve their These specific cores are also suitable to execute specific performance, resource utilization and energy consumption. parts of a program to improve its overall throughput. Differences between cores mean different binary formats From this preliminary point, some aspects must be analyzed and specific concerns when dealing with applications. The and modified. In a first stage, programming and compiling OS also needs to manage the appropriate information to parallel applications should be heterogeneity-aware, schedule resources to achieve the optimal performance. deciding and building the appropriate executing stream. In this paper we present a first approach in Linux to allow Then, a specific management task must be done by the the application to give information to the OS in order to linker and the loader to compose and load the final perform the best resource scheduling for the code executable binary, interpreting its new information and characteristics (where it has to run). Based on the performing the appropriate actions. Finally, the operating continuation model of the Mach microkernel and the device system must manage, at runtime, the physical resources. drivers of Unix-Linux system, the kernel can continue the To take advantage of these specialized units, the OS must execution flow from one core to another (i.e. from PPE to be able to schedule the code parts onto the most appropriate SPE in the Cell BE case). In this way, the OS can unit depending on the code requirements as well as the anticipate costly actions (for example, loading code or available execution units [5]. In this work, programmers data) or reserve resources depending on task needs. can help the OS to manage the system resources more To reach our target, we adapt the operating system as well efficiently by giving some hints about the application as modify the application binary to divide its code parts behaviour. For example, if the programmer knows that a depending on their characteristics and where they have to function or a loop has special characteristics, he can mark run. the code properly. That is why it is necessary to add some additional information in the executables and make the Experimental work has been done for x86 with MMX operating system able to manage and understand the given extension ISA as well as for PPC and Cell BE. information. Some requirements to achieve this OS improvement are: Keywords Heterogeneous multi-core, OS loader, fat binary. Having the application code portions in the binary (or bit stream map) corresponding to the platform where it 1. INTRODUCTION has to run. Current technology trends move to new processors with Having a “fat binary” executable file containing the specialized cores, like Cell BE [6] or the Intel Exoskeleton different binary parts of code [11]. [11] with specific processor elements such as those which are GPU-based [7], or other accelerator elements, including Allowing threads to run sequentially cross-ISA (i.e. DSP and FPGA units. In this scenario, programs must be from one unit to one of a different type) [9]. redesigned in order to fully exploit this heterogeneity and Providing the OS with the requisite information to improve their performance. schedule a thread in the appropriate engine (depending on the ISA, processor affinity or system load). This work is supported by the European Commission in the context of the SARC Integrated Project (EU contract 27648- FP6), the HiPEAC European Network of Excellence and the Ministry of Science and Technology of Spain and the European Union (FEDER) under contract TIN2007-60625. The main contributions included in this paper are: The type of the new sections created for each piece of code compiled for a different ISA: CESOF creates An extensible runtime fully compatible with both read-only data sections and we create text sections2. current and new architectures. CESOF is focused on Cell BE; we want HELF to be A new loader to manage new binary format. adaptable to as many heterogeneous platforms as New OS objects to hold application heterogeneity possible. information. Programming models using the ELF format for The rest of the document is organised in the following way: heterogeneous multi-core are: in the next section, we describe the environment and introduce a brief idea of our target. In the third section we The Cell Superscalar (CellSs) [1]: provides a simple, explain the modifications at application level as well as the flexible programming model that can take advantage API implemented to make the interaction between the user of the performance benefits from the Cell. CellSs is and operating system level easier. In section four we built from two principal components: the CellSs present the OS adaptations to be able to manage, store and compiler and a runtime library. The compiler is a use the new information. In section five we discuss our source-to-source translator that takes in an annotated results and in section six, about the conclusions and future C source file and produces a pair of source files, one work. Finally, in section seven we give the references where for each Cell processor type (PPU and SPU). The readers can find more information related to our work. runtime constructs a data dependency graph to schedule independent annotated functions to execute 2. OUR APPROACH on different SPUs in parallel. CellSs uses CESOF. As mentioned above, hardware performance can be limited The EXO-CHI for the Intel Exoskeleton [11]: uses by its software resource management. So as hardware accelerator elements in a heterogeneous multi- evolves, software has to evolve too, and vice versa. We threaded execution model and provides the tools thought that adapting the executable format to distribute needed to exploit the system performance and reduce application execution into different processor units would power consumption. be an exciting chance to improve system performance. The loader must be able to identify HELF files and classify Our proposal is to include additional information into the the different parts of code depending on the information binary file so that the operating system can understand it added by the programmer or, automatically, by the compiler and manage system resources as efficiently as possible [8]. to the ELF sections. In this way, the operating system will schedule them in the appropriate processor. We looked for a simple and portable format through different architectures. We considered that the best option To achieve this, we have to consider three aspects: would be to modify the ELF format [10] to make the 1. How to adapt binary code in order to split it into transformation of the conventional ELF files easier. different chunks of execution code3. We call our new extension Heterogeneous ELF (HELF) and 2. How to modify the Linux loader in order to we use the ELF sections to store the information that the understand and load heterogeneous binaries. operating system needs. HELF extension philosophy is based on the idea of embedding different binaries compiled 3. Which new data structures have to be added to the for different ISAs in a single binary file. kernel to store the new information needed. The CESOF1 extension used for Cell BE programs also For the moment, we are not focusing on data sharing, so allows different-ISA binary in a single ELF-format file. some changes might be necessary to adapt our proposal to CESOF embeds compiled code for both the PPE and SPE non shared-memory architectures. in a single file. It uses several ELF sections to include the When a code has to continue its execution in a specific embedded SPE executable image with additional PPE core, for reasons such as being compiled for a special ISA symbol information. or to obtain better performance in an accelerator, the only The two main differences between HELF and CESOF are: information that has to be provided is where it continues (an 2 For now, we are just considering self-contained functions, so that we do not concern about data. 1 More information related to CESOF (such as diagrams or 3 This is a programming and compiler concern, so we will code examples) can be found in [2]. not explain the details of this adaptation in this document. address) and which input data it needs (the parameters). We will use as an example a matrix multiplication This is the same as what the explicit continuation model implemented for the Cell BE. The structure of the original does [3]. code is shown in Figure 1. The PPE creates and initializes the matrices, sends them to the SPE and waits for the We have implemented new library calls to specify the entry results. The SPE receive the matrixes, performs the point and the unit identifier. Based on this information, the computation and sends back the results and performance to system is able to get the required data and prepare the the PPE. Finally, the PPE verifies the results and prints out loading and running environment in the new unit. the performance reached. The matrix sizes and the number In the next sections we explain the application and the of SPEs to use are configurable by command line operating system adaptation to allow that.
Recommended publications
  • Cygwin User's Guide
    Cygwin User’s Guide Cygwin User’s Guide ii Copyright © Cygwin authors Permission is granted to make and distribute verbatim copies of this documentation provided the copyright notice and this per- mission notice are preserved on all copies. Permission is granted to copy and distribute modified versions of this documentation under the conditions for verbatim copying, provided that the entire resulting derived work is distributed under the terms of a permission notice identical to this one. Permission is granted to copy and distribute translations of this documentation into another language, under the above conditions for modified versions, except that this permission notice may be stated in a translation approved by the Free Software Foundation. Cygwin User’s Guide iii Contents 1 Cygwin Overview 1 1.1 What is it? . .1 1.2 Quick Start Guide for those more experienced with Windows . .1 1.3 Quick Start Guide for those more experienced with UNIX . .1 1.4 Are the Cygwin tools free software? . .2 1.5 A brief history of the Cygwin project . .2 1.6 Highlights of Cygwin Functionality . .3 1.6.1 Introduction . .3 1.6.2 Permissions and Security . .3 1.6.3 File Access . .3 1.6.4 Text Mode vs. Binary Mode . .4 1.6.5 ANSI C Library . .4 1.6.6 Process Creation . .5 1.6.6.1 Problems with process creation . .5 1.6.7 Signals . .6 1.6.8 Sockets . .6 1.6.9 Select . .7 1.7 What’s new and what changed in Cygwin . .7 1.7.1 What’s new and what changed in 3.2 .
    [Show full text]
  • Chapter 1. Origins of Mac OS X
    1 Chapter 1. Origins of Mac OS X "Most ideas come from previous ideas." Alan Curtis Kay The Mac OS X operating system represents a rather successful coming together of paradigms, ideologies, and technologies that have often resisted each other in the past. A good example is the cordial relationship that exists between the command-line and graphical interfaces in Mac OS X. The system is a result of the trials and tribulations of Apple and NeXT, as well as their user and developer communities. Mac OS X exemplifies how a capable system can result from the direct or indirect efforts of corporations, academic and research communities, the Open Source and Free Software movements, and, of course, individuals. Apple has been around since 1976, and many accounts of its history have been told. If the story of Apple as a company is fascinating, so is the technical history of Apple's operating systems. In this chapter,[1] we will trace the history of Mac OS X, discussing several technologies whose confluence eventually led to the modern-day Apple operating system. [1] This book's accompanying web site (www.osxbook.com) provides a more detailed technical history of all of Apple's operating systems. 1 2 2 1 1.1. Apple's Quest for the[2] Operating System [2] Whereas the word "the" is used here to designate prominence and desirability, it is an interesting coincidence that "THE" was the name of a multiprogramming system described by Edsger W. Dijkstra in a 1968 paper. It was March 1988. The Macintosh had been around for four years.
    [Show full text]
  • Intel® Oneapi Programming Guide
    Intel® oneAPI Programming Guide Intel Corporation www.intel.com Notices and Disclaimers Contents Notices and Disclaimers....................................................................... 5 Chapter 1: Introduction oneAPI Programming Model Overview ..........................................................7 Data Parallel C++ (DPC++)................................................................8 oneAPI Toolkit Distribution..................................................................9 About This Guide.......................................................................................9 Related Documentation ..............................................................................9 Chapter 2: oneAPI Programming Model Sample Program ..................................................................................... 10 Platform Model........................................................................................ 14 Execution Model ...................................................................................... 15 Memory Model ........................................................................................ 17 Memory Objects.............................................................................. 19 Accessors....................................................................................... 19 Synchronization .............................................................................. 20 Unified Shared Memory.................................................................... 20 Kernel Programming
    [Show full text]
  • Pooch Manual In
    What’s New As of August 21, 2011, Pooch is updated to version 1.8.3 for use with OS X 10.7 “Lion”: Pooch users can renew their subscriptions today! Please see http://daugerresearch.com/pooch for more! On November 17, 2009, Pooch was updated to version 1.8: • Linux: Pooch can now cluster nodes running 64-bit Linux, combined with Mac • 64-bit: Major internal revisions for 64-bit, particularly updated data types and structures, for Mac OS X 10.6 "Snow Leopard" and 64-bit Linux • Sockets: Major revisions to internal networking to adapt to BSD Sockets, as recommended by Apple moving forward and required for Linux • POSIX Paths: Major revisions to internal file specification format in favor of POSIX paths, recommended by Apple moving forward and required for Linux • mDNS: Adapted usage of Bonjour service discovery to use Apple's Open Source mDNS library • Pooch Binary directory: Added Pooch binary directory support, making possible launching jobs using a remotely-compiled executable • Minor updates and fixes needed for Mac OS X 10.6 "Snow Leopard" Current Pooch users can renew their subscriptions today! Please see http://daugerresearch.com/pooch for more! On April 16, 2008, Pooch was updated to version 1.7.6: • Mac OS X 10.5 “Leopard” spurs updates in a variety of Pooch technologies: • Network Scan window • Preferences window • Keychain access • Launching via, detection of, and commands to the Terminal • Behind the Login window behavior • Other user interface and infrastructure adjustments • Open MPI support: • Complete MPI support using libraries
    [Show full text]
  • Cygwin User's Guide
    Cygwin User’s Guide i Cygwin User’s Guide Cygwin User’s Guide ii Copyright © 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011, 2012 Red Hat, Inc. Permission is granted to make and distribute verbatim copies of this documentation provided the copyright notice and this per- mission notice are preserved on all copies. Permission is granted to copy and distribute modified versions of this documentation under the conditions for verbatim copying, provided that the entire resulting derived work is distributed under the terms of a permission notice identical to this one. Permission is granted to copy and distribute translations of this documentation into another language, under the above conditions for modified versions, except that this permission notice may be stated in a translation approved by the Free Software Foundation. Cygwin User’s Guide iii Contents 1 Cygwin Overview 1 1.1 What is it? . .1 1.2 Quick Start Guide for those more experienced with Windows . .1 1.3 Quick Start Guide for those more experienced with UNIX . .1 1.4 Are the Cygwin tools free software? . .2 1.5 A brief history of the Cygwin project . .2 1.6 Highlights of Cygwin Functionality . .3 1.6.1 Introduction . .3 1.6.2 Permissions and Security . .3 1.6.3 File Access . .3 1.6.4 Text Mode vs. Binary Mode . .4 1.6.5 ANSI C Library . .5 1.6.6 Process Creation . .5 1.6.6.1 Problems with process creation . .5 1.6.7 Signals . .6 1.6.8 Sockets . .6 1.6.9 Select .
    [Show full text]
  • GNU MP the GNU Multiple Precision Arithmetic Library Edition 6.2.0 17 January 2020
    GNU MP The GNU Multiple Precision Arithmetic Library Edition 6.2.0 17 January 2020 by Torbj¨ornGranlund and the GMP development team This manual describes how to install and use the GNU multiple precision arithmetic library, version 6.2.0. Copyright 1991, 1993-2016, 2018 Free Software Foundation, Inc. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.3 or any later version published by the Free Software Foundation; with no Invariant Sections, with the Front-Cover Texts being \A GNU Manual", and with the Back-Cover Texts being \You have freedom to copy and modify this GNU Manual, like GNU software". A copy of the license is included in Appendix C [GNU Free Documentation License], page 130. i Table of Contents GNU MP Copying Conditions ::::::::::::::::::::::::::::::::::: 1 1 Introduction to GNU MP :::::::::::::::::::::::::::::::::::: 2 1.1 How to use this Manual ::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: 2 2 Installing GMP :::::::::::::::::::::::::::::::::::::::::::::::: 3 2.1 Build Options ::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: 3 2.2 ABI and ISA :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: 8 2.3 Notes for Package Builds ::::::::::::::::::::::::::::::::::::::::::::::::::::::::: 11 2.4 Notes for Particular Systems ::::::::::::::::::::::::::::::::::::::::::::::::::::: 12 2.5 Known Build Problems ::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: 14 2.6 Performance optimization
    [Show full text]
  • Ios Hacking Guide.Pdf
    Hacking iOS Applications a detailed testing guide Prepared by: Dinesh Shetty, Sr. Manager - Information Security @Din3zh 2 Table of Contents 1. Setting Up iOS Pentest Lab ................................................................................................. 5 1.1 Get an iOS Device ................................................................................................................................ 5 1.2 Jailbreaking an iOS Device................................................................................................................... 7 1.3 Installing Required Software and Utilities ........................................................................................ 10 2. Acquiring iOS Binaries ...................................................................................................... 13 3. Generating iOS Binary (.IPA file) from Xcode Source Code: ............................................... 15 3.1 Method I – With A Valid Paid Developer Account. ........................................................................... 15 3.2 Method II - Without a Valid Paid Developer Account ....................................................................... 18 4. Installing iOS Binaries on Physical Devices ........................................................................ 23 4.1 Method I - Using iTunes .................................................................................................................... 23 4.2 Method II - Using Cydia Impactor ....................................................................................................
    [Show full text]
  • Sage Installation Guide Release 9.4
    Sage Installation Guide Release 9.4 The Sage Development Team Aug 24, 2021 CONTENTS 1 Linux package managers 3 2 Install from Pre-built Binaries5 2.1 Download Guide.............................................5 2.2 Linux...................................................5 2.3 macOS..................................................6 2.4 Microsoft Windows (Cygwin)......................................6 3 Install from conda-forge 7 4 Install from Source Code 9 4.1 Supported platforms...........................................9 4.2 Prerequisites............................................... 10 4.3 Additional software........................................... 18 4.4 Step-by-step installation procedure.................................... 20 4.5 Make targets............................................... 25 4.6 Environment variables.......................................... 25 4.7 Installation in a Multiuser Environment................................. 30 5 Launching SageMath 33 5.1 Using a Jupyter Notebook remotely................................... 33 5.2 WSL Post-installation steps....................................... 34 5.3 Setting up SageMath as a Jupyter kernel in an existing Jupyter notebook or JupyterLab installation.. 35 6 Troubleshooting 37 Index 39 i ii Sage Installation Guide, Release 9.4 You can install SageMath either from a package manager, a pre-built binary tarball or from its sources. Installing SageMath from your distribution package manager is the preferred and fastest solution (dependencies will be automatically taken care of and SageMath will be using your system Python). It is the case at least for the following GNU/Linux distributions: Debian version >= 9, Ubuntu version >= 18.04, Arch Linux, and NixOS. If you are in this situation, see Linux package managers. If your operating system does not provide SageMath, you can also use a pre-built binary. See the section Install from Pre-built Binaries. Or you could install the sage package from the conda-forge project.
    [Show full text]
  • Portable Executable
    Portable Executable The Portable Executable (PE) format is a file format for executables, object code, Portable Executable DLLs and others used in 32-bit and 64-bit Filename .acm, .ax, .cpl, .dll, .drv, .efi, versions of Windows operating systems. extension .exe, .mui, .ocx, .scr, .sys, .tsp The PE format is a data structure that Internet application/vnd.microsoft.portable- encapsulates the information necessary for media type executable[1] the Windows OS loader to manage the Developed by Currently: Microsoft wrapped executable code. This includes dynamic library references for linking, API Type of format Binary, executable, object, shared libraries export and import tables, resource Extended from DOS MZ executable management data and thread-local storage COFF (TLS) data. On NT operating systems, the PE format is used for EXE, DLL, SYS (device driver), MUI and other file types. The Unified Extensible Firmware Interface (UEFI) specification states that PE is the standard executable format in EFI environments.[2] On Windows NT operating systems, PE currently supports the x86-32, x86-64 (AMD64/Intel 64), IA-64, ARM and ARM64 instruction set architectures (ISAs). Prior to Windows 2000, Windows NT (and thus PE) supported the MIPS, Alpha, and PowerPC ISAs. Because PE is used on Windows CE, it continues to support several variants of the MIPS, ARM (including Thumb), and SuperH ISAs. [3] Analogous formats to PE are ELF (used in Linux and most other versions of Unix) and Mach-O (used in macOS and iOS). Contents History Technical details Layout Import table Relocations .NET, metadata, and the PE format Use on other operating systems See also References External links History Microsoft migrated to the PE format from the 16-bit NE formats with the introduction of the Windows NT 3.1 operating system.
    [Show full text]
  • A Lightweight Packaging and Execution Environment for Compact, Multi-Architecture Binaries
    GridRun: A lightweight packaging and execution environment for compact, multi-architecture binaries John Shalf Lawrence Berkeley National Laboratory 1 Cyclotron Road, Berkeley California 94720 {[email protected]} Tom Goodale Louisiana State University Baton Rouge, LA 70803 {[email protected]} applications and will continue to play a very important Abstract role in distributed applications for the foreseeable future. Therefore, we focus our attention squarely on GridRun offers a very simple set of tools for creating the issue of simplifying the management and and executing multi-platform binary executables. distribution native executables as multiplatform binary These "fat-binaries" archive native machine code into packages (fat-binaries). compact packages that are typically a fraction the size of the original binary images they store, enabling In order to support a seamless multiplatform execution efficient staging of executables for heterogeneous environment in lieu of virtual machines, we extend the parallel jobs. GridRun interoperates with existing familiar concept of the “fat-binary” to apply to multi- distributed job launchers/managers like Condor and operating system environments. The fat-binary has the Globus GRAM to greatly simplify the logic been a very well-known and successful design pattern required launching native binary applications in for smoothing major instruction-set-architecture distributed heterogeneous environments. transitions within a single Operating System environment. A fat-binary file contains complete Introduction images of the binary executables for each CPU instruction set it supports. The operating system’s Grid computing makes a vast number of file-loader then selects and executes appropriate heterogeneous computing systems available as a binary image for the CPU architecture that is running virtual computing resource.
    [Show full text]
  • Real-Time Streaming Multi-Pattern Search for Constant Alphabet∗
    Real-Time Streaming Multi-Pattern Search for Constant Alphabet∗ Shay Golan1 and Ely Porat2 1 Bar Ilan University, Ramat Gan, Israel [email protected] 2 Bar Ilan University, Ramat Gan, Israel [email protected] Abstract In the streaming multi-pattern search problem, which is also known as the streaming dictionary matching problem, a set D = {P1,P2,...,Pd} of d patterns (strings over an alphabet Σ), called the dictionary, is given to be preprocessed. Then, a text T arrives one character at a time and the goal is to report, before the next character arrives, the longest pattern in the dictionary that is a current suffix of T . We prove that for a constant size alphabet, there exists a randomized Monte-Carlo algorithm for the streaming dictionary matching problem that takes constant time per character and uses O(d log m) words of space, where m is the length of the longest pattern in the dictionary. In the case where the alphabet size is not constant, we introduce two new randomized Monte-Carlo algorithms with the following complexities: O(log log |Σ|) time per character in the worst case and O(d log m) words of space. 1 ε m O( ε ) time per character in the worst case and O(d|Σ| log ε ) words of space for any 0 < ε ≤ 1. These results improve upon the algorithm of Clifford et al. [12] which uses O(d log m) words of space and takes O(log log(m + d)) time per character. 1998 ACM Subject Classification F.2.2 Nonnumerical Algorithms and Problems Keywords and phrases multi-pattern, dictionary, streaming pattern matching, fingerprints Digital Object Identifier 10.4230/LIPIcs.ESA.2017.41 1 Introduction We consider one of the most fundamental pattern matching problems, the dictionary matching problem [12, 16, 5, 6, 30, 20, 7, 21, 17, 18, 4], where a set of patterns D = {P1,P2,...,Pd}, called the dictionary, is given along with a string T , called the text, such that each pattern Pi is a string of length mi, and all the strings are over an alphabet Σ.
    [Show full text]
  • Optimized Composition: Generating Efficient Code for Heterogeneous
    Optimized Composition: Generating Efficient Code for Heterogeneous Systems from Multi-Variant Components, Skeletons and Containers Christoph Kessler Usman Dastgeer Lu Li IDA, Linkoping¨ University IDA, Linkoping¨ University IDA, Linkoping¨ University S-581 83 Linkoping,¨ Sweden S-581 83 Linkoping,¨ Sweden S-581 83 Linkoping,¨ Sweden [email protected] [email protected] [email protected] Abstract—In this survey paper, we review recent work for each call, as well as scheduling and resource allocation on frameworks for the high-level, portable programming of for them and the optimization of operand data transfers heterogeneous multi-/manycore systems (especially, GPU-based between different memory units. We refer to this combined systems) using high-level constructs such as annotated user- level software components, skeletons (i.e., predefined generic problem of code generation and optimization as optimized components) and containers, and discuss the optimization composition. problems that need to be considered in selecting among mul- In this survey paper we give a review of component- tiple implementation variants, generating code and providing based programming frameworks for heterogeneous multi- runtime support for efficient execution on such systems. /manycore (esp., GPU-based) systems and of the specific problems and techniques for generating efficient code. In I. INTRODUCTION particular, we will elaborate on annotated multi-variant Heterogeneous manycore systems offer a remarkable po- components and their composition (Section II), multi- tential for energy-efficient computing and high performance; variant skeletons (Section III), and smart containers for run- in particular, GPU-based systems have become very popular. time memory management and communication optimization However, efficient programming of such systems is difficult (Section IV).
    [Show full text]