DOCSLIB.ORG

Fibers Are Not (P)Threads the Case for Loose Coupling of Asynchronous Programming Models and MPI Through Continuations

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Fibers Are Not (P)Threads the Case for Loose Coupling of Asynchronous Programming Models and MPI Through Continuations Fibers are not (P)Threads The Case for Loose Coupling of Asynchronous Programming Models and MPI Through Continuations Joseph Schuchart Christoph Niethammer José Gracia [email protected] [email protected] [email protected] Höchstleistungsrechenzentrum Höchstleistungsrechenzentrum Höchstleistungsrechenzentrum Stuttgart (HLRS) Stuttgart (HLRS) Stuttgart (HLRS) Stuttgart, Germany Stuttgart, Germany Stuttgart, Germany ABSTRACT multi- and many-core systems by exposing all available concur- Asynchronous programming models (APM) are gaining more and rency to a runtime system. Applications are expressed in terms of more traction, allowing applications to expose the available concur- work-packages (tasks) with well-defined inputs and outputs, which rency to a runtime system tasked with coordinating the execution. guide the runtime scheduler in determining the correct execution While MPI has long provided support for multi-threaded commu- order of tasks. A wide range of task-based programming models nication and non-blocking operations, it falls short of adequately have been developed, ranging from node-local approaches such supporting APMs as correctly and efficiently handling MPI com- as OpenMP [40] and OmpSs [13] to distributed systems such as munication in different models is still a challenge. Meanwhile, new HPX [27], StarPU [3], DASH [49], and PaRSEC [7]. All of them have low-level implementations of light-weight, cooperatively scheduled in common that a set of tasks is scheduled for execution by a set of execution contexts (fibers, aka user-level threads (ULT)) are meant to worker threads based on constraints provided by the user, either in serve as a basis for higher-level APMs and their integration in MPI the form of dependencies or dataflow expressions. implementations has been proposed as a replacement for traditional At the same time, MPI is still the dominant interface for inter- POSIX thread support to alleviate these challenges. process communication in parallel applications [6], providing block- In this paper, we first establish a taxonomy in an attempt to ing and non-blocking point-to-point and collective operations as clearly distinguish different concepts in the parallel software stack. well as I/O capabilities on top of elaborate datatype and process We argue that the proposed tight integration of fiber implemen- group abstractions [37]. Recent years have seen significant improve- tations with MPI is neither warranted nor beneficial and instead ments to the way MPI implementations handle communication by is detrimental to the goal of MPI being a portable communication multiple threads of execution in parallel [20, 41]. abstraction. We propose MPI Continuations as an extension to the However, task-based programming models pose new challenges MPI standard to provide callback-based notifications on completed to the way applications interact with MPI. While MPI provides operations, leading to a clear separation of concerns by providing non-blocking communications that can be used to hide commu- a loose coupling mechanism between MPI and APMs. We show nication latencies, it is left to the application layer to ensure that that this interface is flexible and interacts well with different APMs, all necessary communication operations are eventually initiated namely OpenMP detached tasks, OmpSs-2, and Argobots. to avoid deadlocks, that in-flight communication operations are eventually completed, and that the interactions between communi- KEYWORDS cating tasks are correctly handled. Higher-level tasking approaches such as PaRSEC or HPX commonly track the state of active com- MPI+X, Tasks, Continuations, OpenMP, OmpSs, TAMPI, Fiber, ULT munication operations and regularly test for completion before ACM Reference Format: acting upon such a change in state. With node-local programming Joseph Schuchart, Christoph Niethammer, and José Gracia. 2020. Fibers are models such as OpenMP, this tracking of active communication not (P)Threads: The Case for Loose Coupling of Asynchronous Program- operations is left to the application layer. Unfortunately, the simple ming Models and MPI Through Continuations. In 27th European MPI Users’ approach of test-yield cycles inside a task does not provide optimal Group Meeting (EuroMPI/USA ’20), September 21–24, 2020, Austin, TX, USA. arXiv:2011.06684v1 [cs.DC] 12 Nov 2020 ACM, New York, NY, USA, 12 pages. https://doi.org/10.1145/3416315.3416320 performance due to CPU cycles being wasted on testing and (in the case of OpenMP) may not even be portable [50]. 1 BACKGROUND AND MOTIVATION MPI is thus currently not well equipped to support users and developers of asynchronous programming models, which (among Asynchronous (task-based) programming models have gained more other things) has prompted some higher level runtimes to move and more traction, promising to help users better utilize ubiquitous away from MPI towards more asynchronous APIs [10]. Permission to make digital or hard copies of all or part of this work for personal or Different approaches have been proposed to mitigate this burden, classroom use is granted without fee provided that copies are not made or distributed including Task-Aware MPI (TAMPI) [46] and a tight integration of for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the user-level thread, fiber, and tasking libraries (cf. Section 2) with MPI author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or implementations [32, 34, 45]. In this paper, we argue that the latter republish, to post on servers or to redistribute to lists, requires prior specific permission is the least preferable solution and in fact should be seen critical and/or a fee. Request permissions from [email protected]. 1 EuroMPI/USA ’20, September 21–24, 2020, Austin, TX, USA by the MPI community. After establishing a taxonomy to be used © 2020 Copyright held by the owner/author(s). Publication rights licensed to ACM. ACM ISBN 978-1-4503-8880-1/20/09...$15.00 1This paper is in part an extended version of an argument made online at https: https://doi.org/10.1145/3416315.3416320 //github.com/open-mpi/ompi/pull/6578#issuecomment-602542762. EuroMPI/USA ’20, September 21–24, 2020, Austin, TX, USA Joseph Schuchart, Christoph Niethammer, and José Gracia Process resources such as memory pages and file pointers between different Tasks processes. A process may use one or more light-weight processes. Task Scheduler Light-weight Process (LWP). A light-weight process can be con- Fibers sidered an abstraction of a processor inside the operating system (OS), on which threads are scheduled. LWPs are scheduled preemp- Fiber Scheduler User tively by the OS kernel, i.e., their execution may be preempted and Space User another LWP be scheduled onto the used processor. Different LWPs Threads User Thread Scheduler may share resources such as file descriptors and memory if they Kernel belong to the same process. Space Cooperative and Preemptive Scheduling. These are two main Kernel LWP Threads scheduling strategies to distinguish. With cooperative scheduling, a scheduler assigns resources to a process or thread until it relin- Kernel Scheduler quishes them. The benefits of cooperative scheduling are lower Hardware scheduling overheads (scheduling decisions are made only at well- Processors defined points) and—at least on single-core systems—atomicity of execution between scheduling points [1]. With preemptive schedul- Figure 1: Hierarchy of kernel and user threads, fibers, and ing, the scheduler is able to preempt the execution of a thread after tasks with the different scheduler entities. The user thread the expiration of a time-slice or to schedule processes or threads scheduler is optional and not present in most modern archi- with higher priority. Most modern OS kernels employ preemptive tectures. Tasks may be executed directly on top of threads scheduling to ensure fairness and responsiveness. without the use of fibers. User Thread. User threads are threads of execution that share resources within the confines of a process. # threads may be stati- cally assigned to # LWP (1:1 mapping) or multiplexed onto either throughout this paper (Section 2), we will make the argument that a single (1:N mapping) or " LWPs (N:M mapping) by a user thread such an integration is neither warranted by the MPI standard nor scheduler. In all cases, threads are comprised of some private state beneficial to the goal of supporting the development of portable MPI (stack memory, register context, thread-local variables, signal masks, applications (Section 3). Instead, we propose adding continuations errno value, scheduling priority and policy) and have access to re- to the MPI standard to facilitate a loose coupling between higher- sources shared between threads within a process (heap memory, level concurrency abstractions and MPI (Section 4). We will show global variables, file descriptors, etc). that the interface is flexible and can be used with abstractions on A common example of a user thread implementation are POSIX different levels, including OpenMP detached tasks, OmpSs-2 task threads (pthreads)[24]. In all currently available Unix flavors known events, and Argobots (Section 5). to the authors, a 1:1 mapping between pthreads and LWPs is main- tained, including Linux [31] and various BSD flavors [55–57]. They 2 TAXONOMY are the basic building block for any type of multi-threaded ap- Unfortunately, the
Load more

Recommended publications
  • Other Apis What’S Wrong with Openmp?
    Threaded Programming Other APIs What’s wrong with OpenMP? • OpenMP is designed for programs where you want a fixed number of threads, and you always want the threads to be consuming CPU cycles. – cannot arbitrarily start/stop threads – cannot put threads to sleep and wake them up later • OpenMP is good for programs where each thread is doing (more-or-less) the same thing. • Although OpenMP supports C++, it’s not especially OO friendly – though it is gradually getting better. • OpenMP doesn’t support other popular base languages – e.g. Java, Python What’s wrong with OpenMP? (cont.) Can do this Can do this Can’t do this Threaded programming APIs • Essential features – a way to create threads – a way to wait for a thread to finish its work – a mechanism to support thread private data – some basic synchronisation methods – at least a mutex lock, or atomic operations • Optional features – support for tasks – more synchronisation methods – e.g. condition variables, barriers,... – higher levels of abstraction – e.g. parallel loops, reductions What are the alternatives? • POSIX threads • C++ threads • Intel TBB • Cilk • OpenCL • Java (not an exhaustive list!) POSIX threads • POSIX threads (or Pthreads) is a standard library for shared memory programming without directives. – Part of the ANSI/IEEE 1003.1 standard (1996) • Interface is a C library – no standard Fortran interface – can be used with C++, but not OO friendly • Widely available – even for Windows – typically installed as part of OS – code is pretty portable • Lots of low-level control over behaviour of threads • Lacks a proper memory consistency model Thread forking #include <pthread.h> int pthread_create( pthread_t *thread, const pthread_attr_t *attr, void*(*start_routine, void*), void *arg) • Creates a new thread: – first argument returns a pointer to a thread descriptor.
    [Show full text]
  • Designing an Ultra Low-Overhead Multithreading Runtime for Nim
    Designing an ultra low-overhead multithreading runtime for Nim Mamy Ratsimbazafy Weave [email protected] https://github.com/mratsim/weave Hello! I am Mamy Ratsimbazafy During the day blockchain/Ethereum 2 developer (in Nim) During the night, deep learning and numerical computing developer (in Nim) and data scientist (in Python) You can contact me at [email protected] Github: mratsim Twitter: m_ratsim 2 Where did this talk came from? ◇ 3 years ago: started writing a tensor library in Nim. ◇ 2 threading APIs at the time: OpenMP and simple threadpool ◇ 1 year ago: complete refactoring of the internals 3 Agenda ◇ Understanding the design space ◇ Hardware and software multithreading: definitions and use-cases ◇ Parallel APIs ◇ Sources of overhead and runtime design ◇ Minimum viable runtime plan in a weekend 4 Understanding the 1 design space Concurrency vs parallelism, latency vs throughput Cooperative vs preemptive, IO vs CPU 5 Parallelism is not 6 concurrency Kernel threading 7 models 1:1 Threading 1 application thread -> 1 hardware thread N:1 Threading N application threads -> 1 hardware thread M:N Threading M application threads -> N hardware threads The same distinctions can be done at a multithreaded language or multithreading runtime level. 8 The problem How to schedule M tasks on N hardware threads? Latency vs 9 Throughput - Do we want to do all the work in a minimal amount of time? - Numerical computing - Machine learning - ... - Do we want to be fair? - Clients-server - Video decoding - ... Cooperative vs 10 Preemptive Cooperative multithreading:
    [Show full text]
  • Fiber Context As a first-Class Object
    Document number: P0876R6 Date: 2019-06-17 Author: Oliver Kowalke ([email protected]) Nat Goodspeed ([email protected]) Audience: SG1 fiber_context - fibers without scheduler Revision History . .1 abstract . .2 control transfer mechanism . .2 std::fiber_context as a first-class object . .3 encapsulating the stack . .4 invalidation at resumption . .4 problem: avoiding non-const global variables and undefined behaviour . .5 solution: avoiding non-const global variables and undefined behaviour . .6 inject function into suspended fiber . 11 passing data between fibers . 12 termination . 13 exceptions . 16 std::fiber_context as building block for higher-level frameworks . 16 interaction with STL algorithms . 18 possible implementation strategies . 19 fiber switch on architectures with register window . 20 how fast is a fiber switch . 20 interaction with accelerators . 20 multi-threading environment . 21 acknowledgments . 21 API ................................................................ 22 33.7 Cooperative User-Mode Threads . 22 33.7.1 General . 22 33.7.2 Empty vs. Non-Empty . 22 33.7.3 Explicit Fiber vs. Implicit Fiber . 22 33.7.4 Implicit Top-Level Function . 22 33.7.5 Header <experimental/fiber_context> synopsis . 23 33.7.6 Class fiber_context . 23 33.7.7 Function unwind_fiber() . 27 references . 29 Revision History This document supersedes P0876R5. Changes since P0876R5: • std::unwind_exception removed: stack unwinding must be performed by platform facilities. • fiber_context::can_resume_from_any_thread() renamed to can_resume_from_this_thread(). • fiber_context::valid() renamed to empty() with inverted sense. • Material has been added concerning the top-level wrapper logic governing each fiber. The change to unwinding fiber stacks using an anonymous foreign exception not catchable by C++ try / catch blocks is in response to deep discussions in Kona 2019 of the surprisingly numerous problems surfaced by using an ordinary C++ exception for that purpose.
    [Show full text]
  • Thread and Multitasking
    11/6/17 Thread and multitasking Alberto Bosio, Associate Professor – UM Microelectronic Departement [email protected] Agenda l POSIX Threads Programming l http://www.yolinux.com/TUTORIALS/LinuxTutorialPosixThreads.html l Compiler: http://cygwin.com/index.html 1 11/6/17 Process vs Thread process thread Picture source: https://computing.llnl.gov/tutorials/pthreads/ Shared Memory Model Picture source: https://computing.llnl.gov/tutorials/pthreads/ 2 11/6/17 Simple Thread Example void *func ( ) { /* define local data */ - - - - - - - - - - - - - - - - - - - - - - /* function code */ - - - - - - - - - - - pthread_exit(exit_value); } int main ( ) { pthread_t tid; int exit_value; - - - - - - - - - - - pthread_create (0, 0, func (), 0); - - - - - - - - - - - pthread_join (tid, &exit_value); - - - - - - - - - - - } Basic Functions Purpose Process Model Threads Model Creation of a new thread fork ( ) thr_create( ) Start execution of a new thread exec( ) [ thr_create() builds the new thread and starts the execution Wait for completion of thread wait( ) thr_join() Exit and destroy the thread exit( ) thr_exit() 3 11/6/17 Code comparison main ( ) main() { { fork ( ); thread_create(0,0,func(),0); fork ( ); thread_create(0,0,func(),0); fork ( ); thread_create(0,0,func(),0); } } Creation #include <pthread.h> int pthread_create( pthread_t *thread, const pthread_attr_t *attr, void *(*start_routine)(void*), void *arg); 4 11/6/17 Exit #include <pthread.h> void pthread_exit(void *retval); join #include <pthread.h> int pthread_join( pthread_t thread, void *retval); 5 11/6/17 Joining Passing argument (wrong example) int rc; long t; for(t=0; t<NUM_THREADS; t++) { printf("Creating thread %ld\n", t); rc = pthread_create(&threads[t], NULL, PrintHello, (void *) &t); ... } 6 11/6/17 Passing argument (good example) long *taskids[NUM_THREADS]; for(t=0; t<NUM_THREADS; t++) { taskids[t] = (long *) malloc(sizeof(long)); *taskids[t] = t; printf("Creating thread %ld\n", t); rc = pthread_create(&threads[t], NULL, PrintHello, (void *) taskids[t]); ..
    [Show full text]
  • Fiber-Based Architecture for NFV Cloud Databases
    Fiber-based Architecture for NFV Cloud Databases Vaidas Gasiunas, David Dominguez-Sal, Ralph Acker, Aharon Avitzur, Ilan Bronshtein, Rushan Chen, Eli Ginot, Norbert Martinez-Bazan, Michael Muller,¨ Alexander Nozdrin, Weijie Ou, Nir Pachter, Dima Sivov, Eliezer Levy Huawei Technologies, Gauss Database Labs, European Research Institute fvaidas.gasiunas,david.dominguez1,fi[email protected] ABSTRACT to a specific workload, as well as to change this alloca- The telco industry is gradually shifting from using mono- tion on demand. Elasticity is especially attractive to op- lithic software packages deployed on custom hardware to erators who are very conscious about Total Cost of Own- using modular virtualized software functions deployed on ership (TCO). NFV meets databases (DBs) in more than cloudified data centers using commodity hardware. This a single aspect [20, 30]. First, the NFV concepts of modu- transformation is referred to as Network Function Virtual- lar software functions accelerate the separation of function ization (NFV). The scalability of the databases (DBs) un- (business logic) and state (data) in the design and imple- derlying the virtual network functions is the cornerstone for mentation of network functions (NFs). This in turn enables reaping the benefits from the NFV transformation. This independent scalability of the DB component, and under- paper presents an industrial experience of applying shared- lines its importance in providing elastic state repositories nothing techniques in order to achieve the scalability of a DB for various NFs. In principle, the NFV-ready DBs that are in an NFV setup. The special combination of requirements relevant to our discussion are sophisticated main-memory in NFV DBs are not easily met with conventional execution key-value (KV) stores with stringent high availability (HA) models.
    [Show full text]
  • A Thread Performance Comparison: Windows NT and Solaris on a Symmetric Multiprocessor
    The following paper was originally published in the Proceedings of the 2nd USENIX Windows NT Symposium Seattle, Washington, August 3–4, 1998 A Thread Performance Comparison: Windows NT and Solaris on A Symmetric Multiprocessor Fabian Zabatta and Kevin Ying Brooklyn College and CUNY Graduate School For more information about USENIX Association contact: 1. Phone: 510 528-8649 2. FAX: 510 548-5738 3. Email: [email protected] 4. WWW URL:http://www.usenix.org/ A Thread Performance Comparison: Windows NT and Solaris on A Symmetric Multiprocessor Fabian Zabatta and Kevin Ying Logic Based Systems Lab Brooklyn College and CUNY Graduate School Computer Science Department 2900 Bedford Avenue Brooklyn, New York 11210 {fabian, kevin}@sci.brooklyn.cuny.edu Abstract Manufacturers now have the capability to build high performance multiprocessor machines with common CPU CPU CPU CPU cache cache cache cache PC components. This has created a new market of low cost multiprocessor machines. However, these machines are handicapped unless they have an oper- Bus ating system that can take advantage of their under- lying architectures. Presented is a comparison of Shared Memory I/O two such operating systems, Windows NT and So- laris. By focusing on their implementation of Figure 1: A basic symmetric multiprocessor architecture. threads, we show each system's ability to exploit multiprocessors. We report our results and inter- pretations of several experiments that were used to compare the performance of each system. What 1.1 Symmetric Multiprocessing emerges is a discussion on the performance impact of each implementation and its significance on vari- Symmetric multiprocessing (SMP) is the primary ous types of applications.
    [Show full text]
  • Introduction to Multi-Threading and Vectorization Matti Kortelainen Larsoft Workshop 2019 25 June 2019 Outline
    Introduction to multi-threading and vectorization Matti Kortelainen LArSoft Workshop 2019 25 June 2019 Outline Broad introductory overview: • Why multithread? • What is a thread? • Some threading models – std::thread – OpenMP (fork-join) – Intel Threading Building Blocks (TBB) (tasks) • Race condition, critical region, mutual exclusion, deadlock • Vectorization (SIMD) 2 6/25/19 Matti Kortelainen | Introduction to multi-threading and vectorization Motivations for multithreading Image courtesy of K. Rupp 3 6/25/19 Matti Kortelainen | Introduction to multi-threading and vectorization Motivations for multithreading • One process on a node: speedups from parallelizing parts of the programs – Any problem can get speedup if the threads can cooperate on • same core (sharing L1 cache) • L2 cache (may be shared among small number of cores) • Fully loaded node: save memory and other resources – Threads can share objects -> N threads can use significantly less memory than N processes • If smallest chunk of data is so big that only one fits in memory at a time, is there any other option? 4 6/25/19 Matti Kortelainen | Introduction to multi-threading and vectorization What is a (software) thread? (in POSIX/Linux) • “Smallest sequence of programmed instructions that can be managed independently by a scheduler” [Wikipedia] • A thread has its own – Program counter – Registers – Stack – Thread-local memory (better to avoid in general) • Threads of a process share everything else, e.g. – Program code, constants – Heap memory – Network connections – File handles
    [Show full text]
  • Concurrent Cilk: Lazy Promotion from Tasks to Threads in C/C++
    Concurrent Cilk: Lazy Promotion from Tasks to Threads in C/C++ Christopher S. Zakian, Timothy A. K. Zakian Abhishek Kulkarni, Buddhika Chamith, and Ryan R. Newton Indiana University - Bloomington, fczakian, tzakian, adkulkar, budkahaw, [email protected] Abstract. Library and language support for scheduling non-blocking tasks has greatly improved, as have lightweight (user) threading packages. How- ever, there is a significant gap between the two developments. In previous work|and in today's software packages|lightweight thread creation incurs much larger overheads than tasking libraries, even on tasks that end up never blocking. This limitation can be removed. To that end, we describe an extension to the Intel Cilk Plus runtime system, Concurrent Cilk, where tasks are lazily promoted to threads. Concurrent Cilk removes the overhead of thread creation on threads which end up calling no blocking operations, and is the first system to do so for C/C++ with legacy support (standard calling conventions and stack representations). We demonstrate that Concurrent Cilk adds negligible overhead to existing Cilk programs, while its promoted threads remain more efficient than OS threads in terms of context-switch overhead and blocking communication. Further, it enables development of blocking data structures that create non-fork-join dependence graphs|which can expose more parallelism, and better supports data-driven computations waiting on results from remote devices. 1 Introduction Both task-parallelism [1, 11, 13, 15] and lightweight threading [20] libraries have become popular for different kinds of applications. The key difference between a task and a thread is that threads may block|for example when performing IO|and then resume again.
    [Show full text]
  • The Impact of Process Placement and Oversubscription on Application Performance: a Case Study for Exascale Computing
    Takustraße 7 Konrad-Zuse-Zentrum D-14195 Berlin-Dahlem fur¨ Informationstechnik Berlin Germany FLORIAN WENDE,THOMAS STEINKE,ALEXANDER REINEFELD The Impact of Process Placement and Oversubscription on Application Performance: A Case Study for Exascale Computing ZIB-Report 15-05 (January 2015) Herausgegeben vom Konrad-Zuse-Zentrum fur¨ Informationstechnik Berlin Takustraße 7 D-14195 Berlin-Dahlem Telefon: 030-84185-0 Telefax: 030-84185-125 e-mail: [email protected] URL: http://www.zib.de ZIB-Report (Print) ISSN 1438-0064 ZIB-Report (Internet) ISSN 2192-7782 The Impact of Process Placement and Oversubscription on Application Performance: A Case Study for Exascale Computing Florian Wende, Thomas Steinke, Alexander Reinefeld Abstract: With the growing number of hardware components and the increasing software complexity in the upcoming exascale computers, system failures will become the norm rather than an exception for long-running applications. Fault-tolerance can be achieved by the creation of checkpoints dur- ing the execution of a parallel program. Checkpoint/Restart (C/R) mechanisms allow for both task migration (even if there were no hardware faults) and restarting of tasks after the occurrence of hard- ware faults. Affected tasks are then migrated to other nodes which may result in unfortunate process placement and/or oversubscription of compute resources. In this paper we analyze the impact of unfortunate process placement and oversubscription of compute resources on the performance and scalability of two typical HPC application workloads, CP2K and MOM5. Results are given for a Cray XC30/40 with Aries dragonfly topology. Our results indicate that unfortunate process placement has only little negative impact while oversubscription substantially degrades the performance.
    [Show full text]
  • And Computer Systems I: Introduction Systems
    Welcome to Computer Team! Noah Singer Introduction Welcome to Computer Team! Units Computer and Computer Systems I: Introduction Systems Computer Parts Noah Singer Montgomery Blair High School Computer Team September 14, 2017 Overview Welcome to Computer Team! Noah Singer Introduction 1 Introduction Units Computer Systems 2 Units Computer Parts 3 Computer Systems 4 Computer Parts Welcome to Computer Team! Noah Singer Introduction Units Computer Systems Computer Parts Section 1: Introduction Computer Team Welcome to Computer Team! Computer Team is... Noah Singer A place to hang out and eat lunch every Thursday Introduction A cool discussion group for computer science Units Computer An award-winning competition programming group Systems A place for people to learn from each other Computer Parts Open to people of all experience levels Computer Team isn’t... A programming club A highly competitive environment A rigorous course that requires commitment Structure Welcome to Computer Team! Noah Singer Introduction Units Four units, each with five to ten lectures Computer Each lecture comes with notes that you can take home! Systems Computer Guided activities to help you understand more Parts difficult/technical concepts Special presentations and guest lectures on interesting topics Conventions Welcome to Computer Team! Noah Singer Vocabulary terms are marked in bold. Introduction Definition Units Computer Definitions are in boxes (along with theorems, etc.). Systems Computer Parts Important statements are highlighted in italics. Formulas are written
    [Show full text]
  • Thread Scheduling in Multi-Core Operating Systems Redha Gouicem
    Thread Scheduling in Multi-core Operating Systems Redha Gouicem To cite this version: Redha Gouicem. Thread Scheduling in Multi-core Operating Systems. Computer Science [cs]. Sor- bonne Université, 2020. English. tel-02977242 HAL Id: tel-02977242 https://hal.archives-ouvertes.fr/tel-02977242 Submitted on 24 Oct 2020 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. Ph.D thesis in Computer Science Thread Scheduling in Multi-core Operating Systems How to Understand, Improve and Fix your Scheduler Redha GOUICEM Sorbonne Université Laboratoire d’Informatique de Paris 6 Inria Whisper Team PH.D.DEFENSE: 23 October 2020, Paris, France JURYMEMBERS: Mr. Pascal Felber, Full Professor, Université de Neuchâtel Reviewer Mr. Vivien Quéma, Full Professor, Grenoble INP (ENSIMAG) Reviewer Mr. Rachid Guerraoui, Full Professor, École Polytechnique Fédérale de Lausanne Examiner Ms. Karine Heydemann, Associate Professor, Sorbonne Université Examiner Mr. Etienne Rivière, Full Professor, University of Louvain Examiner Mr. Gilles Muller, Senior Research Scientist, Inria Advisor Mr. Julien Sopena, Associate Professor, Sorbonne Université Advisor ABSTRACT In this thesis, we address the problem of schedulers for multi-core architectures from several perspectives: design (simplicity and correct- ness), performance improvement and the development of application- specific schedulers.
    [Show full text]
  • Table of Contents
    Parallel Programming Models Lawrence Rauchwerger http://parasol.tamu.edu Table of Contents Introduction to Parallelism Introduction to Programming Models Shared Memory Programming Message Passing Programming Shared Memory Models PGAS Languages Other Programming Models 1 Acknowledgement Material in this course has been adapted from various (cited) authoritative sources Presentation has been put together with the help of Dr. Mauro Bianco, Antoniu Pop, Tim Smith and Nathan Thomas – Parasol Lab, Department of Computer Science, Texas A&M University. What Will You Get from Class Ideas about parallel processing Different approaches to parallel programming Additional material in your classnotes 2 Table of Contents Introduction to Parallelism – What is Parallelism ? What is the Goal ? Introduction to Programming Models Shared Memory Programming Message Passing Programming Shared Memory Models PGAS Languages Other Programming Models Introduction to Parallelism Sequential Computing – Single CPU executes stream of instructions. Adapted from: http://www.llnl.gov/computing/tutorials/parallel_comp 3 Introduction to Parallelism Parallel computing – Partition problem into multiple, concurrent streams of instructions. Classification Flynn’s Taxonomy (1966-now) Nowadays SISD SIMD SPMD Single Instruction Single Instruction Single Program Single Data Multiple Data Multiple Data MISD MIMD MPMD Multiple Instructions Multiple Instructions Multiple Program Single Data Multiple Data Multiple Data • Execution models impact the above programming model • Traditional computer is SISD • SIMD is data parallelism while MISD is pure task parallelism • MIMD is a mixed model (harder to program) • SPMD and MPMD are less synchronized than SIMD and MIMD • SPMD is most used model, but MPMD is becoming popular 4 Introduction to Parallelism Goal of parallel computing – Save time - reduce wall clock time.
    [Show full text]