HPX Documentation 1.3.0 The STEjjAR Group May 23, 2019 User documentation 1 What is HPX? 3 2 What’s so special about HPX?5 2.1 Why HPX?................................................5 2.2 Quick start................................................ 11 2.3 Terminology............................................... 17 2.4 Examples................................................. 18 2.5 Manual.................................................. 40 2.6 Additional material............................................ 207 2.7 Overview................................................. 207 2.8 All modules............................................... 208 2.9 API reference............................................... 209 2.10 Contributing to HPX ........................................... 546 2.11 Releases................................................. 552 2.12 About HPX ................................................ 722 3 Index 731 Index 733 i ii HPX Documentation, 1.3.0 If you’re new to HPX you can get started with the Quick start guide. Don’t forget to read the Terminology section to learn about the most important concepts in HPX. The Examples give you a feel for how it is to write real HPX applications and the Manual contains detailed information about everything from building HPX to debugging it. There are links to blog posts and videos about HPX in Additional material. If you can’t find what you’re looking for in the documentation, please: • open an issue on GitHub1; • contact us on IRC, the HPX channel on the C++ Slack2, or on our mailing list3; or • read or ask questions tagged with HPX on StackOverflow4. 1 https://github.com/STEllAR-GROUP/hpx/issues 2 https://cpplang.slack.com 3 [email protected] 4 https://stackoverflow.com/questions/tagged/hpx User documentation 1 HPX Documentation, 1.3.0 2 User documentation CHAPTER 1 What is HPX? HPX is a C++ Standard Library for Concurrency and Parallelism. It implements all of the corresponding facilities as defined by the C++ Standard. Additionally, in HPX we implement functionalities proposed as part of the ongoing C++ standardization process. We also extend the C++ Standard APIs to the distributed case. HPX is developed by the STE||AR group (see People). The goal of HPX is to create a high quality, freely available, open source implementation of a new programming model for conventional systems, such as classic Linux based Beowulf clusters or multi-socket highly parallel SMP nodes. At the same time, we want to have a very modular and well designed runtime system architecture which would allow us to port our implementation onto new computer system architectures. We want to use real-world applications to drive the development of the runtime system, coining out required functionalities and converging onto a stable API which will provide a smooth migration path for developers. The API exposed by HPX is not only modeled after the interfaces defined by the C++11/14/17/20 ISO standard. It also adheres to the programming guidelines used by the Boost collection of C++ libraries. We aim to improve the scalability of today’s applications and to expose new levels of parallelism which are necessary to take advantage of the exascale systems of the future. 3 HPX Documentation, 1.3.0 4 Chapter 1. What is HPX? CHAPTER 2 What’s so special about HPX? • HPX exposes a uniform, standards-oriented API for ease of programming parallel and distributed applications. • It enables programmers to write fully asynchronous code using hundreds of millions of threads. • HPX provides unified syntax and semantics for local and remote operations. • HPX makes concurrency manageable with dataflow and future based synchronization. • It implements a rich set of runtime services supporting a broad range of use cases. • HPX exposes a uniform, flexible, and extendable performance counter framework which can enable runtime adaptivity • It is designed to solve problems conventionally considered to be scaling-impaired. • HPX has been designed and developed for systems of any scale, from hand-held devices to very large scale systems. • It is the first fully functional implementation of the ParalleX execution model. • HPX is published under a liberal open-source license and has an open, active, and thriving developer community. 2.1 Why HPX? Current advances in high performance computing (HPC) continue to suffer from the issues plaguing parallel compu- tation. These issues include, but are not limited to, ease of programming, inability to handle dynamically changing workloads, scalability, and efficient utilization of system resources. Emerging technological trends such as multi- core processors further highlight limitations of existing parallel computation models. To mitigate the aforementioned problems, it is necessary to rethink the approach to parallelization models. ParalleX contains mechanisms such as multi-threading, parcels, global name space support, percolation and local control objects (LCO). By design, Par- alleX overcomes limitations of current models of parallelism by alleviating contention, latency, overhead and starva- tion. With ParalleX, it is further possible to increase performance by at least an order of magnitude on challenging parallel algorithms, e.g., dynamic directed graph algorithms and adaptive mesh refinement methods for astrophysics. An additional benefit of ParalleX is fine-grained control of power usage, enabling reductions in power consumption. 5 HPX Documentation, 1.3.0 2.1.1 ParalleX—a new execution model for future architectures ParalleX is a new parallel execution model that offers an alternative to the conventional computation models, such as message passing. ParalleX distinguishes itself by: • Split-phase transaction model • Message-driven • Distributed shared memory (not cache coherent) • Multi-threaded • Futures synchronization • Local Control Objects (LCOs) • Synchronization for anonymous producer-consumer scenarios • Percolation (pre-staging of task data) The ParalleX model is intrinsically latency hiding, delivering an abundance of variable-grained parallelism within a hierarchical namespace environment. The goal of this innovative strategy is to enable future systems delivering very high efficiency, increased scalability and ease of programming. ParalleX can contribute to significant improvements in the design of all levels of computing systems and their usage from application algorithms and their programming languages to system architecture and hardware design together with their supporting compilers and operating system software. 2.1.2 What is HPX? High Performance ParalleX (HPX) is the first runtime system implementation of the ParalleX execution model. The HPX runtime software package is a modular, feature-complete, and performance oriented representation of the Par- alleX execution model targeted at conventional parallel computing architectures such as SMP nodes and commodity clusters. It is academically developed and freely available under an open source license. We provide HPX to the community for experimentation and application to achieve high efficiency and scalability for dynamic adaptive and ir- regular computational problems. HPX is a C++ library that supports a set of critical mechanisms for dynamic adaptive resource management and lightweight task scheduling within the context of a global address space. It is solidly based on many years of experience in writing highly parallel applications for HPC systems. The two-decade success of the communicating sequential processes (CSP) execution model and its message passing interface (MPI) programming model has been seriously eroded by challenges of power, processor core complexity, multi-core sockets, and heterogeneous structures of GPUs. Both efficiency and scalability for some current (strong scaled) applications and future Exascale applications demand new techniques to expose new sources of algorithm parallelism and exploit unused resources through adaptive use of runtime information. The ParalleX execution model replaces CSP to provide a new computing paradigm embodying the governing principles for organizing and conducting highly efficient scalable computations greatly exceeding the capabilities of today’s problems. HPX is the first practical, reliable, and performance-oriented runtime system incorporating the principal concepts of the ParalleX model publicly provided in open source release form. HPX is designed by the STE||AR5 Group (Systems Technology, Emergent Parallelism, and Algorithm Research) at Louisiana State University (LSU)6’s Center for Computation and Technology (CCT)7 to enable developers to exploit the full processing power of many-core systems with an unprecedented degree of parallelism. STE||AR8 is a research group focusing on system software solutions and scientific application development for hybrid and many-core hard- ware architectures. 5 https://stellar-group.org 6 https://www.lsu.edu 7 https://www.cct.lsu.edu 8 https://stellar-group.org 6 Chapter 2. What’s so special about HPX? HPX Documentation, 1.3.0 For more information about the STE||AR9 Group, see People. 2.1.3 What makes our systems slow? Estimates say that we currently run our computers at way below 100% efficiency. The theoretical peak performance (usually measured in FLOPS10—floating point operations per second) is much higher than any practical peak per- formance reached by any application. This is particularly true for highly parallel hardware. The more hardware parallelism we provide to an application, the better the application must scale in order