Migrating large codebases to C++ Modules Yuka Takahashi - The University of Tokyo Princeton University Oksana Shadura - UNL Vassil Vassilev Yuka Takahashi 13.03.2019 Migrating large codebases to C++ Modules, ACAT 2019 !1 Agenda 1. Motivation of C++ Modules 2. C++ Modules in ROOT 3. C++ Modules in CMSSW 4. CMS Performance Results 5. Conclusion Yuka Takahashi 13.03.2019 Migrating large codebases to C++ Modules, ACAT 2019 !2 Motivation of C++ Modules Yuka Takahashi 13.03.2019 Migrating large codebases to C++ Modules, ACAT 2019 !3 Motivation of C++ Modules C++ Modules technology: - Cache parsed header file information - Avoid header re-parsing - Avoid runtime header parsing (In ROOT) - Part of C++20 Yuka Takahashi 13.03.2019 Migrating large codebases to C++ Modules, ACAT 2019 !4 Motivation of C++ Modules #include <vector> Yuka Takahashi 13.03.2019 Migrating large codebases to C++ Modules, ACAT 2019 !5 Motivation of C++ Modules #include <vector> Textual Include Precompiled Headers (PCH) Modules Expensive Inseparable Fragile Yuka Takahashi 13.03.2019 Migrating large codebases to C++ Modules, ACAT 2019 !6 Motivation of C++ Modules …… TVirtualPad.h …… # 286 "/usr/include/c++/v1/vector" 2 3 #include "TVirtualPad.h" namespace std { inline namespace __1 { template <bool> class __vector_base_common vector #include <vector> Preprocess { __attribute__ #include <set> ((__visibility__("hidden"), __always_inline__)) __vector_base_common() Textual Include{} …… int main() { # 394 "/usr/include/c++/v1/set" 3 namespace std {inline namespace __1 { … set template <…> class set { original code public: typedef _Key key_type; …… .o Compile Parse int main { one big file! …… Yuka Takahashi 13.03.2019 Migrating large codebases to C++ Modules, ACAT 2019 !7 Motivation of C++ Modules Textual Include .c .o 1. Expensive .h .h .c .o Reparse the same header 2. Fragile .c .o Name collisions Users’ code #include <header.h> Rcpp library … #define PI 3.14 double PI = 3.14; … // => double 3.14 = 3.14; Yuka Takahashi 13.03.2019 Migrating large codebases to C++ Modules, ACAT 2019 !8 Motivation of C++ Modules .c .o PCH (Precompiled Headers) PCH .c .o 1. Storing precompiled header information (same as modules) .c .o 2. Stored in one big file - Monolithic allDict.cxx.pch In ROOT Interpreter Yuka Takahashi 13.03.2019 Migrating large codebases to C++ Modules, ACAT 2019 !9 a.pcm Motivation of C++ Modules .c .o Modules b.pcm .c .o - Module files contain parsed header information c.pcm .c .o - PCMs are separated a.pcm b.pcm c.pcm d.pcm e.pcm Each PCM file (a.pcm) corresponds to a library (liba.so) Interpreter In ROOT Yuka Takahashi 13.03.2019 Migrating large codebases to C++ Modules, ACAT 2019 !10 Motivation of C++ Modules Modules - Modules files contain parsed header information - PCMs are separated Compile-time scalability No Fragility Separable Yuka Takahashi 13.03.2019 Migrating large codebases to C++ Modules, ACAT 2019 !11 C++ Modules in ROOT Technology Preview released in ROOT 6.16 Yuka Takahashi 13.03.2019 Migrating large codebases to C++ Modules, ACAT 2019 !12 C++ Modules in ROOT Overview - Dependency Graph rootcling ROOT Dictionaries Clang Cling ROOT binaries files Yuka Takahashi 13.03.2019 Migrating large codebases to C++ Modules, ACAT 2019 !13 C++ Modules in ROOT Overview - Dependency Graph rootcling ROOT Dictionaries Clang Cling ROOT Cling calls Clang API binaries files Yuka Takahashi 13.03.2019 Migrating large codebases to C++ Modules, ACAT 2019 !14 C++ Modules in ROOT Overview - Dependency Graph rootcling ROOT Dictionaries rootcling generates dictionaries (rootmap, rdict) Clang Cling ROOT binaries files Yuka Takahashi 13.03.2019 Migrating large codebases to C++ Modules, ACAT 2019 !15 C++ Modules in ROOT Overview - Dependency Graph rootcling ROOT Dictionaries ROOTMAP Used to map symbols and Clang Cling identifiers to libraries ROOT binaries files Yuka Takahashi 13.03.2019 Migrating large codebases to C++ Modules, ACAT 2019 !16 C++ Modules in ROOT Overview - Dependency Graph rootcling ROOT Dictionaries RDICT Efficiently store information needed for Clang Cling ROOT serialization binaries files Yuka Takahashi 13.03.2019 Migrating large codebases to C++ Modules, ACAT 2019 !17 C++ Modules in ROOT Overview - Dependency Graph rootcling ROOT Dictionaries Clang Cling ROOT Dictionaries are used at ROOT runtime binaries files Yuka Takahashi 13.03.2019 Migrating large codebases to C++ Modules, ACAT 2019 !18 C++ Modules in ROOT Overview - Dependency Graph rootcling ROOT Dictionaries ROOT PCMs Clang Cling ROOT binaries files Yuka Takahashi 13.03.2019 Migrating large codebases to C++ Modules, ACAT 2019 !19 C++ Modules in ROOT Mechanism of loading modules Preloading of modules - Replace some functionality of RDICT and ROOTMAP with a more stable implementation - Load all ROOT modules at the startup time RSS Memory HSimple (4 months) 665MB Preloading of all http://root-bench.cern.ch (login required) Production Development modules Yuka Takahashi 13.03.2019 Migrating large codebases to C++ Modules, ACAT 2019 !20 C++ Modules in ROOT Mechanism of loading modules Global Modules Index - Remove further overhead in ROOT - Mechanism to create the table of symbols and PCM names - ROOT will be able to load corresponding library when a symbol lookup failed - The prototype shows promising results Yuka Takahashi 13.03.2019 Migrating large codebases to C++ Modules, ACAT 2019 !21 C++ Modules in ROOT Mechanism of loading libraries Bloom filter - Hash tables of symbols in .gnu.hash section in shared object files (further read) - ROOT can skip unnecessary libraries by reading it RSS Memory HSimple (4 months) 232MB Bloom filter http://root-bench.cern.ch (login required) Production Development Yuka Takahashi 13.03.2019 Migrating large codebases to C++ Modules, ACAT 2019 !22 C++ Modules in CMSSW Available in CMS CXXMODULE IB Yuka Takahashi 13.03.2019 Migrating large codebases to C++ Modules, ACAT 2019 !23 C++ Modules in CMSSW Overview - Dependency Graph Genreflex CMS (rootcling) Dictionaries SCRAM CMS Build system Runtime C++ CMS Compiler Libraries binaries files Yuka Takahashi 13.03.2019 Migrating large codebases to C++ Modules, ACAT 2019 !24 C++ Modules in CMSSW Overview - Dependency Graph Genreflex CMS (rootcling) Dictionaries SCRAM Genreflex and GCC, CMS Build system executed by SCRAM Runtime C++ CMS Compiler Libraries binaries files Yuka Takahashi 13.03.2019 Migrating large codebases to C++ Modules, ACAT 2019 !25 C++ Modules in CMSSW Overview - Dependency Graph Genreflex CMS (rootcling) Dictionaries SCRAM CMS Dictionaries generated by Genreflex Build system Runtime C++ CMS Compiler Libraries binaries files Yuka Takahashi 13.03.2019 Migrating large codebases to C++ Modules, ACAT 2019 !26 C++ Modules in CMSSW Overview - Dependency Graph Genreflex CMS (rootcling) Dictionaries SCRAM CMS Build system Libraries compiled by gcc Runtime C++ CMS Compiler Libraries binaries files Yuka Takahashi 13.03.2019 Migrating large codebases to C++ Modules, ACAT 2019 !27 C++ Modules in CMSSW Overview - Dependency Graph Genreflex CMS (rootcling) Dictionaries SCRAM CMS C++ CMS Build system Modules PCMs Runtime C++ CMS Compiler Libraries binaries files Yuka Takahashi 13.03.2019 Migrating large codebases to C++ Modules, ACAT 2019 !28 C++ Modules in CMSSW Overview - Dependency Graph Genreflex CMS (rootcling) Dictionaries SCRAM CMS C++ CMS Build system Modules PCMs Runtime - Not all CMS libraries were modularized - Modules can co-exist with the old infrastructure C++ CMS Compiler Libraries binaries files Yuka Takahashi 13.03.2019 Migrating large codebases to C++ Modules, ACAT 2019 !29 C++ Modules in CMSSW Overview - Dependency Graph Genreflex CMS (rootcling) Dictionaries SCRAM CMS C++ CMS Modulemap Build system Modules PCMs Runtime C++ CMS Compiler Libraries binaries files Yuka Takahashi 13.03.2019 Migrating large codebases to C++ Modules, ACAT 2019 !30 C++ Modules in CMSSW Explicit PCMs in CMSSW module.modulemap - Definition file of headers to build a PCM in Clang - Contain all “interface” headers, which are used by libraries module "MathCore" { module “TComplex name” { header "TComplex.h" export * } module <name of the file> { header <relative path to the header file location> } } modulemap will contain all interface header files Autogeneration of modulemap Yuka Takahashi 13.03.2019 Migrating large codebases to C++ Modules, ACAT 2019 !31 C++ Modules in CMSSW Explicit PCMs in CMSSW Autogeneration of modulemap - CMSSW has “interface” headers - Exposed to libraries outside - Automatically generate the modulemap by adding interface headers - Modulemap needs to be generated before the execution of genreflex - Build system is responsible for the autogeneration Yuka Takahashi 13.03.2019 Migrating large codebases to C++ Modules, ACAT 2019 !32 C++ Modules in CMSSW Explicit PCMs in CMSSW Genreflex CMS (rootcling) Dictionaries SCRAM CMS C++ CMS Modulemap Build system Modules PCMs Runtime C++ CMS Compiler Libraries binaries files Yuka Takahashi 13.03.2019 Migrating large codebases to C++ Modules, ACAT 2019 !33 C++ Modules in CMSSW Mechanism of the modulemap modulemap, modulemap overlay file, virtual modulemap overlay / usr/ $build_dir module “header.h” include/ { header “some/directory/ headers.h” } some/ module.modulemap directory/ directories *.h (headers) files Yuka Takahashi 13.03.2019 Migrating large codebases to C++ Modules, ACAT 2019 !34 C++ Modules in CMSSW Mechanism of the modulemap modulemap, modulemap overlay file, virtual modulemap overlay Modulemap for system headers - Modulemap needs to know the location of headers