Gkyl Documentation Release 2.0-Alpha
Total Page:16
File Type:pdf, Size:1020Kb
gkyl Documentation Release 2.0-alpha Ammar Hakim Sep 24, 2021 Contents 1 Installing Gkeyll 3 1.1 gkyl install................................................4 1.2 Postgkyl install..............................................7 1.3 Note on the Windows Linux Subsystem (WSL).............................9 2 Quickstart 13 2.1 A first Gkeyll simulation......................................... 13 2.2 Vlasov example............................................. 18 2.3 Gyrokinetic example........................................... 26 2.4 Fluid example.............................................. 39 3 gkyl Reference 47 3.1 Using gkyl................................................ 47 3.2 gkyl Apps................................................. 54 3.3 App plugins............................................... 74 3.4 gkyl Tools................................................ 91 4 Postgkyl reference 97 4.1 Using Postgkyl.............................................. 97 4.2 Examples................................................. 108 4.3 Function/Command reference...................................... 121 5 Publications and theses 175 5.1 Doctoral Dissertations.......................................... 175 5.2 Algorithms papers............................................ 175 5.3 Physics papers.............................................. 176 6 Presentations 179 7 Developer notes 181 7.1 On use of the Maxima CAS....................................... 181 7.2 Modal basis functions.......................................... 182 7.3 The recovery Maxima code....................................... 184 7.4 Strong-Stability preserving Runge-Kutta time-steppers......................... 184 7.5 Normalized units for the Vlasov-Maxwell system............................ 185 7.6 From normalized to physical units in Vlasov and multi-fluid simulations................ 188 7.7 The eigensystem of the Maxwell equations with extension to perfectly hyperbolic Maxwell equations 190 i 7.8 The eigensystem of the Euler equations................................. 192 7.9 The eigensystem of the ten-moment equations............................. 193 7.10 Handling two-fluid five-moment and ten-moment source terms..................... 197 Bibliography 199 ii gkyl Documentation, Release 2.0-alpha “Magic Chicken Software Framework” – Artificial ‘Intelligence’ view on Gkeyll “Don’t Panic” – The Hitchhiker’s Guide to the Galaxy Gkeyll v2.0 (pronounced as in the book “The Strange Case of Dr. Jekyll and Mr. Hyde”) is a computational plasma physics code mostly written in C/C++ and LuaJIT. Gkeyll contains solvers for gyrokinetic equations, Vlasov-Maxwell equations, and multi-fluid equations. Gkeyll contains ab-initio and novel implementations of a number of algorithms, and perhaps is unique in using a JIT compiled typeless dynamic language for as its main implementation language. The Gkeyll package contains two major parts: the gkyl simulation framework and the the postgkyl post-processing package. Here you will find documentation for the full Gkeyll package. For license and authors see License and Authors. Contents 1 gkyl Documentation, Release 2.0-alpha 2 Contents CHAPTER 1 Installing Gkeyll “Mostly Painless” – Petr Cagas Installation instructions for both the gkyl simulation framework and the postgkyl postprocessing tool are provided below. Contents • Installing Gkeyll – gkyl install * Installing from source (preferred) · Installing using “machine files” (recommended as part of installation from source) · Machine files for non-native systems · Installing from source manually * Installing with Conda * Troubleshooting – Postgkyl install * Installing with Conda (preferred for non-developers) * Installing from source (preferred for developers) * Switching from Conda version to repository – Note on the Windows Linux Subsystem (WSL) * Enabling and installing WSL * Interaction between Windows and Linux; GUI * Known issues 3 gkyl Documentation, Release 2.0-alpha 1.1 gkyl install There are two options for installing gkyl. One can install directly from the source code, or via the Conda package. Installing directly from the source is the preferred option, as this method gives users more control over the instal- lation process. For many users who will wish to run gkyl on a cluster, which will have cluster-built versions of the Message Passing Interface (MPI) for parallel simulations, and potentially other gkyl depedencies, the source build will allow users to set the appropriate paths to the cluster installations of these dependencies. 1.1.1 Installing from source (preferred) To install gkyl from source, first clone the GitHub repository using: git clone https://github.com/ammarhakim/gkyl Navigate into the gkyl directory to begin. In many cases, an installation of gkyl will involve building most of gkyl’s dependencies only require a modern C/C++ compiler and Python 3. The full list of dependencies is: • C/C++ compiler with C++17 support (But NOT Clang >= 12.0 provided by Xcode 12) • Python 3 >=3.6 • MPI compiler with MPI3 support (>=openmpi 3.0 or >=mpich 3.0) • LuaJIT 2.1.0 • ADIOS 1.13.1 (But NOT >=ADIOS 2.0) • Eigen 3.3.7 • CUDA Toolkit >=10.2 (if building with GPU support, NOT FULLY SUPPORTED) The following instructions assume that at minimum the user has both a C/C++ compiler with C++17 support and Python 3. Installing using “machine files” (recommended as part of installation from source) For systems on which gkyl has been built before, the code can be built in three steps using scripts found in the machines/ directory. 1. Install dependencies using a mkdeps script from the machines/ directory: ./machines/mkdeps.[SYSTEM].sh where [SYSTEM] should be replaced by the name of the system you are building on, such as macosx or eddy. By default, installations will be made in ~/gkylsoft/. Even on systems which have installations of gkyl dependen- cies such as MPI, the mkdeps script must be run first to build other gkyl dependencies such as LuaJIT. 2. Configure waf using a configure script from the machines/ directory: ./machines/configure.[SYSTEM].sh NOTE: Steps 1 and 2 should only need to be done on the first build, unless one wishes to change the dependencies. A successful waf configure, on a system without GPU support, will look like: 4 Chapter 1. Installing Gkeyll gkyl Documentation, Release 2.0-alpha bash$ ./machines/configure.macosx.sh ./wafCC=clang CXX=clang++ MPICC=/Users/junoravin/gkylsoft/openmpi/bin/mpicc MPICXX=/ ,!Users/junoravin/gkylsoft/openmpi/bin/mpicxx --out=build --prefix=/Users/junoravin/ ,!gkylsoft/gkyl --cxxflags=-O3,-std=c++17 --luajit-inc-dir=/Users/junoravin/gkylsoft/ ,!luajit/include/luajit-2.1 --luajit-lib-dir=/Users/junoravin/gkylsoft/luajit/lib -- ,!luajit-share-dir=/Users/junoravin/gkylsoft/luajit/share/luajit-2.1.0-beta3 --enable- ,!mpi --mpi-inc-dir=/Users/junoravin/gkylsoft/openmpi/include --mpi-lib-dir=/Users/ ,!junoravin/gkylsoft/openmpi/lib --mpi-link-libs=mpi --enable-adios --adios-inc-dir=/ ,!Users/junoravin/gkylsoft/adios/include --adios-lib-dir=/Users/junoravin/gkylsoft/ ,!adios/lib configure Setting top to : /Users/junoravin/gkyl Setting out to : /Users/junoravin/gkyl/build Checking for 'clang'(C compiler) : clang Checking for 'clang++'(C++ compiler) : clang++ Setting dependency path: : /Users/junoravin/gkylsoft Setting prefix: : /Users/junoravin/gkylsoft/gkyl Checking for LUAJIT : Found LuaJIT Checking for MPI : Found MPI Checking for ADIOS : Found ADIOS Checking for EIGEN : Found EIGEN Checking for Sqlite3 : Using Sqlite3 Checking for NVCC compiler : Not found NVCC 'configure' finished successfully(0.843s) 3. Build the code using: ./waf build install The final result will be a gkyl executable located in the ~/gkylsoft/gkyl/bin/ directory. Feel free to add this directory to your PATH environment variable or create an alias so you can simply call gkyl. Note that if MPI was built as well as the part of the installation, ~/gkylsoft/openmpi/bin/ needs to be added to the PATH as well. Finally, on some distributions, it is required to add ~/gkylsoft/openmpi/lib/ to the LD_LIBRARY_PATH environmental variable. Machine files for non-native systems For systems that do not already have corresponding files in the machines/ directory, we encourage you to add files for your machine. Instructions can be found in machines/README.md. Note: Using Gkeyll on IBM Power9 systems (like Summit or Traverse) is not recommended. This is due to incomplete support for the LuaJIT compiler on Power9. Installing from source manually If machine files are not available, the dependencies, configuration, and build can be done manually. The first step is to build the dependencies. Depending on your system, building dependencies can be complicated. On a Mac or Linux machine you can simply run the mkdeps.sh script in the install-deps directory. To build dependencies cd to: cd gkyl/install-deps First, please check details by running: 1.1. gkyl install 5 gkyl Documentation, Release 2.0-alpha ./mkdeps.sh-h On most supercomputers you will likely need to use the system recommended compilers and MPI libraries. In this case, you should pass the appropriate compilers to mkdeps.sh as follows: ./mkdeps.sh CC=cc CXX=cxx MPICC=mpicc MPICXX=mpicxx.... You should only build libraries not provided by the system. In practice, this likely means LuaJIT, ADIOS and, perhaps Eigen. (Many supercomputer centers at DOE already offer ADIOS builds and should be preferred instead of your own builds). A typical command will be: ./mkdeps.sh--build-adios=yes--build-luajit=yes--build-eigen=yes (in addition, you may need to specify