Easybuild Documentation Release 20170109.01
Total Page:16
File Type:pdf, Size:1020Kb
EasyBuild Documentation Release 20170109.01 Ghent University Fri, 27 Jan 2017 16:57:55 Contents 1 Introductory topics 3 1.1 What is EasyBuild?.........................................3 1.2 Concepts and terminology......................................4 1.2.1 EasyBuild framework....................................4 1.2.2 Easyblocks.........................................4 1.2.3 Toolchains..........................................5 1.2.4 Easyconfig files.......................................6 1.2.5 Extensions..........................................6 1.3 Typical workflow example: building and installing WRF......................6 1.3.1 Searching for available easyconfigs files..........................7 1.3.2 Getting an overview of planned installations........................7 1.3.3 Installing a software stack.................................8 2 Getting started 9 2.1 Installing EasyBuild.........................................9 2.1.1 Requirements........................................9 2.1.2 Bootstrapping EasyBuild.................................. 10 2.1.3 Advanced bootstrapping options.............................. 13 2.1.4 Updating an existing EasyBuild installation........................ 14 2.1.5 Dependencies........................................ 15 2.1.6 Sources........................................... 17 2.1.7 In case of installation issues.................................. 17 2.2 Configuring EasyBuild........................................ 18 2.2.1 Supported configuration types............................... 18 2.2.2 Overview of current configuration (--show-config, --show-full-config).. 22 2.2.3 Available configuration settings.............................. 23 3 Basic usage topics 29 3.1 Using the EasyBuild command line................................. 29 3.1.1 Specifying builds...................................... 29 3.1.2 Commonly used command line options.......................... 31 3.2 Writing easyconfig files: the basics................................. 39 3.2.1 What is an easyconfig (file)?................................ 39 3.2.2 Available easyconfig parameters.............................. 40 3.2.3 Mandatory easyconfig parameters............................. 40 3.2.4 Common easyconfig parameters.............................. 40 3.2.5 Tweaking existing easyconfig files............................. 45 3.2.6 Dynamic values for easyconfig parameters......................... 45 3.2.7 Contributing back...................................... 45 3.3 Understanding EasyBuild logs.................................... 46 3.3.1 Basic information...................................... 46 3.3.2 Navigating log files..................................... 47 i 4 Advanced usage topics 49 4.1 Archived easyconfigs......................................... 49 4.1.1 Toolchain deprecation.................................... 49 4.1.2 Using --consider-archived-easyconfigs ................... 50 4.2 Common toolchains......................................... 50 4.2.1 Definition and motivation.................................. 50 4.2.2 Versioning scheme for common toolchains......................... 52 4.2.3 Update cycle for common toolchains............................ 52 4.2.4 Overview of common toolchains.............................. 52 4.2.5 Customizing common toolchains.............................. 53 4.3 Controlling compiler optimization flags............................... 53 4.3.1 Controlling target architecture specific optimizations via --optarch .......... 53 4.4 EasyBuild on Cray.......................................... 55 4.4.1 Test systems......................................... 56 4.4.2 EasyBuild toolchains.................................... 56 4.4.3 What works already?.................................... 56 4.4.4 Required EasyBuild configuration............................. 57 4.4.5 Major supported/tested applications............................ 57 4.5 Experimental features........................................ 58 4.6 Extended dry run........................................... 58 4.6.1 Important notes....................................... 59 4.6.2 Overview of dry run mechanism.............................. 60 4.6.3 Guidelines for easyblocks.................................. 66 4.6.4 Example output....................................... 69 4.7 Including additional Python modules (--include-*)...................... 69 4.7.1 General aspects of --include-* options........................ 70 4.7.2 Including additional easyblocks (--include-easyblocks)............. 70 4.7.3 Including additional module naming schemes (--include-module-naming-schemes) 71 4.7.4 Including additional toolchains (--include-toolchains).............. 71 4.8 Integration with GitHub....................................... 72 4.8.1 Requirements........................................ 72 4.8.2 Configuration........................................ 73 4.8.3 Checking status of GitHub integration (--check-github)............... 74 4.8.4 Using easyconfigs from pull requests (--from-pr)................... 75 4.8.5 Uploading test reports (--upload-test-report).................. 77 4.8.6 Reviewing easyconfig pull requests (--review-pr).................. 78 4.8.7 Submitting new and updating pull requests (--new-pr, --update-pr)....... 79 4.9 Manipulating dependencies..................................... 84 4.9.1 Filtering out dependencies using --filter-deps ................... 84 4.9.2 Installing dependencies as hidden modules using --hide-deps ............ 84 4.9.3 Using minimal toolchains for dependencies........................ 85 4.10 Packaging support.......................................... 86 4.10.1 Prerequisites........................................ 86 4.10.2 Configuration options.................................... 87 4.10.3 Usage............................................ 87 4.10.4 Packaging existing installations............................... 88 4.11 Partial installations.......................................... 89 4.11.1 Stopping the installation procedure after a step using -s/--stop ............ 89 4.11.2 Installing additional extensions using -k/-skip ..................... 90 4.11.3 Only (re)generating (additional) module files using --module-only ......... 90 4.12 Support for RPATH......................................... 93 4.12.1 What is RPATH?...................................... 94 4.12.2 Why RPATH?........................................ 94 4.12.3 Enabling RPATH linking.................................. 94 4.12.4 Implementation....................................... 94 4.12.5 Filtering RPATH entries via --rpath-filter ..................... 95 4.12.6 Relation to $LD_LIBRARY_PATH ............................ 95 4.13 Submitting jobs using --job .................................... 96 ii 4.13.1 Quick introduction to --job ................................ 96 4.13.2 Configuring --job ..................................... 96 4.13.3 Usage of --job ...................................... 97 4.13.4 Examples.......................................... 98 4.14 Using external modules....................................... 101 4.14.1 Using external modules as dependencies.......................... 101 4.14.2 Metadata for external modules............................... 102 5 Other topics 105 5.1 Code style.............................................. 105 5.1.1 Notes............................................ 105 5.2 Unit tests............................................... 105 5.2.1 What the unit tests are not ................................. 106 5.2.2 Available unit test suites.................................. 106 5.2.3 Applications......................................... 107 5.2.4 Usage............................................ 107 5.3 Useful scripts............................................. 109 5.3.1 fix_broken_easyconfigs.py ........................... 110 5.3.2 install-EasyBuild-develop.sh ......................... 111 5.3.3 clean_gists.py .................................... 111 5.4 Deprecated functionality....................................... 111 5.4.1 Overview of deprecated functionality in EasyBuild version 3.0.2............. 111 5.4.2 Deprecation policy..................................... 112 5.4.3 How to check for use of deprecated functionality..................... 112 5.5 Removed functionality........................................ 113 5.5.1 Overview of removed functionality since EasyBuild v3.0................. 113 5.5.2 Overview of removed functionality since EasyBuild v2.0................. 114 6 Overview of version specific (auto-generated) documentation pages 123 7 Getting help 125 8 Lists and tables 127 9 Appendices 129 9.1 Changelog for EasyBuild documentation.............................. 129 9.2 Configuration Legacy........................................ 133 9.2.1 Porting from legacy configuration style.......................... 133 9.2.2 How EasyBuild used to be configured in the early days.................. 134 9.3 Available easyconfig parameters for EB_WRF........................... 135 9.4 List of easyblocks.......................................... 137 9.5 List of known toolchains....................................... 140 9.6 Alternative installation methods................................... 141 9.6.1 Standard installation of latest release............................ 141 9.6.2 Installation from downloaded sources........................... 143 9.6.3 Installation of latest release from GitHub......................... 143 9.6.4 Installation of latest development version......................... 143 9.6.5 Installation of latest