EasyBuild Documentation Release 20160713.01 Ghent University Mon, 01 Aug 2016 13:51:52 Contents 1 Introductory topics 3 1.1 What is EasyBuild?.........................................3 1.2 Concepts and terminology......................................4 1.2.1 EasyBuild framework....................................4 1.2.2 Easyblocks.........................................4 1.2.3 Toolchains..........................................5 1.2.4 Easyconfig files.......................................5 1.2.5 Extensions..........................................6 1.3 Typical workflow example: building and installing WRF......................6 1.3.1 Searching for available easyconfigs files..........................6 1.3.2 Getting an overview of planned installations........................7 1.3.3 Installing a software stack.................................7 2 Getting started 9 2.1 Installing EasyBuild.........................................9 2.1.1 Requirements........................................9 2.1.2 Bootstrapping EasyBuild.................................. 10 2.1.3 Advanced bootstrapping options.............................. 13 2.1.4 Updating an existing EasyBuild installation........................ 14 2.1.5 Dependencies........................................ 14 2.1.6 Sources........................................... 17 2.1.7 In case of installation issues.................................. 17 2.2 Configuring EasyBuild........................................ 17 2.2.1 Supported configuration types............................... 18 2.2.2 Overview of current configuration (--show-config, --show-full-config).. 22 2.2.3 Available configuration settings.............................. 22 3 Basic usage topics 29 3.1 Using the EasyBuild command line................................. 29 3.1.1 Specifying builds...................................... 29 3.1.2 Commonly used command line options.......................... 31 3.2 Writing easyconfig files: the basics................................. 39 3.2.1 What is an easyconfig (file)?................................ 39 3.2.2 Available easyconfig parameters.............................. 39 3.2.3 Mandatory easyconfig parameters............................. 40 3.2.4 Common easyconfig parameters.............................. 40 3.2.5 Tweaking existing easyconfig files............................. 44 3.2.6 Dynamic values for easyconfig parameters......................... 44 3.2.7 Contributing back...................................... 44 3.3 Understanding EasyBuild logs.................................... 45 3.3.1 Basic information...................................... 45 3.3.2 Navigating log files..................................... 45 i 4 Advanced usage topics 47 4.1 Controlling compiler optimization flags............................... 47 4.1.1 Controlling target architecture specific optimizations via --optarch .......... 47 4.2 Experimental features........................................ 49 4.3 Extended dry run........................................... 49 4.3.1 Important notes....................................... 50 4.3.2 Overview of dry run mechanism.............................. 51 4.3.3 Guidelines for easyblocks.................................. 57 4.3.4 Example output....................................... 60 4.4 Including additional Python modules (--include-*)...................... 60 4.4.1 General aspects of --include-* options........................ 61 4.4.2 Including additional easyblocks (--include-easyblocks)............. 61 4.4.3 Including additional module naming schemes (--include-module-naming-schemes) 62 4.4.4 Including additional toolchains (--include-toolchains).............. 62 4.5 Integration with GitHub....................................... 63 4.5.1 Requirements........................................ 63 4.5.2 Configuration........................................ 64 4.5.3 Checking status of GitHub integration (--check-github)............... 65 4.5.4 Using easyconfigs from pull requests (--from-pr)................... 66 4.5.5 Uploading test reports (--upload-test-report).................. 67 4.5.6 Reviewing easyconfig pull requests (--review-pr).................. 68 4.5.7 Submitting new and updating pull requests (--new-pr, --update-pr)....... 69 4.6 Manipulating dependencies..................................... 73 4.6.1 Filtering out dependencies using --filter-deps ................... 73 4.6.2 Installing dependencies as hidden modules using --hide-deps ............ 74 4.6.3 Using minimal toolchains for dependencies........................ 74 4.7 Packaging support.......................................... 75 4.7.1 Prerequisites........................................ 76 4.7.2 Configuration options.................................... 76 4.7.3 Usage............................................ 77 4.7.4 Packaging existing installations............................... 78 4.8 Partial installations.......................................... 78 4.8.1 Stopping the installation procedure after a step using -s/--stop ............ 78 4.8.2 Installing additional extensions using -k/-skip ..................... 79 4.8.3 Only (re)generating (additional) module files using --module-only ......... 80 4.9 Submitting jobs using --job .................................... 83 4.9.1 Quick introduction to --job ................................ 83 4.9.2 Configuring --job ..................................... 83 4.9.3 Usage of --job ...................................... 84 4.9.4 Examples.......................................... 85 4.10 Using external modules....................................... 88 4.10.1 Using external modules as dependencies.......................... 88 4.10.2 Metadata for external modules............................... 89 5 Other topics 93 5.1 Code style.............................................. 93 5.1.1 Notes............................................ 93 5.2 Unit tests............................................... 93 5.2.1 What the unit tests are not ................................. 94 5.2.2 Available unit test suites.................................. 94 5.2.3 Applications......................................... 95 5.2.4 Usage............................................ 95 5.3 Useful scripts............................................. 97 5.3.1 fix_broken_easyconfigs.py ........................... 98 5.3.2 install-EasyBuild-develop.sh ......................... 99 5.3.3 clean_gists.py .................................... 99 5.4 Deprecated functionality....................................... 99 5.4.1 Overview of deprecated functionality in EasyBuild version 2.8.2............. 99 ii 5.4.2 Deprecation policy..................................... 100 5.4.3 How to check for use of deprecated functionality..................... 101 5.5 Removed functionality........................................ 101 5.5.1 Overview of removed functionality since EasyBuild v2.0................. 101 6 Getting help 109 7 Lists and tables 111 8 Appendices 113 8.1 Changelog for EasyBuild documentation.............................. 113 8.2 Configuration Legacy........................................ 116 8.2.1 Porting from legacy configuration style.......................... 117 8.2.2 How EasyBuild used to be configured in the early days.................. 117 8.3 Available easyconfig parameters for EB_WRF........................... 118 8.4 List of easyblocks.......................................... 120 8.5 List of known toolchains....................................... 123 8.6 Alternative installation methods................................... 124 8.6.1 Standard installation of latest release............................ 124 8.6.2 Installation from downloaded sources........................... 126 8.6.3 Installation of latest release from GitHub......................... 126 8.6.4 Installation of latest development version......................... 126 8.6.5 Installation of latest development version using provided script.............. 127 8.7 Installing environment modules without root permissions...................... 127 8.7.1 Tcl.............................................. 127 8.7.2 Environment Modules................................... 128 8.7.3 Set up your environment.................................. 128 8.8 Installing Lmod without root permissions.............................. 128 8.8.1 Lua............................................. 129 8.8.2 Lmod............................................ 129 8.9 Useful links.............................................. 129 8.10 Sphinx and Read the Docs reference documents online....................... 130 8.11 Third party Sphinx examples, themes and possible extensions................... 130 8.12 EasyBuild release notes....................................... 130 8.12.1 v2.8.2 (July 13th 2016)................................... 130 8.12.2 v2.8.1 (May 30th 2016)................................... 133 8.12.3 v2.8.0 (May 18th 2016)................................... 134 8.12.4 v2.7.0 (March 20th 2016).................................. 137 8.12.5 v2.6.0 (January 26th 2016)................................. 142 8.12.6 v2.5.0 (December 17th 2015)................................ 144 8.12.7 v2.4.0 (November 10th 2015)................................ 146 8.12.8 v2.3.0 (September 2nd 2015)................................ 149 8.12.9 v2.2.0 (July 15th 2015)................................... 151 8.12.10 v2.1.1 (May 18th 2015)................................... 154 8.12.11 v2.1.0 (April 30th 2015).................................. 155 8.12.12 v2.0.0 (March 6th 2015).................................