Jenkins Job Builder Documentation Release 3.10.0 Jenkins Job Builder Maintainers Aug 23, 2021 Contents 1 README 1 1.1 Developers................................................1 1.2 Writing a patch..............................................2 1.3 Unit Tests.................................................2 1.4 Installing without setup.py........................................2 2 Contents 5 2.1 Quick Start Guide............................................5 2.1.1 Use Case 1: Test a job definition................................5 2.1.2 Use Case 2: Updating Jenkins Jobs...............................5 2.1.3 Use Case 3: Working with JSON job definitions........................6 2.1.4 Use Case 4: Deleting a job...................................6 2.1.5 Use Case 5: Providing plugins info...............................6 2.2 Installation................................................6 2.2.1 Documentation.........................................7 2.2.2 Unit Tests............................................7 2.2.3 Test Coverage..........................................7 2.3 Configuration File............................................7 2.3.1 job_builder section.......................................8 2.3.2 jenkins section.........................................9 2.3.3 hipchat section.........................................9 2.3.4 stash section...........................................9 2.3.5 __future__ section.......................................9 2.4 Running.................................................9 2.4.1 Test Mode............................................ 10 2.4.2 Updating Jobs.......................................... 10 2.4.3 Passing Multiple Paths..................................... 10 2.5 Recursive Searching of Paths...................................... 11 2.6 Excluding Paths............................................. 11 2.6.1 Deleting Jobs/Views...................................... 12 2.6.2 Globbed Parameters....................................... 12 2.6.3 Providing Plugins Info..................................... 13 2.6.4 Command Reference...................................... 13 2.7 Job Definitions.............................................. 15 2.7.1 Definitions........................................... 15 2.7.2 Custom Yaml Tags....................................... 32 i 2.7.3 Modules............................................. 41 2.7.4 Module Execution....................................... 328 2.8 Extending................................................. 329 2.8.1 The Builder........................................... 329 2.8.2 XML Processing........................................ 329 2.8.3 Modules............................................. 329 2.8.4 Components........................................... 330 2.8.5 Module Registry........................................ 330 3 Indices and tables 333 Python Module Index 335 Index 337 ii CHAPTER 1 README Jenkins Job Builder takes simple descriptions of Jenkins jobs in YAML or JSON format and uses them to configure Jenkins. You can keep your job descriptions in human readable text format in a version control system to make changes and auditing easier. It also has a flexible template system, so creating many similarly configured jobs is easy. To install: $ pip install --user jenkins-job-builder Online documentation: • https://jenkins-job-builder.readthedocs.io/en/latest/ 1.1 Developers Bug report: • https://storyboard.openstack.org/#!/project/723 Repository: • https://opendev.org/jjb/jenkins-job-builder Cloning: git clone https://opendev.org/jjb/jenkins-job-builder.git Install pre-commit from https://pre-commit.com/#intro in order to run some minimal testing on your commits. A virtual environment is recommended for development. For example, Jenkins Job Builder may be installed from the top level directory: $ virtualenv .venv $ source .venv/bin/activate $ pip install -r test-requirements.txt -e . 1 Jenkins Job Builder Documentation, Release 3.10.0 Patches are submitted via Gerrit at: • https://review.opendev.org Please do not submit GitHub pull requests, they will be automatically closed. Mailing list: • https://groups.google.com/forum/#!forum/jenkins-job-builder IRC: • #openstack-jjb on OFTC More details on how you can contribute is available on our wiki at: • https://docs.openstack.org/infra/manual/developers.html 1.2 Writing a patch We ask that all code submissions be pep8 and pyflakes clean. The easiest way to do that is to run tox before submitting code for review in Gerrit. It will run pep8 and pyflakes in the same manner as the automated test suite that will run on proposed patchsets. When creating new YAML components, please observe the following style conventions: • All YAML identifiers (including component names and arguments) should be lower-case and multiple word identifiers should use hyphens. E.g., “build-trigger”. • The Python functions that implement components should have the same name as the YAML keyword, but should use underscores instead of hyphens. E.g., “build_trigger”. This consistency will help users avoid simple mistakes when writing YAML, as well as developers when matching YAML components to Python implementation. 1.3 Unit Tests Unit tests have been included and are in the tests folder. Many unit tests samples are included as examples in our documentation to ensure that examples are kept current with existing behaviour. To run the unit tests, execute the command: tox-e py34,py27 • Note: View tox.ini to run tests on other versions of Python, generating the documentation and additionally for any special notes on running the test to validate documentation external URLs from behind proxies. 1.4 Installing without setup.py For YAML support, you will need libyaml installed. Mac OS X: $ brew install libyaml Then install the required python packages using pip: 2 Chapter 1. README Jenkins Job Builder Documentation, Release 3.10.0 $ sudo pip install PyYAML python-jenkins 1.4. Installing without setup.py 3 Jenkins Job Builder Documentation, Release 3.10.0 4 Chapter 1. README CHAPTER 2 Contents 2.1 Quick Start Guide This guide was made with the impatient in mind so explanation is sparse. It will guide users through a set of typical use cases for JJB using the same job definitions we use to test JJB. 1. Clone the repository to get the JJB job definition examples 2. The Installation can be either from pypi (released version) or from the clone (master). Usage of the commands below assumes that you are at the root of the cloned directory. 2.1.1 Use Case 1: Test a job definition JJB creates Jenkins XML configuration file from a YAML/JSON definition file and just uploads it to Jenkins. JJB provides a convenient test command to allow you to validate the XML before you attempt to upload it to Jenkins. Test a YAML job definition: jenkins-jobs test tests/yamlparser/fixtures/templates002.yaml The above command prints the generated Jenkins XML to the console. If you prefer to send it to a directory: jenkins-jobs test-o output tests/yamlparser/fixtures/templates002.yaml The output directory will contain files with the XML configurations. 2.1.2 Use Case 2: Updating Jenkins Jobs Once you’ve tested your job definition and are happy with it then you can use the update command to deploy the job to Jenkins. The update command requires a configuration file. An example file is supplied in the etc folder, you should update it to match your Jenkins master: 5 Jenkins Job Builder Documentation, Release 3.10.0 jenkins-jobs--conf etc/jenkins_jobs.ini-sample update tests/yamlparser/fixtures/ ,!templates002.yaml The above command will update your Jenkins master with the generated jobs. Caution: JJB caches Jenkins job information locally. Changes made using the Jenkins UI will not update that cache, which may lead to confusion. See Updating Jobs for more information. 2.1.3 Use Case 3: Working with JSON job definitions You can also define your jobs in json instead of yaml: jenkins-jobs--conf etc/jenkins_jobs.ini-sample update tests/jsonparser/fixtures/ ,!simple.json The above command just uses a simple job definition. You can also convert any of the YAML examples to JSON and feed that to JJB. 2.1.4 Use Case 4: Deleting a job To delete a job: jenkins-jobs--conf etc/jenkins_jobs.ini-sample delete simple The above command deletes the job simple from the Jenkins master. 2.1.5 Use Case 5: Providing plugins info To generate a plugins info, using an account with Administrator rights: jenkins-jobs get-plugins-info-o plugins_info.yaml To run JJB update using the plugins_info.yaml: jenkins-jobs update-p plugins_info.yaml./myjobs Please refer to the jenkins-jobs Command Reference and the Job Definitions pages for more details. 2.2 Installation To install Jenkins Job Builder from source, run: pip install--user jenkins-job-builder A virtual environment is recommended for development. For example, Jenkins Job Builder may be installed from the top level directory: $ virtualenv .venv $ source .venv/bin/activate $ pip install -r test-requirements.txt -e . 6 Chapter 2. Contents Jenkins Job Builder Documentation, Release 3.10.0 Alternatively, the current release can be installed from pypi: sudo pip install jenkins-job-builder The OpenStack project uses Puppet to manage its infrastructure systems, including Jenkins. If you use Puppet, you can use the OpenStack Jenkins module to install Jenkins Job Builder. 2.2.1 Documentation Documentation is included in the doc folder. To generate docs locally execute the command: tox-e docs The generated documentation is then available under doc/build/html/index.html. As