Jenkins-Autojobs Documentation Release 0.17.4
Total Page:16
File Type:pdf, Size:1020Kb
jenkins-autojobs documentation Release 0.17.4 Georgi Valkov February 16, 2017 Contents 1 Installing 3 2 Changes 5 2.1 Changelog................................................5 2.2 Tutorial..................................................8 2.3 Case Study: Git.............................................. 18 3 Development 23 3.1 Testing.................................................. 23 3.2 Todo................................................... 23 4 Similar Projects 25 5 License 27 i ii jenkins-autojobs documentation, Release 0.17.4 Jenkins-autojobs is a set of scripts that automatically create Jenkins jobs from template jobs and the branches in an SCM repository. Jenkins-autojobs supports Git, Mercurial and Subversion. A routine run goes through the following steps: • Read settings from a configuration file. • List branches or refs from SCM. • Creates or updates jobs as configured. In its most basic form, the configuration file specifies: • How to access Jenkins and the SCM repository. • Which branches to process and which to ignore. • Which template job to use for which branches. • How new jobs should be named. Autojobs can also: • Add newly created jobs to Jenkins views. • Cleanup jobs for which a branch no longer exists. • Perform text substitutions on all text elements of a job’s config.xml. • Update jobs when their template job is updated. • Set the enabled/disabled state of new jobs. A new job can inherit the state of its template job, but an updated job can keep its most recent state. Please refer to the tutorial and the example output to get started. You may also have a look at the annotated git, svn and hg config files. Notice: The documentation is in the process of being completely rewritten. Things may seem incomplete and out of place. Contents 1 jenkins-autojobs documentation, Release 0.17.4 2 Contents CHAPTER 1 Installing The latest stable version of jenkins-autojobs can be installed from pypi using pip. $ pip install jenkins-autojobs Jenkins-autojobs depends on a version of lxml with support for XML canonicalization (c14n). Setup will attempt to install one if it is not present on your system. You might have to install the libxml and libxslt development headers if you haven’t already done so: On a Debian compatible OS: $ apt-get install libxml2-dev libxslt1-dev On a Redhat compatible OS: $ yum install libxml2-devel libxslt-devel On Arch Linux or derivatives: $ pacman -S libxslt libxml2 3 jenkins-autojobs documentation, Release 0.17.4 4 Chapter 1. Installing CHAPTER 2 Changes Changelog 0.17.4 (Mar 03, 2016) • More robust handling of the project description element (see ‘issue 56‘_). • Added the verify-ssl, cert-bundle, and client-cert config-file options and the command-line switches to match them. These control the SSL certificate verification to the Jenkins server. 0.17.3 (Feb 10, 2016) • Fix error-prone handling of the //project/description config.xml element (see issue #54). 0.17.2 (Jan 21, 2016) • Actually change the default tag method to description. This was announced as the default behaviour in 0.17.0, but was accidentally omitted from that release (issue #50). Version 0.17.1 was skipped because of a failed upload to PyPi (see issue #74). 0.17.0 (Oct 19, 2015) • Jenkins-autojobs will now use the job description field to store its metadata. Expect to see (created by jenkins-autojobs) and (jenkins-autojobs-tag: $tagname [...]) lines appended to the description of all managed jobs. This metadata was previously stored as elements in the job’s config.xml. Unfortunately, any manual re- configuration of the job would cause Jenkins to remove these extra elements. This issue is described in greater detail in issue #28. The old behavior can be kept by setting the tag-method option to element. • Added the cleanup-filters configuration option, which filters the list of jobs that jenkins-autojobs con- siders for cleanup by view name or regular expression (thanks @bartoszj). Example usage: # Cleanup only jobs in the 'Feature' and 'Release' views as well as all jobs that start with 'scratch'. cleanup-filters: views: -'Feature' 5 jenkins-autojobs documentation, Release 0.17.4 -'Release' jobs: -'^scratch-. *' • Added the list-closed option to the mercurial plugin, which controls the listing of closed branches. The old behaviour and current default value is true. Note that this feature currently works only for local mercurial repositories. 0.16.2 (Oct 02, 2015) • Fix accidentally introduced import error (thanks @bartoszj). 0.16.1 (Sep 30, 2015) • Fix cleanup functionality and improve performance (thanks @bartoszj). 0.16.0 (Sep 23, 2015) • Add the build-on-create option, which triggers a build when the job is created (thanks @bartoszj). • Ignore permission denied errors during job cleanup (thanks @bartoszj). • Fix issue with listing local mercurial branches (thanks ‘@Myz‘_). 0.15.1 (May 05, 2015) • The repo and repo-orig keys are now available to the namefmt and substitute options. They hold the sanitized and raw value of the repo top-level config key. 0.15.0 (Feb 16, 2015) • The * wildcard can now be used in the branches config key of jenkins-makejobs-svn. Example usage: file:///repo/projects/*/branches. 0.14.3 (Jan 02, 2015) • Fix sticky state when template job is enabled (thanks @d-a-n and @sustmi). 0.14.2 (Nov 24, 2014) • Fix reading of scm-username and scm-password from stdin (thanks @yamikuronue). • Fix user input on Python 3. • Fix typo that was preveneting jenkins-autojobs from working on Python 2.6 (thanks @aklemp). 6 Chapter 2. Changes jenkins-autojobs documentation, Release 0.17.4 0.14.1 (Nov 23, 2014) • Ignore the ‘All’ view when adding jobs to views (thanks ‘@myz‘_). • View creation now respects the dryrun (-n) option (thanks @aklemp). • Fix reporting of view creation (thanks @aklemp). • Fix the httpdebug (-t) option on Python 3. The httpdebug option is now available in the config file. 0.14.0 (Oct 27, 2014) • Learn the ability to add generated jobs to specific views. • Fix compatiblity with newer version of the Jenkins Mercurial plugin (thanks @ThomasMatern). 0.13.1 (May 29, 2014) • Add the tag config option to the subversion script (thanks @mrook). 0.13.0 (Apr 08, 2014) • Add the tag config option. • The cleanup option now accepts a tag name. • The substitute option now has access to matched groups (thanks @traviscosgrave). • The substitute and namefmt options can now refer to named capture groups. For example: refs: -'refs/heads/feature-(\d\d)-(?P<name>\w+)-(\d)': namefmt:'wip-{name}-{3}' The above ref config will map the branch feature-random-10 to job wip-random-10. 0.12.0 (Mar 09, 2014) • Fix a bug that made jenkins-autojobs remove all managed jobs if list_branches() failed with cleanup on (thanks @sja). • Use jenkins-webapi 0.2.0. 0.11.0 (Feb 04, 2014) • Add the cleanup config option (thanks @timmipetit). If set to true, jenkins-autojobs will remove all jobs for which a branch no longer exists. • Jenkins-autojobs now adds a createdByJenkinsAutojobs element to the config.xml of jobs that it creates. 2.1. Changelog 7 jenkins-autojobs documentation, Release 0.17.4 0.10.0 (Jan 08, 2014) • Add the sanitize config option (thanks @xgouchet). You can now substitute characters or whole patterns with the sanitize option: sanitize: '@!?#&|\^_$%*':'_' # replace any of '@!?#&|\^_$%*' with '_' 're:colou?r':'color' # replace regex 'colou?r' with 'color' The default is ’@!?#&|\^_$%*’: ’_’, which is the list of characters that are not allowed in job names. 0.9.1 (Jan 08, 2014) • Command line flags -u|-p properly overwrite username and password config keys (thanks @timmipetit). 0.9.0 (Nov 27, 2013) • Add support for Python 3.x. • Add the ‘python’ option to the mercurial yaml config. This sets the Python executable that will be used to call mercurial. This is useful when the default Python in PATH is not Python 2.x. • Use jenkins-webapi instead of python-jenkins. 0.6.0 (Sep 05, 2012) • Add mercurial support. 0.5.0 (Aug 06, 2012) Initial Release. Tutorial This tutorial goes over the steps of installing and configuring jenkins-autojobs. Installing The latest stable version of jenkins-autojobs can be installed from pypi using pip. $ pip install jenkins-autojobs Jenkins-autojobs depends on a version of lxml with support for XML canonicalization (c14n). Setup will attempt to install one if it is not present on your system. You might have to install the libxml and libxslt development headers if you haven’t already done so: On a Debian compatible OS: $ apt-get install libxml2-dev libxslt1-dev 8 Chapter 2. Changes jenkins-autojobs documentation, Release 0.17.4 On a Redhat compatible OS: $ yum install libxml2-devel libxslt-devel On Arch Linux or derivatives: $ pacman -S libxslt libxml2 Usage If jenkins-autojobs was installed succesfully, you’ll find that there are three new scripts on your system: • jenkins-makejobs-git • jenkins-makejobs-svn • jenkins-makejobs-hg All scripts accept the same command-line options and arguments and nearly the same configuration files. Usage: jenkins-makejobs-* [-rvdtjnyoupUYOP] <config.yaml> General Options: -n dry run -v show version and exit -d debug config inheritance -t debug http requests Repository Options: -r <arg> repository url -y <arg> scm username -o <arg> scm password -Y scm username (read from stdin) -O scm password (read from stdin) Jenkins Options: -j <arg> jenkins url -u <arg> jenkins username -p <arg> jenkins password -U jenkins username (read from stdin) -P jenkins password (read from stdin) Template Jobs Autojobs creates jobs from template jobs. Subsequent changes to the template jobs are propagated to all derived jobs (this behavior is configurable). A template job can be any regular Jenkins job. Depending on the SCM plugin you are using, you have to configure certain fields: Git: * Source Code Management: * Git: * Repository URL: https://your.domain/your-project.git * Branch Specifier: master * Checkout/merge to local branch (under advanced): master Subversion: 2.2.