BuildBot This is the BuildBot manual. Copyright (C) 2005, 2006, 2009, 2010 Brian Warner Copying and distribution of this file, with or without modification, are permitted in any medium without royalty provided the copyright notice and this notice are preserved. i Table of Contents 1 Introduction..................................... 1 1.1 History and Philosophy ........................................ 1 1.2 System Architecture............................................ 2 1.2.1 BuildSlave Connections.................................... 3 1.2.2 Buildmaster Architecture .................................. 4 1.2.3 Status Delivery Architecture............................... 6 1.3 Control Flow ................................................... 6 2 Installation ...................................... 8 2.1 Requirements .................................................. 8 2.2 Installing the code ............................................. 8 2.3 Creating a buildmaster ......................................... 9 2.4 Upgrading an Existing Buildmaster............................ 10 2.5 Creating a buildslave.......................................... 10 2.5.1 Buildslave Options ....................................... 12 2.6 Launching the daemons ....................................... 13 2.7 Logfiles ................................................... .... 14 2.8 Shutdown ................................................... 14 2.9 Maintenance .................................................. 15 2.10 Troubleshooting.............................................. 15 2.10.1 Starting the buildslave .................................. 15 2.10.2 Connecting to the buildmaster .......................... 15 2.10.3 Forcing Builds .......................................... 16 3 Concepts ....................................... 17 3.1 Version Control Systems ...................................... 17 3.1.1 Generalizing VC Systems ................................. 17 3.1.2 Source Tree Specifications ................................ 18 3.1.3 How Different VC Systems Specify Sources ............... 19 3.1.4 Attributes of Changes .................................... 20 3.2 Schedulers ................................................... 22 3.3 BuildSet ................................................... 23 3.4 BuildRequest ................................................. 24 3.5 Builder ................................................... .... 24 3.6 Users ................................................... ...... 25 3.6.1 Doing Things With Users ................................ 25 3.6.2 Email Addresses ......................................... 26 3.6.3 IRC Nicknames .......................................... 27 3.6.4 Live Status Clients ....................................... 27 3.7 Build Properties .............................................. 27 ii 4 Configuration .................................. 29 4.1 Config File Format ............................................ 29 4.2 Loading the Config File ....................................... 30 4.3 Testing the Config File ........................................ 30 4.4 Defining the Project .......................................... 31 4.5 Change Sources and Schedulers................................ 32 4.5.1 Scheduler Scheduler ...................................... 33 4.5.2 AnyBranchScheduler ..................................... 34 4.5.3 Dependent Scheduler ..................................... 34 4.5.4 Periodic Scheduler ....................................... 35 4.5.5 Nightly Scheduler ........................................ 35 4.5.6 Try Schedulers ........................................... 37 4.5.7 Triggerable Scheduler .................................... 37 4.6 Merging BuildRequests........................................ 38 4.7 Prioritizing Builders........................................... 39 4.8 Setting the slaveport .......................................... 39 4.9 Buildslave Specifiers........................................... 39 4.9.1 When Buildslaves Go Missing ............................ 40 4.10 On-Demand ("Latent") Buildslaves .......................... 41 4.10.1 Amazon Web Services Elastic Compute Cloud ("AWS EC2") ................................................... 41 4.10.1.1 Get an AWS EC2 Account ......................... 41 4.10.1.2 Create an AMI ..................................... 42 4.10.1.3 Configure the Master with an EC2LatentBuildSlave ................................................... ...... 42 4.10.2 Dangers with Latent Buildslaves......................... 44 4.10.3 Writing New Latent Buildslaves ......................... 45 4.11 Defining Global Properties ................................... 45 4.12 Defining Builders ............................................ 45 4.13 Defining Status Targets ...................................... 47 4.14 Limiting Memory and Disk Usage ............................ 47 4.15 Debug options ............................................... 48 5 Getting Source Code Changes................ 50 5.1 Change Sources ............................................... 50 5.2 Choosing ChangeSources ...................................... 52 5.3 CVSToys - PBService ......................................... 52 5.4 Mail-parsing ChangeSources ................................... 53 5.4.1 Subscribing the Buildmaster.............................. 53 5.4.2 Using Maildirs ........................................... 54 5.4.3 Parsing Email Change Messages .......................... 54 5.4.3.1 FCMaildirSource .................................... 55 5.4.3.2 SyncmailMaildirSource .............................. 55 5.4.3.3 BonsaiMaildirSource................................. 56 5.4.3.4 SVNCommitEmailMaildirSource ..................... 56 5.4.3.5 BzrLaunchpadEmailMaildirSource ................... 56 5.5 PBChangeSource.............................................. 56 5.6 P4Source ................................................... 57 iii 5.7 BonsaiPoller .................................................. 58 5.8 SVNPoller ................................................... 58 5.9 MercurialHook ................................................ 62 5.10 Bzr Hook ................................................... 64 5.11 Bzr Poller ................................................... 65 6 Build Process .................................. 66 6.1 Build Steps ................................................... 66 6.1.1 Common Parameters ..................................... 67 6.1.2 Using Build Properties ................................... 68 6.1.3 Source Checkout ......................................... 70 6.1.3.1 CVS ................................................ 72 6.1.3.2 SVN ................................................ 72 6.1.3.3 Darcs ............................................... 75 6.1.3.4 Mercurial ........................................... 75 6.1.3.5 Arch ................................................ 76 6.1.3.6 Bazaar .............................................. 76 6.1.3.7 Bzr ................................................. 76 6.1.3.8 P4 .................................................. 77 6.1.3.9 Git.................................................. 77 6.1.3.10 Monotone .......................................... 78 6.1.4 ShellCommand ........................................... 78 6.1.5 Simple ShellCommand Subclasses ........................ 80 6.1.5.1 Configure ........................................... 81 6.1.5.2 Compile ............................................. 81 6.1.5.3 Test ................................................. 82 6.1.5.4 TreeSize ............................................. 82 6.1.5.5 PerlModuleTest ..................................... 82 6.1.6 Testing with mysql-test-run .............................. 82 6.1.6.1 SetProperty ......................................... 84 6.1.6.2 SubunitShellCommand .............................. 84 6.1.7 Python BuildSteps ....................................... 84 6.1.7.1 BuildEPYDoc ....................................... 84 6.1.7.2 PyFlakes ............................................ 85 6.1.7.3 PyLint .............................................. 85 6.1.8 Transferring Files ........................................ 85 6.1.9 Steps That Run on the Master ........................... 87 6.1.10 Triggering Schedulers ................................... 87 6.1.11 Writing New BuildSteps................................. 88 6.1.11.1 Writing BuildStep Constructors .................... 88 6.1.11.2 BuildStep LogFiles ................................. 89 6.1.11.3 Reading Logfiles.................................... 90 6.1.11.4 Adding LogObservers .............................. 91 6.1.11.5 BuildStep URLs.................................... 95 6.2 Interlocks ................................................... 96 6.3 Build Factories................................................ 99 6.3.1 BuildStep Objects........................................ 99 6.3.2 BuildFactory ............................................. 99 iv 6.3.2.1 BuildFactory Attributes ............................ 101 6.3.2.2 Quick builds ....................................... 101 6.3.3 Process-Specific build factories .......................... 101 6.3.3.1 GNUAutoconf...................................... 101 6.3.3.2 CPAN ............................................. 102 6.3.3.3 Python distutils .................................... 103 6.3.3.4 Python/Twisted/trial projects...................... 103 7 Status Delivery ............................... 105 7.1 WebStatus