Buildbot Documentation Release 0.8.9
Total Page:16
File Type:pdf, Size:1020Kb
BuildBot Documentation Release 0.8.9 Brian Warner June 14, 2014 Contents 1 Buildbot Tutorial 3 1.1 First Run...............................................3 1.2 First Buildbot run with Docker....................................6 1.3 A Quick Tour.............................................8 1.4 Further Reading........................................... 15 2 Buildbot Manual 21 2.1 Introduction............................................. 21 2.2 Installation.............................................. 26 2.3 Concepts............................................... 38 2.4 Configuration............................................. 47 2.5 Customization............................................ 173 2.6 New-Style Build Steps........................................ 189 2.7 Command-line Tool......................................... 192 2.8 Resources............................................... 202 2.9 Optimization............................................. 202 3 Buildbot Development 203 3.1 Master Organization......................................... 203 3.2 Definitions.............................................. 204 3.3 Buildbot Coding Style........................................ 204 3.4 Buildbot’s Test Suite......................................... 207 3.5 Configuration............................................. 212 3.6 Utilities................................................ 217 3.7 Database............................................... 226 3.8 Build Result Codes.......................................... 242 3.9 File Formats............................................. 243 3.10 Web Status.............................................. 243 3.11 Master-Slave API........................................... 244 3.12 String Encodings........................................... 251 3.13 Metrics................................................ 251 3.14 Classes................................................ 253 4 Release Notes for Buildbot 0.8.9 271 4.1 Master................................................ 271 4.2 Slave................................................. 277 4.3 Details................................................ 277 4.4 Older Versions............................................ 277 5 Indices and Tables 295 6 Copyright 297 i Buildmaster Configuration Index 299 Scheduler Index 301 Change Source Index 303 Build Step Index 305 Status Target Index 307 Command Line Index 309 Python Module Index 311 ii BuildBot Documentation, Release 0.8.9 This is the BuildBot documentation for Buildbot version 0.8.9. If you are evaluating Buildbot and would like to get started quickly, start with the Tutorial. Regular users of Buildbot should consult the Manual, and those wishing to modify Buildbot directly will want to be familiar with the Developer’s Documentation. Contents 1 BuildBot Documentation, Release 0.8.9 2 Contents CHAPTER 1 Buildbot Tutorial Contents: 1.1 First Run 1.1.1 Goal This tutorial will take you from zero to running your first buildbot master and slave as quickly as possible, without changing the default configuration. This tutorial is all about instant gratification and the five minute experience: in five minutes we want to convince you that this project Works, and that you should seriously consider spending some more time learning the system. In this tutorial no configuration or code changes are done. This tutorial assumes that you are running on Unix, but might be adaptable easily to Windows. For the fastest way through in Linux, you can use Docker, the linux container engine. Docker automates all the deployment steps for you. For a more manual approach, you should be able to cut and paste each shell block from this tutorial directly into a terminal. 1.1.2 Getting the code There are many ways to get the code on your machine. For this tutorial, we will use easy_install to install and run buildbot. While this isn’t the preferred method to install buildbot, it is the simplest one to use for the purposes of this tutorial because it should work on all systems. (The preferred method would be to install buildbot via pip.) To make this work, you will need the following installed: • Python (http://www.python.org/) and the development packages for it • virtualenv (http://pypi.python.org/pypi/virtualenv/) • Git (http://git-scm.com/) Preferably, use your package installer to install these. You will also need a working Internet connection, as virtualenv and easy_install will need to download other projects from the Internet. Note: Buildbot does not require root access. Run the commands in this tutorial as a normal, unprivileged user. Let’s dive in by typing at the terminal: 3 BuildBot Documentation, Release 0.8.9 cd mkdir -p tmp/buildbot cd tmp/buildbot virtualenv --no-site-packages sandbox source sandbox/bin/activate easy_install sqlalchemy==0.7.10 easy_install buildbot Note: The requirement to install SQLAlchemy-0.7.10 is due to a conflict between newer versions of SQLAlchemy and SQLAlchemy-Migrate. 1.1.3 Creating a master At the terminal, type: buildbot create-master master mv master/master.cfg.sample master/master.cfg Now start it: buildbot start master tail -f master/twistd.log You will now see all of the log information from the master in this terminal. You should see lines like this: 2011-12-04 10:04:40-0600 [-] Starting factory <buildbot.status.web.baseweb.RotateLogSite instance at 0x2e36638> 2011-12-04 10:04:40-0600 [-] Setting up http.log rotating 10 files of 10000000 bytes each 2011-12-04 10:04:40-0600 [-] WebStatus using (/home/dustin/tmp/buildbot/master/public_html) 2011-12-04 10:04:40-0600 [-] removing 0 old schedulers, updating 0, and adding 1 2011-12-04 10:04:40-0600 [-] adding 1 new changesources, removing 0 2011-12-04 10:04:40-0600 [-] gitpoller: using workdir ’/home/dustin/tmp/buildbot/master/gitpoller-workdir’ 2011-12-04 10:04:40-0600 [-] gitpoller: initializing working dir from git://github.com/buildbot/pyflakes.git 2011-12-04 10:04:40-0600 [-] configuration update complete 2011-12-04 10:04:41-0600 [-] gitpoller: checking out master 2011-12-04 10:04:41-0600 [-] gitpoller: finished initializing working dir from git://github.com/buildbot/pyflakes.git at rev 1a4af6ec1dbb724b884ea14f439b272f30439e4d 1.1.4 Creating a slave Open a new terminal and enter the same sandbox you created before: cd cd tmp/buildbot source sandbox/bin/activate Install the buildslave command: easy_install buildbot-slave Now, create the slave: buildslave create-slave slave localhost:9989 example-slave pass The user:host pair, username, and password should be the same as the ones in master.cfg; verify this is the case by looking at the section for c[’slaves’] and c[’slavePortnum’]: cat master/master.cfg Now, start the slave: buildslave start slave 4 Chapter 1. Buildbot Tutorial BuildBot Documentation, Release 0.8.9 Check the slave’s log: tail -f slave/twistd.log You should see lines like the following at the end of the worker log: 2009-07-29 20:59:18+0200 [Broker,client] message from master: attached 2009-07-29 20:59:18+0200 [Broker,client] SlaveBuilder.remote_print(buildbot-full): message from master: attached 2009-07-29 20:59:18+0200 [Broker,client] sending application-level keepalives every 600 seconds Meanwhile, in the other terminal, in the master log, if you tail the log you should see lines like this: 2011-03-13 18:46:58-0700 [Broker,1,127.0.0.1] slave ’example-slave’ attaching from IPv4Address(TCP, ’127.0.0.1’, 41306) 2011-03-13 18:46:58-0700 [Broker,1,127.0.0.1] Got slaveinfo from ’example-slave’ 2011-03-13 18:46:58-0700 [Broker,1,127.0.0.1] bot attached 2011-03-13 18:46:58-0700 [Broker,1,127.0.0.1] Buildslave example-slave attached to runtests You should now be able to go to http://localhost:8010, where you will see a web page similar to: Click on the Waterfall Display link (http://localhost:8010/waterfall) and you get this: 1.1. First Run 5 BuildBot Documentation, Release 0.8.9 That’s the end of the first tutorial. A bit underwhelming, you say? Well, that was the point! We just wanted to get you to dip your toes in the water. It’s easy to take your first steps, but this is about as far as we can go without touching the configuration. You’ve got a taste now, but you’re probably curious for more. Let’s step it up a little in the second tutorial by changing the configuration and doing an actual build. Continue on to A Quick Tour. 1.2 First Buildbot run with Docker Docker (http://www.docker.io) is an tool that makes building and deploying custom environments a breeze. It uses lightweight linux containers (LXC) and performs quickly, making it a great instrument for the testing community. The next section includes a Docker pre-flight check. If it takes more that 3 minutes to get the ‘Success’ message for you, try the Buildbot pip-based first run instead. 1.2.1 Current Docker dependencies • Linux system, with at least kernel 3.8 and AUFS support. For example, Standard Ubuntu, Debian and Arch systems. • Packages: lxc, iptables, ca-certificates, and bzip2 packages • Local clock on time or slightly in the future for proper SSL communication • Download, launch and test docker is happy in your linux enviroment: mkdir tmp; cd tmp wget -O docker http://get.docker.io/builds/Linux/x86_64/docker-latest chmod 755 docker; sudo ./docker -d & sudo ./docker run -i busybox /bin/echo Success 6 Chapter 1. Buildbot Tutorial BuildBot Documentation, Release 0.8.9 1.2.2 Building and running Buildbot # Download Buildbot Dockerfile. wget https://raw.github.com/buildbot/buildbot/master/master/contrib/Dockerfile # Build the Buildbot container (it will take a few minutes