Avocado Documentation Release 63.0

avocado Documentation Release 63.0

Avocado Development Team

Jul 19, 2018

Contents

1 About Avocado 3

2 Getting Started 5 2.1 Installing Avocado...... 5 2.2 Using Avocado...... 8 2.3 Writing a Simple Test...... 10 2.4 Running A More Complex Test Job...... 10 2.5 Interrupting The Job On First Failed Test (failfast)...... 11 2.6 Ignoring Missing Test References...... 11 2.7 Running Tests With An External Runner...... 12 2.8 Debugging tests...... 13

3 Writing Avocado Tests 15 3.1 Basic example...... 15 3.2 Test statuses...... 16 3.3 Saving test generated (custom) data...... 17 3.4 Accessing test data ﬁles...... 18 3.5 Accessing test parameters...... 18 3.6 Running multiple variants of tests...... 19 3.7 Advanced logging capabilities...... 21 3.8 unittest.TestCase heritage...... 23 3.9 Setup and cleanup methods...... 24 3.10 Running third party test suites...... 24 3.11 Fetching asset ﬁles...... 25 3.12 Test Output Check and Output Record Mode...... 27 3.13 Test log, stdout and stderr in native Avocado modules...... 30 3.14 Setting a Test Timeout...... 31 3.15 Skipping Tests...... 32 3.16 Cancelling Tests...... 33 3.17 Docstring Directives...... 34 3.18 Python unittest Compatibility Limitations And Caveats...... 40 3.19 Environment Variables for Tests...... 41 3.20 SIMPLE Tests BASH extensions...... 42 3.21 SIMPLE Tests Status...... 43 3.22 Wrap Up...... 43

4 Result Formats 45

i 4.1 Results for human beings...... 45 4.2 Machine readable results...... 46 4.3 Multiple results at once...... 49 4.4 Exit Codes...... 49 4.5 Implementing other result formats...... 49

5 Configuration 51 5.1 Config file parsing order...... 51 5.2 Plugin config files...... 52 5.3 Parsing order recap...... 52 5.4 Order of precedence for values used in tests...... 52 5.5 Config plugin...... 53 5.6 Avocado Data Directories...... 53

6 Test discovery 55 6.1 The order of test loaders...... 55 6.2 Running simple tests with arguments...... 56 6.3 Filtering tests by tags...... 56 6.4 Test References...... 56

7 Logging system 59 7.1 Tweaking the UI...... 59 7.2 Storing custom logs...... 59 7.3 Paginator...... 60

8 Sysinfo collection 61

9 Test parameters 63 9.1 TreeNode...... 64 9.2 AvocadoParams...... 64 9.3 Parameter Paths...... 65 9.4 Variant...... 65 9.5 Dumping/Loading Variants...... 65 9.6 Varianter...... 66 9.7 Default params...... 67 9.8 Varianter plugins...... 67 9.9 Multiplexer...... 67 9.10 Multiplex domains...... 68 9.11 MuxPlugin...... 70 9.12 MuxTree...... 70

10 Job Replay 73

11 Job Diff 77

12 Running Tests Remotely 79

13 Subclassing Avocado 81

14 Debugging with GDB 85 14.1 Transparent Execution of Executables...... 85 14.2 avocado.utils.gdb APIs...... 87

15 Wrap executables run by tests 89 15.1 Usage...... 89 15.2 Caveats...... 89 ii 16 Plugin System 91 16.1 Listing plugins...... 91 16.2 Writing a plugin...... 92

17 Utilities 95 17.1 Utilities...... 95

18 Optional Plugins 99 18.1 Optional Plugins...... 99

19 Advanced Topics and Maintenance 119 19.1 Reference Guide...... 119 19.2 Contribution and Community Guide...... 130 19.3 Avocado development tips...... 135 19.4 Releasing avocado...... 137 19.5 Other Resources...... 141

20 API Reference 143 20.1 Test APIs...... 143 20.2 Utilities APIs...... 146 20.3 Internal (Core) APIs...... 203 20.4 Extension (plugin) APIs...... 243 20.5 Optional Plugins API...... 257

21 Avocado Release Notes 267 21.1 Release Notes...... 267

22 Request For Comments (RFCs) 327 22.1 Request For Comments (RFCs)...... 327 22.2 Indices and tables...... 333

Python Module Index 335

iii iv avocado Documentation, Release 63.0

Contents:

Contents 1 avocado Documentation, Release 63.0

2 Contents CHAPTER 1

About Avocado

Avocado is a set of tools and libraries to help with automated testing. One can call it a test framework with beneﬁts. Native tests are written in Python and they follow the unittest pattern, but any executable can serve as a test. Avocado is composed of: • A test runner that lets you execute tests. Those tests can be either written in your language of choice, or be written in Python and use the available libraries. In both cases, you get facilities such as automated log and system information collection. • Libraries that help you write tests in a concise, yet expressive and powerful way. You can ﬁnd more information about what libraries are intended for test writers at Libraries and APIs. • Plugins that can extend and add new functionality to the Avocado Framework. Avocado is built on the experience accumulated with Autotest, while improving on its weaknesses and shortcomings. Avocado tries as much as possible to comply with standard Python testing technology. Tests written using the Avocado API are derived from the unittest class, while other methods suited to functional and performance testing were added. The test runner is designed to help people to run their tests while providing an assortment of system and logging facilities, with no effort, and if you want more features, then you can start using the API features progressively.

3 avocado Documentation, Release 63.0

4 Chapter 1. About Avocado CHAPTER 2

Getting Started

Those who prefer video-introduction, take a look at Other Resources. Either way ﬁrst step towards using Avocado is, quite obviously, installing it.

2.1 Installing Avocado

Avocado is primarily written in Python, so a standard Python installation is possible and often preferable.

Tip: If you are looking for Virtualization speciﬁc testing, also consider looking at Avocado-VT installation instructions after ﬁnishing the Avocado installation.

2.1.1 Installing with standard Python tools

The simplest installation method is through pip. On most POSIX systems with Python 2.7 and pip available, installation can be performed with a single command: pip install--user avocado-framework

This will fetch the Avocado package (and possibly some of its dependecies) from the PyPI repository, and will attempt to install it in the user’s home directory (usually under ~/.local).

Tip: If you want to perform a system-wide installation, drop the --user switch.

If you want even more isolation, Avocado can also be installed in a Python virtual environment. with no additional steps besides creating and activating the “venv” itself: python-m virtualenv/path/to/new/virtual_environment ./path/to/new/virtual_environment/bin/activate pip install avocado-framework

5 avocado Documentation, Release 63.0

Please note that this installs the Avocado core functionality. Many Avocado features are distributed as non- core plugins, also available as additional packages on PyPI. You should be able to find them via pip search avocado-framework-plugin | grep avocado-framework-plugin. Some of them are listed below: • avocado-framework-plugin-result-html: HTML Report for Jobs • avocado-framework-plugin-resultsdb: Propagate Job results to Resultsdb • avocado-framework-plugin-runner-remote: Runner for Remote Execution • avocado-framework-plugin-runner-vm: Runner for libvirt VM Execution • avocado-framework-plugin-runner-docker: Runner for Execution on Docker Containers • avocado-framework-plugin-loader-yaml: Loads tests from YAML files • avocado-framework-plugin-robot: Execution of Robot Framework tests • avocado-framework-plugin-varianter-yaml-to-mux: Parse YAML file into variants

2.1.2 Installing from Packages

Fedora

Avocado is available in stock Fedora 24 and later. The main package name is python-avocado, and can be installed with:

dnf install python-avocado

Other available packages (depending on the Avocado version) may include: • python-avocado-examples: contains example tests and other example ﬁles • python2-avocado-plugins-output-html: HTML job report plugin • python2-avocado-plugins-resultsdb: propagate Job results to Resultsdb • python2-avocado-plugins-runner-remote: execution of jobs on a remote machine • python2-avocado-plugins-runner-vm: execution of jobs on a libvirt based VM • python2-avocado-plugins-runner-docker: execution of jobs on a Docker container • python-avocado-plugins-varianter-yaml-to-mux: parse YAML ﬁle into variants • python2-avocado-plugins-varianter-pict: varianter with combinatorial capabilities by PICT

Fedora from Avocado’s own Repo

The Avocado project also makes the latest release, and the LTS (Long Term Stability) releases available from its own package repository. To use it, first get the package repositories configuration file by running the following command:

sudo curl https://avocado-project.org/data/repos/avocado-fedora.repo-o/etc/yum.

˓→repos.d/avocado.repo

Now check if you have the avocado and avocado-lts repositories conﬁgured by running:

6 Chapter 2. Getting Started avocado Documentation, Release 63.0

sudo dnf repolist avocado avocado-lts ... repo id repo name status avocado Avocado 50 avocado-lts Avocado LTS (Long Term Stability) disabled

Regular users of Avocado will want to use the standard avocado repository, which tracks the latest Avocado releases. For more information about the LTS releases, please refer to RFC: Long Term Stability and to your package management docs on how to switch to the avocado-lts repo. Finally, after deciding between regular Avocado releases or LTS, you can install the RPM packages by running the following commands:

dnf install python-avocado

Additionally, other Avocado packages are available for Fedora: • python-avocado-examples: contains example tests and other example ﬁles • python2-avocado-plugins-output-html: HTML job report plugin • python2-avocado-plugins-resultsdb: propagate Job results to Resultsdb • python2-avocado-plugins-runner-remote: execution of jobs on a remote machine • python2-avocado-plugins-runner-vm: execution of jobs on a libvirt based VM • python2-avocado-plugins-runner-docker: execution of jobs on a Docker container • python-avocado-plugins-varianter-yaml-to-mux: parse YAML ﬁle into variants • python2-avocado-plugins-varianter-pict: varianter with combinatorial capabilities by PICT

Enterprise Linux

Avocado packages for Enterprise Linux are available from the Avocado project RPM repository. Additionally, some packages from the EPEL repo are necessary, so you need to enable it ﬁrst. For EL7, running the following command should do it:

yum install https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm

Then you must use the Avocado project RHEL repo (https://avocado-project.org/data/repos/avocado-el.repo). Run- ning the following command should give you the basic Avocado installation ready:

curl https://avocado-project.org/data/repos/avocado-el.repo-o/etc/yum.repos.d/

˓→avocado.repo yum install python-avocado

2.1. Installing Avocado 7 avocado Documentation, Release 63.0

• python-avocado-plugins-varianter-yaml-to-mux: parse YAML ﬁle into variants • python2-avocado-plugins-varianter-pict: varianter with combinatorial capabilities by PICT The LTS (Long Term Stability) repositories are also available for Enterprise Linux. Please refer to RFC: Long Term Stability and to your package management docs on how to switch to the avocado-lts repo.

OpenSUSE

The OpenSUSE project packages LTS versions of Avocado. You can install packages by running the following commands:

sudo zypper install avocado

Debian

DEB package support is available in the source tree (look at the contrib/packages/debian directory. No actual packages are provided by the Avocado project or the Debian repos.

2.1.3 Generic installation from a GIT repository

First make sure you have a basic set of packages installed. The following applies to Fedora based distributions, please adapt to your platform:

sudo dnf install-y python2 git gcc python-devel python-pip libvirt-devel libffi-

˓→devel openssl-devel libyaml-devel redhat-rpm-config xz-devel

Then to install Avocado from the git repository run:

git clone git://github.com/avocado-framework/avocado.git cd avocado sudo make requirements sudo python setup.py install

Note that python and pip should point to the Python interpreter version 2.7.x. If you’re having trouble to install, you can try again and use the command line utilities python2.7 and pip2.7. Please note that some Avocado functionality may be implemented by optional plugins. To install say, the HTML report plugin, run:

cd optional_plugins/html sudo python setup.py install

If you intend to hack on Avocado, you may want to look at Hacking and Using Avocado.

2.2 Using Avocado

You should ﬁrst experience Avocado by using the test runner, that is, the command line tool that will conveniently run your tests and collect their results.

8 Chapter 2. Getting Started avocado Documentation, Release 63.0

2.2.1 Running Tests

To do so, please run avocado with the run sub-command followed by a test reference, which could be either a path to the ﬁle, or a recognizable name:

$ avocado run /bin/true JOB ID : 381b849a62784228d2fd208d929cc49f310412dc JOB LOG : $HOME/avocado/job-results/job-2014-08-12T15.39-381b849a/job.log (1/1) /bin/true: PASS (0.01 s) RESULTS : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 JOB TIME : 0.11 s JOB HTML : $HOME/avocado/job-results/job-2014-08-12T15.39-381b849a/html/results.html

You probably noticed that we used /bin/true as a test, and in accordance with our expectations, it passed! These are known as simple tests, but there is also another type of test, which we call instrumented tests. See more at Test Types or just keep reading.

Note: Although in most cases running avocado run $test1 $test3 ... is ﬁne, it can lead to argument vs. test name clashes. The safest way to execute tests is avocado run --$argument1 --$argument2 -- $test1 $test2. Everything after – will be considered positional arguments, therefore test names (in case of avocado run)

2.2.2 Listing tests

You have two ways of discovering the tests. You can simulate the execution by using the --dry-run argument: avocado run/bin/true--dry-run JOB ID : 0000000000000000000000000000000000000000 JOB LOG :/tmp/avocado-dry-runSeWniM/job-2015-10-16T15.46-0000000/job.log (1/1)/bin/true: SKIP RESULTS : PASS0| ERROR0| FAIL0| SKIP1| WARN0| INTERRUPT0 JOB TIME : 0.10s JOB HTML :/tmp/avocado-dry-runSeWniM/job-2015-10-16T15.46-0000000/html/results.html which supports all run arguments, simulates the run and even lists the test params. The other way is to use list subcommand that lists the discovered tests If no arguments provided, Avocado lists “default” tests per each plugin. The output might look like this:

$ avocado list INSTRUMENTED /usr/share/doc/avocado/tests/abort.py INSTRUMENTED /usr/share/doc/avocado/tests/datadir.py INSTRUMENTED /usr/share/doc/avocado/tests/doublefail.py INSTRUMENTED /usr/share/doc/avocado/tests/doublefree.py INSTRUMENTED /usr/share/doc/avocado/tests/errortest.py INSTRUMENTED /usr/share/doc/avocado/tests/failtest.py INSTRUMENTED /usr/share/doc/avocado/tests/fiotest.py INSTRUMENTED /usr/share/doc/avocado/tests/gdbtest.py INSTRUMENTED /usr/share/doc/avocado/tests/gendata.py INSTRUMENTED /usr/share/doc/avocado/tests/linuxbuild.py INSTRUMENTED /usr/share/doc/avocado/tests/multiplextest.py INSTRUMENTED /usr/share/doc/avocado/tests/passtest.py INSTRUMENTED /usr/share/doc/avocado/tests/sleeptenmin.py INSTRUMENTED /usr/share/doc/avocado/tests/sleeptest.py (continues on next page)

2.2. Using Avocado 9 avocado Documentation, Release 63.0

(continued from previous page) INSTRUMENTED /usr/share/doc/avocado/tests/synctest.py INSTRUMENTED /usr/share/doc/avocado/tests/timeouttest.py INSTRUMENTED /usr/share/doc/avocado/tests/warntest.py INSTRUMENTED /usr/share/doc/avocado/tests/whiteboard.py ...

These Python ﬁles are considered by Avocado to contain INSTRUMENTED tests. Let’s now list only the executable shell scripts:

$ avocado list | grep ^SIMPLE SIMPLE /usr/share/doc/avocado/tests/env_variables.sh SIMPLE /usr/share/doc/avocado/tests/output_check.sh SIMPLE /usr/share/doc/avocado/tests/simplewarning.sh SIMPLE /usr/share/doc/avocado/tests/failtest.sh SIMPLE /usr/share/doc/avocado/tests/passtest.sh

Here, as mentioned before, SIMPLE means that those files are executables treated as simple tests. You can also give the --verbose or -V flag to display files that were found by Avocado, but are not considered Avocado tests:

$ avocado list examples/gdb-prerun-scripts/ -V Type Test Tag(s) NOT_A_TEST examples/gdb-prerun-scripts/README NOT_A_TEST examples/gdb-prerun-scripts/pass-sigusr1

TEST TYPES SUMMARY ======SIMPLE: 0 INSTRUMENTED: 0 MISSING: 0 NOT_A_TEST: 2

Notice that the verbose ﬂag also adds summary information.

2.3 Writing a Simple Test

This very simple example of simple test written in shell script:

$ echo '#!/bin/bash' > /tmp/simple_test.sh $ echo 'exit 0' >> /tmp/simple_test.sh $ chmod +x /tmp/simple_test.sh

Notice that the ﬁle is given executable permissions, which is a requirement for Avocado to treat it as a simple test. Also notice that the script exits with status code 0, which signals a successful result to Avocado.

2.4 Running A More Complex Test Job

You can run any number of test in an arbitrary order, as well as mix and match instrumented and simple tests:

$ avocado run failtest.py sleeptest.py synctest.py failtest.py synctest.py /tmp/

˓→simple_test.sh JOB ID : 86911e49b5f2c36caeea41307cee4fecdcdfa121 (continues on next page)

10 Chapter 2. Getting Started avocado Documentation, Release 63.0

(continued from previous page) JOB LOG : $HOME/avocado/job-results/job-2014-08-12T15.42-86911e49/job.log (1/6) failtest.py:FailTest.test: FAIL (0.00 s) (2/6) sleeptest.py:SleepTest.test: PASS (1.00 s) (3/6) synctest.py:SyncTest.test: PASS (2.43 s) (4/6) failtest.py:FailTest.test: FAIL (0.00 s) (5/6) synctest.py:SyncTest.test: PASS (2.44 s) (6/6) /tmp/simple_test.sh.1: PASS (0.02 s) RESULTS : PASS 4 | ERROR 0 | FAIL 2 | SKIP 0 | WARN 0 | INTERRUPT 0 JOB TIME : 5.98 s JOB HTML : $HOME/avocado/job-results/job-2014-08-12T15.42-86911e49/html/results.html

2.5 Interrupting The Job On First Failed Test (failfast)

The Avocado run command has the option --failfast on to exit the job on ﬁrst failed test:

$ avocado run --failfast on /bin/true /bin/false /bin/true /bin/true JOB ID : eaf51b8c7d6be966bdf5562c9611b1ec2db3f68a JOB LOG : $HOME/avocado/job-results/job-2016-07-19T09.43-eaf51b8/job.log (1/4) /bin/true: PASS (0.01 s) (2/4) /bin/false: FAIL (0.01 s) Interrupting job (failfast). RESULTS : PASS 1 | ERROR 0 | FAIL 1 | SKIP 2 | WARN 0 | INTERRUPT 0 JOB TIME : 0.12 s JOB HTML : /home/apahim/avocado/job-results/job-2016-07-19T09.43-eaf51b8/html/

˓→results.html

One can also use --failfast off in order to force-disable failfast mode when replaying a job executed with --failfast on.

2.6 Ignoring Missing Test References

When you provide a list of test references, Avocado will try to resolve all of them to tests. If one or more test references can not be resolved to tests, the Job will not be created. Example:

$ avocado run passtest.py badtest.py Unable to resolve reference(s) 'badtest.py' with plugins(s) 'file', 'robot', 'external

˓→', try running 'avocado list -V badtest.py' to see the details.

But if you want to execute the Job anyway, with the tests that could be resolved, you can use --ignore-missing-references on. The same message will appear in the UI, but the Job will be executed:

$ avocado run passtest.py badtest.py --ignore-missing-references on Unable to resolve reference(s) 'badtest.py' with plugins(s) 'file', 'robot', 'external

˓→', try running 'avocado list -V badtest.py' to see the details. JOB ID : 85927c113074b9defd64ea595d6d1c3fdfc1f58f JOB LOG : $HOME/avocado/job-results/job-2017-05-17T10.54-85927c1/job.log (1/1) passtest.py:PassTest.test: PASS (0.02 s) RESULTS : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0 JOB TIME : 0.11 s JOB HTML : $HOME/avocado/job-results/job-2017-05-17T10.54-85927c1/html/results.html

2.5. Interrupting The Job On First Failed Test (failfast) 11 avocado Documentation, Release 63.0

The --ignore-missing-references option accepts the argument off. Since it’s disabled by default, the off argument only makes sense in replay jobs, when the original job was executed with --ignore-missing-references on.

2.7 Running Tests With An External Runner

It’s quite common to have organically grown test suites in most software projects. These usually include a custom built, very speciﬁc test runner that knows how to ﬁnd and run their own tests. Still, running those tests inside Avocado may be a good idea for various reasons, including being able to have results in different human and machine readable formats, collecting system information alongside those tests (the Avocado’s sysinfo functionality), and more. Avocado makes that possible by means of its “external runner” feature. The most basic way of using it is:

$ avocado run --external-runner=/path/to/external_runner foo bar baz

In this example, Avocado will report individual test results for tests foo, bar and baz. The actual results will be based on the return code of individual executions of /path/to/external_runner foo, /path/to/external_runner bar and ﬁnally /path/to/external_runner baz. As another way to explain an show how this feature works, think of the “external runner” as some kind of interpreter and the individual tests as anything that this interpreter recognizes and is able to execute. A UNIX shell, say /bin/sh could be considered an external runner, and ﬁles with shell code could be considered tests:

$ echo "exit 0" > /tmp/pass $ echo "exit 1" > /tmp/fail $ avocado run --external-runner=/bin/sh /tmp/pass /tmp/fail JOB ID : 4a2a1d259690cc7b226e33facdde4f628ab30741 JOB LOG : /home//avocado/job-results/job--/job.log (1/2) /tmp/pass: PASS (0.01 s) (2/2) /tmp/fail: FAIL (0.01 s) RESULTS : PASS 1 | ERROR 0 | FAIL 1 | SKIP 0 | WARN 0 | INTERRUPT 0 JOB TIME : 0.11 s JOB HTML : /home//avocado/job-results/job--/html/results.html

This example is pretty obvious, and could be achieved by giving /tmp/pass and /tmp/fail shell “shebangs” (#!/bin/sh), making them executable (chmod +x /tmp/pass /tmp/fail), and running them as “SIMPLE” tests. But now consider the following example:

$ avocado run --external-runner=/bin/curl http://local-avocado-server:9405/jobs/ \ http://remote-avocado-server:9405/jobs/ JOB ID : 56016a1ffffaba02492fdbd5662ac0b958f51e11 JOB LOG : /home//avocado/job-results/job--/job.log (1/2) http://local-avocado-server:9405/jobs/: PASS (0.02 s) (2/2) http://remote-avocado-server:9405/jobs/: FAIL (3.02 s) RESULTS : PASS 1 | ERROR 0 | FAIL 1 | SKIP 0 | WARN 0 | INTERRUPT 0 JOB TIME : 3.14 s JOB HTML : /home//avocado/job-results/job--/html/results.html

This effectively makes /bin/curl an “external test runner”, responsible for trying to fetch those URLs, and reporting PASS or FAIL for each of them.

12 Chapter 2. Getting Started avocado Documentation, Release 63.0

2.8 Debugging tests

2.8.1 Showing test output

When developing new tests, you frequently want to look straight at the job log, without switching screens or having to “tail” the job log. In order to do that, you can use avocado --show test run ... or avocado run --show-job-log ... options:

$ avocado --show test run examples/tests/sleeptest.py ... Job ID: f9ea1742134e5352dec82335af584d1f151d4b85

START 1-sleeptest.py:SleepTest.test

PARAMS (key=timeout, path=*, default=None) => None PARAMS (key=sleep_length, path=*, default=1) => 1 Sleeping for 1.00 seconds PASS 1-sleeptest.py:SleepTest.test

Test results available in $HOME/avocado/job-results/job-2015-06-02T10.45-f9ea174

As you can see, the UI output is suppressed and only the job log is shown, making this a useful feature for test development and debugging.

2.8.2 Interrupting tests execution

To interrupt a job execution a user can press ctrl+c which after a single press sends SIGTERM to the main test’s process and waits for it to ﬁnish. If this does not help user can press ctrl+c again (after 2s grace period) which destroys the test’s process ungracefully and safely ﬁnishes the job execution always providing the test results. To pause the test execution a user can use ctrl+z which sends SIGSTOP to all processes inherited from the test’s PID. We do our best to stop all processes, but the operation is not atomic and some new processes might not be stopped. Another ctrl+z sends SIGCONT to all processes inherited by the test’s PID resuming the execution. Note the test execution time (concerning the test timeout) are still running while the test’s process is stopped. The test can also be interrupted by an Avocado feature. One example would be the Debugging with GDB Debugging with GDB feature. For custom interactions it is also possible to use other means like pdb or pydevd Avocado development tips breakpoints. Beware it’s not possible to use STDIN from tests (unless dark magic is used).

2.8. Debugging tests 13 avocado Documentation, Release 63.0

14 Chapter 2. Getting Started CHAPTER 3

Writing Avocado Tests

We are going to write an Avocado test in Python and we are going to inherit from avocado.Test. This makes this test a so-called instrumented test.

3.1 Basic example

Let’s re-create an old time favorite, sleeptest1. It is so simple, it does nothing besides sleeping for a while: import time from avocado import Test class SleepTest(Test):

def test(self): sleep_length= self.params.get('sleep_length', default=1) self.log.debug("Sleeping for %.2f seconds", sleep_length) time.sleep(sleep_length)

This is about the simplest test you can write for Avocado, while still leveraging its API power.

3.1.1 What is an Avocado Test

As can be seen in the example above, an Avocado test is a method that starts with test in a class that inherits from avocado.Test.

3.1.2 Multiple tests and naming conventions

You can have multiple tests in a single class.

1 sleeptest is a functional test for Avocado. It’s “old” because we also have had such a test for Autotest for a long time.

15 avocado Documentation, Release 63.0

To do so, just give the methods names that start with test, say test_foo, test_bar and so on. We recommend you follow this naming style, as deﬁned in the PEP8 Function Names section. For the class name, you can pick any name you like, but we also recommend that it follows the CamelCase convention, also known as CapWords, deﬁned in the PEP 8 document under Class Names.

3.1.3 Convenience Attributes

Note that the test class provides you with a number of convenience attributes: • A ready to use log mechanism for your test, that can be accessed by means of self.log. It lets you log debug, info, error and warning messages. • A parameter passing system (and fetching system) that can be accessed by means of self.params. This is hooked to the Varianter, about which you can ﬁnd that more information at Test parameters. • And many more (see avocado.core.test.Test) To minimize the accidental clashes we deﬁne the public ones as properties so if you see something like AttributeError: can't set attribute double you are not overriding these.

3.2 Test statuses

Avocado supports the most common exit statuses: • PASS - test passed, there were no untreated exceptions • WARN - a variant of PASS that keeps track of noteworthy events that ultimately do not affect the test outcome. An example could be soft lockup present in the dmesg output. It’s not related to the test results and unless there are failures in the test it means the feature probably works as expected, but there were certain condition which might be nice to review. (some result plugins does not support this and report PASS instead) • SKIP - the test’s pre-requisites were not satisﬁed and the test’s body was not executed (nor its setUp() and tearDown). • CANCEL - the test was canceled somewhere during the setUp(), the test method or the tearDown(). The setUp() and tearDown methods are executed. • FAIL - test did not result in the expected outcome. A failure points at a (possible) bug in the tested subject, and not in the test itself. When the test (and its) execution breaks, an ERROR and not a FAIL is reported.” • ERROR - this points (probably) at a bug in the test itself, and not in the subject being tested.It is usually caused by uncaught exception and such failures needs to be thoroughly explored and should lead to test modiﬁcation to avoid this failure or to use self.fail along with description how the subject under testing failed to perform it’s task. • INTERRUPTED - this result can’t be set by the test writer, it is only possible when the timeout is reached or when the user hits CTRL+C while executing this test. • other - there are some other internal test statuses, but you should not ever face them. As you can see the FAIL is a neat status, if tests are developed correctly. When writing tests always think about what its setUp should be, what the test body and is expected to go wrong in the test. To support you Avocado supports several methods:

16 Chapter 3. Writing Avocado Tests avocado Documentation, Release 63.0

3.2.1 Test methods

The simplest way to set the status is to use self.fail, self.error or self.cancel directly from test. To remember a warning, one simply writes to self.log.warning logger. This won’t interrupt the test execution, but it will remember the condition and, if there are no failures, will report the test as WARN.

3.2.2 Turning errors into failures

Errors on Python code are commonly signaled in the form of exceptions being thrown. When Avocado runs a test, any unhandled exception will be seen as a test ERROR, and not as a FAIL. Still, it’s common to rely on libraries, which usually raise custom (or builtin) exceptions. Those exceptions would normally result in ERROR but if you are certain this is an odd behavior of the object under testing, you should catch the exception and explain the failure in self.fail method:

try: process.run("stress_my_feature") except process.CmdError as details: self.fail("The stress comamnd failed: %s"% details)

If your test compounds of many executions and you can’t get this exception in other case then expected failure, you can simplify the code by using fail_on decorator:

@avocado.fail_on(process.CmdError) def test(self): process.run("first cmd") process.run("second cmd") process.run("third cmd")

Once again, keeping your tests up-to-date and distinguishing between FAIL and ERROR will save you a lot of time while reviewing the test results.

3.3 Saving test generated (custom) data

Each test instance provides a so called whiteboard. It can be accessed through self.whiteboard. This whiteboard is simply a string that will be automatically saved to test results after the test ﬁnishes (it’s not synced during the execution so when the machine or python crashes badly it might not be present and one should use direct io to the outputdir for critical data). If you choose to save binary data to the whiteboard, it’s your responsibility to encode it ﬁrst (base64 is the obvious choice). Building on the previously demonstrated sleeptest, suppose that you want to save the sleep length to be used by some other script or data analysis tool:

def test(self): sleep_length= self.params.get('sleep_length', default=1) self.log.debug("Sleeping for %.2f seconds", sleep_length) time.sleep(sleep_length) self.whiteboard=" %.2f"% sleep_length

The whiteboard can and should be exposed by files generated by the available test result plugins. The results. json file already includes the whiteboard for each test. Additionally, we’ll save a raw copy of the whiteboard contents on a file named whiteboard, in the same level as the results.json file, for your convenience (maybe you want to use the result of a benchmark directly with your custom made scripts to analyze that particular benchmark result).

3.3. Saving test generated (custom) data 17 avocado Documentation, Release 63.0

If you need to attach several output ﬁles, you can also use self.outputdir, which points to the $RESULTS/ test-results/$TEST_ID/data location and is reserved for arbitrary test result data.

3.4 Accessing test data ﬁles

Some tests can depend on data files, external to the test file itself. Avocado provides a test API that makes it really easy to access such files: get_data(). For Avocado tests (that is, INSTRUMENTED tests) get_data() allows test data files to be accessed from up to three sources: • file level data directory: a directory named after the test file, but ending with .data. For a test file /home/ user/test.py, the file level data directory is /home/user/test.py.data/. • test level data directory: a directory named after the test file and the specific test name. These are useful when different tests part of the same file need different data files (with the same name or not). Considering the previous example of /home/user/test.py, and supposing it contains two tests, MyTest.test_foo and MyTest.test_bar, the test level data directories will be, /home/user/test.py.data/MyTest. test_foo/ and home/user/test.py.data/MyTest.test_bar/ respectively. • variant level data directory: if variants are being used during the test execution, a directory named after the variant will also be considered when looking for test data files. For test file /home/user/test.py, and test MyTest.test_foo, with variant debug-ffff, the data directory path will be /home/user/test.py. data/MyTest.test_foo/debug-ffff/.

Note: Unlike INSTRUMENTED tests, SIMPLE tests only deﬁne file and variant data_dirs, therefore the most-speciﬁc data-dir might look like /bin/echo.data/debug-ffff/.

Avocado looks for data files in the order defined at DATA_SOURCES, which are from most specific one, to most generic one. That means that, if a variant is being used, the variant directory is used first. Then the test level directory is attempted, and finally the file level directory. Additionally you can use get_data(filename, must_exist=False) to get expected location of a possibly non-existing file, which is useful when you intend to create it.

Tip: When running tests you can use the --log-test-data-directories command line option log the test data directories that will be used for that speciﬁc test and execution conditions (such as with or without variants). Look for “Test data directories” in the test logs.

Note: The previously existing API avocado.core.test.Test.datadir, used to allow access to the data directory based on the test ﬁle location only. This API has been removed. If, for whatever reason you still need to access the data directory based on the test ﬁle location only, you can use get_data(filename='', source='file', must_exist=False) instead.

3.5 Accessing test parameters

Each test has a set of parameters that can be accessed through self.params.get($name, $path=None, $default=None) where: • name - name of the parameter (key)

18 Chapter 3. Writing Avocado Tests avocado Documentation, Release 63.0

• path - where to look for this parameter (when not speciﬁed uses mux-path) • default - what to return when param not found The path is a bit tricky. Avocado uses tree to represent parameters. In simple scenarios you don’t need to worry and you’ll ﬁnd all your values in default path, but eventually you might want to check-out Test parameters to understand the details. Let’s say your test receives following params (you’ll learn how to execute them in the following section):

$ avocado variants -m examples/tests/sleeptenmin.py.data/sleeptenmin.yaml --variants 2 ... Variant 1: /run/sleeptenmin/builtin, /run/variants/one_cycle /run/sleeptenmin/builtin:sleep_method => builtin /run/variants/one_cycle:sleep_cycles => 1 /run/variants/one_cycle:sleep_length => 600 ...

In test you can access those params by:

self.params.get("sleep_method") # returns "builtin" self.params.get("sleep_cycles",' *', 10) # returns 1 self.params.get("sleep_length","/ */variants/*" # returns 600

Note: The path is important in complex scenarios where clashes might occur, because when there are multiple values with the same key matching the query avocado raises an exception. As mentioned you can avoid those by using speciﬁc paths or by deﬁning custom mux-path which allows specifying resolving hierarchy. More details can be found in Test parameters.

3.6 Running multiple variants of tests

In the previous section we described how parameters are handled. Now, let’s have a look at how to produce them and execute your tests with different parameters. The variants subsystem is what allows the creation of multiple variations of parameters, and the execution of tests with those parameter variations. This subsystem is pluggable, so you might use custom plugins to produce variants. To keep things simple, let’s use Avocado’s primary implementation, called “yaml_to_mux”. The “yaml_to_mux” plugin accepts YAML ﬁles. Those will create a tree-like structure, store the variables as parameters and use custom tags to mark locations as “multiplex” domains. Let’s use examples/tests/sleeptenmin.py.data/sleeptenmin.yaml ﬁle as an example:

sleeptenmin: !mux builtin: sleep_method: builtin shell: sleep_method: shell variants: !mux one_cycle: sleep_cycles: 1 sleep_length: 600 six_cycles: sleep_cycles: 6 sleep_length: 100 (continues on next page)

3.6. Running multiple variants of tests 19 avocado Documentation, Release 63.0

(continued from previous page) one_hundred_cycles: sleep_cycles: 100 sleep_length: 6 six_hundred_cycles: sleep_cycles: 600 sleep_length: 1

Which produces following structure and parameters:

$ avocado variants -m examples/tests/sleeptenmin.py.data/sleeptenmin.yaml --summary 2

˓→--variants 2 Multiplex tree representation: run sleeptenmin builtin → sleep_method: builtin shell → sleep_method: shell variants one_cycle → sleep_length: 600 → sleep_cycles: 1 six_cycles → sleep_length: 100 → sleep_cycles: 6 one_hundred_cycles → sleep_length: 6 → sleep_cycles: 100 six_hundred_cycles → sleep_length: 1 → sleep_cycles: 600

Multiplex variants (8):

Variant builtin-one_cycle-f659: /run/sleeptenmin/builtin, /run/variants/one_cycle /run/sleeptenmin/builtin:sleep_method => builtin /run/variants/one_cycle:sleep_cycles => 1 /run/variants/one_cycle:sleep_length => 600

Variant builtin-six_cycles-723b: /run/sleeptenmin/builtin, /run/variants/six_cycles /run/sleeptenmin/builtin:sleep_method => builtin /run/variants/six_cycles:sleep_cycles => 6 /run/variants/six_cycles:sleep_length => 100

Variant builtin-one_hundred_cycles-633a: /run/sleeptenmin/builtin, /run/variants/

˓→one_hundred_cycles /run/sleeptenmin/builtin:sleep_method => builtin /run/variants/one_hundred_cycles:sleep_cycles => 100 /run/variants/one_hundred_cycles:sleep_length => 6

Variant builtin-six_hundred_cycles-a570: /run/sleeptenmin/builtin, /run/variants/

˓→six_hundred_cycles /run/sleeptenmin/builtin:sleep_method => builtin /run/variants/six_hundred_cycles:sleep_cycles => 600 /run/variants/six_hundred_cycles:sleep_length => 1

Variant shell-one_cycle-55f5: /run/sleeptenmin/shell, /run/variants/one_cycle (continues on next page)

20 Chapter 3. Writing Avocado Tests avocado Documentation, Release 63.0

(continued from previous page) /run/sleeptenmin/shell:sleep_method => shell /run/variants/one_cycle:sleep_cycles => 1 /run/variants/one_cycle:sleep_length => 600

Variant shell-six_cycles-9e23: /run/sleeptenmin/shell, /run/variants/six_cycles /run/sleeptenmin/shell:sleep_method => shell /run/variants/six_cycles:sleep_cycles => 6 /run/variants/six_cycles:sleep_length => 100

Variant shell-one_hundred_cycles-586f: /run/sleeptenmin/shell, /run/variants/one_

˓→hundred_cycles /run/sleeptenmin/shell:sleep_method => shell /run/variants/one_hundred_cycles:sleep_cycles => 100 /run/variants/one_hundred_cycles:sleep_length => 6

Variant shell-six_hundred_cycles-1e84: /run/sleeptenmin/shell, /run/variants/six_

˓→hundred_cycles /run/sleeptenmin/shell:sleep_method => shell /run/variants/six_hundred_cycles:sleep_cycles => 600 /run/variants/six_hundred_cycles:sleep_length => 1

You can see that it creates all possible variants of each multiplex domain, which are deﬁned by !mux tag in the YAML ﬁle and displayed as single lines in tree view (compare to double lines which are individual nodes with values). In total it’ll produce 8 variants of each test:

$ avocado run --mux-yaml examples/tests/sleeptenmin.py.data/sleeptenmin.yaml --

˓→passtest.py JOB ID : cc7ef22654c683b73174af6f97bc385da5a0f02f JOB LOG : /home/medic/avocado/job-results/job-2017-01-22T11.26-cc7ef22/job.log (1/8) passtest.py:PassTest.test;builtin-one_cycle-f659: PASS (0.01 s) (2/8) passtest.py:PassTest.test;builtin-six_cycles-723b: PASS (0.01 s) (3/8) passtest.py:PassTest.test;builtin-one_hundred_cycles-633a: PASS (0.01 s) (4/8) passtest.py:PassTest.test;builtin-six_hundred_cycles-a570: PASS (0.01 s) (5/8) passtest.py:PassTest.test;shell-one_cycle-55f5: PASS (0.01 s) (6/8) passtest.py:PassTest.test;shell-six_cycles-9e23: PASS (0.01 s) (7/8) passtest.py:PassTest.test;shell-one_hundred_cycles-586f: PASS (0.01 s) (8/8) passtest.py:PassTest.test;shell-six_hundred_cycles-1e84: PASS (0.01 s) RESULTS : PASS 8 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 JOB TIME : 0.16 s

There are other options to inﬂuence the params so please check out avocado run -h and for details use Test parameters.

3.7 Advanced logging capabilities

Avocado provides advanced logging capabilities at test run time. These can be combined with the standard Python library APIs on tests. One common example is the need to follow speciﬁc progress on longer or more complex tests. Let’s look at a very simple test example, but one multiple clear stages on a single test: import logging import time

(continues on next page)

3.7. Advanced logging capabilities 21 avocado Documentation, Release 63.0

(continued from previous page) from avocado import Test progress_log= logging.getLogger("progress") class Plant(Test):

def test_plant_organic(self): rows= self.params.get("rows", default=3)

# Preparing soil for row in range(rows): progress_log.info("%s: preparing soil on row %s", self.name, row)

# Letting soil rest progress_log.info("%s: letting soil rest before throwing seeds", self.name) time.sleep(2)

# Throwing seeds for row in range(rows): progress_log.info("%s: throwing seeds on row %s", self.name, row)

# Let them grow progress_log.info("%s: waiting for Avocados to grow", self.name) time.sleep(5)

# Harvest them for row in range(rows): progress_log.info("%s: harvesting organic avocados on row %s", self.name, row)

From this point on, you can ask Avocado to show your logging stream, either exclusively or in addition to other builtin streams:

$ avocado --show app,progress run plant.py

The outcome should be similar to:

JOB ID : af786f86db530bff26cd6a92c36e99bedcdca95b JOB LOG :/home/cleber/avocado/job-results/job-2016-03-18T10.29-af786f8/job.log (1/1) plant.py:Plant.test_plant_organic: progress:1-plant.py:Plant.test_plant_

˓→organic: preparing soil on row0 progress:1-plant.py:Plant.test_plant_organic: preparing soil on row1 progress:1-plant.py:Plant.test_plant_organic: preparing soil on row2 progress:1-plant.py:Plant.test_plant_organic: letting soil rest before throwing seeds -progress:1-plant.py:Plant.test_plant_organic: throwing seeds on row0 progress:1-plant.py:Plant.test_plant_organic: throwing seeds on row1 progress:1-plant.py:Plant.test_plant_organic: throwing seeds on row2 progress:1-plant.py:Plant.test_plant_organic: waiting for Avocados to grow \progress:1-plant.py:Plant.test_plant_organic: harvesting organic avocados on row0 progress:1-plant.py:Plant.test_plant_organic: harvesting organic avocados on row1 progress:1-plant.py:Plant.test_plant_organic: harvesting organic avocados on row2 PASS (7.01 s) (continues on next page)

22 Chapter 3. Writing Avocado Tests avocado Documentation, Release 63.0

˓→results.html

The custom progress stream is combined with the application output, which may or may not suit your needs or preferences. If you want the progress stream to be sent to a separate ﬁle, both for clarity and for persistence, you can run Avocado like this:

$ avocado run plant.py --store-logging-stream progress

The result is that, besides all the other log ﬁles commonly generated, there will be another log ﬁle named progress. INFO at the job results dir. During the test run, one could watch the progress with:

$ tail -f ~/avocado/job-results/latest/progress.INFO 10:36:59 INFO | 1-plant.py:Plant.test_plant_organic: preparing soil on row 0 10:36:59 INFO | 1-plant.py:Plant.test_plant_organic: preparing soil on row 1 10:36:59 INFO | 1-plant.py:Plant.test_plant_organic: preparing soil on row 2 10:36:59 INFO | 1-plant.py:Plant.test_plant_organic: letting soil rest before

˓→throwing seeds 10:37:01 INFO | 1-plant.py:Plant.test_plant_organic: throwing seeds on row 0 10:37:01 INFO | 1-plant.py:Plant.test_plant_organic: throwing seeds on row 1 10:37:01 INFO | 1-plant.py:Plant.test_plant_organic: throwing seeds on row 2 10:37:01 INFO | 1-plant.py:Plant.test_plant_organic: waiting for Avocados to grow 10:37:06 INFO | 1-plant.py:Plant.test_plant_organic: harvesting organic avocados on

˓→row 0 10:37:06 INFO | 1-plant.py:Plant.test_plant_organic: harvesting organic avocados on

˓→row 1 10:37:06 INFO | 1-plant.py:Plant.test_plant_organic: harvesting organic avocados on

˓→row 2

The very same progress logger, could be used across multiple test methods and across multiple test modules. In the example given, the test name is used to give extra context.

3.8 unittest.TestCase heritage

Since an Avocado test inherits from unittest.TestCase, you can use all the assertion methods that its parent. The code example bellow uses assertEqual, assertTrue and assertIsInstace: from avocado import Test class RandomExamples(Test): def test(self): self.log.debug("Verifying some random math...") four=2 * 2 four_=2+2 self.assertEqual(four, four_,"something is very wrong here!")

self.log.debug("Verifying if a variable is set to True...") variable= True self.assertTrue(variable)

self.log.debug("Verifying if this test is an instance of test.Test") self.assertIsInstance(self, test.Test)

3.8. unittest.TestCase heritage 23 avocado Documentation, Release 63.0

3.8.1 Running tests under other unittest runners nose is another Python testing framework that is also compatible with unittest. Because of that, you can run avocado tests with the nosetests application:

$ nosetests examples/tests/sleeptest.py . ------Ran 1 test in 1.004s

Conversely, you can also use the standard unittest.main() entry point to run an Avocado test. Check out the following code, to be saved as dummy.py: from avocado import Test from unittest import main class Dummy(Test): def test(self): self.assertTrue(True) if __name__ =='__main__': main()

It can be run by:

$ python dummy.py . ------Ran 1 test in 0.000s

But we’d still recommend using avocado.main instead which is our main entry point.

3.9 Setup and cleanup methods

To perform setup actions before/after your test, you may use setUp and tearDown methods. The tearDown method is always executed even on setUp failure so don’t forget to initialize your variables early in the setUp. Example of usage is in the next section Running third party test suites.

3.10 Running third party test suites

It is very common in test automation workloads to use test suites developed by third parties. By wrapping the execution code inside an Avocado test module, you gain access to the facilities and API provided by the framework. Let’s say you want to pick up a test suite written in C that it is in a tarball, uncompress it, compile the suite code, and then executing the test. Here’s an example that does that:

#!/usr/bin/env python import os (continues on next page)

24 Chapter 3. Writing Avocado Tests avocado Documentation, Release 63.0

(continued from previous page)

from avocado import Test from avocado import main from avocado.utils import archive from avocado.utils import build from avocado.utils import process

class SyncTest(Test):

""" Execute the synctest test suite. """ def setUp(self): """ Set default params and build the synctest suite. """ sync_tarball= self.params.get('sync_tarball', default='synctest.tar.bz2') self.sync_length= self.params.get('sync_length', default=100) self.sync_loop= self.params.get('sync_loop', default=10) # Build the synctest suite self.cwd= os.getcwd() tarball_path= self.get_data(sync_tarball) archive.extract(tarball_path, self.workdir) self.workdir= os.path.join(self.workdir,'synctest') build.make(self.workdir)

def test(self): """ Execute synctest with the appropriate params. """ os.chdir(self.workdir) cmd=('./synctest %s %s'% (self.sync_length, self.sync_loop)) process.system(cmd) os.chdir(self.cwd)

if __name__ =="__main__": main()

Here we have an example of the setUp method in action: Here we get the location of the test suite code (tarball) through avocado.Test.get_data(), then uncompress the tarball through avocado.utils.archive. extract(), an API that will decompress the suite tarball, followed by avocado.utils.build.make(), that will build the suite. In this example, the test method just gets into the base directory of the compiled suite and executes the ./ synctest command, with appropriate parameters, using avocado.utils.process.system().

3.11 Fetching asset ﬁles

To run third party test suites as mentioned above, or for any other purpose, we offer an asset fetcher as a method of Avocado Test class. The asset method looks for a list of directories in the cache_dirs key, inside the [datadir. paths] section from the configuration files. Read-only directories are also supported. When the asset file is not

3.11. Fetching asset ﬁles 25 avocado Documentation, Release 63.0

present in any of the provided directories, we will try to download the ﬁle from the provided locations, copying it to the ﬁrst writable cache directory. Example:

cache_dirs=['/usr/local/src/','~/avocado/cache']

In the example above, /usr/local/src/ is a read-only directory. In that case, when we need to fetch the asset from the locations, it will be copied to the ~/avocado/cache directory. If you don’t provide a cache_dirs, we will create a cache directory inside the avocado data_dir location to put the fetched files in. • Use case 1: no cache_dirs key in config files, only the asset name provided in the full url format:

... def setUp(self): stress='http://people.seas.harvard.edu/~apw/stress/stress-1.0.4.tar.gz' tarball= self.fetch_asset(stress) archive.extract(tarball, self.workdir) ...

In this case, fetch_asset() will download the ﬁle from the url provided, copying it to the $data_dir/ cache directory. tarball variable will contains, for example, /home/user/avocado/data/cache/ stress-1.0.4.tar.gz. • Use case 2: Read-only cache directory provided. cache_dirs = ['/mnt/files']:

... def setUp(self): stress='http://people.seas.harvard.edu/~apw/stress/stress-1.0.4.tar.gz' tarball= self.fetch_asset(stress) archive.extract(tarball, self.workdir) ...

In this case, we try to find stress-1.0.4.tar.gz file in /mnt/files directory. If it’s not there, since /mnt/files is read-only, we will try to download the asset file to the $data_dir/cache directory. • Use case 3: Writable cache directory provided, along with a list of locations. cache_dirs = ['~/ avocado/cache']:

... def setUp(self): st_name='stress-1.0.4.tar.gz' st_hash='e1533bc704928ba6e26a362452e6db8fd58b1f0b' st_loc=['http://people.seas.harvard.edu/~apw/stress/stress-1.0.4.tar.gz

˓→', 'ftp://foo.bar/stress-1.0.4.tar.gz'] tarball= self.fetch_asset(st_name, asset_hash=st_hash, locations=st_loc) archive.extract(tarball, self.workdir) ...

In this case, we try to download stress-1.0.4.tar.gz from the provided locations list (if it’s not already in ~/avocado/cache). The hash was also provided, so we will verify the hash. To do so, we first look for a hashfile named stress-1.0.4.tar.gz.sha1 in the same directory. If the hashfile is not present we compute the hash and create the hashfile for further usage. The resulting tarball variable content will be ~/avocado/cache/stress-1.0.4.tar.gz. An exception will take place if we fail to download or to verify the file. Detailing the fetch_asset() attributes:

26 Chapter 3. Writing Avocado Tests avocado Documentation, Release 63.0

• name: The name used to name the fetched file. It can also contains a full URL, that will be used as the first location to try (after serching into the cache directories). • asset_hash: (optional) The expected file hash. If missing, we skip the check. If provided, before computing the hash, we look for a hashfile to verify the asset. If the hashfile is nor present, we compute the hash and create the hashfile in the same cache directory for further usage. • algorithm: (optional) Provided hash algorithm format. Defaults to sha1. • locations: (optional) List of locations that will be used to try to fetch the file from. The supported schemes are http://, https://, ftp:// and file://. You’re required to inform the full url to the file, including the file name. The first success will skip the next locations. Notice that for file:// we just create a symbolic link in the cache directory, pointing to the file original location. • expire: (optional) time period that the cached file will be considered valid. After that period, the file will be dowloaded again. The value can be an integer or a string containing the time and the unit. Example: ‘10d’ (ten days). Valid units are s (second), m (minute), h (hour) and d (day). The expected return is the asset file path or an exception.

3.12 Test Output Check and Output Record Mode

In a lot of occasions, you want to go simpler: just check if the output of a given test matches an expected output. In order to help with this common use case, Avocado provides the --output-check-record option:

--output-check-record {none,stdout,stderr,both,combined,all} Record the output produced by each test (from stdout and stderr) into both the current executing result and into reference files. Reference files are used on subsequent runs to determine if the test produced the expected output or not, and the current executing result is used to check against a previously recorded reference file. Valid values: 'none' (to explicitly disable all recording) 'stdout' (to record standard output *only*), 'stderr' (to record standard error *only*), 'both' (to record standard output and error in separate files), 'combined' (for standard output and error in a single file). 'all' is also a valid but deprecated option that is a synonym of 'both'. This option does not have a default value, but the Avocado test runner will record the test under execution in the most suitable way unless it's explicitly disabled with value 'none'

If this option is used, Avocado will store the content generated by the test in the standard (POSIX) streams, that is, STDOUT and STDERR. Depending on the option chosen, you may end up with different files recorded (into what we call “reference files”): • stdout will produce a file named stdout.expected with the contents from the test process standard output stream (file descriptor 1) • stderr will produce a file named stderr.expected with the contents from the test process standard error stream (file descriptor 2) • both will produce both a file named stdout.expected and a file named stderr.expected • combined: will produce a single file named output.expected, with the content from both test process standard output and error streams (file descriptors 1 and 2)

3.12. Test Output Check and Output Record Mode 27 avocado Documentation, Release 63.0

• none will explicitly disable all recording of test generated output and the generation reference files with that content The reference files will be recorded in the first (most specific) test’s data dir (Accessing test data files). Let’s take as an example the test synctest.py. In a fresh checkout of the Avocado source code you can find the following reference files:

examples/tests/synctest.py.data/stderr.expected examples/tests/synctest.py.data/stdout.expected

From those 2 ﬁles, only stdout.expected has some content:

$ cat examples/tests/synctest.py.data/stdout.expected PAR : waiting PASS : sync interrupted

This means that during a previous test execution, output was recorded with option --output-check-record both and content was generated on the STDOUT stream only:

$ avocado run --output-check-record both synctest.py JOB ID : b6306504351b037fa304885c0baa923710f34f4a JOB LOG : $JOB_RESULTS_DIR/job-2017-11-26T16.42-b630650/job.log (1/1) examples/tests/synctest.py:SyncTest.test: PASS (2.03 s) RESULTS : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0 JOB TIME : 2.26 s

After the reference files are added, the check process is transparent, in the sense that you do not need to provide special flags to the test runner. From this point on, after such as test (one with a reference file recorded) has finished running, Avocado will check if the output generated match the reference(s) file(s) content. If they don’t match, the test will finish with a FAIL status. You can disable this automatic check when a reference file exists by passing --output-check=off to the test runner.

Tip: The avocado.utils.process APIs have a parameter called allow_output_check that let you individually select the output that will be part of the test output and recorded reference ﬁles. Some other APIs built on top of avocado.utils.process, such as the ones in avocado.utils.build also provide the same parameter.

This process works ﬁne also with simple tests, which are programs or shell scripts that returns 0 (PASSed) or != 0 (FAILed). Let’s consider our bogus example:

$ cat output_record.sh #!/bin/bash echo "Hello, world!"

Let’s record the output for this one:

$ scripts/avocado run output_record.sh --output-check-record all JOB ID : 25c4244dda71d0570b7f849319cd71fe1722be8b JOB LOG : $HOME/avocado/job-results/job-2014-09-25T20.49-25c4244/job.log (1/1) output_record.sh: PASS (0.01 s) RESULTS : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 JOB TIME : 0.11 s

After this is done, you’ll notice that a the test data directory appeared in the same level of our shell script, containing 2 ﬁles:

28 Chapter 3. Writing Avocado Tests avocado Documentation, Release 63.0

$ ls output_record.sh.data/ stderr.expected stdout.expected

Let’s look what’s in each of them:

$ cat output_record.sh.data/stdout.expected Hello, world! $ cat output_record.sh.data/stderr.expected $

Now, every time this test runs, it’ll take into account the expected ﬁles that were recorded, no need to do anything else but run the test. Let’s see what happens if we change the stdout.expected ﬁle contents to Hello, Avocado!:

$ scripts/avocado run output_record.sh JOB ID : f0521e524face93019d7cb99c5765aedd933cb2e JOB LOG : $HOME/avocado/job-results/job-2014-09-25T20.52-f0521e5/job.log (1/1) output_record.sh: FAIL (0.02 s) RESULTS : PASS 0 | ERROR 0 | FAIL 1 | SKIP 0 | WARN 0 | INTERRUPT 0 JOB TIME : 0.12 s

Verifying the failure reason:

$ cat $HOME/avocado/job-results/latest/job.log 2017-10-16 14:23:02,567 test L0381 INFO | START 1-output_record.sh 2017-10-16 14:23:02,568 test L0402 DEBUG| Test metadata: 2017-10-16 14:23:02,568 test L0403 DEBUG| filename: $HOME/output_

˓→record.sh 2017-10-16 14:23:02,596 process L0389 INFO | Running '$HOME/output_

˓→record.sh' 2017-10-16 14:23:02,603 process L0499 INFO | Command '$HOME/output_

˓→record.sh' finished with 0 after 0.00131011009216s 2017-10-16 14:23:02,602 process L0479 DEBUG| [stdout] Hello, world! 2017-10-16 14:23:02,603 test L1084 INFO | Exit status: 0 2017-10-16 14:23:02,604 test L1085 INFO | Duration: 0.00131011009216 2017-10-16 14:23:02,604 test L0274 DEBUG| DATA (filename=stdout.

˓→expected) => $HOME/output_record.sh.data/stdout.expected (found at file source dir) 2017-10-16 14:23:02,605 test L0740 DEBUG| Stdout Diff: 2017-10-16 14:23:02,605 test L0742 DEBUG| --- $HOME/output_record.sh.

˓→data/stdout.expected 2017-10-16 14:23:02,605 test L0742 DEBUG| +++ $HOME/avocado/job-

˓→results/job-2017-10-16T14.23-8cba866/test-results/1-output_record.sh/stdout 2017-10-16 14:23:02,605 test L0742 DEBUG| @@ -1 +1 @@ 2017-10-16 14:23:02,605 test L0742 DEBUG| -Hello, Avocado! 2017-10-16 14:23:02,605 test L0742 DEBUG| +Hello, world! 2017-10-16 14:23:02,606 stacktrace L0041 ERROR| 2017-10-16 14:23:02,606 stacktrace L0044 ERROR| Reproduced traceback from:

˓→$HOME/git/avocado/avocado/core/test.py:872 2017-10-16 14:23:02,606 stacktrace L0047 ERROR| Traceback (most recent call

˓→last): 2017-10-16 14:23:02,606 stacktrace L0047 ERROR| File "$HOME/git/avocado/

˓→avocado/core/test.py", line 743, in _check_reference_stdout 2017-10-16 14:23:02,606 stacktrace L0047 ERROR| self.fail('Actual test

˓→sdtout differs from expected one') 2017-10-16 14:23:02,606 stacktrace L0047 ERROR| File "$HOME//git/avocado/

˓→avocado/core/test.py", line 983, in fail 2017-10-16 14:23:02,607 stacktrace L0047 ERROR| raise exceptions.

˓→TestFail(message) (continues on next page)

3.12. Test Output Check and Output Record Mode 29 avocado Documentation, Release 63.0

(continued from previous page) 2017-10-16 14:23:02,607 stacktrace L0047 ERROR| TestFail: Actual test

˓→sdtout differs from expected one 2017-10-16 14:23:02,607 stacktrace L0048 ERROR| 2017-10-16 14:23:02,607 test L0274 DEBUG| DATA (filename=stderr.

˓→expected) => $HOME//output_record.sh.data/stderr.expected (found at file source dir) 2017-10-16 14:23:02,608 test L0965 ERROR| FAIL 1-output_record.sh ->

˓→TestFail: Actual test sdtout differs from expected one

As expected, the test failed because we changed its expectations, so an unified diff was logged. The unified diffs are also present in the files stdout.diff and stderr.diff, present in the test results directory:

$ cat $HOME/avocado/job-results/latest/test-results/1-output_record.sh/stdout.diff --- $HOME/output_record.sh.data/stdout.expected +++ $HOME/avocado/job-results/job-2017-10-16T14.23-8cba866/test-results/1-output_

˓→record.sh/stdout @@ -1 +1 @@ -Hello, Avocado! +Hello, world!

Note: Currently the stdout, stderr and output ﬁles are stored in text mode. Data that can not be decoded according to current locale settings, will be replaced according to https://docs.python.org/3/library/codecs.html#codecs.replace_ errors.

3.13 Test log, stdout and stderr in native Avocado modules

If needed, you can write directly to the expected stdout and stderr ﬁles from the native test scope. It is important to make the distinction between the following entities: • The test logs • The test expected stdout • The test expected stderr The ﬁrst one is used for debugging and informational purposes. Additionally writing to self.log.warning causes test to be marked as dirty and when everything else goes well the test ends with WARN. This means that the test passed but there were non-related unexpected situations described in warning log. You may log something into the test logs using the methods in avocado.Test.log class attributes. Consider the example: class output_test(Test):

def test(self): self.log.info('This goes to the log and it is only informational') self.log.warn('Oh, something unexpected, non-critical happened,' 'but we can continue.') self.log.error('Describe the error here and don't forget to raise ' 'an exception yourself. Writing to self.log.error' 'won't do that for you.') self.log.debug('Everybody look, I had a good lunch today...')

If you need to write directly to the test stdout and stderr streams, Avocado makes two preconﬁgured loggers available for that purpose, named avocado.test.stdout and avocado.test.stderr. You can use Python’s standard

30 Chapter 3. Writing Avocado Tests avocado Documentation, Release 63.0 logging API to write to them. Example: import logging class output_test(Test):

def test(self): stdout= logging.getLogger('avocado.test.stdout') stdout.info('Informational line that will go to stdout') ... stderr= logging.getLogger('avocado.test.stderr') stderr.info('Informational line that will go to stderr')

Avocado will automatically save anything a test generates on STDOUT into a stdout file, to be found at the test results directory. The same applies to anything a test generates on STDERR, that is, it will be saved into a stderr file at the same location. Additionally, when using the runner’s output recording features, namely the --output-check-record argument with values stdout, stderr or all, everything given to those loggers will be saved to the files stdout. expected and stderr.expected at the test’s data directory (which is different from the job/test results directory).

3.14 Setting a Test Timeout

Sometimes your test suite/test might get stuck forever, and this might impact your test grid. You can account for that possibility and set up a timeout parameter for your test. The test timeout can be set through the test parameters, as shown below. sleep_length:5 timeout:3

$ avocado run sleeptest.py --mux-yaml /tmp/sleeptest-example.yaml JOB ID : c78464bde9072a0b5601157989a99f0ba32a288e JOB LOG : $HOME/avocado/job-results/job-2016-11-02T11.13-c78464b/job.log (1/1) sleeptest.py:SleepTest.test: INTERRUPTED (3.04 s) RESULTS : PASS 0 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 1 JOB TIME : 3.14 s JOB HTML : $HOME/avocado/job-results/job-2016-11-02T11.13-c78464b/html/results.html

$ cat $HOME/avocado/job-results/job-2016-11-02T11.13-c78464b/job.log 2016-11-02 11:13:01,133 job L0384 INFO | Multiplex tree representation: 2016-11-02 11:13:01,133 job L0386 INFO | \-- run 2016-11-02 11:13:01,133 job L0386 INFO | -> sleep_length: 5 2016-11-02 11:13:01,133 job L0386 INFO | -> timeout: 3 2016-11-02 11:13:01,133 job L0387 INFO | 2016-11-02 11:13:01,134 job L0391 INFO | Temporary dir: /var/tmp/avocado_

˓→PqDEyC 2016-11-02 11:13:01,134 job L0392 INFO | 2016-11-02 11:13:01,134 job L0399 INFO | Variant 1: /run 2016-11-02 11:13:01,134 job L0402 INFO | 2016-11-02 11:13:01,134 job L0311 INFO | Job ID:

˓→c78464bde9072a0b5601157989a99f0ba32a288e 2016-11-02 11:13:01,134 job L0314 INFO | 2016-11-02 11:13:01,345 sysinfo L0107 DEBUG| Not logging /proc/pci (file

˓→does not exist) (continues on next page)

3.14. Setting a Test Timeout 31 avocado Documentation, Release 63.0

(continued from previous page) 2016-11-02 11:13:01,351 sysinfo L0105 DEBUG| Not logging /proc/slabinfo

˓→(lack of permissions) 2016-11-02 11:13:01,355 sysinfo L0107 DEBUG| Not logging /sys/kernel/debug/

˓→sched_features (file does not exist) 2016-11-02 11:13:01,388 sysinfo L0388 INFO | Commands configured by file: /

˓→etc/avocado/sysinfo/commands 2016-11-02 11:13:01,388 sysinfo L0399 INFO | Files configured by file: /etc/

˓→avocado/sysinfo/files 2016-11-02 11:13:01,388 sysinfo L0419 INFO | Profilers configured by file: /

˓→etc/avocado/sysinfo/profilers 2016-11-02 11:13:01,388 sysinfo L0427 INFO | Profiler disabled 2016-11-02 11:13:01,394 multiplexer L0166 DEBUG| PARAMS (key=timeout, path=*, ˓→default=None) => 3 2016-11-02 11:13:01,395 test L0216 INFO | START 1-sleeptest.py:SleepTest.

˓→test 2016-11-02 11:13:01,396 multiplexer L0166 DEBUG| PARAMS (key=sleep_length, ˓→path=*, default=1) => 5 2016-11-02 11:13:01,396 sleeptest L0022 DEBUG| Sleeping for 5.00 seconds 2016-11-02 11:13:04,411 stacktrace L0038 ERROR| 2016-11-02 11:13:04,412 stacktrace L0041 ERROR| Reproduced traceback from:

˓→$HOME/src/avocado/avocado/core/test.py:454 2016-11-02 11:13:04,412 stacktrace L0044 ERROR| Traceback (most recent call

˓→last): 2016-11-02 11:13:04,413 stacktrace L0044 ERROR| File "/usr/share/doc/avocado/

˓→tests/sleeptest.py", line 23, in test 2016-11-02 11:13:04,413 stacktrace L0044 ERROR| time.sleep(sleep_length) 2016-11-02 11:13:04,413 stacktrace L0044 ERROR| File "$HOME/src/avocado/

˓→avocado/core/runner.py", line 293, in sigterm_handler 2016-11-02 11:13:04,413 stacktrace L0044 ERROR| raise SystemExit("Test

˓→interrupted by SIGTERM") 2016-11-02 11:13:04,414 stacktrace L0044 ERROR| SystemExit: Test interrupted by

˓→SIGTERM 2016-11-02 11:13:04,414 stacktrace L0045 ERROR| 2016-11-02 11:13:04,414 test L0459 DEBUG| Local variables: 2016-11-02 11:13:04,440 test L0462 DEBUG| -> self

˓→SleepTest'>: 1-sleeptest.py:SleepTest.test 2016-11-02 11:13:04,440 test L0462 DEBUG| -> sleep_length : 5 2016-11-02 11:13:04,440 test L0592 ERROR| ERROR 1-sleeptest.py:SleepTest.

˓→test -> TestError: SystemExit('Test interrupted by SIGTERM',): Test interrupted by

˓→SIGTERM

The YAML ﬁle deﬁnes a test parameter timeout which overrides the default test timeout before the runner ends the test forcefully by sending a class:signal.SIGTERM to the test, making it raise a avocado.core.exceptions. TestTimeoutError.

3.15 Skipping Tests

To skip tests is in Avocado, you must use one of the Avocado skip decorators: • @avocado.skip(reason): Skips a test. • @avocado.skipIf(condition, reason): Skips a test if the condition is True. • @avocado.skipUnless(condition, reason): Skips a test if the condition is False Those decorators can be used with both setUp() method and/or and in the test*() methods. The test below:

32 Chapter 3. Writing Avocado Tests avocado Documentation, Release 63.0

import avocado class MyTest(avocado.Test):

@avocado.skipIf(1 ==1,'Skipping on True condition.') def test1(self): pass

@avocado.skip("Don't want this test now.") def test2(self): pass

@avocado.skipUnless(1 ==1,'Skipping on False condition.') def test3(self): pass

Will produce the following result:

$ avocado run test_skip_decorators.py JOB ID : 59c815f6a42269daeaf1e5b93e52269fb8a78119 JOB LOG : $HOME/avocado/job-results/job-2017-02-03T17.41-59c815f/job.log (1/3) test_skip_decorators.py:MyTest.test1: SKIP (2/3) test_skip_decorators.py:MyTest.test2: SKIP (3/3) test_skip_decorators.py:MyTest.test3: PASS (0.02 s) RESULTS : PASS 1 | ERROR 0 | FAIL 0 | SKIP 2 | WARN 0 | INTERRUPT 0 JOB TIME : 0.13 s JOB HTML : $HOME/avocado/job-results/job-2017-02-03T17.41-59c815f/html/results.html

Notice the test3 was not skipped because the provided condition was not False. Using the skip decorators, nothing is actually executed. We will skip the setUp() method, the test method and the tearDown() method.

Note: It’s an erroneous condition, reported with test status ERROR, to use any of the skip decorators on the tearDown() method.

3.16 Cancelling Tests

You can cancel a test calling self.cancel() at any phase of the test (setUp(), test method or tearDown()). Test will ﬁnish with CANCEL status and will not make the Job to exit with a non-0 status. Example:

#!/usr/bin/env python from avocado import Test from avocado import main from avocado.utils.process import run from avocado.utils.software_manager import SoftwareManager class CancelTest(Test):

""" (continues on next page)

3.16. Cancelling Tests 33 avocado Documentation, Release 63.0

(continued from previous page) Example tests that cancel the current test from inside the test. """

def setUp(self): sm= SoftwareManager() self.pkgs= sm.list_all(software_components= False)

def test_iperf(self): if 'iperf-2.0.8-6.fc25.x86_64' not in self.pkgs: self.cancel('iperf is not installed or wrong version') self.assertIn('pthreads', run('iperf -v', ignore_status=True).stderr)

def test_gcc(self): if 'gcc-6.3.1-1.fc25.x86_64' not in self.pkgs: self.cancel('gcc is not installed or wrong version') self.assertIn('enable-gnu-indirect-function', run('gcc -v', ignore_status=True).stderr) if __name__ =="__main__": main()

In a system missing the iperf package but with gcc installed in the correct version, the result will be:

JOB ID : 39c1f120830b9769b42f5f70b6b7bad0b1b1f09f JOB LOG : $HOME/avocado/job-results/job-2017-03-10T16.22-39c1f12/job.log (1/2) /home/apahim/avocado/tests/test_cancel.py:CancelTest.test_iperf: CANCEL (1.15

Notice that using the self.cancel() will cancel the rest of the test from that point on, but the tearDown() will still be executed. Depending on the result format you’re referring to, the CANCEL status is mapped to a corresponding valid status in that format. See the table below:

Format Corresponding Status json cancel xunit skipped tap ok html CANCEL (warning)

3.17 Docstring Directives

Some Avocado features, usually only available to instrumented tests, depend on setting directives on the test’s class docstring. A docstring directive is composed of a marker (a literal :avocado: string), followed by the custom content itself, such as :avocado: directive. This is similar to docstring directives such as :param my_param: description and shouldn’t be a surprise to most Python developers.

34 Chapter 3. Writing Avocado Tests avocado Documentation, Release 63.0

The reason Avocado uses those docstring directives (instead of real Python code) is that the inspection done while looking for tests does not involve any execution of code. For a detailed explanation about what makes a docstring format valid or not, please refer to our section on Docstring Directives Rules. Now let’s follow with some docstring directives examples.

3.17.1 Explicitly enabling or disabling tests

If your test is a method in a class that directly inherits from avocado.Test, then Avocado will find it as one would expect. Now, the need may arise for more complex tests, to use more advanced Python features such as inheritance. For those tests that are written in a class not directly inherting from avocado.Test, Avocado may need your help, because Avocado uses only static analysis to examine the files. For example, suppose that you define a new test class that inherits from the Avocado base test class, that is, avocado. Test, and put it in mylibrary.py: from avocado import Test class MyOwnDerivedTest(Test): def __init__(self, methodName='test', name=None, params=None, base_logdir=None, job=None, runner_queue=None): super(MyOwnDerivedTest, self).__init__(methodName, name, params, base_logdir, job, runner_queue) self.log('Derived class example')

Then you implement your actual test using that derived class, in mytest.py: import mylibrary class MyTest(mylibrary.MyOwnDerivedTest):

def test1(self): self.log('Testing something important')

def test2(self): self.log('Testing something even more important')

If you try to list the tests in that ﬁle, this is what you’ll get: scripts/avocado list mytest.py -V Type Test Tag(s) NOT_A_TEST mytest.py

TEST TYPES SUMMARY ======ACCESS_DENIED: 0 BROKEN_SYMLINK: 0 EXTERNAL: 0 FILTERED: 0 INSTRUMENTED: 0 (continues on next page)

3.17. Docstring Directives 35 avocado Documentation, Release 63.0

(continued from previous page) MISSING: 0 NOT_A_TEST: 1 SIMPLE: 0 VT: 0

You need to give avocado a little help by adding a docstring directive. That docstring directive is :avocado: enable. It tells the Avocado safe test detection code to consider it as an avocado test, regardless of what the (admit- tedly simple) detection code thinks of it. Let’s see how that works out. Add the docstring, as you can see the example below: import mylibrary class MyTest(mylibrary.MyOwnDerivedTest): """ :avocado: enable """ def test1(self): self.log('Testing something important')

def test2(self): self.log('Testing something even more important')

Now, trying to list the tests on the mytest.py ﬁle again: scripts/avocado list mytest.py -V Type Test Tag(s) INSTRUMENTED mytest.py:MyTest.test1 INSTRUMENTED mytest.py:MyTest.test2

TEST TYPES SUMMARY ======ACCESS_DENIED: 0 BROKEN_SYMLINK: 0 EXTERNAL: 0 FILTERED: 0 INSTRUMENTED: 2 MISSING: 0 NOT_A_TEST: 0 SIMPLE: 0 VT: 0

You can also use the :avocado: disable docstring directive, that works the opposite way: something that would be considered an Avocado test, but we force it to not be listed as one. The docstring :avocado: disable is evaluated ﬁrst by Avocado, meaning that if both :avocado: disable and :avocado: enable are present in the same docstring, the test will not be listed.

3.17.2 Recursively Discovering Tests

In addition to the :avocado: enable and :avocado: disable docstring directives, Avocado has support for the :avocado: recursive directive. It is intended to be used in inherited classes when you want to tell Avocado to also discover the ancestor classes. The :avocado: recursive directive will direct Avocado to evaluate all the ancestors of the class until the base class, the one derived from from avocado.Test.

36 Chapter 3. Writing Avocado Tests avocado Documentation, Release 63.0

Example: File /usr/share/doc/avocado/tests/test_base_class.py: from avocado import Test class BaseClass(Test):

def test_basic(self): pass

File /usr/share/doc/avocado/tests/test_ﬁrst_child.py: from test_base_class import BaseClass class FirstChild(BaseClass):

def test_first_child(self): pass

File /usr/share/doc/avocado/tests/test_second_child.py: from test_first_child import FirstChild class SecondChild(FirstChild): """ :avocado: recursive """

def test_second_child(self): pass

Using only test_second_child.py as a test reference will result in:

$ avocado list test_second_child.py INSTRUMENTED test_second_child.py:SecondChild.test_second_child INSTRUMENTED test_second_child.py:SecondChild.test_first_child INSTRUMENTED test_second_child.py:SecondChild.test_basic

Notice that the :avocado: disable docstring will be ignored in ancestors during the recursive discovery. What means that even if an ancestor contains the docstring :avocado: disable, that ancestor will still be included in the results.

3.17.3 Categorizing tests

Avocado allows tests to be given tags, which can be used to create test categories. With tags set, users can select a subset of the tests found by the test resolver (also known as test loader). To make this feature easier to grasp, let’s work with an example: a single Python source code ﬁle, named perf.py, that contains both disk and network performance tests: from avocado import Test class Disk(Test): (continues on next page)

3.17. Docstring Directives 37 avocado Documentation, Release 63.0

(continued from previous page)

""" Disk performance tests

:avocado: tags=disk,slow,superuser,unsafe """

def test_device(self): device= self.params.get('device', default='/dev/vdb') self.whiteboard= measure_write_to_disk(device) class Network(Test):

""" Network performance tests

:avocado: tags=net,fast,safe """

def test_latency(self): self.whiteboard= measure_latency()

def test_throughput(self): self.whiteboard= measure_throughput() class Idle(Test):

""" Idle tests """

def test_idle(self): self.whiteboard="test achieved nothing"

Warning: All docstring directives in Avocado require a strict format, that is, :avocado: followed by one or more spaces, and then followed by a single value with no white spaces in between. This means that an attempt to write a docstring directive like :avocado: tags=foo, bar will be interpreted as :avocado: tags=foo,.

Usually, listing and executing tests with the Avocado test runner would reveal all three tests:

$ avocado list perf.py INSTRUMENTED perf.py:Disk.test_device INSTRUMENTED perf.py:Network.test_latency INSTRUMENTED perf.py:Network.test_throughput INSTRUMENTED perf.py:Idle.test_idle

If you want to list or run only the network based tests, you can do so by requesting only tests that are tagged with net:

$ avocado list perf.py --filter-by-tags=net INSTRUMENTED perf.py:Network.test_latency INSTRUMENTED perf.py:Network.test_throughput

38 Chapter 3. Writing Avocado Tests avocado Documentation, Release 63.0

Now, suppose you’re not in an environment where you’re confortable running a test that will write to your raw disk devices (such as your development workstation). You know that some tests are tagged with safe while others are tagged with unsafe. To only select the “safe” tests you can run:

$ avocado list perf.py --filter-by-tags=safe INSTRUMENTED perf.py:Network.test_latency INSTRUMENTED perf.py:Network.test_throughput

But you could also say that you do not want the “unsafe” tests (note the minus sign before the tag):

$ avocado list perf.py --filter-by-tags=-unsafe INSTRUMENTED perf.py:Network.test_latency INSTRUMENTED perf.py:Network.test_throughput

Tip: The - sign may cause issues with some shells. One know error condition is to use spaces between --filter-by-tags and the negated tag, that is, --filter-by-tags -unsafe will most likely not work. To be on the safe side, use --filter-by-tags=-tag.

If you require tests to be tagged with multiple tags, just add them separate by commas. Example:

$ avocado list perf.py --filter-by-tags=disk,slow,superuser,unsafe INSTRUMENTED perf.py:Disk.test_device

If no test contains all tags given on a single –ﬁlter-by-tags parameter, no test will be included:

$ avocado list perf.py --filter-by-tags=disk,slow,superuser,safe | wc -l 0

Test tags can be applied to test classes and to test methods. Tags are evaluated per method, meaning that the class tags will be inherited by all methods, being merged with method local tags. Example: from avocado import Test class MyClass(Test): """ :avocado: tags=furious """

def test1(self): """ :avocado: tags=fast """ pass

def test2(self): """ :avocado: tags=slow """ pass

If you use the tag furious, all tests will be included:

$ avocado list furious_tests.py --filter-by-tags=furious INSTRUMENTED test_tags.py:MyClass.test1 INSTRUMENTED test_tags.py:MyClass.test2

3.17. Docstring Directives 39 avocado Documentation, Release 63.0

But using fast and furious will include only test1:

$ avocado list furious_tests.py --filter-by-tags=fast,furious INSTRUMENTED test_tags.py:MyClass.test1

Multiple –ﬁlter-by-tags

While multiple tags in a single option will require tests with all the given tags (effectively a logical AND operation), it’s also possible to use multiple --filter-by-tags (effectively a logical OR operation). For instance To include all tests that have the disk tag and all tests that have the net tag, you can run:

$ avocado list perf.py --filter-by-tags=disk --filter-by-tags=net INSTRUMENTED perf.py:Disk.test_device INSTRUMENTED perf.py:Network.test_latency INSTRUMENTED perf.py:Network.test_throughput

Including tests without tags

The normal behavior when using --filter-by-tags is to require the given tags on all tests. In some situations, though, it may be desirable to include tests that have no tags set. For instance, you may want to include tests of certain types that do not have support for tags (such as SIMPLE tests) or tests that have not (yet) received tags. Consider this command:

$ avocado list perf.py /bin/true --filter-by-tags=disk INSTRUMENTED perf.py:Disk.test_device

Since it requires the disk tag, only one test was returned. By using the --filter-by-tags-include-empty option, you can force the inclusion of tests without tags:

$ avocado list perf.py /bin/true --filter-by-tags=disk --filter-by-tags-include-empty SIMPLE /bin/true INSTRUMENTED perf.py:Idle.test_idle INSTRUMENTED perf.py:Disk.test_device

3.18 Python unittest Compatibility Limitations And Caveats

When executing tests, Avocado uses different techniques than most other Python unittest runners. This brings some compatibility limitations that Avocado users should be aware.

3.18.1 Execution Model

One of the main differences is a consequence of the Avocado design decision that tests should be self contained and isolated from other tests. Additionally, the Avocado test runner runs each test in a separate process. If you have a unittest class with many test methods and run them using most test runners, you’ll ﬁnd that all test methods run under the same process. To check that behavior you could add to your setUp method: def setUp(self): print("PID: %s", os.getpid())

If you run the same test under Avocado, you’ll ﬁnd that each test is run on a separate process.

40 Chapter 3. Writing Avocado Tests avocado Documentation, Release 63.0

3.18.2 Class Level setUp and tearDown

Because of Avocado’s test execution model (each test is run on a separate process), it doesn’t make sense to support unittest’s unittest.TestCase.setUpClass() and unittest.TestCase.tearDownClass(). Test classes are freshly instantiated for each test, so it’s pointless to run code in those methods, since they’re supposed to keep class state between tests. The setUp method is the only place in avocado where you are allowed to call the skip method, given that, if a test started to be executed, by deﬁnition it can’t be skipped anymore. Avocado will do its best to enforce this boundary, so that if you use skip outside setUp, the test upon execution will be marked with the ERROR status, and the error message will instruct you to ﬁx your test’s code. If you require a common setup to a number of tests, the current recommended approach is to to write regular setUp and tearDown code that checks if a given state was already set. One example for such a test that requires a binary installed by a package:

from avocado import Test

from avocado.utils import software_manager from avocado.utils import path as utils_path from avocado.utils import process

class BinSleep(Test):

""" Sleeps using the /bin/sleep binary """ def setUp(self): self.sleep= None try: self.sleep= utils_path.find_command('sleep') except utils_path.CmdNotFoundError: software_manager.install_distro_packages({'fedora':['coreutils']}) self.sleep= utils_path.find_command('sleep')

def test(self): process.run("%s 1"% self.sleep)

If your test setup is some kind of action that will last accross processes, like the installation of a software package given in the previous example, you’re pretty much covered here. If you need to keep other type of data a class across test executions, you’ll have to resort to saving and restoring the data from an outside source (say a “pickle” ﬁle). Finding and using a reliable and safe location for saving such data is currently not in the Avocado supported use cases.

3.19 Environment Variables for Tests

Avocado exports some information, including test parameters, as environment variables to the running test. While these variables are available to all tests, they are usually more interesting to SIMPLE tests. The reason is that SIMPLE tests can not make direct use of Avocado API. INSTRUMENTED tests will usually have more powerful ways, to access the same information. Here is a list of the variables that Avocado currently exports to tests:

3.19. Environment Variables for Tests 41 avocado Documentation, Release 63.0

Environemnt Meaning Example Variable AVO- Version of Avocado test runner 0.12.0 CADO_VERSION AVO- Base directory of Avocado tests $HOME/Downloads/avocado- CADO_TEST_BASEDIR source/avocado AVO- Work directory for the test /var/tmp/avocado_Bjr_rd/my_test.sh CADO_TEST_WORKDIR AVO- Temporary directory created by the teststmpdir /var/tmp/avocado_XhEdo/ CADO_TESTS_COMMON_TMPDIRplugin. The directory is persistent throughout the tests in the same Job AVO- Log directory for the test $HOME/logs/job-results/job- CADO_TEST_LOGDIR 2014-09-16T14.38-ac332e6/test- results/$HOME/my_test.sh.1 AVO- Log ﬁle for the test $HOME/logs/job-results/job- CADO_TEST_LOGFILE 2014-09-16T14.38-ac332e6/test- results/$HOME/my_test.sh.1/debug.log AVO- Output directory for the test $HOME/logs/job-results/job- CADO_TEST_OUTPUTDIR 2014-09-16T14.38-ac332e6/test- results/$HOME/my_test.sh.1/data AVO- The system information directory $HOME/logs/job-results/job- CADO_TEST_SYSINFODIR 2014-09-16T14.38-ac332e6/test- results/$HOME/my_test.sh.1/sysinfo *** All variables from –mux-yaml TIMEOUT=60; IO_WORKERS=10; VM_BYTES=512M; . . .

Warning: AVOCADO_TEST_SRCDIR was present in earlier versions, but has been deprecated on version 60.0, and removed on version 62.0. Please use AVOCADO_TEST_WORKDIR instead.

Warning: AVOCADO_TEST_DATADIR was present in earlier versions, but has been deprecated on version 60.0, and removed on version 62.0. The test data ﬁles (and directories) are now dynamically evaluated and are not available as environment variables

3.20 SIMPLE Tests BASH extensions

SIMPLE tests written in shell can use a few Avocado utilities. In your shell code, check if the libraries are available with something like:

AVOCADO_SHELL_EXTENSIONS_DIR=$(avocado exec-path 2>/dev/null)

And if available, injects that directory containing those utilities into the PATH used by the shell, making those utilities readily accessible: if [ $? == 0 ]; then PATH=$AVOCADO_SHELL_EXTENSIONS_DIR:$PATH fi

42 Chapter 3. Writing Avocado Tests avocado Documentation, Release 63.0

For a full list of utilities, take a look into at the directory return by avocado exec-path (if any). Also, the example test examples/tests/simplewarning.sh can serve as further inspiration.

Tip: These extensions may be available as a separate package. For RPM packages, look for the bash sub-package.

3.21 SIMPLE Tests Status

With SIMPLE tests, Avocado checks the exit code of the test to determine whether the test PASSed or FAILed. If your test exits with exit code 0 but you still want to set a different test status in some conditions, Avocado can search a given regular expression in the test outputs and, based on that, set the status to WARN or SKIP. To use that feature, you have to set the proper keys in the conﬁguration ﬁle. For instance, to set the test status to SKIP when the test outputs a line like this: ‘11:08:24 Test Skipped’:

[simpletests.output] skip_regex = ^\d\d:\d\d:\d\d Test Skipped$

That conﬁguration will make avocado to search the Python Regular Expression on both stdout and stderr. If you want to limit the search for only one of them, there’s another key for that conﬁguration, resulting in:

[simpletests.output] skip_regex = ^\d\d:\d\d:\d\d Test Skipped$ skip_location = stderr

The equivalent settings can be present for the WARN status. For instance, if you want to set the test status to WARN when the test outputs a line starting with string WARNING:, the conﬁguration ﬁle will look like this:

[simpletests.output] skip_regex = ^\d\d:\d\d:\d\d Test Skipped$ skip_location = stderr warn_regex = ^WARNING: warn_location = all

3.22 Wrap Up

We recommend you take a look at the example tests present in the examples/tests directory, that contains a few samples to take some inspiration from. That directory, besides containing examples, is also used by the Av- ocado self test suite to do functional testing of Avocado itself. Although one can inspire in https://github.com/ avocado-framework-tests where people are allowed to share their basic system tests. It is also recommended that you take a look at the API Reference. for more possibilities.

3.21. SIMPLE Tests Status 43 avocado Documentation, Release 63.0

44 Chapter 3. Writing Avocado Tests CHAPTER 4

Result Formats

A test runner must provide an assortment of ways to clearly communicate results to interested parties, be them humans or machines.

Note: There are several optional result plugins, you can ﬁnd them in Result plugins.

4.1 Results for human beings

Avocado has two different result formats that are intended for human beings: • Its default UI, which shows the live test execution results on a command line, text based, UI. • The HTML report, which is generated after the test job ﬁnishes running.

4.1.1 Avocado command line UI

A regular run of Avocado will present the test results in a live fashion, that is, the job and its test(s) results are constantly updated:

$ avocado run sleeptest.py failtest.py synctest.py JOB ID : 5ffe479262ea9025f2e4e84c4e92055b5c79bdc9 JOB LOG : $HOME/avocado/job-results/job-2014-08-12T15.57-5ffe4792/job.log (1/3) sleeptest.py:SleepTest.test: PASS (1.01 s) (2/3) failtest.py:FailTest.test: FAIL (0.00 s) (3/3) synctest.py:SyncTest.test: PASS (1.98 s) RESULTS : PASS 1 | ERROR 1 | FAIL 1 | SKIP 0 | WARN 0 | INTERRUPT 0 JOB TIME : 3.27 s JOB HTML : $HOME/avocado/job-results/job-2014-08-12T15.57-5ffe4792/html/results.html

The most important thing is to remember that programs should never need to parse human output to ﬁgure out what happened to a test job run.

45 avocado Documentation, Release 63.0

4.2 Machine readable results

Another type of results are those intended to be parsed by other applications. Several standards exist in the test community, and Avocado can in theory support pretty much every result standard out there. Out of the box, Avocado supports a couple of machine readable results. They are always generated and stored in the results directory in results.$type ﬁles, but you can ask for a different location too.

4.2.1 xunit

The default machine readable output in Avocado is xunit. xunit is an XML format that contains test results in a structured form, and are used by other test automation projects, such as jenkins. If you want to make Avocado to generate xunit output in the standard output of the runner, simply use:

$ avocado run sleeptest.py failtest.py synctest.py --xunit -

˓→[CDATA[Traceback (most recent call last): File "/home/medic/Work/Projekty/avocado/avocado/avocado/core/test.py", line 490, in

˓→_run_avocado raise test_exception TestFail: This test is supposed to fail ]]>

˓→avocado/core/test.py:435 14:46:53 ERROR| Traceback (most recent call last): 14:46:53 ERROR| File "/home/medic/Work/Projekty/avocado/avocado/examples/tests/

˓→failtest.py", line 17, in test 14:46:53 ERROR| self.fail('This test is supposed to fail') 14:46:53 ERROR| File "/home/medic/Work/Projekty/avocado/avocado/avocado/core/test.py

˓→", line 585, in fail 14:46:53 ERROR| raise exceptions.TestFail(message) 14:46:53 ERROR| TestFail: This test is supposed to fail 14:46:53 ERROR| 14:46:53 ERROR| FAIL 2-failtest.py:FailTest.test -> TestFail: This test is supposed

˓→to fail 14:46:53 INFO | ]]>

Note: The dash - in the option –xunit, it means that the xunit result should go to the standard output.

46 Chapter 4. Result Formats avocado Documentation, Release 63.0

Note: In case your tests produce very long outputs, you can limit the number of embedded characters by –xunit- max-test-log-chars. If the output in the log ﬁle is longer it only attaches up-to max-test-log-chars characters one half starting from the beginning of the content, the other half from the end of the content.

4.2.2 JSON

JSON is a widely used data exchange format. The JSON Avocado plugin outputs job information, similarly to the xunit output plugin:

$ avocado run sleeptest.py failtest.py synctest.py --json - { "cancel": 0, "debuglog": "/home/cleber/avocado/job-results/job-2016-08-09T13.53-10715c4/job.log

˓→", "errors": 0, "failures": 1, "job_id": "10715c4645d2d2b57889d7a4317fcd01451b600e", "pass": 2, "skip": 0, "tests": [ { "end": 1470761623.176954, "fail_reason": "None", "logdir": "/home/cleber/avocado/job-results/job-2016-08-09T13.53-10715c4/

˓→test-results/1-sleeptest.py:SleepTest.test", "logfile": "/home/cleber/avocado/job-results/job-2016-08-09T13.53-10715c4/

˓→test-results/1-sleeptest.py:SleepTest.test/debug.log", "start": 1470761622.174918, "status": "PASS", "id": "1-sleeptest.py:SleepTest.test", "time": 1.0020360946655273, "whiteboard": "" }, { "end": 1470761623.193472, "fail_reason": "This test is supposed to fail", "logdir": "/home/cleber/avocado/job-results/job-2016-08-09T13.53-10715c4/

˓→test-results/2-failtest.py:FailTest.test", "logfile": "/home/cleber/avocado/job-results/job-2016-08-09T13.53-10715c4/

˓→test-results/2-failtest.py:FailTest.test/debug.log", "start": 1470761623.192334, "status": "FAIL", "id": "2-failtest.py:FailTest.test", "time": 0.0011379718780517578, "whiteboard": "" }, { "end": 1470761625.656061, "fail_reason": "None", "logdir": "/home/cleber/avocado/job-results/job-2016-08-09T13.53-10715c4/

˓→test-results/3-synctest.py:SyncTest.test", "logfile": "/home/cleber/avocado/job-results/job-2016-08-09T13.53-10715c4/

˓→test-results/3-synctest.py:SyncTest.test/debug.log", "start": 1470761623.208165, (continues on next page)

4.2. Machine readable results 47 avocado Documentation, Release 63.0

(continued from previous page) "status": "PASS", "id": "3-synctest.py:SyncTest.test", "time": 2.4478960037231445, "whiteboard": "" } ], "time": 3.4510700702667236, "total": 3 }

Note: The dash - in the option –json, it means that the xunit result should go to the standard output.

Bear in mind that there’s no documented standard for the Avocado JSON result format. This means that it will probably grow organically to accommodate newer Avocado features. A reasonable effort will be made to not break backwards compatibility with applications that parse the current form of its JSON result.

4.2.3 TAP

Provides the basic TAP (Test Anything Protocol) results, currently in v12. Unlike most existing avocado machine readable outputs this one is streamlined (per test results):

$ avocado run sleeptest.py --tap - 1..1 # debug.log of sleeptest.py:SleepTest.test: # 12:04:38 DEBUG| PARAMS (key=sleep_length, path=*, default=1) => 1 # 12:04:38 DEBUG| Sleeping for 1.00 seconds # 12:04:39 INFO | PASS 1-sleeptest.py:SleepTest.test # 12:04:39 INFO | ok 1 sleeptest.py:SleepTest.test

4.2.4 Silent result

This result disables all stdout logging (while keeping the error messages being printed to stderr). One can then use the return code to learn about the result:

$ avocado --silent run failtest.py $ echo $? 1

In practice, this would usually be used by scripts that will in turn run Avocado and check its results:

#!/bin/bash ... $ avocado --silent run /path/to/my/test.py if [ $? == 0 ]; then echo "great success!" elif ... more details regarding exit codes in Exit Codes section.

48 Chapter 4. Result Formats avocado Documentation, Release 63.0

4.3 Multiple results at once

You can have multiple results formats at once, as long as only one of them uses the standard output. For example, it is ﬁne to use the xunit result on stdout and the JSON result to output to a ﬁle:

$ avocado run sleeptest.py synctest.py --xunit - --json /tmp/result.json

$ cat /tmp/result.json { "debuglog": "/home/cleber/avocado/job-results/job-2016-08-09T13.55-1a94ad6/job.

˓→log", "errors": 0, ... }

But you won’t be able to do the same without the –json ﬂag passed to the program:

$ avocado run sleeptest.py synctest.py --xunit - --json - Options --json --xunit are trying to use stdout simultaneously Please set at least one of them to a file to avoid conflicts

That’s basically the only rule, and a sane one, that you need to follow.

4.4 Exit Codes

Avocado exit code tries to represent different things that can happen during an execution. That means exit codes can be a combination of codes that were ORed together as a single exit code. The ﬁnal exit code can be de-bundled so users can have a good idea on what happened to the job. The single individual exit codes are: • AVOCADO_ALL_OK (0) • AVOCADO_TESTS_FAIL (1) • AVOCADO_JOB_FAIL (2) • AVOCADO_FAIL (4) • AVOCADO_JOB_INTERRUPTED (8) If a job ﬁnishes with exit code 9, for example, it means we had at least one test that failed and also we had at some point a job interruption, probably due to the job timeout or a CTRL+C.

4.5 Implementing other result formats

If you are looking to implement a new machine or human readable output format, you can refer to avocado. plugins.xunit and use it as a starting point.

4.3. Multiple results at once 49 avocado Documentation, Release 63.0

If your result is something that is produced at once, based on the complete job outcome, you should create a new class that inherits from avocado.core.plugin_interfaces.Result and implements the avocado. core.plugin_interfaces.Result.render() method. But, if your result implementation is something that outputs information live before/during/after tests, then the avocado.core.plugin_interfaces.ResultEvents interface is to one to look at. It will require you to implement the methods that will perform actions (write to a ﬁle/stream) for each of the deﬁned events on a Job and test execution. You can take a look at Plugin System for more information on how to write a plugin that will activate and execute the new result format.

50 Chapter 4. Result Formats CHAPTER 5

Conﬁguration

Avocado utilities have a certain default behavior based on educated, reasonable (we hope) guesses about how users like to use their systems. Of course, different people will have different needs and/or dislike our defaults, and that’s why a configuration system is in place to help with those cases The Avocado config file format is based on the (informal) INI file ‘specification’, that is implemented by Python’s ConfigParser. The format is simple and straightforward, composed by sections, that contain a number of keys and values. Take for example a basic Avocado config file:

[datadir.paths] base_dir= /var/lib/avocado test_dir= /usr/share/doc/avocado/tests data_dir= /var/lib/avocado/data logs_dir= ~/avocado/job-results

The datadir.paths section contains a number of keys, all of them related to directories used by the test runner. The base_dir is the base directory to other important Avocado directories, such as log, data and test directories. You can also choose to set those other important directories by means of the variables test_dir, data_dir and logs_dir. You can do this by simply editing the conﬁg ﬁles available.

5.1 Conﬁg ﬁle parsing order

Avocado starts by parsing what it calls system wide config file, that is shipped to all Avocado users on a system wide directory, /etc/avocado/avocado.conf. Then it’ll verify if there’s a local user config file, that is located usually in ~/.config/avocado/avocado.conf. The order of the parsing matters, so the system wide file is parsed, then the user config file is parsed last, so that the user can override values at will. There is another directory that will be scanned by extra config files, /etc/avocado/conf.d. This directory may contain plugin config files, and extra additional config files that the system administrator/avocado developers might judge necessary to put there. Please note that for base directories, if you chose a directory that can’t be properly used by Avocado (some directories require read access, others, read and write access), Avocado will fall back to some defaults. So if your regular user wants to write logs to /root/avocado/logs, Avocado will not use that directory, since it can’t write files to that place. A new location, by default ~/avocado/job-results will be selected instead.

51 avocado Documentation, Release 63.0

The order of files described in this section is only valid if avocado was installed in the system. For people using avocado from git repos (usually avocado developers), that did not install it in the system, keep in mind that avocado will read the config files present in the git repos, and will ignore the system wide config files. Running avocado config will let you know which files are actually being used.

5.2 Plugin conﬁg ﬁles

Plugins can also be configured by config files. In order to not disturb the main Avocado config file, those plugins, if they wish so, may install additional config files to /etc/avocado/conf.d/[pluginname].conf, that will be parsed after the system wide config file. Users can override those values as well at the local config file level. Considering the config for the hypothethical plugin salad:

[salad.core] base= ceasar dressing= ceasar

If you want, you may change dressing in your config file by simply adding a [salad.core] new section in your local config file, and set a different value for dressing there.

5.3 Parsing order recap

So the file parsing order is: • /etc/avocado/avocado.conf • /etc/avocado/conf.d/*.conf • ~/.config/avocado/avocado.conf In this order, meaning that what you set on your local config file may override what’s defined in the system wide files.

Note: Please note that if avocado is running from git repos, those files will be ignored in favor of in tree configuration files. This is something that would normally only affect people developing avocado, and if you are in doubt, avocado config will tell you exactly which files are being used in any given situation.

Note: When avocado runs inside virtualenv than path for global conﬁg ﬁles is also changed. For example, avocado.conf comes from the virual-env path venv/etc/avocado/avocado.conf.

5.4 Order of precedence for values used in tests

Since you can use the config system to alter behavior and values used in tests (think paths to test programs, for example), we established the following order of precedence for variables (from least precedence to most): • default value (from library or test code) • global config file • local (user) config file • command line switch

52 Chapter 5. Conﬁguration avocado Documentation, Release 63.0

• test parameters So the least important value comes from the library or test code default, going all the way up to the test parameters system.

5.5 Conﬁg plugin

A configuration plugin is provided for users that wish to quickly see what’s defined in all sections of their Avocado configuration, after all the files are parsed in their correct resolution order. Example:

$ avocado config Config files read (in order): /etc/avocado/avocado.conf $HOME/.config/avocado/avocado.conf

Section.Key Value runner.base_dir /var/lib/avocado runner.test_dir /usr/share/doc/avocado/tests runner.data_dir /var/lib/avocado/data runner.logs_dir ~/avocado/job-results

The command also shows the order in which your conﬁg ﬁles were parsed, giving you a better understanding of what’s going on. The Section.Key nomenclature was inspired in git config --list output.

5.6 Avocado Data Directories

When running tests, we are frequently looking to: • Locate tests • Write logs to a given location • Grab files that will be useful for tests, such as ISO files or VM disk images Avocado has a module dedicated to find those paths, to avoid cumbersome path manipulation magic that people had to do in previous test frameworks1. If you want to list all relevant directories for your test, you can use avocado config –datadir command to list those directories. Executing it will give you an output similar to the one seen below:

$ avocado config --datadir Config files read (in order): /etc/avocado/avocado.conf $HOME/.config/avocado/avocado.conf

Avocado replaces config dirs that can't be accessed with sensible defaults. Please edit your local config file to customize values

Avocado Data Directories: base $HOME/avocado tests $HOME/Code/avocado/examples/tests data $HOME/avocado/data logs $HOME/avocado/job-results

1 For example, autotest.

5.5. Conﬁg plugin 53 avocado Documentation, Release 63.0

Note that, while Avocado will do its best to use the config values you provide in the config file, if it can’t write values to the locations provided, it will fall back to (we hope) reasonable defaults, and we notify the user about that in the output of the command. The relevant API documentation and meaning of each of those data directories is in avocado.core.data_dir, so it’s highly recommended you take a look. You may set your preferred data dirs by setting them in the Avocado config files. The only exception for important data dirs here is the Avocado tmp dir, used to place temporary files used by tests. That directory will be in normal circumstances /var/tmp/avocado_XXXXX, (where XXXXX is in actuality a random string) securely created on /var/tmp/, unless the user has the $TMPDIR environment variable set, since that is customary among unix programs. The next section of the documentation explains how you can see and set config values that modify the behavior for the Avocado utilities and plugins.

54 Chapter 5. Conﬁguration CHAPTER 6

Test discovery

In this section you can learn how tests are being discovered and how to affect this process.

6.1 The order of test loaders

Avocado supports different types of test starting with SIMPLE tests, which are simply executable files, then unittest- like tests called INSTRUMENTED up to some tests like the avocado-vt ones, which uses complex matrix of tests from config files that don’t directly map to existing files. Given the number of loaders, the mapping from test names on the command line to executed tests might not always be unique. Additionally some people might always (or for given run) want to execute only tests of a single type. To adjust this behavior you can either tweak plugins.loaders in avocado settings (/etc/avocado/), or tem- porarily using --loaders (option of avocado run) option. This option allows you to specify order and some params of the available test loaders. You can specify either loader_name (file), loader_name + TEST_TYPE (file.SIMPLE) and for some loaders even additional params passed after : (external:/bin/echo -e. You can also supply @DEFAULT, which injects into that position all the remaining unused loaders. To get help about --loaders:

$ avocado run --loaders ? $ avocado run --loaders external:?

Example of how --loaders affects the produced tests (manually gathered as some of them result in error):

$ avocado run passtest.py boot this_does_not_exist /bin/echo > INSTRUMENTED passtest.py:PassTest.test > VT io-github-autotest-qemu.boot > MISSING this_does_not_exist > SIMPLE /bin/echo $ avocado run passtest.py boot this_does_not_exist /bin/echo --loaders @DEFAULT

˓→"external:/bin/echo -e" (continues on next page)

55 avocado Documentation, Release 63.0

(continued from previous page) > INSTRUMENTED passtest.py:PassTest.test > VT io-github-autotest-qemu.boot > EXTERNAL this_does_not_exist > SIMPLE /bin/echo $ avocado run passtest.py boot this_does_not_exist /bin/echo --loaders file.SIMPLE

˓→file.INSTRUMENTED @DEFAULT external.EXTERNAL:/bin/echo > INSTRUMENTED passtest.py:PassTest.test > VT io-github-autotest-qemu.boot > EXTERNAL this_does_not_exist > SIMPLE /bin/echo

6.2 Running simple tests with arguments

This used to be supported out of the box by running avocado run "test arg1 arg2" but it was quite con- fusing and removed. It is still possible to achieve that by using shell and one can even combine normal tests and the parametrized ones:

$ avocado run --loaders file external:/bin/sh -- existing_file.py "'/bin/echo

˓→something'" nonexisting-file

This will run 3 tests, the ﬁrst one is a normal test deﬁned by existing_file.py (most probably an instrumented test). Then we have /bin/echo which is going to be executed via /bin/sh -c '/bin/echo something'. The last one would be nonexisting-file which would execute /bin/sh -c nonexisting-file which most probably fails. Note that you are responsible for quotating the test-id (see the "'/bin/echo something'" example).

6.3 Filtering tests by tags

6.4 Test References

A Test Reference is a string that can be resolved into (interpreted as) one or more tests by the Avocado Test Resolver. Each resolver (a.k.a. loader) can handle the Test References differently. For example, External Loader will use the Test Reference as an argument for the external command, while the File Loader will expect a file path. If you don’t specify the loader that you want to use, all of the available loaders will be used to resolve the provided Test References. One by one, the Test References will be resolved by the first loader able to create a test list out of that reference. Below you can find some extra details about the specific builtin Avocado loaders. For Loaders introduced to Avocado via plugins (VT, Robot, . . . ), please refer to the corresponding loader/plugin documentation.

56 Chapter 6. Test discovery avocado Documentation, Release 63.0

6.4.1 File Loader

For the File Loader, the loader responsible for discovering INSTRUMENTED, PyUNITTEST (classic python unittests) and SIMPLE tests. If the file corresponds to an INSTRUMENTED or PyUNITTEST test, you can filter the Test IDs by adding to the Test Reference a : followed by a regular expression. For instance, if you want to list all tests that are present in the gdbtest.py file, you can use the list command below:

$ avocado list /usr/share/doc/avocado/tests/gdbtest.py INSTRUMENTED /usr/share/doc/avocado/tests/gdbtest.py:GdbTest.test_start_exit INSTRUMENTED /usr/share/doc/avocado/tests/gdbtest.py:GdbTest.test_existing_commands_

˓→raw INSTRUMENTED /usr/share/doc/avocado/tests/gdbtest.py:GdbTest.test_existing_commands INSTRUMENTED /usr/share/doc/avocado/tests/gdbtest.py:GdbTest.test_load_set_breakpoint_

˓→run_exit_raw INSTRUMENTED /usr/share/doc/avocado/tests/gdbtest.py:GdbTest.test_load_set_breakpoint_

˓→run_exit INSTRUMENTED /usr/share/doc/avocado/tests/gdbtest.py:GdbTest.test_generate_core INSTRUMENTED /usr/share/doc/avocado/tests/gdbtest.py:GdbTest.test_set_multiple_break INSTRUMENTED /usr/share/doc/avocado/tests/gdbtest.py:GdbTest.test_disconnect_raw INSTRUMENTED /usr/share/doc/avocado/tests/gdbtest.py:GdbTest.test_disconnect INSTRUMENTED /usr/share/doc/avocado/tests/gdbtest.py:GdbTest.test_remote_exec INSTRUMENTED /usr/share/doc/avocado/tests/gdbtest.py:GdbTest.test_stream_messages INSTRUMENTED /usr/share/doc/avocado/tests/gdbtest.py:GdbTest.test_connect_multiple_

˓→clients INSTRUMENTED /usr/share/doc/avocado/tests/gdbtest.py:GdbTest.test_server_exit INSTRUMENTED /usr/share/doc/avocado/tests/gdbtest.py:GdbTest.test_multiple_servers INSTRUMENTED /usr/share/doc/avocado/tests/gdbtest.py:GdbTest.test_interactive INSTRUMENTED /usr/share/doc/avocado/tests/gdbtest.py:GdbTest.test_interactive_args INSTRUMENTED /usr/share/doc/avocado/tests/gdbtest.py:GdbTest.test_exit_status INSTRUMENTED /usr/share/doc/avocado/tests/gdbtest.py:GdbTest.test_server_stderr INSTRUMENTED /usr/share/doc/avocado/tests/gdbtest.py:GdbTest.test_server_stdout INSTRUMENTED /usr/share/doc/avocado/tests/gdbtest.py:GdbTest.test_interactive_stdout INSTRUMENTED /usr/share/doc/avocado/tests/gdbtest.py:GdbTest.test_remote

To ﬁlter the results, listing only the tests that have test_interactive in their test method names, you can execute:

$ avocado list /usr/share/doc/avocado/tests/gdbtest.py:test_interactive INSTRUMENTED /usr/share/doc/avocado/tests/gdbtest.py:GdbTest.test_interactive INSTRUMENTED /usr/share/doc/avocado/tests/gdbtest.py:GdbTest.test_interactive_args INSTRUMENTED /usr/share/doc/avocado/tests/gdbtest.py:GdbTest.test_interactive_stdout

As the string after the : is a regular expression, three tests were ﬁltered in. You can manipulate the regular expression to have only the test with that exact name:

$ avocado list /usr/share/doc/avocado/tests/gdbtest.py:test_interactive$ INSTRUMENTED /usr/share/doc/avocado/tests/gdbtest.py:GdbTest.test_interactive

The regular expression enables you to have more complex ﬁlters. Example:

$ avocado list /usr/share/doc/avocado/tests/gdbtest.py:GdbTest.test_[le].*raw INSTRUMENTED /usr/share/doc/avocado/tests/gdbtest.py:GdbTest.test_existing_commands_

˓→raw INSTRUMENTED /usr/share/doc/avocado/tests/gdbtest.py:GdbTest.test_load_set_breakpoint_

˓→run_exit_raw

6.4. Test References 57 avocado Documentation, Release 63.0

Once the test reference is providing you the expected outcome, you can replace the list subcommand with the run subcommand to execute your tests:

$ avocado run /usr/share/doc/avocado/tests/gdbtest.py:GdbTest.test_[le].*raw JOB ID : 333912fb02698ed5339a400b832795a80757b8af JOB LOG : $HOME/avocado/job-results/job-2017-06-14T14.54-333912f/job.log (1/2) /usr/share/doc/avocado/tests/gdbtest.py:GdbTest.test_existing_commands_raw:

˓→PASS (0.59 s) (2/2) /usr/share/doc/avocado/tests/gdbtest.py:GdbTest.test_load_set_breakpoint_run_

Warning: Specially when using regular expressions, it’s recommended to individually enclose your Test References in quotes to avoid bash of corrupting them. In that case, the command from the example above would be: avocado run "/usr/share/doc/avocado/tests/gdbtest.py:GdbTest. test_[le].*raw"

6.4.2 External Loader

Using the External Loader, Avocado will consider that and External Runner will be in place and so Avocado doesn’t really need to resolve the references. Instead, Avocado will pass the references as parameters to the External Runner. Example:

$ avocado run 20 Unable to resolve reference(s) '20' with plugins(s) 'file', 'robot', 'vt', 'external', try running 'avocado list -V 20' to see the details.

In the command above, no loaders can resolve 20 as a test. But running the command above with the External Runner /bin/sleep will make Avocado to actually execute /bin/sleep 20 and check for its return code:

$ avocado run 20 --loaders external:/bin/sleep JOB ID : 42215ece2894134fb9379ee564aa00f1d1d6cb91 JOB LOG : $HOME/avocado/job-results/job-2017-06-19T11.17-42215ec/job.log (1/1) 20: PASS (20.03 s) RESULTS : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0 JOB TIME : 20.13 s JOB HTML : $HOME/avocado/job-results/job-2017-06-19T11.17-42215ec/html/results.html

Warning: It’s safer to put your Test References at the end of the command line, after a –. That will avoid argument vs. Test References clashes. In that case, everything after the – will be considered positional arguments, therefore Test References. Considering that syntax, the command for the example above would be: avocado run --loaders external:/bin/sleep -- 20

58 Chapter 6. Test discovery CHAPTER 7

Logging system

This section describes the logging system used in avocado and avocado tests.

7.1 Tweaking the UI

Avocado uses python’s logging system to produce UI and to store test’s output. The system is quite ﬂexible and allows you to tweak the output to your needs either by built-in stream sets, or directly by using the stream name. To tweak them you can use avocado –show STREAM[:LEVEL][,STREAM[:LEVEL],. . . ]. Built-in streams with description (followed by list of associated python streams): app The text based UI (avocado.app) test Output of the executed tests (avocado.test, “”) debug Additional messages useful to debug avocado (avocado.app.debug) remote Fabric/paramiko debug messages, useful to analyze remote execution (avocado.fabric, paramiko) early Early logging before the logging system is set. It includes the test output and lots of output produced by used libraries. (“”, avocado.test) Additionally you can specify “all” or “none” to enable/disable all of pre-deﬁned streams and you can also supply custom python logging streams and they will be passed to the standard output.

Warning: Messages with importance greater or equal WARN in logging stream “avocado.app” are always enabled and they go to the standard error.

7.2 Storing custom logs

When you run a test, you can also store custom logging streams into the results directory by avocado run –store- logging-stream [STREAM[:LEVEL] [STREAM[:LEVEL] . . . ]], which will produce $STREAM.$LEVEL ﬁles per each

59 avocado Documentation, Release 63.0

(unique) entry in the test results directory.

Note: You have to specify separated logging streams. You can’t use the built-in streams in this function.

Note: Currently the custom streams are stored only per job, not per each individual test.

7.3 Paginator

Some subcommands (list, plugins, . . . ) support “paginator”, which, on compatible terminals, basically pipes the colored output to less to simplify browsing of the produced output. One can disable it by –paginator {on|off}.

60 Chapter 7. Logging system CHAPTER 8

Sysinfo collection

Avocado comes with a sysinfo plugin, which automatically gathers some system information per each job or even between tests. This is very useful when later we want to know what caused the test’s failure. This system is config- urable but we provide a sane set of defaults for you. In the default Avocado configuration (/etc/avocado/avocado.conf) there is a section sysinfo.collect where you can enable/disable the sysinfo collection as well as configure the basic environment. In sysinfo. collectibles section you can define basic paths of where to look for what commands/tasks should be performed before/during the sysinfo collection. Avocado supports three types of tasks: 1. commands - file with new-line separated list of commands to be executed before and after the job/test (single execution commands). It is possible to set a timeout which is enforced per each executed command in [sysinfo.collect] by setting “commands_timeout” to a positive number. 2. files - file with new-line separated list of files to be copied 3. profilers - file with new-line separated list of commands to be executed before the job/test and killed at the end of the job/test (follow-like commands) Additionally this plugin tries to follow the system log via journalctl if available. By default these are collected per-job but you can also run them per-test by setting per_test = True in the sysinfo.collect section. The sysinfo can also be enabled/disabled on the cmdline if needed by --sysinfo on|off. After the job execution you can find the collected information in $RESULTS/sysinfo of $RESULTS/ test-results/$TEST/sysinfo. They are categorized into pre, post and profile folders and the file- names are safely-escaped executed commands or file-names. You can also see the sysinfo in html results when you have html results plugin enabled.

Warning: If you are using avocado from sources, you need to manually place the commands/files/profilers into the /etc/avocado/sysinfo directories or adjust the paths in $AVOCADO_SRC/etc/avocado/avocado.conf.

61 avocado Documentation, Release 63.0

62 Chapter 8. Sysinfo collection CHAPTER 9

Test parameters

Note: This section describes in detail what test parameters are and how the whole variants mechanism works in Avo- cado. If you’re interested in the basics, see Accessing test parameters or practical view by examples in Yaml_to_mux plugin.

Avocado allows passing parameters to tests, which effectively results in several different variants of each test. These parameters are available in (test’s) self.params and are of avocado.core.varianter.AvocadoParams type. The data for self.params are supplied by avocado.core.varianter.Varianter which asks all registered plugins for variants or uses default when no variants are deﬁned. Overall picture of how the params handling works is:

+------+ || // Test uses variant to produce AvocadoParams | Test| || +-----^-----+ | // single variant is passed to Test | +------+ | Runner| // iterates through tests and variants to run all +-----^-----+ // desired combinations specified by "--execution-order" | | +------+ provide variants+------+ ||<------|| | Varianter API|| Varianter plugins API| ||------>|| +------+ update defaults+------+ ^^ || | // default params injected | // All plugins are invoked (continues on next page)

63 avocado Documentation, Release 63.0

(continued from previous page) +------+| // in turns |+------++------+|| || avocado-virt|| other providers||| |+------++------+|| +------+| | +------+-----+ || || v v +------++------+ | yaml_to_mux plugin|| Other variant plugin(s)| +-----^------++------+ | | // yaml is parsed to MuxTree, | // multiplexed and yields variants +------+ |+------++------+| ||--mux-yaml||--mux-inject|| |+------++------+| +------+

Let’s introduce the basic keywords.

9.1 TreeNode

avocado.core.tree.TreeNode Is a node object allowing to create tree-like structures with parent->multiple_children relations and storing params. It can also report it’s environment, which is set of params gathered from root to this node. This is used in tests where instead of passing the full tree only the leaf nodes are passed and their environment represents all the values of the tree.

9.2 AvocadoParams

avocado.core.varianter.AvocadoParams Is a “database” of params present in every (instrumented) avocado test. It’s produced during avocado.core. test.Test’s __init__ from a variant. It accepts a list of TreeNode objects; test name avocado.core.test. TestID (for logging purposes) and a list of default paths (Parameter Paths). In test it allows querying for data by using:

self.params.get($name, $path=None, $default=None)

Where: • name - name of the parameter (key) • path - where to look for this parameter (when not speciﬁed uses mux-path) • default - what to return when param not found Each variant deﬁnes a hierarchy, which is preserved so AvocadoParams follows it to return the most appropriate value or raise Exception on error.

64 Chapter 9. Test parameters avocado Documentation, Release 63.0

9.3 Parameter Paths

As test params are organized in trees, it’s possible to have the same variant in several locations. When they are produced from the same TreeNode, it’s not a problem, but when they are a different values there is no way to distinguish which should be reported. One way is to use specific paths, when asking for params, but sometimes, usually when combining upstream and downstream variants, we want to get our values first and fall-back to the upstream ones when they are not found. For example let’s say we have upstream values in /upstream/sleeptest and our values in /downstream/ sleeptest. If we asked for a value using path "*", it’d raise an exception being unable to distinguish whether we want the value from /downstream or /upstream. We can set the parameter paths to ["/downstream/*", "/upstream/*"] to make all relative calls (path starting with *) to first look in nodes in /downstream and if not found look into /upstream. More practical overview of parameter paths is in Yaml_to_mux plugin in Resolution order section.

9.4 Variant

Variant is a set of params produced by Varianter‘_s and passed to the test by the test runner as ‘‘params‘ argument. The simplest variant is None, which still produces an empty AvocadoParams. Also, the Variant can also be a tuple(list, paths) or just the list of avocado.core.tree.TreeNode with the params.

9.5 Dumping/Loading Variants

Depending on the number of parameters, generating the Variants can be very compute intensive. As the Variants are generated as part of the Job execution, that compute intensive task will be executed by the systems under test, causing a possibly unwanted cpu load on those systems. To avoid such situation, you can acquire the resulting JSON serialized variants file, generated out of the variants computation, and load that file on the system where the Job will be executed. There are two ways to acquire the JSON serialized variants file: • Using the --json-variants-dump option of the avocado variants command:

$ avocado variants --mux-yaml examples/yaml_to_mux/hw/hw.yaml --json-variants-

˓→dump variants.json ...

$ file variants.json variants.json: ASCII text, with very long lines, with no line terminators

• Getting the auto-generated JSON serialized variants ﬁle after a Avocado Job execution:

$ avocado run passtest.py --mux-yaml examples/yaml_to_mux/hw/hw.yaml ...

$ file $HOME/avocado/job-results/latest/jobdata/variants.json $HOME/avocado/job-results/latest/jobdata/variants.json: ASCII text, with very

˓→long lines, with no line terminators

Once you have the variants.json ﬁle, you can load it on the system where the Job will take place:

9.3. Parameter Paths 65 avocado Documentation, Release 63.0

$ avocado run passtest.py --json-variants-load variants.json JOB ID : f2022736b5b89d7f4cf62353d3fb4d7e3a06f075 JOB LOG : $HOME/avocado/job-results/job-2018-02-09T14.39-f202273/job.log (1/6) passtest.py:PassTest.test;intel-scsi-56d0: PASS (0.04 s) (2/6) passtest.py:PassTest.test;intel-virtio-3d4e: PASS (0.02 s) (3/6) passtest.py:PassTest.test;amd-scsi-fa43: PASS (0.02 s) (4/6) passtest.py:PassTest.test;amd-virtio-a59a: PASS (0.02 s) (5/6) passtest.py:PassTest.test;arm-scsi-1c14: PASS (0.03 s) (6/6) passtest.py:PassTest.test;arm-virtio-5ce1: PASS (0.04 s) RESULTS : PASS 6 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0 JOB TIME : 0.51 s JOB HTML : $HOME/avocado/job-results/job-2018-02-09T14.39-f202273/results.html

9.6 Varianter avocado.core.varianter.Varianter Is an internal object which is used to interact with the variants mechanism in Avocado. It’s lifecycle is compound of two stages. First it allows the core/plugins to inject default values, then it is parsed and only allows querying for values, number of variants and such. Example workﬂow of avocado run passtest.py -m example.yaml is: avocado run passtest.py -m example.yaml | + parser.finish -> Varianter.__init__ // dispatcher initializes all plugins | + $PLUGIN -> args.default_avocado_params.add_default_param // could be used to

˓→representation to log | + runner.run_suite -> Varianter.get_number_of_tests | + runner._iter_variants -> Varianter.itertests // Yields variants

In order to allow force-updating the Varianter it supports ignore_new_data, which can be used to ignore new data. This is used by Job Replay to replace the current run Varianter with the one loaded from the replayed job. The workﬂow with ignore_new_data could look like this: avocado run --replay latest -m example.yaml | + $PLUGIN -> args.default_avocado_params.add_default_param // could be used to

˓→insert default values | + replay.run -> Varianter.is_parsed | (continues on next page)

66 Chapter 9. Test parameters avocado Documentation, Release 63.0

(continued from previous page) + replay.run // Varianter object is replaced with the replay job's one | // Varianter.ignore_new_data is set | + $PLUGIN -> args.default_avocado_params.add_default_param // is ignored as new

˓→data are not accepted | + job.run_tests -> Varianter.is_parsed | + job._log_variants -> Varianter.to_str | + runner.run_suite -> Varianter.get_number_of_tests | + runner._iter_variants -> Varianter.itertests

The Varianter itself can only produce an empty variant with the Default params, but it invokes all Varianter plugins and if any of them reports variants it yields them instead of the default variant.

9.7 Default params

The Default params is a mechanism to specify default values in Varianter or Varianter plugins. Their purpose is usually to deﬁne values dependent on the system which should not affect the test’s results. One example is a qemu binary location which might differ from one host to another host, but in the end they should result in qemu being executable in test. For this reason the Default params do not affects the test’s variant-id (at least not in the ofﬁcial Varianter plugins). These params can be set from plugin/core by getting default_avocado_params from args and using:

default_avocado_params.add_default_parma(self, name, key, value, path=None)

Where: • name - name of the plugin which injects data (not yet used for anything, but we plan to allow white/black listing) • key - the parameter’s name • value - the parameter’s value • path - the location of this parameter. When the path does not exists yet, it’s created out of TreeNode.

9.8 Varianter plugins

avocado.core.plugin_interfaces.Varianter A plugin interface that can be used to build custom plugins which are used by Varianter to get test variants. For inspiration see avocado_varianter_yaml_to_mux.YamlToMux which is an optional varianter plugin. Details about this plugin can be found here Yaml_to_mux plugin.

9.9 Multiplexer

avocado.core.mux Multiplexer or simply Mux is an abstract concept, which was the basic idea behind the tree-like params structure with the support to produce all possible variants. There is a core implementation of basic building blocks

9.7. Default params 67 avocado Documentation, Release 63.0

that can be used when creating a custom plugin. There is a demonstration version of plugin using this concept in avocado_varianter_yaml_to_mux which adds a parser and then uses this multiplexer concept to deﬁne an avocado plugin to produce variants from yaml (or json) ﬁles.

9.9.1 Multiplexer concept

As mentioned earlier, this is an in-core implementation of building blocks intended for writing Varianter plugins based on a tree with Multiplex domains defined. The available blocks are: • MuxTree - Object which represents a part of the tree and handles the multiplexation, which means producing all possible variants from a tree-like object. • MuxPlugin - Base class to build Varianter plugins • MuxTreeNode - Inherits from TreeNode and adds the support for control flags (MuxTreeNode.ctrl) and multiplex domains (MuxTreeNode.multiplex). And some support classes and methods eg. for filtering and so on.

9.10 Multiplex domains

A default AvocadoParams tree with variables could look like this:

Multiplex tree representation: paths → tmp: /var/tmp → qemu: /usr/libexec/qemu-kvm environ → debug: False

The multiplexer wants to produce similar structure, but also to be able to define not just one variant, but to define all possible combinations and then report the slices as variants. We use the term Multiplex domains to define that children of this node are not just different paths, but they are different values and we only want one at a time. In the representation we use double-line to visibily distinguish between normal relation and multiplexed relation. Let’s modify our example a bit:

Multiplex tree representation: paths → tmp: /var/tmp → qemu: /usr/libexec/qemu-kvm environ production → debug: False debug → debug: True

The difference is that environ is now a multiplex node and it’s children will be yielded one at a time producing two variants:

Variant 1: paths → tmp: /var/tmp → qemu: /usr/libexec/qemu-kvm environ production (continues on next page)

68 Chapter 9. Test parameters avocado Documentation, Release 63.0

(continued from previous page) → debug: False Variant 2: paths → tmp: /var/tmp → qemu: /usr/libexec/qemu-kvm environ debug → debug: False

Note that the multiplex is only about direct children, therefore the number of leaves in variants might differ:

Multiplex tree representation: paths → tmp: /var/tmp → qemu: /usr/libexec/qemu-kvm environ production → debug: False debug system → debug: False program → debug: True

Produces one variant with /paths and /environ/production and other variant with /paths, /environ/ debug/system and /environ/debug/program. As mentioned earlier the power is not in producing one variant, but in deﬁning huge scenarios with all possible variants. By using tree-structure with multiplex domains you can avoid most of the ugly ﬁlters you might know from Jenkin’s sparse matrix jobs. For comparison let’s have a look at the same example in avocado:

Multiplex tree representation: os distro redhat fedora version 20 21 flavor workstation cloud rhel 5 6 arch i386 x86_64

Which produces:

Variant1:/os/distro/redhat/fedora/version/20,/os/distro/redhat/fedora/flavor/

˓→workstation,/os/arch/i386 Variant2:/os/distro/redhat/fedora/version/20,/os/distro/redhat/fedora/flavor/

˓→workstation,/os/arch/x86_64 Variant3:/os/distro/redhat/fedora/version/20,/os/distro/redhat/fedora/flavor/

˓→cloud,/os/arch/i386 (continues on next page)

9.10. Multiplex domains 69 avocado Documentation, Release 63.0

(continued from previous page) Variant4:/os/distro/redhat/fedora/version/20,/os/distro/redhat/fedora/flavor/

˓→cloud,/os/arch/x86_64 Variant5:/os/distro/redhat/fedora/version/21,/os/distro/redhat/fedora/flavor/

˓→workstation,/os/arch/i386 Variant6:/os/distro/redhat/fedora/version/21,/os/distro/redhat/fedora/flavor/

˓→workstation,/os/arch/x86_64 Variant7:/os/distro/redhat/fedora/version/21,/os/distro/redhat/fedora/flavor/

˓→cloud,/os/arch/i386 Variant8:/os/distro/redhat/fedora/version/21,/os/distro/redhat/fedora/flavor/

˓→cloud,/os/arch/x86_64 Variant9:/os/distro/redhat/rhel/5,/os/arch/i386 Variant 10:/os/distro/redhat/rhel/5,/os/arch/x86_64 Variant 11:/os/distro/redhat/rhel/6,/os/arch/i386 Variant 12:/os/distro/redhat/rhel/6,/os/arch/x86_64

Versus Jenkin’s sparse matrix:

os_version = fedora20 fedora21 rhel5 rhel6 os_flavor = none workstation cloud arch = i386 x86_64

filter = ((os_version == "rhel5").implies(os_flavor == "none") && (os_version == "rhel6").implies(os_flavor == "none")) && !(os_version == "fedora20" && os_flavor == "none") && !(os_version == "fedora21" && os_flavor == "none")

Which is still relatively simple example, but it grows dramatically with inner-dependencies.

9.11 MuxPlugin

avocado.core.mux.MuxPlugin Deﬁnes the full interface required by avocado.core.plugin_interfaces.Varianter. The plugin writer should inherit from this MuxPlugin, then from the Varianter and call the:

self.initialize_mux(root, paths, debug)

Where: • root - is the root of your params tree (compound of TreeNode -like nodes) • paths - is the Parameter paths to be used in test with all variants • debug - whether to use debug mode (requires the passed tree to be compound of TreeNodeDebug-like nodes which stores the origin of the variant/value/environment as the value for listing purposes and is __NOT__ intended for test execution. This method must be called before the Varianter’s second stage (the latest opportunity is during self. update_defaults). The MuxPlugin’s code will take care of the rest.

9.12 MuxTree

This is the core feature where the hard work happens. It walks the tree and remembers all leaf nodes or uses list of MuxTrees when another multiplex domain is reached while searching for a leaf.

70 Chapter 9. Test parameters avocado Documentation, Release 63.0

When it’s asked to report variants, it combines one variant of each remembered item (leaf node always stays the same, but MuxTree circles through it’s values) which recursively produces all possible variants of different multiplex domains.

9.12. MuxTree 71 avocado Documentation, Release 63.0

72 Chapter 9. Test parameters CHAPTER 10

Job Replay

In order to reproduce a given job using the same data, one can use the --replay option for the run command, informing the hash id from the original job to be replayed. The hash id can be partial, as long as the provided part corresponds to the initial characters of the original job id and it is also unique enough. Or, instead of the job id, you can use the string latest and avocado will replay the latest job executed. Let’s see an example. First, running a simple job with two test references:

$ avocado run /bin/true /bin/false JOB ID : 825b860b0c2f6ec48953c638432e3e323f8d7cad JOB LOG : $HOME/avocado/job-results/job-2016-01-11T16.14-825b860/job.log (1/2) /bin/true: PASS (0.01 s) (2/2) /bin/false: FAIL (0.01 s) RESULTS : PASS 1 | ERROR 0 | FAIL 1 | SKIP 0 | WARN 0 | INTERRUPT 0 JOB TIME : 0.12 s JOB HTML : $HOME/avocado/job-results/job-2016-01-11T16.14-825b860/html/results.html

Now we can replay the job by running:

$ avocado run --replay 825b86 JOB ID : 55a0d10132c02b8cc87deb2b480bfd8abbd956c3 SRC JOB ID : 825b860b0c2f6ec48953c638432e3e323f8d7cad JOB LOG : $HOME/avocado/job-results/job-2016-01-11T16.18-55a0d10/job.log (1/2) /bin/true: PASS (0.01 s) (2/2) /bin/false: FAIL (0.01 s) RESULTS : PASS 1 | ERROR 0 | FAIL 1 | SKIP 0 | WARN 0 | INTERRUPT 0 JOB TIME : 0.11 s JOB HTML : $HOME/avocado/job-results/job-2016-01-11T16.18-55a0d10/html/results.html

The replay feature will retrieve the original test references, the variants and the conﬁguration. Let’s see another example, now using a mux YAML ﬁle:

$ avocado run /bin/true /bin/false --mux-yaml mux-environment.yaml JOB ID : bd6aa3b852d4290637b5e771b371537541043d1d JOB LOG : $HOME/avocado/job-results/job-2016-01-11T21.56-bd6aa3b/job.log (continues on next page)

73 avocado Documentation, Release 63.0

(continued from previous page) (1/4) /bin/true;first-c49a: PASS (0.01 s) (2/4) /bin/true;second-f05f: PASS (0.01 s) (3/4) /bin/false;first-c49a: FAIL (0.04 s) (4/4) /bin/false;second-f05f: FAIL (0.04 s) RESULTS : PASS 2 | ERROR 0 | FAIL 2 | SKIP 0 | WARN 0 | INTERRUPT 0 JOB TIME : 0.19 s JOB HTML : $HOME/avocado/job-results/job-2016-01-11T21.56-bd6aa3b/html/results.html

We can replay the job as is, using $ avocado run --replay latest, or replay the job ignoring the variants, as below:

$ avocado run --replay bd6aa3b --replay-ignore variants Ignoring variants from source job with --replay-ignore. JOB ID : d5a46186ee0fb4645e3f7758814003d76c980bf9 SRC JOB ID : bd6aa3b852d4290637b5e771b371537541043d1d JOB LOG : $HOME/avocado/job-results/job-2016-01-11T22.01-d5a4618/job.log (1/2) /bin/true: PASS (0.01 s) (2/2) /bin/false: FAIL (0.01 s) RESULTS : PASS 1 | ERROR 0 | FAIL 1 | SKIP 0 | WARN 0 | INTERRUPT 0 JOB TIME : 0.12 s JOB HTML : $HOME/avocado/job-results/job-2016-01-11T22.01-d5a4618/html/results.html

Also, it is possible to replay only the variants that faced a given result, using the option --replay-test-status. See the example below:

$ avocado run --replay bd6aa3b --replay-test-status FAIL JOB ID : 2e1dc41af6ed64895f3bb45e3820c5cc62a9b6eb SRC JOB ID : bd6aa3b852d4290637b5e771b371537541043d1d JOB LOG : $HOME/avocado/job-results/job-2016-01-12T00.38-2e1dc41/job.log (1/4) /bin/true;first-c49a: SKIP (2/4) /bin/true;second-f05f: SKIP (3/4) /bin/false;first-c49a: FAIL (0.03 s) (4/4) /bin/false;second-f05f: FAIL (0.04 s) RESULTS : PASS 0 | ERROR 0 | FAIL 24 | SKIP 24 | WARN 0 | INTERRUPT 0 JOB TIME : 0.29 s JOB HTML : $HOME/avocado/job-results/job-2016-01-12T00.38-2e1dc41/html/results.html

Of which one special example is --replay-test-status INTERRUPTED or simply --replay-resume, which SKIPs the executed tests and only executes the ones which were CANCELED or not executed after a CAN- CELED test. This feature should work even on hard interruptions like system crash. When replaying jobs that were executed with the --failfast on option, you can disable the failfast option using --failfast off in the replay job. To be able to replay a job, avocado records the job data in the same job results directory, inside a subdirectory named replay. If a given job has a non-default path to record the logs, when the replay time comes, we need to inform where the logs are. See the example below:

$ avocado run /bin/true --job-results-dir /tmp/avocado_results/ JOB ID : f1b1c870ad892eac6064a5332f1bbe38cda0aaf3 JOB LOG : /tmp/avocado_results/job-2016-01-11T22.10-f1b1c87/job.log (1/1) /bin/true: PASS (0.01 s) RESULTS : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 JOB TIME : 0.11 s JOB HTML : /tmp/avocado_results/job-2016-01-11T22.10-f1b1c87/html/results.html

Trying to replay the job, it fails:

74 Chapter 10. Job Replay avocado Documentation, Release 63.0

$ avocado run --replay f1b1 can't find job results directory in '$HOME/avocado/job-results'

In this case, we have to inform where the job results directory is located:

$ avocado run --replay f1b1 --replay-data-dir /tmp/avocado_results JOB ID : 19c76abb29f29fe410a9a3f4f4b66387570edffa SRC JOB ID : f1b1c870ad892eac6064a5332f1bbe38cda0aaf3 JOB LOG : $HOME/avocado/job-results/job-2016-01-11T22.15-19c76ab/job.log (1/1) /bin/true: PASS (0.01 s) RESULTS : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 JOB TIME : 0.11 s JOB HTML : $HOME/avocado/job-results/job-2016-01-11T22.15-19c76ab/html/results.html

75 avocado Documentation, Release 63.0

76 Chapter 10. Job Replay CHAPTER 11

Job Diff

Avocado Diff plugin allows users to easily compare several aspects of two given jobs. The basic usage is:

$ avocado diff 7025aaba 384b949c --- 7025aaba9c2ab8b4bba2e33b64db3824810bb5df +++ 384b949c991b8ab324ce67c9d9ba761fd07672ff @@ -1,15 +1,15 @@

COMMAND LINE -/usr/bin/avocado run sleeptest.py +/usr/bin/avocado run passtest.py

TOTAL TIME -1.00 s +0.00 s

TEST RESULTS -1-sleeptest.py:SleepTest.test: PASS +1-passtest.py:PassTest.test: PASS

...

Avocado Diff can compare and create an uniﬁed diff of: • Command line. • Job time. • Variants and parameters. • Tests results. • Conﬁguration. • Sysinfo pre and post. Only sections with different content will be included in the results. You can also enable/disable those sections with --diff-filter. Please see avocado diff --help for more information.

77 avocado Documentation, Release 63.0

Jobs can be identiﬁed by the Job ID, by the results directory or by the key latest. Example:

$ avocado diff ~/avocado/job-results/job-2016-08-03T15.56-4b3cb5b/ latest --- 4b3cb5bbbb2435c91c7b557eebc09997d4a0f544 +++ 57e5bbb3991718b216d787848171b446f60b3262 @@ -1,9 +1,9 @@

COMMAND LINE -/usr/bin/avocado run perfmon.py +/usr/bin/avocado run passtest.py

TOTAL TIME -11.91 s +0.00 s

TEST RESULTS -1-test.py:Perfmon.test: FAIL +1-examples/tests/passtest.py:PassTest.test: PASS

Along with the uniﬁed diff, you can also generate the html (option --html) diff ﬁle and, optionally, open it on your preferred browser (option --open-browser):

$ avocado diff 7025aaba 384b949c --html /tmp/myjobdiff.html /tmp/myjobdiff.html

If the option --open-browser is used without the --html, we will create a temporary html file. For those wiling to use a custom diff tool instead of the Avocado Diff tool, we offer the option --create-reports, so we create two temporary files with the relevant content. The file names are printed and user can copy/paste to the custom diff tool command line:

$ avocado diff 7025aaba 384b949c --create-reports /var/tmp/avocado_diff_7025aab_zQJjJh.txt /var/tmp/avocado_diff_384b949_AcWq02.txt

$ diff -u /var/tmp/avocado_diff_7025aab_zQJjJh.txt /var/tmp/avocado_diff_384b949_

˓→AcWq02.txt --- /var/tmp/avocado_diff_7025aab_zQJjJh.txt 2016-08-10 21:48:43.547776715 +0200 +++ /var/tmp/avocado_diff_384b949_AcWq02.txt 2016-08-10 21:48:43.547776715 +0200 @@ -1,250 +1,19 @@

COMMAND LINE ======-/usr/bin/avocado run sleeptest.py +/usr/bin/avocado run passtest.py

TOTAL TIME ======-1.00 s +0.00 s

...

78 Chapter 11. Job Diff CHAPTER 12

Running Tests Remotely

The Avocado results are easily copyable between machines, so you can manually run your jobs on various machines and copy the results dir to your main machines while still being able to re-run/diff/. . . them. Anyway if you want even smoother remote execution, you can use our Remote runner plugins optional plugins to run jobs on a remote systems (machine, container, . . . ) reporting the results locally.

79 avocado Documentation, Release 63.0

80 Chapter 12. Running Tests Remotely CHAPTER 13

Subclassing Avocado

Subclassing Avocado Test class to extend its features is quite straight forward and it might constitute a very useful resource to have some shared/recurrent code hosted in your project repository. In this documentation we propose an project organization that will allow you to create and install your so called sub-framework. Let’s use, as an example, a project called Apricot Framework. Here’s the proposed ﬁlesystem structure:

~/git/apricot (master)$ tree . apricot __init__.py test.py README.rst setup.py tests test_example.py VERSION

• setup.py: In the setup.py it is important to specify the avocado-framework package as a dependency:

from setuptools import setup, find_packages

setup(name='apricot', description='Apricot - Avocado SubFramwork', version=open("VERSION","r").read().strip(), author='Apricot Developers', author_email='[email protected]', packages=['apricot'], include_package_data=True, install_requires=['avocado-framework'] )

• VERSION: Version your project as you wish:

81 avocado Documentation, Release 63.0

1.0

• apricot/__init__.py: Make your new test class available in your module root:

__all__=['ApricotTest']

from apricot.test import ApricotTest

• apricot/test.py: Here you will be basically extending the Avocado Test class with your own methods and routines:

from avocado import Test

class ApricotTest(Test): def setUp(self): self.log.info("setUp() executed from Apricot")

def some_useful_method(self): return True

• tests/test_example.py: And this is how your test will look like. The most important item here is to use the docstring :avocado: recursive, so the Avocado test loader will be able to recognize your test class as an Avocado Test class:

from apricot import ApricotTest

class MyTest(ApricotTest): """ :avocado: recursive """ def test(self): self.assertTrue(self.some_useful_method())

To (non-intrusively) install your module, use:

~/git/apricot (master)$ python setup.py develop --user running develop running egg_info writing requirements to apricot.egg-info/requires.txt writing apricot.egg-info/PKG-INFO writing top-level names to apricot.egg-info/top_level.txt writing dependency_links to apricot.egg-info/dependency_links.txt reading manifest file 'apricot.egg-info/SOURCES.txt' writing manifest file 'apricot.egg-info/SOURCES.txt' running build_ext Creating /home/apahim/.local/lib/python2.7/site-packages/apricot.egg-link (link to .) apricot 1.0 is already the active version in easy-install.pth

Installed /home/apahim/git/apricot Processing dependencies for apricot==1.0 Searching for avocado-framework==55.0 Best match: avocado-framework 55.0 avocado-framework 55.0 is already the active version in easy-install.pth

Using /home/apahim/git/avocado Searching for stevedore==1.25.0 Best match: stevedore 1.25.0 (continues on next page)

82 Chapter 13. Subclassing Avocado avocado Documentation, Release 63.0

(continued from previous page) Adding stevedore 1.25.0 to easy-install.pth file

Using /usr/lib/python2.7/site-packages Searching for six==1.10.0 Best match: six 1.10.0 Adding six 1.10.0 to easy-install.pth file

Using /usr/lib/python2.7/site-packages Searching for pbr==3.1.1 Best match: pbr 3.1.1 Adding pbr 3.1.1 to easy-install.pth file Installing pbr script to /home/apahim/.local/bin

Using /usr/lib/python2.7/site-packages Finished processing dependencies for apricot==1.0

And to run your test:

~/git/apricot$ avocado run tests/test_example.py JOB ID : 02c663eb77e0ae6ce67462a398da6972791793bf JOB LOG : $HOME/avocado/job-results/job-2017-11-16T12.44-02c663e/job.log (1/1) tests/test_example.py:MyTest.test: PASS (0.03 s) RESULTS : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0 JOB TIME : 0.95 s JOB HTML : $HOME/avocado/job-results/job-2017-11-16T12.44-02c663e/results.html

83 avocado Documentation, Release 63.0

84 Chapter 13. Subclassing Avocado CHAPTER 14

Debugging with GDB

Avocado has two different types of GDB support that complement each other: • Transparent execution of executables inside the GNU Debugger. This takes standard and possibly unmodiﬁed tests that uses the avocado.utils.process APIs for running processes. By using a command line option, the executable is run on GDB. This allows the user to interact with GDB, but to the test itself, things are pretty much transparent. • The avocado.utils.gdb APIs that allows a test to interact with GDB, including setting a executable to be run, setting breakpoints or any other types of commands. This requires a test written with that approach and API in mind.

Tip: Even though this section describes the use of the Avocado GDB features, which allow live debugging of binaries inside Avocado tests, it’s also possible to debug some application ofﬂine by using tools such as rr. Avocado ships with an example wrapper script (to be used with --wrapper) for that purpose.

14.1 Transparent Execution of Executables

This feature adds a few command line options to the Avocado run command:

$ avocado run --help ... GNU Debugger support:

--gdb-run-bin EXECUTABLE[:BREAKPOINT] Run a given executable inside the GNU debugger, pausing at a given breakpoint (defaults to "main") --gdb-prerun-commands EXECUTABLE:COMMANDS After loading an executable in GDB, but before actually running it, execute the GDB commands in the given file. EXECUTABLE is optional, if omitted (continues on next page)

85 avocado Documentation, Release 63.0

(continued from previous page) COMMANDS will apply to all executables --gdb-coredump {on,off} Automatically generate a core dump when the inferior process received a fatal signal such as SIGSEGV or SIGABRT ...

To get started you want to use --gdb-run-bin, as shown in the example bellow.

14.1.1 Example

The simplest way is to just run avocado run --gdb-run-bin=doublefree examples/tests/ doublefree.py, which wraps each executed executable with name doublefree inside GDB server and stops at the executable entry point. Optionally you can specify single breakpoint using --gdb-run-bin=doublefree:$breakpoint (eg: doublefree:1) or just doublefree: to stop only when an interruption happens (eg: SIGABRT). It’s worth mentioning that when breakpoint is not reached, the test ﬁnishes without any interruption. This is helpful when you identify regions where you should never get in your code, or places which interests you and you can run your code in production and GDB variants. If after a long time you get to this place, the test notiﬁes you and you can investigate the problem. This is demonstrated in examples/tests/doublefree_nasty.py test. To unveil the power of Avocado, run this test using:

avocado run--gdb-run-bin=doublefree: examples/tests/doublefree_nasty.py--gdb-prerun-

˓→commands examples/tests/doublefree_nasty.py.data/gdb_pre--mux-yaml examples/tests/

˓→doublefree_nasty.py.data/iterations.yaml

which executes 100 iterations of this test while setting all breakpoints from the examples/tests/ doublefree_nasty.py.data/gdb_pre ﬁle (you can specify whatever GDB supports, not only breakpoints). As you can see this test usually passes, but once in a while it gets into the problematic area. Imagine this is very hard to spot (dependent on HW registers, . . . ) and this is one way to combine regular testing and the possibility of debugging hard-to-get parts of your code.

14.1.2 Caveats

Currently, when using the Avocado GDB plugin, that is, when using the –gdb-run-bin option, there are some caveats you should be aware of: • It is not currently compatible with Avocado’s –output-check-record feature • There’s no way to perform proper input to the process, that is, manipulate its STDIN • The process STDERR content is mixed with the content generated by gdbserver on its own STDERR (because they are in fact, the same thing) But, you can still depend on the process STDOUT, as exempliﬁed by this ﬁctional test:

from avocado import Test from avocado.utils import process

class HelloOutputTest(Test):

def test(self): (continues on next page)

86 Chapter 14. Debugging with GDB avocado Documentation, Release 63.0

(continued from previous page) result= process.run("/path/to/hello", ignore_status= True) self.assertIn("hello\n", result.stdout)

If run under GDB or not, result.stdout behavior and content is expected to be the same.

14.1.3 Reasons for the caveats

There are a two basic reasons for the mentioned caveats: • The architecture of Avocado’s GDB feature • GDB’s own behavior and limitations When using the Avocado GDB plugin, that is, –gdb-run-bin, Avocado runs a gdbserver instance transparently and controls it by means of a gdb process. When a given event happens, say a breakpoint is reached, it disconnects its own gdb from the server, and allows the user to use a standard gdb to connect to the gdbserver. This provides a natural and seamless user experience. But, gdbserver has some limitations at this point, including: • Not being able to set a controlling tty • Not separating its own STDERR content from the application being run These limitations are being addressed both on Avocado and GDB, and will be resolved in future Avocado versions.

14.1.4 Workaround

If the application you’re running as part of your test can read input from alternative sources (including devices, ﬁles or the network) and generate output likewise, then you should not be further limited.

14.1.5 GDB support and avocado-virt

Another current limitation is the use of avocado-virt and avocado GDB support. The supported API for transparent debugging is currently limited to avocado.utils.process.run(), and does not cover advanced uses of the avocado.utils.process.SubProcess class. The avocado-virt extension, though, uses avocado.utils.process.SubProcess class to execute qemu in the background. This limitation will be addressed in future versions of avocado and avocado-virt.

14.2 avocado.utils.gdb APIs

Avocado’s GDB module, provides three main classes that lets a test writer interact with a gdb process, a gdbserver process and also use the GDB remote protocol for interaction with a remote target. Please refer to avocado.utils.gdb for more information.

14.2.1 Example

Take a look at examples/tests/modify_variable.py test:

14.2. avocado.utils.gdb APIs 87 avocado Documentation, Release 63.0

def test(self): """ Execute 'print_variable'. """ path= os.path.join(self.workdir,'print_variable') app= gdb.GDB() app.set_file(path) app.set_break(6) app.run() self.log.info("\n".join(app.read_until_break())) app.cmd("set variable a = 0xff") app.cmd("c") out=" \n".join(app.read_until_break()) self.log.info(out) app.exit() self.assertIn("MY VARIABLE'A' IS: ff", out)

You can see that instead of running the executable using process.run we invoke avocado.utils.gdb.GDB. This allows us to automate the interaction with the GDB in means of setting breakpoints, executing commands and querying for output. When you check the output (--show-job-log) you can see that despite declaring the variable as 0, ff is injected and printed instead.

88 Chapter 14. Debugging with GDB CHAPTER 15

Wrap executables run by tests

Avocado allows the instrumentation of executables being run by a test in a transparent way. The user speciﬁes a script (“the wrapper”) to be used to run the actual program called by the test. If the instrumentation script is implemented correctly, it should not interfere with the test behavior. That is, the wrapper should avoid changing the return status, standard output and standard error messages of the original executable. The user can be speciﬁc about which program to wrap (with a shell-like glob), or if that is omitted, a global wrapper that will apply to all programs called by the test.

15.1 Usage

This feature is implemented as a plugin, that adds the –wrapper option to the Avocado run command. For a detailed explanation, please consult the Avocado man page. Example of a transparent way of running strace as a wrapper:

#!/bin/sh exec strace -ff -o $AVOCADO_TEST_LOGDIR/strace.log -- $@

To have all programs started by test.py wrapped with ~/bin/my-wrapper.sh:

$ scripts/avocado run --wrapper ~/bin/my-wrapper.sh tests/test.py

To have only my-binary wrapped with ~/bin/my-wrapper.sh:

$ scripts/avocado run --wrapper ~/bin/my-wrapper.sh:*my-binary tests/test.py

15.2 Caveats

• It is not possible to debug with GDB (–gdb-run-bin) and use wrappers (–wrapper) at the same time. These two options are mutually exclusive.

89 avocado Documentation, Release 63.0

• You can only set one (global) wrapper. If you need functionality present in two wrappers, you have to combine those into a single wrapper script. • Only executables that are run with the avocado.utils.process APIs (and other API modules that make use of it, like mod:avocado.utils.build) are affected by this feature.

90 Chapter 15. Wrap executables run by tests CHAPTER 16

Plugin System

Avocado has a plugin system that can be used to extended it in a clean way.

16.1 Listing plugins

The avocado command line tool has a builtin plugins command that lets you list available plugins. The usage is pretty simple:

$ avocado plugins Plugins that add new commands (avocado.plugins.cli.cmd): exec-path Returns path to avocado bash libraries and exits. run Run one or more tests (native test, test alias, binary or script) sysinfo Collect system information ... Plugins that add new options to commands (avocado.plugins.cli): remote Remote machine options for 'run' subcommand journal Journal options for the 'run' subcommand ...

Since plugins are (usually small) bundles of Python code, they may fail to load if the Python code is broken for any reason. Example:

$ avocado plugins Failed to load plugin from module "avocado.plugins.exec_path": ImportError('No module

˓→named foo',) Plugins that add new commands (avocado.plugins.cli.cmd): run Run one or more tests (native test, test alias, binary or script) sysinfo Collect system information ...

91 avocado Documentation, Release 63.0

16.2 Writing a plugin

What better way to understand how an Avocado plugin works than creating one? Let’s use another old time favorite for that, the “Print hello world” theme.

16.2.1 Code example

Let’s say you want to write a plugin that adds a new subcommand to the test runner, hello. This is how you’d do it:

from avocado.core.output import LOG_JOB from avocado.core.plugin_interfaces import CLICmd

class HelloWorld(CLICmd):

name='hello' description='The classical Hello World! plugin example.'

def run(self, args): LOG_JOB.info(self.description)

As you can see, this plugins inherits from avocado.core.plugin_interfaces.CLICmd. This speciﬁc base class allows for the creation of new commands for the Avocado CLI tool. The only mandatory method to be implemented is run and it’s the plugin main entry point. This plugin uses avocado.core.output.LOG_JOB to produce the hello world output in the Job log. One can also use avocado.core.output.LOG_UI to produce output in the human readable output.

16.2.2 Registering Plugins

Avocado makes use of the Stevedore library to load and activate plugins. Stevedore itself uses setuptools and its entry points to register and ﬁnd Python objects. So, to make your new plugin visible to Avocado, you need to add to your setuptools based setup.py ﬁle something like:

setup(name='mypluginpack', ... entry_points={ 'avocado.plugins.cli':[ 'hello = mypluginpack.hello:HelloWorld', ] } ...

Then, by running either $ python setup.py install or $ python setup.py develop your plugin should be visible to Avocado.

16.2.3 Fully qualiﬁed named for a plugin

The plugin registry mentioned earlier, (setuptools and its entry points) is global to a given Python installation. Avocado uses the namespace prefix avocado.plugins. to avoid name clashes with other software. Now, inside Avocado itself, there’s no need keep using the avocado.plugins. prefix. Take for instance, the Job Pre/Post plugins are defined on setup.py:

92 Chapter 16. Plugin System avocado Documentation, Release 63.0

'avocado.plugins.job.prepost':[ 'jobscripts = avocado.plugins.jobscripts:JobScripts' ]

The setuptools entry point namespace is composed of the mentioned prefix avocado.plugins., which is is then followed by the Avocado plugin type, in this case, job.prepost. Inside avocado itself, the fully qualified name for a plugin is the plugin type, such as job.prepost concatenated to the name used in the entry point definition itself, in this case, jobscripts. To summarize, still using the same example, the fully qualified Avocado plugin name is going to be job.prepost. jobscripts.

16.2.4 Disabling a plugin

Even though a plugin can be installed and registered under setuptools entry points, it can be explicitly disabled in Avocado. The mechanism available to do so is to add entries to the disable key under the plugins section of the Avocado conﬁguration ﬁle. Example:

[plugins] disable=['cli.hello','job.prepost.jobscripts']

The exact effect on Avocado when a plugin is disabled depends on the plugin type. For instance, by disabling plugins of type cli.cmd, the command implemented by the plugin should no longer be available on the Avocado command line application. Now, by disabling a job.prepost plugin, those won’t be executed before/after the execution of the jobs.

16.2.5 Default plugin execution order

In many situations, such as result generation, not one, but all of the enabled plugin types will be executed. The order in which the plugins are executed follows the lexical order of the entry point name. For example, for the JSON result plugin, whose fully qualiﬁed name is result.json, has an entry point name of json, as can be seen on its registration code in setup.py:

... entry_points={ 'avocado.plugins.result':[ 'json = avocado.plugins.jsonresult:JSONResult', ...

If it sounds too complicated, it isn’t. It just means that for plugins of the same type, a plugin named automated will be executed before the plugin named uploader. In the default Avocado set of result plugins, it means that the JSON plugin (json) will be executed before the XUnit plugin (xunit). If the HTML result plugin is installed and enabled (html) it will be executed before both JSON and XUnit.

16.2.6 Conﬁguring the plugin execution order

On some circumstances it may be necessary to change the order in which plugins are executed. To do so, add a order entry a conﬁguration ﬁle section named after the plugin type. For job.prepost plugin types, the section name has to be named plugins.job.prepost, and it would look like this:

16.2. Writing a plugin 93 avocado Documentation, Release 63.0

[plugins.job.prepost] order=['myplugin','jobscripts']

That conﬁguration sets the job.prepost.myplugin plugin to execute before the standard Avocado job. prepost.jobscripts does.

16.2.7 Wrap Up

We have briefly discussed the making of Avocado plugins. We recommend the Stevedore documentation and also a look at the avocado.core.plugin_interfaces module for the various plugin interface definitions. Some plugins examples are available in the Avocado source tree, under examples/plugins. Finally, exploring the real plugins shipped with Avocado in avocado.plugins is the final “documentation” source.

94 Chapter 16. Plugin System CHAPTER 17

Utilities

17.1 Utilities

The following pages are the documentation for some of the Avocado utilities:

17.1.1 vmimage

This utility provides a API to download/cache VM images (QCOW) from the ofﬁcial distributions repositories.

Basic Usage

Import vmimage module:

>>> from avocado.utils import vmimage

Get an image, which consists in an object with the path of the dowloaded/cached base image and the path of the external snapshot created out of that base image:

>>> image= vmimage.get() >>> image >>> image.name 'Fedora' >>> image.path '/tmp/Fedora-Cloud-Base-26-1.5.x86_64-d369c285.qcow2' >>> image.get() '/tmp/Fedora-Cloud-Base-26-1.5.x86_64-e887c743.qcow2' >>> image.path '/tmp/Fedora-Cloud-Base-26-1.5.x86_64-e887c743.qcow2' >>> image.version 26 (continues on next page)

95 avocado Documentation, Release 63.0

(continued from previous page) >>> image.base_image '/tmp/Fedora-Cloud-Base-26-1.5.x86_64.qcow2'

If you provide more details about the image, the object is expected to reﬂect those details:

>>> image= vmimage.get(arch='aarch64') >>> image >>> image.name 'FedoraSecondary' >>> image.path '/tmp/Fedora-Cloud-Base-26-1.5.aarch64-07b8fbda.qcow2'

>>> image= vmimage.get(version=7) >>> image >>> image.path '/tmp/CentOS-7-x86_64-GenericCloud-1708-dd8139c5.qcow2'

Notice that, unlike the base_image attribute, the path attribute will be always different in each instance, as it actually points to an external snapshot created out of the base image:

>>> i1= vmimage.get() >>> i2= vmimage.get() >>> i1.path == i2.path False

Custom Image Provider

If you need your own Image Provider, you can extend the vmimage.IMAGE_PROVIDERS list, including your provider class. For instance, using the vmimage utility in an Avocado test, we could add our own provider with: from avocado import Test from avocado.utils import vmimage class MyProvider(vmimage.ImageProviderBase):

name='MyDistro'

def __init__(self, version='[0-9]+', build='[0-9]+.[0-9]+', arch=os.uname()[4]): """ :params version: The regular expression that represents your distro version numbering. :params build: The regular expression that represents your build version numbering. :params arch: The default architecture to look images for. """ super(MyProvider, self).__init__(version, build, arch)

# The URL which contains a list of the distro versions self.url_versions='https://dl.fedoraproject.org/pub/fedora/linux/releases/'

# The URL which contains a list of distro images (continues on next page)

96 Chapter 17. Utilities avocado Documentation, Release 63.0

(continued from previous page) self.url_images= self.url_versions+' {version}/CloudImages/{arch}/images/'

# The images naming pattern self.image_pattern='Fedora-Cloud-Base- {version}-{build}.{arch}.qcow2$' class MyTest(Test):

def setUp(self): vmimage.IMAGE_PROVIDERS.add(MyProvider) image= vmimage.get('MyDistro') ...

def test(self): ...

17.1. Utilities 97 avocado Documentation, Release 63.0

98 Chapter 17. Utilities CHAPTER 18

Optional Plugins

18.1 Optional Plugins

The following pages are the documentation for some of the Avocado optional plugins:

18.1.1 Remote runner plugins

There are currently three optional plugins to help you run your tests remotely: • Running Tests on a Remote Host - Over SSH • Running Tests on a Virtual Machine - Using libvirt • Running Tests on a Docker container - Using docker

Running Tests on a Remote Host

(avocado-framework-plugin-runner-remote) Avocado lets you run tests directly in a remote machine with SSH connection, provided that you properly set it up by installing Avocado in it. You can check if this feature (a plugin) is enabled by running:

$ avocado plugins ... remote Remote machine options for 'run' subcommand ...

Assuming this feature is enabled, you should be able to pass the following options when using the run command in the Avocado command line tool:

99 avocado Documentation, Release 63.0

--remote-hostname REMOTE_HOSTNAME Specify the hostname to login on remote machine --remote-port REMOTE_PORT Specify the port number to login on remote machine. Default: 22 --remote-username REMOTE_USERNAME Specify the username to login on remote machine --remote-password REMOTE_PASSWORD Specify the password to login on remote machine

From these options, you are normally going to use –remote-hostname and –remote-username in case you did set up your VM with password-less SSH connection (through SSH keys).

Remote Setup

Make sure you have: 1. Avocado packages installed. You can see more info on how to do that in the Getting Started section. 2. The remote machine IP address or fully qualiﬁed hostname and the SSH port number. 3. All pre-requisites for your test to run installed inside the remote machine (gcc, make and others if you want to compile a 3rd party test suite written in C, for example). Optionally, you may have password less SSH login on your remote machine enabled.

Running your test

Once the remote machine is properly set, you may run your test. Example:

$ scripts/avocado run --remote-hostname 192.168.122.30 --remote-username fedora

˓→examples/tests/sleeptest.py examples/tests/failtest.py REMOTE LOGIN : [email protected]:22 JOB ID : 60ddd718e7d7bb679f258920ce3c39ce73cb9779 JOB LOG : $HOME/avocado/job-results/job-2014-10-23T11.45-a329461/job.log (1/2) examples/tests/sleeptest.py: PASS (1.00 s) (2/2) examples/tests/failtest.py: FAIL (0.00 s) RESULTS : PASS 1 | ERROR 0 | FAIL 1 | SKIP 0 | WARN 0 | INTERRUPT 0 JOB TIME : 1.11 s

A bit of extra logging information is added to your job summary, mainly to distinguish the regular execution from the remote one. Note here that we did not need –remote-password because an SSH key was already set.

Running Tests on a Virtual Machine

(avocado-framework-plugin-runner-vm) Sometimes you don’t want to run a given test directly in your own machine (maybe the test is dangerous, maybe you need to run it in another Linux distribution, so on and so forth). For those scenarios, Avocado lets you run tests directly in VMs deﬁned as libvirt domains in your system, provided that you properly set them up. You can check if this feature (a plugin) is enabled by running:

100 Chapter 18. Optional Plugins avocado Documentation, Release 63.0

$ avocado plugins ... vm Virtual Machine options for 'run' subcommand ...

Assuming this feature is enabled, you should be able to pass the following options when using the run command in the Avocado command line tool:

--vm Run tests on Virtual Machine --vm-hypervisor-uri VM_HYPERVISOR_URI Specify hypervisor URI driver connection --vm-domain VM_DOMAIN Specify domain name (Virtual Machine name) --vm-hostname VM_HOSTNAME Specify VM hostname to login. By default Avocado attempts to automatically find the VM IP address. --vm-username VM_USERNAME Specify the username to login on VM --vm-password VM_PASSWORD Specify the password to login on VM --vm-cleanup Restore VM to a previous state, before running the tests

From these options, you are normally going to use –vm-domain, –vm-hostname and –vm-username in case you did set up your VM with password-less SSH connection (through SSH keys). If your VM has the qemu-guest-agent installed, you can skip the --vm-hostname option. Avocado will then probe the VM IP from the agent.

Virtual Machine Setup

Make sure you have: 1. A libvirt domain with the Avocado packages installed. You can see more info on how to do that in the Getting Started section. 2. The domain IP address or fully qualiﬁed hostname. 3. All pre-requesites for your test to run installed inside the VM (gcc, make and others if you want to compile a 3rd party test suite written in C, for example). Optionally, you may have password less SSH login on your VM enabled.

Running your test

Once the virtual machine is properly set, you may run your test. Example:

$ scripts/avocado run --vm-domain fedora20 --vm-username autotest --vm examples/tests/

˓→sleeptest.py examples/tests/failtest.py VM DOMAIN : fedora20 VM LOGIN : [email protected] JOB ID : 60ddd718e7d7bb679f258920ce3c39ce73cb9779 JOB LOG : $HOME/avocado/job-results/job-2014-09-16T18.41-60ddd71/job.log (1/2) examples/tests/sleeptest.py:SleepTest.test: PASS (1.00 s) (2/2) examples/tests/failtest.py:FailTest.test: FAIL (0.01 s) (continues on next page)

18.1. Optional Plugins 101 avocado Documentation, Release 63.0

A bit of extra logging information is added to your job summary, mainly to distinguish the regular execution from the remote one. Note here that we did not need –vm-password because the SSH key is already set.

Running Tests on a Docker container

(avocado-framework-plugin-runner-docker) Avocado also lets you run tests on a Docker container, starting and cleaning it up automatically with every execution. You can check if this feature (a plugin) is enabled by running:

$ avocado plugins ... docker Run tests inside docker container ...

Docker container images

Avocado needs to be present inside the container image in order for the test execution to be properly performed. There’s one ready to use image (ldoktor/fedora-avocado) in the default image repository (docker.io):

$ sudo docker pull ldoktor/fedora-avocado Using default tag: latest Trying to pull repository docker.io/ldoktor/fedora-avocado ... latest: Pulling from docker.io/ldoktor/fedora-avocado ... Status: Downloaded newer image for docker.io/ldoktor/fedora-avocado:latest

Use custom docker images

One of the possible ways to use (and develop) Avocado is to create a docker image with your development tree. This is a good way to test your development branch without breaking your system. To do so, you can following a few simple steps. Begin by fetching the source code as usual:

$ git clone github.com/avocado-framework/avocado.git avocado.git

You may want to make some changes to Avocado:

$ cd avocado.git $ patch -p1 < MY_PATCH

Finally build a docker image:

$ docker build -t fedora-avocado-custom -f contrib/docker/Dockerfile.fedora .

And now you can run tests with your modiﬁed Avocado inside your container:

$ avocado run --docker fedora-avocado-custom examples/tests/passtest.py

102 Chapter 18. Optional Plugins avocado Documentation, Release 63.0

Running your test

Assuming your system is properly set to run Docker, including having an image with Avocado, you can run a test inside the container with a command similar to:

$ avocado run passtest.py warntest.py failtest.py --docker ldoktor/fedora-avocado --

˓→docker-cmd "sudo docker" JOB ID : db309f5daba562235834f97cad5f4458e3fe6e32 JOB LOG : $HOME/avocado/job-results/job-2016-07-25T08.01-db309f5/job.log DOCKER : Container id

˓→'4bcbcd69801211501a0dde5926c0282a9630adbe29ecb17a21ef04f024366943' DOCKER : Container name 'job-2016-07-25T08.01-db309f5.avocado' (1/3) /avocado_remote_test_dir/$HOME/passtest.py:PassTest.test: PASS (0.00 s) (2/3) /avocado_remote_test_dir/$HOME/warntest.py:WarnTest.test: WARN (0.00 s) (3/3) /avocado_remote_test_dir/$HOME/failtest.py:FailTest.test: FAIL (0.00 s) RESULTS : PASS 1 | ERROR 0 | FAIL 1 | SKIP 0 | WARN 1 | INTERRUPT 0 JOB TIME : 0.10 s JOB HTML : $HOME/avocado/job-results/job-2016-07-25T08.01-db309f5/html/results.html

Environment Variables

Running remote instances os Avocado, for example using remote or vm plugins, the remote environment has a different set of environment variables. If you want to make available remotely variables that are available in the local environment, you can use the run option –env-keep. See the example below:

$ export MYVAR1=foobar $ env MYVAR2=foobar2 avocado run passtest.py --env-keep MYVAR1,MYVAR2 --remote-

˓→hostname 192.168.122.30 --remote-username fedora

By doing that, both MYVAR1 and MYVAR2 will be available in remote environment.

Known Issues

Given the modular architecture of Avocado, the fact that the remote feature is a plugin and also the fact that the plugins are engaged in no particular order, other plugins will not have the information that we are in a remote execution. As consequence, plugins that look for local resources that are available only remotely can fail. That’s the case of the so called multiplex plugin. If you’re using the multiplex plugin (-m or --mux-yaml) options in addition to the remote plugin (or any derived plugin, like vm or docker), the multiplex files must exist locally in the provided path. Notice the multiplex files must be also available remotely in the provided path, since we don’t copy files for remote executions.

18.1.2 Result plugins

Optional plugins providing various types of job results.

HTML results Plugin

This optional plugin creates beautiful human readable results. To install the HTML plugin from pip, use:

pip install avocado-framework-plugin-result-html

18.1. Optional Plugins 103 avocado Documentation, Release 63.0

Once installed it produces the results in job results dir:

$ avocado run sleeptest.py failtest.py synctest.py ... JOB HTML : /home/medic/avocado/job-results/job-2014-08-12T15.57-5ffe4792/html/

˓→results.html ...

This can be disabled via –html-job-result on|off. One can also specify a custom location via –html . Last but not least –open-browser can be used to start browser automatically once the job ﬁnishes.

Results Upload Plugin

This optional plugin is intended to upload the Avocado Job results to a dedicated sever. To install the Result Upload plugin from pip, use: pip install avocado-framework-plugin-result-upload

Usage: avocado run passtest.py--result-upload-url www @avocadologs.example.com:/var/www/html

Avocado logs will be available at following URL: • ssh

[email protected]:/var/www/html/job-2017-04-21T12.54-1cefe11

• html (If web server is enabled)

http://avocadologs.example.com/job-2017-04-21T12.54-1cefe11/

Such links may be refered by other plugins, such as the ResultsDB plugin By default upload will be handled by following command rsync-arz-e'ssh -o LogLevel=error -o stricthostkeychecking=no -o

˓→userknownhostsfile=/dev/null -o batchmode=yes -o passwordauthentication=no'

Optionally, you can customize uploader command, for example following command upload logs to Google storage: avocado run passtest.py--result-upload-url='gs://avocadolog'--result-upload-cmd=

˓→'gsutil -m cp -r'

You can also set the ResultUpload URL and command using a conﬁg ﬁle:

[plugins.result_upload] url= www @avocadologs.example.com:/var/www/htmlavocado/job-results command='rsync -arzq'

And then run the Avocado command without the explicit cmd options. Notice that the command line options will have precedence over the conﬁguration ﬁle.

ResultsDB Plugin

This optional plugin is intended to propagate the Avocado Job results to a given ResultsDB API URL.

104 Chapter 18. Optional Plugins avocado Documentation, Release 63.0

To install the ResultsDB plugin from pip, use: pip install avocado-framework-plugin-resultsdb

Usage: avocado run passtest.py--resultsdb-api http://resultsdb.example.com/api/v2.0/

Optionally, you can provide the URL where the Avocado logs are published: avocado run passtest.py--resultsdb-api http://resultsdb.example.com/api/v2.0/--

˓→resultsdb-logs http://avocadologs.example.com/

The –resultsdb-logs is a convenience option that will create links to the logs in the ResultsDB records. The links will then have the following formats: • ResultDB group (Avocado Job):

http://avocadologs.example.com/job-2017-04-21T12.54-1cefe11/

• ResultDB result (Avocado Test):

http://avocadologs.example.com/job-2017-04-21T12.54-1cefe11/test-results/1-

˓→passtest.py:PassTest.test/

You can also set the ResultsDB API URL and logs URL using a conﬁg ﬁle:

[plugins.resultsdb] api_url= http://resultsdb.example.com/api/v2.0/ logs_url= http://avocadologs.example.com/

And then run the Avocado command without the –resultsdb-api and –resultsdb-logs options. Notice that the command line options will have precedence over the conﬁguration ﬁle.

18.1.3 Robot Plugin

This optional plugin enables Avocado to work with tests originally written using the Robot Framework API. To install the Robot plugin from pip, use:

$ sudo pip install avocado-framework-plugin-robot

After installed, you can list/run Robot tests the same way you do with other types of tests. To list the tests, execute:

$ avocado list ~/path/to/robot/tests/test.robot

Directories are also accepted. To run the tests, execute:

$ avocado run ~/path/to/robot/tests/test.robot

18.1.4 Yaml_to_mux plugin avocado_varianter_yaml_to_mux

18.1. Optional Plugins 105 avocado Documentation, Release 63.0

This plugin utilizes the in-core multiplexation mechanism to produce variants out of a yaml file. This section is example-based, if you are interested in test parameters and/or multiplexation overview, please take a look at Test parameters. As mentioned earlier, it inherits from the avocado.core.mux.MuxPlugin and the only thing it implements is the argument parsing to get some input and a custom yaml parser (which is also capable of parsing json). The YAML file is perfect for this task as it’s easily read by both, humans and machines. Let’s start with an example (line numbers at the first columns are for documentation purposes only, they are not part of the multiplex file format):

1 hw: 2 cpu: !mux 3 intel: 4 cpu_CFLAGS:'-march=core2' 5 amd: 6 cpu_CFLAGS:'-march=athlon64' 7 arm: 8 cpu_CFLAGS:'-mabi=apcs-gnu -march=armv8-a -mtune=arm8' 9 disk: !mux 10 scsi: 11 disk_type:'scsi' 12 virtio: 13 disk_type:'virtio' 14 distro: !mux 15 fedora: 16 init:'systemd' 17 mint: 18 init:'systemv' 19 env: !mux 20 debug: 21 opt_CFLAGS:'-O0 -g' 22 prod: 23 opt_CFLAGS:'-O2'

Warning: On some architectures misbehaving versions of CYaml Python library were reported and Av- ocado always fails with unacceptable character #x0000: control characters are not allowed. To workaround this issue you need to either update the PyYaml to the version which works properly, or you need to remove the python2.7/site-packages/yaml/cyaml.py or disable CYaml import in Avocado sources. For details check out the Github issue

There are couple of key=>value pairs (lines 4,6,8,11,13,. . . ) and there are named nodes which define scope (lines 1,2,3,5,7,9,. . . ). There are also additional flags (lines 2, 9, 14, 19) which modifies the behavior.

Nodes

They define context of the key=>value pairs allowing us to easily identify for what this values might be used for and also it makes possible to define multiple values of the same keys with different scope. Due to their purpose the YAML automatic type conversion for nodes names is disabled, so the value of node name is always as written in the YAML file (unlike values, where yes converts to True and such). Nodes are organized in parent-child relationship and together they create a tree. To view this structure use avocado variants --tree -m :

106 Chapter 18. Optional Plugins avocado Documentation, Release 63.0

run hw cpu intel amd arm disk scsi virtio distro fedora mint env debug prod

You can see that hw has 2 children cpu and disk. All parameters deﬁned in parent node are inherited to children and extended/overwritten by their values up to the leaf nodes. The leaf nodes (intel, amd, arm, scsi, . . . ) are the most important as after multiplexation they form the parameters available in tests.

Keys and Values

Every value other than dict (4,6,8,11) is used as value of the antecedent node. Each node can deﬁne key/value pairs (lines 4,6,8,11,. . . ). Additionally each children node inherits values of it’s parent and the result is called node environment. Given the node structure bellow: devtools: compiler:'cc' flags: -'-O2' debug:'-g' fedora: compiler:'gcc' flags: -'-Wall' osx: compiler:'clang' flags: -'-arch i386' -'-arch x86_64'

And the rules defined as: • Scalar values (Booleans, Numbers and Strings) are overwritten by walking from the root until the final node. • Lists are appended (to the tail) whenever we walk from the root to the final node. The environment created for the nodes fedora and osx are: • Node //devtools/fedora environment compiler: 'gcc', flags: ['-O2', '-Wall'] • Node //devtools/osx environment compiler: 'clang', flags: ['-O2', '-arch i386', '-arch x86_64'] Note that due to different usage of key and values in environment we disabled the automatic value conversion for keys while keeping it enabled for values. This means that the value can be of any YAML supported value, eg. bool, None,

18.1. Optional Plugins 107 avocado Documentation, Release 63.0

list or custom type, while the key is always string.

Variants

In the end all leaves are gathered and turned into parameters, more speciﬁcally into AvocadoParams:

setup: graphic: user:"guest" password:"pass" text: user:"root" password:"123456"

produces [graphic, text]. In the test code you’ll be able to query only those leaves. Intermediary or root nodes are available. The example above generates a single test execution with parameters separated by path. But the most powerful multiplexer feature is that it can generate multiple variants. To do that you need to tag a node whose children are ment to be multiplexed. Effectively it returns only leaves of one child at the time.In order to generate all possible variants multiplexer creates cartesian product of all of these variants:

cpu: !mux intel: amd: arm: fmt: !mux qcow2: raw:

Produces 6 variants:

/cpu/intel,/fmt/qcow2 /cpu/intel,/fmt/raw ... /cpu/arm,/fmt/raw

The !mux evaluation is recursive so one variant can expand to multiple ones:

fmt: !mux qcow: !mux 2: 2v3: raw:

Results in:

/fmt/qcow2/2 /fmt/qcow2/2v3 /raw

Resolution order

You can see that only leaves are part of the test parameters. It might happen that some of these leaves contain different values of the same key. Then you need to make sure your queries separate them by different paths. When the

108 Chapter 18. Optional Plugins avocado Documentation, Release 63.0

path matches multiple results with different origin, an exception is raised as it’s impossible to guess which key was originally intended. To avoid these problems it’s recommended to use unique names in test parameters if possible, to avoid the mentioned clashes. It also makes it easier to extend or mix multiple YAML files for a test. For multiplex YAML files that are part of a framework, contain default configurations, or serve as plugin configurations and other advanced setups it is possible and commonly desirable to use non-unique names. But always keep those points in mind and provide sensible paths. Multiplexer also supports default paths. By default it’s /run/* but it can be overridden by --mux-path, which accepts multiple arguments. What it does it splits leaves by the provided paths. Each query goes one by one through those sub-trees and first one to hit the match returns the result. It might not solve all problems, but it can help to combine existing YAML files with your ones:

qa: # large and complex read-only file, content injected into /qa tests: timeout: 10 ... my_variants: !mux # your YAML file injected into /my_variants short: timeout: 1 long: timeout: 1000

You want to use an existing test which uses params.get('timeout', '*'). Then you can use --mux-path '/my_variants/*' '/qa/*' and it’ll ﬁrst look in your variants. If no matches are found, then it would proceed to /qa/* Keep in mind that only slices deﬁned in mux-path are taken into account for relative paths (the ones starting with *)

Injecting ﬁles

You can run any test with any YAML ﬁle by:

avocado run sleeptest.py--mux-yaml file.yaml

This puts the content of file.yaml into /run location, which as mentioned in previous section, is the default mux-path path. For most simple cases this is the expected behavior as your files are available in the default path and you can safely use params.get(key). When you need to put a file into a different location, for example when you have two files and you don’t want the content to be merged into a single place becoming effectively a single blob, you can do that by giving a name to your YAML file:

avocado run sleeptest.py--mux-yaml duration:duration.yaml

The content of duration.yaml is injected into /run/duration. Still when keys from other files don’t clash, you can use params.get(key) and retrieve from this location as it’s in the default path, only extended by the duration intermediary node. Another benefit is you can merge or separate multiple files by using the same or different name, or even a complex (relative) path. Last but not least, advanced users can inject the file into whatever location they prefer by:

avocado run sleeptest.py--mux-yaml/my/variants/duration:duration.yaml

Simple params.get(key) won’t look in this location, which might be the intention of the test writer. There are several ways to access the values:

18.1. Optional Plugins 109 avocado Documentation, Release 63.0

• absolute location params.get(key, '/my/variants/duration') • absolute location with wildcards params.get(key, '/my/*) (or /*/duration/*...) • set the mux-path avocado run ... --mux-path /my/* and use relative path It’s recommended to use the simple injection for single YAML files, relative injection for multiple simple YAML files and the last option is for very advanced setups when you either can’t modify the YAML files and you need to specify custom resolution order or you are specifying non-test parameters, for example parameters for your plugin, which you need to separate from the test parameters.

Special values

As you might have noticed, we are using mapping/dicts to deﬁne the structure of the params. To avoid surprises we disallowed the smart typing of mapping keys so:

on: on

Won’t become True: True, but the key will be preserved as string on: True. You might also want to use dict as values in your params. This is also supported but as we can’t easily distinguish whether that value is a value or a node (structure), you have to either embed it into another object (list, ..) or you have to clearly state the type (yaml tag !!python/dict). Even then the value won’t be a standard dictionary, but it’ll be collections.OrderedDict and similarly to nodes structure all keys are preserved as strings and no smart type detection is used. Apart from that it should behave similarly as dict, only you get the values ordered by the order they appear in the ﬁle.

Multiple ﬁles

You can provide multiple files. In such scenario final tree is a combination of the provided files where later nodes with the same name override values of the preceding corresponding node. New nodes are appended as new children:

file-1.yaml: debug: CFLAGS:'-O0 -g' prod: CFLAGS:'-O2'

file-2.yaml: prod: CFLAGS:'-Os' fast: CFLAGS:'-Ofast'

results in:

debug: CFLAGS:'-O0 -g' prod: CFLAGS:'-Os' # overriden fast: CFLAGS:'-Ofast' # appended

It’s also possible to include existing ﬁle into another a given node in another ﬁle. This is done by the !include : $path directive:

110 Chapter 18. Optional Plugins avocado Documentation, Release 63.0

os: fedora: !include : fedora.yaml gentoo: !include : gentoo.yaml

Warning: Due to YAML nature, it’s mandatory to put space between !include and the colon (:) that must follow it.

The file location can be either absolute path or relative path to the YAML file where the !include is called (even when it’s nested). Whole file is merged into the node where it’s defined.

Advanced YAML tags

There are additional features related to YAML ﬁles. Most of them require values separated by ":". Again, in all such cases it’s mandatory to add a white space ("") between the tag and the ":", otherwise ":" is part of the tag name and the parsing fails.

!include

Includes other ﬁle and injects it into the node it’s speciﬁed in:

my_other_file: !include : other.yaml

The content of /my_other_file would be parsed from the other.yaml. It’s the hardcoded equivalent of the -m $using:$path. Relative paths start from the original ﬁle’s directory.

!using

Prepends path to the node it’s deﬁned in:

!using : /foo bar: !using : baz

bar is put into baz becoming /baz/bar and everything is put into /foo. So the ﬁnal path of bar is /foo/baz/ bar.

!remove_node

Removes node if it existed during the merge. It can be used to extend incompatible YAML ﬁles:

18.1. Optional Plugins 111 avocado Documentation, Release 63.0

os: fedora: windows: 3.11: 95: os: !remove_node : windows windows: win3.11: win95:

Removes the windows node from structure. It’s different from ﬁlter-out as it really removes the node (and all children) from the tree and it can be replaced by you new structure as shown in the example. It removes windows with all children and then replaces this structure with slightly modiﬁed version. As !remove_node is processed during merge, when you reverse the order, windows is not removed and you end-up with /windows/{win3.11,win95,3.11,95} nodes.

!remove_value

It’s similar to !remove_node only with values.

!mux

Children of this node will be multiplexed. This means that in ﬁrst variant it’ll return leaves of the ﬁrst child, in second the leaves of the second child, etc. Example is in section Variants

!ﬁlter-only

Deﬁnes internal ﬁlters. They are inherited by children and evaluated during multiplexation. It allows one to specify the only compatible branch of the tree with the current variant, for example:

cpu: arm: !filter-only : /disk/virtio disk: virtio: scsi:

will skip the [arm, scsi] variant and result only in [arm, virtio] _Note: It’s possible to use !filter-only multiple times with the same parent and all allowed variants will be included (unless they are filtered-out by !filter-out)_ _Note2: The evaluation order is 1. filter-out, 2. filter-only. This means when you booth filter-out and filter-only a branch it won’t take part in the multiplexed variants._

!ﬁlter-out

Similarly to !filter-only only it skips the specified branches and leaves the remaining ones. (in the same example the use of !filter-out : /disk/scsi results in the same behavior). The difference is when a new disk type is introduced, !filter-only still allows just the specified variants, while !filter-out only removes the specified ones.

112 Chapter 18. Optional Plugins avocado Documentation, Release 63.0

As for the speed optimization, currently Avocado is strongly optimized towards fast !filter-out so it’s highly recommended using them rather than !filter-only, which takes signiﬁcantly longer to process.

Complete example

Let’s take a second look at the ﬁrst example:

1 hw: 2 cpu: !mux 3 intel: 4 cpu_CFLAGS: '-march=core2' 5 amd: 6 cpu_CFLAGS: '-march=athlon64' 7 arm: 8 cpu_CFLAGS: '-mabi=apcs-gnu -march=armv8-a -mtune=arm8' 9 disk: !mux 10 scsi: 11 disk_type: 'scsi' 12 virtio: 13 disk_type: 'virtio' 14 distro: !mux 15 fedora: 16 init: 'systemd' 17 mint: 18 init: 'systemv' 19 env: !mux 20 debug: 21 opt_CFLAGS: '-O0 -g' 22 prod: 23 opt_CFLAGS: '-O2'

After ﬁlters are applied (simply removes non-matching variants), leaves are gathered and all variants are generated:

$ avocado variants -m selftests/.data/mux-environment.yaml Variants generated: Variant 1: /hw/cpu/intel, /hw/disk/scsi, /distro/fedora, /env/debug Variant 2: /hw/cpu/intel, /hw/disk/scsi, /distro/fedora, /env/prod Variant 3: /hw/cpu/intel, /hw/disk/scsi, /distro/mint, /env/debug Variant 4: /hw/cpu/intel, /hw/disk/scsi, /distro/mint, /env/prod Variant 5: /hw/cpu/intel, /hw/disk/virtio, /distro/fedora, /env/debug Variant 6: /hw/cpu/intel, /hw/disk/virtio, /distro/fedora, /env/prod Variant 7: /hw/cpu/intel, /hw/disk/virtio, /distro/mint, /env/debug Variant 8: /hw/cpu/intel, /hw/disk/virtio, /distro/mint, /env/prod Variant 9: /hw/cpu/amd, /hw/disk/scsi, /distro/fedora, /env/debug Variant 10: /hw/cpu/amd, /hw/disk/scsi, /distro/fedora, /env/prod Variant 11: /hw/cpu/amd, /hw/disk/scsi, /distro/mint, /env/debug Variant 12: /hw/cpu/amd, /hw/disk/scsi, /distro/mint, /env/prod Variant 13: /hw/cpu/amd, /hw/disk/virtio, /distro/fedora, /env/debug Variant 14: /hw/cpu/amd, /hw/disk/virtio, /distro/fedora, /env/prod Variant 15: /hw/cpu/amd, /hw/disk/virtio, /distro/mint, /env/debug Variant 16: /hw/cpu/amd, /hw/disk/virtio, /distro/mint, /env/prod Variant 17: /hw/cpu/arm, /hw/disk/scsi, /distro/fedora, /env/debug Variant 18: /hw/cpu/arm, /hw/disk/scsi, /distro/fedora, /env/prod Variant 19: /hw/cpu/arm, /hw/disk/scsi, /distro/mint, /env/debug Variant 20: /hw/cpu/arm, /hw/disk/scsi, /distro/mint, /env/prod Variant 21: /hw/cpu/arm, /hw/disk/virtio, /distro/fedora, /env/debug Variant 22: /hw/cpu/arm, /hw/disk/virtio, /distro/fedora, /env/prod (continues on next page)

18.1. Optional Plugins 113 avocado Documentation, Release 63.0

(continued from previous page) Variant 23: /hw/cpu/arm, /hw/disk/virtio, /distro/mint, /env/debug Variant 24: /hw/cpu/arm, /hw/disk/virtio, /distro/mint, /env/prod

Where the ﬁrst variant contains:

/hw/cpu/intel/=> cpu_CFLAGS:-march=core2 /hw/disk/=> disk_type: scsi /distro/fedora/=> init: systemd /env/debug/=> opt_CFLAGS:-O0-g

The second one:

/hw/cpu/intel/=> cpu_CFLAGS:-march=core2 /hw/disk/=> disk_type: scsi /distro/fedora/=> init: systemd /env/prod/=> opt_CFLAGS:-O2

From this example you can see that querying for /env/debug works only in the ﬁrst variant, but returns nothing in the second variant. Keep this in mind and when you use the !mux ﬂag always query for the pre-mux path, /env/* in this example.

18.1.5 PICT Varianter plugin avocado_varianter_pict This plugin uses a third-party tool to provide variants created by “Pair-Wise” algorithms, also known as Combinatorial Independent Testing.

Installing PICT

PICT is a free software (MIT licensed) tool that implements combinatorial testing. More information about it can be found at https://github.com/Microsoft/pict/. If you’re building from sources, make sure you have a C++ compiler such as GCC or clang, and make. The included Makefile should work out of the box and give you a pict binary. Then copy the pict binary to a location in your $PATH. Alternatively, you may use the plugin --pict-binary command line option to provide a speciﬁc location of the pict binary, but that is not as convenient as having it on your $PATH.

Using the PICT Varianter Plugin

The following listing is a sample (simple) PICT ﬁle: arch: intel, amd block_driver: scsi, ide, virtio net_driver: rtl8139, e1000, virtio guest: windows, linux host: rhel6, rhel7, rhel8

To list the variants generated with the default combination order (2, that is, do a pairwise idenpendent combinatorial testing):

114 Chapter 18. Optional Plugins avocado Documentation, Release 63.0

$ avocado variants --pict-parameter-file=params.pict Pict Variants (11): Variant amd-scsi-rtl8139-windows-rhel6-acff: /run ... Variant amd-ide-e1000-linux-rhel6-eb43: /run

To list the variants generated with a 3-way combination:

$ avocado variants --pict-parameter-file=examples/params.pict \ --pict-order-of-combinations=3

Pict Variants (28): Variant intel-ide-virtio-windows-rhel7-aea5: /run ... Variant intel-scsi-e1000-linux-rhel7-9f61: /run

To run tests, just replace the variants avocado command for run:

$ avocado run --pict-parameter-file=params.pict /bin/true

The tests given in the command line should then be executed with all variants produced by the combinatorial algorithm implemented by PICT.

18.1.6 YAML Loader (yaml_loader)

This plugin is related to Yaml_to_mux plugin and it understands the same content, only it works on loader-level, rather than on test variants level. The result is that this plugin tries to open the test reference as if it was a file specifying variants and if it succeeds it iterates through variants and looks for test_reference entries. On success it attempts to discover the reference using either loader defined by test_reference_resolver_class or it fall-backs to FileLoader when not specified. Then it assigns the current variant’s params to all of the discovered tests. This way one can freely assign various variants to different tests. Currently supported special keys are: • test_reference - reference to be discovered as test • test_reference_resolver_class - loadable location of a loader class to be used to discover the test_reference • test_reference_resolver_args - those arguments will override the avocado arguments passed to the test_resolver_class (only resolver args will be modified) • test_reference_resolver_extra - extra_params to be passed to the test_resolver_class. • mux_suite_test_name_prefix - test name prefix to be added to each discovered test (is useful to distinguish between different variants of the same test) Keep in mind YAML files (in Avocado) are ordered, therefor variant name won’t re-arrange the test order. The only exception is when you use the same variant name twice, then the second one will get merged into the first one. Also note that in case of no test_reference or just when no tests are discovered in the current variant, there is no error, no warning and the loader reports the discovered tests (if any) without the variant which did not produced any tests. The simplest way to learn about this plugin is to look at examples in examples/yaml_to_mux_loader/.

18.1.7 Golang Plugin

This optional plugin enables Avocado to list and run tests written using the Go testing package.

18.1. Optional Plugins 115 avocado Documentation, Release 63.0

To install the Golang plugin from pip, use:

$ sudo pip install avocado-framework-plugin-golang

After installed, you can list/run Golang tests providing the package name:

~$ avocado list golang.org/x/text/unicode/norm GOLANG golang.org/x/text/unicode/norm:TestFlush GOLANG golang.org/x/text/unicode/norm:TestInsert GOLANG golang.org/x/text/unicode/norm:TestDecomposition GOLANG golang.org/x/text/unicode/norm:TestComposition GOLANG golang.org/x/text/unicode/norm:TestProperties GOLANG golang.org/x/text/unicode/norm:TestIterNext GOLANG golang.org/x/text/unicode/norm:TestIterSegmentation GOLANG golang.org/x/text/unicode/norm:TestPlaceHolder GOLANG golang.org/x/text/unicode/norm:TestDecomposeSegment GOLANG golang.org/x/text/unicode/norm:TestFirstBoundary GOLANG golang.org/x/text/unicode/norm:TestNextBoundary GOLANG golang.org/x/text/unicode/norm:TestDecomposeToLastBoundary GOLANG golang.org/x/text/unicode/norm:TestLastBoundary GOLANG golang.org/x/text/unicode/norm:TestSpan GOLANG golang.org/x/text/unicode/norm:TestIsNormal GOLANG golang.org/x/text/unicode/norm:TestIsNormalString GOLANG golang.org/x/text/unicode/norm:TestAppend GOLANG golang.org/x/text/unicode/norm:TestAppendString GOLANG golang.org/x/text/unicode/norm:TestBytes GOLANG golang.org/x/text/unicode/norm:TestString GOLANG golang.org/x/text/unicode/norm:TestLinking GOLANG golang.org/x/text/unicode/norm:TestReader GOLANG golang.org/x/text/unicode/norm:TestWriter GOLANG golang.org/x/text/unicode/norm:TestTransform GOLANG golang.org/x/text/unicode/norm:TestTransformNorm GOLANG golang.org/x/text/unicode/norm:TestCharacterByCharacter GOLANG golang.org/x/text/unicode/norm:TestStandardTests GOLANG golang.org/x/text/unicode/norm:TestPerformance

And the Avocado test reference syntax to ﬁlter the tests you want to execute is also available in this plugin:

~$ avocado list golang.org/x/text/unicode/norm:TestTransform GOLANG golang.org/x/text/unicode/norm:TestTransform GOLANG golang.org/x/text/unicode/norm:TestTransformNorm

To run the tests, just switch from list to run:

~$ avocado run golang.org/x/text/unicode/norm:TestTransform JOB ID : aa6e36547ba304fd724779eff741b6180ee78a54 JOB LOG : $HOME/avocado/job-results/job-2017-10-06T16.06-aa6e365/job.log (1/2) golang.org/x/text/unicode/norm:TestTransform: PASS (1.89 s) (2/2) golang.org/x/text/unicode/norm:TestTransformNorm: PASS (1.87 s) RESULTS : PASS 2 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0 JOB TIME : 4.61 s JOB HTML : $HOME/avocado/job-results/job-2017-10-06T16.06-aa6e365/results.html

The content of the individual tests output is recorded in the default location:

~$ head ~/avocado/job-results/latest/test-results/1-golang.org_x_text_unicode_norm_

˓→TestTransform/debug.log 16:06:53 INFO | Running '/usr/bin/go test -v golang.org/x/text/unicode/norm -run ˓→TestTransform' (continues on next page)

116 Chapter 18. Optional Plugins avocado Documentation, Release 63.0

(continued from previous page) 16:06:55 DEBUG| [stdout] === RUN TestTransform 16:06:55 DEBUG| [stdout] --- PASS: TestTransform (0.00s) 16:06:55 DEBUG| [stdout] === RUN TestTransformNorm 16:06:55 DEBUG| [stdout] === RUN TestTransformNorm/NFC/0 16:06:55 DEBUG| [stdout] === RUN TestTransformNorm/NFC/0/fn 16:06:55 DEBUG| [stdout] === RUN TestTransformNorm/NFC/0/NFD 16:06:55 DEBUG| [stdout] === RUN TestTransformNorm/NFC/0/NFKC 16:06:55 DEBUG| [stdout] === RUN TestTransformNorm/NFC/0/NFKD 16:06:55 DEBUG| [stdout] === RUN TestTransformNorm/NFC/1

18.1.8 GLib Plugin

This optional plugin enables Avocado to list and run tests written using the GLib Test Framework. To list the tests, just provide the test ﬁle path:

$ avocado list --loaders glib -- tests/check-qnum GLIB tests/check-qnum:/qnum/from_int GLIB tests/check-qnum:/qnum/from_uint GLIB tests/check-qnum:/qnum/from_double GLIB tests/check-qnum:/qnum/from_int64 GLIB tests/check-qnum:/qnum/get_int GLIB tests/check-qnum:/qnum/get_uint GLIB tests/check-qnum:/qnum/to_qnum GLIB tests/check-qnum:/qnum/to_string

Notice that you have to be explicit about the test loader you’re using, otherwise, since the test files are executable binaries, the FileLoader will report the file as a SIMPLE test, making the whole test suite to be executed as one test only from the Avocado perspective. The Avocado test reference syntax to filter the tests you want to execute is also available in this plugin:

$ avocado list --loaders glib -- tests/check-qnum:int GLIB tests/check-qnum:/qnum/from_int GLIB tests/check-qnum:/qnum/from_uint GLIB tests/check-qnum:/qnum/from_int64 GLIB tests/check-qnum:/qnum/get_int GLIB tests/check-qnum:/qnum/get_uint

To run the tests, just switch from list to run:

$ avocado run --loaders glib -- tests/check-qnum:int JOB ID : 380a2b3d65b3fce9f8062d84f8635712d6e03133 JOB LOG : $HOME/avocado/job-results/job-2018-02-23T18.02-380a2b3/job.log (1/5) tests/check-qnum:/qnum/from_int: PASS (0.03 s) (2/5) tests/check-qnum:/qnum/from_uint: PASS (0.03 s) (3/5) tests/check-qnum:/qnum/from_int64: PASS (0.04 s) (4/5) tests/check-qnum:/qnum/get_int: PASS (0.03 s) (5/5) tests/check-qnum:/qnum/get_uint: PASS (0.03 s) RESULTS : PASS 5 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0 JOB TIME : 0.46 s JOB HTML : $HOME/avocado/job-results/job-2018-02-23T18.02-380a2b3/results.html

18.1. Optional Plugins 117 avocado Documentation, Release 63.0

18.1.9 Avocado-ec2 Plugin

This plugin allows you to run tests on Amazon EC2 instances. Details available here

18.1.10 Azure Plugin

This plugin allows you to run tests on Microsoft Azure Virtual Machine instances. Details available here

118 Chapter 18. Optional Plugins CHAPTER 19

Advanced Topics and Maintenance

19.1 Reference Guide

This guide presents information on the Avocado basic design and its internals.

19.1.1 Job, test and identiﬁers

Job ID

The Job ID is a random SHA1 string that uniquely identiﬁes a given job. The full form of the SHA1 string is used is most references to a job:

$ avocado run sleeptest.py JOB ID : 49ec339a6cca73397be21866453985f88713ac34 ...

But a shorter version is also used at some places, such as in the job results location:

JOB LOG : $HOME/avocado/job-results/job-2015-06-10T10.44-49ec339/job.log

Test References

A Test Reference is a string that can be resolved into (interpreted as) one or more tests by the Avocado Test Resolver. A given resolver plugin is free to interpret a test reference, it is completely abstract to the other components of Avocado.

Note: Mapping the Test References to tests can be affected by command-line switches like –external-runner, which completelly changes the meaning of the given strings.

119 avocado Documentation, Release 63.0

Test Name

A test name is an arbitrarily long string that unambiguously points to the source of a single test. In other words the Avocado Test Resolver, as configured for a particular job, should return one and only one test as the interpretation of this name. This name can be as specific as necessary to make it unique. Therefore it can contain an arbitrary number of variables, prefixes, suffixes, tags, etc. It all depends on user preferences, what is supported by Avocado via its Test Resolvers and the context of the job. The output of the Test Resolver when resolving Test References should always be a list of unambiguous Test Names (for that particular job). Notice that although the Test Name has to be unique, one test can be run more than once inside a job. By definition, a Test Name is a Test Reference, but the reciprocal is not necessarily true, as the latter can represent more than one test. Examples of Test Names:

'/bin/true' 'passtest.py:Passtest.test' 'file:///tmp/passtest.py:Passtest.test' 'multiple_tests.py:MultipleTests.test_hello' 'type_specific.io-github-autotest-qemu.systemtap_tracing.qemu.qemu_free'

Variant IDs

The varianter component creates different sets of variables (known as “variants”), to allow tests to be run individually in each of them. A Variant ID is an arbitrary and abstract string created by the varianter plugin to identify each variant. It should be unique per variant inside a set. In other words, the varianter plugin generates a set of variants, identiﬁed by unique IDs. A simpler implementation of the varianter uses serial integers as Variant IDs. A more sophisticated implementation could generate Variant IDs with more semantic, potentially representing their contents.

Test ID

A test ID is a string that uniquely identiﬁes a test in the context of a job. When considering a single job, there are no two tests with the same ID. A test ID should encapsulate the Test Name and the Variant ID, to allow direct identiﬁcation of a test. In other words, by looking at the test ID it should be possible to identify: • What’s the test name • What’s the variant used to run this test (if any) Test IDs don’t necessarily keep their uniqueness properties when considered outside of a particular job, but two identical jobs run in the exact same environment should generate a identical sets of Test IDs. Syntax:

-[;]

Example of Test IDs:

120 Chapter 19. Advanced Topics and Maintenance avocado Documentation, Release 63.0

'1-/bin/true' '2-passtest.py:Passtest.test;quiet-' '3-file:///tmp/passtest.py:Passtest.test' '4-multiple_tests.py:MultipleTests.test_hello;maximum_debug-df2f' '5-type_specific.io-github-autotest-qemu.systemtap_tracing.qemu.qemu_free'

19.1.2 Test Types

Avocado at its simplest conﬁguration can run three different types of tests1. You can mix and match those in a single job.

Instrumented

These are tests written in Python or BASH with the Avocado helpers that use the Avocado test API. To be more precise, the Python ﬁle must contain a class derived from avocado.test.Test. This means that an executable written in Python is not always an instrumented test, but may work as a simple test. The instrumented tests allows the writer ﬁner control over the process including logging, test result status and other more sophisticated test APIs. Test statuses PASS, WARN, START and SKIP are considered as successful builds. The ABORT, ERROR, FAIL, ALERT, RUNNING, NOSTATUS and INTERRUPTED are considered as failed ones.

Python unittest

The discovery of classical python unittest is also supported, although unlike python unittest we still use static analysis to get individual tests so dynamically created cases are not recognized. Also note that test result SKIP is reported as CANCEL in Avocado as SKIP test meaning differs from our deﬁnition. Apart from that there should be no surprises when running unittests via Avocado.

Simple

Any executable in your box. The criteria for PASS/FAIL is the return code of the executable. If it returns 0, the test PASSes, if it returns anything else, it FAILs.

19.1.3 Test Statuses

Avocado sticks to the following deﬁnitions of test statuses: • `PASS`: The test passed, which means all conditions being tested have passed. • `FAIL`: The test failed, which means at least one condition being tested has failed. Ideally, it should mean a problem in the software being tested has been found. • `ERROR`: An error happened during the test execution. This can happen, for example, if there’s a bug in the test runner, in its libraries or if a resource breaks unexpectedly. Uncaught exceptions in the test code will also result in this status. • `SKIP`: The test runner decided a requested test should not be run. This can happen, for example, due to missing requirements in the test environment or when there’s a job timeout.

1 Avocado plugins can introduce additional test types.

19.1. Reference Guide 121 avocado Documentation, Release 63.0

19.1.4 Libraries and APIs

The Avocado libraries and its APIs are a big part of what Avocado is. But, to avoid having any issues you should understand what parts of the Avocado libraries are intended for test writers and their respective API stability promises.

Test APIs

At the most basic level there’s the Test APIs which you should use when writing tests in Python and planning to make use of any other utility library. The Test APIs can be found in the avocado main module, and its most important member is the avocado.Test class. By conforming to the avocado.Test API, that is, by inheriting from it, you can use the full set of utility libraries. The Test APIs are guaranteed to be stable across a single major version of Avocado. That means that a test written for a given version of Avocado should not break on later minor versions because of Test API changes.

Utility Libraries

There are also a large number of utility libraries that can be found under the avocado.utils namespace. These are very general in nature and can help you speed up your test development. The utility libraries may receive incompatible changes across minor versions, but these will be done in a staged fashion. If a given change to an utility library can cause test breakage, it will ﬁrst be documented and/or deprecated, and only on the next subsequent minor version it will actually be changed. What this means is that upon updating to later minor versions of Avocado, you should look at the Avocado Release Notes for changes that may impact your tests.

Core (Application) Libraries

Finally, everything under avocado.core is part of the application’s infrastructure and should not be used by tests. Extensions and Plugins can use the core libraries, but API stability is not guaranteed at any level.

19.1.5 Test Resolution

When you use the Avocado runner, frequently you’ll provide paths to files, that will be inspected, and acted upon depending on their contents. The diagram below shows how Avocado analyzes a file and decides what to do with it: It’s important to note that the inspection mechanism is safe (that is, python classes and files are not actually loaded and executed on discovery and inspection stage). Due to the fact Avocado doesn’t actually load the code and classes, the introspection is simple and will not catch things like buggy test modules, missing imports and miscellaneous bugs in the code you want to list or run. We recommend only running tests from sources you trust, use of static checking and reviews in your test development process. Due to the simple test inspection mechanism, avocado will not recognize test classes that inherit from a class derived from avocado.Test. Please refer to the Writing Avocado Tests documentation on how to use the tags functionality to mark derived classes as avocado test classes.

122 Chapter 19. Advanced Topics and Maintenance avocado Documentation, Release 63.0

19.1.6 Results Speciﬁcation

On a machine that executed tests, job results are available under [job-results]/job-[timestamp]-[short job ID], where logdir is the conﬁgured Avocado logs directory (see the data dir plugin), and the directory name includes a timestamp, such as job-2014-08-12T15.44-565e8de. A typical results directory structure can be seen below

$HOME/avocado/job-results/job-2014-08-13T00.45-4a92bc0/ id jobdata args cmdline config multiplex pwd test_references job.log results.json results.xml sysinfo post brctl_show cmdline cpuinfo current_clocksource df_-mP (continues on next page)

19.1. Reference Guide 123 avocado Documentation, Release 63.0

(continued from previous page) dmesg_-c dmidecode fdisk_-l gcc_--version hostname ifconfig_-a interrupts ip_link ld_--version lscpu lspci_-vvnn meminfo modules mount mounts numactl_--hardware_show partitions scaling_governor uname_-a uptime version pre brctl_show cmdline cpuinfo current_clocksource df_-mP dmesg_-c dmidecode fdisk_-l gcc_--version hostname ifconfig_-a interrupts ip_link ld_--version lscpu lspci_-vvnn meminfo modules mount mounts numactl_--hardware_show partitions scaling_governor uname_-a uptime version profile test-results tests sleeptest.py.1 data debug.log sysinfo post pre (continues on next page)

124 Chapter 19. Advanced Topics and Maintenance avocado Documentation, Release 63.0

(continued from previous page) sleeptest.py.2 data debug.log sysinfo post pre sleeptest.py.3 data debug.log sysinfo post pre

22 directories, 65 files

From what you can see, the results dir has: 1. A human readable id in the top level, with the job SHA1. 2. A human readable job.log in the top level, with human readable logs of the task 3. Subdirectory jobdata, that contains machine readable data about the job. 4. A machine readable results.xml and results.json in the top level, with a summary of the job information in xUnit/json format. 5. A top level sysinfo dir, with sub directories pre, post and profile, that store sysinfo ﬁles pre/post/during job, respectively. 6. Subdirectory test-results, that contains a number of subdirectories (ﬁlesystem-friendly test ids). Those test ids represent instances of test execution results.

Test execution instances speciﬁcation

The instances should have: 1. A top level human readable job.log, with job debug information 2.A sysinfo subdir, with sub directories pre, post and profile that store sysinfo files pre test, post test and profiling info while the test was running, respectively. 3.A data subdir, where the test can output a number of files if necessary.

Test execution environment

Each test is executed in a separate process. Due to how the underlying operating system works, a lot of the attributes of the parent process (the Avocado test runner) are passed down to the test process. On GNU/Linux systems, a child process should be “an exact duplicate of the parent process, except” some items that are documented in the fork(2) man page. Besides those operating system exceptions, the Avocado test runner changes the test process in the following ways: 1. The standard input (STDIN) is set to a null device. This is truth both for sys.stdin and for file descriptor 0. Both will point to the same open null device file. 2. The standard output (STDOUT), as in sys.stdout, is redirected so that it doesn’t interfere with the test runner’s own output. All content written to the test’s sys.stdout will be available in the logs under the output prefix.

19.1. Reference Guide 125 avocado Documentation, Release 63.0

Warning: The ﬁle descriptor 1 (AKA /dev/stdout, AKA /proc/self/fd/1, etc) is not currently redirected for INSTRUMENTED tests. Any attempt to write directly to the ﬁle descriptor will interfere with the runner’s own output stream. This behavior will be addressed in a future version.

3. The standard error (STDERR), as in sys.stderr, is redirected so that it doesn’t interfere with the test runner’s own errors. All content written to the test’s sys.stderr will be available in the logs under the output preﬁx.

Warning: The ﬁle descriptor 2 (AKA /dev/stderr, AKA /proc/self/fd/2, etc) is not currently redirected for INSTRUMENTED tests. Any attempt to write directly to the ﬁle descriptor will interfere with the runner’s own error stream. This behavior will be addressed in a future version.

4. A custom handler for signal SIGTERM which will simply raise an exception (with the appropriate message) to be handled by the Avocado test runner, stating the fact that the test was interrupted by such a signal.

Tip: By following the backtrace that is given alongside the in the test log (look for RuntimeError: Test interrupted by SIGTERM) a user can quickly grasp at which point the test was interrupted.

Note: If the test handles SIGTERM differently and doesn’t ﬁnish the test process quickly enough, it will receive then a SIGKILL which is supposed to deﬁnitely end the test process.

5. A number of environment variables that are set by Avocado, all preﬁxed with AVOCADO_. If you want to see for yourself what is described here, you may want to run the example test test_env.py and examine its log messages.

19.1.7 Pre and post plugins

Avocado provides interfaces with which custom plugins can register to be called at various times. For instance, it’s possible to trigger custom actions before and after the execution of a job, or before and after the execution of the tests from a job test suite. Let’s discuss each interface brieﬂy.

Before and after jobs

Avocado supports plug-ins which are (guaranteed to be) executed before the ﬁrst test and after all tests ﬁnished. The interfaces are avocado.core.plugin_interfaces.JobPre and avocado.core. plugin_interfaces.JobPost, respectively. The pre method of each installed plugin of type job.prepost will be called by the run command, that is, anytime an avocado run command is executed.

Note: Conditions such as the SystemExit or KeyboardInterrupt execeptions being raised can interrupt the execution of those plugins.

Then, immediately after that, the job’s run method is called, which attempts to run all job phases, from test suite creation to test execution.

126 Chapter 19. Advanced Topics and Maintenance avocado Documentation, Release 63.0

Unless a SystemExit or KeyboardInterrupt is raised, or yet another major external event (like a system condition that Avocado can not control) it will attempt to run the post methods of all the installed plugins of type job.prepost. This even includes job executions where the pre plugin executions were interrupted.

Before and after the execution of tests

If you followed the previous section, you noticed that the job’s run method was said to run all the test phases. Here’s a sequence of the job phases: 1. Creation of the test suite 2. Pre tests hook 3. Tests execution 4. Post tests hook Plugin writers can have their own code called at Avocado during a job by writing a that will be called at phase number 2 (pre_tests) by writing a method according to the avocado.core.plugin_interfaces. JobPreTests() interface. Accordingly, plugin writers can have their own called at phase number 4 (post_tests) by writing a method according to the avocado.core.plugin_interfaces. JobPostTests() interface. Note that there’s no guarantee that all of the ﬁrst 3 job phases will be executed, so a failure in phase 1 (create_test_suite), may prevent the phase 2 (pre_tests) and/or 3 (run_tests) from from being executed. Now, no matter what happens in the attempted execution of job phases 1 through 3, job phase 4 (post_tests) will be attempted to be executed. To make it extra clear, as long as the Avocado test runner is still in execution (that is, has not been terminated by a system condition that it can not control), it will execute plugin’s post_tests methods. As a concrete example, a plugin’ post_tests method would not be executed after a SIGKILL is sent to the Avocado test runner on phases 1 through 3, because the Avocado test runner would be promptly interrupted. But, a SIGTERM and KeyboardInterrupt sent to the Avocado test runner under phases 1 though 3 would still cause the test runner to run post_tests (phase 4). Now, if during phase 4 a KeyboardInterrupt or SystemExit is received, the remaining plugins’ post_tests methods will NOT be executed.

Jobscripts plugin

Avocado ships with a plugin (installed by default) that allows running scripts before and after the actual execution of Jobs. A user can be sure that, when a given “pre” script is run, no test in that job has been run, and when the “post” scripts are run, all the tests in a given job have already ﬁnished running.

Conﬁguration

By default, the script directory location is:

/etc/avocado/scripts/job

Inside that directory, that is a directory for pre-job scripts:

/etc/avocado/scripts/job/pre.d

And for post-job scripts:

19.1. Reference Guide 127 avocado Documentation, Release 63.0

/etc/avocado/scripts/job/post.d

All the configuration about the Pre/Post Job Scripts are placed under the avocado.plugins.jobscripts config section. To change the location for the pre-job scripts, your configuration should look something like this:

[plugins.jobscripts] pre=/my/custom/directory/ for/pre/job/scripts/

Accordingly, to change the location for the post-job scripts, your conﬁguration should look something like this:

[plugins.jobscripts] post=/my/custom/directory/ for/post/scripts/

A couple of other conﬁguration options are available under the same section: • warn_non_existing_dir: gives warnings if the conﬁgured (or default) directory set for either pre or post scripts do not exist • warn_non_zero_status: gives warnings if a given script (either pre or post) exits with non-zero status

Script Execution Environment

All scripts are run in separate process with some environment variables set. These can be used in your scripts in any way you wish: • AVOCADO_JOB_UNIQUE_ID: the unique job-id. • AVOCADO_JOB_STATUS: the current status of the job. • AVOCADO_JOB_LOGDIR: the ﬁlesystem location that holds the logs and various other ﬁles for a given job run. Note: Even though these variables should all be set, it’s a good practice for scripts to check if they’re set before using their values. This may prevent unintended actions such as writing to the current working directory instead of to the AVOCADO_JOB_LOGDIR if this is not set. Finally, any failures in the Pre/Post scripts will not alter the status of the corresponding jobs.

19.1.8 Job Cleanup

It’s possible to register a callback function that will be called when all the tests have finished running. This effectively allows for a test job to clean some state it may have left behind. At the moment, this feature is not intended to be used by test writers, but it’s seen as a feature for Avocado extensions to make use. To register a callback function, your code should put a message in a very specific format in the “runner queue”. The Avocado test runner code will understand that this message contains a (serialized) function that will be called once all tests finish running. Example:

from avocado import Test

def my_cleanup(path_to_file): if os.path.exists(path_to_file): os.unlink(path_to_file)

(continues on next page)

128 Chapter 19. Advanced Topics and Maintenance avocado Documentation, Release 63.0

(continued from previous page) class MyCustomTest(Test): ... cleanup_file='/tmp/my-custom-state' self.runner_queue.put({"func_at_exit": self.my_cleanup, "args": (cleanup_file), "once": True}) ...

This results in the my_cleanup function being called with positional argument cleanup_file. Because once was set to True, only one unique combination of function, positional arguments and keyword arguments will be registered, not matter how many times they’re attempted to be registered. For more information check avocado.utils.data_structures.CallbackRegister.register().

19.1.9 Docstring Directives Rules

Avocado INSTRUMENTED tests, those written in Python and using the avocado.Test API, can make use of special directives speciﬁed as docstrings. To be considered valid, the docstring must match this pattern: avocado.core.safeloader. DOCSTRING_DIRECTIVE_RE_RAW. An Avocado docstring directive has two parts: 1. The marker, which is the literal string :avocado:. 2. The content, a string that follows the marker, separated by at least one white space or tab. The following is a list of rules that makes a docstring directive be a valid one: • It should start with :avocado:, which is the docstring directive “marker” • At least one whitespace or tab must follow the marker and preceed the docstring directive “content” • The “content”, which follows the marker and the space, must begin with an alphanumeric character, that is, characters within “a-z”, “A-Z” or “0-9”. • After at least one alphanumeric character, the content may contain the following special symbols too: _, ,, = and :. • An end of string (or end of line) must immediately follow the content.

19.1.10 Signal Handlers

Avocado normal operation is related to run code written by users/test-writers. It means the test code can carry its own handlers for different signals or even ignore then. Still, as the code is being executed by Avocado, we have to make sure we will finish all the subprocesses we create before ending our execution. Signals sent to the Avocado main process will be handled as follows: • SIGSTP/Ctrl+Z: On SIGSTP, Avocado will pause the execution of the subprocesses, while the main process will still be running, respecting the timeout timer and waiting for the subprocesses to finish. A new SIGSTP will make the subprocesses to resume the execution. • SIGINT/Ctrl+C: This signal will be forwarded to the test process and Avocado will wait until it’s finished. If the test process does not finish after receiving a SIGINT, user can send a second SIGINT (after the 2 seconds ignore period). The second SIGINT will make Avocado to send a SIGKILL to the whole subprocess tree and then complete the main process execution.

19.1. Reference Guide 129 avocado Documentation, Release 63.0

• SIGTERM: This signal will make Avocado to terminate immediately. A SIGKILL will be sent to the whole subprocess tree and the main process will exit without completing the execution. Notice that it’s a best-effort attempt, meaning that in case of fork-bomb, newly created processes might still be left behind.

19.2 Contribution and Community Guide

Useful pointers on how to participate of the Avocado community and contribute.

19.2.1 Hacking and Using Avocado

Since version 0.31.0, our plugin system requires Setuptools entry points to be registered. If you’re hacking on Avocado and want to use the same, possibly modiﬁed, source for running your tests and experiments, you may do so with one additional step:

$ make develop

On POSIX systems this will create an “egg link” to your original source tree under “$HOME/.local/lib/pythonX.Y/site- packages”. Then, on your original source tree, an “egg info” directory will be created, containing, among other things, the Setuptools entry points mentioned before. This works like a symlink, so you only need to run this once (unless you add a new entry-point, then you need to re-run it to make it available). Avocado supports various plugins, which are distributed as separate projects, for example “avocado-vt” and “avocado- virt”. These also need to be deployed and linked in order to work properly with the avocado from sources (installed version works out of the box). To simplify this you can use make requirements-plugins from the main avocado project to install requirements of the plugins and make link to link and develop the plugins. The workﬂow could be:

$ cd $AVOCADO_PROJECTS_DIR $ git clone $AVOCADO_GIT $ git clone $AVOCADO_PROJECT2 $ # Add more projects $ cd avocado # go into the main avocado project dir $ make requirements-plugins $ make link

You should see the process and status for each directory.

19.2.2 Contact information

• Avocado-devel mailing list: https://www.redhat.com/mailman/listinfo/avocado-devel • Avocado IRC channel: irc.oftc.net #avocado

19.2.3 Contributing to Avocado

Avocado uses github and the github pull request development model. You can ﬁnd a primer on how to use github pull requests here. Every Pull Request you send will be automatically tested by Travis CI and review will take place in the Pull Request as well. For people who don’t like the github development model, there is the option of sending the patches to the Mailing List, following a workﬂow more traditional in Open Source development communities. The patches will be reviewed in the Mailing List, should you opt for that. Then a maintainer will collect the patches, integrate them on a branch, and then those patches will be submitted as a github Pull Request. This process tries to ensure that every contributed patch goes through the CI jobs before it is considered good for inclusion.

130 Chapter 19. Advanced Topics and Maintenance avocado Documentation, Release 63.0

Git workﬂow

• Fork the repository in github. • Clone from your fork:

$ git clone [email protected]:/avocado.git

• Enter the directory:

$ cd avocado

• Create a remote, pointing to the upstream:

$ git remote add upstream [email protected]:avocado-framework/avocado.git

• Conﬁgure your name and e-mail in git:

$ git config --global user.name "Your Name" $ git config --global user.email [email protected]

• Golden tip: never work on local branch master. Instead, create a new local branch and checkout to it:

$ git checkout -b my_new_local_branch

• Code and then commit your changes:

$ git add new-file.py $ git commit -s # or "git commit -as" to commit all changes

• Write a good commit message, pointing motivation, issues that you’re addressing. Usually you should try to explain 3 points in the commit message: motivation, approach and effects:

header<- Limited to 72 characters. No period. <- Blank line message<- Any number of lines, limited to 72 characters per line. <- Blank line Reference:<- External references, one per line (issue, trello,...) Signed-off-by:<- Signature and acknowledgment of licensing terms when contributing to the project (created by git commit-s)

• Make sure your code is working (install your version of avocado, test your change, run make check to make sure you didn’t introduce any regressions). • Paste the job.log ﬁle content from the previous step in a pastebin service, like fpaste.org. If you have fpaste installed, you can simply run:

$ fpaste ~/avocado/job-results/latest/job.log

• Rebase your local branch on top of upstream master:

$ git fetch $ git rebase upstream/master (resolve merge conflicts, if any)

• Push your commit(s) to your fork:

19.2. Contribution and Community Guide 131 avocado Documentation, Release 63.0

$ git push origin my_new_local_branch

• Create the Pull Request on github. Add the relevant information to the Pull Request description. • In the Pull Request discussion page, comment with the link to the job.log output/ﬁle. • Check if your Pull Request passes the CI (travis). Your Pull Request will probably be ignored until it’s all green. Now you’re waiting for feedback on github Pull Request page. Once you get some, join the discussion, answer the questions, make clear if you’re going to change the code based on some review and, if not, why. Feel free to disagree with the reviewer, they probably have different use cases and opinions, which is expected. Try describing yours and suggest other solutions, if necessary. New versions of your code should not be force-updated (unless explicitly requested by the code reviewer). Instead, you should: • Create a new branch out of your previous branch:

$ git checkout my_new_local_branch $ git checkout -b my_new_local_branch_v2

• Code, and amend the commit(s) and/or create new commits. If you have more than one commit in the PR, you will probably need to rebase interactively to amend the right commits. git cola or git citool can be handy here. • Rebase your local branch on top of upstream master:

$ git fetch $ git rebase upstream/master (resolve merge conflicts, if any)

• Push your changes:

$ git push origin my_new_local_branch_v2

• Create a new Pull Request for this new branch. In the Pull Request description, point the previous Pull Request and the changes the current Pull Request introduced when compared to the previous Pull Request(s). • Close the previous Pull Request on github. After your PR gets merged, you can sync the master branch on your local repository propagate the sync to the master branch in your fork repository on github:

$ git checkout master $ git pull upstream master $ git push

From time to time, you can remove old branches to avoid pollution:

# To list branches along with time reference: $ git for-each-ref --sort='-authordate:iso8601' --format=' %(authordate:iso8601)%09

˓→%(refname)' refs/heads # To remove branches from your fork repository: $ git push origin :my_old_branch

Signing commits

Optionally you can sign the commits using GPG signatures. Doing it is simple and it helps from unauthorized code being merged without notice.

132 Chapter 19. Advanced Topics and Maintenance avocado Documentation, Release 63.0

All you need is a valid GPG signature, git configuration, slightly modified workflow to use the signature and eventually even setup in github so one benefits from the “nice” UI. Get a GPG signature:

# Google for howto, but generally it works like this $ gpg --gen-key # defaults are usually fine (using expiration is recommended) $ gpg --send-keys $YOUR_KEY # to propagate the key to outer world

Enable it in git:

$ git config --global user.signingkey $YOUR_KEY

(optional) Link the key with your GH account:

1. Login to github 2. Go to settings->SSH and GPG keys 3. Add New GPG key 4. run $(gpg -a --export $YOUR_EMAIL) in shell to see your key 5. paste the key there

Use it:

# You can sign commits by using '-S' $ git commit -S # You can sign merges by using '-S' $ git merge -S

Warning: You can not use the merge button on github to do signed merges as github does not have your private key.

Licensing

Except where otherwise indicated in a given source ﬁle, all original contributions to Avocado are licensed under the GNU General Public License version 2 (GPLv2) or any later version. By contributing you agree that these contributions are your own (or approved by your employer) and you grant a full, complete, irrevocable copyright license to all users and developers of the Avocado project, present and future, pursuant to the license of the project.

Code Review

Every single Pull Request in Avocado has to be reviewed by at least one other developer. All members of the core team have permission to merge a Pull Request, but there are some conditions that have to be fulﬁlled before merging the code: • Pull Request has to pass the CI tests. • One ‘Approved’ code review should be given. • No explicit disapproval should be present. Pull Requests failing in CI will not be merged, and reviews won’t be given to them until all the problems are sorted out. In case of a weird failure, or false-negative (eg. due to too many commits in a single PR), please reach the developers by @name/email/irc or other means.

19.2. Contribution and Community Guide 133 avocado Documentation, Release 63.0

While reviewing the code, one should: • Verify that the code is sound and clean. • Run the highest level of selftests per each new commit in the merge. The contrib/scripts/ avocado-check-pr.sh contrib script should simplify this step. • Verify that code works to its purpose. • Make sure the commits organization is proper (i.e. code is well organized in atomic commits, there’s no extra/unwanted commits, . . . ). • Provide an in-line feedback with explicit questions and/or requests of improvements. • Provide a general feedback in the review message, being explicit about what’s expected for the next Pull Request version, if that’s the case. When the Pull Request is approved, the reviewer will merge the code or wait for someone with merge permission to merge it.

Using avocado-check-pr.sh

The contrib/scripts/avocado-check-pr.sh script is here to simplify the per-commit-check. You can simply prepare the merge and initiate AVOCADO_CHECK_LEVEL=99 contrib/scripts/ avocado-check-pr.sh to run all checks per each commit between your branch and the same branch on the origin/master (you can specify different remote origin). Use ./contrib/scripts/avocado-check-pr.sh -h to learn more about the options. I can recommend: AVOCADO_PARALLEL_CHECK=yes AVOCADO_CHECK_LEVEL=99 ./contrib/scripts/ avocado-check-pr.sh -i -v and due to PARALLEL false-negatives running while :; do read AAA; python -m unittest $AAA; done in second terminal to re-check potential failures.

Note: Before ﬁrst use you might need to create ~/.config/github_checker.ini and ﬁll github user/token entries (while on it you can also specify some defaults)

19.2.4 Tests Repositories

We encourage you or your company to create public Avocado tests repositories so the community can also benefit of your tests. We will be pleased to advertise your repository here in our documentation. List of known community and third party maintained repositories: • https://github.com/avocado-framework-tests/avocado-misc-tests: Community maintained Avocado miscellaneous tests repository. There you will find, among others, performance tests like lmbench, stress, cpu tests like ebizzy and generic tests like ltp. Some of them were ported from Autotest Client Tests repository. • https://github.com/scylladb/scylla-cluster-tests: Avocado tests for Scylla Clusters. Those tests can automatically create a scylla cluster, some loader machines and then run operations defined by the test writers, such as database workloads.

134 Chapter 19. Advanced Topics and Maintenance avocado Documentation, Release 63.0

19.3 Avocado development tips

19.3.1 Interrupting test

In case you want to “pause” the running test, you can use SIGTSTP (ctrl+z) signal sent to the main avocado process. This signal is forwarded to test and it’s children processes. To resume testing you repeat the same signal. Note: that the job/test timeouts are still enabled on stopped processes.

19.3.2 In tree utils

You can ﬁnd handy utils in avocado.utils.debug: measure_duration

Decorator can be used to print current duration of the executed function and accumulated duration of this decorated function. It’s very handy when optimizing. Usage: from avocado.utils import debug ... @debug.measure_duration def your_function(...):

During the execution look for:

PERF::(0.1s, 11.3s) PERF::(0.2s, 11.5s)

19.3.3 Line-proﬁler

You can measure line-by-line performance by using line_profiler. You can install it using pip: pip install line_profiler and then simply mark the desired function with @profile (no need to import it from anywhere). Then you execute: kernprof-l-v./scripts/avocado run... and when the process finishes you’ll see the profiling information. (sometimes the binary is called kernprof.py)

19.3.4 Remote debug with Eclipse

Eclipse is a nice debugging frontend which allows remote debugging. It’s very simple. The only thing you need is Eclipse with pydev plugin. The simplest way is to use pip install pydevd and then you set the breakpoint by: import pydevd pydevd.settrace(host="$IP_ADDR_OF_ECLIPSE_MACHINE", stdoutToServer=False,

˓→stderrToServer=False, port=5678, suspend=True, trace_only_current_thread=False,

˓→overwrite_prev_trace=False, patch_multiprocessing=False)

19.3. Avocado development tips 135 avocado Documentation, Release 63.0

Before you run the code, you need to start the Eclipse’s debug server. Switch to Debug perspective (you might need to open it ﬁrst Window->Perspective->Open Perspective). Then start the server from Pydev->Start Debug Server. Now whenever the pydev.settrace() code is executed, it contacts Eclipse debug server (port 8000 by default, don’t forget to open it) and you can easily continue in execution. This works on every remote machine which has access to your Eclipse’s port 8000 (you can override it).

19.3.5 Using Trello cards in Eclipse

Eclipse allows us to create tasks. They are pretty cool as you see the status (not started, started, current, done) and by switching tasks it automatically resumes where you previously ﬁnished (opened ﬁles, . . . ) Avocado is planned using Trello, which is not yet supported by Eclipse. Anyway there is a way to at least get read-only list of your commits. This guide is based on https://docs.google.com/document/d/ 1jvmJcCStE6QkJ0z5ASddc3fNmJwhJPOFN7X9-GLyabM/ which didn’t work well with lables and descriptions. The only difference is you need to use Query Pattern:

\"url\":\"https://trello.com/[^/]*/[^/]*/({Id}[^\"]+)({Description})\"

Setup Trello key: 1. Create a Trello account 2. Get (developer_key) here: https://trello.com/1/appKey/generate 3. Get user_token from following address (replace key with your key): https://trello.com/1/authorize?key=\protect\ T1\textdollardeveloper_key&name=Mylyn%20Tasks&expiration=never&response_type=token 4. Address with your assigned tasks (task_addr) is: https://trello.com/1/members/my/cards?key=developer_key& token=\protect\T1\textdollaruser_token Open it in web browser and you should see [] or [$list_of_cards] without any passwords. Configure Eclipse: 1. We’re going to need Web Templates, which are not yet upstream. We need to use incubator version. 2. Help->Install New Software. . . 3. -> Add 4. Name: Incubator 5. Location: http://download.eclipse.org/mylyn/incubator/3.10 6. -> OK 7. Select Mylyn Tasks Connector: Web Templates (Advanced) (Incubation) (use filter text to find it) 8. Install it (Next->Agree->Next. . . ) 9. Restart Eclipse 10. Open the Mylyn Team Repositories Window->Show View->Other.. . ->Mylyn->Team Repositories 11. Right click the Team Repositories and select New->Repository 12. Use Task Repository -> Next 13. Use Web Template (Advanced) -> Next 14. In the Properties for Task Repository dialog box, enter https://trello.com 15. In the Server field and give the repository a label (eg. Trello API). 16. In the Additional Settings section set applicationkey = $developer_key and userkey = $user_token.

136 Chapter 19. Advanced Topics and Maintenance avocado Documentation, Release 63.0

17. In the Advanced Configuration set the Task URL to https://trello.com/c/ 18. Set New Task URL to https://trello.com 19. Set the Query Request URL (no changes required): https://trello.com/1/members/my/cards?key=\protect\ T1\textdollar\protect\T1\textbraceleftapplicationkey\protect\T1\textbraceright&token=\protect\T1\textdollar\ protect\T1\textbraceleftuserkey\protect\T1\textbraceright 20. For the Query Pattern enter “url”:”https://trello.com/[^/]*/[^/]*/({Id}[^”]+)({Description})” 21. -> Finish Create task query: 1. Create a query by opening the Mylyn Task List. 2. Right click the pane and select New Query. 3. Select Trello API as the repository. 4. -> Next 5. Enter the name of your query. 6. Expand the Advanced Configuration and make sure the Query Pattern is filled in 7. Press Preview to confirm that there are no errors. 8. Press Finish. 9. Trello tasks assigned to you will now appear in the Mylyn Task List. Note you can start using tasks by clicking the small bubble in front of the name. This closes all editors. Try opening some and then click the bubble again. They should get closed. When you click the bubble third time, it should resume all the open editors from before. My usual workflow is: 1. git checkout $branch 2. Eclipse: select task 3. git commit . . . 4. Eclipse: unselect task 5. git checkout $other_branch 6. Eclipse: select another_task This way you always have all the files present and you can easily resume your work.

19.4 Releasing avocado

So you have all PRs approved, the Sprint meeting is done and now Avocado is ready to be released. Great, let’s go over (most of) the details you need to pay attention to.

19.4.1 Bump the version number

For the Avocado versioning, two ﬁles need to receive a manual version update: • VERSION • python-avocado.spec

19.4. Releasing avocado 137 avocado Documentation, Release 63.0 followed by make propagate-version to propagate this change to all optional and “linkabe” plugins sharing the parent dir (eg. avocado-vt). Don’t forget to commit the changes of “linked” plugins as they live in different repositories. An example diff (after the make propagate-version) looks like this: diff --git a/VERSION b/VERSION index dd0353d..aafccd8 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -48.0 +49.0 diff --git a/optional_plugins/html/VERSION b/optional_plugins/html/VERSION index dd0353d..aafccd8 100644 --- a/optional_plugins/html/VERSION +++ b/optional_plugins/html/VERSION @@ -1 +1 @@ -48.0 +49.0 diff --git a/optional_plugins/robot/VERSION b/optional_plugins/robot/VERSION index dd0353d..aafccd8 100644 --- a/optional_plugins/robot/VERSION +++ b/optional_plugins/robot/VERSION @@ -1 +1 @@ -48.0 +49.0 diff --git a/optional_plugins/runner_docker/VERSION b/optional_plugins/runner_docker/

˓→VERSION index dd0353d..aafccd8 100644 --- a/optional_plugins/runner_docker/VERSION +++ b/optional_plugins/runner_docker/VERSION @@ -1 +1 @@ -48.0 +49.0 diff --git a/optional_plugins/runner_remote/VERSION b/optional_plugins/runner_remote/

˓→VERSION index dd0353d..aafccd8 100644 --- a/optional_plugins/runner_remote/VERSION +++ b/optional_plugins/runner_remote/VERSION @@ -1 +1 @@ -48.0 +49.0 diff --git a/optional_plugins/runner_vm/VERSION b/optional_plugins/runner_vm/VERSION index dd0353d..aafccd8 100644 --- a/optional_plugins/runner_vm/VERSION +++ b/optional_plugins/runner_vm/VERSION @@ -1 +1 @@ -48.0 +49.0 diff --git a/python-avocado.spec b/python-avocado.spec index 6a4b067..4b9dba8 100644 --- a/python-avocado.spec +++ b/python-avocado.spec @@ -12,7 +12,7 @@

Summary: Framework with tools and libraries for Automated Testing Name: python-%{srcname} (continues on next page)

138 Chapter 19. Advanced Topics and Maintenance avocado Documentation, Release 63.0

(continued from previous page) -Version: 48.0 +Version: 49.0 Release: 1%{?dist} License: GPLv2 Group: Development/Tools @@ -259,6 +259,9 @@ examples of how to write tests on your own. %{_datadir}/avocado/wrappers

%changelog +* Wed Apr 12 2017 Lukas Doktor - 49.0-0 +- Testing release + * Mon Apr 3 2017 Cleber Rosa - 48.0-1 - Updated exclude directives and files for optional plugins

You can ﬁnd on git such commits that will help you get oriented for other repos.

19.4.2 Which repositories you should pay attention to

In general, a release of avocado includes taking a look and eventually release content in the following repositories: • avocado • avocado-vt

19.4.3 Tag all repositories

When everything is in good shape, commit the version changes and tag that commit in master with:

$ git tag -u $(GPG_ID) -s $(RELEASE) -m 'Avocado Release $(RELEASE)'

Then the tag should be pushed to the GIT repository with:

$ git push --tags

19.4.4 Build RPMs

Go to the source directory and do:

$ make rpm ... + exit 0

This should be all. It will build packages using mock, targeting your default conﬁguration. That usually means the same platform you’re currently on.

19.4.5 Sign Packages

All the packages should be signed for safer public consumption. The process is, of course, dependent on the private keys, put is based on running:

19.4. Releasing avocado 139 avocado Documentation, Release 63.0

$ rpm --resign

For more information look at the rpmsign(8) man page.

19.4.6 Upload packages to repository

The current distribution method is based on serving content over HTTP. That means that repository metadata is created locally and synchronized to the well know public Web server. A process similar to:

$ cd $REPO_ROOT && for DIR in epel-?-noarch fedora-??-noarch; \ do cd $DIR && createrepo -v . && cd ..; done;

Creates the repo metadata locally. Then a command similar to:

$ rsync -va $REPO_ROOT user@repo_web_server:/path

Is used to copy the content over.

19.4.7 Write release notes

Release notes give an idea of what has changed on a given development cycle. Good places to go for release notes are: 1. Git logs 2. Trello Cards (Look for the Done lists) 3. Github compare views: https://github.com/avocado-framework/avocado/compare/0.28.0. . . 0.29.0 Go there and try to write a text that represents the changes that the release encompasses.

19.4.8 Upload package to PyPI

Users may also want to get Avocado from the PyPI repository, so please upload there as well. To help with the process, please run:

$ make pypi

And follow the URL and brief instructions given.

19.4.9 Conﬁgure Read The Docs

On https://readthedocs.org/dashboard/avocado-framework/edit/: • Click in Versions. In Choose Active Versions, ﬁnd the version you’re releasing and check the Active option. Submit. • Click in Versions again. In Default Version, select the new version you’re releasing. Submit.

19.4.10 Send e-mails to avocado-devel and other places

Send the e-mail with the release notes to avocado-devel and virt-test-devel.

140 Chapter 19. Advanced Topics and Maintenance avocado Documentation, Release 63.0

19.5 Other Resources

This is a collection of some other varied Avocado related sources on the web:

19.5.1 Presentations

• Testing Framework Internals (DevConf 2017) • Auto Testing for AArch64 Virtualization (Linaro connect San Francisco 2017) • libvirt integration and testing for enterprise KVM/ARM (Linaro Connect Budapest 2017) • Automated Testing Framework (PyCon CZ 2016) • Avocado and Jenkins (DevConf 2016) • Avocado: Next Gen Testing Toolbox (DevConf 2015) • Avocado workshop (DevConf 2015) mindmap with all commands/content and a partial video • Avocado: Open Source Testing Made Easy (LinuxCon 2015)

19.5.2 Public test repositories

• Avocado Misc Tests • Cockpit tests • Modularity framework tests (uses custom docker image) • Azure Tests (requires plugin)

19.5. Other Resources 141 avocado Documentation, Release 63.0

142 Chapter 19. Advanced Topics and Maintenance CHAPTER 20

API Reference

20.1 Test APIs

This is the bare mininum set of APIs that users should use, and can rely on, while writing tests.

20.1.1 Module contents avocado.main alias of avocado.core.job.TestProgram class avocado.Test(methodName=’test’, name=None, params=None, base_logdir=None, job=None, runner_queue=None) Bases: unittest.case.TestCase, avocado.core.test.TestData Base implementation for the test class. You’ll inherit from this to write your own tests. Typically you’ll want to implement setUp(), test*() and tear- Down() methods on your own tests. Initializes the test. Parameters • methodName – Name of the main method to run. For the sake of compatibility with the original unittest class, you should not set this. • name (avocado.core.test.TestID) – Pretty name of the test name. For normal tests, written with the avocado API, this should not be set. This is reserved for internal Avocado use, such as when running random executables as tests. • base_logdir – Directory where test logs should go. If None provided, it’ll use avocado.data_dir.create_job_logs_dir(). • job – The job that this test is part of. basedir The directory where this test (when backed by a ﬁle) is located at

143 avocado Documentation, Release 63.0

cache_dirs Returns a list of cache directories as set in config file. cancel(message=None) Cancels the test. This method is expected to be called from the test method, not anywhere else, since by definition, we can only cancel a test that is currently under execution. If you call this method outside the test method, avocado will mark your test status as ERROR, and instruct you to fix your test in the error message. Parameters message (str) – an optional message that will be recorded in the logs error(message=None) Errors the currently running test. After calling this method a test will be terminated and have its status as ERROR. Parameters message (str) – an optional message that will be recorded in the logs fail(message=None) Fails the currently running test. After calling this method a test will be terminated and have its status as FAIL. Parameters message (str) – an optional message that will be recorded in the logs fail_class fail_reason fetch_asset(name, asset_hash=None, algorithm=None, locations=None, expire=None) Method o call the utils.asset in order to fetch and asset file supporting hash check, caching and multiple locations. Parameters • name – the asset filename or URL • asset_hash – asset hash (optional) • algorithm – hash algorithm (optional, defaults to avocado.utils.asset. DEFAULT_HASH_ALGORITHM) • locations – list of URLs from where the asset can be fetched (optional) • expire – time for the asset to expire Raises EnvironmentError – When it fails to fetch the asset Returns asset file local path filename Returns the name of the file (path) that holds the current test get_state() Serialize selected attributes representing the test state Returns a dictionary containing relevant test state data Return type dict job The job this test is associated with log The enhanced test log

144 Chapter 20. API Reference avocado Documentation, Release 63.0

logdir Path to this test’s logging dir logfile Path to this test’s main debug.log file name Returns the Test ID, which includes the test name Return type TestID outputdir Directory available to test writers to attach files to the results params Parameters of this test (AvocadoParam instance) report_state() Send the current test state to the test runner process run_avocado() Wraps the run method, for execution inside the avocado runner. Result Unused param, compatibility with unittest.TestCase. runner_queue The communication channel between test and test runner running Whether this test is currently being executed set_runner_queue(runner_queue) Override the runner_queue status The result status of this test teststmpdir Returns the path of the temporary directory that will stay the same for all tests in a given Job. time_elapsed = -1 duration of the test execution (always recalculated from time_end - time_start time_end = -1 (unix) time when the test finished (could be forced from test) time_start = -1 (unix) time when the test started (could be forced from test) timeout = None Test timeout (the timeout from params takes precedence) traceback whiteboard = '' Arbitrary string which will be stored in $logdir/whiteboard location when the test finishes. workdir This property returns a writable directory that exists during the entire test execution, but will be cleaned up once the test finishes. It can be used on tasks such as decompressing source tarballs, building software, etc.

20.1. Test APIs 145 avocado Documentation, Release 63.0 avocado.fail_on(exceptions=None) Fail the test when decorated function produces exception of the speciﬁed type. (For example, our method may raise IndexError on tested software failure. We can either try/catch it or use this decorator instead) Parameters exceptions – Tuple or single exception to be assumed as test fail [Exception] Note self.error and self.cancel behavior remains intact Note To allow simple usage param “exceptions” must not be callable avocado.skip(message=None) Decorator to skip a test. avocado.skipIf(condition, message=None) Decorator to skip a test if a condition is True. avocado.skipUnless(condition, message=None) Decorator to skip a test if a condition is False. exception avocado.TestError Bases: avocado.core.exceptions.TestBaseException Indicates that the test was not fully executed and an error happened. This is the sort of exception you raise if the test was partially executed and could not complete due to a setup, conﬁguration, or another fatal condition. status = 'ERROR' exception avocado.TestFail Bases: avocado.core.exceptions.TestBaseException, exceptions.AssertionError Indicates that the test failed. TestFail inherits from AssertionError in order to keep compatibility with vanilla python unittests (they only consider failures the ones deriving from AssertionError). status = 'FAIL' exception avocado.TestCancel Bases: avocado.core.exceptions.TestBaseException Indicates that a test was canceled. Should be thrown when the cancel() test method is used. status = 'CANCEL'

20.2 Utilities APIs

This is a set of utility APIs that Avocado provides as added value to test writers. It’s suppose to be generic, without any knowledge of Avocado and reusable in different projects.

Note: In the current version there is a hidden knowledge of avocado logging streams. More about this issue can be found here https://trello.com/c/4QyUgWsW/720-get-rid-of-avocado-test-loggers-from-avocado-utils

146 Chapter 20. API Reference avocado Documentation, Release 63.0

20.2.1 Subpackages

avocado.utils.external package

Submodules

avocado.utils.external.gdbmi_parser module

avocado.utils.external.gdbmi_parser.compare(a, b) avocado.utils.external.gdbmi_parser.parse(tokens) avocado.utils.external.gdbmi_parser.process(input_message) avocado.utils.external.gdbmi_parser.scan(input_message)

avocado.utils.external.spark module

class avocado.utils.external.spark.GenericASTBuilder(AST, start) Bases: avocado.utils.external.spark.GenericParser buildASTNode(args, lhs) nonterminal(token_type, args) preprocess(rule, func) terminal(token) class avocado.utils.external.spark.GenericASTMatcher(start, ast) Bases: avocado.utils.external.spark.GenericParser foundMatch(args, func) match(ast=None) match_r(node) preprocess(rule, func) resolve(input_list) class avocado.utils.external.spark.GenericASTTraversal(ast)

default(node) postorder(node=None) preorder(node=None) prune() typestring(node) exception avocado.utils.external.spark.GenericASTTraversalPruningException Bases: exceptions.Exception class avocado.utils.external.spark.GenericParser(start)

add(input_set, item, i=None, predecessor=None, causal=None) addRule(doc, func, _preprocess=1)

20.2. Utilities APIs 147 avocado Documentation, Release 63.0

ambiguity(rules) augment(start) buildTree(nt, item, tokens, k) causal(key) collectRules() computeNull() deriveEpsilon(nt) error(token) finalState(tokens) goto(state, sym) gotoST(state, st) gotoT(state, t) isnullable(sym) makeNewRules() makeSet(token, sets, i) makeSet_fast(token, sets, i) makeState(state, sym) makeState0() parse(tokens) predecessor(key, causal) preprocess(rule, func) resolve(input_list) skip(lhs_rhs, pos=0) typestring(token) class avocado.utils.external.spark.GenericScanner(ﬂags=0)

error(s, pos) makeRE(name) position(newpos=None) reflect() t_default(s) ( . | n )+ tokenize(s)

148 Chapter 20. API Reference avocado Documentation, Release 63.0

Module contents

20.2.2 Submodules

20.2.3 avocado.utils.archive module

Module to help extract and create compressed archives. exception avocado.utils.archive.ArchiveException Bases: exceptions.Exception Base exception for all archive errors. class avocado.utils.archive.ArchiveFile(filename, mode=’r’) Bases: object Class that represents an Archive file. Archives are ZIP files or Tarballs. Creates an instance of ArchiveFile. Parameters • filename – the archive file name. • mode – file mode, r read, w write. add(filename, arcname=None) Add file to the archive. Parameters • filename – file to archive. • arcname – alternative name for the file in the archive. close() Close archive. extract(path=’.’) Extract all files from the archive. Parameters path – destination path. list() List files to the standard output. classmethod open(filename, mode=’r’) Creates an instance of ArchiveFile. Parameters • filename – the archive file name. • mode – file mode, r read, w write. avocado.utils.archive.compress(filename, path) Compress files in an archive. Parameters • filename – archive file name. • path – origin directory path to files to compress. No individual files allowed.

20.2. Utilities APIs 149 avocado Documentation, Release 63.0 avocado.utils.archive.create(filename, path) Compress files in an archive. Parameters • filename – archive file name. • path – origin directory path to files to compress. No individual files allowed. avocado.utils.archive.extract(filename, path) Extract files from an archive. Parameters • filename – archive file name. • path – destination path to extract to. avocado.utils.archive.is_archive(filename) Test if a given file is an archive. Parameters filename – file to test. Returns True if it is an archive. avocado.utils.archive.uncompress(filename, path) Extract files from an archive. Parameters • filename – archive file name. • path – destination path to extract to.

20.2.4 avocado.utils.asset module

Asset fetcher from multiple locations class avocado.utils.asset.Asset(name, asset_hash, algorithm, locations, cache_dirs, expire=None) Bases: object Try to fetch/verify an asset file from multiple locations. Initialize the Asset() class. Parameters • name – the asset filename. url is also supported • asset_hash – asset hash • algorithm – hash algorithm • locations – list of locations fetch asset from • cache_dirs – list of cache directories • expire – time in seconds for the asset to expire fetch() Fetches the asset. First tries to find the asset on the provided cache_dirs list. Then tries to download the asset from the locations list provided. Raises EnvironmentError – When it fails to fetch the asset Returns The path for the file on the cache directory.

150 Chapter 20. API Reference avocado Documentation, Release 63.0 avocado.utils.asset.DEFAULT_HASH_ALGORITHM = 'sha1' The default hash algorithm to use on asset cache operations exception avocado.utils.asset.UnsupportedProtocolError Bases: exceptions.EnvironmentError Signals that the protocol of the asset URL is not supported

20.2.5 avocado.utils.astring module

Operations with strings (conversion and sanitation). The unusual name aims to avoid causing name clashes with the stdlib module string. Even with the dot notation, people may try to do things like import string . . . from avocado.utils import string And not notice until their code starts failing. avocado.utils.astring.ENCODING = 'UTF-8' On import evaluated value representing the system encoding based on system locales using locale. getpreferredencoding(). Use this value wisely as some ﬁles are dumped in different encoding. avocado.utils.astring.FS_UNSAFE_CHARS = '<>:"/\\|?*;' String containing all fs-unfriendly chars (Windows-fat/Linux-ext3) avocado.utils.astring.bitlist_to_string(data) Transform from bit list to ASCII string. Parameters data – Bit list to be transformed avocado.utils.astring.is_bytes(data) Checks if the data given is a sequence of bytes And not a “text” type, that can be of multi-byte characters. Also, this does NOT mean a bytearray type. Parameters data – the instance to be checked if it falls under the deﬁnition of an array of bytes. avocado.utils.astring.is_text(data) Checks if the data given is a suitable for holding text That is, if it can hold text that requires more than one byte for each character. avocado.utils.astring.iter_tabular_output(matrix, header=None, strip=False) Generator for a pretty, aligned string representation of a nxm matrix. This representation can be used to print any tabular data, such as database results. It works by scanning the lengths of each element in each column, and determining the format string dynamically. Parameters • matrix – Matrix representation (list with n rows of m elements). • header – Optional tuple or list with header elements to be displayed. • strip – Optionally remove trailing whitespace from each row. avocado.utils.astring.shell_escape(command) Escape special characters from a command so that it can be passed as a double quoted (” “) string in a (ba)sh command. Parameters command – the command string to escape. Returns The escaped command string. The required englobing double quotes are NOT added and so should be added at some point by the caller.

20.2. Utilities APIs 151 avocado Documentation, Release 63.0

See also: http://www.tldp.org/LDP/abs/html/escapingsection.html avocado.utils.astring.string_safe_encode(input_str) People tend to mix unicode streams with encoded strings. This function tries to replace any input with a valid utf-8 encoded ascii stream. On Python 3, it’s a terrible idea to try to mess with encoding, so this function is limited to converting other types into strings, such as numeric values that are often the members of a matrix. Parameters input_str – possibly unsafe string or other object that can be turned into a string Returns a utf-8 encoded ascii stream avocado.utils.astring.string_to_bitlist(data) Transform from ASCII string to bit list. Parameters data – String to be transformed avocado.utils.astring.string_to_safe_path(input_str) Convert string to a valid file/dir name. This takes a string that may contain characters that are not allowed on FAT (Windows) filesystems and/or ext3 (Linux) filesystems, and replaces them for safe (boring) underlines. It limits the size of the path to be under 255 chars, and make hidden paths (starting with “.”) non-hidden by making them start with “_”. Parameters input_str – String to be converted Returns String which is safe to pass as a file/dir name (on recent fs) avocado.utils.astring.strip_console_codes(output, custom_codes=None) Remove the Linux console escape and control sequences from the console output. Make the output readable and can be used for result check. Now only remove some basic console codes using during boot up. Parameters • output (string) – The output from Linux console • custom_codes – The codes added to the console codes which is not covered in the default codes Returns the string without any special codes Return type string avocado.utils.astring.tabular_output(matrix, header=None, strip=False) Pretty, aligned string representation of a nxm matrix. This representation can be used to print any tabular data, such as database results. It works by scanning the lengths of each element in each column, and determining the format string dynamically. Parameters • matrix – Matrix representation (list with n rows of m elements). • header – Optional tuple or list with header elements to be displayed. • strip – Optionally remove trailing whitespace from each row. Returns String with the tabular output, lines separated by unix line feeds. Return type str avocado.utils.astring.to_text(data, encoding=’UTF-8’, errors=’strict’) Convert anything to text decoded text

152 Chapter 20. API Reference avocado Documentation, Release 63.0

When the data is bytes, it’s decoded. When it’s not of string types it’s re-formatted into text and returned. Otherwise (it’s string) it’s returned unchanged. Parameters • data (either bytes or other data that will be returned unchanged) – data to be transformed into text • encoding – encoding of the data (only used when decoding is necessary) • errors – how to handle encode/decode errors, see: https://docs.python.org/3/library/ codecs.html#error-handlers

20.2.6 avocado.utils.aurl module

URL related functions. The strange name is to avoid accidental naming collisions in code. avocado.utils.aurl.is_url(path) Return True if path looks like an URL. Parameters path – path to check. Return type Boolean.

20.2.7 avocado.utils.build module avocado.utils.build.configure(path, configure=None) Configures the source tree for a subsequent build Most source directories coming from official released tarballs will have a “configure” script, but source code snapshots may have “autogen.sh” instead (which usually creates and runs a “configure” script itself). This function will attempt to run the first one found (if a configure script name not given explicitly). Parameters configure (str or None) – the name of the configure script (None for trying to find one automatically) Returns the configure script exit status, or None if no script was found and executed avocado.utils.build.make(path, make=’make’, env=None, extra_args=”, ignore_status=None, al- low_output_check=None, process_kwargs=None) Run make, adding MAKEOPTS to the list of options. Parameters • make – what make command name to use. • env – dictionary with environment variables to be set before calling make (e.g.: CFLAGS). • extra – extra command line arguments to pass to make. • allow_output_check (str) – Whether to log the command stream outputs (stdout and stderr) of the make process in the test stream files. Valid values: ‘stdout’, for allowing only standard output, ‘stderr’, to allow only standard error, ‘all’, to allow both standard output and error, and ‘none’, to allow none to be recorded (default). The default here is ‘none’, because usually we don’t want to use the compilation output as a reference in tests. Returns exit status of the make process avocado.utils.build.run_make(path, make=’make’, extra_args=”, process_kwargs=None) Run make, adding MAKEOPTS to the list of options.

20.2. Utilities APIs 153 avocado Documentation, Release 63.0

Parameters • path – directory from where to run make • make – what make command name to use. • extra_args – extra command line arguments to pass to make. • process_kwargs – Additional key word arguments to the underlying process running the make. Returns the make command result object

20.2.8 avocado.utils.cpu module

Get information from the current’s machine CPU. avocado.utils.cpu.cpu_has_flags(flags) Check if a list of flags are available on current CPU info Parameters flags (list)–A list of cpu flags that must exists on the current CPU. Returns bool True if all the flags were found or False if not Return type list avocado.utils.cpu.cpu_online_list() Reports a list of indexes of the online cpus avocado.utils.cpu.get_cpu_arch() Work out which CPU architecture we’re running on avocado.utils.cpu.get_cpu_vendor_name() Get the current cpu vendor name Returns string ‘intel’ or ‘amd’ or ‘power7’ depending on the current CPU architecture. Return type string avocado.utils.cpu.get_cpufreq_governor() Get current cpu frequency governor avocado.utils.cpu.get_cpuidle_state() Get current cpu idle values Returns Dict of cpuidle states values for all cpus Return type Dict of dicts avocado.utils.cpu.offline(cpu) Offline given CPU avocado.utils.cpu.online(cpu) Online given CPU avocado.utils.cpu.online_cpus_count() Return Number of Online cpus in the system avocado.utils.cpu.set_cpufreq_governor(governor=’random’) To change the given cpu frequency governor Parameters governor – frequency governor profile name whereas random is default option to choose random profile among available ones.

154 Chapter 20. API Reference avocado Documentation, Release 63.0

avocado.utils.cpu.set_cpuidle_state(state_number=’all’, disable=True, setstate=None) Set/Reset cpu idle states for all cpus Parameters • state_number – cpuidle state number, default: all all states • disable – whether to disable/enable given cpu idle state, default is to disable (True) • setstate – cpuidle state value, output of get_cpuidle_state() avocado.utils.cpu.total_cpus_count() Return Number of Total cpus in the system including ofﬂine cpus

20.2.9 avocado.utils.crypto module avocado.utils.crypto.hash_file(filename, size=None, algorithm=’md5’) Calculate the hash value of filename. If size is not None, limit to first size bytes. Throw exception if something is wrong with filename. Can be also implemented with bash one-liner (assuming size%1024==0):

dd if=filename bs=1024 count=size/1024| sha1sum-

Parameters • filename – Path of the file that will have its hash calculated. • algorithm – Method used to calculate the hash (default is md5). • size – If provided, hash only the first size bytes of the file. Returns Hash of the file, if something goes wrong, return None.

20.2.10 avocado.utils.data_factory module

Generate data useful for the avocado framework and tests themselves. avocado.utils.data_factory.generate_random_string(length, ignore=’!"#$%&\’()*+, - ./:;<=>?@[\\]^_‘{|}~’, convert=”) Generate a random string using alphanumeric characters. Parameters • length (int) – Length of the string that will be generated. • ignore (str) – Characters that will not include in generated string. • convert (str) – Characters that need to be escaped (prepend “”). Returns The generated random string. avocado.utils.data_factory.make_dir_and_populate(basedir=’/tmp’) Create a directory in basedir and populate with a number of ﬁles. The ﬁles just have random text contents. Parameters basedir (str) – Base directory where directory should be generated. Returns Path of the dir created and populated. Return type str

20.2. Utilities APIs 155 avocado Documentation, Release 63.0

20.2.11 avocado.utils.data_structures module

This module contains handy classes that can be used inside avocado core code or plugins. class avocado.utils.data_structures.Borg Multiple instances of this class will share the same state. This is considered a better design pattern in Python than more popular patterns, such as the Singleton. Inspired by Alex Martelli’s article mentioned below: See http://www.aleax.it/5ep.html class avocado.utils.data_structures.CallbackRegister(name, log) Bases: object Registers pickable functions to be executed later. Parameters name – Human readable identiﬁer of this register register(func, args, kwargs, once=False) Register function/args to be called on self.destroy() :param func: Pickable function :param args: Pick- able positional arguments :param kwargs: Pickable keyword arguments :param once: Add unique (func,args,kwargs) combination only once run() Call all registered function unregister(func, args, kwargs) Unregister (func,args,kwargs) combination :param func: Pickable function :param args: Pickable positional arguments :param kwargs: Pickable keyword arguments class avocado.utils.data_structures.DataSize(data) Bases: object Data Size object with builtin unit-converted attributes. Parameters data (str) – Data size plus optional unit string. i.e. ‘10m’. No unit string means the data size is in bytes. MULTIPLIERS = {'b': 1, 'g': 1073741824, 'k': 1024, 'm': 1048576, 't': 1099511627776} b g k m t unit value exception avocado.utils.data_structures.InvalidDataSize Bases: exceptions.ValueError Signals that the value given to DataSize is not valid. class avocado.utils.data_structures.LazyProperty(f_get) Bases: object Lazily instantiated property.

156 Chapter 20. API Reference avocado Documentation, Release 63.0

Use this decorator when you want to set a property that will only be evaluated the first time it’s accessed. Inspired by the discussion in the Stack Overflow thread below: See http://stackoverflow.com/questions/15226721/ avocado.utils.data_structures.comma_separated_ranges_to_list(string) Provides a list from comma separated ranges Parameters string – string of comma separated range Return list list of integer values in comma separated range avocado.utils.data_structures.compare_matrices(matrix1, matrix2, threshold=0.05) Compare 2 matrices nxm and return a matrix nxm with comparison data and stats. When the first columns match, they are considered as header and included in the results intact. Parameters • matrix1 – Reference Matrix of floats; first column could be header. • matrix2 – Matrix that will be compared; first column could be header • threshold – Any difference greater than this percent threshold will be reported. Returns Matrix with the difference in comparison, number of improvements, number of regressions, total number of comparisons. avocado.utils.data_structures.geometric_mean(values) Evaluates the geometric mean for a list of numeric values. This implementation is slower but allows unlimited number of values. :param values: List with values. :return: Single value representing the geometric mean for the list values. :see: http://en.wikipedia.org/wiki/Geometric_mean avocado.utils.data_structures.ordered_list_unique(object_list) Returns an unique list of objects, with their original order preserved avocado.utils.data_structures.time_to_seconds(time) Convert time in minutes, hours and days to seconds. :param time: Time, optionally including the unit (i.e. ‘10d’)

20.2.12 avocado.utils.debug module

This ﬁle contains tools for (not only) Avocado developers. avocado.utils.debug.log_calls(length=None, cls_name=None) Use this as decorator to log the function call altogether with arguments. :param length: Max message length :param cls_name: Optional class name preﬁx avocado.utils.debug.log_calls_class(length=None) Use this as decorator to log the function methods’ calls. :param length: Max message length avocado.utils.debug.measure_duration(func) Use this as decorator to measure duration of the function execution. The output is “Function $name: ($cur- rent_duration, $accumulated_duration)”

20.2.13 avocado.utils.disk module

Disk utilities avocado.utils.disk.freespace(path)

20.2. Utilities APIs 157 avocado Documentation, Release 63.0

20.2.14 avocado.utils.distro module

This module provides the client facilities to detect the Linux Distribution it’s running under. class avocado.utils.distro.LinuxDistro(name, version, release, arch) Bases: object Simple collection of information for a Linux Distribution Initializes a new Linux Distro Parameters • name (str) – a short name that precisely distinguishes this Linux Distribution among all others. • version (str) – the major version of the distribution. Usually this is a single number that denotes a large development cycle and support file. • release (str) – the release or minor version of the distribution. Usually this is also a single number, that is often omitted or starts with a 0 when the major version is initially release. It’s often associated with a shorter development cycle that contains incremental a collection of improvements and fixes. • arch (str) – the main target for this Linux Distribution. It’s common for some architectures to ship with packages for previous and still compatible architectures, such as it’s the case with Intel/AMD 64 bit architecture that support 32 bit code. In cases like this, this should be set to the 64 bit architecture name. class avocado.utils.distro.Probe Bases: object Probes the machine and does it best to confirm it’s the right distro CHECK_FILE = None Points to a file that can determine if this machine is running a given Linux Distribution. This servers a first check that enables the extra checks to carry on. CHECK_FILE_CONTAINS = None Sets the content that should be checked on the file pointed to by CHECK_FILE_EXISTS. Leave it set to None (its default) to check only if the file exists, and not check its contents CHECK_FILE_DISTRO_NAME = None The name of the Linux Distribution to be returned if the file defined by CHECK_FILE_EXISTS exist. CHECK_VERSION_REGEX = None A regular expression that will be run on the file pointed to by CHECK_FILE_EXISTS check_name_for_file() Checks if this class will look for a file and return a distro The conditions that must be true include the file that identifies the distro file being set (CHECK_FILE) and the name of the distro to be returned (CHECK_FILE_DISTRO_NAME) check_name_for_file_contains() Checks if this class will look for text on a file and return a distro The conditions that must be true include the file that identifies the distro file being set (CHECK_FILE), the text to look for inside the distro file (CHECK_FILE_CONTAINS) and the name of the distro to be returned (CHECK_FILE_DISTRO_NAME) check_release() Checks if this has the conditions met to look for the release number

158 Chapter 20. API Reference avocado Documentation, Release 63.0

check_version() Checks if this class will look for a regex in ﬁle and return a distro get_distro() Returns the LinuxDistro this probe detected name_for_file() Get the distro name if the CHECK_FILE is set and exists name_for_file_contains() Get the distro if the CHECK_FILE is set and has content release() Returns the release of the distro version() Returns the version of the distro avocado.utils.distro.register_probe(probe_class) Register a probe to be run during autodetection avocado.utils.distro.detect() Attempts to detect the Linux Distribution running on this machine Returns the detected LinuxDistro or UNKNOWN_DISTRO Return type LinuxDistro

20.2.15 avocado.utils.download module

Methods to download URLs and regular files. avocado.utils.download.get_file(src, dst, permissions=None, hash_expected=None, hash_algorithm=’md5’, download_retries=1) Gets a file from a source location, optionally using caching. If no hash_expected is provided, simply download the file. Else, keep trying to download the file until down- load_failures exceeds download_retries or the hashes match. If the hashes match, return dst. If download_failures exceeds download_retries, raise an EnvironmentError. Parameters • src – source path or URL. May be local or a remote file. • dst – destination path. • permissions – (optional) set access permissions. • hash_expected – Hash string that we expect the file downloaded to have. • hash_algorithm – Algorithm used to calculate the hash string (md5, sha1). • download_retries – Number of times we are going to retry a failed download. Raise EnvironmentError. Returns destination path. avocado.utils.download.url_download(url, filename, data=None, timeout=300) Retrieve a file from given url. Parameters • url – source URL.

20.2. Utilities APIs 159 avocado Documentation, Release 63.0

• filename – destination path. • data – (optional) data to post. • timeout – (optional) default timeout in seconds. Returns None. avocado.utils.download.url_download_interactive(url, output_file, title=”, chunk_size=102400) Interactively downloads a given file url to a given output file. Parameters • url (string) – URL for the file to be download • output_file (string) – file name or absolute path on which to save the file to • title (string) – optional title to go along the progress bar • chunk_size (integer) – amount of data to read at a time avocado.utils.download.url_open(url, data=None, timeout=5) Wrapper to urllib2.urlopen() with timeout addition. Parameters • url – URL to open. • data – (optional) data to post. • timeout – (optional) default timeout in seconds. Returns file-like object. Raises URLError.

20.2.16 avocado.utils.ﬁlelock module

Utility for individual file access control implemented via PID lock files. exception avocado.utils.filelock.AlreadyLocked Bases: exceptions.Exception class avocado.utils.filelock.FileLock(filename, timeout=0) Bases: object Creates an exclusive advisory lock for a file. All processes should use and honor the advisory locking scheme, but uncooperative processes are free to ignore the lock and access the file in any way they choose. exception avocado.utils.filelock.LockFailed Bases: exceptions.Exception

20.2.17 avocado.utils.gdb module

Module that provides communication with GDB via its GDB/MI interpreter class avocado.utils.gdb.GDB(path=’/usr/bin/gdb’, *extra_args) Bases: object Wraps a GDB subprocess for easier manipulation DEFAULT_BREAK = 'main'

160 Chapter 20. API Reference avocado Documentation, Release 63.0

REQUIRED_ARGS = ['--interpreter=mi', '--quiet'] cli_cmd(command) Sends a cli command encoded as an MI command Parameters command (str) – a regular GDB cli command Returns a CommandResult instance Return type CommandResult cmd(command) Sends a command and parses all lines until prompt is received Parameters command (str) – the GDB command, hopefully in MI language Returns a CommandResult instance Return type CommandResult cmd_exists(command) Checks if a given command exists Parameters command (str) – a GDB MI command, including the dash (-) preﬁx Returns either True or False Return type bool connect(port) Connects to a remote debugger (a gdbserver) at the given TCP port This uses the “extended-remote” target type only Parameters port (int) – the TCP port number Returns a CommandResult instance Return type CommandResult del_break(number) Deletes a breakpoint by its number Parameters number (int) – the breakpoint number Returns a CommandResult instance Return type CommandResult disconnect() Disconnects from a remote debugger Returns a CommandResult instance Return type CommandResult exit() Exits the GDB application gracefully Returns the result of subprocess.POpen.wait(), that is, a subprocess.POpen. returncode Return type int or None read_gdb_response(timeout=0.01, max_tries=100) Read raw responses from GDB Parameters

20.2. Utilities APIs 161 avocado Documentation, Release 63.0

• timeout (float) – the amount of time to way between read attempts • max_tries (int) – the maximum number of cycles to try to read until a response is obtained Returns a string containing a raw response from GDB Return type str read_until_break(max_lines=100) Read lines from GDB until a break condition is reached Parameters max_lines (int) – the maximum number of lines to read Returns a list of messages read Return type list of str run(args=None) Runs the application inside the debugger Parameters args (builtin.list) – the arguments to be passed to the binary as command line arguments Returns a CommandResult instance Return type CommandResult send_gdb_command(command) Send a raw command to the GNU debugger input Parameters command (str) – the GDB command, hopefully in MI language Returns None set_break(location, ignore_error=False) Sets a new breakpoint on the binary currently being debugged Parameters location (str) – a breakpoint location expression as accepted by GDB Returns a CommandResult instance Return type CommandResult set_file(path) Sets the ﬁle that will be executed Parameters path (str) – the path of the binary that will be executed Returns a CommandResult instance Return type CommandResult class avocado.utils.gdb.GDBServer(path=’/usr/bin/gdbserver’, port=None, wait_until_running=True, *extra_args) Bases: object Wraps a gdbserver instance Initializes a new gdbserver instance Parameters • path (str) – location of the gdbserver binary • port (int) – tcp port number to listen on for incoming connections

162 Chapter 20. API Reference avocado Documentation, Release 63.0

• wait_until_running (bool) – wait until the gdbserver is running and accepting connections. It may take a little after the process is started and it is actually bound to the allocated port • extra_args – optional extra arguments to be passed to gdbserver INIT_TIMEOUT = 5.0 The time to optionally wait for the server to initialize itself and be ready to accept new connections PORT_RANGE = (20000, 20999) The range from which a port to GDB server will try to be allocated from REQUIRED_ARGS = ['--multi'] The default arguments used when starting the GDB server process exit(force=True) Quits the gdb_server process Most correct way of quitting the GDB server is by sending it a command. If no GDB client is connected, then we can try to connect to it and send a quit command. If this is not possible, we send it a signal and wait for it to ﬁnish. Parameters force (bool) – if a forced exit (sending SIGTERM) should be attempted Returns None class avocado.utils.gdb.GDBRemote(host, port, no_ack_mode=True, extended_mode=True) Bases: object Initializes a new GDBRemote object. A GDBRemote acts like a client that speaks the GDB remote protocol, documented at: https://sourceware.org/gdb/current/onlinedocs/gdb/Remote-Protocol.html Caveat: we currently do not support communicating with devices, only with TCP sockets. This limitation is basically due to the lack of use cases that justify an implementation, but not due to any technical shortcoming. Parameters • host (str) – the IP address or host name • port (int) – the port number where the the remote GDB is listening on • no_ack_mode (bool) – if the packet transmission conﬁrmation mode should be disabled • extended_mode – if the remote extended mode should be enabled cmd(command_data, expected_response=None) Sends a command data to a remote gdb server Limitations: the current version does not deal with retransmissions. Parameters • command_data (str) – the remote command to send the the remote stub • expected_response (str) – the (optional) response that is expected as a response for the command sent Raises RetransmissionRequestedError, UnexpectedResponseError Returns raw data read from from the remote server Return type str

20.2. Utilities APIs 163 avocado Documentation, Release 63.0

connect() Connects to the remote target and initializes the chosen modes set_extended_mode() Enable extended mode. In extended mode, the remote server is made persistent. The ‘R’ packet is used to restart the program being debugged. Original documentation at: https://sourceware.org/gdb/current/onlinedocs/gdb/Packets.html#extended-mode start_no_ack_mode() Request that the remote stub disable the normal +/- protocol acknowledgments. Original documentation at: https://sourceware.org/gdb/current/onlinedocs/gdb/General-Query-Packets.html#QStartNoAckMode

20.2.18 avocado.utils.genio module

Avocado generic IO related functions. exception avocado.utils.genio.GenIOError Bases: exceptions.Exception Base Exception Class for all IO exceptions avocado.utils.genio.ask(question, auto=False) Prompt the user with a (y/n) question. Parameters • question (str) – Question to be asked • auto (bool) – Whether to return “y” instead of asking the question Returns User answer Return type str avocado.utils.genio.close_log_file(filename) avocado.utils.genio.log_line(filename, line) Write a line to a file. Parameters • filename – Path of file to write to, either absolute or relative to the dir set by set_log_file_dir(). • line – Line to write. avocado.utils.genio.read_all_lines(filename) Return all lines of a given file This utility method returns an empty list in any error scenario, that is, it doesn’t attempt to identify error paths and raise appropriate exceptions. It does exactly the opposite to that. This should be used when it’s fine or desirable to have an empty set of lines if a file is missing or is unreadable. Parameters filename (str) – Path to the file. Returns all lines of the file as list Return type builtin.list avocado.utils.genio.read_file(filename) Read the entire contents of file.

164 Chapter 20. API Reference avocado Documentation, Release 63.0

Parameters filename (str) – Path to the file. Returns File contents Return type str avocado.utils.genio.read_one_line(filename) Read the first line of filename. Parameters filename (str) – Path to the file. Returns First line contents Return type str avocado.utils.genio.set_log_file_dir(directory) Set the base directory for log files created by log_line(). Parameters dir – Directory for log files. avocado.utils.genio.write_file(filename, data) Write data to a file. Parameters • filename (str) – Path to the file. • line (str) – Line to be written. avocado.utils.genio.write_file_or_fail(filename, data) Write to a file and raise exception on write failure Parameters • filename (str) – Path to file • data (str) – Data to be written to file Raises GenIOError – On write Failure avocado.utils.genio.write_one_line(filename, line) Write one line of text to filename. Parameters • filename (str) – Path to the file. • line (str) – Line to be written.

20.2.19 avocado.utils.git module

APIs to download/update git repositories from inside python scripts. class avocado.utils.git.GitRepoHelper(uri, branch=’master’, lbranch=None, commit=None, destination_dir=None, base_uri=None) Bases: object Helps to deal with git repos, mostly fetching content from a repo Instantiates a new GitRepoHelper Parameters • uri (string) – git repository url • branch (string) – git remote branch

20.2. Utilities APIs 165 avocado Documentation, Release 63.0

• lbranch (string) – git local branch name, if different from remote • commit (string) – specific commit to download • destination_dir (string) – path of a dir where to save downloaded code • base_uri (string) – a closer, usually local, git repository url from where to fetch content first from checkout(branch=None, commit=None) Performs a git checkout for a given branch and start point (commit) Parameters • branch – Remote branch name. • commit – Specific commit hash. execute() Performs all steps necessary to initialize and download a git repo. This includes the init, fetch and checkout steps in one single utility method. fetch(uri) Performs a git fetch from the remote repo get_top_commit() Returns the topmost commit id for the current branch. Returns Commit id. get_top_tag() Returns the topmost tag for the current branch. Returns Tag. git_cmd(cmd, ignore_status=False) Wraps git commands. Parameters • cmd – Command to be executed. • ignore_status – Whether we should suppress error.CmdError exceptions if the command did return exit code !=0 (True), or not suppress them (False). init() Initializes a directory for receiving a verbatim copy of git repo This creates a directory if necessary, and either resets or inits the repo avocado.utils.git.get_repo(uri, branch=’master’, lbranch=None, commit=None, destina- tion_dir=None, base_uri=None) Utility function that retrieves a given git code repository. Parameters • uri (string) – git repository url • branch (string) – git remote branch • lbranch (string) – git local branch name, if different from remote • commit (string) – specific commit to download • destination_dir (string) – path of a dir where to save downloaded code

166 Chapter 20. API Reference avocado Documentation, Release 63.0

• base_uri (string) – a closer, usually local, git repository url from where to fetch content ﬁrst from

20.2.20 avocado.utils.iso9660 module

Basic ISO9660 file-system support. This code does not attempt (so far) to implement code that knows about ISO9660 internal structure. Instead, it uses commonly available support either in userspace tools or on the Linux kernel itself (via mount). avocado.utils.iso9660.iso9660(path) Checks the available tools on a system and chooses class accordingly This is a convenience function, that will pick the first available iso9660 capable tool. Parameters path (str) – path to an iso9660 image file Returns an instance of any iso9660 capable tool Return type Iso9660IsoInfo, Iso9660IsoRead, Iso9660Mount or None class avocado.utils.iso9660.Iso9660IsoInfo(path) Bases: avocado.utils.iso9660.MixInMntDirMount, avocado.utils.iso9660. BaseIso9660 Represents a ISO9660 filesystem This implementation is based on the cdrkit’s isoinfo tool read(path) Abstract method to read data from path Parameters path – path to the file Returns data content from the file Return type str class avocado.utils.iso9660.Iso9660IsoRead(path) Bases: avocado.utils.iso9660.MixInMntDirMount, avocado.utils.iso9660. BaseIso9660 Represents a ISO9660 filesystem This implementation is based on the libcdio’s iso-read tool close() Cleanups and frees any resources being used copy(src, dst) Simplistic version of copy that relies on read() Parameters • src (str) – source path • dst (str) – destination path Return type None read(path) Abstract method to read data from path Parameters path – path to the file Returns data content from the file

20.2. Utilities APIs 167 avocado Documentation, Release 63.0

Return type str class avocado.utils.iso9660.Iso9660Mount(path) Bases: avocado.utils.iso9660.BaseIso9660 Represents a mounted ISO9660 filesystem. initializes a mounted ISO9660 filesystem Parameters path (str) – path to the ISO9660 file close() Perform umount operation on the temporary dir Return type None copy(src, dst) Parameters • src (str) – source • dst (str) – destination Return type None mnt_dir read(path) Read data from path Parameters path (str) – path to read data Returns data content Return type str

20.2.21 avocado.utils.kernel module class avocado.utils.kernel.KernelBuild(version, config_path=None, work_dir=None, data_dirs=None) Bases: object Build the Linux Kernel from official tarballs. Creates an instance of KernelBuild. Parameters • version – kernel version (“3.19.8”). • config_path – path to config file. • work_dir – work directory. • data_dirs – list of directories to keep the downloaded kernel Returns None. SOURCE = 'linux-{version}.tar.gz' URL = 'https://www.kernel.org/pub/linux/kernel/v{major}.x/' build(binary_package=False) Build kernel from source.

168 Chapter 20. API Reference avocado Documentation, Release 63.0

Parameters binary_package – when True, the appropriate platform package is built for install() to use configure() Conﬁgure/prepare kernel source to build. download(url=None) Download kernel source. Parameters url (str or None) – override the url from where to fetch the kernel source tarball install() Install built kernel. uncompress() Uncompress kernel source. avocado.utils.kernel.check_version(version) This utility function compares the current kernel version with the version parameter and gives assertion error if the version parameter is greater. Parameters version (string) – version to be compared with current kernel version

20.2.22 avocado.utils.linux_modules module

Linux kernel modules APIs avocado.utils.linux_modules.BUILTIN = 2 Config built-in to kernel (=y) avocado.utils.linux_modules.MODULE = 1 Config compiled as loadable module (=m) avocado.utils.linux_modules.NOT_SET = 0 Config commented out or not set avocado.utils.linux_modules.check_kernel_config(config_name) Reports the configuration of $config_name of the current kernel Parameters config_name (str) – Name of kernel config to search Returns Config status in running kernel (NOT_SET, BUILTIN, MODULE) Return type int avocado.utils.linux_modules.get_loaded_modules() Gets list of loaded modules. :return: List of loaded modules. avocado.utils.linux_modules.get_submodules(module_name) Get all submodules of the module. Parameters module_name (str) – Name of module to search for Returns List of the submodules Return type builtin.list avocado.utils.linux_modules.load_module(module_name) Checks if a module has already been loaded. :param module_name: Name of module to check :return: True if module is loaded, False otherwise :rtype: Bool avocado.utils.linux_modules.loaded_module_info(module_name) Get loaded module details: Size and Submodules.

20.2. Utilities APIs 169 avocado Documentation, Release 63.0

Parameters module_name (str) – Name of module to search for Returns Dictionary of module name, size, submodules if present, ﬁlename, version, number of modules using it, list of modules it is dependent on, list of dictionary of param name and type Return type dict avocado.utils.linux_modules.module_is_loaded(module_name) Is module loaded Parameters module_name (str) – Name of module to search for Returns True if module is loaded Return type bool avocado.utils.linux_modules.parse_lsmod_for_module(l_raw, module_name, escape=True) Use a regexp to parse raw lsmod output and get module information :param l_raw: raw output of lsmod :type l_raw: str :param module_name: Name of module to search for :type module_name: str :param escape: Escape regexp tokens in module_name, default True :type escape: bool :return: Dictionary of module info, name, size, submodules if present :rtype: dict avocado.utils.linux_modules.unload_module(module_name) Removes a module. Handles dependencies. If even then it’s not possible to remove one of the modules, it will throw an error.CmdError exception. Parameters module_name (str) – Name of the module we want to remove.

20.2.23 avocado.utils.lv_utils module exception avocado.utils.lv_utils.LVException Bases: exceptions.Exception Base Exception Class for all exceptions avocado.utils.lv_utils.get_diskspace(disk) Get the entire disk space of a given disk Parameters disk – Name of the disk to ﬁnd free space Returns size in bytes avocado.utils.lv_utils.lv_check(vg_name, lv_name) Check whether provided Logical volume exists. Parameters • vg_name – Name of the volume group • lv_name – Name of the logical volume avocado.utils.lv_utils.lv_create(vg_name, lv_name, lv_size, force_ﬂag=True) Create a Logical volume in a volume group. The volume group must already exist. Parameters • vg_name – Name of the volume group • lv_name – Name of the logical volume • lv_size – Size for the logical volume to be created

170 Chapter 20. API Reference avocado Documentation, Release 63.0 avocado.utils.lv_utils.lv_list() List available group volumes. :return list available logical volumes avocado.utils.lv_utils.lv_mount(vg_name, lv_name, mount_loc, create_ﬁlesystem=”) Mount a Logical volume to a mount location. Parameters • vg_name – Name of volume group • lv_name – Name of the logical volume • create_filesystem – Can be one of ext2, ext3, ext4, vfat or empty if the ﬁlesystem was already created and the mkfs process is skipped Mount_loc Location to mount the logical volume avocado.utils.lv_utils.lv_reactivate(vg_name, lv_name, timeout=10) In case of unclean shutdowns some of the lvs is still active and merging is postponed. Use this function to attempt to deactivate and reactivate all of them to cause the merge to happen. Parameters • vg_name – Name of volume group • lv_name – Name of the logical volume • timeout – Timeout between operations avocado.utils.lv_utils.lv_remove(vg_name, lv_name) Remove a logical volume. Parameters • vg_name – Name of the volume group • lv_name – Name of the logical volume avocado.utils.lv_utils.lv_revert(vg_name, lv_name, lv_snapshot_name) Revert the origin to a snapshot. Parameters • vg_name – An existing volume group • lv_name – An existing logical volume • lv_snapshot_name – Name of the snapshot be to reverted avocado.utils.lv_utils.lv_revert_with_snapshot(vg_name, lv_name, lv_snapshot_name, lv_snapshot_size) Perform Logical volume merge with snapshot and take a new snapshot. Parameters • vg_name – Name of volume group in which lv has to be reverted • lv_name – Name of the logical volume to be reverted • lv_snapshot_name – Name of the snapshot be to reverted • lv_snapshot_size – Size of the snapshot avocado.utils.lv_utils.lv_take_snapshot(vg_name, lv_name, lv_snapshot_name, lv_snapshot_size) Take a snapshot of the original Logical volume.

20.2. Utilities APIs 171 avocado Documentation, Release 63.0

Parameters • vg_name – An existing volume group • lv_name – An existing logical volume • lv_snapshot_name – Name of the snapshot be to created • lv_snapshot_size – Size of the snapshot avocado.utils.lv_utils.lv_umount(vg_name, lv_name) Unmount a Logical volume from a mount location. Parameters • vg_name – Name of volume group • lv_name – Name of the logical volume avocado.utils.lv_utils.thin_lv_create(vg_name, thinpool_name=’lvthinpool’, thin- pool_size=’1.5G’, thinlv_name=’lvthin’, thinlv_size=’1G’) Create a thin volume from given volume group. Parameters • vg_name – An exist volume group • thinpool_name – The name of thin pool • thinpool_size – The size of thin pool to be created • thinlv_name – The name of thin volume • thinlv_size – The size of thin volume avocado.utils.lv_utils.vg_check(vg_name) Check whether provided volume group exists. Parameters vg_name – Name of the volume group. avocado.utils.lv_utils.vg_create(vg_name, pv_list, force=False) Create a volume group by using the block special devices Parameters • vg_name – Name of the volume group • pv_list – List of physical volumes • force – Create volume group forcefully avocado.utils.lv_utils.vg_list() List available volume groups. :return List of volume groups. avocado.utils.lv_utils.vg_ramdisk(disk, vg_name, ramdisk_vg_size, ramdisk_basedir, ramdisk_sparse_ﬁlename) Create vg on top of ram memory to speed up lv performance. When disk is speciﬁed size of the physical volume is taken from existing disk space. Parameters • disk – Name of the disk in which volume groups are created. • vg_name – Name of the volume group. • ramdisk_vg_size – Size of the ramdisk virtual group (MB).

172 Chapter 20. API Reference avocado Documentation, Release 63.0

• ramdisk_basedir – Base directory for the ramdisk sparse file. • ramdisk_sparse_filename – Name of the ramdisk sparse file. Returns ramdisk_filename, vg_ramdisk_dir, vg_name, loop_device Raises LVException – On failure Sample ramdisk params: - ramdisk_vg_size = “40000” - ramdisk_basedir = “/tmp” - ramdisk_sparse_filename = “virtual_hdd” Sample general params: - vg_name=’autotest_vg’, - lv_name=’autotest_lv’, - lv_size=‘1G’, - lv_snapshot_name=’autotest_sn’, - lv_snapshot_size=‘1G’ The ramdisk volume group size is in MB. avocado.utils.lv_utils.vg_ramdisk_cleanup(ramdisk_filename=None, vg_ramdisk_dir=None, vg_name=None, loop_device=None) Inline cleanup function in case of test error. It detects whether the components were initialized and if so it tries to remove them. In case of failure it raises summary exception. Parameters • ramdisk_filename – Name of the ramdisk sparse file. • vg_ramdisk_dir – Location of the ramdisk file Vg_name Name of the volume group Loop_device Name of the disk or loop device Raises LVException – In case it fail to clean things detected in system avocado.utils.lv_utils.vg_remove(vg_name) Remove a volume group. Parameters vg_name – Name of the volume group

20.2.24 avocado.utils.memory module exception avocado.utils.memory.MemError Bases: exceptions.Exception called when memory operations fails class avocado.utils.memory.MemInfo Bases: object Representation of /proc/meminfo avocado.utils.memory.check_hotplug() Check kernel support for memory hotplug Returns True if hotplug supported, else False Return type ‘bool’ avocado.utils.memory.drop_caches() Writes back all dirty pages to disk and clears all the caches. avocado.utils.memory.freememtotal() Read MemFree from meminfo.

20.2. Utilities APIs 173 avocado Documentation, Release 63.0

avocado.utils.memory.get_blk_string(block) Format the given block id to string Parameters block – memory block id or block string. Returns returns string memory198 if id 198 is given Return type string avocado.utils.memory.get_buddy_info(chunk_sizes, nodes=’all’, zones=’all’) Get the fragement status of the host. It uses the same method to get the page size in buddyinfo. The expression to evaluate it is:

2^chunk_size * page_size

The chunk_sizes can be string make up by all orders that you want to check split with blank or a mathematical expression with >, < or =. For example: • The input of chunk_size could be: 0 2 4, and the return will be {'0': 3, '2': 286, '4': 687} • If you are using expression: >=9 the return will be {'9': 63, '10': 225}

Parameters • chunk_size (string) – The order number shows in buddyinfo. This is not the real page size. • nodes (string) – The numa node that you want to check. Default value is all • zones (string) – The memory zone that you want to check. Default value is all Returns A dict using the chunk_size as the keys Return type dict avocado.utils.memory.get_huge_page_size() Get size of the huge pages for this system. Returns Huge pages size (KB). avocado.utils.memory.get_num_huge_pages() Get number of huge pages for this system. Returns Number of huge pages. avocado.utils.memory.get_page_size() Get linux page size for this system. :return Kernel page size (Bytes). avocado.utils.memory.get_thp_value(feature) Gets the value of the thp feature arg passed Param feature Thp feature to get value avocado.utils.memory.hotplug(block) Online the memory for the given block id. Parameters block – memory block id or or memory198 avocado.utils.memory.hotunplug(block) Ofﬂine the memory for the given block id.

174 Chapter 20. API Reference avocado Documentation, Release 63.0

Parameters block – memory block id. avocado.utils.memory.is_hot_pluggable(block) Check if the given memory block is hotpluggable Parameters block – memory block id. Returns True if hotpluggable, else False Return type ‘bool’ avocado.utils.memory.memtotal() Read Memtotal from meminfo. avocado.utils.memory.memtotal_sys() Reports actual memory size according to online-memory blocks available via “/sys” Returns system memory in Kb as float avocado.utils.memory.node_size() Return node size. Returns Node size. avocado.utils.memory.numa_nodes() Get a list of NUMA nodes present on the system. Returns List with nodes. avocado.utils.memory.numa_nodes_with_memory() Get a list of NUMA nodes present with memory on the system. Returns List with nodes which has memory. avocado.utils.memory.read_from_meminfo(key) Retrieve key from meminfo. Parameters key – Key name, such as MemTotal. avocado.utils.memory.read_from_numa_maps(pid, key) Get the process numa related info from numa_maps. This function only use to get the numbers like anon=1. Parameters • pid (String) – Process id • key (String) – The item you want to check from numa_maps Returns A dict using the address as the keys Return type dict avocado.utils.memory.read_from_smaps(pid, key) Get specific item value from the smaps of a process include all sections. Parameters • pid (String) – Process id • key (String) – The item you want to check from smaps Returns The value of the item in kb Return type int avocado.utils.memory.read_from_vmstat(key) Get specific item value from vmstat

20.2. Utilities APIs 175 avocado Documentation, Release 63.0

Parameters key (String) – The item you want to check from vmstat Returns The value of the item Return type int avocado.utils.memory.rounded_memtotal() Get memtotal, properly rounded. Returns Total memory, KB. avocado.utils.memory.set_num_huge_pages(num) Set number of huge pages. Parameters num – Target number of huge pages. avocado.utils.memory.set_thp_value(feature, value) Sets THP feature to a given value Parameters • feature (str) – Thp feature to set • value (str) – Value to be set to feature

20.2.25 avocado.utils.multipath module

Module with multipath related utility functions. It needs root access. avocado.utils.multipath.device_exists(path) Checks if a given path exists. Returns True if path exists, False if does not exist. avocado.utils.multipath.fail_path(path) failing the individual paths :param disk_path: disk path. Example: sda, sdb. :return: True or False avocado.utils.multipath.flush_path(path_name) Flushes the given multipath. Returns Returns False if command fails, True otherwise. avocado.utils.multipath.form_conf_mpath_file(blacklist=”, defaults_extra=”) Form a multipath configuration file, and restart multipath service. Parameters • blacklist – Entry in conf file to indicate blacklist section. • defaults_extra – Extra entry in conf file in defaults section. avocado.utils.multipath.get_mpath_name(wwid) Get multipath name for a given wwid. Parameters wwid – wwid of multipath device. Returns Name of multipath device. avocado.utils.multipath.get_multipath_details() Get multipath details as a dictionary, as given by the command: multipathd show maps json Returns Dictionary of multipath output in json format. avocado.utils.multipath.get_multipath_wwids() Get list of multipath wwids.

176 Chapter 20. API Reference avocado Documentation, Release 63.0

Returns List of multipath wwids. avocado.utils.multipath.get_path_status(disk_path) Return the status of a path in multipath. Parameters disk_path – disk path. Example: sda, sdb. Returns Tuple in the format of (dm status, dev status, checker status) avocado.utils.multipath.get_paths(wwid) Get list of paths, given a multipath wwid. Returns List of paths. avocado.utils.multipath.get_policy(wwid) Gets path_checker policy, given a multipath wwid. Returns path checker policy. avocado.utils.multipath.get_size(wwid) Gets size of device, given a multipath wwid. Returns size of multipath device. avocado.utils.multipath.get_svc_name() Gets the multipath service name based on distro. avocado.utils.multipath.is_path_a_multipath(disk_path) Check if given disk path is part of a multipath. Parameters disk_path – disk path. Example: sda, sdb. Returns True if part of multipath, else False. avocado.utils.multipath.reinstate_path(path) reinstating the individual paths :param disk_path: disk path. Example: sda, sdb. :return: True or False

20.2.26 avocado.utils.network module

Module with network related utility functions class avocado.utils.network.PortTracker Bases: avocado.utils.data_structures.Borg Tracks ports used in the host machine. find_free_port(start_port=None) register_port(port) release_port(port) avocado.utils.network.find_free_port(start_port, end_port, address=’localhost’, sequent=True) Return a host free port in the range [start_port, end_port]. Parameters • start_port – header of candidate port range • end_port – ender of candidate port range • sequent – Find port sequentially, random order if it’s False • address – Socket address to bind or connect

20.2. Utilities APIs 177 avocado Documentation, Release 63.0

avocado.utils.network.find_free_ports(start_port, end_port, count, address=’localhost’, sequent=True) Return count of host free ports in the range [start_port, end_port]. Parameters • start_port – header of candidate port range • end_port – ender of candidate port range • count – Initial number of ports known to be free in the range. • address – Socket address to bind or connect • sequent – Find port sequentially, random order if it’s False avocado.utils.network.is_port_free(port, address) Return True if the given port is available for use. Parameters • port – Port number • address – Socket address to bind or connect

20.2.27 avocado.utils.output module

Utility functions for user friendly display of information. class avocado.utils.output.ProgressBar(minimum=0, maximum=100, width=75, title=”) Bases: object Displays interactively the progress of a given task Inspired/adapted from https://gist.github.com/t0xicCode/3306295 Initializes a new progress bar Parameters • minimum (integer) – minimum (initial) value on the progress bar • maximum (integer) – maximum (ﬁnal) value on the progress bar • with – number of columns, that is screen width append_amount(amount) Increments the current amount value. draw() Prints the updated text to the screen. update_amount(amount) Performs sanity checks and update the current amount. update_percentage(percentage) Updates the progress bar to the new percentage. avocado.utils.output.display_data_size(size) Display data size in human readable units (SI). Parameters size (int) – Data size, in Bytes. Returns Human readable string with data size, using SI preﬁxes.

178 Chapter 20. API Reference avocado Documentation, Release 63.0

20.2.28 avocado.utils.partition module

Utility for handling partitions. class avocado.utils.partition.MtabLock Bases: object mtab = None class avocado.utils.partition.Partition(device, loop_size=0, mountpoint=None) Bases: object Class for handling partitions and filesystems Parameters • device – The device in question (e.g.”/dev/hda2”). If device is a file it will be mounted as loopback. • loop_size – Size of loopback device (in MB). Defaults to 0. • mountpoint – Where the partition to be mounted to. get_mountpoint(filename=None) Find the mount point of this partition object. Parameters filename – where to look for the mounted partitions information (default None which means it will search /proc/mounts and/or /etc/mtab) Returns a string with the mount point of the partition or None if not mounted static list_mount_devices() Lists mounted file systems and swap on devices. static list_mount_points() Lists the mount points. mkfs(fstype=None, args=”) Format a partition to filesystem type Parameters • fstype – the filesystem type, such as “ext3”, “ext2”. Defaults to previously set type or “ext2” if none has set. • args – arguments to be passed to mkfs command. mount(mountpoint=None, fstype=None, args=”) Mount this partition to a mount point Parameters • mountpoint – If you have not provided a mountpoint to partition object or want to use a different one, you may specify it here. • fstype – Filesystem type. If not provided partition object value will be used. • args – Arguments to be passed to “mount” command. unmount(force=True) Umount this partition. It’s easier said than done to umount a partition. We need to lock the mtab file to make sure we don’t have any locking problems if we are umounting in parallel. When the unmount fails and force==True we unmount the partition ungracefully.

20.2. Utilities APIs 179 avocado Documentation, Release 63.0

Returns 1 on success, 2 on force umount success Raises PartitionError – On failure exception avocado.utils.partition.PartitionError(partition, reason, details=None) Bases: exceptions.Exception Generic PartitionError

20.2.29 avocado.utils.path module

Avocado path related functions. exception avocado.utils.path.CmdNotFoundError(cmd, paths) Bases: exceptions.Exception Indicates that the command was not found in the system after a search. Parameters • cmd – String with the command. • paths – List of paths where we looked after. class avocado.utils.path.PathInspector(path) Bases: object get_first_line() has_exec_permission() is_empty() is_python() is_script(language=None) avocado.utils.path.find_command(cmd, default=None) Try to find a command in the PATH, paranoid version. Parameters • cmd – Command to be found. • default – Command path to use as a fallback if not found in the standard directories. Raise avocado.utils.path.CmdNotFoundError in case the command was not found and no default was given. avocado.utils.path.get_path(base_path, user_path) Translate a user specified path to a real path. If user_path is relative, append it to base_path. If user_path is absolute, return it as is. Parameters • base_path – The base path of relative user specified paths. • user_path – The user specified path. avocado.utils.path.init_dir(*args) Wrapper around os.path.join that creates dirs based on the final path. Parameters args – List of dir arguments that will be os.path.joined. Returns directory. Return type str

180 Chapter 20. API Reference avocado Documentation, Release 63.0

avocado.utils.path.usable_ro_dir(directory) Verify whether dir exists and we can access its contents. If a usable RO is there, use it no questions asked. If not, let’s at least try to create one. Parameters directory – Directory avocado.utils.path.usable_rw_dir(directory) Verify whether we can use this dir (read/write). Checks for appropriate permissions, and creates missing dirs as needed. Parameters directory – Directory

20.2.30 avocado.utils.pci module

Module for all PCI devices related functions. avocado.utils.pci.get_cfg(dom_pci_address) Gets the hardware configuration data of the given PCI address. Note Specific for ppc64 processor. Parameters dom_pci_address – Partial PCI address including domain addr and at least bus addr (0003:00, 0003:00:1f.2, . . . ) Returns dictionary of configuration data of a PCI address. avocado.utils.pci.get_disks_in_pci_address(pci_address) Gets disks in a PCI address. Parameters pci_address – Any segment of a PCI address (1f, 0000:00:1f, . . . ) Returns list of disks in a PCI address. avocado.utils.pci.get_domains() Gets all PCI domains. Example, it returns [‘0000’, ‘0001’, . . . ] Returns List of PCI domains. avocado.utils.pci.get_driver(pci_address) Gets the kernel driver in use of given PCI address. (first match only) Parameters pci_address – Any segment of a PCI address (1f, 0000:00:1f, . . . ) Returns driver of a PCI address. avocado.utils.pci.get_interfaces_in_pci_address(pci_address, pci_class) Gets interface in a PCI address. e.g: host = pci.get_interfaces_in_pci_address(“0001:01:00.0”, “net”) [‘enP1p1s0f0’] host = pci.get_interfaces_in_pci_address(“0004:01:00.0”, “fc_host”) [‘host6’]

Parameters • pci_address – Any segment of a PCI address (1f, 0000:00:1f, . . . ) • class – Adapter type (FC(fc_host), FCoE(net), NIC(net), SCSI(scsi)..) Returns list of generic interfaces in a PCI address.

avocado.utils.pci.get_mask(pci_address) Gets the mask of PCI address. (ﬁrst match only) Note There may be multiple memory entries for a PCI address.

20.2. Utilities APIs 181 avocado Documentation, Release 63.0

Note This mask is calculated only with the first such entry. Parameters pci_address – Any segment of a PCI address (1f, 0000:00:1f, . . . ) Returns mask of a PCI address. avocado.utils.pci.get_memory_address(pci_address) Gets the memory address of a PCI address. (first match only) Note There may be multiple memory address for a PCI address. Note This function returns only the first such address. Parameters pci_address – Any segment of a PCI address (1f, 0000:00:1f, . . . ) Returns memory address of a pci_address. avocado.utils.pci.get_nics_in_pci_address(pci_address) Gets network interface(nic) in a PCI address. Parameters pci_address – Any segment of a PCI address (1f, 0000:00:1f, . . . ) Returns list of network interfaces in a PCI address. avocado.utils.pci.get_num_interfaces_in_pci(dom_pci_address) Gets number of interfaces of a given partial PCI address starting with full domain address. Parameters dom_pci_address – Partial PCI address including domain address (0000, 0000:00:1f, 0000:00:1f.2, etc) Returns number of devices in a PCI domain. avocado.utils.pci.get_pci_addresses() Gets list of PCI addresses in the system. Does not return the PCI Bridges/Switches. Returns list of full PCI addresses including domain (0000:00:14.0) avocado.utils.pci.get_pci_class_name(pci_address) Gets pci class name for given pci bus address e.g: >>> pci.get_pci_class_name(“0000:01:00.0”) ‘scsi_host’

Parameters pci_address – Any segment of a PCI address(1f, 0000:00:if, . . . ) Returns class name for corresponding pci bus address avocado.utils.pci.get_pci_fun_list(pci_address) Gets list of functions in the given PCI address. Example: in address 0000:03:00, functions are 0000:03:00.0 and 0000:03:00.1 Parameters pci_address – Any segment of a PCI address (1f, 0000:00:1f, . . . ) Returns list of functions in a PCI address. avocado.utils.pci.get_pci_id(pci_address) Gets PCI id of given address. (ﬁrst match only) Parameters pci_address – Any segment of a PCI address (1f, 0000:00:1f, . . . ) Returns PCI ID of a PCI address. avocado.utils.pci.get_pci_id_from_sysfs(full_pci_address) Gets the PCI ID from sysfs of given PCI address. Parameters full_pci_address – Full PCI address including domain (0000:03:00.0) Returns PCI ID of a PCI address from sysfs.

182 Chapter 20. API Reference avocado Documentation, Release 63.0

avocado.utils.pci.get_pci_prop(pci_address, prop) Gets specific PCI ID of given PCI address. (first match only) Parameters • pci_address – Any segment of a PCI address (1f, 0000:00:1f, . . . ) • part – prop of PCI ID. Returns specific PCI ID of a PCI address. avocado.utils.pci.get_slot_from_sysfs(full_pci_address) Gets the PCI slot of given address. Note Specific for ppc64 processor. Parameters full_pci_address – Full PCI address including domain (0000:03:00.0) Returns Removed port related details using re, only returns till physical slot of the adapter. avocado.utils.pci.get_slot_list() Gets list of PCI slots in the system. Note Specific for ppc64 processor. Returns list of slots in the system. avocado.utils.pci.get_vpd(dom_pci_address) Gets the VPD (Virtual Product Data) of the given PCI address. Note Specific for ppc64 processor. Parameters dom_pci_address – Partial PCI address including domain addr and at least bus addr (0003:00, 0003:00:1f.2, . . . ) Returns dictionary of VPD of a PCI address.

20.2.31 avocado.utils.process module

Functions dedicated to ﬁnd and run external commands. avocado.utils.process.CURRENT_WRAPPER = None The active wrapper utility script. exception avocado.utils.process.CmdError(command=None, result=None, addi- tional_text=None) Bases: exceptions.Exception class avocado.utils.process.CmdResult(command=”, stdout=”, stderr=”, exit_status=None, duration=0, pid=None, encoding=None) Bases: object Command execution result. Parameters • command (str) – the command line itself • exit_status (int) – exit code of the process • stdout (bytes) – content of the process stdout • stderr (bytes) – content of the process stderr • duration (float) – elapsed wall clock time running the process

20.2. Utilities APIs 183 avocado Documentation, Release 63.0

• pid (int) – ID of the process • encoding (str) – the encoding to use for the text version of stdout and stderr, by default avocado.utils.astring.ENCODING stderr = None The raw stderr (bytes) stderr_text stdout = None The raw stdout (bytes) stdout_text class avocado.utils.process.FDDrainer(fd, result, name=None, logger=None, log- ger_prefix=’%s’, stream_logger=None, ig- nore_bg_processes=False, verbose=False) Bases: object Reads data from a file descriptor in a thread, storing locally in a file-like data object. Parameters • fd (int) – a file descriptor that will be read (drained) from • result (a CmdResult instance) – a CmdResult instance associated with the process used to detect if the process is still running and if there’s still data to be read. • name (str) – a descriptive name that will be passed to the Thread name • logger (logging.Logger) – the logger that will be used to (interactively) write the content from the file descriptor • logger_prefix (str with one %-style string formatter) – the prefix used when logging the data • ignore_bg_processes (boolean) – When True the process does not wait for child processes which keep opened stdout/stderr streams after the main process finishes (eg. forked daemon which did not closed the stdout/stderr). Note this might result in missing output produced by those daemons after the main thread finishes and also it allows those daemons to be running after the process finishes. • verbose (boolean) – whether to log in both the logger and stream_logger flush() start() class avocado.utils.process.GDBSubProcess(cmd, verbose=True, allow_output_check=None, shell=False, env=None, sudo=False, ig- nore_bg_processes=False, encoding=None) Bases: object Runs a subprocess inside the GNU Debugger Creates the subprocess object, stdout/err, reader threads and locks. Parameters • cmd (str) – Command line to run. • allow_output_check – Currently ignored in GDBSubProcess • shell – Currently ignored in GDBSubProcess • env – Currently ignored in GDBSubProcess

184 Chapter 20. API Reference avocado Documentation, Release 63.0

• sudo – Currently ignored in GDBSubProcess • ignore_bg_processes – Currently ignored in GDBSubProcess • encoding (str) – the encoding to use for the text representation of the command result stdout and stderr, by default avocado.utils.astring.ENCODING Params verbose Currently ignored in GDBSubProcess create_and_wait_on_resume_fifo(path) Creates a FIFO file and waits until it’s written to Parameters path (str) – the path that the file will be created Returns first character that was written to the fifo Return type str generate_core() generate_gdb_connect_cmds() generate_gdb_connect_sh() handle_break_hit(response) handle_fatal_signal(response) run(timeout=None) wait_for_exit() Waits until debugger receives a message about the binary exit avocado.utils.process.OUTPUT_CHECK_RECORD_MODE = None The current output record mode. It’s not possible to record both the ‘stdout’ and ‘stderr’ streams, and at the same time in the right order, the combined ‘output’ stream. So this setting defines the mode. class avocado.utils.process.SubProcess(cmd, verbose=True, allow_output_check=None, shell=False, env=None, sudo=False, ig- nore_bg_processes=False, encoding=None) Bases: object Run a subprocess in the background, collecting stdout/stderr streams. Creates the subprocess object, stdout/err, reader threads and locks. Parameters • cmd (str) – Command line to run. • verbose (bool) – Whether to log the command run and stdout/stderr. • allow_output_check (str) – Whether to record the output from this process (from stdout and stderr) in the test’s output record files. Valid values: ‘stdout’, for standard output only, ‘stderr’ for standard error only, ‘both’ for both standard output and error in separate files, ‘combined’ for standard output and error in a single file, and ‘none’ to disable all recording. ‘all’ is also a valid, but deprecated, option that is a synonym of ‘both’. If an explicit value is not given to this parameter, that is, if None is given, it defaults to using the module level configuration, as set by OUTPUT_CHECK_RECORD_MODE. If the module level configuration itself is not set, it defaults to ‘none’. • shell (bool) – Whether to run the subprocess in a subshell. • env (dict) – Use extra environment variables.

20.2. Utilities APIs 185 avocado Documentation, Release 63.0

• sudo (bool) – Whether the command requires admin privileges to run, so that sudo will be prepended to the command. The assumption here is that the user running the command has a sudo configuration such that a password won’t be prompted. If that’s not the case, the command will straight out fail. • ignore_bg_processes – When True the process does not wait for child processes which keep opened stdout/stderr streams after the main process finishes (eg. forked daemon which did not closed the stdout/stderr). Note this might result in missing output produced by those daemons after the main thread finishes and also it allows those daemons to be running after the process finishes. • encoding (str) – the encoding to use for the text representation of the command result stdout and stderr, by default avocado.utils.astring.ENCODING Raises ValueError if incorrect values are given to parameters get_pid() Reports PID of this process get_stderr() Get the full stderr of the subprocess so far. Returns Standard error of the process. Return type str get_stdout() Get the full stdout of the subprocess so far. Returns Standard output of the process. Return type str kill() Send a signal.SIGKILL to the process. poll() Call the subprocess poll() method, fill results if rc is not None. run(timeout=None, sig=15) Start a process and wait for it to end, returning the result attr. If the process was already started using .start(), this will simply wait for it to end. Parameters • timeout (float) – Time (seconds) we’ll wait until the process is finished. If it’s not, we’ll try to terminate it and get a status. • sig (int) – Signal to send to the process in case it did not end after the specified timeout. Returns The command result object. Return type A CmdResult instance. send_signal(sig) Send the specified signal to the process. Parameters sig – Signal to send. start() Start running the subprocess. This method is particularly useful for background processes, since you can start the subprocess and not block your test flow.

186 Chapter 20. API Reference avocado Documentation, Release 63.0

Returns Subprocess PID. Return type int stop() Stop background subprocess. Call this method to terminate the background subprocess and wait for it results. terminate() Send a signal.SIGTERM to the process. wait() Call the subprocess poll() method, fill results if rc is not None. avocado.utils.process.UNDEFINED_BEHAVIOR_EXCEPTION = None Exception to be raised when users of this API need to know that the execution of a given process resulted in undefined behavior. One concrete example when a user, in an interactive session, let the inferior process exit before before avocado resumed the debugger session. Since the information is unknown, and the behavior is undefined, this situation will be flagged by an exception. avocado.utils.process.WRAP_PROCESS = None The global wrapper. If set, run every process under this wrapper. avocado.utils.process.WRAP_PROCESS_NAMES_EXPR = [] Set wrapper per program names. A list of wrappers and program names. Format: [ (‘/path/to/wrapper.sh’, ‘progname’), . . . ] class avocado.utils.process.WrapSubProcess(cmd, verbose=True, al- low_output_check=None, shell=False, env=None, wrapper=None, sudo=False, ig- nore_bg_processes=False, encoding=None) Bases: avocado.utils.process.SubProcess Wrap subprocess inside an utility program. avocado.utils.process.binary_from_shell_cmd(cmd) Tries to find the first binary path from a simple shell-like command. Note It’s a naive implementation, but for commands like: VAR=VAL binary -args || true gives the right result (binary) Parameters cmd (unicode string) – simple shell-like binary Returns first found binary from the cmd avocado.utils.process.can_sudo(cmd=None) Check whether sudo is available (or running as root) Parameters cmd – unicode string with the commands avocado.utils.process.cmd_split(cmd) Splits a command line into individual components This is a simple wrapper around shlex.split(), which has the requirement of having text (not bytes) as its argument on Python 3, but bytes on Python 2. Parameters cmd – text (a multi byte string) encoded as ‘utf-8’ avocado.utils.process.get_children_pids(ppid, recursive=False) Get all PIDs of children/threads of parent ppid param ppid: parent PID param recursive: True to return all levels of sub-processes return: list of PIDs of all children/threads of ppid

20.2. Utilities APIs 187 avocado Documentation, Release 63.0

avocado.utils.process.get_sub_process_klass(cmd) Which sub process implementation should be used Either the regular one, or the GNU Debugger version Parameters cmd – the command arguments, from where we extract the binary name avocado.utils.process.getoutput(cmd, timeout=None, verbose=False, ignore_status=True, allow_output_check=’combined’, shell=True, env=None, sudo=False, ignore_bg_processes=False) Because commands module is removed in Python3 and it redirect stderr to stdout, we port commands.getoutput to make code compatible Return output (stdout or stderr) of executing cmd in a shell. Parameters • cmd (str) – Command line to run. • timeout (float) – Time limit in seconds before attempting to kill the running process. This function will take a few seconds longer than ‘timeout’ to complete if it has to kill the process. • verbose (bool) – Whether to log the command run and stdout/stderr. • ignore_status – Whether to raise an exception when command returns =! 0 (False), or not (True). • allow_output_check (str) – Whether to record the output from this process (from stdout and stderr) in the test’s output record files. Valid values: ‘stdout’, for standard output only, ‘stderr’ for standard error only, ‘both’ for both standard output and error in separate files, ‘combined’ for standard output and error in a single file, and ‘none’ to disable all recording. ‘all’ is also a valid, but deprecated, option that is a synonym of ‘both’. If an explicit value is not given to this parameter, that is, if None is given, it defaults to using the module level configuration, as set by OUTPUT_CHECK_RECORD_MODE. If the module level configuration itself is not set, it defaults to ‘none’. • shell (bool) – Whether to run the command on a subshell • env (dict) – Use extra environment variables • sudo (bool) – Whether the command requires admin privileges to run, so that sudo will be prepended to the command. The assumption here is that the user running the command has a sudo configuration such that a password won’t be prompted. If that’s not the case, the command will straight out fail. • ignore_bg_processes (bool) – Whether to ignore background processes Returns Command output(stdout or stderr). Return type str avocado.utils.process.getstatusoutput(cmd, timeout=None, verbose=False, ig- nore_status=True, allow_output_check=’combined’, shell=True, env=None, sudo=False, ig- nore_bg_processes=False) Because commands module is removed in Python3 and it redirect stderr to stdout, we port commands.getstatusoutput to make code compatible Return (status, output) of executing cmd in a shell. Parameters • cmd (str) – Command line to run. • timeout (float) – Time limit in seconds before attempting to kill the running process. This function will take a few seconds longer than ‘timeout’ to complete if it has to kill the process.

188 Chapter 20. API Reference avocado Documentation, Release 63.0

• verbose (bool) – Whether to log the command run and stdout/stderr. • ignore_status – Whether to raise an exception when command returns =! 0 (False), or not (True). • allow_output_check (str) – Whether to record the output from this process (from stdout and stderr) in the test’s output record files. Valid values: ‘stdout’, for standard output only, ‘stderr’ for standard error only, ‘both’ for both standard output and error in separate files, ‘combined’ for standard output and error in a single file, and ‘none’ to disable all recording. ‘all’ is also a valid, but deprecated, option that is a synonym of ‘both’. If an explicit value is not given to this parameter, that is, if None is given, it defaults to using the module level configuration, as set by OUTPUT_CHECK_RECORD_MODE. If the module level configuration itself is not set, it defaults to ‘none’. • shell (bool) – Whether to run the command on a subshell • env (dict) – Use extra environment variables • sudo (bool) – Whether the command requires admin privileges to run, so that sudo will be prepended to the command. The assumption here is that the user running the command has a sudo configuration such that a password won’t be prompted. If that’s not the case, the command will straight out fail. • ignore_bg_processes (bool) – Whether to ignore background processes Returns Exit status and command output(stdout and stderr). Return type tuple avocado.utils.process.kill_process_by_pattern(pattern) Send SIGTERM signal to a process with matched pattern. Parameters pattern – normally only matched against the process name avocado.utils.process.kill_process_tree(pid, sig=9, send_sigcont=True) Signal a process and all of its children. If the process does not exist – return. Parameters • pid – The pid of the process to signal. • sig – The signal to send to the processes. avocado.utils.process.pid_exists(pid) Return True if a given PID exists. Parameters pid – Process ID number. avocado.utils.process.process_in_ptree_is_defunct(ppid) Verify if any processes deriving from PPID are in the defunct state. Attempt to verify if parent process and any children from PPID is defunct (zombie) or not. Parameters ppid – The parent PID of the process to verify. avocado.utils.process.run(cmd, timeout=None, verbose=True, ignore_status=False, al- low_output_check=None, shell=False, env=None, sudo=False, ignore_bg_processes=False, encoding=None) Run a subprocess, returning a CmdResult object. Parameters • cmd (str) – Command line to run.

20.2. Utilities APIs 189 avocado Documentation, Release 63.0

• timeout (float) – Time limit in seconds before attempting to kill the running process. This function will take a few seconds longer than ‘timeout’ to complete if it has to kill the process. • verbose (bool) – Whether to log the command run and stdout/stderr. • ignore_status (bool) – Whether to raise an exception when command returns =! 0 (False), or not (True). • allow_output_check (str) – Whether to record the output from this process (from stdout and stderr) in the test’s output record files. Valid values: ‘stdout’, for standard output only, ‘stderr’ for standard error only, ‘both’ for both standard output and error in separate files, ‘combined’ for standard output and error in a single file, and ‘none’ to disable all recording. ‘all’ is also a valid, but deprecated, option that is a synonym of ‘both’. If an explicit value is not given to this parameter, that is, if None is given, it defaults to using the module level configuration, as set by OUTPUT_CHECK_RECORD_MODE. If the module level configuration itself is not set, it defaults to ‘none’. • shell (bool) – Whether to run the command on a subshell • env (dict) – Use extra environment variables • sudo – Whether the command requires admin privileges to run, so that sudo will be prepended to the command. The assumption here is that the user running the command has a sudo configuration such that a password won’t be prompted. If that’s not the case, the command will straight out fail. • encoding (str) – the encoding to use for the text representation of the command result stdout and stderr, by default avocado.utils.astring.ENCODING Returns An CmdResult object. Raise CmdError, if ignore_status=False. avocado.utils.process.safe_kill(pid, signal) Attempt to send a signal to a given process that may or may not exist. Parameters signal – Signal number. avocado.utils.process.should_run_inside_gdb(cmd) Whether the given command should be run inside the GNU debugger Parameters cmd – the command arguments, from where we extract the binary name avocado.utils.process.should_run_inside_wrapper(cmd) Whether the given command should be run inside the wrapper utility. Parameters cmd – the command arguments, from where we extract the binary name avocado.utils.process.split_gdb_expr(expr) Splits a GDB expr into (binary_name, breakpoint_location) Returns avocado.gdb.GDB.DEFAULT_BREAK as the default breakpoint if one is not given. Parameters expr (str) – an expression of the form [:] Returns a (binary_name, breakpoint_location) tuple Return type tuple avocado.utils.process.system(cmd, timeout=None, verbose=True, ignore_status=False, al- low_output_check=None, shell=False, env=None, sudo=False, ig- nore_bg_processes=False, encoding=None) Run a subprocess, returning its exit code.

190 Chapter 20. API Reference avocado Documentation, Release 63.0

Parameters • cmd (str) – Command line to run. • timeout (float) – Time limit in seconds before attempting to kill the running process. This function will take a few seconds longer than ‘timeout’ to complete if it has to kill the process. • verbose (bool) – Whether to log the command run and stdout/stderr. • ignore_status (bool) – Whether to raise an exception when command returns =! 0 (False), or not (True). • allow_output_check (str) – Whether to record the output from this process (from stdout and stderr) in the test’s output record files. Valid values: ‘stdout’, for standard output only, ‘stderr’ for standard error only, ‘both’ for both standard output and error in separate files, ‘combined’ for standard output and error in a single file, and ‘none’ to disable all recording. ‘all’ is also a valid, but deprecated, option that is a synonym of ‘both’. If an explicit value is not given to this parameter, that is, if None is given, it defaults to using the module level configuration, as set by OUTPUT_CHECK_RECORD_MODE. If the module level configuration itself is not set, it defaults to ‘none’. • shell (bool) – Whether to run the command on a subshell • env (dict) – Use extra environment variables. • sudo – Whether the command requires admin privileges to run, so that sudo will be prepended to the command. The assumption here is that the user running the command has a sudo configuration such that a password won’t be prompted. If that’s not the case, the command will straight out fail. • encoding (str) – the encoding to use for the text representation of the command result stdout and stderr, by default avocado.utils.astring.ENCODING Returns Exit code. Return type int Raise CmdError, if ignore_status=False. avocado.utils.process.system_output(cmd, timeout=None, verbose=True, ig- nore_status=False, allow_output_check=None, shell=False, env=None, sudo=False, ig- nore_bg_processes=False, strip_trail_nl=True, encoding=None) Run a subprocess, returning its output. Parameters • cmd (str) – Command line to run. • timeout (float) – Time limit in seconds before attempting to kill the running process. This function will take a few seconds longer than ‘timeout’ to complete if it has to kill the process. • verbose (bool) – Whether to log the command run and stdout/stderr. • ignore_status – Whether to raise an exception when command returns =! 0 (False), or not (True). • allow_output_check (str) – Whether to record the output from this process (from stdout and stderr) in the test’s output record files. Valid values: ‘stdout’, for standard output only, ‘stderr’ for standard error only, ‘both’ for both standard output and error in separate files, ‘combined’ for standard output and error in a single file, and ‘none’ to disable all

20.2. Utilities APIs 191 avocado Documentation, Release 63.0

recording. ‘all’ is also a valid, but deprecated, option that is a synonym of ‘both’. If an explicit value is not given to this parameter, that is, if None is given, it defaults to using the module level configuration, as set by OUTPUT_CHECK_RECORD_MODE. If the module level configuration itself is not set, it defaults to ‘none’. • shell (bool) – Whether to run the command on a subshell • env (dict) – Use extra environment variables • sudo (bool) – Whether the command requires admin privileges to run, so that sudo will be prepended to the command. The assumption here is that the user running the command has a sudo configuration such that a password won’t be prompted. If that’s not the case, the command will straight out fail. • ignore_bg_processes (bool) – Whether to ignore background processes • strip_trail_nl (bool) – Whether to strip the trailing newline • encoding (str) – the encoding to use for the text representation of the command result stdout and stderr, by default avocado.utils.astring.ENCODING Returns Command output. Return type bytes Raise CmdError, if ignore_status=False.

20.2.32 avocado.utils.runtime module

Module that contains runtime conﬁguration avocado.utils.runtime.CURRENT_JOB = None Sometimes it’s useful for the framework and API to know about the job that is currently running, if one exists avocado.utils.runtime.CURRENT_TEST = None Sometimes it’s useful for the framework and API to know about the test that is currently running, if one exists

20.2.33 avocado.utils.script module

Module to handle scripts creation. avocado.utils.script.DEFAULT_MODE = 509 What is commonly known as “0775” or “u=rwx,g=rwx,o=rx” avocado.utils.script.READ_ONLY_MODE = 292 What is commonly known as “0444” or “u=r,g=r,o=r” class avocado.utils.script.Script(path, content, mode=509, open_mode=’w’) Bases: object Class that represents a script. Creates an instance of Script. Note that when the instance inside a with statement, it will automatically call save() and then remove() for you. Parameters • path – the script ﬁle name. • content – the script content. • mode – set ﬁle mode, defaults what is commonly known as 0775.

192 Chapter 20. API Reference avocado Documentation, Release 63.0

remove() Remove script from the file system. Returns True if script has been removed, otherwise False. save() Store script to file system. Returns True if script has been stored, otherwise False. class avocado.utils.script.TemporaryScript(name, content, prefix=’avocado_script’, mode=509, open_mode=’w’) Bases: avocado.utils.script.Script Class that represents a temporary script. Creates an instance of TemporaryScript. Note that when the instance inside a with statement, it will automatically call save() and then remove() for you. When the instance object is garbage collected, it will automatically call remove() for you. Parameters • name – the script file name. • content – the script content. • prefix – prefix for the temporary directory name. • mode – set file mode, default to 0775. remove() Remove script from the file system. Returns True if script has been removed, otherwise False. avocado.utils.script.make_script(path, content, mode=509) Creates a new script stored in the file system. Parameters • path – the script file name. • content – the script content. • mode – set file mode, default to 0775. Returns the script path. avocado.utils.script.make_temp_script(name, content, prefix=’avocado_script’, mode=509) Creates a new temporary script stored in the file system. Parameters • path – the script file name. • content – the script content. • prefix – the directory prefix Default to ‘avocado_script’. • mode – set file mode, default to 0775. Returns the script path.

20.2. Utilities APIs 193 avocado Documentation, Release 63.0

20.2.34 avocado.utils.service module

avocado.utils.service.ServiceManager(run=) Detect which init program is being used, init or systemd and return a class has methods to start/stop services. # Get the system service manager >> service_manager = ServiceManager() # Stating service/unit “sshd” >> service_manager.start(“sshd”) # Getting a list of available units >> units = service_manager.list() # Disabling and stopping a list of services >> services_to_disable = [‘ntpd’, ‘httpd’] >> for s in services_to_disable: >> service_manager.disable(s) >> service_manager.stop(s) Returns SysVInitServiceManager or SystemdServiceManager Return type _GenericServiceManager avocado.utils.service.SpecificServiceManager(service_name, run=) # Get the specific service manager for sshd >>> sshd = SpecificServiceManager(“sshd”) >>> sshd.start() >>> sshd.stop() >>> sshd.reload() >>> sshd.restart() >>> sshd.condrestart() >>> sshd.status() >>> sshd.enable() >>> sshd.disable() >>> sshd.is_enabled() Parameters service_name (str) – systemd unit or init.d service to manager Returns SpecificServiceManager that has start/stop methods Return type _SpecificServiceManager avocado.utils.service.convert_systemd_target_to_runlevel(target) Convert systemd target to runlevel. Parameters target (str) – systemd target Returns sys_v runlevel Return type str Raises ValueError – when systemd target is unknown avocado.utils.service.convert_sysv_runlevel(level) Convert runlevel to systemd target. Parameters level (str or int) – sys_v runlevel Returns systemd target Return type str Raises ValueError – when runlevel is unknown avocado.utils.service.get_name_of_init(run=) Internal function to determine what executable is PID 1 It does that by checking /proc/1/exe. Fall back to checking /proc/1/cmdline (local execution). Returns executable name for PID 1, aka init Return type str avocado.utils.service.service_manager(run=) Detect which init program is being used, init or systemd and return a class has methods to start/stop services. # Get the system service manager >> service_manager = ServiceManager() # Stating service/unit “sshd” >> service_manager.start(“sshd”)

194 Chapter 20. API Reference avocado Documentation, Release 63.0

# Getting a list of available units >> units = service_manager.list() # Disabling and stopping a list of services >> services_to_disable = [‘ntpd’, ‘httpd’] >> for s in services_to_disable: >> service_manager.disable(s) >> service_manager.stop(s) Returns SysVInitServiceManager or SystemdServiceManager Return type _GenericServiceManager avocado.utils.service.specific_service_manager(service_name, run=) # Get the specific service manager for sshd >>> sshd = SpecificServiceManager(“sshd”) >>> sshd.start() >>> sshd.stop() >>> sshd.reload() >>> sshd.restart() >>> sshd.condrestart() >>> sshd.status() >>> sshd.enable() >>> sshd.disable() >>> sshd.is_enabled() Parameters service_name (str) – systemd unit or init.d service to manager Returns SpecificServiceManager that has start/stop methods Return type _SpecificServiceManager avocado.utils.service.sys_v_init_command_generator(command) Generate lists of command arguments for sys_v style inits. Parameters command (str) – start,stop,restart, etc. Returns list of commands to pass to process.run or similar function Return type builtin.list avocado.utils.service.sys_v_init_result_parser(command) Parse results from sys_v style commands. command status: return true if service is running. command is_enabled: return true if service is enabled. command list: return a dict from service name to status. command others: return true if operate success. Parameters command (str.) – command. Returns different from the command. avocado.utils.service.systemd_command_generator(command) Generate list of command line argument strings for systemctl. One argument per string for compatibility Popen WARNING: If systemctl detects that it is running on a tty it will use color, pipe to $PAGER, change column sizes and not truncate unit names. Use –no-pager to suppress pager output, or set PAGER=cat in the environment. You may need to take other steps to suppress color output. See https://bugzilla.redhat.com/show_bug.cgi?id=713567 Parameters command (str) – start,stop,restart, etc. Returns List of command and arguments to pass to process.run or similar functions Return type builtin.list avocado.utils.service.systemd_result_parser(command) Parse results from systemd style commands. command status: return true if service is running. command is_enabled: return true if service is enabled. command list: return a dict from service name to status. command others: return true if operate success. Parameters command (str.) – command. Returns different from the command.

20.2. Utilities APIs 195 avocado Documentation, Release 63.0

20.2.35 avocado.utils.software_manager module

Software package management library. This is an abstraction layer on top of the existing distributions high level package managers. It supports package operations useful for testing purposes, and multiple high level package managers (here called backends). If you want to make this lib to support your particular package manager/distro, please implement the given backend class. author Higor Vieira Alves author Lucas Meneghel Rodrigues author Ramon de Carvalho Valle copyright IBM 2008-2009 copyright Red Hat 2009-2014 class avocado.utils.software_manager.AptBackend Bases: avocado.utils.software_manager.DpkgBackend Implements the apt backend for software manager. Set of operations for the apt package manager, commonly found on Debian and Debian based distributions, such as Ubuntu Linux. Initializes the base command and the debian package repository. add_repo(repo) Add an apt repository. Parameters repo – Repository string. Example: ‘deb http://archive.ubuntu.com/ubuntu/ mav- erick universe’ build_dep(name) Installed build-dependencies of a given package [name]. Parameters name – parameter package to install build-dependencies for. Return True If packages are installed properly get_source(name, path) Download source for provided package. Returns the path with source placed. Parameters name – parameter wildcard package to get the source for Return path path of ready-to-build source install(name) Installs package [name]. Parameters name – Package name. provides(path) Return a list of packages that provide [path]. Parameters path – File path. remove(name) Remove package [name]. Parameters name – Package name. remove_repo(repo) Remove an apt repository.

196 Chapter 20. API Reference avocado Documentation, Release 63.0

Parameters repo – Repository string. Example: ‘deb http://archive.ubuntu.com/ubuntu/ mav- erick universe’ upgrade(name=None) Upgrade all packages of the system with eventual new versions. Optionally, upgrade individual packages. Parameters name (str) – optional parameter wildcard spec to upgrade class avocado.utils.software_manager.BaseBackend Bases: object This class implements all common methods among backends. install_what_provides(path) Installs package that provides [path]. Parameters path – Path to ﬁle. class avocado.utils.software_manager.DnfBackend Bases: avocado.utils.software_manager.YumBackend Implements the dnf backend for software manager. DNF is the successor to yum in recent Fedora. Initializes the base command and the DNF package repository. class avocado.utils.software_manager.DpkgBackend Bases: avocado.utils.software_manager.BaseBackend This class implements operations executed with the dpkg package manager. dpkg is a lower level package manager, used by higher level managers such as apt and aptitude. INSTALLED_OUTPUT = 'install ok installed' PACKAGE_TYPE = 'deb' check_installed(name) list_all() List all packages available in the system. list_files(package) List ﬁles installed by package [package]. Parameters package – Package name. Returns List of paths installed by package. class avocado.utils.software_manager.RpmBackend Bases: avocado.utils.software_manager.BaseBackend This class implements operations executed with the rpm package manager. rpm is a lower level package manager, used by higher level managers such as yum and zypper. PACKAGE_TYPE = 'rpm' SOFTWARE_COMPONENT_QRY = 'rpm %{NAME} %{VERSION} %{RELEASE} %{SIGMD5} %{ARCH}' check_installed(name, version=None, arch=None) Check if package [name] is installed. Parameters

20.2. Utilities APIs 197 avocado Documentation, Release 63.0

• name – Package name. • version – Package version. • arch – Package architecture. list_all(software_components=True) List all installed packages. Parameters software_components – log in a format suitable for the SoftwareComponent schema list_files(name) List files installed on the system by package [name]. Parameters name – Package name. prepare_source(spec_file, dest_path=None) Rpmbuild the spec path and return build dir Parameters spec_path – spec path to install Return path build directory rpm_install(file_path) Install the rpm file [file_path] provided. Parameters file_path – Rpm file path. Return True if file is installed properly class avocado.utils.software_manager.SoftwareManager Bases: object Package management abstraction layer. It supports a set of common package operations for testing purposes, and it uses the concept of a backend, a helper class that implements the set of operations of a given package management tool. Lazily instantiate the object class avocado.utils.software_manager.SystemInspector Bases: object System inspector class. This may grow up to include more complete reports of operating system and machine properties. Probe system, and save information for future reference. get_package_management() Determine the supported package management systems present on the system. If more than one package management system installed, try to find the best supported system. class avocado.utils.software_manager.YumBackend(cmd=’yum’) Bases: avocado.utils.software_manager.RpmBackend Implements the yum backend for software manager. Set of operations for the yum package manager, commonly found on Yellow Dog Linux and Red Hat based distributions, such as Fedora and Red Hat Enterprise Linux. Initializes the base command and the yum package repository. add_repo(url) Adds package repository located on [url].

198 Chapter 20. API Reference avocado Documentation, Release 63.0

Parameters url – Universal Resource Locator of the repository. build_dep(name) Install build-dependencies for package [name] Parameters name – name of the package Return True If build dependencies are installed properly get_source(name, dest_path) Downloads the source package and prepares it in the given dest_path to be ready to build. Parameters • name – name of the package • dest_path – destination_path Return ﬁnal_dir path of ready-to-build directory install(name) Installs package [name]. Handles local installs. provides(name) Returns a list of packages that provides a given capability. Parameters name – Capability name (eg, ‘foo’). remove(name) Removes package [name]. Parameters name – Package name (eg. ‘ipython’). remove_repo(url) Removes package repository located on [url]. Parameters url – Universal Resource Locator of the repository. upgrade(name=None) Upgrade all available packages. Optionally, upgrade individual packages. Parameters name (str) – optional parameter wildcard spec to upgrade class avocado.utils.software_manager.ZypperBackend Bases: avocado.utils.software_manager.RpmBackend Implements the zypper backend for software manager. Set of operations for the zypper package manager, found on SUSE Linux. Initializes the base command and the yum package repository. add_repo(url) Adds repository [url]. Parameters url – URL for the package repository. get_source(name, dest_path) Downloads the source package and prepares it in the given dest_path to be ready to build Parameters • name – name of the package • dest_path – destination_path

20.2. Utilities APIs 199 avocado Documentation, Release 63.0

Return ﬁnal_dir path of ready-to-build directory install(name) Installs package [name]. Handles local installs. Parameters name – Package Name. provides(name) Searches for what provides a given ﬁle. Parameters name – File path. remove(name) Removes package [name]. remove_repo(url) Removes repository [url]. Parameters url – URL for the package repository. upgrade(name=None) Upgrades all packages of the system. Optionally, upgrade individual packages. Parameters name (str) – Optional parameter wildcard spec to upgrade avocado.utils.software_manager.install_distro_packages(distro_pkg_map, interactive=False) Installs packages for the currently running distribution This utility function checks if the currently running distro is a key in the distro_pkg_map dictionary, and if there is a list of packages set as its value. If these conditions match, the packages will be installed using the software manager interface, thus the native packaging system if the currently running distro. Parameters distro_pkg_map (dict) – mapping of distro name, as returned by utils.get_os_vendor(), to a list of package names Returns True if any packages were actually installed, False otherwise avocado.utils.software_manager.main()

20.2.36 avocado.utils.stacktrace module

Traceback standard module plus some additional APIs. avocado.utils.stacktrace.analyze_unpickable_item(path_preﬁx, obj) Recursive method to obtain unpickable objects along with location Parameters • path_prefix – Path to this object • obj – The sub-object under introspection Returns [($path_to_the_object, $value), . . . ] avocado.utils.stacktrace.log_exc_info(exc_info, logger=”) Log exception info to logger_name. Parameters • exc_info – Exception info produced by sys.exc_info()

200 Chapter 20. API Reference avocado Documentation, Release 63.0

• logger – Name or logger instance (defaults to ‘’) avocado.utils.stacktrace.log_message(message, logger=”) Log message to logger. Parameters • message – Message • logger – Name or logger instance (defaults to ‘’) avocado.utils.stacktrace.prepare_exc_info(exc_info) Prepare traceback info. Parameters exc_info – Exception info produced by sys.exc_info() avocado.utils.stacktrace.str_unpickable_object(obj) Return human readable string identifying the unpickable objects Parameters obj – The object for analysis Raises ValueError – In case the object is pickable avocado.utils.stacktrace.tb_info(exc_info) Prepare traceback info. Parameters exc_info – Exception info produced by sys.exc_info()

20.2.37 avocado.utils.vmimage module

Provides VM images acquired from ofﬁcial repositories class avocado.utils.vmimage.CentOSImageProvider(version=’[0-9]+’, build=’[0-9]{4}’, arch=’x86_64’) Bases: avocado.utils.vmimage.ImageProviderBase CentOS Image Provider name = 'CentOS' class avocado.utils.vmimage.DebianImageProvider(version=’[0-9]+.[0-9]+.[0-9]+-.*’, build=None, arch=’x86_64’) Bases: avocado.utils.vmimage.ImageProviderBase Debian Image Provider name = 'Debian' class avocado.utils.vmimage.FedoraImageProvider(version=’[0-9]+’, build=’[0-9]+.[0- 9]+’, arch=’x86_64’) Bases: avocado.utils.vmimage.ImageProviderBase Fedora Image Provider get_image_url() Probes the higher image available for the current parameters. name = 'Fedora' class avocado.utils.vmimage.FedoraSecondaryImageProvider(version=’[0-9]+’, build=’[0-9]+.[0-9]+’, arch=’x86_64’) Bases: avocado.utils.vmimage.ImageProviderBase Fedora Secondary Image Provider

20.2. Utilities APIs 201 avocado Documentation, Release 63.0

get_image_url() Probes the higher image available for the current parameters. name = 'FedoraSecondary' avocado.utils.vmimage.IMAGE_PROVIDERS = set([, , , , , ]) List of available providers classes class avocado.utils.vmimage.Image(name, url, version, arch, checksum, algorithm, cache_dir) Bases: object base_image get() path class avocado.utils.vmimage.ImageProviderBase(version, build, arch) Bases: object Base class to deﬁne the common methods and attributes of an image. Intended to be sub-classed by the speciﬁc image providers. get_image_url() Probes the higher image available for the current parameters. get_version() Probes the higher version available for the current parameters. version exception avocado.utils.vmimage.ImageProviderError Bases: exceptions.Exception Generic error class for ImageProvider class avocado.utils.vmimage.JeosImageProvider(version=’[0-9]+’, build=None, arch=’x86_64’) Bases: avocado.utils.vmimage.ImageProviderBase JeOS Image Provider name = 'JeOS' class avocado.utils.vmimage.UbuntuImageProvider(version=’[0-9]+.[0-9]+’, build=None, arch=’x86_64’) Bases: avocado.utils.vmimage.ImageProviderBase Ubuntu Image Provider name = 'Ubuntu' class avocado.utils.vmimage.VMImageHtmlParser(pattern) Bases: HTMLParser.HTMLParser Custom HTML parser to extract the href items that match a given pattern handle_starttag(tag, attrs) avocado.utils.vmimage.get(name=None, version=None, build=None, arch=None, checksum=None, algorithm=None, cache_dir=None) Wrapper to get the best Image Provider, according to the parameters provided. Parameters • name – (optional) Name of the Image Provider, usually matches the distro name. • version – (optional) Version of the system image.

202 Chapter 20. API Reference avocado Documentation, Release 63.0

• build – (optional) Build number of the system image. • arch – (optional) Architecture of the system image. • checksum – (optional) Hash of the system image to match after download. • algorithm – (optional) Hash type, used when the checksum is provided. • cache_dir – (optional) Local system path where the images and the snapshots will be held. Returns Image Provider instance that can provide the image according to the parameters. avocado.utils.vmimage.list_providers() List the available Image Providers

20.2.38 avocado.utils.wait module

avocado.utils.wait.wait_for(func, timeout, ﬁrst=0.0, step=1.0, text=None, args=None, kwargs=None) Wait until func() evaluates to True. If func() evaluates to True before timeout expires, return the value of func(). Otherwise return None. Parameters • timeout – Timeout in seconds • first – Time to sleep before ﬁrst attempt • step – Time to sleep between attempts in seconds • text – Text to print while waiting, for debug purposes • args – Positional arguments to func • kwargs – Keyword arguments to func

20.2.39 Module contents

20.3 Internal (Core) APIs

Internal APIs that may be of interest to Avocado hackers.

20.3. Internal (Core) APIs 203 avocado Documentation, Release 63.0

20.3.1 Subpackages avocado.core.restclient package

Subpackages avocado.core.restclient.cli package

Subpackages avocado.core.restclient.cli.actions package

Submodules avocado.core.restclient.cli.actions.base module avocado.core.restclient.cli.actions.base.action(function) Simple function that marks functions as CLI actions Parameters function – the function that will receive the CLI action mark avocado.core.restclient.cli.actions.server module

Module that implements the actions for the CLI App when the job toplevel command is used avocado.core.restclient.cli.actions.server.list_brief(app) Shows the server API list avocado.core.restclient.cli.actions.server.status(app) Shows the server status

Module contents avocado.core.restclient.cli.args package

Submodules avocado.core.restclient.cli.args.base module

This module has base action arguments that are used on other top level commands These top level commands import these deﬁnitions for uniformity and consistency sake avocado.core.restclient.cli.args.server module

This module has actions for the server command

204 Chapter 20. API Reference avocado Documentation, Release 63.0

Module contents

Submodules avocado.core.restclient.cli.app module

This is the main entry point for the rest client cli application class avocado.core.restclient.cli.app.App Bases: object Base class for CLI application Initializes a new app instance. This class is intended both to be used by the stock client application and also to be reused by custom applications. If you want, say, to limit the amount of command line actions and its arguments, you can simply supply another argument parser class to this constructor. Of course another way to customize it is to inherit from this and modify its members at will. dispatch_action() Calls the actions that was speciﬁed via command line arguments. This involves loading the relevant module ﬁle. initialize_connection() Initialize the connection instance run() Main entry point for application avocado.core.restclient.cli.parser module

REST client application command line parsing class avocado.core.restclient.cli.parser.Parser(**kwargs) Bases: argparse.ArgumentParser The main CLI Argument Parser. Initializes a new parser add_arguments_on_all_modules(prefix=’avocado.core.restclient.cli.args’) Add arguments that are present on all Python modules at a given prefix Parameters prefix – a Python module namespace add_arguments_on_module(name, prefix) Add arguments that are present on a given Python module Parameters name – the name of the Python module, without the namespace

20.3. Internal (Core) APIs 205 avocado Documentation, Release 63.0

Module contents

Submodules avocado.core.restclient.connection module

This module provides connection classes the avocado server. A connection is a simple wrapper around a HTTP request instance. It is this basic object that allows methods to be called on the remote server. avocado.core.restclient.connection.get_default() Returns the global, default connection to avocado-server Returns an avocado.core.restclient.connection.Connection instance class avocado.core.restclient.connection.Connection(hostname=None, port=None, username=None, password=None) Bases: object Connection to the avocado server Initializes a connection to an avocado-server instance Parameters • hostname (str) – the hostname or IP address to connect to • port (int) – the port number where avocado-server is running • username (str) – the name of the user to be authenticated as • password (str) – the password to use for authentication check_min_version(data=None) Checks the minimum server version get_api_list() Gets the list of APIs the server makes available to the current user get_url(path=None) Returns a representation of the current connection as an HTTP URL ping() Tests connectivity to the currently set avocado-server This is intentionally a simple method that will only return True if a request is made, and a response is received from the server. request(path, method=, check_status=True, **data) Performs a request to the server This method is heavily used by upper level API methods, and more often than not, those upper level API methods should be used instead. Parameters • path (str) – the path on the server where the resource lives • method – the method you want to call on the remote server, defaults to a HTTP GET

206 Chapter 20. API Reference avocado Documentation, Release 63.0

• check_status – whether to check the HTTP status code that comes with the response. If set to True, it will depend on the method chosen. If set to False, no check will be performed. If an integer is given then that speciﬁc status will be checked for. • data – keyword arguments to be passed to the remote method Returns JSON data

avocado.core.restclient.response module

Module with base model functions to manipulate JSON data class avocado.core.restclient.response.BaseResponse(json_data) Bases: object Base class that provides commonly used features for response handling REQUIRED_DATA = [] exception avocado.core.restclient.response.InvalidJSONError Bases: exceptions.Exception Data given to a loader/decoder is not valid JSON exception avocado.core.restclient.response.InvalidResultResponseError Bases: exceptions.Exception Returned result response does not conform to expectation Even though the result may be a valid json, it may not have the required or expected information that would normally be sent by avocado-server. class avocado.core.restclient.response.ResultResponse(json_data) Bases: avocado.core.restclient.response.BaseResponse Provides a wrapper around an ideal result response This class should be instantiated with the JSON data received from an avocado-server, and will check if the required data members are present and thus the response is well formed. REQUIRED_DATA = ['count', 'next', 'previous', 'results']

Module contents

20.3.2 Submodules

20.3.3 avocado.core.app module

The core Avocado application. class avocado.core.app.AvocadoApp Bases: object Avocado application. run()

20.3. Internal (Core) APIs 207 avocado Documentation, Release 63.0

20.3.4 avocado.core.data_dir module

Library used to let avocado tests find important paths in the system. The general reasoning to find paths is: • When running in tree, don’t honor avocado.conf. Also, we get to run/display the example tests shipped in tree. • When avocado.conf is in /etc/avocado, or ~/.config/avocado, then honor the values there as much as possible. If they point to a location where we can’t write to, use the next best location available. • The next best location is the default system wide one. • The next best location is the default user specific one. avocado.core.data_dir.clean_tmp_files() Try to clean the tmp directory by removing it. This is a useful function for avocado entry points looking to clean after tests/jobs are done. If OSError is raised, silently ignore the error. avocado.core.data_dir.create_job_logs_dir(base_dir=None, unique_id=None) Create a log directory for a job, or a stand alone execution of a test. Parameters • base_dir – Base log directory, if None, use value from configuration. • unique_id – The unique identification. If None, create one. Return type str avocado.core.data_dir.get_base_dir() Get the most appropriate base dir. The base dir is the parent location for most of the avocado other important directories. Examples: • Log directory • Data directory • Tests directory avocado.core.data_dir.get_cache_dirs() Returns the list of cache dirs, according to configuration and convention avocado.core.data_dir.get_data_dir() Get the most appropriate data dir location. The data dir is the location where any data necessary to job and test operations are located. Examples: • ISO files • GPG files • VM images • Reference bitmaps avocado.core.data_dir.get_datafile_path(*args) Get a path relative to the data dir. Parameters args – Arguments passed to os.path.join. Ex (‘images’, ‘jeos.qcow2’)

208 Chapter 20. API Reference avocado Documentation, Release 63.0 avocado.core.data_dir.get_logs_dir() Get the most appropriate log dir location. The log dir is where we store job/test logs in general. avocado.core.data_dir.get_test_dir() Get the most appropriate test location. The test location is where we store tests written with the avocado API. The heuristics used to determine the test dir are: 1) If an explicit test dir is set in the conﬁguration system, it is used. 2) If user is running Avocado out of the source tree, the example test dir is used 3) System wide test dir is used 4) User default test dir (~/avocado/tests) is used avocado.core.data_dir.get_tmp_dir(basedir=None) Get the most appropriate tmp dir location. The tmp dir is where artifacts produced by the test are kept. Examples: • Copies of a test suite source code • Compiled test suite source code

20.3.5 avocado.core.decorators module avocado.core.decorators.fail_on(exceptions=None) Fail the test when decorated function produces exception of the speciﬁed type. (For example, our method may raise IndexError on tested software failure. We can either try/catch it or use this decorator instead) Parameters exceptions – Tuple or single exception to be assumed as test fail [Exception] Note self.error and self.cancel behavior remains intact Note To allow simple usage param “exceptions” must not be callable avocado.core.decorators.skip(message=None) Decorator to skip a test. avocado.core.decorators.skipIf(condition, message=None) Decorator to skip a test if a condition is True. avocado.core.decorators.skipUnless(condition, message=None) Decorator to skip a test if a condition is False.

20.3.6 avocado.core.defaults module

The Avocado core defaults avocado.core.defaults.ENCODING = 'utf-8' The encoding used by default on all data input

20.3.7 avocado.core.dispatcher module

Extensions/plugins dispatchers.

20.3. Internal (Core) APIs 209 avocado Documentation, Release 63.0 class avocado.core.dispatcher.CLICmdDispatcher Bases: avocado.core.dispatcher.Dispatcher Calls extensions on configure/run Automatically adds all the extension with entry points registered under ‘avocado.plugins.cli.cmd’ class avocado.core.dispatcher.CLIDispatcher Bases: avocado.core.dispatcher.Dispatcher Calls extensions on configure/run Automatically adds all the extension with entry points registered under ‘avocado.plugins.cli’ class avocado.core.dispatcher.Dispatcher(namespace, invoke_kwds=None) Bases: stevedore.enabled.EnabledExtensionManager Base dispatcher for various extension types NAMESPACE_PREFIX = 'avocado.plugins.' Default namespace prefix for Avocado extensions enabled(extension) Checks configuration for explicit mention of plugin in a disable list If configuration section or key doesn’t exist, it means no plugin is disabled. fully_qualified_name(extension) Returns the Avocado fully qualified plugin name Parameters extension (stevedore.extension.Extension) – an Stevedore Exten- sion instance map_method(method_name, *args) Maps method_name on each extension in case the extension has the attr Parameters • method_name – Name of the method to be called on each ext • args – Arguments to be passed to all called functions map_method_with_return(method_name, *args, **kwargs) The same as map_method but additionally reports the list of returned values and optionally deepcopies the passed arguments Parameters • method_name – Name of the method to be called on each ext • args – Arguments to be passed to all called functions • kwargs – Key-word arguments to be passed to all called functions if “deepcopy” == True is present in kwargs the args and kwargs are deepcopied before passing it to each called function. names() Returns the names of the discovered extensions This differs from stevedore.extension.ExtensionManager.names() in that it returns names in a predictable order, by using standard sorted(). plugin_type() Subset of entry points namespace for this dispatcher

210 Chapter 20. API Reference avocado Documentation, Release 63.0

Given an entry point avocado.plugins.foo, plugin type is foo. If entry point does not conform to the Avocado standard preﬁx, it’s returned unchanged. settings_section() Returns the conﬁg section name for the plugin type handled by itself static store_load_failure(manager, entrypoint, exception) class avocado.core.dispatcher.JobPrePostDispatcher Bases: avocado.core.dispatcher.Dispatcher Calls extensions before Job execution Automatically adds all the extension with entry points registered under ‘avocado.plugins.job.prepost’ class avocado.core.dispatcher.ResultDispatcher Bases: avocado.core.dispatcher.Dispatcher class avocado.core.dispatcher.ResultEventsDispatcher(args) Bases: avocado.core.dispatcher.Dispatcher class avocado.core.dispatcher.VarianterDispatcher Bases: avocado.core.dispatcher.Dispatcher map_method(method_name, *args, **kwargs) Maps method_name on each extension in case the extension has the attr Parameters • method_name – Name of the method to be called on each ext • args – Arguments to be passed to all called functions map_method_copy(method_name, *args, **kwargs) The same as map_method, but use copy.deepcopy on each passed arg

20.3.8 avocado.core.exceptions module

Exception classes, useful for tests, and other parts of the framework code. exception avocado.core.exceptions.JobBaseException Bases: exceptions.Exception The parent of all job exceptions. You should be never raising this, but just in case, we’ll set its status’ as FAIL. status = 'FAIL' exception avocado.core.exceptions.JobError Bases: avocado.core.exceptions.JobBaseException A generic error happened during a job execution. status = 'ERROR' exception avocado.core.exceptions.OptionValidationError Bases: exceptions.Exception An invalid option was passed to the test runner status = 'ERROR'

20.3. Internal (Core) APIs 211 avocado Documentation, Release 63.0

exception avocado.core.exceptions.TestAbortError Bases: avocado.core.exceptions.TestBaseException Indicates that the test was prematurely aborted. status = 'ERROR' exception avocado.core.exceptions.TestBaseException Bases: exceptions.Exception The parent of all test exceptions. You should be never raising this, but just in case, we’ll set its status’ as FAIL. status = 'FAIL' exception avocado.core.exceptions.TestCancel Bases: avocado.core.exceptions.TestBaseException Indicates that a test was canceled. Should be thrown when the cancel() test method is used. status = 'CANCEL' exception avocado.core.exceptions.TestError Bases: avocado.core.exceptions.TestBaseException Indicates that the test was not fully executed and an error happened. This is the sort of exception you raise if the test was partially executed and could not complete due to a setup, conﬁguration, or another fatal condition. status = 'ERROR' exception avocado.core.exceptions.TestFail Bases: avocado.core.exceptions.TestBaseException, exceptions.AssertionError Indicates that the test failed. TestFail inherits from AssertionError in order to keep compatibility with vanilla python unittests (they only consider failures the ones deriving from AssertionError). status = 'FAIL' exception avocado.core.exceptions.TestInterruptedError Bases: avocado.core.exceptions.TestBaseException Indicates that the test was interrupted by the user (Ctrl+C) status = 'INTERRUPTED' exception avocado.core.exceptions.TestNotFoundError Bases: avocado.core.exceptions.TestBaseException Indicates that the test was not found in the test directory. status = 'ERROR' exception avocado.core.exceptions.TestSetupFail Bases: avocado.core.exceptions.TestBaseException Indicates an error during a setup or cleanup procedure. status = 'ERROR'

212 Chapter 20. API Reference avocado Documentation, Release 63.0

exception avocado.core.exceptions.TestSkipError Bases: avocado.core.exceptions.TestBaseException Indicates that the test is skipped. Should be thrown when various conditions are such that the test is inappropriate. For example, inappropriate architecture, wrong OS version, program being tested does not have the expected capability (older version). status = 'SKIP' exception avocado.core.exceptions.TestTimeoutInterrupted Bases: avocado.core.exceptions.TestBaseException Indicates that the test did not ﬁnish before the timeout speciﬁed. status = 'INTERRUPTED' exception avocado.core.exceptions.TestWarn Bases: avocado.core.exceptions.TestBaseException Indicates that bad things (may) have happened, but not an explicit failure. status = 'WARN'

20.3.9 avocado.core.exit_codes module

Avocado exit codes. These codes are returned on the command line and may be used by applications that interface (that is, run) the Avocado command line application. Besides main status about the execution of the command line application, these exit status may also give extra, although limited, information about test statuses. avocado.core.exit_codes.AVOCADO_ALL_OK = 0 Both job and tests PASSed avocado.core.exit_codes.AVOCADO_FAIL = 4 Something else went wrong and avocado failed (or crashed). Commonly used on command line validation errors. avocado.core.exit_codes.AVOCADO_GENERIC_CRASH = -1 Avocado generic crash avocado.core.exit_codes.AVOCADO_JOB_FAIL = 2 Something went wrong with an Avocado Job execution, usually by an explicit avocado.core. exceptions.JobError exception. avocado.core.exit_codes.AVOCADO_JOB_INTERRUPTED = 8 The job was explicitly interrupted. Usually this means that a user hit CTRL+C while the job was still running. avocado.core.exit_codes.AVOCADO_TESTS_FAIL = 1 Job went ﬁne, but some tests FAILed or ERRORed

20.3.10 avocado.core.job module

Job module - describes a sequence of automated test operations. class avocado.core.job.Job(args=None) Bases: object A Job is a set of operations performed on a test machine.

20.3. Internal (Core) APIs 213 avocado Documentation, Release 63.0

Most of the time, we are interested in simply running tests, along with setup operations and event recording. Creates an instance of Job class. Parameters args – an instance of argparse.Namespace. cleanup() Cleanup the temporary job handlers (dirs, global setting, . . . ) create_test_suite() Creates the test suite for this Job This is a public Job API as part of the documented Job phases logdir = None The log directory for this job, also known as the job results directory. If it’s set to None, it means that the job results directory has not yet been created. post_tests() Run the post tests execution hooks By default this runs the plugins that implement the avocado.core.plugin_interfaces. JobPostTests interface. pre_tests() Run the pre tests execution hooks By default this runs the plugins that implement the avocado.core.plugin_interfaces. JobPreTests interface. run() Runs all job phases, returning the test execution results. This method is supposed to be the simplified interface for jobs, that is, they run all phases of a job. Returns Integer with overall job status. See avocado.core.exit_codes for more information. run_tests() The actual test execution phase setup() Setup the temporary job handlers (dirs, global setting, . . . ) test_suite = None The list of discovered/resolved tests that will be attempted to be run by this job. If set to None, it means that test resolution has not been attempted. If set to an empty list, it means that no test was found during resolution. time_elapsed = None The total amount of time the job took from start to finish, or -1 if it has not been started by means of the run() method time_end = None The time at which the job has finished or -1 if it has not been started by means of the run() method. time_start = None The time at which the job has started or -1 if it has not been started by means of the run() method. class avocado.core.job.TestProgram Bases: object Convenience class to make avocado test modules executable. parse_args(argv)

214 Chapter 20. API Reference avocado Documentation, Release 63.0

run_tests() avocado.core.job.main alias of avocado.core.job.TestProgram

20.3.11 avocado.core.job_id module avocado.core.job_id.create_unique_job_id() Create a 40 digit hex number to be used as a job ID string. (similar to SHA1) Returns 40 digit hex number string Return type str

20.3.12 avocado.core.jobdata module

Record/retrieve job information avocado.core.jobdata.get_id(path, jobid) Gets the full Job ID using the results directory path and a partial Job ID or the string ‘latest’. avocado.core.jobdata.get_resultsdir(logdir, jobid) Gets the job results directory using a Job ID. avocado.core.jobdata.record(args, logdir, variants, references=None, cmdline=None) Records all required job information. avocado.core.jobdata.retrieve_args(resultsdir) Retrieves the job args from the results directory. avocado.core.jobdata.retrieve_cmdline(resultsdir) Retrieves the job command line from the results directory. avocado.core.jobdata.retrieve_config(resultsdir) Retrieves the job settings from the results directory. avocado.core.jobdata.retrieve_pwd(resultsdir) Retrieves the job pwd from the results directory. avocado.core.jobdata.retrieve_references(resultsdir) Retrieves the job test references from the results directory. avocado.core.jobdata.retrieve_variants(resultsdir) Retrieves the job variants object from the results directory.

20.3.13 avocado.core.loader module

Test loader module. avocado.core.loader.ALL = > Compatibility alias (to be removed) to DiscoverMode.ALL avocado.core.loader.AVAILABLE = > Compatibility alias (to be removed) to DiscoverMode.AVAILABLE class avocado.core.loader.AccessDeniedPath Bases: object Dummy object to represent reference pointing to a inaccessible path

20.3. Internal (Core) APIs 215 avocado Documentation, Release 63.0

class avocado.core.loader.BrokenSymlink Bases: object Dummy object to represent reference pointing to a BrokenSymlink path avocado.core.loader.DEFAULT = > Compatibility alias (to be removed) to DiscoverMode.DEFAULT class avocado.core.loader.DiscoverMode Bases: enum.Enum ALL =