avocado Documentation Release 63.0 Avocado Development Team Jul 19, 2018 Contents 1 About Avocado 3 2 Getting Started 5 2.1 Installing Avocado............................................5 2.2 Using Avocado..............................................8 2.3 Writing a Simple Test.......................................... 10 2.4 Running A More Complex Test Job................................... 10 2.5 Interrupting The Job On First Failed Test (failfast)........................... 11 2.6 Ignoring Missing Test References.................................... 11 2.7 Running Tests With An External Runner................................ 12 2.8 Debugging tests............................................. 13 3 Writing Avocado Tests 15 3.1 Basic example.............................................. 15 3.2 Test statuses............................................... 16 3.3 Saving test generated (custom) data................................... 17 3.4 Accessing test data files......................................... 18 3.5 Accessing test parameters........................................ 18 3.6 Running multiple variants of tests.................................... 19 3.7 Advanced logging capabilities...................................... 21 3.8 unittest.TestCase heritage.................................... 23 3.9 Setup and cleanup methods....................................... 24 3.10 Running third party test suites...................................... 24 3.11 Fetching asset files............................................ 25 3.12 Test Output Check and Output Record Mode.............................. 27 3.13 Test log, stdout and stderr in native Avocado modules......................... 30 3.14 Setting a Test Timeout.......................................... 31 3.15 Skipping Tests.............................................. 32 3.16 Cancelling Tests............................................. 33 3.17 Docstring Directives........................................... 34 3.18 Python unittest Compatibility Limitations And Caveats...................... 40 3.19 Environment Variables for Tests..................................... 41 3.20 SIMPLE Tests BASH extensions.................................... 42 3.21 SIMPLE Tests Status........................................... 43 3.22 Wrap Up................................................. 43 4 Result Formats 45 i 4.1 Results for human beings........................................ 45 4.2 Machine readable results......................................... 46 4.3 Multiple results at once......................................... 49 4.4 Exit Codes................................................ 49 4.5 Implementing other result formats.................................... 49 5 Configuration 51 5.1 Config file parsing order......................................... 51 5.2 Plugin config files............................................ 52 5.3 Parsing order recap............................................ 52 5.4 Order of precedence for values used in tests............................... 52 5.5 Config plugin............................................... 53 5.6 Avocado Data Directories........................................ 53 6 Test discovery 55 6.1 The order of test loaders......................................... 55 6.2 Running simple tests with arguments.................................. 56 6.3 Filtering tests by tags........................................... 56 6.4 Test References.............................................. 56 7 Logging system 59 7.1 Tweaking the UI............................................. 59 7.2 Storing custom logs........................................... 59 7.3 Paginator................................................. 60 8 Sysinfo collection 61 9 Test parameters 63 9.1 TreeNode................................................. 64 9.2 AvocadoParams............................................. 64 9.3 Parameter Paths............................................. 65 9.4 Variant.................................................. 65 9.5 Dumping/Loading Variants....................................... 65 9.6 Varianter................................................. 66 9.7 Default params.............................................. 67 9.8 Varianter plugins............................................. 67 9.9 Multiplexer................................................ 67 9.10 Multiplex domains............................................ 68 9.11 MuxPlugin................................................ 70 9.12 MuxTree................................................. 70 10 Job Replay 73 11 Job Diff 77 12 Running Tests Remotely 79 13 Subclassing Avocado 81 14 Debugging with GDB 85 14.1 Transparent Execution of Executables.................................. 85 14.2 avocado.utils.gdb APIs..................................... 87 15 Wrap executables run by tests 89 15.1 Usage................................................... 89 15.2 Caveats.................................................. 89 ii 16 Plugin System 91 16.1 Listing plugins.............................................. 91 16.2 Writing a plugin............................................. 92 17 Utilities 95 17.1 Utilities.................................................. 95 18 Optional Plugins 99 18.1 Optional Plugins............................................. 99 19 Advanced Topics and Maintenance 119 19.1 Reference Guide............................................. 119 19.2 Contribution and Community Guide................................... 130 19.3 Avocado development tips........................................ 135 19.4 Releasing avocado............................................ 137 19.5 Other Resources............................................. 141 20 API Reference 143 20.1 Test APIs................................................. 143 20.2 Utilities APIs............................................... 146 20.3 Internal (Core) APIs........................................... 203 20.4 Extension (plugin) APIs......................................... 243 20.5 Optional Plugins API........................................... 257 21 Avocado Release Notes 267 21.1 Release Notes.............................................. 267 22 Request For Comments (RFCs) 327 22.1 Request For Comments (RFCs)..................................... 327 22.2 Indices and tables............................................ 333 Python Module Index 335 iii iv avocado Documentation, Release 63.0 Contents: Contents 1 avocado Documentation, Release 63.0 2 Contents CHAPTER 1 About Avocado Avocado is a set of tools and libraries to help with automated testing. One can call it a test framework with benefits. Native tests are written in Python and they follow the unittest pattern, but any executable can serve as a test. Avocado is composed of: • A test runner that lets you execute tests. Those tests can be either written in your language of choice, or be written in Python and use the available libraries. In both cases, you get facilities such as automated log and system information collection. • Libraries that help you write tests in a concise, yet expressive and powerful way. You can find more information about what libraries are intended for test writers at Libraries and APIs. • Plugins that can extend and add new functionality to the Avocado Framework. Avocado is built on the experience accumulated with Autotest, while improving on its weaknesses and shortcomings. Avocado tries as much as possible to comply with standard Python testing technology. Tests written using the Avocado API are derived from the unittest class, while other methods suited to functional and performance testing were added. The test runner is designed to help people to run their tests while providing an assortment of system and logging facilities, with no effort, and if you want more features, then you can start using the API features progressively. 3 avocado Documentation, Release 63.0 4 Chapter 1. About Avocado CHAPTER 2 Getting Started Those who prefer video-introduction, take a look at Other Resources. Either way first step towards using Avocado is, quite obviously, installing it. 2.1 Installing Avocado Avocado is primarily written in Python, so a standard Python installation is possible and often preferable. Tip: If you are looking for Virtualization specific testing, also consider looking at Avocado-VT installation instruc- tions after finishing the Avocado installation. 2.1.1 Installing with standard Python tools The simplest installation method is through pip. On most POSIX systems with Python 2.7 and pip available, installation can be performed with a single command: pip install--user avocado-framework This will fetch the Avocado package (and possibly some of its dependecies) from the PyPI repository, and will attempt to install it in the user’s home directory (usually under ~/.local). Tip: If you want to perform a system-wide installation, drop the --user switch. If you want even more isolation, Avocado can also be installed in a Python virtual environment. with no additional steps besides creating and activating the “venv” itself: python-m virtualenv/path/to/new/virtual_environment ./path/to/new/virtual_environment/bin/activate pip install avocado-framework 5 avocado Documentation, Release 63.0 Please note that this installs the Avocado core functionality. Many Avocado features are distributed as non- core plugins, also available as additional packages on PyPI. You should be able to find them via pip search avocado-framework-plugin | grep avocado-framework-plugin. Some of them are listed