TESTING IN QEMU AND LIBVIRT Beyond make and make check

YASH MANKAD INC.

26th October 2018 AGENDA

What Why How Where

1 KVM Forum 2018 AGENDA

What is the state of testing in QEMU and libvirt ? Why How Where

2 KVM Forum 2018 AGENDA

What is the state of testing in QEMU and libvirt ? Why should we change ? How Where

3 KVM Forum 2018 AGENDA

What is the state of testing in QEMU and libvirt ? Why should we change ? How will it work ? Where

4 KVM Forum 2018 AGENDA

What is the state of testing in QEMU and libvirt ? Why should we change ? How will it work ? Where will it run ?

5 KVM Forum 2018 STATE OF VIRT TESTING

QEMU libvirt Tests included in source test suite present in source are unit tests code runs unit tests CI environments build QEMU CI environment uses the from source using 'make' and autogen.sh script to build and run 'make check' runs gmake Sub-system tests such as 'check' ensures XML can run iotests, coverity, and dynamic qemu code analysis

6 KVM Forum 2018 STATE OF QEMU TESTING

Runs for every patch Ubuntu 12/14.04 x86_64 Runs Debian, Ubuntu sent to qemu-devel hosts, MAC OS X amd-64 ubuntu, fedora, centos Runs make Runs make on No functional tests No functional tests checkpatch.pl Runs make check

7 KVM Forum 2018 STATE OF LIBVIRT TESTING

on

Runs Ubuntu in docker Runs CentOS, Debian, Fedora, rawhide and make syntax-check make install make and make install make dist

8 KVM Forum 2018 WHY SHOULD WE CHANGE ?

9 KVM Forum 2018 WHY SHOULD WE CHANGE ?

Why look beyond make, make check, make install, etc ?

Run more functional tests Run on baremetal nodes Run on different architectures

10 KVM Forum 2018 WHY SHOULD WE CHANGE ?

======Add repo ======$ sudo curl Lack of integrated test frameworks for https://avocado­project.org/data/repos/avocado­fedora.re ­o /etc/yum.repos.d/avocado.repo Virtualization ======Install Avocado ======Avocado and Avocado-VT $ dnf install python­avocado $ dnf install avocado­plugins­vt Pros ======Bootstrap Avocado ======Can run in bare metal or virtual environment $ avocado vt­bootstrap ­­vt­type =[qemu|libvirt|..] Has extensive functional and regression tests ======Pick the tests ======$ avocado list ­­vt­type libvirt | wc ­l Cons 12097 $ avocado list ­­vt­type qemu | wc ­l Not packaged for all distros 3419

Lot of dependencies ======Run the tests ======$ avocado run /path/to/test

11 KVM Forum 2018 HOW WILL IT WORK ?

12 KVM Forum 2018 HOW WILL IT WORK ?

13 KVM Forum 2018 HOW WILL IT WORK ?

QEMU $ cd qemu Functional tests added to developer workflow $ tree tests/acceptance/ Have access to a predefined “VM” tests/acceptance/ ├── avocado_qemu self.vm │ └── __init__.py The VM is a QEMUMachine instance (from ├── boot_linux_console.py scripts/qemu.py) ├── README.rst ├── version.py Add command line arguments with add_args() └── vnc.py Launch the VM with launch() Send QMP commands with command() $ avocado run tests/acceptance ­ OR ­ Called using make check-acceptance $ make check­acceptance Cleber Rosa's talk: https://sched.co/Fzud

14 KVM Forum 2018 HOW WILL IT WORK ?

libvirt ======Install the TCK tests ======libvirt-tck test suite from -Sys-Virt-TCK $ dnf install perl­Sys­Virt­TCK Functional, integration tests for libvirt drivers ­­enablerepo=updates­testing

Wasn't actively maintained....we will be ======Run the tests ======changing that $ libvirt­tck ­v Supports and runs a wide range of tests domain storage nwfilter, etc. Packaged for Fedora

15 KVM Forum 2018 WHERE WILL IT RUN ?

16 KVM Forum 2018 WHERE WILL IT RUN ?

I strongly encourage the use of bare metal ci.centos.org environments of Virt testing

CentOS CI infrastructure They have 38 x86_64 bare metal nodes available for CI Also have 64 aarch64 and ppc64le VMs Can be requested using duffy CentOS is working on adding aarch64 hardware to this infrastructure.

17 KVM Forum 2018 WHERE WILL IT RUN ?

Get onboard with Fedora's CI initiative

Tie in functional tests with existing environments Both avocado_qemu and libvirt-tck can run in virtual environments Create a meaningful feedback loop for developers and feature owners

18 KVM Forum 2018 SUMMARY

QEMU and libvirt CI runs a lot of 'make' Only unit and basic acceptance tests

Let's start running functional, integration tests for QEMU and libvirt Will provide more coverage, run on additional architectures

Have a functional CI for Virt packages

19 KVM Forum 2018 THANK YOU QUESTIONS ?

More questions ? Yash Mankad