TESTING IN QEMU AND LIBVIRT Beyond make and make check YASH MANKAD RED HAT INC. 26th October 2018 AGENDA What Why How Where 1 KVM Forum 2018 AGENDA What is the state of testing in QEMU and libvirt ? Why How Where 2 KVM Forum 2018 AGENDA What is the state of testing in QEMU and libvirt ? Why should we change ? How Where 3 KVM Forum 2018 AGENDA What is the state of testing in QEMU and libvirt ? Why should we change ? How will it work ? Where 4 KVM Forum 2018 AGENDA What is the state of testing in QEMU and libvirt ? Why should we change ? How will it work ? Where will it run ? 5 KVM Forum 2018 STATE OF VIRT TESTING QEMU libvirt Tests included in qemu source test suite present in source are unit tests code runs unit tests CI environments build QEMU CI environment uses the from source using 'make' and autogen.sh script to build and run 'make check' runs gmake Sub-system tests such as 'check' ensures XML can run iotests, coverity, and dynamic qemu code analysis 6 KVM Forum 2018 STATE OF QEMU TESTING Runs for every patch Ubuntu 12/14.04 x86_64 Runs Debian, Ubuntu sent to qemu-devel hosts, MAC OS X amd-64 ubuntu, fedora, centos Runs make Runs make on docker No functional tests No functional tests checkpatch.pl Runs make check 7 KVM Forum 2018 STATE OF LIBVIRT TESTING on Runs Ubuntu in docker Runs CentOS, Debian, Fedora, rawhide and make syntax-check freebsd make install make and make install make dist 8 KVM Forum 2018 WHY SHOULD WE CHANGE ? 9 KVM Forum 2018 WHY SHOULD WE CHANGE ? Why look beyond make, make check, make install, etc ? Run more functional tests Run on baremetal nodes Run on different architectures 10 KVM Forum 2018 WHY SHOULD WE CHANGE ? ========== Add repo ============= $ sudo curl Lack of integrated test frameworks for https://avocado­project.org/data/repos/avocado­fedora.re ­o /etc/yum.repos.d/avocado.repo Virtualization ======= Install Avocado ======== Avocado and Avocado-VT $ dnf install python­avocado $ dnf install avocado­plugins­vt Pros ======= Bootstrap Avocado ====== Can run in bare metal or virtual environment $ avocado vt­bootstrap ­­vt­type =[qemu|libvirt|..] Has extensive functional and regression tests ======== Pick the tests ========= $ avocado list ­­vt­type libvirt | wc ­l Cons 12097 $ avocado list ­­vt­type qemu | wc ­l Not packaged for all distros 3419 Lot of dependencies ======== Run the tests ========= $ avocado run /path/to/test 11 KVM Forum 2018 HOW WILL IT WORK ? 12 KVM Forum 2018 HOW WILL IT WORK ? 13 KVM Forum 2018 HOW WILL IT WORK ? QEMU $ cd qemu Functional tests added to developer workflow $ tree tests/acceptance/ Have access to a predefined “VM” tests/acceptance/ ├── avocado_qemu self.vm │ └── __init__.py The VM is a QEMUMachine instance (from ├── boot_linux_console.py scripts/qemu.py) ├── README.rst ├── version.py Add command line arguments with add_args() └── vnc.py Launch the VM with launch() Send QMP commands with command() $ avocado run tests/acceptance ­ OR ­ Called using make check-acceptance $ make check­acceptance Cleber Rosa's talk: https://sched.co/Fzud 14 KVM Forum 2018 HOW WILL IT WORK ? libvirt ====== Install the TCK tests ========= libvirt-tck test suite from Perl-Sys-Virt-TCK $ dnf install perl­Sys­Virt­TCK Functional, integration tests for libvirt drivers ­­enablerepo=updates­testing Wasn't actively maintained....we will be ====== Run the tests ========== changing that $ libvirt­tck ­v Supports and runs a wide range of tests domain storage nwfilter, etc. Packaged for Fedora 15 KVM Forum 2018 WHERE WILL IT RUN ? 16 KVM Forum 2018 WHERE WILL IT RUN ? I strongly encourage the use of bare metal ci.centos.org environments of Virt testing CentOS CI infrastructure They have 38 x86_64 bare metal nodes available for CI Also have 64 aarch64 and ppc64le VMs Can be requested using duffy CentOS is working on adding aarch64 hardware to this infrastructure. 17 KVM Forum 2018 WHERE WILL IT RUN ? Get onboard with Fedora's CI initiative Tie in functional tests with existing environments Both avocado_qemu and libvirt-tck can run in virtual environments Create a meaningful feedback loop for developers and feature owners 18 KVM Forum 2018 SUMMARY QEMU and libvirt CI runs a lot of 'make' Only unit and basic acceptance tests Let's start running functional, integration tests for QEMU and libvirt Will provide more coverage, run on additional architectures Have a functional CI for Virt packages 19 KVM Forum 2018 THANK YOU QUESTIONS ? More questions ? Yash Mankad <[email protected]> .