Testing oVirt Node in 4min (video)

My focus over the last year or so lay on bringing test automation to oVirt Node.
It was challenging because oVirt Node is based on a LiveCD - at boot and post-installtion. (The whole LiveCD is used as a r/o rootfs.)

To allow an automated testing of oVirt Node, I've been working on igor. It allows us to test oVirt Node on real hardware an in VMs.

When you throw all the new features (see below - libvirt-only, new igorc, new igor events service, junit-reports for jobs) together you can do a complete testsuite run on an oVirt Node ISO with one command.

And this is how it looks (view it in fullscreen to see all the nifty details, but this will leave you without the subtitles explaining what's happening):

0 00:00:00,000 --> 00:00:00,100 1 00:00:00,100 --> 00:00:03,000 Launching igorc 2 00:00:03,000 --> 00:00:12,000 igorc (left) extracts LiveCD, creates a profile and submits a new job 3 00:00:27,000 --> 00:00:32,000 igord created a VM (right) and boots it up (from a CD derived from the igord profile) 4 00:00:40,000 --> 00:01:00,000 VM (right) boots and the autoinstall is performed 5 00:01:36,000 --> 00:01:45,000 Installation finished (Ctrl-Alt-Del is sent to the VM to reboot [that's a known bug]) 6 00:01:58,000 --> 00:02:05,000 The VM now boots from HD 7 00:02:20,000 --> 00:02:27,000 An igor-service is now started in the background (within the VM) to communicate with igord 8 00:02:34,000 --> 00:02:39,000 The igor-service tells igord about the ocmpletion of the first testcase, which is then picked up by igorc (left). 9 00:02:39,000 --> 00:02:45,000 A couple of more testcases were completed (left) and a reboot is initiated (by the igor-service within the VM) 10 00:02:48,000 --> 00:03:10,000 The VM (right) reboots 11 00:03:46,000 --> 00:03:59,000 All testcases passed and the VM is torn down by igord

This is a big step forward - even if there are still some issues outstanding to achieve the goal to make testcase development fun.

Now that we've seen the fancy part some background and open issues.
One pitfall - up to this week - was the hurdle to get igor up an running. Igor used to require Cobbler - and cobbler is not easy to setup on Fedora 18 (which I use to build an test oVirt Node - which itself is based on Fedora 18 packages).

Anyhow - long story short - Igor has a "feature complete" "backend" for libvirt now, that means, igor doesn't need cobbler anymore. Furthermore I've added a brand new igor client (called igorc) which communicates with igord (the daemon doing all the coordination work).
This client has some "advanced" features  like pretty printing of junit results (Igor offers the result of the testruns in junit's XML format).

Some open issues:

• ovirt-node needs a target to build a testable ISO
• igor needs a feature to upload testsuites from the client side

All of this is up in the igor repository. ovirt-node related patches (e.g. merging of the igor plugin are pending). Just follow the node-devel ml to see when is is ready for daily usage.
That's it for now - thanks for watching.

Getting EFI (OVMF) into libvirt

Finally it's working like a charm - booting a VM with EFI (instead of a BIOS).

lersek pointed me to a page describing how to test SecureBoot with Fedora this implies the usage of EFI. And that page contains a link to an rpm carrying the EFI payload. Awesome.

After installing the RPM you only need to tell libvirt to use that bootloader instead of the default one.

All you need to do:

# Install a prerequirement
$ sudo dnf install seavgabios-bin ipxe-roms ipxe-roms-qemu

# Install OVMF
$ sudo rpm -ivh http://fedorapeople.org/~crobinso/\
    secureboot/edk2.manual-0-0.20130221.944c84a6.x86_64.rpm

# Create a VM pointing to OVMF
$ sudo virt-install --name f18-uefi \
                    --ram 2048 \
                    --boot loader=/usr/share/edk2.manual/\
                                  ovmf-x64/OVMF-pure-efi.fd \
                    --disk f18-uefi.qcow,format=qcow2,size=10 \
                    --os-variant fedora18 \
                    --cdrom /path/to/Fedora-18-x86_64-DVD.iso

That's it! I'm truly amazed.

Btw.: oVirt Node can also be tested this way. Sadly it currently crashes under EFI.

Doing PXE with libvirt / virt-manager

Installing hosts using PXE is a well known thing.
Why not do it within libvirt? Or: How do I do this in libvirt?
Do I need to setup my own dhcp server to pass the bootp option? Nope.
Just use libvirts default dnsmasq and add the bootp dhcp option.

All you need to do is editing the default network configuration using virsh (no way o do it from virt-manager).

# virsh net-destroy default
# virsh net-edit
  Now add "<bootp file='/pxelinux.0' server='$PXESERVERIP' />" under /network/ip/dhcp
# virsh net-start default

All done.
Just have a look at the definition here to read about more features.

Using oVirt Node with virt-manager

Virtualization is already an ubiquitous technique.
Fedora provides packages for many of the Linux virtualization components through the yum virtualization group.

$ sudo yum groupinstall virtualization

Well, anyway - When doing virtualization you need a host, hosting your virtualized guests. If you don't want to do this on your local machine - because it hasn't got the capabilities, isn't beefy enough, ... - you can use oVirt Node as a hypervisor on a second machine which you can easily manage from Fedora using virt-manager.
This can be useful for a small working group or developers.

oVirt Node is based on Fedora and optimized to quickly get a hypervisor up an running. You actually do not need to care about all the constraints - networking, services, storage, ... - you need to consider if you setup a hypervisor yourself (which can also be done with  Fedora). It is also stripped down (~150MB) to preserve most of the RAM and storage space to the virtualized guests.

Anyhow:

1. Download oVirt Node
2. Install it on a machine with a recent Intel or AMD processor
3. Log into the installed Node using admin and
1. Configure a network interface
2. Press F2 to drop to the console and run
3. /usr/libexec/ovirt-config-password 
1. set a root password
2. enable SSH access
4. Optional: ssh-copy-id your ssh key to node to allow a password-less login
5. User virt-manager to create a new connection (File -> New Connection) to the installed Node (IP can be found on the Node's Status page)
   URI: qemu+ssh://$OVIRTNODE/system

($OVIRTNODE needs to replaced accordingly)
Actually oVirt Node is intended to be used with oVirt Engine, which can manage from one up to a couple of hundreds (?) of Nodes.
But the Engine setup itself is not as easy as just using virt-manager :)
At least - Engine would be the next step to get used to the oVirt components.

P.s.: You can use virsh vol-upload to get some data onto the node.

Auto-Installing oVirt Node

oVirt - maybe you've heard about it. It's a project to create an open IaaS "virtualization management system" - So a bit like OpenStack, but different.
Fedora is the base for oVirt's hypervisor: "Node". Basicaly this is a stripped down Fedora, enriched with a couple of packages to prvide just enough to host some virtual guests and do some basic configuration.

Personally I'd like to use Node in conjunction with Gnome Boxes or virt-manager. But this is currently not possible - but we might get closer to it when solving this bug.
Anyhow, to quickly install oVirt Node you just need to add two (or three) additional kernel arguments:

BOOTIF=ethX storage_init

You should/could also add

adminpw=$ADMINPW

ADMINPW=$(openssl passwd -salt SALT) is a salted password, so you can log in (as admin) after the installation. Alternatively you can boot into single mode to reset the password.

The parameters above install oVirt node without user intervention,  setup networking on ethX and erase all data on the disk and create a defautl (lvm based) partitioning scheme.
The next step would be adding Node to oVirt Engine - or wait until it can be managed by virt-manager, which is much quicker to set-up :)