https://gitlab.com/fdroid/fdroidserver/issues/238#note_24000153
"Our hard-coded image size meta-data (1000) is for some interpreted as less
than the size of the box-image by my kvm setup. This makes grub/initrd
refuse to boot. So I've changed the metadata size to 9999 which resulted in
an actually booting vm. I can log in on the builder-vm via virt-manager
and virsh.
`vagrant package` does not work with KVM, so we have to hack together our
own until someone implements it (suppose we should do it). This is a hacked
up version based on:
d7d440ea8f/tools/create_box.sh#238
This prevents v.destroy() from running if Vagrantfile.yaml does not exist,
since that is required for vagrant to run: is the core config including the
name of the box, etc. Otherwise, it would exit with an error.
This also does complete cleanup when using libvirt.
For people using slow, expensive, and/or flaky internet, liberal use of
caching can make a huge difference. The restricted environment of the
gpjenkins box has been a good test environment for this (Tor-only,
whitelist of allowed IPs to visit, home internet connection).
By running the whole program in a main() function, it can be wrapped in
try/finally in order to stop the background display thread. This is also
done in ./fdroid, its standard practice for Python CLI utilities.
I ran into some annoying issues with UTF-8 output in the vagrant logs, and
it was hard to solve. So I switched to using python-vagrant, which handles
it all for us. Its been around since 2012, has a number of contributors,
and is still actively maintained, so it seems like a good bet. I also
packaged it for Debian, including a backport in jessie-backports.
On Debian/jessie, do `apt-get install python3-vagrant/jessie-backports`
No point in running any other code if the script is just going to bail
out with an error. This assumes that ./makebuildserver is only ever
run from a git clone of fdroidserver.git.
VirtualBox runs as the same user as `fdroid`, so the cache does not need to
be accessible by the world. On libvirt, libvirtd runs the VMs as its own
user, so in that case, the cache dirs must have permissions to let that
user access them.
buildserver running in qemu/kvm to support KVM on KVM
jenkins.debian.net runs in QEMU/KVM instances, so in order to run the F-Droid buildserver there, it needs to work inside of a KVM guest. The best way I found to do that is to create QEMU/KVM instances via KVM's "nested" virtualization support. This collection of commits enables using QEMU/KVM as the buildserver when `./makebuildserver` detects that it is running inside of KVM. Otherwise, the old behavior is default: running in VirtualBox.
I have run these tests inside of ubuntu/16.04 on bare metal, which uses VirtualBox, and ubuntu/16.04 KVM guest, which uses QEMU/KVM. It'll also run on the Guardian Project jenkins box, which is Debian/jessie.
@mvdan @CiaranG @krt
See merge request !168
For running in QEMU/KVM guests like on jenkins.debian.net, this sets up the
whole process automatically. This only really covers the case where this
is running in a KVM guest, and the original case of running VirtualBox on
bare metal. It could be extended to cover more cases if someone wanted to.
This is the last thing using Chef, which adds a lot of time to the time it
takes to fully provision the buildserver. This slows down development on
the things we are actually using, like running all builds on
jenkins.debian.net.
#210#165
