No point in running any other code if the script is just going to bail
out with an error. This assumes that ./makebuildserver is only ever
run from a git clone of fdroidserver.git.
VirtualBox runs as the same user as `fdroid`, so the cache does not need to
be accessible by the world. On libvirt, libvirtd runs the VMs as its own
user, so in that case, the cache dirs must have permissions to let that
user access them.
buildserver running in qemu/kvm to support KVM on KVM
jenkins.debian.net runs in QEMU/KVM instances, so in order to run the F-Droid buildserver there, it needs to work inside of a KVM guest. The best way I found to do that is to create QEMU/KVM instances via KVM's "nested" virtualization support. This collection of commits enables using QEMU/KVM as the buildserver when `./makebuildserver` detects that it is running inside of KVM. Otherwise, the old behavior is default: running in VirtualBox.
I have run these tests inside of ubuntu/16.04 on bare metal, which uses VirtualBox, and ubuntu/16.04 KVM guest, which uses QEMU/KVM. It'll also run on the Guardian Project jenkins box, which is Debian/jessie.
@mvdan @CiaranG @krt
See merge request !168
For running in QEMU/KVM guests like on jenkins.debian.net, this sets up the
whole process automatically. This only really covers the case where this
is running in a KVM guest, and the original case of running VirtualBox on
bare metal. It could be extended to cover more cases if someone wanted to.
This is the last thing using Chef, which adds a lot of time to the time it
takes to fully provision the buildserver. This slows down development on
the things we are actually using, like running all builds on
jenkins.debian.net.
#210#165
This reverts commit 82d09560c6.
It doesn't work - the setup scripts are expecting a ".bin" file (which
is apparently a 7z archive), but what's actually got is a ".zip".
Conflicts:
buildserver/provision-android-ndk
Without this, running makebuildserver from a clean master results in the
following:
$ ./makebuildserver
Traceback (most recent call last):
File "./makebuildserver", line 74, in <module>
del(config['__builtins__']) # added by compile/exec
KeyError: '__builtins__'
This is because a clean checkout has no config, thus exec is never
actually ran.
Amusingly, the commit changed more than just this so a simple git revert
is not possible.
The problem is that the zip is replaced with each point release (24.0.1
at the time of writing) and there is no way to get a URL to a single,
non-changing version. Hence any caching or checksums are completely
worthless and will break every few weeks.
Vagrantfile is now committed and not changed between configurations. It is
configed by translating the python config file's dict to a YAML file, which
Vagrantfile now loads and uses. This makes it a lot easier for vagrant
users and python programmers to understand, and hopefully makes it easier
to maintain and test with.
Python can easily output dicts as YAML, and a Vagrantfile is a ruby script,
which can easily read YAML. Going this route means that Vagrantfile can
ultimately be committed to git, and the configuration will happen all via
Python dicts output as YAML. That makes it drastically easier to follow
the code, and to make modifications.
easy_install does not provide any download caching, while pip does. This
also moves the python module installing a shell script that takes python
packages as args. That will allow for future uses like allowing app
metadata to include pip modules that they need.
This makes it so there is only a single `apt-get install` command run,
instead of one command per-package like with the chef script. It also adds
`apt-get upgrade` to make sure that the base box is fully up-to-date.
This is part of the effort to remove moving parts from the whole build
server setup. Why wrap shell scripts in ruby and chef if we can just
directly run a shell script?
`fdroid build` handles setting the NDK env vars since the NDK version can
change depending on the app being build. Unlike ANDROID_HOME, there is no
single global NDK location. The NDK installs are all versioned.
The progress bar updating at each 1k chunk spams the terminal to such an
extent it stops responding to keypresses (at least with a fast
connection). It also seems extremely inefficient to be writing the file
in 1k chunks. This makes it 64k, which is more reasonable but quite
probably still too small.
