Hans-Christoph Steiner
12c80f9062
update outdated pylint setup
...
The logilab-astng lib is dead, replaced by python-astroid. The crazy astng
plugin is no longer needed also.
#281
2017-05-23 22:34:16 +02:00
Hans-Christoph Steiner
361ce5ca41
Merge branch 'python-vagrant-copy-caches' into 'master'
...
complete staging buildserver setup on jenkins.debian.net using nested KVM instances
See merge request !176
2017-05-23 18:38:57 +00:00
Hans-Christoph Steiner
a01e302cde
makebuildserver: make config['domain'] local var where its needed
2017-05-23 20:08:29 +02:00
Michael Pöhn
c8234919df
refactored vm related code from build.py to vmtools.py
2017-05-23 20:08:29 +02:00
Michael Pöhn
fc660048eb
removed unused buildserver code
2017-05-23 20:08:08 +02:00
Hans-Christoph Steiner
9ef936c21a
leave VirtualBox vagrant package as it was originally
...
We only need Vagrantfile hacks for KVM.
2017-05-23 20:06:47 +02:00
Michael Pöhn
8f1fabfed6
restart builder vm when ssh connection fails
2017-05-23 20:06:47 +02:00
Hans-Christoph Steiner
6106b962a2
build: delete bad builder/ symlinks
...
If builder/ is a symlink but is not detected as a directory by
os.path.isdir(), then it is a broken symlink.
2017-05-23 20:06:06 +02:00
Michael Pöhn
4546929d7f
wait a sec after suspending
2017-05-23 20:06:06 +02:00
Michael Pöhn
48159f005a
reset buildserver vm if vagrant uuid not present
2017-05-23 20:06:06 +02:00
Michael Pöhn
510efaa024
makebuildserver vbox logmessage fix
2017-05-23 20:06:06 +02:00
Michael Pöhn
0ec5422952
use uuid for vbox snapshots again
2017-05-23 20:06:06 +02:00
Michael Pöhn
358b00d7aa
removed useless vm validity check; attempted to fix vbox support
2017-05-23 20:06:06 +02:00
Michael Pöhn
1b1d6b7d96
deal with outdated box images in libvirt storage pool
2017-05-23 20:06:06 +02:00
Michael Pöhn
b01d48a4fd
makebuildserver deal with apt cache lock
2017-05-23 20:06:06 +02:00
Hans-Christoph Steiner
acf25a3999
build server use up instead or resume; logging
2017-05-23 20:06:06 +02:00
Hans-Christoph Steiner
c749c68486
test script for vmtools
2017-05-23 20:06:06 +02:00
Michael Pöhn
3187d2cbcf
revised build server creation
2017-05-23 20:06:06 +02:00
Michael Pöhn
8abd3f1cbc
auto-lookup vm provider based on available executables; more fault tolerant vagrant package
2017-05-23 20:06:06 +02:00
Hans-Christoph Steiner
bba6b8ab0a
fixed reading libvirt box image size
2017-05-23 20:06:06 +02:00
Michael Pöhn
1bd51966b8
vmtools debug logging for check_output calls
2017-05-23 20:06:06 +02:00
Hans-Christoph Steiner
5dbcd0e9bd
added box handling to vmtools
2017-05-23 20:06:06 +02:00
Hans-Christoph Steiner
01b6473823
refactored kvm_package to vmtools
2017-05-23 20:06:06 +02:00
Hans-Christoph Steiner
5580a685db
added makebuildserver option for keeping vagrant box
...
This is very useful for debugging this process, and also for people
who might want to keep a working copy of the box.
2017-05-23 20:06:06 +02:00
Michael Pöhn
d180aa2658
fix virsh destroy parameters
2017-05-23 20:06:06 +02:00
Michael Pöhn
4347c10d9d
use configured vm provider in when calling destroy in makebuildserver
2017-05-23 20:06:06 +02:00
Michael Pöhn
a414aa00ff
use overhauled mv destroy code in build.py
2017-05-23 20:06:06 +02:00
Michael Pöhn
92fada803e
overhauled and moved destroying builder vm to vmtools.py
2017-05-23 20:04:08 +02:00
Michael Pöhn
fb03e17849
fdroid build: added sleep after destroy/undefine
2017-05-23 20:04:08 +02:00
Michael Pöhn
718d01dea2
makebuildserver added sleep after destroy/undefine
2017-05-23 20:04:08 +02:00
Michael Pöhn
3c4b1dec84
makebuildserver more robust codepath for vagrant destroy
2017-05-23 20:04:08 +02:00
Michael Pöhn
7e8f7c65bc
makebuildserver added failsafe when destroy vagrant vm
2017-05-23 20:04:08 +02:00
Michael Pöhn
70a827d59a
makebuildserver use virsh instead of libvirt for forcing domain off
2017-05-23 20:04:08 +02:00
Michael Pöhn
8e5446068b
makebuildserver debug logging when initial provisioning fails
2017-05-23 20:04:08 +02:00
Michael Pöhn
16b609215e
overhauled makebuildserver libvirt vm cleanup
2017-05-23 20:04:08 +02:00
Michael Pöhn
440509cf8a
makebuildserver debugging log details
2017-05-23 20:04:08 +02:00
Michael Pöhn
2aa5523011
makebuildserver prune gloabal vagrant status when purging broken VMs
2017-05-23 20:04:08 +02:00
Michael Pöhn
8cfd3ca770
delete .vagrant dir when cleaning up buildserver vm
2017-05-23 20:04:08 +02:00
Michael Pöhn
437ff7c3f0
jenkins makebuildserver fail if vagrant box was not created
2017-05-23 20:04:08 +02:00
Hans-Christoph Steiner
2993674aa8
calculate correct size for buildserver-box in makebuildserver
...
https://gitlab.com/fdroid/fdroidserver/issues/238#note_24000153
"Our hard-coded image size meta-data (1000) is for some interpreted as less
than the size of the box-image by my kvm setup. This makes grub/initrd
refuse to boot. So I've changed the metadata size to 9999 which resulted in
an actually booting vm. I can log in on the builder-vm via virt-manager
and virsh.
2017-05-23 20:04:08 +02:00
Michael Pöhn
413c3836d5
auto-reset broken builder vm
2017-05-23 20:04:08 +02:00
Michael Pöhn
ce3c959ce5
scan vm provider again after re-creating builder vm
2017-05-23 20:04:08 +02:00
Michael Pöhn
c54e0565d7
added some debug listings to jenkins makebuildserver script
2017-05-23 20:04:08 +02:00
Michael Pöhn
34cddd3be8
delete associated libvirt domain/image when destroying builder vm
2017-05-23 20:04:08 +02:00
Michael Pöhn
fad98eeb7f
build: fixed kvm snapshot support; makebuildserver: setup kvm ssh credentials
2017-05-23 20:04:08 +02:00
Michael Pöhn
fa2d44ee94
added libvirt vm-provider support to build.py
2017-05-23 20:04:08 +02:00
Hans-Christoph Steiner
69e4b91d3f
makebuildserver: package up KVM VM as a vagrant box
...
`vagrant package` does not work with KVM, so we have to hack together our
own until someone implements it (suppose we should do it). This is a hacked
up version based on:
d7d440ea8f/tools/create_box.sh#238
2017-05-23 20:04:08 +02:00
Hans-Christoph Steiner
988ac21e7f
buildserver: make --clean destroy reliably
...
This prevents v.destroy() from running if Vagrantfile.yaml does not exist,
since that is required for vagrant to run: is the core config including the
name of the box, etc. Otherwise, it would exit with an error.
This also does complete cleanup when using libvirt.
2017-05-23 20:04:08 +02:00
Hans-Christoph Steiner
299ed82a88
buildserver: consolidate boxfile export code into one block
...
This is just for clarity, and moving more code into the main() function.
2017-05-23 20:04:08 +02:00
Hans-Christoph Steiner
daade7656a
buildserver: add copy_caches_from_host config option
...
For people using slow, expensive, and/or flaky internet, liberal use of
caching can make a huge difference. The restricted environment of the
gpjenkins box has been a good test environment for this (Tor-only,
whitelist of allowed IPs to visit, home internet connection).
2017-05-23 20:04:08 +02:00
