Marcus
86cb8cbce7
Merge branch 'build-tools_r27.0.2' into 'master'
...
makebuildserver: add build-tools_r27.0.2
See merge request fdroid/fdroidserver!413
2017-12-15 12:14:00 +00:00
relan
b2213f1e62
makebuildserver: add build-tools_r27.0.2
2017-12-15 14:34:40 +03:00
relan
b16669b2a0
makebuildserver: add Gradle 4.4
2017-12-15 14:32:34 +03:00
Hans-Christoph Steiner
1d83d4396c
makebuildserver: quiet rsync for copy_caches_from_host
2017-12-11 22:11:16 +01:00
Hans-Christoph Steiner
6640f276b0
Revert "makebuildserver: update SHA-256 for platform-27_r01.zip"
...
Looks like Google switched back to the old binary, which I guess is
good news?
https://issuetracker.google.com/issues/70292819
This reverts commit 956660085a!401
2017-12-08 09:44:10 +01:00
Hans-Christoph Steiner
0e37a18d83
makebuildserver: fix join() syntax error, it needs a list/tuple
...
If only there was a way to test this without taking hours to run...
fixes 964ef996a0
2017-12-07 11:52:52 +01:00
Hans-Christoph Steiner
956660085a
makebuildserver: update SHA-256 for platform-27_r01.zip
...
Someone forgot to call this _r02.zip:
-ro.build.version.incremental=4402310
+ro.build.version.incremental=4458339
https://verification.f-droid.org/build-metadata/platform-27_r01.html
!364
2017-12-06 22:05:09 +01:00
Hans-Christoph Steiner
964ef996a0
makebuildserver: make copy_caches_from_host do rsync like fdroid build
...
This rsync hung because of an SSH unknown key prompt. Since this is just
the vm host sshing to the vm guest, it is not essential to check the host
keys.
2017-12-06 20:20:17 +01:00
Marcus Hoffmann
f9b853ab91
makebuildserver: add ndk r16
2017-11-26 17:17:55 +01:00
relan
c859ffe145
makebuildserver: add Gradle 4.3.1
2017-11-14 10:36:12 +03:00
relan
16465c7a17
makebuildserver: add build-tools_r26.0.3
2017-11-12 10:48:00 +03:00
Sergey Eremin
0ec17495af
makebuildserver: add build-tools_r27.0.1
2017-11-10 18:38:58 +03:00
relan
db65ef5ed3
makebuildserver: add build-tools_r27
2017-11-05 10:59:01 +03:00
Hans-Christoph Steiner
ccccc0743f
Merge branch 'gradle-4.3' into 'master'
...
makebuildserver: add Gradle 4.3
See merge request fdroid/fdroidserver!365
2017-11-03 13:15:11 +00:00
relan
86512d3655
makebuildserver: add Gradle 4.3
2017-10-31 17:19:57 +03:00
relan
156fbcc377
makebuildserver: add platform-27_r01
...
Add Android 8.1.0 SDK platform.
2017-10-31 17:15:42 +03:00
Marcus
fde964479d
Merge branch 'makebuildserver/default_memory' into 'master'
...
makebuildserver: change mem default 4 GB
See merge request fdroid/fdroidserver!358
2017-10-25 12:41:25 +00:00
Marcus Hoffmann
f9444386e6
makebuildserver: change mem default 2 GB
...
1 GB is not enough for a modern android toolchain, so we change it to
something actually usable.
2017-10-25 14:13:35 +02:00
Marcus Hoffmann
7859244ce7
makebuildserver: remove reference to f-droid.org/jessie64.box
...
Also improve error message when the initial vagrant up fails, this is
most likely due to the missing jessie64.box file.
2017-10-20 04:41:50 +02:00
relan
dca96a6159
makebuildserver: add Gradle 4.2.1
...
N.B. Actual address of the distribution server is downloads.gradle.org;
services.gradle.org redirects to it.
2017-10-08 21:12:37 +03:00
Marcus Hoffmann
f25cfe1b37
add build-tools 26.0.2
2017-10-04 19:03:41 +02:00
relan
3fdddd1179
makebuildserver: add Gradle 4.2
2017-09-23 10:04:39 +03:00
Marcus Hoffmann
4b605e8b3e
remove trailing whitespace
2017-09-03 19:30:00 +02:00
Marcus Hoffmann
90180bb902
makebuildserver: add comment about not upgrading sdk-tools
2017-09-03 18:33:03 +02:00
Marcus Hoffmann
36ef93520c
Merge branch 'mr/verify_ca_certs_v2' into 'master'
...
makebuildserver: verify https all requests to avoid urllib3 warnings
See merge request !316
2017-08-28 15:05:24 +00:00
Marcus Hoffmann
69666839f7
makebuildserver: update platform and build tool releases
...
Update platform-25 to r03
Update platform-26 to r02
Add build-tools_r26.0.1
sha256 signatures locally computed after verifying the sha1 signature provided
by the repository2-1.xml file.
All these will be downloaded and installed in the buildserver through
provision-android-sdk anyway but this adds them to the cache.
2017-08-17 00:56:59 +02:00
Marcus Hoffmann
ae613ab277
makebuildserver: verify https all requests to avoid urllib3 warnings
...
According to http://docs.python-requests.org/en/master/user/advanced/#ca-certificates
python-requests >= 2.4.0 will use the certifi provided certificates.
We already recommend installing python3-certifi in the docs.
On debian requests is patched to use the system trust store instead.
2017-08-17 00:49:53 +02:00
Hans-Christoph Steiner
1955e2f153
Merge branch 'gradle-updates' into 'master'
...
makebuildserver: add Gradle 3.5.1, 4.0.1, 4.0.2 and 4.1
See merge request !313
2017-08-15 15:49:58 +00:00
relan
a09952ca2b
makebuildserver: add Gradle 3.5.1, 4.0.1, 4.0.2 and 4.1
...
Android Gradle plugin 3.0.0 requires Gradle 4.1.
2017-08-14 08:53:57 +03:00
relan
6f05cf85f9
makebuildserver: upgrade NDK r15b to r15c
2017-08-11 10:19:40 +03:00
relan
86b403b599
makebuildserver: add NDK r15b
2017-07-04 11:44:25 +03:00
Boris Kraut
1b88f886e7
makebs: add gradle4.0
2017-06-17 01:39:44 +02:00
Sergey Eremin
eb64b0889d
makebuildserver: bump tools, add api26 and build-tools 26
2017-06-09 20:35:04 +03:00
Boris Kraut
a57bff78d4
makebs: update to ndk-r14b and build-tools-25.0.3
2017-05-26 23:21:20 +02:00
Hans-Christoph Steiner
a01e302cde
makebuildserver: make config['domain'] local var where its needed
2017-05-23 20:08:29 +02:00
Michael Pöhn
fc660048eb
removed unused buildserver code
2017-05-23 20:08:08 +02:00
Hans-Christoph Steiner
9ef936c21a
leave VirtualBox vagrant package as it was originally
...
We only need Vagrantfile hacks for KVM.
2017-05-23 20:06:47 +02:00
Michael Pöhn
510efaa024
makebuildserver vbox logmessage fix
2017-05-23 20:06:06 +02:00
Michael Pöhn
b01d48a4fd
makebuildserver deal with apt cache lock
2017-05-23 20:06:06 +02:00
Hans-Christoph Steiner
5dbcd0e9bd
added box handling to vmtools
2017-05-23 20:06:06 +02:00
Hans-Christoph Steiner
01b6473823
refactored kvm_package to vmtools
2017-05-23 20:06:06 +02:00
Hans-Christoph Steiner
5580a685db
added makebuildserver option for keeping vagrant box
...
This is very useful for debugging this process, and also for people
who might want to keep a working copy of the box.
2017-05-23 20:06:06 +02:00
Michael Pöhn
4347c10d9d
use configured vm provider in when calling destroy in makebuildserver
2017-05-23 20:06:06 +02:00
Michael Pöhn
a414aa00ff
use overhauled mv destroy code in build.py
2017-05-23 20:06:06 +02:00
Michael Pöhn
92fada803e
overhauled and moved destroying builder vm to vmtools.py
2017-05-23 20:04:08 +02:00
Michael Pöhn
fb03e17849
fdroid build: added sleep after destroy/undefine
2017-05-23 20:04:08 +02:00
Michael Pöhn
718d01dea2
makebuildserver added sleep after destroy/undefine
2017-05-23 20:04:08 +02:00
Michael Pöhn
3c4b1dec84
makebuildserver more robust codepath for vagrant destroy
2017-05-23 20:04:08 +02:00
Michael Pöhn
7e8f7c65bc
makebuildserver added failsafe when destroy vagrant vm
2017-05-23 20:04:08 +02:00
Michael Pöhn
70a827d59a
makebuildserver use virsh instead of libvirt for forcing domain off
2017-05-23 20:04:08 +02:00
Michael Pöhn
8e5446068b
makebuildserver debug logging when initial provisioning fails
2017-05-23 20:04:08 +02:00
Michael Pöhn
16b609215e
overhauled makebuildserver libvirt vm cleanup
2017-05-23 20:04:08 +02:00
Michael Pöhn
440509cf8a
makebuildserver debugging log details
2017-05-23 20:04:08 +02:00
Michael Pöhn
2aa5523011
makebuildserver prune gloabal vagrant status when purging broken VMs
2017-05-23 20:04:08 +02:00
Michael Pöhn
8cfd3ca770
delete .vagrant dir when cleaning up buildserver vm
2017-05-23 20:04:08 +02:00
Hans-Christoph Steiner
2993674aa8
calculate correct size for buildserver-box in makebuildserver
...
https://gitlab.com/fdroid/fdroidserver/issues/238#note_24000153
"Our hard-coded image size meta-data (1000) is for some interpreted as less
than the size of the box-image by my kvm setup. This makes grub/initrd
refuse to boot. So I've changed the metadata size to 9999 which resulted in
an actually booting vm. I can log in on the builder-vm via virt-manager
and virsh.
2017-05-23 20:04:08 +02:00
Michael Pöhn
fad98eeb7f
build: fixed kvm snapshot support; makebuildserver: setup kvm ssh credentials
2017-05-23 20:04:08 +02:00
Hans-Christoph Steiner
69e4b91d3f
makebuildserver: package up KVM VM as a vagrant box
...
`vagrant package` does not work with KVM, so we have to hack together our
own until someone implements it (suppose we should do it). This is a hacked
up version based on:
d7d440ea8f/tools/create_box.sh#238
2017-05-23 20:04:08 +02:00
Hans-Christoph Steiner
988ac21e7f
buildserver: make --clean destroy reliably
...
This prevents v.destroy() from running if Vagrantfile.yaml does not exist,
since that is required for vagrant to run: is the core config including the
name of the box, etc. Otherwise, it would exit with an error.
This also does complete cleanup when using libvirt.
2017-05-23 20:04:08 +02:00
Hans-Christoph Steiner
299ed82a88
buildserver: consolidate boxfile export code into one block
...
This is just for clarity, and moving more code into the main() function.
2017-05-23 20:04:08 +02:00
Hans-Christoph Steiner
daade7656a
buildserver: add copy_caches_from_host config option
...
For people using slow, expensive, and/or flaky internet, liberal use of
caching can make a huge difference. The restricted environment of the
gpjenkins box has been a good test environment for this (Tor-only,
whitelist of allowed IPs to visit, home internet connection).
2017-05-23 20:04:08 +02:00
Hans-Christoph Steiner
4cde71552f
buildserver: run_via_vagrant_ssh() to run cmds via python-vagrant
...
This moves the last vagrant call in a subprocess.
2017-05-23 20:04:08 +02:00
Hans-Christoph Steiner
6464ec55b7
buildserver: move code into main() method to always stop thread
...
By running the whole program in a main() function, it can be wrapped in
try/finally in order to stop the background display thread. This is also
done in ./fdroid, its standard practice for Python CLI utilities.
2017-05-23 20:04:08 +02:00
Hans-Christoph Steiner
675500ad88
buildserver: display verbose logging in a background tail
2017-05-23 20:04:08 +02:00
Hans-Christoph Steiner
4b03c3d42d
buildserver: replace custom code with python-vagrant
...
I ran into some annoying issues with UTF-8 output in the vagrant logs, and
it was hard to solve. So I switched to using python-vagrant, which handles
it all for us. Its been around since 2012, has a number of contributors,
and is still actively maintained, so it seems like a good bet. I also
packaged it for Debian, including a backport in jessie-backports.
On Debian/jessie, do `apt-get install python3-vagrant/jessie-backports`
2017-05-23 20:04:08 +02:00
Andrew Patrikalakis
9ee96117d8
Check for libvirt specific shutoff message
2017-04-29 18:38:44 -07:00
Boris Kraut
10e91d1be2
makebs: add gradle 3.5
2017-04-15 15:08:13 +02:00
Jan Christian Grünhage
f637b461f0
Update support repository.
2017-03-29 10:33:38 +00:00
Boris Kraut
53be33055b
makebs: update support repo to r46
2017-03-21 21:50:29 +01:00
Boris Kraut
ed310f0404
makebs: add ndk r14
2017-03-19 22:27:51 +01:00
Hans-Christoph Steiner
1e9406778f
buildserver: remove my silly mistake
...
Should have done this through a merge request... code review!
2017-03-14 20:14:06 +01:00
Hans-Christoph Steiner
ee82c30cb8
buildserver: bump to android_m2repository r45
2017-03-14 20:04:50 +01:00
Boris Kraut
e8d0a8ede8
makebs: add gradle 3.4.1
2017-03-10 14:29:36 +01:00
Boris Kraut
add1ad6eb5
makebs: add gradle 3.4
2017-02-26 03:23:49 +01:00
Boris Kraut
687061f47e
makebs: update support repo to r43, closes fdroid/rfp#49
2017-02-23 01:27:10 +01:00
Hans-Christoph Steiner
1d91a400a2
buildserver: update to latest tools and m2repository
2017-02-13 20:10:48 +01:00
Boris Kraut
e382f09fd6
makebs: bump support repo to r43
2017-02-11 14:40:40 +01:00
Boris Kraut
f32f4a68d1
makebs: upgrade support repo to r42
2017-02-04 15:01:15 +01:00
Hans-Christoph Steiner
4741f75f03
buildserver: update android_m2repository to latest (r41)
2017-01-12 10:19:38 +01:00
Boris Kraut
d63a1fde19
makebs: install gradle 3.3
2017-01-08 21:13:24 +01:00
est31
d6da0a283d
Makebs: add build tools 25.0.2
2016-12-17 23:36:48 +01:00
est31
d39f54f04d
Add gradle 3.2.1
2016-12-02 08:16:04 +01:00
Boris Kraut
51083fdeb4
makebs: add buildtools 25.0.1
2016-12-01 08:44:47 +01:00
est31
ddefec33cd
Add ndk 13b
2016-11-25 21:18:37 +01:00
Daniel Martí
8cdf1b0223
makebs: add gradle 3.2
2016-11-15 21:07:35 +00:00
Daniel Martí
483338843f
makebs: update to support repo r40
2016-11-15 20:56:04 +00:00
Daniel Martí
82b1d7ad14
all: make newer pycodestyle happy
...
Apparently the "two empty lines" rule is now stricter.
2016-11-15 20:55:06 +00:00
Daniel Martí
98b7538789
makebs: bump m2repo, add api25 and build-tools 25
2016-10-23 16:47:19 +01:00
Ciaran Gultnieks
aa4aa45703
Merge branch 'build-tools' into 'master'
...
makebs: add build-tools 24.0.3
See merge request !175
2016-10-05 18:09:46 +00:00
Daniel Martí
b34cef8776
makebs: add build-tools 24.0.3
2016-10-03 09:10:41 +01:00
Hans-Christoph Steiner
1b09ab5a79
buildserver: run dir check as very first thing
...
No point in running any other code if the script is just going to bail
out with an error. This assumes that ./makebuildserver is only ever
run from a git clone of fdroidserver.git.
2016-09-28 09:52:49 +02:00
Hans-Christoph Steiner
a3d32c65a4
buildserver: use py3's pathlib to generate cache file URL
...
This should handle edge cases better, like odd characters in the path, etc.
2016-09-28 09:52:49 +02:00
est31
91c5fb567a
Add qt sdk support
2016-09-27 14:36:12 +02:00
Hans-Christoph Steiner
1001e5e9b9
buildserver: bochs can also be used with KVM
...
jenkins.debian.net is being detected as 'bochs' rather than 'qemu'.
2016-09-23 17:41:41 +02:00
Hans-Christoph Steiner
1e8fd01c1e
buildserver: only check cache permissions when using libvirt
...
VirtualBox runs as the same user as `fdroid`, so the cache does not need to
be accessible by the world. On libvirt, libvirtd runs the VMs as its own
user, so in that case, the cache dirs must have permissions to let that
user access them.
2016-09-23 15:20:05 +02:00
Hans-Christoph Steiner
f6a487eee4
buildserver: wipe snapshot from libvirt store on --clean
2016-09-23 15:06:41 +02:00
Ciaran Gultnieks
5667d16498
Merge branch 'buildserver-qemu-kvm' into 'master'
...
buildserver running in qemu/kvm to support KVM on KVM
jenkins.debian.net runs in QEMU/KVM instances, so in order to run the F-Droid buildserver there, it needs to work inside of a KVM guest. The best way I found to do that is to create QEMU/KVM instances via KVM's "nested" virtualization support. This collection of commits enables using QEMU/KVM as the buildserver when `./makebuildserver` detects that it is running inside of KVM. Otherwise, the old behavior is default: running in VirtualBox.
I have run these tests inside of ubuntu/16.04 on bare metal, which uses VirtualBox, and ubuntu/16.04 KVM guest, which uses QEMU/KVM. It'll also run on the Guardian Project jenkins box, which is Debian/jessie.
@mvdan @CiaranG @krt
See merge request !168
2016-09-23 12:25:12 +00:00
est31
56a0077183
makebs: add gradle 3.1
2016-09-23 07:50:52 +02:00
Hans-Christoph Steiner
68ec7c2e88
buildserver: auto-detect and use libvirt's QEMU+KVM
...
For running in QEMU/KVM guests like on jenkins.debian.net, this sets up the
whole process automatically. This only really covers the case where this
is running in a KVM guest, and the original case of running VirtualBox on
bare metal. It could be extended to cover more cases if someone wanted to.
2016-09-19 16:33:12 +02:00
Hans-Christoph Steiner
f306e32636
buildserver: make sure cachedir is accessible to VM instance
...
It seems that KVM/QEMU's 9p sharing is more sensitive to file perms. If
~/.cache is 0700, then ~/.cache/fdroidserver cannot be mounted in the guest
2016-09-19 16:33:12 +02:00
