kamp/fdroidserver
1
0
Fork 0
mirror of https://gitlab.com/fdroid/fdroidserver.git synced 2024-07-14 13:00:08 +02:00
Code Activity
5,453 Commits 1 Branch 58 Tags 48 MiB
43df6ab64a
Commit Graph

266 Commits

Author SHA1 Message Date
Hans-Christoph Steiner
1d83d4396c makebuildserver: quiet rsync for copy_caches_from_host 2017-12-11 22:11:16 +01:00
Hans-Christoph Steiner
6640f276b0 Revert "makebuildserver: update SHA-256 for platform-27_r01.zip" 
Looks like Google switched back to the old binary, which I guess is
good news?
https://issuetracker.google.com/issues/70292819

This reverts commit 956660085a.

!401
 2017-12-08 09:44:10 +01:00
Hans-Christoph Steiner
0e37a18d83 makebuildserver: fix join() syntax error, it needs a list/tuple 
If only there was a way to test this without taking hours to run...

fixes 964ef996a0
 2017-12-07 11:52:52 +01:00
Hans-Christoph Steiner
956660085a makebuildserver: update SHA-256 for platform-27_r01.zip 
Someone forgot to call this _r02.zip:
-ro.​build.​version.​incremental=4402310
+ro.​build.​version.​incremental=4458339

https://verification.f-droid.org/build-metadata/platform-27_r01.html

!364
 2017-12-06 22:05:09 +01:00
Hans-Christoph Steiner
964ef996a0 makebuildserver: make copy_caches_from_host do rsync like fdroid build 
This rsync hung because of an SSH unknown key prompt.  Since this is just
the vm host sshing to the vm guest, it is not essential to check the host
keys.
 2017-12-06 20:20:17 +01:00
Marcus Hoffmann
f9b853ab91 makebuildserver: add ndk r16 2017-11-26 17:17:55 +01:00
relan
c859ffe145 makebuildserver: add Gradle 4.3.1 2017-11-14 10:36:12 +03:00
relan
16465c7a17 makebuildserver: add build-tools_r26.0.3 2017-11-12 10:48:00 +03:00
Sergey Eremin
0ec17495af makebuildserver: add build-tools_r27.0.1 2017-11-10 18:38:58 +03:00
relan
db65ef5ed3 makebuildserver: add build-tools_r27 2017-11-05 10:59:01 +03:00
Hans-Christoph Steiner
ccccc0743f Merge branch 'gradle-4.3' into 'master' 
makebuildserver: add Gradle 4.3

See merge request fdroid/fdroidserver!365
 2017-11-03 13:15:11 +00:00
relan
86512d3655 makebuildserver: add Gradle 4.3 2017-10-31 17:19:57 +03:00
relan
156fbcc377 makebuildserver: add platform-27_r01 
Add Android 8.1.0 SDK platform.
 2017-10-31 17:15:42 +03:00
Marcus
fde964479d Merge branch 'makebuildserver/default_memory' into 'master' 
makebuildserver: change mem default 4 GB

See merge request fdroid/fdroidserver!358
 2017-10-25 12:41:25 +00:00
Marcus Hoffmann
f9444386e6 makebuildserver: change mem default 2 GB 
1 GB is not enough for a modern android toolchain, so we change it to
something actually usable.
 2017-10-25 14:13:35 +02:00
Marcus Hoffmann
7859244ce7 makebuildserver: remove reference to f-droid.org/jessie64.box 
Also improve error message when the initial vagrant up fails, this is
most likely due to the missing jessie64.box file.
 2017-10-20 04:41:50 +02:00
relan
dca96a6159 makebuildserver: add Gradle 4.2.1 
N.B. Actual address of the distribution server is downloads.gradle.org;
services.gradle.org redirects to it.
 2017-10-08 21:12:37 +03:00
Marcus Hoffmann
f25cfe1b37 add build-tools 26.0.2 2017-10-04 19:03:41 +02:00
relan
3fdddd1179 makebuildserver: add Gradle 4.2 2017-09-23 10:04:39 +03:00
Marcus Hoffmann
4b605e8b3e remove trailing whitespace 2017-09-03 19:30:00 +02:00
Marcus Hoffmann
90180bb902 makebuildserver: add comment about not upgrading sdk-tools 2017-09-03 18:33:03 +02:00
Marcus Hoffmann
36ef93520c Merge branch 'mr/verify_ca_certs_v2' into 'master' 
makebuildserver: verify https all requests to avoid urllib3 warnings

See merge request !316
 2017-08-28 15:05:24 +00:00
Marcus Hoffmann
69666839f7 makebuildserver: update platform and build tool releases 
Update platform-25 to r03
Update platform-26 to r02
Add build-tools_r26.0.1

sha256 signatures locally computed after verifying the sha1 signature provided
by the repository2-1.xml file.

All these will be downloaded and installed in the buildserver through
provision-android-sdk anyway but this adds them to the cache.
 2017-08-17 00:56:59 +02:00
Marcus Hoffmann
ae613ab277 makebuildserver: verify https all requests to avoid urllib3 warnings 
According to http://docs.python-requests.org/en/master/user/advanced/#ca-certificates
python-requests >= 2.4.0 will use the certifi provided certificates.
We already recommend installing python3-certifi in the docs.
On debian requests is patched to use the system trust store instead.
 2017-08-17 00:49:53 +02:00
Hans-Christoph Steiner
1955e2f153 Merge branch 'gradle-updates' into 'master' 
makebuildserver: add Gradle 3.5.1, 4.0.1, 4.0.2 and 4.1

See merge request !313
 2017-08-15 15:49:58 +00:00
relan
a09952ca2b makebuildserver: add Gradle 3.5.1, 4.0.1, 4.0.2 and 4.1 
Android Gradle plugin 3.0.0 requires Gradle 4.1.
 2017-08-14 08:53:57 +03:00
relan
6f05cf85f9 makebuildserver: upgrade NDK r15b to r15c 2017-08-11 10:19:40 +03:00
relan
86b403b599 makebuildserver: add NDK r15b 2017-07-04 11:44:25 +03:00
Boris Kraut
1b88f886e7 makebs: add gradle4.0 2017-06-17 01:39:44 +02:00
Sergey Eremin
eb64b0889d makebuildserver: bump tools, add api26 and build-tools 26 2017-06-09 20:35:04 +03:00
Boris Kraut
a57bff78d4 makebs: update to ndk-r14b and build-tools-25.0.3 2017-05-26 23:21:20 +02:00
Hans-Christoph Steiner
a01e302cde makebuildserver: make config['domain'] local var where its needed 2017-05-23 20:08:29 +02:00
Michael Pöhn
fc660048eb removed unused buildserver code 2017-05-23 20:08:08 +02:00
Hans-Christoph Steiner
9ef936c21a leave VirtualBox vagrant package as it was originally 
We only need Vagrantfile hacks for KVM.
 2017-05-23 20:06:47 +02:00
Michael Pöhn
510efaa024 makebuildserver vbox logmessage fix 2017-05-23 20:06:06 +02:00
Michael Pöhn
b01d48a4fd makebuildserver deal with apt cache lock 2017-05-23 20:06:06 +02:00
Hans-Christoph Steiner
5dbcd0e9bd added box handling to vmtools 2017-05-23 20:06:06 +02:00
Hans-Christoph Steiner
01b6473823 refactored kvm_package to vmtools 2017-05-23 20:06:06 +02:00
Hans-Christoph Steiner
5580a685db added makebuildserver option for keeping vagrant box 
This is very useful for debugging this process, and also for people
who might want to keep a working copy of the box.
 2017-05-23 20:06:06 +02:00
Michael Pöhn
4347c10d9d use configured vm provider in when calling destroy in makebuildserver 2017-05-23 20:06:06 +02:00
Michael Pöhn
a414aa00ff use overhauled mv destroy code in build.py 2017-05-23 20:06:06 +02:00
Michael Pöhn
92fada803e overhauled and moved destroying builder vm to vmtools.py 2017-05-23 20:04:08 +02:00
Michael Pöhn
fb03e17849 fdroid build: added sleep after destroy/undefine 2017-05-23 20:04:08 +02:00
Michael Pöhn
718d01dea2 makebuildserver added sleep after destroy/undefine 2017-05-23 20:04:08 +02:00
Michael Pöhn
3c4b1dec84 makebuildserver more robust codepath for vagrant destroy 2017-05-23 20:04:08 +02:00
Michael Pöhn
7e8f7c65bc makebuildserver added failsafe when destroy vagrant vm 2017-05-23 20:04:08 +02:00
Michael Pöhn
70a827d59a makebuildserver use virsh instead of libvirt for forcing domain off 2017-05-23 20:04:08 +02:00
Michael Pöhn
8e5446068b makebuildserver debug logging when initial provisioning fails 2017-05-23 20:04:08 +02:00
Michael Pöhn
16b609215e overhauled makebuildserver libvirt vm cleanup 2017-05-23 20:04:08 +02:00
Michael Pöhn
440509cf8a makebuildserver debugging log details 2017-05-23 20:04:08 +02:00
Michael Pöhn
2aa5523011 makebuildserver prune gloabal vagrant status when purging broken VMs 2017-05-23 20:04:08 +02:00
Michael Pöhn
8cfd3ca770 delete .vagrant dir when cleaning up buildserver vm 2017-05-23 20:04:08 +02:00
Hans-Christoph Steiner
2993674aa8 calculate correct size for buildserver-box in makebuildserver 
https://gitlab.com/fdroid/fdroidserver/issues/238#note_24000153
"Our hard-coded image size meta-data (1000) is for some interpreted as less
than the size of the box-image by my kvm setup. This makes grub/initrd
refuse to boot. So I've changed the metadata size to 9999 which resulted in
an actually booting vm. I can log in on the builder-vm via virt-manager
and virsh.
 2017-05-23 20:04:08 +02:00
Michael Pöhn
fad98eeb7f build: fixed kvm snapshot support; makebuildserver: setup kvm ssh credentials 2017-05-23 20:04:08 +02:00
Hans-Christoph Steiner
69e4b91d3f makebuildserver: package up KVM VM as a vagrant box 
`vagrant package` does not work with KVM, so we have to hack together our
own until someone implements it (suppose we should do it). This is a hacked
up version based on:
d7d440ea8f/tools/create_box.sh

#238
 2017-05-23 20:04:08 +02:00
Hans-Christoph Steiner
988ac21e7f buildserver: make --clean destroy reliably 
This prevents v.destroy() from running if Vagrantfile.yaml does not exist,
since that is required for vagrant to run: is the core config including the
name of the box, etc.  Otherwise, it would exit with an error.

This also does complete cleanup when using libvirt.
 2017-05-23 20:04:08 +02:00
Hans-Christoph Steiner
299ed82a88 buildserver: consolidate boxfile export code into one block 
This is just for clarity, and moving more code into the main() function.
 2017-05-23 20:04:08 +02:00
Hans-Christoph Steiner
daade7656a buildserver: add copy_caches_from_host config option 
For people using slow, expensive, and/or flaky internet, liberal use of
caching can make a huge difference.  The restricted environment of the
gpjenkins box has been a good test environment for this (Tor-only,
whitelist of allowed IPs to visit, home internet connection).
 2017-05-23 20:04:08 +02:00
Hans-Christoph Steiner
4cde71552f buildserver: run_via_vagrant_ssh() to run cmds via python-vagrant 
This moves the last vagrant call in a subprocess.
 2017-05-23 20:04:08 +02:00
Hans-Christoph Steiner
6464ec55b7 buildserver: move code into main() method to always stop thread 
By running the whole program in a main() function, it can be wrapped in
try/finally in order to stop the background display thread.  This is also
done in ./fdroid, its standard practice for Python CLI utilities.
 2017-05-23 20:04:08 +02:00
Hans-Christoph Steiner
675500ad88 buildserver: display verbose logging in a background tail 2017-05-23 20:04:08 +02:00
Hans-Christoph Steiner
4b03c3d42d buildserver: replace custom code with python-vagrant 
I ran into some annoying issues with UTF-8 output in the vagrant logs, and
it was hard to solve.  So I switched to using python-vagrant, which handles
it all for us.  Its been around since 2012, has a number of contributors,
and is still actively maintained, so it seems like a good bet. I also
packaged it for Debian, including a backport in jessie-backports.

On Debian/jessie, do `apt-get install python3-vagrant/jessie-backports`
 2017-05-23 20:04:08 +02:00
Andrew Patrikalakis
9ee96117d8 Check for libvirt specific shutoff message 2017-04-29 18:38:44 -07:00
Boris Kraut
10e91d1be2 makebs: add gradle 3.5 2017-04-15 15:08:13 +02:00
Jan Christian Grünhage
f637b461f0 Update support repository. 2017-03-29 10:33:38 +00:00
Boris Kraut
53be33055b makebs: update support repo to r46 2017-03-21 21:50:29 +01:00
Boris Kraut
ed310f0404 makebs: add ndk r14 2017-03-19 22:27:51 +01:00
Hans-Christoph Steiner
1e9406778f buildserver: remove my silly mistake 
Should have done this through a merge request... code review!
 2017-03-14 20:14:06 +01:00
Hans-Christoph Steiner
ee82c30cb8 buildserver: bump to android_m2repository r45 2017-03-14 20:04:50 +01:00
Boris Kraut
e8d0a8ede8 makebs: add gradle 3.4.1 2017-03-10 14:29:36 +01:00
Boris Kraut
add1ad6eb5 makebs: add gradle 3.4 2017-02-26 03:23:49 +01:00
Boris Kraut
687061f47e makebs: update support repo to r43, closes fdroid/rfp#49 2017-02-23 01:27:10 +01:00
Hans-Christoph Steiner
1d91a400a2 buildserver: update to latest tools and m2repository 2017-02-13 20:10:48 +01:00
Boris Kraut
e382f09fd6 makebs: bump support repo to r43 2017-02-11 14:40:40 +01:00
Boris Kraut
f32f4a68d1 makebs: upgrade support repo to r42 2017-02-04 15:01:15 +01:00
Hans-Christoph Steiner
4741f75f03 buildserver: update android_m2repository to latest (r41) 2017-01-12 10:19:38 +01:00
Boris Kraut
d63a1fde19 makebs: install gradle 3.3 2017-01-08 21:13:24 +01:00
est31
d6da0a283d Makebs: add build tools 25.0.2 2016-12-17 23:36:48 +01:00
est31
d39f54f04d Add gradle 3.2.1 2016-12-02 08:16:04 +01:00
Boris Kraut
51083fdeb4 makebs: add buildtools 25.0.1 2016-12-01 08:44:47 +01:00
est31
ddefec33cd Add ndk 13b 2016-11-25 21:18:37 +01:00
Daniel Martí
8cdf1b0223 makebs: add gradle 3.2 2016-11-15 21:07:35 +00:00
Daniel Martí
483338843f makebs: update to support repo r40 2016-11-15 20:56:04 +00:00
Daniel Martí
82b1d7ad14 all: make newer pycodestyle happy 
Apparently the "two empty lines" rule is now stricter.
 2016-11-15 20:55:06 +00:00
Daniel Martí
98b7538789 makebs: bump m2repo, add api25 and build-tools 25 2016-10-23 16:47:19 +01:00
Ciaran Gultnieks
aa4aa45703 Merge branch 'build-tools' into 'master' 
makebs: add build-tools 24.0.3



See merge request !175
 2016-10-05 18:09:46 +00:00
Daniel Martí
b34cef8776 makebs: add build-tools 24.0.3 2016-10-03 09:10:41 +01:00
Hans-Christoph Steiner
1b09ab5a79 buildserver: run dir check as very first thing 
No point in running any other code if the script is just going to bail
out with an error.  This assumes that ./makebuildserver is only ever
run from a git clone of fdroidserver.git.
 2016-09-28 09:52:49 +02:00
Hans-Christoph Steiner
a3d32c65a4 buildserver: use py3's pathlib to generate cache file URL 
This should handle edge cases better, like odd characters in the path, etc.
 2016-09-28 09:52:49 +02:00
est31
91c5fb567a Add qt sdk support 2016-09-27 14:36:12 +02:00
Hans-Christoph Steiner
1001e5e9b9 buildserver: bochs can also be used with KVM 
jenkins.debian.net is being detected as 'bochs' rather than 'qemu'.
 2016-09-23 17:41:41 +02:00
Hans-Christoph Steiner
1e8fd01c1e buildserver: only check cache permissions when using libvirt 
VirtualBox runs as the same user as `fdroid`, so the cache does not need to
be accessible by the world.  On libvirt, libvirtd runs the VMs as its own
user, so in that case, the cache dirs must have permissions to let that
user access them.
 2016-09-23 15:20:05 +02:00
Hans-Christoph Steiner
f6a487eee4 buildserver: wipe snapshot from libvirt store on --clean 2016-09-23 15:06:41 +02:00
Ciaran Gultnieks
5667d16498 Merge branch 'buildserver-qemu-kvm' into 'master' 
buildserver running in qemu/kvm to support KVM on KVM

jenkins.debian.net runs in QEMU/KVM instances, so in order to run the F-Droid buildserver there, it needs to work inside of a KVM guest.  The best way I found to do that is to create QEMU/KVM instances via KVM's "nested" virtualization support.  This collection of commits enables using QEMU/KVM as the buildserver when `./makebuildserver` detects that it is running inside of KVM.  Otherwise, the old behavior is default: running in VirtualBox.

I have run these tests inside of ubuntu/16.04 on bare metal, which uses VirtualBox, and ubuntu/16.04 KVM guest, which uses QEMU/KVM.  It'll also run on the Guardian Project jenkins box, which is Debian/jessie.

@mvdan @CiaranG @krt

See merge request !168
 2016-09-23 12:25:12 +00:00
est31
56a0077183 makebs: add gradle 3.1 2016-09-23 07:50:52 +02:00
Hans-Christoph Steiner
68ec7c2e88 buildserver: auto-detect and use libvirt's QEMU+KVM 
For running in QEMU/KVM guests like on jenkins.debian.net, this sets up the
whole process automatically.  This only really covers the case where this
is running in a KVM guest, and the original case of running VirtualBox on
bare metal.  It could be extended to cover more cases if someone wanted to.
 2016-09-19 16:33:12 +02:00
Hans-Christoph Steiner
f306e32636 buildserver: make sure cachedir is accessible to VM instance 
It seems that KVM/QEMU's 9p sharing is more sensitive to file perms.  If
~/.cache is 0700, then ~/.cache/fdroidserver cannot be mounted in the guest
 2016-09-19 16:33:12 +02:00
Daniel Martí
8797cc01a8 makebs: bump support repo to 38 2016-09-15 20:07:25 +01:00
Daniel Martí
02c006a685 makebs: bump tools to 25.2.2 2016-09-15 20:07:16 +01:00
Hans-Christoph Steiner
14de399bb7 buildserver: remove Kivy, unused since 2013 and out of date 
This is the last thing using Chef, which adds a lot of time to the time it
takes to fully provision the buildserver.  This slows down development on
the things we are actually using, like running all builds on
jenkins.debian.net.

#210 #165
 2016-09-12 14:58:08 +02:00