1
0
mirror of https://gitlab.com/fdroid/fdroidserver.git synced 2024-08-16 03:10:09 +02:00
Commit Graph

3890 Commits

Author SHA1 Message Date
Hans-Christoph Steiner
a2a630c83c buildserver: only include latest m2 when provisioning
I think the `android update sdk` tool is installing all of the m2 files
that are present in the temp cache, and it seems to do it in order of
newest to oldest.  Well done, and I thought that tool couldn't get any
worse.  So only include the latest version of android_m2repository*.zip in
the temp cache.
2016-09-19 16:33:12 +02:00
Hans-Christoph Steiner
0a0ba6de84 buildserver: ensure dirs to mount cache exist in guest VM instance
It seems that the 9p synced folder setup is not as flexible and easy as the
VirtualBox one, so we have to do more little things like this.
2016-09-19 16:33:12 +02:00
Hans-Christoph Steiner
988647a387 buildserver: fix jenkins build to support QEMU/KVM 2016-09-19 16:33:12 +02:00
Hans-Christoph Steiner
3651e7eb9a buildserver: retry apt-get downloads 20 times
Try harder before failing the whole buildserver setup.
2016-09-19 16:33:12 +02:00
Hans-Christoph Steiner
68ec7c2e88 buildserver: auto-detect and use libvirt's QEMU+KVM
For running in QEMU/KVM guests like on jenkins.debian.net, this sets up the
whole process automatically.  This only really covers the case where this
is running in a KVM guest, and the original case of running VirtualBox on
bare metal.  It could be extended to cover more cases if someone wanted to.
2016-09-19 16:33:12 +02:00
Hans-Christoph Steiner
f306e32636 buildserver: make sure cachedir is accessible to VM instance
It seems that KVM/QEMU's 9p sharing is more sensitive to file perms.  If
~/.cache is 0700, then ~/.cache/fdroidserver cannot be mounted in the guest
2016-09-19 16:33:12 +02:00
Hans-Christoph Steiner
0ae2df6646 buildserver: support optionally using qemu+kvm
This makes it possible to run the full build process in a KVM virtual
machine, like jenkins.debian.net.

closes #190
2016-09-19 16:33:12 +02:00
Hans-Christoph Steiner
5af53238d3 buildserver: ignore android sdk repos of proprietary bits
The Android SDK by default includes some Google repositories of their
proprietary SDKs.  We of course do not want that stuff ever.  We also do
not need the emulator images since this process does not currently install
or run an emulator.
2016-09-19 16:33:12 +02:00
Hans-Christoph Steiner
b5e57a7aa6 example file for public read only S3 bucket policy
This is the locked down S3 policy as created by Benetech for their Secure
App Generator project.
2016-09-15 23:20:18 +02:00
Hans-Christoph Steiner
008b4a31eb skip fdroid import test if gitlab is not available
This prevents CI build failures when gitlab is deploying or has other
issues that might cause a 500.
2016-09-15 23:20:18 +02:00
Daniel Martí
89010a1ba5 Merge branch 'gradle-wrapper' into 'master'
Make the gradle wrapper smarter

@eighthave PTAL

This will help with issues like https://gitlab.com/fdroid/fdroiddata/issues/515

See merge request !166
2016-09-15 20:45:44 +00:00
Hans-Christoph Steiner
8fc27680cb Merge branch 'makebs-updates' into 'master'
Makebs updates



See merge request !167
2016-09-15 20:38:43 +00:00
Daniel Martí
8797cc01a8 makebs: bump support repo to 38 2016-09-15 20:07:25 +01:00
Daniel Martí
02c006a685 makebs: bump tools to 25.2.2 2016-09-15 20:07:16 +01:00
Daniel Martí
737dbc4f0a gradle: support gradle-wrapper.properties version
Brings our wrapper closer to gradlew's functionality.

Updates #98.
2016-09-15 19:16:38 +01:00
Daniel Martí
12653741cc gradle: delay calculating some versions
Simplifies the wrapper script.
2016-09-15 19:10:18 +01:00
Daniel Martí
071376b7f4 Merge branch 'control-errors-warnings' into 'master'
make metadata exceptions optional based on CLI flag

In many cases, there are times where metadata errors need to be ignored, or
at least not stop the command from running.  For example, there will
inevitably be new metadata fields added, in which case a packaged version
of fdroidserver will throw errors on each one.  This adds a standard -W
flag to customize the response: ignore, default, or error.

* by default, the errors are still errors
* `fdroid readmeta -W` will just print errors
* `fdroid readmeta -Wignore` will not even print errors

https://gitlab.com/fdroid/fdroidserver/issues/150

See merge request !164
2016-09-12 14:16:42 +00:00
Daniel Martí
352c1ff448 Merge branch 'buildserver-fixes-pre-kvm' into 'master'
buildserver fixes pre-KVM

This is a collection of little fixes to the buildserver setup that I did while implementing the QEMU+KVM support.  Since I was running the whole process repeatedly, I had opportunity to find and fix little things like this.

See merge request !165
2016-09-12 14:16:15 +00:00
Hans-Christoph Steiner
14de399bb7 buildserver: remove Kivy, unused since 2013 and out of date
This is the last thing using Chef, which adds a lot of time to the time it
takes to fully provision the buildserver.  This slows down development on
the things we are actually using, like running all builds on
jenkins.debian.net.

#210 #165
2016-09-12 14:58:08 +02:00
Hans-Christoph Steiner
91c1e8313f buildserver: download apt package first to increase reliability
This does not have the careful result rechecking that chef has, when it
installs each package in the list one at a time.  So to help with failures
caused by a package failing to download, first try downloading all the
package, then run the install.  The install pass will try to download any
missing packages.

Really, this should use ansible or perhaps chef again since those include
lots of tricks around this stuff.
2016-09-12 14:58:08 +02:00
Hans-Christoph Steiner
7b64bdcf0b buildserver: only specify lib*-dev to future proof package list
Using libssl-dev will work on all releases of Debian, but Debian/stretch
does not have libssl1.0.0.
2016-09-12 14:58:08 +02:00
Hans-Christoph Steiner
73947092fa buildserver: test build of Checkey and old F-Droid
The buildserver should still be able to build old versions of apps, and
Checkey is a verified, reproducible build.
2016-09-12 14:58:08 +02:00
Hans-Christoph Steiner
b717271c61 buildserver: force a known-good version of chef
This forces the release channel and version of chef-solo to install on the
guest VM.  I was getting really massive, odd stacktraces without specifying
this, and chef is only used for Kivy now anyway.
2016-09-12 14:58:08 +02:00
Hans-Christoph Steiner
dc2f53b48c buildserver: vagrant-cachier conflicts with custom apt cache
The technique where /var/cache/apt is mounted as a shared folder conflicts
with vagrant-cachier's workings.  Therefore, ignore vagrant-cachier if the
user selects ./makebuildserver's custom apt cache.  The shared folder way
has the advantage for CI builds of storing the cache outside of
VAGRANT_HOME, which is set to be in the git project.  That gets wiped by
`git clean -fdx` on each CI build.
2016-09-12 14:58:08 +02:00
Hans-Christoph Steiner
d522988d8c make metadata exceptions optional based on CLI flag
In many cases, there are times where metadata errors need to be ignored, or
at least not stop the command from running.  For example, there will
inevitably be new metadata fields added, in which case a packaged version
of fdroidserver will throw errors on each one.  This adds a standard -W
flag to customize the response: ignore, default, or error.

* by default, the errors are still errors
* `fdroid readmeta -W` will just print errors
* `fdroid readmeta -Wignore` will not even print errors

https://gitlab.com/fdroid/fdroidserver/issues/150
2016-09-12 12:55:48 +02:00
Daniel Martí
34f451361f Bump to 0.7.0 2016-09-11 11:52:11 +02:00
Daniel Martí
9f88bc6324 Merge branch 'updates' into 'master'
all: bump to build-tools 24.0.2

Update the docker image to include it there too.

See merge request !163
2016-09-10 10:06:44 +00:00
Daniel Martí
fd7a8c83c8 all: bump to build-tools 24.0.2
Update the docker image to include it there too.
2016-09-10 11:45:39 +02:00
Hans-Christoph Steiner
12d0fbf8c5 Merge branch 'buildserver-fixes' into 'master'
buildserver fixes

updates to the Android SDK packages used by `./makebuildserver`

See merge request !162
2016-09-05 08:48:49 +00:00
Hans-Christoph Steiner
91ceb5aff3 update to platform-24_r02 in ./makebuildserver 2016-08-30 19:48:07 +02:00
Hans-Christoph Steiner
d831ccab20 add build-tools_r24.0.2-linux.zip to ./makebuildserver 2016-08-30 19:17:30 +02:00
Hans-Christoph Steiner
bf1dff7986 new build-tools_r24.0.1.zip sha256, Google keeps changing it
I validated this versus the Google repository XML downloaded from two
separate paths from:
https://dl.google.com/android/repository/repository-12.xml

Using their SHA1:
84f18c392919a074fcbb9b1d967984e6b2fef8b4
2016-08-30 19:16:31 +02:00
Hans-Christoph Steiner
0bcd4f958b Merge branch 'patch-1' into 'master'
Add buildserver lxml requirement for pEp



See merge request !161
2016-08-30 16:09:10 +00:00
huss
261fde8ac7 Add buildserver lxml requirement for pEp 2016-08-28 12:47:12 +00:00
Daniel Martí
b92ee2b5c6 Merge branch 'fix-push-requests' into 'master'
Fix push requests

Two bug fixes found in the process of implementing the client side of the push requests in https://gitlab.com/fdroid/fdroidclient/merge_requests/386

This is related to !156.

See merge request !160
2016-08-26 22:26:17 +00:00
Hans-Christoph Steiner
c98ce54991 make build reproducible by fixing sort order in docs
From Debian https://bugs.debian.org/835463
 Description: Make the build reproducible
 Author: Chris Lamb <lamby@debian.org>
 Last-Update: 2016-08-26
2016-08-26 13:41:14 +02:00
Hans-Christoph Steiner
6126b55136 rename server request from "delete" to "uninstall"
This matches the Android API's current Intent action for this, rather than
the deprecated one:

https://gitlab.com/fdroid/fdroidclient/blob/v0.101-alpha5/app/src/main/java/org/fdroid/fdroid/installer/DefaultInstallerActivity.java#L147
https://developer.android.com/reference/android/content/Intent.html#ACTION_UNINSTALL_PACKAGE
https://developer.android.com/reference/android/content/Intent.html#ACTION_DELETE
2016-08-24 23:05:34 +02:00
Hans-Christoph Steiner
a6a8d34528 sanitize mirror URLs to prevent path segments from being removed
urllib.parse.urljoin() will strip off the last path segment before joining
if that last path segment does not end with /.  That's a "feature" to make
it easy to replace file names.  Here it was stripping off the essential
'fdroid' segment, making URLs like:

https://foo.com/repo

when they should be

https://foo.com/fdroid/repo
2016-08-24 23:05:34 +02:00
Daniel Martí
1c49c3af1d Merge branch 'ci-bump' into 'master'
CI: bump docker image



See merge request !159
2016-08-22 09:25:19 +00:00
Daniel Martí
a006f277bc CI: bump docker image 2016-08-21 11:11:01 +02:00
Daniel Martí
baea79f739 makebs: add gradle 3.0 2016-08-18 17:53:27 +02:00
Daniel Martí
8bcaed37a1 makebs: bump m2repository to r36 (24.2.0) 2016-08-18 17:48:01 +02:00
Daniel Martí
8d6375b1bb makebs: bump tools to r25.2.1 2016-08-18 17:47:49 +02:00
Daniel Martí
b5fcea73c5 gradle: plugins 2.2 and 2.1.3 require 2.14.1 2016-08-18 17:38:36 +02:00
Hans-Christoph Steiner
62c59c8534 Merge branch 'push-install' into 'master'
set up install/delete lists for "push" commands from server

It is now possible for the server operator to specify lists of apps that must be installed or deleted on the client (aka "push installs).  If
the user has opted in, or the device is already setup to respond to these requests, then fdroidclient will automatically install/delete
the packageNames listed.  This is protected by the same signing key as the app index metadata.

It generates single XML elements with the data set in the attributes. This keeps the XML compact and easily extensible, e.g. for adding versionCode, signingKey, etc as attributes:

```xml
    <install packageName="com.fsck.k9"/>
    <install packageName="at.bitfire.davdroid"/>
    <delete packageName="com.facebook.orca"/>
```

See merge request !156
2016-08-17 09:18:31 +00:00
Hans-Christoph Steiner
85632ba00e set up install/delete lists for "push" commands from server
It is now possible for the server operator to specify lists of apps that
must be installed or deleted on the client (aka "push installs).  If
the user has opted in, or the device is already setup to respond to
these requests, then fdroidclient will automatically install/delete
the packageNames listed.  This is protected by the same signing key
as the app index metadata.

It generates single XML elements with the data set in the attributes. This
keeps the XML compact and easily extensible, e.g. for adding versionCode,
signingKey, etc as attributes:

    <install packageName="com.fsck.k9"/>
    <install packageName="at.bitfire.davdroid"/>
    <delete packageName="com.facebook.orca"/>

Copyright: 2016 Blue Jay Wireless
Signed-off-by: Hans-Christoph Steiner <hans@eds.org>

closes #177
2016-08-16 21:02:15 +02:00
Daniel Martí
329e0247d5 Merge branch 'ndk11' into 'master'
Add ndk r11c

NDK11 is required by some apps (some versions of VLC) which will not build with r10e or r12b. As always, please test this before merging: I havent added a NDK before.

See merge request !155
2016-08-14 09:15:04 +00:00
Daniel Martí
76ecacc693 makebs: add automake
Version 1.14 is needed for k9 and some other apps.
2016-08-14 09:57:29 +02:00
Boris Kraut
815961acc5 Add ndk r11c 2016-08-14 00:59:31 +02:00
Daniel Martí
ec685342a4 Merge branch 'gradle-2.14.1' into 'master'
Add gradle 2.14.1

I hope verything is doen correctly, didnt add any gradle versions since moving from chef.. (or at all?)

See merge request !154
2016-08-12 20:36:35 +00:00