kamp/fdroidserver
1
0
Fork 0
mirror of https://gitlab.com/fdroid/fdroidserver.git synced 2024-11-05 06:50:10 +01:00
Code Activity
6,715 Commits 1 Branch 59 Tags 40 MiB
f44bd8d928
Commit Graph

407 Commits

Author SHA1 Message Date
Hans-Christoph Steiner
28a4f9481a
provision-android-ndk: do all configuration via args 
This makes it so it is easy to reuse this script in other contexts, like
in Docker, GitLab CI, etc.
 2021-06-01 14:33:41 +02:00
Hans-Christoph Steiner
adafd4560c
provision-android-sdk: ensure ANDROID_HOME parent dir is accessible by all 
https://gitlab.com/fdroid/fdroidserver/-/merge_requests/926#note_586778883
 2021-05-28 10:24:55 +02:00
Hans-Christoph Steiner
7a1d236c8d
only support zipballs in NDK provisioning 
Since I discovered there is an r10e zipball, this can now get all NDKs
in zipball form.
fdroid/android-sdk-transparency-log@447fea86e7

closes #902
 2021-05-28 09:13:36 +02:00
Hans-Christoph Steiner
9f77044d0d
auto-detect NDKs installed in standard paths 
'ndk_paths' will be automatically filled out from well known sources
like $ANDROID_HOME/ndk-bundle and $ANDROID_HOME/ndk/*.  If a required
version is missing in the buildserver VM, it will be automatically
downloaded and installed into the standard $ANDROID_HOME/ndk/
directory.  Manually setting it here will override the auto-detected
values.  The keys can either be the "release" (e.g. r21e) or the
"revision" (e.g. 21.4.7075529).

https://developer.android.com/studio/projects/configure-agp-ndk#agp_version_41
* sdkmanager installs "ndk;12.3.4567890" into $ANDROID_SDK_ROOT/ndk/
* sdkmanager installs "ndk-bundle" into $ANDROID_SDK_ROOT/ndk-bundle/
 2021-05-28 09:13:34 +02:00
Hans-Christoph Steiner
93145a43fb buildserver: use standardized paths for the NDKs 
This should help with reproducibility since some tools like Python still
include the build paths in the binaries.
https://gitlab.com/fdroid/fdroidserver/-/merge_requests/919#note_578180986

The default ANDROID_SDK_ROOT base dir of /opt/android-sdk is hard-coded in
buildserver/Vagrantfile.  The $ANDROID_HOME/ndk subdir is where Android
Studio will install the NDK into versioned subdirs.
https://developer.android.com/studio/projects/configure-agp-ndk#agp_version_41
 2021-05-26 09:21:52 +02:00
Hans-Christoph Steiner
bb77d7a6d2 buildserver: standardize SDK install location /opt/android-sdk 
_/opt/android-sdk_ was chosen for a number of reasons:
* _/opt_ is [standardized][1] for packages like the Android SDK, which has its own directory layout.
* _android-sdk_ is used rather than the upstream directory name from the ZIP (e.g. _android-sdk-linux_) so that the path is the same on all platforms.
* On platforms without official _/opt_ support ([macOS][2], [Windows][3], [FreeBSD][4], etc.), it does not conflict with any existing system directory.

[1]: https://refspecs.linuxfoundation.org/FHS_3.0/fhs/ch03s13.html
[2]: https://developer.apple.com/library/archive/documentation/FileManagement/Conceptual/FileSystemProgrammingGuide/FileSystemOverview/FileSystemOverview.html#//apple_ref/doc/uid/TP40010672-CH2-SW7
[3]: https://en.wikipedia.org/wiki/Directory_structure#Windows_10
[4]: https://www.freebsd.org/cgi/man.cgi?query=hier&sektion=7&format=html
 2021-05-25 17:30:32 +02:00
Hans-Christoph Steiner
07da062a34 buildserver: fix ssh BatchMode config 2021-05-25 17:21:26 +02:00
Hans-Christoph Steiner
ec2cace222
buildserver: trim pre-installed NDK list down to the bare minimum 
This keeps the Long Term Support release and the latest release installed.
r10e was kept in because it needs a special extraction method, since it is
a .bin file, not a .zip.  r12b is kept in because it is the old default.
Here is a survey of the NDK versions used in the most recent Builds entry
in each app that uses the NDK:

{'r10e': 6,
 'r12b': 93,
 'r13b': 4,
 'r14b': 5,
 'r15c': 7,
 'r16b': 14,
 'r17b': 4,
 'r17c': 7,
 'r18b': 9,
 'r19c': 17,
 'r20': 1,
 'r20b': 22,
 'r21': 3,
 'r21d': 56,
 'r21e': 65,
 'r22': 9,
 'r22b': 15,
 'r9b': 1}

#517


import glob
import os
import yaml

try:
    from yaml import CSafeLoader as SafeLoader
except ImportError:
    from yaml import SafeLoader

ndks = dict()
for f in glob.glob('metadata/*.yml'):
    with open(f) as fp:
        app = yaml.load(fp, Loader=SafeLoader)
        if app.get('Disable'):
            continue
        build = app.get('Builds', [])[-1]
        if build.get('disabled'):
            continue
        ndk = build.get('ndk')
        if ndk and ndk[1] == '9':
            print(f, build)
        elif ndk and int(ndk[2:3]) < 18:
            print(f, build)
        if ndk:
            print(f, ndk)
            if ndk not in ndks:
                ndks[ndk] = 0
            ndks[ndk] += 1

import pprint

pprint.pprint(ndks)
 2021-05-25 17:06:28 +02:00
relan
6d9410597b Upgrade NDK r22 to r22b 2021-04-07 12:12:04 +03:00
Hans-Christoph Steiner
ee84eb762c buildserver: SharedFoldersEnableSymlinksCreate: false 2021-03-23 11:01:40 +01:00
Hans-Christoph Steiner
4b05854ffc buildserver: only install the latest gradle version 
gradle checksums are now handled by gradlew-fdroid

refs #370 #517 #737
 2021-03-11 08:40:34 +01:00
Licaon_Kter
d683a40f96 Add NDK 21e 2021-03-10 16:36:14 +02:00
Licaon_Kter
5396b79da7 Fix after https://gitlab.com/fdroid/fdroidserver/-/merge_requests/853 2021-02-12 09:14:27 +00:00
TacoTheDank
c88a3f56a1 Update NDK to r22 and add buildtools 30.0.3 2020-12-26 19:51:22 -05:00
Hans-Christoph Steiner
e927ed02a8
provision-apt-get-install: fix gpg from stretch-backports-sloppy 
!822

The gpg dependencies are complicated.  The previous setup was always
failing with:
The following information may help to resolve the situation:
The following packages have unmet dependencies:
 gpg : Depends: gpgconf (= 2.2.20-1~bpo9+1)
       Depends: libassuan0 (>= 2.5.0) but 2.4.3-2 is to be installed
       Depends: libgpg-error0 (>= 1.35) but 1.26-2 is to be installed

Thanks to @izzysoft to spotting this.
 2020-11-09 14:37:55 +01:00
Michael Pöhn
196dc75481 Merge branch 'yaml-config' into 'master' 
move config.py to config.yml

See merge request fdroid/fdroidserver!571
 2020-11-05 23:00:32 +00:00
Michael Pöhn
51a099cb3d Merge branch 'gpgaptfix' into 'master' 
Use newer gpg from sloppy

See merge request fdroid/fdroidserver!822
 2020-11-04 15:22:39 +00:00
licaon-kter
2fc0178972 Use newer gpg from sloppy 2020-11-01 14:02:48 +02:00
Hans-Christoph Steiner
a9fdb5b401 init: switch to config.yml as the default format 2020-10-23 14:20:49 +02:00
Hans-Christoph Steiner
fd41b70e27 purge common.use_androguard, it is now used by default 
Up until now, the buildserver has not included androguard.  Since a
good version of androguard (v3.3.3+) is included in stretch-backports
and the buildserver is already setup to use stretch-backports, this
sets up the buildserver with androguard.

closes #627
 2020-10-22 11:58:55 +02:00
Marcus Hoffmann
ba28d631b7 add now required packages to buildserver installation 
Dependencies introduced in ffdroid/fdroidserver!798
 2020-09-27 12:47:03 +02:00
Hans-Christoph Steiner
e659a5353d
provision-apt-get-install: gpg needs to come from stretch-backports 
https://gitlab.com/eighthave/fdroiddata/-/jobs/611438997
Otherwise it fails to install:

```
  gpg : Depends: gpgconf (= 2.2.20-1~bpo9+1)
        Depends: libassuan0 (>= 2.5.0) but 2.4.3-2 is to be installed
        Depends: libgpg-error0 (>= 1.35) but 1.26-2 is to be installed
```

* 7e8c244473
* !769
 2020-06-25 18:08:31 +02:00
Hans-Christoph Steiner
7e8c244473 provision-apt-get-install: ensure gpg is installed 
The basebox installs gpg, but when this script is used to provision
a GitLab CI Runner, gpg was missing.  It is used in some builds.
 2020-06-24 21:43:58 +02:00
TacoTheDank
0216513da6 Upgrade NDK r21c to r21d 2020-06-10 15:55:53 -04:00
Hans-Christoph Steiner
b1eb086259 Merge branch 'patch-2' into 'master' 
Add NDK r21c

See merge request fdroid/fdroidserver!755
 2020-06-02 19:59:38 +00:00
TacoTheDank
edd41b3c58 Upgrade NDK r21b to r21c 2020-05-29 16:23:20 -04:00
Hans-Christoph Steiner
1ed4ed61c7 buildserver: explicitly include sudo as a dependency 
The basebox currently provides sudo, but that may not always be the case.
This makes the sudo dependency explicit, so that this provisioning script
can also be used in other settings, like GitLab CI.
 2020-05-27 22:09:52 +02:00
relan
13ac46af07 makebuildserver: upgrade NDK r21 to r21b 2020-05-14 08:43:33 +03:00
Licaon_Kter
9e8aef6771 Add stretch-backports-sloppy repo too 2020-03-30 07:10:51 -04:00
relan
1b001cf1b5 provision-gradle: remove fake proxy configuration 
The prohibition of HTTP causes connection errors on some builds:

    IOException: https://dl.google.com/android/repository/addons_list-3.xml
    java.net.ConnectException: Connection refused (Connection refused)
    IOException: https://dl.google.com/android/repository/addons_list-2.xml
    java.net.ConnectException: Connection refused (Connection refused)
    IOException: https://dl.google.com/android/repository/addons_list-1.xml
    java.net.ConnectException: Connection refused (Connection refused)
    Failed to download any source lists!

Partly revert a746917022.
 2020-03-15 17:41:29 +03:00
Hans-Christoph Steiner
d9722f4453 buildserver: use long timeouts for gradle downloads 
default is 30 seconds, this uses 10 minutes to avoid things like:

* What went wrong:
A problem occurred configuring root project 'org.fdroid.fdroid'.
> Could not resolve all files for configuration ':classpath'.
   > Could not download auto-value.jar (com.google.auto.value:auto-value:1.5.2)
      > Could not get resource 'https://repo.maven.apache.org/maven2/com/google/auto/value/auto-value/1.5.2/auto-value-1.5.2.jar'.
         > Read timed out

* https://stackoverflow.com/a/49646993
* https://github.com/gradle/gradle/issues/4629#issuecomment-393182135
* https://github.com/gradle/gradle/pull/3371/files
 2020-02-13 22:32:51 +01:00
Hans-Christoph Steiner
bfe8f05de6
fix syntax error from ae86dc3d38 
fdroid/fdroidserver!713

[skip ci]
 2020-02-11 12:48:42 +01:00
Hans-Christoph Steiner
a746917022 provision-gradle: disable daemon and block HTTP repos 
closes #624
#712
 2020-02-05 20:36:25 +01:00
Hans-Christoph Steiner
ae86dc3d38 buildserver: set quiet options to stop apt-get spamming build logs 
https://gitlab.com/fdroid/fdroidserver/issues/636#note_266483988
 2020-02-05 20:36:25 +01:00
Marcus Hoffmann
fdcb1ad23f Revert "Revert "Merge branch 'ndk21' into 'master'"" 
This reverts commit 999b15a909.
 2020-01-15 22:48:40 +01:00
Marcus Hoffmann
999b15a909 Revert "Merge branch 'ndk21' into 'master'" 
This reverts commit 9b4cb8b76a, reversing
changes made to 3bdbbfd45b.
 2020-01-15 22:09:41 +01:00
Licaon_Kter
3c3b3dbcf2 Add NDK21 2020-01-15 12:28:36 +00:00
Licaon_Kter
126f6bf839 Add NDK r20b 2019-12-16 00:07:03 +00:00
Marcus Hoffmann
bdf67784f5 makebuildserver: add ndk r20 2019-07-31 15:40:57 +02:00
Marcus
7272689ced Merge branch 'master' into 'master' 
Added newer ndks, gradles, latest sdk-license, and update java 1.8 version

See merge request fdroid/fdroidserver!637
 2019-04-11 11:42:18 +00:00
Michael Pöhn
67731470cc Revert "Merge branch 'write-yaml-overhaul' into 'master'" 
This reverts merge request !630
 2019-04-01 10:24:00 +00:00
Taco
457cf22361 Added newer ndks, gradles, latest sdk-license, and update java 1.8 version 2019-03-30 17:10:21 -04:00
Michael Pöhn
63afc0acb5 use pyyaml for writing metadata instead of ruamel 2019-03-18 22:45:35 +01:00
Marcus Hoffmann
daff6658c0 ndk r19b: missed a few places 2019-02-20 13:18:24 +01:00
Marcus Hoffmann
8d9f885b1c makebuildserver: add ndk r19b 2019-02-19 15:43:35 +01:00
Marcus Hoffmann
eee2cb03cf add ndk r19 2019-01-19 23:51:46 +01:00
Marcus Hoffmann
e1c547cfdf Revert "Revert "Merge branch 'random-fixes' into 'master'"" 
This reverts commit f6f2fb0b89.

Only one of the included commit should have been reverted.
 2019-01-10 14:48:29 +01:00
Marcus
f6f2fb0b89 Revert "Merge branch 'random-fixes' into 'master'" 
This reverts merge request !605
 2019-01-10 13:46:49 +00:00
Hans-Christoph Steiner
4c5864c975 buildserver: add config option to use 'nfs' instead of '9p' 
'9p' is not possible when running KVM in VMware.  Also, 'nfs' might provide
more reliably operation on systems that are properly setup for it.
 2019-01-03 21:00:01 +01:00
Hans-Christoph Steiner
d6545732e4 buildserver: disable all automatic apt-get updates 
Prevents errors due to:
dpkg: error: dpkg status database is locked by another process

closes #437
 2018-12-12 17:38:26 +01:00