kamp/fdroidserver
1
0
Fork 0
mirror of https://gitlab.com/fdroid/fdroidserver.git synced 2024-11-12 18:20:11 +01:00
Code Activity
6,650 Commits 1 Branch 59 Tags 42 MiB
153b5d4392
Commit Graph

6650 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Hans-Christoph Steiner
153b5d4392
stop modifying default_config when running fill_config_defaults() 
dicts and lists are passed by reference in assignments, so this needs to
copy them instead.
 2021-05-28 10:24:58 +02:00
Hans-Christoph Steiner
45bd89b3a2
build: missing NDK is now a warning not a critical error 2021-05-28 10:24:56 +02:00
Hans-Christoph Steiner
adafd4560c
provision-android-sdk: ensure ANDROID_HOME parent dir is accessible by all 
https://gitlab.com/fdroid/fdroidserver/-/merge_requests/926#note_586778883
 2021-05-28 10:24:55 +02:00
Hans-Christoph Steiner
6eb895a700
makebuilserver: only install most recent two releases 2021-05-28 10:24:53 +02:00
Hans-Christoph Steiner
22c83c9142 Merge branch 'modernize-ndk-handling' into 'master' 
buildserver: standardize SDK install location /opt/android-sdk

Closes #902

See merge request fdroid/fdroidserver!927
 2021-05-28 07:35:38 +00:00
Hans-Christoph Steiner
09fa49a7a3
make get_android_tools_versions() search ndk_paths from config 2021-05-28 09:13:38 +02:00
Hans-Christoph Steiner
7a1d236c8d
only support zipballs in NDK provisioning 
Since I discovered there is an r10e zipball, this can now get all NDKs
in zipball form.
fdroid/android-sdk-transparency-log@447fea86e7

closes #902
 2021-05-28 09:13:36 +02:00
Hans-Christoph Steiner
9f77044d0d
auto-detect NDKs installed in standard paths 
'ndk_paths' will be automatically filled out from well known sources
like $ANDROID_HOME/ndk-bundle and $ANDROID_HOME/ndk/*.  If a required
version is missing in the buildserver VM, it will be automatically
downloaded and installed into the standard $ANDROID_HOME/ndk/
directory.  Manually setting it here will override the auto-detected
values.  The keys can either be the "release" (e.g. r21e) or the
"revision" (e.g. 21.4.7075529).

https://developer.android.com/studio/projects/configure-agp-ndk#agp_version_41
* sdkmanager installs "ndk;12.3.4567890" into $ANDROID_SDK_ROOT/ndk/
* sdkmanager installs "ndk-bundle" into $ANDROID_SDK_ROOT/ndk-bundle/
 2021-05-28 09:13:34 +02:00
Hans-Christoph Steiner
4686c06f62 metadata: allow ndk: to be str or list of release or revision 
There are two version numbers used for NDKs: the "release" and the
"revision".  The "release" is used in the download URL and zipball and the
"revision" is used in the source.properties and the gradle ndkVersion field.

Also, there are some builds which need multiple NDKs installed, so this
makes it possible to have a list of release/revision entries in build.ndk.
This does not yet add full support since _fdroidserver/build.py_ will also
need changes.
 2021-05-28 09:13:27 +02:00
Hans-Christoph Steiner
096532dddb Merge branch 'standardize-sdk-ndk-paths' into 'master' 
buildserver: standardize SDK and NDK install locations

See merge request fdroid/fdroidserver!926
 2021-05-28 06:57:44 +00:00
Hans-Christoph Steiner
93145a43fb buildserver: use standardized paths for the NDKs 
This should help with reproducibility since some tools like Python still
include the build paths in the binaries.
https://gitlab.com/fdroid/fdroidserver/-/merge_requests/919#note_578180986

The default ANDROID_SDK_ROOT base dir of /opt/android-sdk is hard-coded in
buildserver/Vagrantfile.  The $ANDROID_HOME/ndk subdir is where Android
Studio will install the NDK into versioned subdirs.
https://developer.android.com/studio/projects/configure-agp-ndk#agp_version_41
 2021-05-26 09:21:52 +02:00
Hans-Christoph Steiner
bb77d7a6d2 buildserver: standardize SDK install location /opt/android-sdk 
_/opt/android-sdk_ was chosen for a number of reasons:
* _/opt_ is [standardized][1] for packages like the Android SDK, which has its own directory layout.
* _android-sdk_ is used rather than the upstream directory name from the ZIP (e.g. _android-sdk-linux_) so that the path is the same on all platforms.
* On platforms without official _/opt_ support ([macOS][2], [Windows][3], [FreeBSD][4], etc.), it does not conflict with any existing system directory.

[1]: https://refspecs.linuxfoundation.org/FHS_3.0/fhs/ch03s13.html
[2]: https://developer.apple.com/library/archive/documentation/FileManagement/Conceptual/FileSystemProgrammingGuide/FileSystemOverview/FileSystemOverview.html#//apple_ref/doc/uid/TP40010672-CH2-SW7
[3]: https://en.wikipedia.org/wiki/Directory_structure#Windows_10
[4]: https://www.freebsd.org/cgi/man.cgi?query=hier&sektion=7&format=html
 2021-05-25 17:30:32 +02:00
Hans-Christoph Steiner
07da062a34 buildserver: fix ssh BatchMode config 2021-05-25 17:21:26 +02:00
Hans-Christoph Steiner
be9036b1f1 Merge branch 'auto-install-ndk' into 'master' 
auto-download missing NDKs if they're known and can be verified

Closes #369 and #717

See merge request fdroid/fdroidserver!919
 2021-05-25 15:12:03 +00:00
Hans-Christoph Steiner
9d44fa7919
gitlab-ci: auto-generate merge request when NDK release found 
Following the pattern of the gradle bot, this will check the transparency
log for any new NDK release.  If there are any, it will make a merge
request from @fdroid-bot.
 2021-05-25 17:06:30 +02:00
Hans-Christoph Steiner
ec2cace222
buildserver: trim pre-installed NDK list down to the bare minimum 
This keeps the Long Term Support release and the latest release installed.
r10e was kept in because it needs a special extraction method, since it is
a .bin file, not a .zip.  r12b is kept in because it is the old default.
Here is a survey of the NDK versions used in the most recent Builds entry
in each app that uses the NDK:

{'r10e': 6,
 'r12b': 93,
 'r13b': 4,
 'r14b': 5,
 'r15c': 7,
 'r16b': 14,
 'r17b': 4,
 'r17c': 7,
 'r18b': 9,
 'r19c': 17,
 'r20': 1,
 'r20b': 22,
 'r21': 3,
 'r21d': 56,
 'r21e': 65,
 'r22': 9,
 'r22b': 15,
 'r9b': 1}

#517


import glob
import os
import yaml

try:
    from yaml import CSafeLoader as SafeLoader
except ImportError:
    from yaml import SafeLoader

ndks = dict()
for f in glob.glob('metadata/*.yml'):
    with open(f) as fp:
        app = yaml.load(fp, Loader=SafeLoader)
        if app.get('Disable'):
            continue
        build = app.get('Builds', [])[-1]
        if build.get('disabled'):
            continue
        ndk = build.get('ndk')
        if ndk and ndk[1] == '9':
            print(f, build)
        elif ndk and int(ndk[2:3]) < 18:
            print(f, build)
        if ndk:
            print(f, ndk)
            if ndk not in ndks:
                ndks[ndk] = 0
            ndks[ndk] += 1

import pprint

pprint.pprint(ndks)
 2021-05-25 17:06:28 +02:00
Hans-Christoph Steiner
9fc2a23713
build: remove default NDK, closes #717 2021-05-25 17:06:26 +02:00
Hans-Christoph Steiner
69fcd6a024
build: auto-download missing NDKS if they're known and can be verified 
refs #517 #717
 2021-05-25 17:06:24 +02:00
Hans-Christoph Steiner
7a7ecbf9dc
move sha256sum() and sha256base64() to common 2021-05-25 17:06:23 +02:00
Hans-Christoph Steiner
0fbd04f1c2
remove redundant imports and pointless code 2021-05-25 17:06:19 +02:00
Hans-Christoph Steiner
95af082082 Merge branch 'gradle-release-checksums.py' into 'master' 
update to gradle v7.0.2

See merge request fdroid/fdroidserver!924
 2021-05-17 17:11:02 +00:00
fdroid-bot
927aa95cd8 gradle v7.0.2 2021-05-17 16:59:55 +00:00
Hans-Christoph Steiner
e4a7726033 gitlab-ci: fix archlinux docker image 2021-05-13 17:20:41 +02:00
Hans-Christoph Steiner
72cef85201 Merge branch 'fix_autoname' into 'master' 
Don't return Gradle variables as string

See merge request fdroid/fdroidserver!909
 2021-05-13 13:05:17 +00:00
Jochen Sprickerhof
84b6617faf Don't return Gradle variables as string 
This is used in checkupdates and currently sets the AutoName to
something like ${displayName}:

4ae415f73c
 2021-05-13 12:53:19 +00:00
Hans-Christoph Steiner
a1dfd69392 Merge branch 'disabled_needsupdated' into 'master' 
[status_json] Don't list disabled builds as needs updated

Closes #898

See merge request fdroid/fdroidserver!917
 2021-05-13 12:42:46 +00:00
Jochen Sprickerhof
31c0078b53 [status_json] Add list of archivePolicy0 2021-05-13 12:30:53 +00:00
Jochen Sprickerhof
33fa918a69 [status_json] Don't list disabled builds as needs updated 
Closes: #898
 2021-05-13 12:30:53 +00:00
Hans-Christoph Steiner
1bad5b5c6c Merge branch 'gradle-release-checksums.py' into 'master' 
update to gradle v7.0.1

See merge request fdroid/fdroidserver!918
 2021-05-13 12:13:33 +00:00
fdroid-bot
e0f825d636 gradle v7.0.1 2021-05-13 12:02:17 +00:00
Hans-Christoph Steiner
01947e0c30 Merge branch 'disabled_logs' into 'master' 
Don't remove logs of disabled builds

See merge request fdroid/fdroidserver!907
 2021-05-13 12:01:34 +00:00
Jochen Sprickerhof
22c32752de Don't remove logs of disabled builds 
We need them to debug problems.
 2021-05-13 11:49:55 +00:00
Hans-Christoph Steiner
58ff81e7b5 Merge branch 'sphinx' into 'master' 
set up docs based on docstrings and publish HTML versions

See merge request fdroid/fdroidserver!911
 2021-05-10 14:34:59 +00:00
Benedikt Brückmann
6791c3a550 Only publish pages on default branch 2021-05-10 14:31:51 +00:00
Benedikt Brückmann
245aeec972 Publish docu to GL Pages 2021-05-10 14:31:51 +00:00
Benedikt Brückmann
bdcd843d75 Publish docu to GL Pages 2021-05-10 14:31:51 +00:00
Benedikt Brückmann
47cd1dd055 Add missing index.rst 2021-05-10 14:31:51 +00:00
Benedikt Brückmann
a15581faa9 Draft: set up docs based on docstrings and publish HTML versions 2021-05-10 14:31:51 +00:00
Hans-Christoph Steiner
022d0796d8 gitlab-ci: use latest alpine/bandit to fix CVE flagged by safety 2021-05-10 16:05:59 +02:00
fdroid-bot
2b21584b35 gradle v7.0 2021-05-10 06:48:16 +00:00
Jochen Sprickerhof
3c7a135c20 Merge branch 'fix-year-in-changelog' into 'master' 
Fix: Changelog.md dates incorrect year

Closes #896

See merge request fdroid/fdroidserver!910
 2021-05-10 06:47:32 +00:00
Benedikt Brückmann
b220ddc555 Fix: Changelog.md dates incorrect year 
closes fdroid/fdroidserver#896
 2021-05-03 09:59:29 +02:00
Jochen Sprickerhof
a99bebbb5e Merge branch 'keep-failed-build-in-test-mode' into 'master' 
`fdroid build --test` keeps unsigned APKs in tmp/ that fail to reproduce

See merge request fdroid/fdroidserver!904
 2021-04-16 08:18:47 +00:00
Hans-Christoph Steiner
0b0bc803b7 build: --test now keeps unsigned APKs in tmp/ that fail to reproduce 
Before, whenever an unsigned APK failed to reproduce, it was just deleted.
That makes debugging hard.  This makes it keep the unsigned APK, which is
written in tmp/ when using --test.

@jspricke this is related to !864
 2021-04-16 09:46:23 +02:00
Hans-Christoph Steiner
6097caef91
build: fix typo in regex for Binaries: verification 2021-04-16 09:40:21 +02:00
Hans-Christoph Steiner
3d69e767d8
common: test abs and rel paths in get_all_gradle_and_manifests() 2021-04-16 09:40:19 +02:00
Hans-Christoph Steiner
634864f206
gitlab-ci: document jobs 2021-04-16 09:40:12 +02:00
Hans-Christoph Steiner
184d2f97d2
jenkins-build-all: stop deleting previous builds results 2021-04-15 16:52:54 +02:00
Hans-Christoph Steiner
760fdf3bd2
jenkins-build-all: fix failure when trying to rm a dir 
+ for f in metadata/*/signatures/*
++++ dirname metadata/de.schildbach.wallet/signatures/370
+++ dirname metadata/de.schildbach.wallet/signatures
++ basename metadata/de.schildbach.wallet
Wed 14 Apr 2021 11:50:34 PM UTC - cleanup in progress...
Wed 14 Apr 2021 11:50:34 PM UTC - cleanup done.
+ appid=de.schildbach.wallet
++ basename metadata/de.schildbach.wallet/signatures/370
+ versionCode=370
+ rm -f repo/de.schildbach.wallet_427.apk repo/de.schildbach.wallet_427.apk.asc repo/de.schildbach.wallet_427.log.gz repo/de.schildbach.wallet_test repo/de.schildbach.wallet_test_427.apk repo/de.schildbach.wallet_test_427.apk.asc repo/de.schildbach.wallet_test_427.log.gz 'archive/de.schildbach.wallet_*' 'unsigned/de.schildbach.wallet_*'
rm: cannot remove 'repo/de.schildbach.wallet_test': Is a directory
 2021-04-15 09:22:11 +02:00
Jochen Sprickerhof
f149be7296 Merge branch 'bump_metadata' into 'master' 
Bump METADATA_VERSION for apksigner transition

See merge request fdroid/fdroidserver!905
 2021-04-14 22:21:16 +00:00