allowlist and blocklist are much clearer terms with no cultural baggage.
This changes all "whitelist" references to "allowlist", and all "blacklist"
references to "blocklist".
config.yml requires ASCII or UTF-8 encoding because this code does not
auto-detect the file's encoding. That is left up to the YAML library.
YAML allows ASCII, UTF-8, UTF-16, and UTF-32 encodings. Since it is a
good idea to manage config.yml (WITHOUT PASSWORDS!) in git, it makes
sense to use a globally standard encoding.
Android Studio recommends "you use UTF-8 encoding whenever possible",
so this code assumes the files use UTF-8. UTF-8 is also the default
encoding on GNU/Linux and macOS.
https://sites.google.com/a/android.com/tools/knownissues/encoding
Windows will probably default to UTF16, since that's the native
encoding for files. So forcing things to use UTF-8 should help
compatibility.
com.github.jameshnsears.quoteunquote defines flavours 'fdroid' and
'fdroidS'. The old code used flavour in line, which matches both and the
wrong one was selected.
Closes: #912
'ndk_paths' will be automatically filled out from well known sources
like $ANDROID_HOME/ndk-bundle and $ANDROID_HOME/ndk/*. If a required
version is missing in the buildserver VM, it will be automatically
downloaded and installed into the standard $ANDROID_HOME/ndk/
directory. Manually setting it here will override the auto-detected
values. The keys can either be the "release" (e.g. r21e) or the
"revision" (e.g. 21.4.7075529).
https://developer.android.com/studio/projects/configure-agp-ndk#agp_version_41
* sdkmanager installs "ndk;12.3.4567890" into $ANDROID_SDK_ROOT/ndk/
* sdkmanager installs "ndk-bundle" into $ANDROID_SDK_ROOT/ndk-bundle/
There are two version numbers used for NDKs: the "release" and the
"revision". The "release" is used in the download URL and zipball and the
"revision" is used in the source.properties and the gradle ndkVersion field.
Also, there are some builds which need multiple NDKs installed, so this
makes it possible to have a list of release/revision entries in build.ndk.
This does not yet add full support since _fdroidserver/build.py_ will also
need changes.
The .txt format was the last place where the lowercase "builds" was used,
this converts references everywhere to be "Builds". This makes it possible
to load metadata YAML files with any YAML parser, then have it possible to
use fdroidserver methods on that data, like metadata.write_metadata().
The test files in tests/metadata/dump/*.yaml were manually edited by cutting
the builds: block and putting it the sort order for Builds: so the contents
should be unchanged.
```
sed -i \
-e 's/app\.builds/app.get('Builds', \[\])/g' \
-e "s/app\.get(Builds, \[\]) =/app\['Builds'] =/g" \
-e "s/app\.get(Builds, \[\]) =/app\['Builds'] =/g" \
-e "s/app\.get(Builds, \[\])/app.get('Builds', \[\])/g" \
-e "s/app\.get('Builds', \[\])\.append/app\['Builds'\].append/g" \
-e "s/app\['builds'\]/app.get('Builds', [])/g" \
*/*.*
```
Ideally, an fdroid repo should be running from a clean git repo, so that
all changes are tracked in git. This change is useful in seeing which
changes and/or files are not in git. If there are modified files, the
dirty flag will be set, so this info can help debugging that.
Back when fdroidserver was built around aapt, that was needed to
guarantee that a compatible version of aapt was used. Now, aapt is
only optionally used for getting the APK ID, so this was just
complicating maintenance.
None of the config options in config.py require Python code. YAML is a
common config data format, and it is also used for build metadata. It is
also much safer to use since it can be pure data, without anything
executable in it. This also reduces the attack surface of the fdroid
process by eliminating a guaranteed place to write to get code executed.
With config.py, any exploit that can get local write access can turn that
into execute access by writing to the config.py, then cleaning up after
itself once it has what it needs. Switching to YAML removes that vector
entirely.
Also, this removes the config_file argument. It is not used in either
fdroidserver or repomaker. Also, it probably wouldn't work since so
much of the code assumes that the current working dir is the root of the
repo.
Up until now, the buildserver has not included androguard. Since a
good version of androguard (v3.3.3+) is included in stretch-backports
and the buildserver is already setup to use stretch-backports, this
sets up the buildserver with androguard.
closes#627
