Python encode/decode libs work directly with dicts, so the internal dict
can just be passed directly to any of these libs (pyyaml, pyjson, msgpack,
simplejson, etc). This still generates the exact same index.xml as before.
This converts the internal format for the repo timestamp to a datetime
instance, which can be easily converted to UNIX time in seconds for XML
and UNIX time in milliseconds for the new index formats. UNIX time in
milliseconds is directly serialized into a java.util.Date instance by
Jackson.
The ever troublesome gpjenkins box needs to use HTTPS mirrors. Plus it
improves the security of the buildserver, since there have been CVEs that
HTTPS would protect against:
https://www.debian.org/security/2016/dsa-3733
Google is pushing gradle towards downloading all the SDK components that it
needs, rather than having a preconfigured SDK installed. The buildserver
strongly supports the old model, with added checksum checking even. We can
still support the old model by pre-configuring the SDK and locking it down
as root. This can then also support the new model by setting the file perms
so that new packages can be auto-installed, but they cannot overwrite any
packages that come pre-installed and pre-verified.
fdroiddata!2096
closes#247
A Binary Transparency Log is a append only log of all binaries published by
a repo. This is useful for people to find whether the binary they have
matches what F-Droid has published, and also makes it more difficult for
the published history to be changed without notice, or for a server to give
specific users custom malware binaries.
https://www.eff.org/deeplinks/2014/02/open-letter-to-tech-companies
gradle will now automatically download and install missing bits of the
Android SDK. While we prefer to have the SDK packages fully verified, we
should allow this behavior on the buildserver to ensure that builds work
even when the buildserver can't be updated. Since each build starts from a
clean snapshot, this auto-installed build-tools will only be used for the
single build, so it won't affect other apps.
The new ConstraintLayout library in Android Support has some new custom way
of handling the license. I suspect that they are going to use this new way
with all of the bits that gradle downloads. We also have to support it for
apps that use it, including soon fdroidclient.
fdroiddata!2094
ci-images!1
FDroidPopen is used for running many commands - from git to gradle to
custom commands via flags like build=. When any of these invoke calls to
custom build systems or upstream programs/scripts, it's not safe to
assume that the output will be utf8.
Unfortunately, this currently leads to crashes and failed builds:
ERROR: Could not build app org.kiwix.kiwixmobile due to unknown error: Traceback (most recent call last):
File "/home/vagrant/fdroidserver/fdroidserver/build.py", line 1155, in main
options.onserver, options.refresh):
File "/home/vagrant/fdroidserver/fdroidserver/build.py", line 951, in trybuild
build_local(app, build, vcs, build_dir, output_dir, srclib_dir, extlib_dir, tmp_dir, force, onserver, refresh)
File "/home/vagrant/fdroidserver/fdroidserver/build.py", line 648, in build_local
p = FDroidPopen(['bash', '-x', '-c', cmd], cwd=root_dir)
File "/home/vagrant/fdroidserver/fdroidserver/common.py", line 1786, in FDroidPopen
result.output = result.output.decode('utf-8')
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb7 in position 5397290: invalid start byte
One way to fix this would be to use one of the python libraries that
guess an encoding. But a much safer option is to tell the decode method
to ignore non-utf8 bytes, as opposed to crashing on them.
Since it is now possible to build and include arbitrary files, like OTA
update ZIP files, the update procedure needs to look for non-APK files that
match the packageName_versionCode pattern of fdroid-generated files.
!193
admin#14
privileged-extension#9
`fdroid build` will generate source tarballs, and now with support for
adding any file to a repo, we need to explicitly ignore the fdroid-
generated source tarballs. If people want to include source tarballs in a
repo, they still can, as long as that source tarball doesn't use the
`fdroid build` tarball naming scheme.
This forces <uses-permission maxSdkVersion=""> to be an integer in the
internal dict, and forces it to have no decimal point in XML. Having it as
an integer in the internal dict means data will pass directly through to
the fdroidclient Apk instance, where it is ultimately an integer.
fdroidclient should handle no minSdkVersion fine, since it just parses the
text in <sdkver></sdkver> as an int, and uses a default value if there is
an Exception.
A .asc or .sig file is a detached PGPG signature, `fdroid gpgsign`
generates them. It makes no sense for them to be ever treated as a file
for distribution.
This also adds to forgotten forms of index files.
This lets index-v1 be parsed directly into class instances because the
field/instance var names match exactly. The original index v0 element
must retain the 'lastupdated' name for backwards compatibility.
