This complements the binary transparency logging that happens on the
server side !226. Anyone can set up an efficient tracker of any
F-Droid repo which stores all index files that it sees. It uses HEAD
requests and ETag checking to be as efficient as possible, so that
this can be automatically run at a frequent pace.
This was failing on environments that did not have any LANG or LC_* locale
variables set. This is a valid setup, and is common in headless setups, so
it needs to be handled.
This also adds a new pass of the test suite without the locale env vars set
so that this situation is also tests on gitlab-ci, not only gpjenkins.
The error this caused was:
UnicodeEncodeError: 'ascii' codec can't encode characters in position 6-18: ordinal not in range(128)
In 6c2cf2ccdd, the names of some essential
data bits changed. If those names are not in the tmp/apkcache, then
`fdroid update` shouldn't crash but instead just ignore that cache entry.
tmp/apkcache should be deleted since the metadata version was bumped, but
I guess that does not always happen.
With the binary transparency log stored in git, it makes that more readable
and as small as possible if the index files are fully sorted. That will
reduce the differences between two copies of an index file to a bare
minimum.
If in the future we implement some kind of binary diff transfer, this will
also help there.
Google is making gradle automatically download Android SDK components that
are needed by the build, but not already present. We need to support that
since it would be a lot of work to fight it. Plus, since each build starts
from the fresh snapshot, it should not be such a big deal to let each
build install stuff during the process.
closes#268
Google has their own utility for verifying APK signatures on a desktop
machine since Java's jarsigner is bad for the task. For example, it
acts as if an unsigned APK validates. And to check whether an APK is
unsigned using jarsigner is difficult.
apksigner also does the v2 signatures, so it will have to be used
eventually anyway. It is already in Debian/stretch and can be
available in jessie-backports if need be.
https://android.googlesource.com/platform/tools/apksighttps://packages.debian.org/apksigner
There are two SHA1 git commit IDs that fdroidserver includes in the builds
it makes: fdroidserverid and buildserverid. Originally, these were inserted
into AndroidManifest.xml, but that makes the build not reproducible. So
instead they are included as separate files in the APK's META-INF/ folder.
If those files exist in the signed APK, they will be part of the signature
and need to also be included in the unsigned APK for it to validate.
This makes the jarsigner the ultimate and only judge of whether two APKs
match. This is the best tool since APK signatures are jar signatures. This
should be eventually updated to use the official Android APK signing tool
called apksigner.
https://android.googlesource.com/platform/tools/apksig/
The ZIP format allows multiple entries with the exact same filename, and on
top of that, it does not allow deleting or updating entries. To make the
`fdroid verify` procedure failsafe, it needs to create a new temporary APK
that is made up on the contents of the "unsigned APK" and the signature
from the "signed APK". Since it would be possible to give a signed APK as
in the unsigned one's position, `fdroid verify` was not able to update the
signature since it was just adding the new signature to the end of the ZIP
file. When reading a ZIP, the first entry is used.
This is a bit different than index.jar: instead of their being index.xml
and index_unsigned.jar, the presense of index-v1.json means that there is
unsigned data. That file is then stuck into a jar and signed by the
signing process. index-v1.json is never published to the repo. It is
included in the binary transparency log, if that is enabled.
