Normally, just 'repo/' is created by default, e.g. `fdroid init`. If APKs
are dumped into 'repo/', then have invalid signatures, then they'll be
automatically moved to 'archive/', which therefore needs to exist.
The new policy is to move APKs with invalid signatures to the archive,
and only add those APKs to the archive's index if they have valid MD5
signatures.
closes#323closes#292
In April 2017, Oracle's jarsigner and Google's apksigner both switched to
considering any APK signature that uses MD5 as unsigned. Any old build
is likely to have a MD5 signature. This sets up the archive as the only
place where these "disabled algorithms" are allowed in the repo, and
marks any APK signed by a "disabled algorithm" as having a "known
vulnerability"
This also now automatically moves APKs with invalid signatures to the
archive section.
#323
The original logic was checking keepversions against the len() of ALL the
APKs in the repo/archive. The correct thing is to check against the
number of APKs available for the given packageName/appid.
closes#166
This exploit is old, and was fixed in 4.4. But it was easy to exploit,
so it is still worth scanning for it. It is also easy to scan for, since
valid APKs should not have files with duplicate names. In theory, this
could look for duplicate file names for any file, but this limits the
false positives by only checking names of files related to executing code.
fdroidclient#40
`fdroid update` crashed for apps that only had screenshots but no graphics
or localized texts because destdir was not being set in that case. This
fixes that and adds a test case.
closes#320!286
If working with a random grabbag of APKs, there can be all sorts of
issues like corrupt entries in the ZIP, bad signatures, signatures that
are invalid since they use MD5, etc. Moving these two checks later means
that the APKs can be renamed still.
This does change how common.getsig() works. For years, it returned
None if the signature check failed. Now that I've started working
with giant APK collections gathered from the wild, I can see that
`fdroid update` needs to be able to first index what's there, then
make decisions based on that information. So that means separating
the getsig() fingerprint fetching from the APK signature verification.
This is not hugely security sensitive, since the APKs still have to
get past the Android checks, e.g. update signature checks. Plus the
APK hash is already included in the signed index.
With a generic file, the file name is the only guaranteed name metadata
field. So if the name is not specified in the metadata, then the name
is set to the filename. This changes that so that the file extension is
stripped from that generated name.
This is useful for parsing APK files, which can include packageName,
versionCode, and optionally 7 char signing key ID (i.e. <sig>).
This also can set the packageName and versionCoe for non APK files, so
that it is easy to assign them to metadata files, and to allow for
upgrades by setting the versionCode in the filename.
uses the standard package.name_123.apk. If that exists, it appends the
shasum. If that exists, then its a duplicate, so its deleted. This should
help @SergeWinters with his 12,000 APKs.
This is some very messy logic built up since 2010. This will all go away
once we have a python3 version of androguard available.
The removed imports and `dir(APK)` is to silence pyflakes
closes#303
_ is a valid character for Java package names, so it should also work in
the repo file naming scheme. This makes it so it only splits the file
name based on the last _.
This is a little omission. keys that are used in metadata/*.yml all start
with an UpperCase letter, but in fdroidserver, index-v1.json, and
fdroidclient, it is all camelCase with lowercase first letter. The keys
from the 'localized' section are currently never in metadata/*.yml, so
these keys never get downcase. This change will break fdroidclient
versions that do not also have this change, but since we're in alpha, that
should be fine.
If support for a 'localized' section is added to metadata/*.yml, then the
keys there should probably be UpperCase CamelCase to match the other keys.
glob != regexp, [A-Z-.@] means one of those chars is required in that
position, so the glob pattern was requiring a least 3 chars. Locales are
usually just two lower case letters, e.g. vi, de, ar.
This uses the "What's New" entry for the CurrentVersionCode and includes it
as the current WhatsNew metadata for the App class.
Things like fastlane supply and Google Play support a "What's New" entry
per-APK, but fdroidclient does not current use anything but the current
version of this data. Right now, it seems we probably only want to have
the latest WhatsNew in the index to save space.
In theory, we could make the WhatsNew data structure follow the structure
of fastlane/Play, but that would quite a bit of complexity for something
that might never be used.
fdroidclient#910
Since `fdroid server update` is the place where all uploads to servers
happens, it makes sense to also handle the git push for the binary
transparency log here instead of `fdroid btlog`
This was failing on environments that did not have any LANG or LC_* locale
variables set. This is a valid setup, and is common in headless setups, so
it needs to be handled.
This also adds a new pass of the test suite without the locale env vars set
so that this situation is also tests on gitlab-ci, not only gpjenkins.
The error this caused was:
UnicodeEncodeError: 'ascii' codec can't encode characters in position 6-18: ordinal not in range(128)
In 6c2cf2ccdd, the names of some essential
data bits changed. If those names are not in the tmp/apkcache, then
`fdroid update` shouldn't crash but instead just ignore that cache entry.
tmp/apkcache should be deleted since the metadata version was bumped, but
I guess that does not always happen.
With the binary transparency log stored in git, it makes that more readable
and as small as possible if the index files are fully sorted. That will
reduce the differences between two copies of an index file to a bare
minimum.
If in the future we implement some kind of binary diff transfer, this will
also help there.
