The .txt format was the last place where the lowercase "builds" was used,
this converts references everywhere to be "Builds". This makes it possible
to load metadata YAML files with any YAML parser, then have it possible to
use fdroidserver methods on that data, like metadata.write_metadata().
The test files in tests/metadata/dump/*.yaml were manually edited by cutting
the builds: block and putting it the sort order for Builds: so the contents
should be unchanged.
```
sed -i \
-e 's/app\.builds/app.get('Builds', \[\])/g' \
-e "s/app\.get(Builds, \[\]) =/app\['Builds'] =/g" \
-e "s/app\.get(Builds, \[\]) =/app\['Builds'] =/g" \
-e "s/app\.get(Builds, \[\])/app.get('Builds', \[\])/g" \
-e "s/app\.get('Builds', \[\])\.append/app\['Builds'\].append/g" \
-e "s/app\['builds'\]/app.get('Builds', [])/g" \
*/*.*
```
Ideally, an fdroid repo should be running from a clean git repo, so that
all changes are tracked in git. This change is useful in seeing which
changes and/or files are not in git. If there are modified files, the
dirty flag will be set, so this info can help debugging that.
Back when fdroidserver was built around aapt, that was needed to
guarantee that a compatible version of aapt was used. Now, aapt is
only optionally used for getting the APK ID, so this was just
complicating maintenance.
None of the config options in config.py require Python code. YAML is a
common config data format, and it is also used for build metadata. It is
also much safer to use since it can be pure data, without anything
executable in it. This also reduces the attack surface of the fdroid
process by eliminating a guaranteed place to write to get code executed.
With config.py, any exploit that can get local write access can turn that
into execute access by writing to the config.py, then cleaning up after
itself once it has what it needs. Switching to YAML removes that vector
entirely.
Also, this removes the config_file argument. It is not used in either
fdroidserver or repomaker. Also, it probably wouldn't work since so
much of the code assumes that the current working dir is the root of the
repo.
Up until now, the buildserver has not included androguard. Since a
good version of androguard (v3.3.3+) is included in stretch-backports
and the buildserver is already setup to use stretch-backports, this
sets up the buildserver with androguard.
closes#627
This makes apksigner a hard requirement of the signing procedure.
We'll first try to find a globally installed version from PATH and if
that's not available fall back to using a version from build-tools.
Future TODO: always sign with apksigner, blocked on signature transplant
support for apksigv2/v3
Closesfdroid/fdroidserver#634Closesfdroid/fdroidserver#827
import is a strict keyword in Python, so it is not possible to import a
module called 'import', even with things like:
* import fdroidserver.import
* from fdroidserver import import
These days, the location that overrides all the others is in the android{}
block of the build.gradle file that loads the com.android.application
plugin. So this should be the preferred place to read these values.
test files GPL licensed: https://github.com/Integreight/1Sheeld-Android-App
This was done with much help from @uniqx. This is the first level of
supporting APK Signatures v1, v2, and v3. This is enough to include
APKs with any combo of v1/v2/v3 signatures. For this to work at all,
apksigner and androguard 3.3.3+ must be installed.
closes#399
androguard parses the whole APK before handing the instance back, this uses
the primitives to just find the <application android:debuggable=""> value,
then stop parsing.
#557
Normally, androguard parses the entire APK before it is possible to get any
values from it. This uses androguard primitives to only attempt to parse
the AndroidManifest.xml, then to quit as soon as it gets what it needs.
This greatly speeds up the parsing (1 minute vs 60 minutes).
fdroid/fdroidserver#557
For some reason, the parser stopped working intermittently, even
though the format has been the same since aapt 23 or earlier. Then
also, some of the test cases pointed to symlinks that were no longer
generated, and one test app now has a blank versionName.
Strange that this wasn't caught in the gitlab-ci runs. !484
FAIL: test_get_api_id_aapt (__main__.CommonTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "./common.TestCase", line 578, in testA_get_api_id_aapt
self.assertEqual(versionName, vn)
AssertionError: '0.1' != "0.1' platformBuildVersionName='4.3.1-1425645"
- 0.1
+ 0.1' platformBuildVersionName='4.3.1-1425645
These were both spamming the output with lots of confusing messages, even
when --verbose was not used. Jarsigner especially has confusing messages,
since it has warnings that do not pertain to APK signatures at all, like
the ones about timestamps and missing Certificate Authority.
closes#405
This came about testing on OSX, where there are often multiple versions of
the JDK installed. This was choosing the oldest version. It should
choose the most recent version.
This is useful for parsing APK files, which can include packageName,
versionCode, and optionally 7 char signing key ID (i.e. <sig>).
This also can set the packageName and versionCoe for non APK files, so
that it is easy to assign them to metadata files, and to allow for
upgrades by setting the versionCode in the filename.
Google has their own utility for verifying APK signatures on a desktop
machine since Java's jarsigner is bad for the task. For example, it
acts as if an unsigned APK validates. And to check whether an APK is
unsigned using jarsigner is difficult.
apksigner also does the v2 signatures, so it will have to be used
eventually anyway. It is already in Debian/stretch and can be
available in jessie-backports if need be.
https://android.googlesource.com/platform/tools/apksighttps://packages.debian.org/apksigner
The ZIP format allows multiple entries with the exact same filename, and on
top of that, it does not allow deleting or updating entries. To make the
`fdroid verify` procedure failsafe, it needs to create a new temporary APK
that is made up on the contents of the "unsigned APK" and the signature
from the "signed APK". Since it would be possible to give a signed APK as
in the unsigned one's position, `fdroid verify` was not able to update the
signature since it was just adding the new signature to the end of the ZIP
file. When reading a ZIP, the first entry is used.
