This expands the gradle wrapper shell script used by the buildserver for
usage outside the buildserver environment. It also allows downloading
whitelisted versions of gradle if they are not yet deployed to the
buildserver by simply upsating the copy of fdroidserver (in contrast to
having to reprovision the whole buildserver).
We first move the buildserver/gradle shell script to the repo root
as gradlew-fdroid, as it's an fdroid specific gradle wrapper.
We also now sync it inside the build VM before each build.
We then add a list of whitelisted gradle distributions taken from the
makebuildserver script.
The script additionally now reads two env vars which tell it where to
expect installed versions of gradle and where it might store downloaded
gradle .zip files. Both of those are configurable from config.py. As the
first should normally just be a subdir of the second it's not exposed in
the example config.py but only used by the buildserver config.py.
Default config now uses this internal gradle wrapper but a path to a
custom wrapper or specific gradle distribution can still be set from
config.py.
Closesfdroid/fdroidserver#98
Ref: fdroid/fdroidserver#370
For something like a verification server, this avoids having `fdroid verify`
redownload the remote APK from f-droid.org every time its run. For normal
users, it should download a fresh copy each time to avoid false errors
based on confusion over anything that might have changed the local copy of
the remote APK.
This patch has been used on verification.f-droid.org for a while now. It is
the last thing keeping verification.f-droid.org from using fdroidserver
straight from stretch-backports.
On versions of diffoscope before 87, like the version included in Ubuntu
xenial LTS, it would crash saying:
ValueError: max_diff_block_lines (100) cannot be smaller than max_page_diff_block_lines (128)
https://bugs.debian.org/875451
This commit fixes two bugs with reproducible builds:
* Files added by the buildserver to META-INF (fdroidserverid and buildserverid)
were causing signature verification to always fail when --on-server was used.
Since they are not needed anymore, they are no longer added to APKs.
* When showing a diff between both APK files, `jar xf` did not extract
the full APK properly which was causing useless diffs.
Instead of using jar, python's zipfile library is used instead.
For some reason, the parser stopped working intermittently, even
though the format has been the same since aapt 23 or earlier. Then
also, some of the test cases pointed to symlinks that were no longer
generated, and one test app now has a blank versionName.
Strange that this wasn't caught in the gitlab-ci runs. !484
FAIL: test_get_api_id_aapt (__main__.CommonTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "./common.TestCase", line 578, in testA_get_api_id_aapt
self.assertEqual(versionName, vn)
AssertionError: '0.1' != "0.1' platformBuildVersionName='4.3.1-1425645"
- 0.1
+ 0.1' platformBuildVersionName='4.3.1-1425645
This file is written freshly each time before use, so it does not need
to be ekpt around. It was the only file making the fdroiddata.git
repo dirty on the f-droid.org infrastructure.
This also adds stricter file permissions to avoid an attacker changing
those settings during operation.
fixes de35f1b05b:
Traceback (most recent call last):
File "/home/hans/code/fdroid/server/fdroid", line 164, in <module>
main()
File "/home/hans/code/fdroid/server/fdroid", line 138, in main
mod.main()
File "/export/share/code/fdroid/server/fdroidserver/update.py", line 1932, in main
apks, cachechanged = process_apks(apkcache, repodirs[0], knownapks, options.use_date_from_apk)
File "/export/share/code/fdroid/server/fdroidserver/update.py", line 1459, in process_apks
use_date_from_apk, ada, True)
File "/export/share/code/fdroid/server/fdroidserver/update.py", line 1332, in process_apk
apk = scan_apk(apkfile)
File "/export/share/code/fdroid/server/fdroidserver/update.py", line 1051, in scan_apk
scan_apk_androguard(apk, apk_file)
File "/export/share/code/fdroid/server/fdroidserver/update.py", line 1220, in scan_apk_androguard
res_id = arsc.get_id(apk['packageName'], res_id)[1]
TypeError: 'NoneType' object is not subscriptable
For example https://f-droid.org/archive/com.abitsinc.andr_5.apk:
$ aapt dump badging archive/com.abitsinc.andr_5.apk |head -1
package: name='com.abitsinc.andr' versionCode='5' versionName='5' platformBuildVersionName='2.3.3'
Instead of just crashing, first try to use the versionName as written in the
build metadata, otherwise just let it be blank. A blank versionName will
cause fdroidclient < 1.3 to crash. Blank versionNames are not allowed in
the .txt metadata format, only .yml.
closes#477closes#478
closes fdroidclient#1416
closes fdroidclient#1417
closes fdroidclient#1418
fdroiddata!3061
