1
0
mirror of https://gitlab.com/fdroid/fdroidserver.git synced 2024-11-15 19:30:31 +01:00
Commit Graph

8637 Commits

Author SHA1 Message Date
Hans-Christoph Steiner
660f8756e5 Merge branch 'docstrings' into 'master'
docstrings and code format from !1436

See merge request fdroid/fdroidserver!1482
2024-05-08 13:55:10 +00:00
Gregor Düster
dbdefe200c Format files with ruff 2024-05-08 15:41:51 +02:00
Gregor Düster
df27405d8b [build] Add more docstrings 2024-05-08 15:35:28 +02:00
Gregor Düster
1c70758940 [btlog] Add more docstrings 2024-05-08 15:31:43 +02:00
Gregor Düster
4109e8fb03 [checkupdates] Add module docstring 2024-05-08 15:31:38 +02:00
Gregor Düster
97346a2cba [nightly] Add more docstrings 2024-05-08 15:31:32 +02:00
Gregor Düster
6c27fec94b [import] Add more docstrings 2024-05-08 15:31:29 +02:00
Michael Pöhn
5d8c6b8cd5 Merge branch 'fix-ubuntu-ppa-job' into 'master'
Fix ubuntu ppa job

See merge request fdroid/fdroidserver!1481
2024-05-08 13:17:23 +00:00
Hans-Christoph Steiner
9f62445f38 gitlab-ci: fix ubuntu_lts_ppa job to work with Ubuntu/noble 2024-05-08 13:16:00 +00:00
Michael Pöhn
80705579c2 Merge branch 'getcert' into 'master'
get_first_signer_certificate: check all v1 v2 and v3 certs

Closes #1128

See merge request fdroid/fdroidserver!1466
2024-05-08 13:14:05 +00:00
Hans-Christoph Steiner
ad9f0a9022 include @obfusk's proof-of-concept APKs in test suite
https://github.com/obfusk/fdroid-fakesigner-poc/releases/tag/poc-apks
2024-05-07 16:22:59 +02:00
Hans-Christoph Steiner
fc4a9c96a5 test APK signatures with a cert chain are parsed like apksigner
Microsoft and SanDisk sign APKs with a X.509 certificate chain of
trust, so there are actually three certificates included. apksigner
only cares about one certificate and ignores the other certificates in
the chain.

The correct values come from:

    apksigner verify --print-certs 883cbdae7aeb2e4b122e8ee8d89966c7062d0d49107a130235fa220a5b994a79.apk

X.509 certificates are machine generated and just data, so are not
copyrightable.  So I included SANAPPSI.* directly.
2024-05-07 16:22:59 +02:00
Hans-Christoph Steiner
accdd65f91 also handle APKs entirely without JAR/v1 signatures
future-proofing!
2024-05-07 16:22:59 +02:00
Hans-Christoph Steiner
9a327b5097 reliable implementation of get_first_signer_certificate()
This keeps key pieces of @linsui's algorithm, specifically the check
that all certificates are the same.  apksigner also does this check.

closes #1128
2024-05-07 16:22:59 +02:00
Hans-Christoph Steiner
a8fd360a88 skip AndroidManifest.xml and resources when fetching v2+ certs 2024-05-07 16:22:59 +02:00
FC (Fay) Stegerman
6f5fd2b132 PoC + writeup + patch
6c6dc25112/fdroidserver.patch (L28)

https://github.com/androguard/androguard/issues/1030
refs #1128

(this is an excerpt of the original patch)
2024-05-07 16:22:59 +02:00
Hans-Christoph Steiner
312f822764 androguard is required, stop using use_androguard() 2024-05-07 16:22:59 +02:00
linsui
2fea71a6c7 get_first_signer_certificate: check all v1 v2 and v3 certs 2024-05-07 16:22:59 +02:00
Hans-Christoph Steiner
93f361c623 replace decade old pyasn1 crypto impl with working asn1crypto
For some APKs, get_certificate() was returning a different result than
apksigner and keytool.  So I just took the algorithm from androguard, which
uses asn1crypto instead of pyasn1.  So that removes a dependency as well.
asn1crypto is already required by androguard.

The original get_certificate() came from 6e2d0a9e1
2024-05-07 16:22:59 +02:00
Hans-Christoph Steiner
4666330a4d Merge branch 'gradlefile' into 'master'
throw error if gradle build method is used but no build.gradle file is found

See merge request fdroid/fdroidserver!1479
2024-05-07 14:14:26 +00:00
linsui
7104411296 throw error if gradle build method is used but no build.gradle file is found 2024-05-07 14:13:47 +00:00
Hans-Christoph Steiner
99bd544ab9 Merge branch 'fedora-40-ci-failure' into 'master'
make it easier to support the Fedora job

See merge request fdroid/fdroidserver!1474
2024-05-07 14:11:53 +00:00
Hans-Christoph Steiner
5df3d27126 gitlab-ci: stay on Fedora 39 until it is no longer supported
We can rely on the debian:testing job to test the bleeding edge, and it is
a lot easier to troubleshoot.

The Fedora job is a lot harder to troubleshoot than the Debian-based jobs,
and they are often quite bleeding edge.  Currently, there is a change to
either Python or an image processing lib (Pillow?) that now compresses PNGs
differently than all previous releases.  That breaks the tests based on
processing images and checking the SHA-256 matches.

70e7e720b9
fdroidserver!669
2024-05-07 12:58:23 +00:00
Hans-Christoph Steiner
1b65e33835 make it easy to keep test artifacts from jobs
When troubleshooting things that are difficult to reproduce locally, like
different behaviors in the fedora_latest job, these changes make it easy to
keep the test files around after the tests run.  For example, if PNGs are
processed differently by newer Python versions.
2024-05-07 12:58:23 +00:00
Hans-Christoph Steiner
299e3e5f4c index: handle image processing diffs across various Python versions
Apparently, the newest Python thingies strip the PNGs a tiny bit smaller,
so a fixed file size will lead to the test failing:

https://gitlab.com/fdroid/fdroidserver/-/jobs/6703386074
```
Traceback (most recent call last):
  File "/builds/fdroid/fdroidserver/tests/index.TestCase", line 704, in test_package_metadata
    self.assertEqual(36027, metadata['featureGraphic']['en-US']['size'])
AssertionError: 36027 != 35619
```
2024-05-07 12:58:23 +00:00
Hans-Christoph Steiner
1cb1394de3 Merge branch 'debugkey' into 'master'
lint: blocklist known AOSP debug keys in AASK

See merge request fdroid/fdroidserver!1478
2024-05-07 11:33:12 +00:00
Hans-Christoph Steiner
9a9b5beeaa simplify test setup
I'm in the midst of working towards getting rid of the "config" instances
that are in the subcommand module, e.g. `fdroidserver.lint.config`
2024-05-07 11:33:04 +00:00
Hans-Christoph Steiner
14c8647909 add additional tests 2024-05-07 11:33:04 +00:00
linsui
d243cbd030 lint: blocklist known AOSP debug keys in AASK 2024-05-07 11:33:04 +00:00
Michael Pöhn
08c553e1cb Merge branch 'deploy' into 'master'
deploy: retry when git push fails

See merge request fdroid/fdroidserver!1480
2024-05-06 06:36:58 +00:00
linsui
f82d648cb1 deploy: retry when git push fails 2024-05-05 20:53:35 +00:00
Michael Pöhn
79586fd9e3 Merge branch 'fix-icons-in-fdroid-mirror' into 'master'
mirror: apparently the 'icons' dir does not always exist

See merge request fdroid/fdroidserver!1476
2024-05-05 17:14:49 +00:00
Hans-Christoph Steiner
6f07538cdc mirror: apparently the 'icons' dir does not always exist
https://apt.izzysoft.de/fdroid/archive/com.futsch1.medtimer_28.apk:
2024-04-21 18:55:20 ERROR 404: Not Found.
https://apt.izzysoft.de/fdroid/archive/com.hardbacknutter.nevertoomanybooks_75.apk:
2024-04-21 18:55:20 ERROR 404: Not Found.
https://apt.izzysoft.de/fdroid/archive/com.jovial.jrpn15_20.apk:
2024-04-21 18:55:20 ERROR 404: Not Found.
https://apt.izzysoft.de/fdroid/archive/com.jovial.jrpn2_20.apk:
2024-04-21 18:55:20 ERROR 404: Not Found.
https://apt.izzysoft.de/fdroid/archive/com.logger.app_8.apk:
2024-04-21 18:55:21 ERROR 404: Not Found.
https://apt.izzysoft.de/fdroid/archive/com.lorenzovainigli.foodexpirationdates.foss_22.apk:
2024-04-21 18:55:21 ERROR 404: Not Found.
https://apt.izzysoft.de/fdroid/archive/com.openathena_36.apk:
2024-04-21 18:55:21 ERROR 404: Not Found.
https://apt.izzysoft.de/fdroid/archive/com.oppzippy.openscq30_21.apk:
2024-04-21 18:55:21 ERROR 404: Not Found.
https://apt.izzysoft.de/fdroid/archive/com.sanmer.mrepo_713.apk:
2024-04-21 18:55:21 ERROR 404: Not Found.
https://apt.izzysoft.de/fdroid/archive/com.starry.myne_325.apk:
2024-04-21 18:55:21 ERROR 404: Not Found.
https://apt.izzysoft.de/fdroid/archive/dev.imranr.obtainium_22602.apk:
2024-04-21 18:55:22 ERROR 404: Not Found.
https://apt.izzysoft.de/fdroid/archive/dev.sanmer.pi_137.apk:
2024-04-21 18:55:22 ERROR 404: Not Found.
https://apt.izzysoft.de/fdroid/archive/org.breezyweather_50108.apk:
2024-04-21 18:55:22 ERROR 404: Not Found.
2024-04-21 18:55:22,448 ERROR: no "icon" in org.breezyweather
2024-04-21 18:55:22,449 ERROR: no "icon" in com.lorenzovainigli.foodexpirationdates.foss
2024-04-21 18:55:22,449 ERROR: no "icon" in com.jovial.jrpn15
2024-04-21 18:55:22,450 ERROR: no "icon" in com.jovial.jrpn2
2024-04-21 18:55:22,450 ERROR: no "icon" in com.logger.app
2024-04-21 18:55:22,450 ERROR: no "icon" in com.futsch1.medtimer
2024-04-21 18:55:22,451 ERROR: no "icon" in com.sanmer.mrepo
2024-04-21 18:55:22,451 ERROR: no "icon" in com.starry.myne
2024-04-21 18:55:22,451 ERROR: no "icon" in com.hardbacknutter.nevertoomanybooks
2024-04-21 18:55:22,452 ERROR: no "icon" in dev.imranr.obtainium
2024-04-21 18:55:22,452 ERROR: no "icon" in com.openathena
2024-04-21 18:55:22,453 ERROR: no "icon" in com.oppzippy.openscq30
2024-04-21 18:55:22,453 ERROR: no "icon" in dev.sanmer.pi
2024-04-21 18:55:22,454 CRITICAL: Unknown exception found!
Traceback (most recent call last):
  File "/home/fdroid/code/fdroid/fdroidserver/fdroid", line 22, in <module>
    fdroidserver.__main__.main()
  File "/home/fdroid/code/fdroid/fdroidserver/fdroidserver/__main__.py", line 230, in main
    raise e
  File "/home/fdroid/code/fdroid/fdroidserver/fdroidserver/__main__.py", line 211, in main
    mod.main()
  File "/home/fdroid/code/fdroid/fdroidserver/fdroidserver/mirror.py", line 269, in main
    _run_wget(os.path.join(basedir, section, icondir), urls[icondir])
KeyError: 'icons'
2024-05-02 18:29:42 +02:00
Hans-Christoph Steiner
32ef77ecb6 Merge branch 'androguard-modernization' into 'master'
port to androguard >= 4 and drop support for older than 3.3.5

See merge request fdroid/fdroidserver!1462
2024-04-25 11:09:27 +00:00
Hans-Christoph Steiner
be59b38ac1 update: handle ValueError from apkInspector in androguard 4.1
androguard 4.1 uses a new lib called apkInspector instead of zipfile.ZipFile
so that it can handle usable but invalid ZIP files.  It will also throw
ValueError on some things, for example:

Traceback (most recent call last):
  File "/builds/eighthave/fdroidserver/fdroidserver-2.3a0/tests/update.TestCase", line 878, in test_scan_apk_bad_zip
    fdroidserver.update.scan_apk(apkfile)
  File "/builds/eighthave/fdroidserver/fdroidserver-2.3a0/fdroidserver/update.py", line 1586, in scan_apk
    scan_apk_androguard(apk, apk_file)
  File "/builds/eighthave/fdroidserver/fdroidserver-2.3a0/fdroidserver/update.py", line 1725, in scan_apk_androguard
    apkobject = common.get_androguard_APK(apkfile)
  File "/builds/eighthave/fdroidserver/fdroidserver-2.3a0/fdroidserver/common.py", line 2673, in get_androguard_APK
    return APK(apkfile)
  File "/usr/local/lib/python3.10/dist-packages/androguard/core/apk/__init__.py", line 273, in __init__
    self.zip = ZipEntry.parse(filename, False)
  File "/usr/local/lib/python3.10/dist-packages/apkInspector/headers.py", line 410, in parse
    eocd = EndOfCentralDirectoryRecord.parse(apk_file)
  File "/usr/local/lib/python3.10/dist-packages/apkInspector/headers.py", line 59, in parse
    raise ValueError("End of central directory record (EOCD) signature not found")
ValueError: End of central directory record (EOCD) signature not found
2024-04-25 13:00:23 +02:00
Hans-Christoph Steiner
ef4ec74882 some parts of androguard 4.x use loguru instead of logging 2024-04-25 12:39:12 +02:00
Hans-Christoph Steiner
1c84f63247 replace deprecated get_element() which was removed in 4.x
/usr/lib/python3/dist-packages/androguard/core/bytecodes/apk.py:884: DeprecationWarning: This method is deprecated since 3.3.5.

It was added in 3.3.5.  Debian/bullseye and Ubuntu/20.04/focal both include
new enough versions.  Debian/buster's is too old (3.3.3).
2024-04-25 12:39:12 +02:00
Hans-Christoph Steiner
5b7abc0423 single function to tame androguard's verbose default output
# Conflicts:
#	fdroidserver/common.py
2024-04-25 12:39:12 +02:00
Hans-Christoph Steiner
7a144a4762 port to androguard >= 4 and drop support for older than 3.3.3
This also makes androguard a hard requirement, which has been true for a
while anyway.  So the code that handles androguard as an optional
requirement is removed.  androguard from Debian/buster is new enough, so
this does not seem like it will cause any problems.
2024-04-25 12:39:12 +02:00
Hans-Christoph Steiner
cdc7c98707 common.get_androguard_APK() is no longer private to the module 2024-04-25 12:39:12 +02:00
Hans-Christoph Steiner
a742df3758
add CHANGELOG entry for v2.2.2 2024-04-25 12:39:00 +02:00
Hans-Christoph Steiner
e45e37b2df Merge branch 'altstore-index-fix' into 'master'
🐛 index fix: skip altstore when no IPAs present

See merge request fdroid/fdroidserver!1473
2024-04-24 12:05:03 +00:00
Michael Pöhn
005a33732c apply review suggestions 2024-04-24 11:56:07 +00:00
Michael Pöhn
bd6afa4365
🐛 index fix: skip altstore when no IPAs present 2024-04-24 12:45:29 +02:00
Hans-Christoph Steiner
07929ffe66 Merge branch 'black' into 'master'
move fdroidserver/deploy.py to black format

See merge request fdroid/fdroidserver!1472
2024-04-24 08:51:46 +00:00
Hans-Christoph Steiner
79f148167a deploy: automatically convert to black format 2024-04-24 08:42:15 +00:00
Hans-Christoph Steiner
cb04d801d8 deploy: manually move hard cases to black code format 2024-04-24 08:42:15 +00:00
Hans-Christoph Steiner
eacfb8095b Merge branch 'alt-store' into 'master'
iOS metadata + alt-store source support

See merge request fdroid/fdroidserver!1465
2024-04-24 08:39:05 +00:00
Hans-Christoph Steiner
9716b5e1ab index: manual black format 2024-04-24 10:29:50 +02:00
Michael Pöhn
d00a87ed6c
🏏 alt-store index: incorporate review feedback 2024-04-23 17:28:30 +02:00