1
0
mirror of https://gitlab.com/fdroid/fdroidserver.git synced 2024-11-15 03:20:10 +01:00
Commit Graph

951 Commits

Author SHA1 Message Date
Hans-Christoph Steiner
ad9f0a9022 include @obfusk's proof-of-concept APKs in test suite
https://github.com/obfusk/fdroid-fakesigner-poc/releases/tag/poc-apks
2024-05-07 16:22:59 +02:00
Hans-Christoph Steiner
fc4a9c96a5 test APK signatures with a cert chain are parsed like apksigner
Microsoft and SanDisk sign APKs with a X.509 certificate chain of
trust, so there are actually three certificates included. apksigner
only cares about one certificate and ignores the other certificates in
the chain.

The correct values come from:

    apksigner verify --print-certs 883cbdae7aeb2e4b122e8ee8d89966c7062d0d49107a130235fa220a5b994a79.apk

X.509 certificates are machine generated and just data, so are not
copyrightable.  So I included SANAPPSI.* directly.
2024-05-07 16:22:59 +02:00
Hans-Christoph Steiner
accdd65f91 also handle APKs entirely without JAR/v1 signatures
future-proofing!
2024-05-07 16:22:59 +02:00
Hans-Christoph Steiner
9a327b5097 reliable implementation of get_first_signer_certificate()
This keeps key pieces of @linsui's algorithm, specifically the check
that all certificates are the same.  apksigner also does this check.

closes #1128
2024-05-07 16:22:59 +02:00
Hans-Christoph Steiner
1b65e33835 make it easy to keep test artifacts from jobs
When troubleshooting things that are difficult to reproduce locally, like
different behaviors in the fedora_latest job, these changes make it easy to
keep the test files around after the tests run.  For example, if PNGs are
processed differently by newer Python versions.
2024-05-07 12:58:23 +00:00
Hans-Christoph Steiner
299e3e5f4c index: handle image processing diffs across various Python versions
Apparently, the newest Python thingies strip the PNGs a tiny bit smaller,
so a fixed file size will lead to the test failing:

https://gitlab.com/fdroid/fdroidserver/-/jobs/6703386074
```
Traceback (most recent call last):
  File "/builds/fdroid/fdroidserver/tests/index.TestCase", line 704, in test_package_metadata
    self.assertEqual(36027, metadata['featureGraphic']['en-US']['size'])
AssertionError: 36027 != 35619
```
2024-05-07 12:58:23 +00:00
Hans-Christoph Steiner
9a9b5beeaa simplify test setup
I'm in the midst of working towards getting rid of the "config" instances
that are in the subcommand module, e.g. `fdroidserver.lint.config`
2024-05-07 11:33:04 +00:00
Hans-Christoph Steiner
14c8647909 add additional tests 2024-05-07 11:33:04 +00:00
linsui
d243cbd030 lint: blocklist known AOSP debug keys in AASK 2024-05-07 11:33:04 +00:00
Hans-Christoph Steiner
5b7abc0423 single function to tame androguard's verbose default output
# Conflicts:
#	fdroidserver/common.py
2024-04-25 12:39:12 +02:00
Hans-Christoph Steiner
7a144a4762 port to androguard >= 4 and drop support for older than 3.3.3
This also makes androguard a hard requirement, which has been true for a
while anyway.  So the code that handles androguard as an optional
requirement is removed.  androguard from Debian/buster is new enough, so
this does not seem like it will cause any problems.
2024-04-25 12:39:12 +02:00
Hans-Christoph Steiner
cdc7c98707 common.get_androguard_APK() is no longer private to the module 2024-04-25 12:39:12 +02:00
Michael Pöhn
bd6afa4365
🐛 index fix: skip altstore when no IPAs present 2024-04-24 12:45:29 +02:00
Michael Pöhn
d00a87ed6c
🏏 alt-store index: incorporate review feedback 2024-04-23 17:28:30 +02:00
Michael Pöhn
f2118b35a3
🏟️ fix ci 2024-04-23 16:06:10 +02:00
Michael Pöhn
45efb88f85
🕴️ add test for make_altstore 2024-04-23 16:05:58 +02:00
Michael Pöhn
86db8c93cc
🩹 fix parse_ipa tests 2024-04-23 16:05:47 +02:00
Michael Pöhn
f742799a9d
🏟️ add test for _parse_from_pbxproj
Also fix lint issues
2024-04-23 16:05:34 +02:00
Michael Pöhn
450765490b
🗺️ add test for _get_ipa_ico 2024-04-23 16:05:22 +02:00
Michael Pöhn
6152abee08 🪀 tests for insert_localized_ios_app_metadata
Split some functions from insert_localized_ios_app_metadata into
sub-functions and implemented separate tests for each.
2024-04-03 14:55:35 +00:00
Michael Pöhn
bbf17ee59c 🧆 improve and test parse_ios_screenshot_name 2024-04-03 14:55:35 +00:00
Michael Pöhn
293caf5ff9 🪪 iOS fastlane screenshot support 2024-04-03 14:55:35 +00:00
Hans-Christoph Steiner
9c65bed4a5 check for <application android:testOnly="true">
This adds a check for "testOnly" to the existing "debuggable" check, since
they are very similar.  We should really be refactoring all the checks into
a more reasonable setup.  Since "debuggable" and "testOnly" are both set in
the same place (`<application>` in _AndroidManifest.xml_) and are both set
by the same process (running debug builds), I thought it would be OK to
include both in the same place.  Plus it was a one-line change.
2024-04-03 13:47:04 +00:00
Hans-Christoph Steiner
e269e41b12
publish: --error-on-failed to exit when signing/verifying fails
Since we have limited visibility into @CiaranG's signing server, it is hard
to make changes to the publishing process, especially ones that might break
@CiaranG's automation.  So `fdroid publish` mostly reports success by
moving an APK from unsigned/ to repo/.  In some cases, we want immediate
failure, like in CI.  So this adds `--error-on-failed` for that purpose.
2024-04-03 15:33:18 +02:00
proletarius101
ac4694fcd3
fix Could not initialize class org.codehaus.groovy.runtime.InvokerHelper
https://gitlab.com/proletarius101/fdroidserver/-/jobs/6068794117#L547
2024-03-14 13:14:41 +01:00
proletarius101
f2bdf5214a
install openjdk 11 from bullseye repo instead in the test cases 2024-03-14 13:14:41 +01:00
proletarius101
0a3f785254
drop gradle support before 7.3
debian bookworm supports only Java 17+, which is compatible with gradle 7.3+: https://docs.gradle.org/current/userguide/compatibility.html
2024-03-14 13:14:41 +01:00
proletarius101
a0a766cc8e
drop gradle support before 2.0
debian bullseye supports only Java 8+, which is compatible with gradle 2.0+: https://docs.gradle.org/current/userguide/compatibility.html
2024-03-14 13:14:41 +01:00
linsui
a002e46997 scanner: update maven repo regex 2024-03-12 07:57:11 +00:00
Hans-Christoph Steiner
cff3364fdf split out mirrors data structure into standalone, tested function 2024-03-06 13:31:46 +00:00
Hans-Christoph Steiner
9749282b44 net: fix test_download_file_url_parsing
self.assertTrue(requests_get.called) will always be true because .called
will contain a MagicMock instance.
2024-03-06 13:31:46 +00:00
Hans-Christoph Steiner
7904f12d05 net: add test of automatic retries in download_file()
The existing logic from d1ddd525c in !1225 is confusing because it adds its
own retry loop on top of the retry mechanism that is built into requests.
So this test confirms that setting `download_file(retries=3)` actually
results in more than three retries.
2024-03-06 13:31:46 +00:00
Hans-Christoph Steiner
074dda4b61 use tiny entry.jar rather than large index-v1.jar for net test case 2024-03-06 13:31:46 +00:00
Hans-Christoph Steiner
261be20109 convert tests/testcommon.py to black code format 2024-03-06 13:31:46 +00:00
Nitai Sasson
8931e038d6 add 'non-changeable' to NonFreeNet description 2024-02-21 17:13:58 +01:00
proletarius101
947217549a feat: add servergitmirrors as a dict support 2024-02-14 16:50:39 +00:00
Hans-Christoph Steiner
031a130395 scanner: add refresh_config config item for buildserver
Includes some cosmetic changes from black.
2024-01-25 17:04:38 +01:00
Hans-Christoph Steiner
d6aa3caaca
scanner: test of --refresh controls triggering a refresh 2024-01-25 14:03:46 +01:00
Hans-Christoph Steiner
1d9ec42758 scanner: update test counts based on new default rules
The new rules announce more things.
2024-01-25 12:31:05 +01:00
Hans-Christoph Steiner
681392d8c2 scanner: script to update default rules from SUSS 2024-01-25 11:51:22 +01:00
Hans-Christoph Steiner
810387a009 deploy: update_serverwebroots() for testable logic
This moves all of the serverwebroot: logic into a function, and adds tests.
I did this because I ran into issues in the logic in main():

Traceback (most recent call last):
  File "/builds/eighthave/fdroidserver/fdroid", line 22, in <module>
    fdroidserver.__main__.main()
  File "/builds/eighthave/fdroidserver/fdroidserver/__main__.py", line 230, in main
    raise e
  File "/builds/eighthave/fdroidserver/fdroidserver/__main__.py", line 211, in main
    mod.main()
  File "/builds/eighthave/fdroidserver/fdroidserver/deploy.py", line 753, in main
    s = serverwebroot.rstrip('/').split(':')
AttributeError: 'dict' object has no attribute 'rstrip'
2024-01-23 18:49:12 +01:00
Hans-Christoph Steiner
fbf097d390 deploy: update_serverwebroot() works w/o options/config
Since update_serverwebroot() is part of the public API, this function should
work without setting `fdroidserver.deploy.options` or
`fdroidserver.deploy.config`.
2024-01-23 17:48:47 +01:00
Hans-Christoph Steiner
7a656d45e3 config: convert serverwebroot: to list-of-dicts format
This allows for more metadata about the server and deploy mode.
2024-01-22 22:11:49 +01:00
Hans-Christoph Steiner
3f50372d8d config: test cases for serverwebroot: with string and list 2024-01-22 22:11:49 +01:00
Hans-Christoph Steiner
dc7170e709 gitlab-ci: install biplist if available, otherwise skip test_parse_ipa
Fedora does not have a biplist package.
2024-01-11 16:02:06 +01:00
Hans-Christoph Steiner
6eee83db47 run black on new .ipa test cases 2024-01-11 16:02:06 +01:00
Michael Pöhn
8b5a61bb25 ⛰️ make ipa related test cases more robust 2024-01-11 12:02:53 +01:00
Michael Pöhn
7211e9f9b4 🍲 add unit test for update.scan_repo_for_ipas 2024-01-11 11:34:21 +01:00
Michael Pöhn
995118bcd2 🥔 add strapped IPA file and test for parse_ipa 2024-01-11 11:34:21 +01:00
Michael Pöhn
c288317530 🪨 version string conversion: error handling+tests 2024-01-11 11:34:21 +01:00
Hans-Christoph Steiner
0849e664e7 metadata_v0: use git rev-parse instead of git describe 2024-01-09 14:54:38 +01:00
Hans-Christoph Steiner
96fc49d7fc lint: check syntax of countryCode: fields for mirrors 2024-01-09 09:32:37 +00:00
Hans-Christoph Steiner
4511da68b9 lint: support linting config files 2024-01-09 09:32:37 +00:00
Hans-Christoph Steiner
15b983f48d handle all cases of @ in srclibs
expands on fdroidserver!1422
2024-01-07 10:50:29 +01:00
Hans-Christoph Steiner
8c8b9021f5 tests/run-tests: disable nightly.TestCase on non-Linux until its ported 2023-12-15 09:15:19 +01:00
Hans-Christoph Steiner
52f27aea75 tests/run-tests: temp disable git mirror test on non-Linux
I looked into this but couldn't figure it out.  It is the only failing
test on macOS, so it seemed worth just adding this single quirk for now.
2023-12-15 09:15:19 +01:00
Hans-Christoph Steiner
865673964e gitlab-ci: enable shellcheck on tests/run-tests 2023-12-15 08:40:01 +01:00
Hans-Christoph Steiner
45d1363a0e tests/run-tests: use $sed var for all calls that need GNU sed
FreeBSD, macOS, and others come with BSD sed, which does not support all
the things that GNU sed does.  Also, BSD rmdir does not have any extended
options like --ignore-fail-on-non-empty.
2023-12-13 22:30:19 +01:00
Hans-Christoph Steiner
e2949cbd61 port to looseversion 2023-12-08 10:48:01 +01:00
Andrew Gunnerson
679ba6f552
Add support for Python 3.12
Python 3.12 completely removed the builtin `distutils` module. This
commit replaces its use with the `packaging` package, which is an
external dependency, but maintained by the Python developers.

Signed-off-by: Andrew Gunnerson <accounts+gitlab@chiller3.com>
2023-12-07 18:15:05 -05:00
Hans-Christoph Steiner
6d541e3ef6
establish config/mirrors.yml as a way to set up mirrors
The mirrors: entry in config.yml is great for quick access and shorter
mirror lists.  Now that we are adding a lot more metadata to the mirrors,
including potentially the full text of the privacy policy, having this
in its own file should make these cases easier to manage.
2023-12-07 14:38:26 +01:00
Hans-Christoph Steiner
f13d9235ce remove jq dependency in the tests, port to GNU sed
This avoids spreading the jq dependency to more places, like:

https://gitlab.com/eighthave/ansible-role-install-fdroidserver-dependencies/-/jobs/5559798360

And is helpful when porting to non-Debian platforms.
2023-11-23 12:24:49 +00:00
linsui
f99f430e9e deinit git submodule 2023-11-14 01:06:49 +08:00
Jochen Sprickerhof
521ff4fe17 Add default locale for generated categories
The client expects at least a name to be set.

Regression of 2c566cf6.
2023-09-21 14:14:06 +00:00
Hans-Christoph Steiner
3757add164 test_sdk_exists to be based on apksigner, that's the requirement
Before, lots of pieces of the Android SDK were required for fdroidserver to
operate, like aapt, zipalign, etc.  Now, apksigner is the only requirement.

%"support APK Signature v2+"
!889
2023-09-17 23:31:07 +02:00
Hans-Christoph Steiner
48559ecec5 category with no apps should be ignored, even if defined in config
https://gitlab.com/fdroid/fdroidclient/-/issues/2619#note_1421280589

The test needed to change because the test index files contained category
definitions that were not ever used in the "copy tests/repo, generate java/gpg
keys, update, and gpgsign" test in tests/run-tests.
2023-09-08 10:45:53 +02:00
Hans-Christoph Steiner
2c566cf68f update: add all categories in metadata files to repo definitions
!1366 makes it so categories are now defined by the repo. Categories can be
defined in the config so that lint has a list of categories to enforce. This
also provides a place for localization and icons for the categories. The old
way of defining categories was just listing them in app metadata files. This
restores that way of functioning when using index-v2.

closes #1137
2023-09-08 10:45:51 +02:00
Hans-Christoph Steiner
64b8ee772c throw useful error if a config YAML file is not a dict 2023-09-08 10:42:27 +02:00
Hans-Christoph Steiner
00aa595f37 deploy: give useful error if rsync is not installed 2023-09-08 10:42:27 +02:00
linsui
8188bb6bfa metadata: set mapping indentation to 2 2023-09-08 08:18:27 +00:00
Hans-Christoph Steiner
4e28fad55a use constants for names used in the config dict
Hopefully this helps with the Anti-Features case confusion:
* antifeatures
* antiFeatures
* AntiFeatures
2023-07-25 14:31:09 +00:00
Hans-Christoph Steiner
6b82710039 test load_localized_config() with categories 2023-06-02 13:28:53 +02:00
Hans-Christoph Steiner
bc58959315 purge remnants of repo/categories.txt, which is long unused 2023-06-02 13:26:03 +02:00
Hans-Christoph Steiner
1c3a87e002 lint: get Categories from config 2023-06-02 13:26:03 +02:00
Jochen Sprickerhof
27ea667438
Add test for !1370
The bug fixed in !1370 was due to a reuse of the per app anti features
dictionary in the per version one and adding the per version anti
features into it. This was not caught by the test suite as the app
versions are processed from newest to oldest and the test data had only
anti features in the oldest version. This patch adds an anti feature to
a newest version so only those in between should not have it.
2023-06-02 10:26:27 +02:00
Hans-Christoph Steiner
d96de4d8c2 update: test archive_old_apks() with ArchivePolicy: 0 2023-06-01 20:31:26 +02:00
Hans-Christoph Steiner
9ef2088ace
add unit tests 2023-05-30 23:02:36 +02:00
Gregor Düster
337974cbed
metadata: Make ArchivePolicy an interger internally 2023-05-30 23:02:34 +02:00
Hans-Christoph Steiner
fac7ceffe3 metadata: remove non-values from Builds: entries 2023-05-29 16:50:28 +00:00
Hans-Christoph Steiner
b055559df7 metadata: remove STRING/INT conversion on output
The type conversion should all happen in post_parse_yaml_metadata whenever
possible.  Also, when `if` blocks end in `return`, it is clearer if no
`elif` or `else` is used.
2023-05-29 16:50:28 +00:00
Hans-Christoph Steiner
070dae1431 versionCode is an int everywhere since !1176 fixed #332 2023-05-29 16:50:28 +00:00
Hans-Christoph Steiner
e64f121c0c metadata: type conversion happens at parsing, not at writing
These test cases were writing assuming they had to transform the data
format.  That is no longer the case.  Going forward, the parsing process
converts everything to a standardized format.  That will hopefully be
enforceable by the JSON Schema in the future.
2023-05-29 16:50:28 +00:00
Hans-Christoph Steiner
26b2cffdcc metadata: tests for converting Builds: entries for writing out
* The metadata.Builds() class initializes all possible flags, then the
  flags with init values are filtered out when writing out YAML.
* TYPE_SCRIPT flags with one entry will be converted to a string.
2023-05-29 16:50:28 +00:00
Hans-Christoph Steiner
d3521d7374 metadata: case-insensitive sort for AntiFeatures Categories 2023-05-29 16:50:28 +00:00
Hans-Christoph Steiner
2aa0403208 metadata: handle SHA-256 values that parse as decimal ints
https://gitlab.com/fdroid/fdroidserver/-/merge_requests/1350#note_1370665635
2023-05-25 16:53:02 +02:00
Hans-Christoph Steiner
642e444cfa metadata: _normalize_type_list for TYPE_LIST quirks and errors
This should reduce surprises when dealing with filenames in things like
`rm:`.  So any float/int/bool value can be used directly, without quoting.

* A plain str/int/float value is interpreted as a list of one string.
* Dictionaries as values throws error.
* A set is treated like a list.
2023-05-25 16:53:02 +02:00
Hans-Christoph Steiner
4711b632b8 metadata: _normalize_type_int to handle exceptions 2023-05-25 16:53:02 +02:00
Hans-Christoph Steiner
9f606d0fbb metadata: auto-convert YAML special float values: .nan .inf -.inf
Even for people who know what the special floats not-a-number, infinity,
and negative infinity, they don't necessarily know the YAML 1.2 syntax for
these.  I didn't.  And I've spent some quality time fighting things with
those values.  They are also easy to reliably convert to string values.
2023-05-25 16:53:02 +02:00
Hans-Christoph Steiner
8374842faa metadata: normalize TYPE_BOOL to YAML 1.2 booleans
This makes the internal representation always be a boolean, and that also
means that YAML 1.2 booleans will be written out, e.g. rewritemeta.
2023-05-25 16:53:02 +02:00
Hans-Christoph Steiner
aa98d67c86 metadata: test None in post_metadata_parse 2023-05-25 15:10:05 +02:00
Hans-Christoph Steiner
df7c909884 build: add test to ensure Prepare: is only run in VM 2023-05-25 13:03:25 +00:00
Hans-Christoph Steiner
dbf163e03f test to check that the right Builds flag values get removed 2023-05-23 12:11:09 +02:00
Hans-Christoph Steiner
895e0553a0 error if a YAML Anti-Feature conflicts with a localized file
https://gitlab.com/fdroid/fdroidserver/-/issues/683#note_1383597734
2023-05-23 10:47:03 +02:00
Hans-Christoph Steiner
b0c05842d8 metadata: RequiresRoot is boolean, according to docs
"Set this optional field to ‘True’ if the application requires root
privileges to be usable."
https://f-droid.org/docs/Build_Metadata_Reference/#RequiresRoot
2023-05-23 10:47:03 +02:00
Hans-Christoph Steiner
0393e46af9 metadata: handle TYPE_STRINGMAP when writing out YAML 2023-05-23 10:47:03 +02:00
Hans-Christoph Steiner
2cb12f9594 metadata: break out write_yaml to standalone function and add unit tests 2023-05-23 10:47:03 +02:00
Hans-Christoph Steiner
784bebfee9 metadata: keep manually added NoSourceSince in AntiFeatures
If the metadata file contains NoSourceSince:, it is added to the collection
of Anti-Features.  When rewriting the .yml file, NoSourceSince should only
be written into the AntiFeatures: collection if there are manual changes,
e.g. the user had provided translations.
2023-05-23 10:47:03 +02:00
Hans-Christoph Steiner
7c1d7fb4b3 metadata: check error messages are printed for more cases 2023-05-23 10:47:03 +02:00
Hans-Christoph Steiner
061ca38afd define "string map" type for new Anti-Features explanations
closes #683
2023-05-23 10:47:03 +02:00
Jochen Sprickerhof
2488cb5710 scanner: replace global dict by dataclass 2023-05-12 17:58:04 +00:00