1
0
mirror of https://gitlab.com/fdroid/fdroidserver.git synced 2024-11-04 22:40:12 +01:00
Commit Graph

825 Commits

Author SHA1 Message Date
Hans-Christoph Steiner
9ef2088ace
add unit tests 2023-05-30 23:02:36 +02:00
Gregor Düster
337974cbed
metadata: Make ArchivePolicy an interger internally 2023-05-30 23:02:34 +02:00
Hans-Christoph Steiner
fac7ceffe3 metadata: remove non-values from Builds: entries 2023-05-29 16:50:28 +00:00
Hans-Christoph Steiner
b055559df7 metadata: remove STRING/INT conversion on output
The type conversion should all happen in post_parse_yaml_metadata whenever
possible.  Also, when `if` blocks end in `return`, it is clearer if no
`elif` or `else` is used.
2023-05-29 16:50:28 +00:00
Hans-Christoph Steiner
070dae1431 versionCode is an int everywhere since !1176 fixed #332 2023-05-29 16:50:28 +00:00
Hans-Christoph Steiner
e64f121c0c metadata: type conversion happens at parsing, not at writing
These test cases were writing assuming they had to transform the data
format.  That is no longer the case.  Going forward, the parsing process
converts everything to a standardized format.  That will hopefully be
enforceable by the JSON Schema in the future.
2023-05-29 16:50:28 +00:00
Hans-Christoph Steiner
26b2cffdcc metadata: tests for converting Builds: entries for writing out
* The metadata.Builds() class initializes all possible flags, then the
  flags with init values are filtered out when writing out YAML.
* TYPE_SCRIPT flags with one entry will be converted to a string.
2023-05-29 16:50:28 +00:00
Hans-Christoph Steiner
d3521d7374 metadata: case-insensitive sort for AntiFeatures Categories 2023-05-29 16:50:28 +00:00
Hans-Christoph Steiner
2aa0403208 metadata: handle SHA-256 values that parse as decimal ints
https://gitlab.com/fdroid/fdroidserver/-/merge_requests/1350#note_1370665635
2023-05-25 16:53:02 +02:00
Hans-Christoph Steiner
642e444cfa metadata: _normalize_type_list for TYPE_LIST quirks and errors
This should reduce surprises when dealing with filenames in things like
`rm:`.  So any float/int/bool value can be used directly, without quoting.

* A plain str/int/float value is interpreted as a list of one string.
* Dictionaries as values throws error.
* A set is treated like a list.
2023-05-25 16:53:02 +02:00
Hans-Christoph Steiner
4711b632b8 metadata: _normalize_type_int to handle exceptions 2023-05-25 16:53:02 +02:00
Hans-Christoph Steiner
9f606d0fbb metadata: auto-convert YAML special float values: .nan .inf -.inf
Even for people who know what the special floats not-a-number, infinity,
and negative infinity, they don't necessarily know the YAML 1.2 syntax for
these.  I didn't.  And I've spent some quality time fighting things with
those values.  They are also easy to reliably convert to string values.
2023-05-25 16:53:02 +02:00
Hans-Christoph Steiner
8374842faa metadata: normalize TYPE_BOOL to YAML 1.2 booleans
This makes the internal representation always be a boolean, and that also
means that YAML 1.2 booleans will be written out, e.g. rewritemeta.
2023-05-25 16:53:02 +02:00
Hans-Christoph Steiner
aa98d67c86 metadata: test None in post_metadata_parse 2023-05-25 15:10:05 +02:00
Hans-Christoph Steiner
df7c909884 build: add test to ensure Prepare: is only run in VM 2023-05-25 13:03:25 +00:00
Hans-Christoph Steiner
dbf163e03f test to check that the right Builds flag values get removed 2023-05-23 12:11:09 +02:00
Hans-Christoph Steiner
895e0553a0 error if a YAML Anti-Feature conflicts with a localized file
https://gitlab.com/fdroid/fdroidserver/-/issues/683#note_1383597734
2023-05-23 10:47:03 +02:00
Hans-Christoph Steiner
b0c05842d8 metadata: RequiresRoot is boolean, according to docs
"Set this optional field to ‘True’ if the application requires root
privileges to be usable."
https://f-droid.org/docs/Build_Metadata_Reference/#RequiresRoot
2023-05-23 10:47:03 +02:00
Hans-Christoph Steiner
0393e46af9 metadata: handle TYPE_STRINGMAP when writing out YAML 2023-05-23 10:47:03 +02:00
Hans-Christoph Steiner
2cb12f9594 metadata: break out write_yaml to standalone function and add unit tests 2023-05-23 10:47:03 +02:00
Hans-Christoph Steiner
784bebfee9 metadata: keep manually added NoSourceSince in AntiFeatures
If the metadata file contains NoSourceSince:, it is added to the collection
of Anti-Features.  When rewriting the .yml file, NoSourceSince should only
be written into the AntiFeatures: collection if there are manual changes,
e.g. the user had provided translations.
2023-05-23 10:47:03 +02:00
Hans-Christoph Steiner
7c1d7fb4b3 metadata: check error messages are printed for more cases 2023-05-23 10:47:03 +02:00
Hans-Christoph Steiner
061ca38afd define "string map" type for new Anti-Features explanations
closes #683
2023-05-23 10:47:03 +02:00
Jochen Sprickerhof
2488cb5710 scanner: replace global dict by dataclass 2023-05-12 17:58:04 +00:00
Hans-Christoph Steiner
8bc9a3da73 test_parse_yaml_metadata_continue_on_warning checks logging calls 2023-05-11 10:37:45 +02:00
Hans-Christoph Steiner
86b643f87b metadata: test to check that metadata/*.yml overrides .fdroid.yml
This actually uncovered that .fdroid.yml isn't really working. But that is
a problem for another day.
2023-05-11 10:37:45 +02:00
Hans-Christoph Steiner
bb99986630 metadata: fix crash if .fdroid.yml but its not a git repo 2023-05-11 10:37:45 +02:00
Hans-Christoph Steiner
d5a1439457 lint: Anti-Features validator uses names from config 2023-05-11 10:29:04 +02:00
Hans-Christoph Steiner
d6dba05ec3 test load_localized_locale() and translated Anti-Features 2023-05-10 13:34:09 +02:00
Hans-Christoph Steiner
f9864dc3a2 rewritemeta: split into remove_blank_flags_from_builds()
This takes this key bit of functionality, splits it out as its own
function, and adds some unit tests.
2023-05-10 09:21:14 +02:00
Hans-Christoph Steiner
f7830a41f1 deploy: ensure mirrors and binary transparency always create 'master'
If there was a global default on a machine that was something other than
'master', these things would crash with:

Traceback (most recent call last):
  File "/home/hans/code/fdroid/server/fdroid", line 22, in <module>
    fdroidserver.__main__.main()
  File "/home/hans/code/fdroid/server/fdroidserver/__main__.py", line 230, in main
    raise e
  File "/home/hans/code/fdroid/server/fdroidserver/__main__.py", line 211, in main
    mod.main()
  File "/home/hans/code/fdroid/server/fdroidserver/deploy.py", line 833, in main
    push_binary_transparency(BINARY_TRANSPARENCY_DIR,
  File "/home/hans/code/fdroid/server/fdroidserver/deploy.py", line 705, in push_binary_transparency
    local.pull('master')
  File "/usr/lib/python3/dist-packages/git/remote.py", line 1045, in pull
    res = self._get_fetch_info_from_stderr(proc, progress, kill_after_timeout=kill_after_timeout)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/git/remote.py", line 848, in _get_fetch_info_from_stderr
    proc.wait(stderr=stderr_text)
  File "/usr/lib/python3/dist-packages/git/cmd.py", line 604, in wait
    raise GitCommandError(remove_password_if_present(self.args), status, errstr)
git.exc.GitCommandError: Cmd('git') failed due to: exit code(1)
  cmdline: git pull -v -- local master
  stderr: 'fatal: couldn't find remote ref master'
2023-05-10 09:21:14 +02:00
Hans-Christoph Steiner
0735bfa7e5 remove obsolete test case 2023-05-09 20:45:16 +02:00
Hans-Christoph Steiner
43b278b9d6
build: fix loading config files in tests
The new test case works with the default config.
2023-05-05 09:49:34 +02:00
Hans-Christoph Steiner
1e4e2489aa
add keep_when_not_allowed config/option 2023-05-05 09:49:32 +02:00
Hans-Christoph Steiner
386fb55b99
keep old test case intact 2023-05-05 09:49:27 +02:00
Jason A. Donenfeld
26472c22ce build: check AllowedAPKSigningKeys in reproducible build scenario
The builder should check the `AllowedAPKSigningKeys` at build time, so
that the CI can check if somebody gives a wrong value that doesn't match
a compared RB binary.  In the event it fails, it gives useful
information, and in the event it succeeds, it makes it clear that this
build has verification back to the developer's original key.

Also, add tests for this to the test suite.
2023-05-05 06:14:31 +00:00
Hans-Christoph Steiner
9a9705a667 update test_read_metadata to use ruamel.yaml and YAML 1.2
I tried to get this to indent the .yaml files properly so yamllint defaults
work with tests/metadata/dump/*.yaml, but it didn't take for some reason:

    yaml.indent(mapping=4, sequence=4, offset=2)
2023-05-02 13:52:38 +02:00
Hans-Christoph Steiner
28ea6cea7f add tests of TYPE_LIST parsing 2023-05-02 13:52:06 +02:00
Hans-Christoph Steiner
0b3fd725c3 metadata.TestCase: metadata.warnings_action = 'error' as default
The default state for the tests should be to fail on errors.
2023-05-02 13:51:44 +02:00
Hans-Christoph Steiner
8300ed051b ruamel.yaml is now required and the packages are all new enough 2023-05-02 13:41:39 +02:00
Hans-Christoph Steiner
e5fda54693 add test_check_metadata_AntiFeatures 2023-05-02 13:37:07 +02:00
linsui
1a0a0a1839 add litecoin address native segwit format 2023-05-02 11:31:34 +00:00
Hans-Christoph Steiner
1bcd9a8489 metadata: handle empty files and dicts, and error out on non-dicts 2023-04-28 10:55:20 +02:00
Hans-Christoph Steiner
2b81a66b79 App.comments is no more since !772 2023-04-28 10:44:19 +02:00
Hans-Christoph Steiner
dbe21b2b94 metadata: transform all TYPE_STRING values w/ _normalize_type_string()
Before this, there were separate post-parse paths for app-fields versus
build-flags.  This makes all TYPE_STRING values always go through the same
post-parse code path.
2023-04-28 10:44:19 +02:00
Hans-Christoph Steiner
a8531a03a6 metadata: refactor into one function to do YAML post processing
It used to be that there had to be separate post processing steps depending
on whether it was parsing .txt or .yml.  The .txt format is long gone!

!772
2023-04-28 10:44:19 +02:00
Hans-Christoph Steiner
3869e1374b metadata: force TYPE_STRING fields to string in internal dict
* YAML 1.2's boolean is 'true' so this makes the conversion correct.
* rewritemeta would also have to be changed to support this.
2023-04-27 21:51:10 +02:00
Hans-Christoph Steiner
c0ae09e0df metadata: remove strange app arg construct from parse_yaml_metadata()
My guess is that this is some kind of vestige of the old code structure,
back when there was .txt and .yml formats.  This makes it a normal Python
function: input as arg, return value is the result.
2023-04-27 21:51:10 +02:00
Hans-Christoph Steiner
1166258145 map out type conversions in metadata.PostMetadataParseTest suite 2023-04-27 21:50:49 +02:00
Hans-Christoph Steiner
27a0df9ddb metadata: failfast=False like the rest of the tests 2023-04-27 21:48:27 +02:00