1
0
mirror of https://gitlab.com/fdroid/fdroidserver.git synced 2024-11-04 22:40:12 +01:00
Commit Graph

796 Commits

Author SHA1 Message Date
Hans-Christoph Steiner
f9864dc3a2 rewritemeta: split into remove_blank_flags_from_builds()
This takes this key bit of functionality, splits it out as its own
function, and adds some unit tests.
2023-05-10 09:21:14 +02:00
Hans-Christoph Steiner
f7830a41f1 deploy: ensure mirrors and binary transparency always create 'master'
If there was a global default on a machine that was something other than
'master', these things would crash with:

Traceback (most recent call last):
  File "/home/hans/code/fdroid/server/fdroid", line 22, in <module>
    fdroidserver.__main__.main()
  File "/home/hans/code/fdroid/server/fdroidserver/__main__.py", line 230, in main
    raise e
  File "/home/hans/code/fdroid/server/fdroidserver/__main__.py", line 211, in main
    mod.main()
  File "/home/hans/code/fdroid/server/fdroidserver/deploy.py", line 833, in main
    push_binary_transparency(BINARY_TRANSPARENCY_DIR,
  File "/home/hans/code/fdroid/server/fdroidserver/deploy.py", line 705, in push_binary_transparency
    local.pull('master')
  File "/usr/lib/python3/dist-packages/git/remote.py", line 1045, in pull
    res = self._get_fetch_info_from_stderr(proc, progress, kill_after_timeout=kill_after_timeout)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/git/remote.py", line 848, in _get_fetch_info_from_stderr
    proc.wait(stderr=stderr_text)
  File "/usr/lib/python3/dist-packages/git/cmd.py", line 604, in wait
    raise GitCommandError(remove_password_if_present(self.args), status, errstr)
git.exc.GitCommandError: Cmd('git') failed due to: exit code(1)
  cmdline: git pull -v -- local master
  stderr: 'fatal: couldn't find remote ref master'
2023-05-10 09:21:14 +02:00
Hans-Christoph Steiner
0735bfa7e5 remove obsolete test case 2023-05-09 20:45:16 +02:00
Hans-Christoph Steiner
43b278b9d6
build: fix loading config files in tests
The new test case works with the default config.
2023-05-05 09:49:34 +02:00
Hans-Christoph Steiner
1e4e2489aa
add keep_when_not_allowed config/option 2023-05-05 09:49:32 +02:00
Hans-Christoph Steiner
386fb55b99
keep old test case intact 2023-05-05 09:49:27 +02:00
Jason A. Donenfeld
26472c22ce build: check AllowedAPKSigningKeys in reproducible build scenario
The builder should check the `AllowedAPKSigningKeys` at build time, so
that the CI can check if somebody gives a wrong value that doesn't match
a compared RB binary.  In the event it fails, it gives useful
information, and in the event it succeeds, it makes it clear that this
build has verification back to the developer's original key.

Also, add tests for this to the test suite.
2023-05-05 06:14:31 +00:00
Hans-Christoph Steiner
9a9705a667 update test_read_metadata to use ruamel.yaml and YAML 1.2
I tried to get this to indent the .yaml files properly so yamllint defaults
work with tests/metadata/dump/*.yaml, but it didn't take for some reason:

    yaml.indent(mapping=4, sequence=4, offset=2)
2023-05-02 13:52:38 +02:00
Hans-Christoph Steiner
28ea6cea7f add tests of TYPE_LIST parsing 2023-05-02 13:52:06 +02:00
Hans-Christoph Steiner
0b3fd725c3 metadata.TestCase: metadata.warnings_action = 'error' as default
The default state for the tests should be to fail on errors.
2023-05-02 13:51:44 +02:00
Hans-Christoph Steiner
8300ed051b ruamel.yaml is now required and the packages are all new enough 2023-05-02 13:41:39 +02:00
Hans-Christoph Steiner
e5fda54693 add test_check_metadata_AntiFeatures 2023-05-02 13:37:07 +02:00
linsui
1a0a0a1839 add litecoin address native segwit format 2023-05-02 11:31:34 +00:00
Hans-Christoph Steiner
1bcd9a8489 metadata: handle empty files and dicts, and error out on non-dicts 2023-04-28 10:55:20 +02:00
Hans-Christoph Steiner
2b81a66b79 App.comments is no more since !772 2023-04-28 10:44:19 +02:00
Hans-Christoph Steiner
dbe21b2b94 metadata: transform all TYPE_STRING values w/ _normalize_type_string()
Before this, there were separate post-parse paths for app-fields versus
build-flags.  This makes all TYPE_STRING values always go through the same
post-parse code path.
2023-04-28 10:44:19 +02:00
Hans-Christoph Steiner
a8531a03a6 metadata: refactor into one function to do YAML post processing
It used to be that there had to be separate post processing steps depending
on whether it was parsing .txt or .yml.  The .txt format is long gone!

!772
2023-04-28 10:44:19 +02:00
Hans-Christoph Steiner
3869e1374b metadata: force TYPE_STRING fields to string in internal dict
* YAML 1.2's boolean is 'true' so this makes the conversion correct.
* rewritemeta would also have to be changed to support this.
2023-04-27 21:51:10 +02:00
Hans-Christoph Steiner
c0ae09e0df metadata: remove strange app arg construct from parse_yaml_metadata()
My guess is that this is some kind of vestige of the old code structure,
back when there was .txt and .yml formats.  This makes it a normal Python
function: input as arg, return value is the result.
2023-04-27 21:51:10 +02:00
Hans-Christoph Steiner
1166258145 map out type conversions in metadata.PostMetadataParseTest suite 2023-04-27 21:50:49 +02:00
Hans-Christoph Steiner
27a0df9ddb metadata: failfast=False like the rest of the tests 2023-04-27 21:48:27 +02:00
Hans-Christoph Steiner
6620bf0208
enable black on tests: api, gpgsign, main, net, vcs 2023-04-27 11:34:14 +02:00
Hans-Christoph Steiner
c7070b2799
index.TestCase: manually convert to black code format
* switch to `from fdroidserver import common, index, signindex...`
2023-04-27 11:34:07 +02:00
Hans-Christoph Steiner
98decf8cfa index: use 'fake.url' as test domain when no networking is used 2023-04-27 10:35:09 +02:00
Hans-Christoph Steiner
7c692a4532 index-v2 'mirrors' fully settable from config
This lets mirrors: in config.yml be the same list-of-dicts format as it is
in index-v2.  This also includes a data format conversion to maintain the
right format for the old, unchanging index v0 and v1 formats.

#928
#1107
2023-04-27 07:53:02 +00:00
Hans-Christoph Steiner
ceef07d2f2 test of current internal format of mirrors list in config 2023-04-27 07:53:02 +00:00
Hans-Christoph Steiner
ad960c6ecd
remove last maven: quirks and make it always a string
It turns out that the maven: field was originally declared as a TYPE_STRING,
given that it was not given a different type in metadata.py's 	flagtypes.
The code was confused because it was given a default value of `False` rather
than `None` as the rest of the TYPE_STRING fields have.

This construct in build.py means maven: should always be a string:

        if '@' in build.maven:
            maven_dir = os.path.join(root_dir, build.maven.split('@', 1)[1])
        else:
            maven_dir = root_dir
2023-04-24 11:08:48 +02:00
Hans-Christoph Steiner
a107e810d4
build: add test case for maven: builds 2023-04-24 11:08:46 +02:00
Hans-Christoph Steiner
7641c30389
add real test of moving files with deploy.update_serverwebroot() 2023-04-24 11:08:42 +02:00
Gregor Düster
d971f19e92
metadata: Remove code ensuring YAML 1.2 compatibility
This is no longer necessary b/c the current YAML parser, ruamel.yaml,
implements YAML 1.2 in contrast to the previous parser PyYAML.
2023-04-18 17:08:44 +02:00
Hans-Christoph Steiner
36d2a8f899
all ndk paths in config must be strings
The paths in the config must be strings because they are used in things
like env vars where they must be strings.  Plus lots of other places in the
code assumes they are strings.  This is the first step to defining the
border of where paths can be pathlib.Path() and where they must be strings.
2023-04-11 13:54:19 +02:00
Hans-Christoph Steiner
898624dcac
common.get_android_tools_versions() for modern NDK dir layout
NDK r10e is still in use, but is so old, it does not have the modern
sdkmanager package metadata.
2023-04-11 13:54:17 +02:00
Hans-Christoph Steiner
0dd5a7db64
remove NDK download handling in favor of fdroid/sdkmanager 2023-04-11 13:54:05 +02:00
Jochen Sprickerhof
d59a88f844 Strip ipfsCIDv1 before testing index-v2 2023-04-11 10:29:27 +00:00
Hans-Christoph Steiner
94badac317 set skipUnless for test that relies on ANDROID_HOME
This job just started failing, but wasn't before:
https://gitlab.com/eighthave/fdroidserver/-/jobs/4060582594

But I look at it, and it looks right that it fails.  So how on earth
was it succeeding before?  Basically the `os.getenv('ANDROID_HOME')`
returns `None` when `ANDROID_HOME` is not set. It is not set in both the
jobs, so how did it not stacktrace before?
2023-04-04 22:41:46 +02:00
Jochen Sprickerhof
889b8cb372 Don't copy per version anti features to all versions
make_v0() modified the apps data structure to copy an anti feature for a
specific version to all versions resulting in index-v1 and -v2 to
contain wrong anti feature annotations. This patch fixes this and adds a
test that the data structure is no longer modified.

The bug shadowed bugs in the AF implementation of -v1 and -v2 resulting
in not coping the version specific data. This is corrected as well.
This is also tested now.

For -v2 the AF dict is now sorted to make the result reproducible.

Finally The NoSourceSince AF was added as a per version and overall AF
in -v1 and is now only applied as an overall AF and the test is updated
accordingly.
2023-04-04 14:20:41 +00:00
Jochen Sprickerhof
484f289919 index.TestCase: support more arguments (like -k) 2023-04-04 14:20:41 +00:00
Hans-Christoph Steiner
8f4167660e disable pylint no-member on tests/extra/manual-vmtools-test.py
This script is some of the only documentation of certain processes that
run in the buildserver setup.  It is not really maintained, but is still
here as a kind of reference.
2023-03-20 17:38:10 +01:00
FestplattenSchnitzel
a4c1ca48ad pylint: Reenable R1722 consider-using-sys-exit
* R1722: Consider using 'sys.exit' instead (consider-using-sys-exit)
2023-03-20 16:33:16 +01:00
FestplattenSchnitzel
abc752314b pylint: Reenable E1101 no-member 2023-03-20 16:33:16 +01:00
Hans-Christoph Steiner
b92e280eab fix code format for new black rule 2023-03-20 14:47:48 +01:00
Hans-Christoph Steiner
e6ff596b4c update: handle messed up namespaces in AndroidManifest.xml 2023-03-20 13:24:59 +00:00
Hans-Christoph Steiner
b1f482009a
tame verbose logging in index tests by default 2023-03-09 12:35:31 +01:00
Hans-Christoph Steiner
f3e49f4bcb
download_repo_index_v2() for verified downloading of index-v2 2023-03-09 12:35:29 +01:00
Hans-Christoph Steiner
a557764b4d
sign tests/repo/index-v1.jar once per index.TestCase run
Signing files is a slow operation, especially with jarsigner. This speeds
up the full test run from 10-12 seconds to 2-3 seconds, which makes it
possible to run the tests interactively again. And it stops signing the
file entirely for tests that do not even touch that file.

In the long run, it would probably make sense to have each test case sign
the file as it needs it, but that's a much bigger change.
2023-03-09 12:34:57 +01:00
Hans-Christoph Steiner
dd16076651
add verify_jar_signature() to verify entry.jar 2023-03-09 12:34:55 +01:00
Hans-Christoph Steiner
cfe399888b
add new test module for the public API 2023-03-09 12:34:54 +01:00
FestplattenSchnitzel
2dda9db1f1
Remove support for Buildozer 2023-03-09 12:03:57 +01:00
Hans-Christoph Steiner
98448e0481 purge all references to zipalign, that is delegated to other things
Things like apksigner and @obfusk's tools handle this now.
jarsigner is used in the test, since that's the most common use of
`common.find_sdk_tools_cmd()`.

closes #1100
2023-02-22 14:45:51 +01:00
Hans-Christoph Steiner
5af5ed2759 fix test_fill_config_defaults_java for non-amd64 arches 2023-02-21 11:20:16 +01:00