1
0
mirror of https://gitlab.com/fdroid/fdroidserver.git synced 2024-11-05 06:50:10 +01:00
Commit Graph

796 Commits

Author SHA1 Message Date
Hans-Christoph Steiner
24df262f6b handle str and pathlib.Path in getvcs() 2023-02-21 11:20:15 +01:00
Hans-Christoph Steiner
9e9c909380 tests/run-tests: silence err_handler output 2023-02-19 22:54:38 +01:00
Hans-Christoph Steiner
b8f59097f7 purge all vestiges of the unused fdroid stats, closes #839
* for f in locale/*/LC_MESSAGES/fdroidserver.po; do msgattrib --set-obsolete --no-wrap --ignore-file=locale/fdroidserver.pot -o $f $f; done
* sed -i 's, \.\./fdroidserver/stats\.py,,' locale/*/LC_MESSAGES/fdroidserver.po
2023-02-19 22:54:38 +01:00
Hans-Christoph Steiner
58cfce106b add test_sign_apk_fail and test_sign_apk_corrupt 2023-02-19 13:39:48 +01:00
Hans-Christoph Steiner
f468270aa8
support Java 20, which is already in Debian/unstable, closes #1070 2023-02-17 16:35:20 +01:00
Hans-Christoph Steiner
0c9f62a5fe signindex: add JSON check for entry.json
Ran this to generate/refresh the test index files:
`cd tests; ../fdroid update --nosign --pretty`

* converts config.py description to a single line, since the values are no
  longer stripped, so this give the same output.

closes #835
2023-02-17 14:34:53 +00:00
Hans-Christoph Steiner
b697845bda add test case for index.package_metadata() 2023-02-16 21:45:39 +01:00
linsui
58f167b7e5 Add build.binary 2023-02-16 20:16:52 +08:00
linsui
33def096f5 Add postbuild 2023-02-16 12:11:26 +00:00
Simon Brand
4a581bdfb6 Remove path workarounds for old python versions 2023-02-15 19:25:48 +00:00
Jochen Sprickerhof
6118925387 Skip nightly test on Fedora with pip in $HOME 2023-02-08 18:32:13 +01:00
Jochen Sprickerhof
5715e377b0 nightly test: cleaup test directory 2023-02-08 18:14:38 +01:00
Jochen Sprickerhof
55c3a28568 run-tests: make mirror test independent of update test 2023-02-08 18:14:38 +01:00
Jochen Sprickerhof
a667074d89 run-tests: make scanner test independent on importer test 2023-02-08 18:14:38 +01:00
Jochen Sprickerhof
b1fca3f722 run-tests: always cleanup testfiles 2023-02-08 18:14:38 +01:00
Jochen Sprickerhof
6ba40b7349 common tests: use patch.dict for os.environ 2023-02-08 18:14:36 +01:00
Jochen Sprickerhof
d29a486e31 tests: use context manager and/or standard setup temp files 2023-02-08 18:12:35 +01:00
Jochen Sprickerhof
1eeb992118
Don't create unused testfiles directory 2023-02-06 14:52:35 +01:00
Jochen Sprickerhof
d5400549f6
update test: set update options 2023-02-06 14:52:35 +01:00
Hans-Christoph Steiner
9d2cc1ecc5 fix pylint C1803: 'icons_src == {}' can be simplified to 'not icons_src' as an empty dict is falsey (use-implicit-booleaness-not-comparison) 2023-02-02 16:02:49 +01:00
linsui
5858d8fcb4 remove liberapayID 2023-01-16 17:02:56 +00:00
FestplattenSchnitzel
aa71dd57f6
update.TestCase: Avoid creating metadata with duplicate keys 2023-01-15 12:52:41 +01:00
FestplattenSchnitzel
671a264dfe
Use ruamel.yaml in metadata.py and metadata.TestCase
This is to read metadata based on YAML 1.2 rather than 1.1.
2023-01-15 12:52:41 +01:00
Hans-Christoph Steiner
947d94e0a9
deploy: support GitLab Job Artifacts as a mirror 2022-11-16 20:16:32 +01:00
Hans-Christoph Steiner
f24613b701
index: fix git-mirror size check for GitLab Pages
The test case had the wrong folder setup, this was confirmed on a production
repo setup.
2022-11-16 20:16:25 +01:00
Hans-Christoph Steiner
c2567d71d1
nightly: fail if *-nightly git repo is not publicly available 2022-11-16 20:16:21 +01:00
Hans-Christoph Steiner
15bd7057f0 nightly: add tests 2022-11-16 16:50:14 +01:00
Hans-Christoph Steiner
1c5506ae05 nightly: support OpenSSL 3.0 with Paramiko
OpenSSL 3.0 changed the default output format from PKCS#1 to PKCS#8,
which paramiko does not support.

https://www.openssl.org/docs/man3.0/man1/openssl-rsa.html#traditional
https://github.com/paramiko/paramiko/issues/1015
2022-11-16 14:40:08 +01:00
Hans-Christoph Steiner
01f7dfd0b3 index-v2.jar is not a valid file, remove references
entry.jar is the signed file, it references index-v2.json.
2022-11-15 13:22:09 +00:00
linsui
d25995c084 Set ArchivePolicy based on VercodeOperation/signature 2022-11-15 18:50:16 +08:00
Michael Pöhn
24d88705fa 🔍 add scanner_signature_sources config option
This adds the option to configure which set of signatures `fdroid
scanner` should use, by configuring it in `config.yml`. It allows
fetching signatures in our custom json format. It also adds 3 additional
sources: 'suss', 'exodus', 'etip'
2022-11-15 09:20:26 +00:00
Hans-Christoph Steiner
bd51b2e99f build: read VM CPUs/RAM from builder/Vagrantfile
makebuildserver.config.py is no more, builder/Vagrantfile is now where the
CPU and memory is configured for the buildserver VM.  In fact, that was
always the actual place, the makebuildserver.config.py thing was just
confused.

This should have been part of !1222
2022-11-15 08:57:41 +01:00
Jochen Sprickerhof
1bb963d768
jarsigner: allow weak signatures
openjdk-11 11.0.17 in Debian unstable fails to verify weak signatures:

jarsigner -verbose -strict -verify tests/signindex/guardianproject.jar

         131 Fri Dec 02 20:10:00 CET 2016 META-INF/MANIFEST.MF
         252 Fri Dec 02 20:10:04 CET 2016 META-INF/1.SF
        2299 Fri Dec 02 20:10:04 CET 2016 META-INF/1.RSA
           0 Fri Dec 02 20:09:58 CET 2016 META-INF/
 m  ?  48743 Fri Dec 02 20:09:58 CET 2016 index.xml

  s = signature was verified
  m = entry is listed in manifest
  k = at least one certificate was found in keystore
  ? = unsigned entry

- Signed by "EMAILADDRESS=root@guardianproject.info, CN=guardianproject.info, O=Guardian Project, OU=FDroid Repo, L=New York, ST=New York, C=US"
    Digest algorithm: SHA1 (disabled)
    Signature algorithm: SHA1withRSA (disabled), 4096-bit key

WARNING: The jar will be treated as unsigned, because it is signed with a weak algorithm that is now disabled by the security property:

  jdk.jar.disabledAlgorithms=MD2, MD5, RSA keySize < 1024, DSA keySize < 1024, SHA1 denyAfter 2019-01-01, include jdk.disabled.namedCurves
2022-11-14 17:59:56 +01:00
Jochen Sprickerhof
24630dfe68
Drop old getsig.java test
getsig.java was replaced by a Python implementation in 6e2d0a9e (2014)
and the test was only there to compare the results for the transition.

Dropping this as it no longer works starting with 11.0.17+8.
2022-11-14 17:58:37 +01:00
Jochen Sprickerhof
557fe87d44 Run shell scripts with -e (Closes: #1035)
Make sudo, init prebuild, build and Prepare fields lists and only
concatenate them with '; ' before execution. This allows arbitrary
commands inside the fileds (even && and ';') as we don't need to split
the commands again for rewritemeta.
2022-11-03 12:25:31 +00:00
Hans-Christoph Steiner
97f3e78fa8 fixup from 8b484b37bd in !1240 2022-10-31 10:56:02 +01:00
Hans-Christoph Steiner
99e0c28d37 run black on tests/updates.TestCase to fix lint failure 2022-10-31 10:19:43 +01:00
Hans-Christoph Steiner
8b484b37bd
add tests for Fastlane changelogs including default.txt 2022-10-31 09:42:19 +01:00
linsui
618e2c7d72 change VercodeOperation to TYPE_LIST 2022-10-27 19:11:37 +08:00
Jochen Sprickerhof
5e1bdfc278
Fix ipfs test config 2022-10-24 11:43:47 +02:00
FC Stegerman
d1ddd525c1
net.download_file(): retry on errors 2022-10-22 23:15:13 +02:00
Hans-Christoph Steiner
2b5c35829f tests/run-tests: fix "fatal: transport 'file' not allowed"
https://gitlab.com/eighthave/fdroidserver/-/jobs/3202418899
2022-10-20 17:21:43 +02:00
linsui
68b58c043f Change VercodeOperation to list
... so that we can calculate multiple version codes to support multi-arch autoupdate
2022-10-15 17:19:49 +08:00
linsui
0d39169840 checkupdates.TestCase: remove indentation
These assert functions doesn't need to be in the mock context.
2022-10-15 16:09:19 +08:00
linsui
5a28f20301 make versionCode/build.timeout an integer
Co-authored-by: Jochen Sprickerhof <git@jochen.sprickerhof.de>
2022-10-14 08:42:26 +00:00
linsui
6f73a87eb1 fix typo: Build -> Builds 2022-10-14 08:42:26 +00:00
Hans-Christoph Steiner
353ee84a6f
test that get_apk_id throws exceptions (issuebot#60) 2022-10-11 16:36:11 +02:00
Michael Pöhn
e967fc61cf scanner: add regression tests for signautre load logic 2022-10-06 14:50:30 +02:00
Michael Pöhn
b7233dfb2e ignore cache write test case on older python versions 2022-10-06 12:09:07 +02:00
Michael Pöhn
59b1899d79 scanner: switch form iso8601 to unix timestamp for python3.5 support 2022-10-06 12:09:07 +02:00
Michael Pöhn
9560ed955c avoid running into native-date object issue
Seem I ran into this issue: https://bugs.python.org/issue47228  This
change tries to fix it by using utcnow insteas of astimezone.
2022-10-06 12:09:07 +02:00
Michael Pöhn
0921863fa6 scanner: update suss defaults after removal of allowlisted feature 2022-10-06 12:09:07 +02:00
Michael Pöhn
a8bcaa3d70 scanner: implement caching rules for suss 2022-10-06 12:09:06 +02:00
Michael Pöhn
bfcc30b854 add --refresh to scanner 2022-10-06 12:09:06 +02:00
Michael Pöhn
1e6694112a rename to suss 2022-10-06 12:09:06 +02:00
Michael Pöhn
c9b59b525d fix timestamp check; remove dead code 2022-10-06 12:09:05 +02:00
Michael Pöhn
c10633eac5 convert fdroid scanner --exodus to SignatureDataController 2022-10-06 12:08:26 +02:00
Michael Pöhn
d5ef1b2e95 add --clear-cache option to scanner 2022-10-06 12:08:26 +02:00
Michael Pöhn
f56b1f3012 basic downloading for scan_binary signatures 2022-10-06 12:08:23 +02:00
Hans-Christoph Steiner
3de6063a01 scanner: open DEX/ZIP by file magic; throw errors on bad filenames 2022-09-30 17:56:15 +00:00
Hans-Christoph Steiner
aa190d532f scanner.TestCase: manually convert to black code format
I manually changed some code structures to give a decent code format.
2022-09-30 17:56:15 +00:00
Jochen Sprickerhof
f96f247095
Add test for git getref 2022-09-15 13:32:11 +02:00
linsui
ac96e43e13 scanner: add some rules 2022-09-14 15:02:37 +00:00
pmmayero
0ad45a94a8 Addition of IPFS CIDv1 to Index
IPFS CIDv1 is only generated for APKs and "repo files"
2022-09-14 09:28:02 +00:00
Jochen Sprickerhof
483d6da51c
Cleanup tests/dump_internal_metadata_format.py
- Update usage documentation.
- Use argparse.
- Simplify config handling (similar to readmeta.py).
- Drop code for fdroidserver before 0.7.0.
- Reformat with black.
2022-09-08 21:40:26 +02:00
Hans-Christoph Steiner
b854f3bab7 update: fully isolate testInsertStoreMetadata test
The test was failing due to something on my local filesystem, this
runs the test in a new dir every time.
2022-09-08 18:45:30 +02:00
Jochen Sprickerhof
5b79e7aea8
Fix trailing whitespace 2022-09-05 17:50:13 +02:00
Jochen Sprickerhof
c89a9f0e8b
Add timeout argument to requests.(get,post) 2022-09-05 17:50:10 +02:00
Hans-Christoph Steiner
c6cf0468ed nightly: update Raw URLs to fix breakage and avoid redirects 2022-08-26 07:18:19 +00:00
FestplattenSchnitzel
7c89e923f6 Move methods specific to import to it's module 2022-08-24 22:21:37 +02:00
FestplattenSchnitzel
7b7f863c65 [import] Rename to import_subcommand internally
This enables normal import of the module without the need for
workarounds.
2022-08-24 22:21:35 +02:00
linsui
832ed18fca scanner: also scan classpath 2022-08-24 21:34:55 +02:00
Hans-Christoph Steiner
9e58fc8cda
verify: normalize dicts via JSON for reliable comparisons
13016c5d63 in !602 used a set to prevent
duplicate entries, but that worked poorly because it required lots of
data wrapping.  Instead, just normalize to JSON, then equality is easy.
2022-08-24 19:16:33 +02:00
linsui
516a0c2ce8 Fix tag match with GitPython 2022-08-24 10:50:09 +00:00
linsui
f99dbec012 scanner: fix maven url check for kts files 2022-08-08 15:52:47 +00:00
Jochen Sprickerhof
eb79522a36 Fix flake8 E275 missing whitespace after keyword 2022-07-31 09:48:14 +02:00
Jochen Sprickerhof
7822db2881 Catch DefusedXmlException (as ValueError)
defusedxml can't handle the nbsp in the strings.xml (etree can).
2022-07-25 09:30:25 +02:00
Michael Pöhn
ab579be6b5 add some basic tests for scanner.main 2022-07-17 15:52:52 +02:00
Michael Pöhn
42d9ac446c get tests working on ci 2022-07-17 15:50:07 +02:00
Michael Pöhn
1c2b084410 🔧 improve scanner.scan_apk tests
Refactor test function it a TestCase and split into separate test cases.
Fix and improve tests for scanning apks with embedded apks.
2022-07-17 15:50:07 +02:00
Michael Pöhn
07a366a4d6 add tests for scanner.load_exodus_trackers_signatures 2022-07-17 15:50:07 +02:00
Michael Pöhn
a1677b5cb0 add test case for _exodus_compile_signatures 2022-07-17 15:50:07 +02:00
Hans-Christoph Steiner
e2d9dedbb1 build: add test to ensure the flags are passed to the buildserver
!1141
2022-06-08 21:15:51 +02:00
Hans-Christoph Steiner
2bf6848391 test_find_apksigner_system_package_android_home if build_tools is new enough 2022-06-08 20:39:44 +02:00
Hans-Christoph Steiner
fe22958476
run-tests: skip tests that require apksigner when running on Java8
The buildserver VM has not been upgraded yet to bullseye, so it is still on
Debian/stretch.  The buildserver VM does not need to run `fdroid update`,
`fdroid signindex`, etc. so this new apksigner requirement should not
affect app builds even though they are stuck on Debian/stretch.
2022-06-07 16:56:25 +02:00
Hans-Christoph Steiner
3182b77d18
use apksigner to sign index-v2 with modern, supported algorithms
The current signing method uses apksigner to sign the JAR so that it
will automatically select algorithms that are compatible with Android
SDK 23, which added the most recent algorithms:
https://developer.android.com/reference/java/security/Signature

This signing method uses then inherits the default signing algothim
settings, since Java and Android both maintain those.  That helps
avoid a repeat of being stuck on an old signing algorithm.  That means
specifically that this call to apksigner does not specify any of the
algorithms.

The old indexes must be signed by SHA1withRSA otherwise they will no
longer be compatible with old Androids.

apksigner 30.0.0+ is available in Debian/bullseye, Debian/buster-backports,
Ubuntu 21.10, and Ubuntu 20.04 from the fdroid PPA.  Here's a quick way to
test:

for f in `ls -1 /opt/android-sdk/build-tools/*/apksigner | sort ` /usr/bin/apksigner; do printf "$f : "; $f sign --v4-signing-enabled false; done

closes #1005
2022-06-07 16:56:23 +02:00
Hans-Christoph Steiner
7544761e86
index: add test for --nosign config and file generation 2022-06-07 13:22:06 +02:00
Hans-Christoph Steiner
c6dcc82ca4 allow common.get_apk_id() to be used in the API
If a project uses fdroidserver as a library, then just calls
common.get_apk_id(), it will now work.  Before, that project would have had
to include something like `common.config = {}` to avoid a stacktrace.
2022-05-26 22:18:21 +02:00
Hans-Christoph Steiner
2639909f90 add test cases for parsing smartcardoptions config field 2022-05-26 16:49:43 +02:00
Hans-Christoph Steiner
cbd4828561 include index-v2 in signindex tests 2022-05-24 11:35:46 +02:00
Hans-Christoph Steiner
2448f070e9 fix tests and docstring error 2022-05-23 15:34:30 +02:00
Jochen Sprickerhof
b07d23ff5c Don't include disabled apks in the index
This needs a rerun of `fdroid update --clean`.

In case a build is disabled delete_disabled_builds takes care of
deleting it from the repo. But this only works if the apk follows the
normal name pattern. Otherwise it will stay in the folder and be picked
up by process_apks and added to the index.

Closes: #1002
2022-05-23 11:47:53 +00:00
Jochen Sprickerhof
d70e5c2cd9 Index v2 2022-05-23 10:39:17 +00:00
Hans-Christoph Steiner
73c31e6e63
signindex: do not remove index-v1.json after signing
With ~index-v2, the model is changing to offer the plain JSON file for easy
consumption.  Then gpgsign will also provide a detached PGP signature for
systems that would rather verify based on PGP signatures than JAR signatures.

!1080
closes #969
2022-05-17 15:38:06 +02:00
Jochen Sprickerhof
08e3e445fb Merge androguard_test into update.TestCase 2022-05-10 14:40:03 +00:00
Jochen Sprickerhof
e82888837e Fix androguard_test
- SdkToolsPopen expects a list.
- info.guardianproject.urzip.apk is not in the repo.
- scan_apk_aapt was removed in 08931f45 so define output.
2022-05-10 14:40:03 +00:00
Hans-Christoph Steiner
107593d3ba net: improve parsing filenames from URL
This is more relevant to issuebot than fdroidserver, but it can't hurt here
also.
2022-05-09 20:59:15 +02:00
linsui
df1e0e30d5 fix tests 2022-05-07 02:17:26 +00:00
linsui
90bc8e1e8f scan gradle.kts 2022-05-07 00:43:55 +08:00
Jochen Sprickerhof
94ccd022c4 Add apk.embedded_1.apk test 2022-05-03 14:59:52 +00:00