Marcus Hoffmann
4cd96d4a9f
use yaml safeloader in tests
...
Try to use CSafeLoader when possible because its significantly faster.
Use the normal Safeloader otherwise. (This mirrors the non-test code
behaviour)
2020-09-10 14:18:36 +02:00
Marcus Hoffmann
768a91370c
publish: use apksigner for signing apks with targetSDK>=30
...
This makes apksigner a hard requirement of the signing procedure.
We'll first try to find a globally installed version from PATH and if
that's not available fall back to using a version from build-tools.
Future TODO: always sign with apksigner, blocked on signature transplant
support for apksigv2/v3
Closes fdroid/fdroidserver#634
Closes fdroid/fdroidserver#827
2020-09-09 18:09:28 +02:00
Marcus Hoffmann
bc6fa986d1
fix vcs test
...
It doesn't need a valid sdk_path, so just set it to something.
The test was failing when ANDROID_HOME wasn't set in the env.
2020-08-25 23:09:28 +02:00
Marcus Hoffmann
34717fe88a
fix typoes
2020-08-25 21:32:54 +02:00
Marcus Hoffmann
059ebd4bc9
tests: add check that we trigger a scanner error without setting it as extlib
2020-08-25 21:32:29 +02:00
Marcus Hoffmann
a301a1ba93
add test for correct added
date for apps
2020-08-25 21:32:29 +02:00
Marcus Hoffmann
fce4721695
add a vcs.TestCase
2020-08-25 21:32:29 +02:00
Marcus Hoffmann
61736f3f50
scanner: add test for #759
2020-08-25 21:32:29 +02:00
Marcus Hoffmann
f46e99a5c4
test for #796
...
The extlib test is in build because it tests the interaction between
prepare_source with a later scan as it is run from build.py
2020-08-25 21:31:07 +02:00
Marcus
8801d37649
Merge branch 'publish_json' into 'master'
...
Some publish.py improvements
Closes #820
See merge request fdroid/fdroidserver!787
2020-08-25 12:33:39 +00:00
Marcus Hoffmann
882f8cfe19
test_check_for_key_collisions: test with an actual collision
...
Genrated with this script:
https://gitlab.com/fdroid/fdroidserver/-/merge_requests/787#note_401275883
2020-08-24 21:19:59 +02:00
Marcus Hoffmann
d9a6bfb0a9
CI: install pyjks as dependency for tests
2020-08-24 21:11:55 +02:00
Marcus Hoffmann
a114c73c2d
publish: factor out the signing key creation into a method
2020-08-24 19:34:08 +02:00
Marcus Hoffmann
7813a17cf8
publish: extract a few functions out of main
...
publish is currently not reusable from other modules as everything is
happening in main. It's also not testable from python unittests.
There's already a function for getting the key_alias, so we can use
that.
Introduce tests for the split out functions.
2020-08-24 19:34:08 +02:00
Michael Pöhn
cb368a674c
remove txt srclib support
2020-08-20 20:40:15 +02:00
Michael Pöhn
2ec90bb490
remove support for rewriting to txt
2020-08-20 20:40:15 +02:00
Hans-Christoph Steiner
29eea3f739
Merge branch 'androguard_minsdk' into 'master'
...
common: use androguard to figure out minSDK version
Closes #816
See merge request fdroid/fdroidserver!785
2020-08-20 12:45:28 +00:00
Marcus Hoffmann
ba28b44ae7
common: use androguard to figure out minSDK version
...
Closes : #816
2020-08-19 19:49:39 +02:00
Hans-Christoph Steiner
f8a1b45df5
gradlew-fdroid: fix random failures due to empty $line, closes #815
2020-08-19 16:28:58 +02:00
Hans-Christoph Steiner
10fa912c16
deploy: remove git remotes not enabled in servergitmirrors
...
If the repo operator removes an item from servergitmirrors, it should auto-
matically remove the remote.
2020-07-30 14:57:19 +02:00
Marcus
1902bfb6fc
Merge branch 'fix_760' into 'master'
...
update.py (and index.py) refactoring
Closes #524 and #760
See merge request fdroid/fdroidserver!756
2020-06-24 22:37:51 +00:00
Marcus Hoffmann
3ebc44c54f
fix tests after they switched to yaml
2020-06-25 00:07:42 +02:00
Marcus Hoffmann
07caa88920
don't include the localized metadata things for /archive
...
We haven't done this so far and it's a potential big change in archive
size and update performance.
2020-06-25 00:07:42 +02:00
Marcus Hoffmann
8c71637d43
update: make copy_triple_t_store_metadata and insert_localized_app_metadata not assume /repo
...
This will enable copying the localized metadata to the archive as well.
2020-06-24 23:53:36 +02:00
Marcus Hoffmann
03881154c6
metadata: make linkresolver an actual object
...
Previously this was magically capturing the apps dict when passing it around as a
function. This also moved the code to the metadata module.
Add a test doing read_metadata where the linkresolver is used. This
happens when the apps we read have a [[app.id]] link to another app.
2020-06-24 23:53:35 +02:00
Marcus Hoffmann
a4177e5ec3
add test for correct whatsnew handling without CVC
2020-06-24 23:53:35 +02:00
Hans-Christoph Steiner
d0f426e076
replace $$srclib$$ with an absolute path
...
closes #725
2020-06-24 22:40:26 +02:00
Hans-Christoph Steiner
238f048257
update: fix crash when liberapay: or open_collective: not in FUNDING.yml
...
closes #799
2020-06-24 21:39:43 +02:00
Hans-Christoph Steiner
82eceebd13
Merge branch 'parse-donation-links-from-funding.yml' into 'master'
...
update: insert donation links based on FUNDING.yml
See merge request fdroid/fdroidserver!754
2020-06-18 04:44:08 +00:00
Hans-Christoph Steiner
2c4e9beacb
scanner: add test with abs/rel paths as run from fdroid build
...
https://gitlab.com/fdroid/fdroidserver/-/issues/791#note_361018050
!767
2020-06-17 10:33:55 +02:00
Hans-Christoph Steiner
62c8fd5999
add Liberapay: field with username as data
...
Liberapay was originally included using a numeric ID, since they had
not yet finalized the public URLs. Now it is a username. So this
logic prefers the username in Liberapay: field, and keeps the old
LiberapayID: to ease migration. LiberapayID: will not override
Liberapay:. Clients are expected to prefer Liberapay: over LiberapayID:
2020-06-16 15:35:28 +02:00
Hans-Christoph Steiner
0183592526
update: insert donation links based on FUNDING.yml
...
GitHub has specified FUNDING.yml, a file to include in a git repo for
pointing people to donation links. Since F-Droid also points people
to donation links, this parses them to fill out Donate:
and OpenCollective:. Specifying those in the metadata file takes
precedence over the FUNDING.yml. This follows the same pattern as how
`fdroid update` includes Fastlane/Triple-T metadata. This lets the
git repo maintain those specific donations links themselves.
https://help.github.com/en/articles/displaying-a-sponsor-button-in-your-repository#about-funding-files
The test file was generated using:
```python
import os, re, yaml
found = dict()
for root, dirs, files in os.walk('.'):
for f in files:
if f == 'FUNDING.yml':
with open(os.path.join(root, f)) as fp:
data = yaml.safe_load(fp)
for k, v in data.items():
if k not in found:
found[k] = set()
if not v:
continue
if isinstance(v, list):
for i in v:
found[k].add(i)
else:
found[k].add(v)
with open('gather-funding-names.yaml', 'w') as fp:
output = dict()
for k, v in found.items():
output[k] = sorted(v)
yaml.dump(output, fp, default_flow_style=False)
```
2020-06-16 15:35:26 +02:00
Marcus
f84818c15e
scanner: add a simple scan for blacklisted classes after build step
...
add com.android.billing to blacklist, see
https://gitlab.com/fdroid/fdroiddata/-/issues/2070#note_360611289
2020-06-15 18:03:19 +00:00
Hans-Christoph Steiner
0837289935
scanner: fix regex for matching URLs in gradle maven{} blocks
...
closes #465
This script generated gradle-maven-blocks.yaml:
```python
import os
import re
import yaml
pat = re.compile(r'\smaven\s*{[^}]+}')
finds = set()
for root, dirs, files in os.walk('.'):
for f in files:
if '.gradle' in f:
with open(os.path.join(root, f), errors='surrogateescape') as fp:
contents = fp.read()
for m in pat.findall(contents):
finds.add(m)
with open('finds.yaml', 'w') as fp:
yaml.dump(sorted(finds), fp, default_flow_style=False)
```
2020-06-10 19:42:23 +02:00
Hans-Christoph Steiner
6590f3869e
scanner: error/warn on dex/gz/zip, closes #394
2020-06-10 19:42:23 +02:00
Hans-Christoph Steiner
ee54dbce87
scanner: safely check options, for things using this API
...
https://gitlab.com/fdroid/fdroidserver/-/issues/771#note_353495799
2020-06-10 19:42:23 +02:00
Hans-Christoph Steiner
75acf63efa
scanner: remove all gradle wrapper files
2020-06-10 19:42:23 +02:00
Hans-Christoph Steiner
d898ad0412
scanner: make AARs and JARs trigger an error
...
refs #491
2020-06-10 19:42:23 +02:00
Michael Pöhn
a97b3ca4dd
implement plugin system review suggestsions
2020-06-10 18:43:23 +02:00
Michael Pöhn
77167e098e
plugin system: regex instead of import bases plugin parsing
2020-06-10 18:43:11 +02:00
Michael Pöhn
b257a3411a
stick to default python sys.path
2020-06-10 12:11:47 +02:00
Michael Pöhn
bf815251ec
rough plugin system implementation
2020-06-10 12:11:47 +02:00
Hans-Christoph Steiner
0d1fddb82e
convert inline tests to .yml
2020-06-10 10:43:27 +02:00
Hans-Christoph Steiner
f0fb3b64f8
remove obselete tests
2020-06-10 10:43:25 +02:00
Hans-Christoph Steiner
37f37ebd88
use default accepted_formats since all the files are .yml anyway
2020-06-10 10:43:21 +02:00
Hans-Christoph Steiner
b63e9e68c5
move all test metadata files to .yml format
...
for f in metadata/*.txt; do ../fdroid rewritemeta --to yml $(basename $f | sed 's,\.txt,,'); done
2020-06-10 09:56:31 +02:00
Hans-Christoph Steiner
13d9a122bf
metadata: validate STRING and INT build field types
...
This converts float/int to string for things like commit: or versionName:.
For versionCode, which must be an integer, it throws an exception if the
data is any other type.
2020-06-02 22:06:01 +02:00
Hans-Christoph Steiner
7a84679b0d
gradlew-fdroid: fix parsing when files have Windows linefeeds
...
The new test file should have Windows linefeeds, if I got the git config
correct.
2020-05-29 21:55:10 +02:00
Hans-Christoph Steiner
fc885c9b5c
scanner: test for get_gradle_compile_commands()
2020-05-27 22:09:52 +02:00
Hans-Christoph Steiner
95c3ab2454
skip yamllint test if yamllint is not installed
...
!721
2020-05-27 22:07:17 +02:00
Hans-Christoph Steiner
67332d83a5
scanner: add --json option for outputting machine readable results
...
* makes per-build entries in per-app entries
* `fdroid scanner --json --verbose` will output logging messages to stderr
* removed " at line N" from one message to make them uniform keys
* this will be used in issuebot
This is a second attempt with tests for how `fdroid build` calls the
scanner functions. closes #771 . It was previously merged in !748 then
reverted in 68c072c72e
2020-05-27 08:35:14 +02:00
Hans-Christoph Steiner
5b9944fcde
add test for things fdroid build
cleans
...
This needs a lot of mocking because build.build_local() is a
gianormous single function.
2020-05-26 09:48:55 +02:00
Marcus Hoffmann
68c072c72e
Revert "scanner: add --json option for outputting machine readable results"
...
This reverts commit cdaf62e5d9
.
See: https://gitlab.com/fdroid/fdroidserver/-/merge_requests/748#note_347769371
2020-05-24 10:24:40 +02:00
Hans-Christoph Steiner
cdaf62e5d9
scanner: add --json option for outputting machine readable results
...
* makes per-build entries in per-app entries
* `fdroid scanner --json --verbose` will output logging messages to stderr
* removed " at line N" from one message to make them uniform keys
* this will be used in issuebot
2020-05-14 21:58:48 +02:00
Hans-Christoph Steiner
a0e3b01e94
metadata: parsed srclibs must always return a dict as the container
2020-05-14 16:28:13 +02:00
Michael Pöhn
b076e8cba7
add srclib spec parser
2020-04-25 17:22:48 +02:00
Michael Pöhn
d24484a950
simple testcase for common.run_yamllint
2020-04-24 15:47:47 +02:00
Michael Pöhn
4e69ff582f
run yamllint when parsing failed and also on fdroid lint runs
2020-04-24 15:47:31 +02:00
Michael Pöhn
5741e6930b
yml srclibs: support Subdir as list
2020-04-16 11:31:59 +02:00
Michael Pöhn
1ac7d612b1
yml srclibs: support multiline Prepare
2020-04-16 11:31:59 +02:00
Michael Pöhn
58776da694
get tests working
2020-04-16 11:31:59 +02:00
Michael Pöhn
ee3d8d2f18
srclib: yml parsing + tests for yml and txt
2020-04-16 11:31:59 +02:00
Michael Pöhn
fa7885063b
scrlib: add test for getsrclib
2020-04-16 11:31:59 +02:00
Michael Pöhn
286220fe11
srclib: add test for getsrclibvcs
2020-04-16 11:31:59 +02:00
Michael Pöhn
2c86d38028
srclib: no spaces in dict key
2020-04-16 11:31:59 +02:00
Michael Pöhn
c6a97939f1
rename parse_srclib to parese_txt_srclib + test case
2020-04-16 11:31:59 +02:00
Hans-Christoph Steiner
b7901952a1
deploy: make androidobservatory and virustotal functions reusable
...
This should not change the logic at all, just make the loop runs into
standalone functions.
2020-03-11 22:13:39 +01:00
Hans-Christoph Steiner
ab2291475b
import: mv reusable functions to common.py to avoid import_proxy.py
...
import is a strict keyword in Python, so it is not possible to import a
module called 'import', even with things like:
* import fdroidserver.import
* from fdroidserver import import
2020-03-11 13:41:13 +01:00
Marcus
9d24f2e4a7
add opencollective metadata and index field
2020-03-10 14:56:03 +00:00
Hans-Christoph Steiner
202291d66c
integration test for creating and deploying status JSON files
...
----------------------------
2020-02-19 14:50:37 +01:00
Hans-Christoph Steiner
4bb590b6e5
publish: write status JSON
2020-02-19 14:50:24 +01:00
Hans-Christoph Steiner
271b74af7d
fix remove_signing_keys() for Kotlin gradles files (*.gradle.kts)
2020-02-13 22:32:51 +01:00
Hans-Christoph Steiner
3de2d0f56f
add basic test suite for gradlew-fdroid
...
!707
fdroiddata#6216
The se.manyver app is licensed MPL, the files came from:
81d247a6cd
2020-02-13 22:32:51 +01:00
Hans-Christoph Steiner
83ffeb855f
prefer build.gradle with Android Plugin as source of package/version/code
...
These days, the location that overrides all the others is in the android{}
block of the build.gradle file that loads the com.android.application
plugin. So this should be the preferred place to read these values.
test files GPL licensed: https://github.com/Integreight/1Sheeld-Android-App
2020-02-13 22:32:51 +01:00
Hans-Christoph Steiner
1153ac24fd
import: overhaul URL validation to use urllib.parse
...
Python provides us a lovely URL parser with some level of validation built
in. The parsed URL is then much easier to validate.
2020-02-13 13:51:52 +01:00
Hans-Christoph Steiner
e9a6c84efd
import: split URL parsing from code cloning
...
This makes things testable and easier to follow.
2020-02-13 13:51:52 +01:00
Hans-Christoph Steiner
bfe587979d
import: make it work most of the time with git repos
...
This includes real tests too.
2020-02-13 13:51:52 +01:00
Hans-Christoph Steiner
5fb368916f
index: fix no JAR test case that fails on example.org
...
http://example.org/index-v1.jar now returns the HTTP header
"Content-Encoding: gzip" but then the reply is plain HTML. That
triggers a ContentDecodingError instead of an HTTPError, so this
changes the test to success on any RequestsException.
2020-02-05 20:33:40 +01:00
Hans-Christoph Steiner
427856b5fd
tests: use same mirrors order in all tests
...
This makes it so running `../fdroid update --nosign --pretty` in tests/ no
longer creates a diff in the tests/*/index* files. It matches the order
set in tests/run-tests.
2020-01-31 15:38:10 +01:00
Hans-Christoph Steiner
6ce22bff56
update: strip newlines from name/summary/video entries
...
These entries are hardcoded as a single line in all the app stores, so
newlines should be stripped to get the data simple to use. This is in
contrast with the on-disk format for Fastlane and Triple-T, which includes
a newline in the title.txt and short_description.txt files. I think all
files in those systems are normalized to end in a newline.
2020-01-31 15:37:59 +01:00
Hans-Christoph Steiner
fdede914cd
tests: properly name CheckupdatesTest class
2020-01-31 15:37:32 +01:00
Hans-Christoph Steiner
3df276cc3c
fix all bandit B310 urllib_urlopen
...
"Audit url open for permitted schemes. Allowing use of ‘file:’’ or custom
schemes is often unexpected."
https://bandit.readthedocs.io/en/latest/blacklists/blacklist_calls.html#b310-urllib-urlopen
2020-01-31 15:37:30 +01:00
Marcus Hoffmann
8961485125
remove dscanner subcommand
...
It's unused and unmaintained. It could potentially be revived as a
plugin at a later point.
2020-01-31 14:17:12 +01:00
Michael Pöhn
f2a80ffa3d
fix tests on old python version
2020-01-28 11:43:09 +01:00
Michael Pöhn
b25eeb66a1
fix code style
2020-01-28 11:43:09 +01:00
Michael Pöhn
f21481ca81
add some simple tests for main
2020-01-28 11:43:09 +01:00
Hans-Christoph Steiner
e76a0c9d6a
git_mirror_size_limit config option to set max git mirror size
...
GitHub and GitLab have some kinds of limits on how big a git repo can be,
this makes that option configurable. This also is very useful for tests.
2020-01-14 11:36:08 +01:00
Gerhard Olsson
cf9bff1d56
AutoUpdateMode: Do not include + in suffix
...
Add suffix to version only
2020-01-02 13:27:12 +00:00
Marcus Hoffmann
83526e09a3
update: fix unarchiving of allowed disabled algorithm
...
6d0b1bbe6fae0909683f2c6a154515bc4bfcb674 didn't handle the
allow_disabled_algorithm case at all, so we add it back.
This additionally fixes a (previously existing) bug where setting
allow_disabled_algorithms to True didn't move apks back from archive to
repo. Introduce a new test for this.
The disabled_algorithm archiving logic is still all over the place so
ideally that needs a future refactor.
2020-01-01 15:25:43 +01:00
Marcus Hoffmann
7f6efa74f5
tests: add test for correctly handling CVC when archiving
2019-12-24 17:35:54 +01:00
Hans-Christoph Steiner
7d40e89341
checkupdates: split out vercode parsing into testable function
2019-12-03 23:51:48 +01:00
Jochen Sprickerhof
bbee2cf707
Add unit test for string_is_integer()
2019-12-03 21:49:44 +01:00
Mike Hardy
3648ef1f2e
Update scanner.TestCase to match new regex regime
...
scanning for analytics is now handled by firebase/play-services regexes
2019-11-02 18:54:54 +00:00
Hans-Christoph Steiner
0a8af2c55f
Merge branch 'fix-readline-in-srclibs' into 'master'
...
build: fix bad regexs when removing signingConfig from srclibs
See merge request fdroid/fdroidserver!686
2019-10-25 08:18:08 +00:00
Hans-Christoph Steiner
afaa24f2fd
build: fix bad regexs when removing signingConfig from srclibs
...
I went through the source of all apps in fdroiddata for examples, and found
some that use readLine() for things totally unrelated to signingConfigs.
https://gitlab.com/fdroid/fdroiddata/merge_requests/4775#note_234132902
2019-10-23 12:44:47 +02:00
Michael Pöhn
7fa3c34e5b
update tests for fixed log deployment+changelog
2019-10-15 15:19:18 +02:00
Hans-Christoph Steiner
5904aef5a7
update: parse Triple-T v2.x layout
...
Thanks to the very nice example from Piwigo, included here under the GPLv3+
https://github.com/Piwigo/Piwigo-Android
https://gitlab.com/fdroid/fdroiddata/merge_requests/5579#note_225834868
2019-10-04 14:11:40 +02:00
Hans-Christoph Steiner
3801db064a
lint: improve HTTPS check
...
It was missing some domains, so I added another rule. @IzzySoft pointed
out it was redudnant, so this removes the redudant rule and fixes the
original.
https://gitlab.com/fdroid/fdroidserver/merge_requests/681/diffs#note_225263464
2019-10-04 11:12:48 +02:00
Hans-Christoph Steiner
af4e231f7d
lint: enforce HTTPS on GitHub and GitLab pages
...
The always provide HTTPS, so let's enforce it.
2019-10-04 11:12:48 +02:00
Hans-Christoph Steiner
17320c23f4
lint: don't trip up on projects with 'master' in the name
...
https://gitlab.com/fdroid/fdroiddata/merge_requests/5557#note_223283359
2019-10-04 11:12:48 +02:00
Michael Pöhn
d5ab303d83
lint: license override config option + require FSF/OSI approved licenses by default
2019-10-02 22:14:09 +02:00
Hans-Christoph Steiner
c7048f2c39
update: log errors on bad graphics, and then ignore the file
...
Python PIL is not so tolerant, so bad EXIF causes crashes:
File "/var/lib/jenkins/userContent/reproducible/reproducible_fdroid_build_apps/fdroidserver/update.py", line 2088, in main
insert_localized_app_metadata(apps)
File "/var/lib/jenkins/userContent/reproducible/reproducible_fdroid_build_apps/fdroidserver/update.py", line 978, in insert_localized_app_metadata
_strip_and_copy_image(os.path.join(root, f), destdir)
File "/var/lib/jenkins/userContent/reproducible/reproducible_fdroid_build_apps/fdroidserver/update.py", line 754, in _strip_and_copy_image
in_image = Image.open(fp)
File "/usr/lib/python3/dist-packages/PIL/Image.py", line 2687, in open
% (filename if filename else fp))
OSError: cannot identify image file <_io.BufferedReader name='build/org.sw24softwares.starkeverben/fastlane/metadata/android/en-US/images/featureGraphic.png'>
2019-09-26 22:06:21 +02:00
Hans-Christoph Steiner
cf2c1d0270
run-tests needs a source of APKs to function
...
The test runs on jenkins.debian.net were failing because they
specified a dir that did not exist.
2019-09-25 12:25:39 +02:00
Hans-Christoph Steiner
25548023e0
gitlab-ci: check gradle checksums against official list
2019-09-12 14:18:07 +02:00
relan
079754c56c
scanner: add a test for the local Debian Maven repo
2019-08-29 21:33:12 +03:00
Hans-Christoph Steiner
70e7e720b9
update: use graphics filename with hash in index to support caching
...
Using a filename based on the hash of the contents means that the caching
algorithms for fdroidclient and browsers can safely cache the file forever
using the filename, since this guarantees that the contents will never
change for a given filename.
This does not cover screenshots, only icon.png, featureGraphic.png,
tvBanner.png, and promoGraphic.png.
fdroidserver#689
fdroid-website!453
2019-08-29 08:23:51 +02:00
Hans-Christoph Steiner
508af00e84
update: only copy graphics and screenshots if mtime/size has changed
...
Instead of copying every time, trust the filesystem to tell us when the
file has changed.
2019-08-29 08:23:47 +02:00
Hans-Christoph Steiner
f13c41e3ba
update: remove Provides: from template.yml test case
...
!654
dcf3837bcb
2019-08-28 15:57:40 +02:00
Hans-Christoph Steiner
7090ce63dd
Merge branch 'remove-provides-in-yml' into 'master'
...
Remove `Provides:` in yaml metadata
Closes #676
See merge request fdroid/fdroidserver!654
2019-08-28 13:39:59 +00:00
Michael Pöhn
8e5232076f
do not delete yml metadata when raumel not installed
2019-08-27 15:38:40 +02:00
Michael Pöhn
0885303672
add additional rewritemeta test for yml
2019-08-27 15:38:40 +02:00
Michael Pöhn
3951d93196
add rewritemeta.TestCase
2019-08-27 15:38:40 +02:00
Michael Pöhn
39c589fb85
Merge branch 'update-create-template' into 'master'
...
make metadata template behave well for empty values
Closes #681
See merge request fdroid/fdroidserver!657
2019-07-24 11:02:38 +00:00
Michael Pöhn
d0368d0ad8
common add parse_androidmanifests_ignore test
2019-07-23 22:44:45 +02:00
Michael Pöhn
bad888856a
checkupdates: add check_http ignore test
2019-07-23 22:01:18 +02:00
Michael Pöhn
23280b6029
checkupdates add check_http test
2019-07-23 22:01:18 +02:00
Michael Pöhn
1c7af1dc2c
add test for checkupdates_app()
2019-07-23 22:01:18 +02:00
Michael Pöhn
e2fed09af1
make metadata template behaves well for empty values
2019-07-22 01:34:55 +02:00
Michael Pöhn
dcf3837bcb
parse yaml: ignore (and warn) deprecated field: Provides
2019-07-15 15:45:02 +02:00
Michael Pöhn
6e48663230
test that write yaml does not write provides
2019-07-11 03:35:23 +02:00
Michael Pöhn
723bd110a6
test that yaml parsing does not accept provides
2019-07-11 03:35:23 +02:00
Hans-Christoph Steiner
051596dd0d
tests: conditionally disable tests that can't work with apksigner
...
apksigner treats MD5 signatures as valid, fdroid does not.
2019-07-10 14:35:03 +02:00
Hans-Christoph Steiner
26af94974a
Merge branch 'rsync-buildslogs-to-webroot-repo' into 'master'
...
build: rsync buildlogs to <webroot>/repo
See merge request fdroid/fdroidserver!651
2019-07-10 08:55:04 +00:00
Michael Pöhn
2c87b5e6f9
deploy build logs: no timestamps
2019-07-06 16:48:56 +02:00
Michael Pöhn
f30983368c
build: rsync buildlogs to <webroot>/repo
2019-07-06 16:48:56 +02:00
Hans-Christoph Steiner
a248a69692
tests: skip disabled_algorithms test when apksigner is present
...
apksigner doesn't treat MD5 signatures as deprecated, so that portion of
the tests would always fail.
2019-07-04 16:45:50 +02:00
Hans-Christoph Steiner
a9aa8788e0
tests: only run source tarball test if running from git clone
2019-07-03 22:33:15 +02:00
Hans-Christoph Steiner
aa1e958360
tests: only run hooks/pre-commit if its present (not in source tarball)
2019-07-03 20:46:30 +02:00
Hans-Christoph Steiner
a0f5ee661e
tests: common.test_sign_apk requires aapt to run
2019-07-03 09:07:36 +02:00
Hans-Christoph Steiner
57b9d1e316
tests: handle when apksigner considers MD5 signatures valid
2019-07-02 22:17:06 +02:00
Michael Pöhn
66105de833
improve litecoin validation + tests
2019-05-28 11:04:43 +02:00
Michael Pöhn
06cec2041d
improve bitcoin validation regex + testcases
2019-05-07 22:43:05 +02:00
Hans-Christoph Steiner
dd2f9d60f8
publish: fix stupid error in repro-signing and add integration test
...
stoopid mistake in ea84014f9b
reported
by @CiaranG
2019-04-11 14:06:51 +02:00
Marcus
7272689ced
Merge branch 'master' into 'master'
...
Added newer ndks, gradles, latest sdk-license, and update java 1.8 version
See merge request fdroid/fdroidserver!637
2019-04-11 11:42:18 +00:00
Michael Pöhn
67731470cc
Revert "Merge branch 'write-yaml-overhaul' into 'master'"
...
This reverts merge request !630
2019-04-01 10:24:00 +00:00
Taco
457cf22361
Added newer ndks, gradles, latest sdk-license, and update java 1.8 version
2019-03-30 17:10:21 -04:00
Michael Pöhn
2683b37044
yml metadata write: do not use local functions
2019-03-19 01:01:18 +01:00
Michael Pöhn
881a79fa84
test writing all yaml fields
2019-03-18 22:45:35 +01:00
Michael Pöhn
63afc0acb5
use pyyaml for writing metadata instead of ruamel
2019-03-18 22:45:35 +01:00
Hans-Christoph Steiner
d03b121152
update: allow tests to pass when apksigner is not installed
...
This is only for the v2/v3 signatures.
fdroid/fdroidserver#627
2019-02-12 12:43:21 +01:00
Hans-Christoph Steiner
50ca3967cc
update: fix tests when running without apksigner
2019-02-03 15:50:39 +01:00
Hans-Christoph Steiner
17dc231dc9
update: fix running without androguard
...
Soon, we can rip out all the aapt parsing stuff, but not yet!
2019-02-03 15:50:39 +01:00
Michael Pöhn
dd695c650e
update: treat target and min sdk version as int
2019-02-01 09:56:21 +01:00
Hans-Christoph Steiner
d96f5ff660
support APK Signature V2 when apksigner is installed
...
This was done with much help from @uniqx. This is the first level of
supporting APK Signatures v1, v2, and v3. This is enough to include
APKs with any combo of v1/v2/v3 signatures. For this to work at all,
apksigner and androguard 3.3.3+ must be installed.
closes #399
2019-02-01 09:17:56 +01:00
Francesco Cervigni
984d276c1b
Added test aapt output files for build-tools 28.0.3
2019-01-13 19:27:02 +01:00
Michael Pöhn
31ca2092a1
yaml parsing: script build flags can now be lists
2018-11-29 21:53:37 +01:00
Michael Pöhn
c15a7508e7
write yaml script metadata as lists
2018-11-29 21:53:37 +01:00
Michael Pöhn
a21635ae2e
fix metadata test case: write yaml prebuild
2018-11-29 21:53:37 +01:00
Michael Pöhn
723815a25b
fix metadata.Testcase:test_rewrite_yaml_special_build_params
2018-11-29 21:53:37 +01:00
Michael Pöhn
d0a129c216
add test for parsing build field prebuild as string
2018-11-29 21:53:37 +01:00
Michael Pöhn
942de28fa5
yaml metadata: split prebuild build field to list
2018-11-29 21:53:37 +01:00
Hans-Christoph Steiner
57556aceee
remove redundant open() arg: encoding='utf8'
...
By default, open() returns a str:
https://docs.python.org/3/library/functions.html#open
By default, str is UTF-8:
https://docs.python.org/3/library/stdtypes.html#str
This used to matter on Python 2.x, but this code is 3.x only now.
2018-10-19 15:01:34 +02:00
Hans-Christoph Steiner
bfdf581201
import: use valid placeholder values for versionCode/versionName
...
fdroid/fdroidserver!559
closes fdroid/fdroidserver#548
2018-10-10 16:02:34 +02:00
Michael Pöhn
67e46694d3
add test for allowing to write placeholder values to yaml metadata files
2018-10-10 16:02:34 +02:00
Hans-Christoph Steiner
3bea689f74
add another lint field type test case
2018-10-10 16:02:34 +02:00
Hans-Christoph Steiner
e33c1c74b1
tests: include info.zwanenburg.caffeinetile in index tests
2018-10-10 16:02:34 +02:00
Hans-Christoph Steiner
8b251da79f
update: do not set targetSdkVersion if the APK is missing it
...
699b3e4c69
got it wrong for targetSdkVersion.
Also, one confusing thing is that aapt outputs "sdkVersion: '3'" for
com.politedroid_3.apk but no "sdkVersion:" for no.min.target.sdk_987.apk.
F-Droid never really supported running on android-1 or android-2, so it
seems pointless to debug support for them.
2018-10-10 15:29:02 +02:00
Hans-Christoph Steiner
85993eb2f8
lint: check fields for proper type, e.g. list vs. string
...
fdroid/fdroidserver#578
2018-10-09 23:31:24 +02:00
Hans-Christoph Steiner
6b57cb6b7c
fix strict Application ID checks
...
* upper case letters are allowed at all positions
* there must be a "." separator
2018-09-24 17:07:27 +02:00
Hans-Christoph Steiner
11d46072ab
use androguard primitives to speed up finding debuggable flag
...
androguard parses the whole APK before handing the instance back, this uses
the primitives to just find the <application android:debuggable=""> value,
then stop parsing.
#557
2018-09-21 14:56:46 +02:00
Hans-Christoph Steiner
a3cecc16a3
use partial androguard binary XML parsing to speed up APK ID lookup
...
Normally, androguard parses the entire APK before it is possible to get any
values from it. This uses androguard primitives to only attempt to parse
the AndroidManifest.xml, then to quit as soon as it gets what it needs.
This greatly speeds up the parsing (1 minute vs 60 minutes).
fdroid/fdroidserver#557
2018-09-21 14:56:46 +02:00
Hans-Christoph Steiner
4c4da3d176
update: remove 'SET' debug antifeature, closes #565
...
oops, this slipped in in 3011953d0e
Bump the metadata version to purge the cache
2018-09-18 10:57:10 +02:00
Hans-Christoph Steiner
92c4f048de
update: test scan_apk() with both aapt and androguard
...
#568
2018-09-18 10:55:51 +02:00
Hans-Christoph Steiner
807bf3d26b
build: reuse common methods for getting metadata from APKs
...
This splits out the code that gets the list of native ABIs supported, then
uses the standard methods for the rest.
2018-09-18 10:55:51 +02:00
Hans-Christoph Steiner
487c4d02f3
handle package: line output from aapt v28
...
fdroid/fdroiddata!3484
fdroid/fdroiddata!3562
fdroid/fdroidserver!548
2018-09-18 10:55:51 +02:00
Hans-Christoph Steiner
d1acef0405
tests: generate aapt output for every version to make tests easy
2018-09-18 10:55:51 +02:00
Hans-Christoph Steiner
9d12b1dc61
add strict, tested validation of Android/F-Droid package names
...
Android has stricter rules than Java for Package Names, but anything the
Python regex thinks is valid must be valid according to Java's rules too.
https://developer.android.com/studio/build/application-id
2018-09-07 14:17:39 +02:00
Hans-Christoph Steiner
5d161cc9fd
validate appid when reading metadata files
...
The metadata file must be named after the Application ID of the app it is
describing, and Android Application IDs must be valid Java Package Names.
2018-09-03 22:56:08 +02:00
Hans-Christoph Steiner
3011953d0e
convert apkcache from pickle to JSON
...
pickle can serialize executable code, while JSON is only ever pure data.
The APK cache is only ever pure data, so no need for the security risks of
pickle. For example, if some malicious thing gets write access on the
`fdroid update` machine, it can write out a custom tmp/apkcache which would
then be executed. That is not possible with JSON.
This does just ignore any existing cache and rebuilds from scratch. That is
so we don't need to maintain pickle anywhere, and to ensure there are no
glitches from a conversion from pickle to JSON.
closes #163
2018-09-03 18:07:40 +02:00
Hans-Christoph Steiner
4d13a904f3
use defusedxml to avoid DoS attacks while loading XML
2018-08-29 17:44:54 +02:00
Hans-Christoph Steiner
4503e7a92a
replace unneeded eval() call and support negative versionCodes
2018-08-29 17:24:04 +02:00
Hans-Christoph Steiner
05be4bc814
Merge branch '471-lint-check-unknown-keys' into 'master'
...
check for unknown app fields and build flags when parsing yml
Closes #471
See merge request fdroid/fdroidserver!554
2018-08-14 10:50:59 +00:00
Hans-Christoph Steiner
53bd5acb09
Merge branch 'make-python-tests-run-independent-of-cwd' into 'master'
...
Make python tests run independent of cwd
See merge request fdroid/fdroidserver!556
2018-08-14 10:34:47 +00:00
Michael Pöhn
82563850b7
fix test metadata case for Builds app field
2018-08-14 12:31:12 +02:00
Michael Pöhn
6a06944945
better exception and waring message when yaml metadata checks fail; build flags check optional
2018-08-14 12:31:12 +02:00
Michael Pöhn
e83221eb03
improve yaml build flag check cost; added unit test for unknown app fields, builds flags checks when parsing yaml
2018-08-14 12:31:12 +02:00
Michael Pöhn
7d5f4d3ab9
added cwd setup boiler-plate to *.TestCase where missing
2018-08-14 10:34:13 +02:00
Michael Pöhn
f153a61277
made build.TestCase run independant of cwd
2018-08-14 10:10:19 +02:00
Michael Pöhn
37e0dce73b
made index.TestCase run independant of cwd
2018-08-14 10:08:03 +02:00
Michael Pöhn
edb8d105b0
made update.TestCase run independent of cwd
2018-08-14 10:02:15 +02:00
Hans-Christoph Steiner
1b37b521c8
common: fix tests when running without any Android SDK installed
...
fdroid/fdroidserver!552
fdroid/fdroidserver#541
2018-08-07 14:51:42 +02:00
Michael Pöhn
55466f1166
fix fixtures in tests/metadata/dump
2018-08-07 00:02:07 +02:00
Michael Pöhn
5d5617bf40
fix: minor bugs in testcases for: common, metadata, update
2018-08-06 23:53:49 +02:00
Michael Pöhn
28ec93c587
fix: run all testcase in tests/run-tests
2018-08-06 23:53:49 +02:00
Hans-Christoph Steiner
64b999ca19
publish: test case that successfully verifies based on Binaries:
...
This is a stupid test of the process just to exercise the most basic path.
It should always succeed since it is verifying two copes of the same APK.
2018-07-12 23:53:05 +02:00
Hans-Christoph Steiner
b03c4a7353
NoSourceSince Anti-Feature should only be added to index
...
Before, it was being added to the metadata, so that rewritemeta would
output it.
fdroid/fdroidserver!529
#508
2018-07-10 23:32:07 +02:00
Hans-Christoph Steiner
669401640a
tests: only run gpgsign tests if gpg is present and executable
...
This allows the full test suite to run in Debian/Ubuntu autopkgtest with
only the Depends: installed, and nothing from Recommends:.
2018-06-25 10:52:12 +02:00
Michael Pöhn
752f6a4ba2
applied review suggestions for deploying (build) logs
2018-06-19 15:18:13 +02:00
Michael Pöhn
88e64df3ef
deploying build logs to server after each individual build run
2018-06-19 12:24:52 +02:00
Michael Pöhn
4c53c71fcf
added helper function for uploading build logs with rsync
2018-06-19 12:24:52 +02:00
Michael Pöhn
af980fbe7e
added tests for update_serverwebroot
2018-06-19 12:24:52 +02:00
Michael Pöhn
429bc21da7
make common.TestCase run independant of cwd
2018-06-19 12:24:52 +02:00
Hans-Christoph Steiner
ff90c0246e
fix PEP8 W605 invalid escape sequence
...
Python 3.7 will get a lot stricter with escape sequences. They must be
valid.
* https://lintlyci.github.io/Flake8Rules/rules/W605.html
* https://docs.python.org/3/whatsnew/3.6.html#deprecated-python-behavior
2018-05-29 13:51:47 +02:00
Hans-Christoph Steiner
e180e03a5a
tests: fix index-v1.json unicode test
2018-05-28 09:29:11 +02:00
Hans-Christoph Steiner
91068d8e00
tests: rename unicode test file to fix "file name too long" errors
...
pristine-tar and mkdir barf on this file.
2018-05-25 17:27:58 +02:00
Hans-Christoph Steiner
5ff1b5ef37
verify: exit with error code if any APK fails to verify
2018-05-25 11:56:13 +02:00
Hans-Christoph Steiner
0a15a9b65b
tests: fix and test fdroid deploy
2018-05-25 10:32:09 +02:00
Hans-Christoph Steiner
1daa1063c5
tests: keep strerr totally clean on successful runs
...
Debian's autopkgtest defaults to considering any output on stderr as a sign
that the tests failed. it is simple to make this achieve that standard, so
it seems worth it for now at least.
2018-05-25 10:32:01 +02:00
Michael Pöhn
6b1f242d25
added tests for common.calculate_math_string
2018-05-22 13:24:37 +02:00
Marcus Hoffmann
a4b6549927
tests: adapt to spdx v4.0.0
2018-05-16 22:49:57 +02:00