1
0
mirror of https://gitlab.com/fdroid/fdroidserver.git synced 2024-07-14 13:00:08 +02:00
Commit Graph

435 Commits

Author SHA1 Message Date
Michael Pöhn
4e69ff582f run yamllint when parsing failed and also on fdroid lint runs 2020-04-24 15:47:31 +02:00
Michael Pöhn
5741e6930b yml srclibs: support Subdir as list 2020-04-16 11:31:59 +02:00
Michael Pöhn
1ac7d612b1 yml srclibs: support multiline Prepare 2020-04-16 11:31:59 +02:00
Michael Pöhn
58776da694 get tests working 2020-04-16 11:31:59 +02:00
Michael Pöhn
ee3d8d2f18 srclib: yml parsing + tests for yml and txt 2020-04-16 11:31:59 +02:00
Michael Pöhn
fa7885063b scrlib: add test for getsrclib 2020-04-16 11:31:59 +02:00
Michael Pöhn
286220fe11 srclib: add test for getsrclibvcs 2020-04-16 11:31:59 +02:00
Michael Pöhn
2c86d38028 srclib: no spaces in dict key 2020-04-16 11:31:59 +02:00
Michael Pöhn
c6a97939f1 rename parse_srclib to parese_txt_srclib + test case 2020-04-16 11:31:59 +02:00
Hans-Christoph Steiner
b7901952a1 deploy: make androidobservatory and virustotal functions reusable
This should not change the logic at all, just make the loop runs into
standalone functions.
2020-03-11 22:13:39 +01:00
Hans-Christoph Steiner
ab2291475b import: mv reusable functions to common.py to avoid import_proxy.py
import is a strict keyword in Python, so it is not possible to import a
module called 'import', even with things like:

* import fdroidserver.import
* from fdroidserver import import
2020-03-11 13:41:13 +01:00
Marcus
9d24f2e4a7 add opencollective metadata and index field 2020-03-10 14:56:03 +00:00
Hans-Christoph Steiner
202291d66c
integration test for creating and deploying status JSON files
----------------------------
2020-02-19 14:50:37 +01:00
Hans-Christoph Steiner
4bb590b6e5
publish: write status JSON 2020-02-19 14:50:24 +01:00
Hans-Christoph Steiner
271b74af7d fix remove_signing_keys() for Kotlin gradles files (*.gradle.kts) 2020-02-13 22:32:51 +01:00
Hans-Christoph Steiner
3de2d0f56f add basic test suite for gradlew-fdroid
!707
fdroiddata#6216

The se.manyver app is licensed MPL, the files came from:
81d247a6cd
2020-02-13 22:32:51 +01:00
Hans-Christoph Steiner
83ffeb855f prefer build.gradle with Android Plugin as source of package/version/code
These days, the location that overrides all the others is in the android{}
block of the build.gradle file that loads the com.android.application
plugin.  So this should be the preferred place to read these values.

test files GPL licensed: https://github.com/Integreight/1Sheeld-Android-App
2020-02-13 22:32:51 +01:00
Hans-Christoph Steiner
1153ac24fd import: overhaul URL validation to use urllib.parse
Python provides us a lovely URL parser with some level of validation built
in.  The parsed URL is then much easier to validate.
2020-02-13 13:51:52 +01:00
Hans-Christoph Steiner
e9a6c84efd import: split URL parsing from code cloning
This makes things testable and easier to follow.
2020-02-13 13:51:52 +01:00
Hans-Christoph Steiner
bfe587979d import: make it work most of the time with git repos
This includes real tests too.
2020-02-13 13:51:52 +01:00
Hans-Christoph Steiner
5fb368916f
index: fix no JAR test case that fails on example.org
http://example.org/index-v1.jar now returns the HTTP header
"Content-Encoding: gzip" but then the reply is plain HTML. That
triggers a ContentDecodingError instead of an HTTPError, so this
changes the test to success on any RequestsException.
2020-02-05 20:33:40 +01:00
Hans-Christoph Steiner
427856b5fd
tests: use same mirrors order in all tests
This makes it so running `../fdroid update --nosign --pretty` in tests/ no
longer creates a diff in the tests/*/index* files.  It matches the order
set in tests/run-tests.
2020-01-31 15:38:10 +01:00
Hans-Christoph Steiner
6ce22bff56
update: strip newlines from name/summary/video entries
These entries are hardcoded as a single line in all the app stores, so
newlines should be stripped to get the data simple to use.  This is in
contrast with the on-disk format for Fastlane and Triple-T, which includes
a newline in the title.txt and short_description.txt files.  I think all
files in those systems are normalized to end in a newline.
2020-01-31 15:37:59 +01:00
Hans-Christoph Steiner
fdede914cd
tests: properly name CheckupdatesTest class 2020-01-31 15:37:32 +01:00
Hans-Christoph Steiner
3df276cc3c
fix all bandit B310 urllib_urlopen
"Audit url open for permitted schemes. Allowing use of ‘file:’’ or custom
schemes is often unexpected."

https://bandit.readthedocs.io/en/latest/blacklists/blacklist_calls.html#b310-urllib-urlopen
2020-01-31 15:37:30 +01:00
Marcus Hoffmann
8961485125 remove dscanner subcommand
It's unused and unmaintained. It could  potentially be revived as a
plugin at a later point.
2020-01-31 14:17:12 +01:00
Michael Pöhn
f2a80ffa3d fix tests on old python version 2020-01-28 11:43:09 +01:00
Michael Pöhn
b25eeb66a1 fix code style 2020-01-28 11:43:09 +01:00
Michael Pöhn
f21481ca81 add some simple tests for main 2020-01-28 11:43:09 +01:00
Hans-Christoph Steiner
e76a0c9d6a
git_mirror_size_limit config option to set max git mirror size
GitHub and GitLab have some kinds of limits on how big a git repo can be,
this makes that option configurable.  This also is very useful for tests.
2020-01-14 11:36:08 +01:00
Gerhard Olsson
cf9bff1d56 AutoUpdateMode: Do not include + in suffix
Add suffix to version only
2020-01-02 13:27:12 +00:00
Marcus Hoffmann
83526e09a3 update: fix unarchiving of allowed disabled algorithm
6d0b1bbe6fae0909683f2c6a154515bc4bfcb674 didn't handle the
allow_disabled_algorithm case at all, so we add it back.

This additionally fixes a (previously existing) bug where setting
allow_disabled_algorithms to True didn't move apks back from archive to
repo. Introduce a new test for this.

The disabled_algorithm archiving logic is still all over the place so
ideally that needs a future refactor.
2020-01-01 15:25:43 +01:00
Marcus Hoffmann
7f6efa74f5 tests: add test for correctly handling CVC when archiving 2019-12-24 17:35:54 +01:00
Hans-Christoph Steiner
7d40e89341
checkupdates: split out vercode parsing into testable function 2019-12-03 23:51:48 +01:00
Jochen Sprickerhof
bbee2cf707 Add unit test for string_is_integer() 2019-12-03 21:49:44 +01:00
Mike Hardy
3648ef1f2e Update scanner.TestCase to match new regex regime
scanning for analytics is now handled by firebase/play-services regexes
2019-11-02 18:54:54 +00:00
Hans-Christoph Steiner
0a8af2c55f Merge branch 'fix-readline-in-srclibs' into 'master'
build: fix bad regexs when removing signingConfig from srclibs

See merge request fdroid/fdroidserver!686
2019-10-25 08:18:08 +00:00
Hans-Christoph Steiner
afaa24f2fd
build: fix bad regexs when removing signingConfig from srclibs
I went through the source of all apps in fdroiddata for examples, and found
some that use readLine() for things totally unrelated to signingConfigs.

https://gitlab.com/fdroid/fdroiddata/merge_requests/4775#note_234132902
2019-10-23 12:44:47 +02:00
Michael Pöhn
7fa3c34e5b update tests for fixed log deployment+changelog 2019-10-15 15:19:18 +02:00
Hans-Christoph Steiner
5904aef5a7
update: parse Triple-T v2.x layout
Thanks to the very nice example from Piwigo, included here under the GPLv3+
https://github.com/Piwigo/Piwigo-Android

https://gitlab.com/fdroid/fdroiddata/merge_requests/5579#note_225834868
2019-10-04 14:11:40 +02:00
Hans-Christoph Steiner
3801db064a lint: improve HTTPS check
It was missing some domains, so I added another rule.  @IzzySoft pointed
out it was redudnant, so this removes the redudant rule and fixes the
original.

https://gitlab.com/fdroid/fdroidserver/merge_requests/681/diffs#note_225263464
2019-10-04 11:12:48 +02:00
Hans-Christoph Steiner
af4e231f7d lint: enforce HTTPS on GitHub and GitLab pages
The always provide HTTPS, so let's enforce it.
2019-10-04 11:12:48 +02:00
Hans-Christoph Steiner
17320c23f4 lint: don't trip up on projects with 'master' in the name
https://gitlab.com/fdroid/fdroiddata/merge_requests/5557#note_223283359
2019-10-04 11:12:48 +02:00
Michael Pöhn
d5ab303d83 lint: license override config option + require FSF/OSI approved licenses by default 2019-10-02 22:14:09 +02:00
Hans-Christoph Steiner
c7048f2c39 update: log errors on bad graphics, and then ignore the file
Python PIL is not so tolerant, so bad EXIF causes crashes:

  File "/var/lib/jenkins/userContent/reproducible/reproducible_fdroid_build_apps/fdroidserver/update.py", line 2088, in main
    insert_localized_app_metadata(apps)
  File "/var/lib/jenkins/userContent/reproducible/reproducible_fdroid_build_apps/fdroidserver/update.py", line 978, in insert_localized_app_metadata
    _strip_and_copy_image(os.path.join(root, f), destdir)
  File "/var/lib/jenkins/userContent/reproducible/reproducible_fdroid_build_apps/fdroidserver/update.py", line 754, in _strip_and_copy_image
    in_image = Image.open(fp)
  File "/usr/lib/python3/dist-packages/PIL/Image.py", line 2687, in open
    % (filename if filename else fp))
OSError: cannot identify image file <_io.BufferedReader name='build/org.sw24softwares.starkeverben/fastlane/metadata/android/en-US/images/featureGraphic.png'>
2019-09-26 22:06:21 +02:00
Hans-Christoph Steiner
cf2c1d0270
run-tests needs a source of APKs to function
The test runs on jenkins.debian.net were failing because they
specified a dir that did not exist.
2019-09-25 12:25:39 +02:00
Hans-Christoph Steiner
25548023e0
gitlab-ci: check gradle checksums against official list 2019-09-12 14:18:07 +02:00
relan
079754c56c scanner: add a test for the local Debian Maven repo 2019-08-29 21:33:12 +03:00
Hans-Christoph Steiner
70e7e720b9
update: use graphics filename with hash in index to support caching
Using a filename based on the hash of the contents means that the caching
algorithms for fdroidclient and browsers can safely cache the file forever
using the filename, since this guarantees that the contents will never
change for a given filename.

This does not cover screenshots, only icon.png, featureGraphic.png,
tvBanner.png, and promoGraphic.png.

fdroidserver#689
fdroid-website!453
2019-08-29 08:23:51 +02:00
Hans-Christoph Steiner
508af00e84
update: only copy graphics and screenshots if mtime/size has changed
Instead of copying every time, trust the filesystem to tell us when the
file has changed.
2019-08-29 08:23:47 +02:00