1
0
mirror of https://gitlab.com/fdroid/fdroidserver.git synced 2024-11-19 21:30:10 +01:00
Commit Graph

501 Commits

Author SHA1 Message Date
Hans-Christoph Steiner
8c1cf724e1 init: force keystore to PKCS12 format
Java 8 supports PKCS12, Java 9+ uses PKCS12 by default, which should have
a .p12 file extension.  `fdroid init` has always just added .jks which is
the old default format.

* https://docs.oracle.com/en/java/javase/12/tools/keytool.html#GUID-5990A2E4-78E3-47B7-AE75-6D1826259549__GUID-A8B9E662-C1C2-4A0E-9307-A8464F0E95D4
* https://openjdk.java.net/jeps/229
2020-09-23 16:40:37 +02:00
Hans-Christoph Steiner
08f724651e remove "force_build_tools" config option, closes #738
The `force_build_tools` config option was added a long time ago to
brute force the _build-tools_ version by trying to replace the value
in `build.gradle` files. This is never something that should be used
in production, since the app's build metadata should specify this kind
of thing. And now that we're moving towards _androguard_ for
everything except fdroid build and fdroid publish, _build-tools_ will
no longer even be used in the other commands.
2020-09-16 16:30:49 +02:00
Marcus Hoffmann
7de601a5b5 fallback to minsdk when targetsdk isn't set
Androguard already has a function always returning an int here, so let's
use that.

Also put in a guard against minsdk not being set.
2020-09-14 14:12:18 +02:00
Marcus Hoffmann
b2f6483671 use new find_apksigner in test_scan_apk 2020-09-10 18:59:39 +02:00
Marcus Hoffmann
7eb32feaa5 skip new signing test when we can't find apksigner
Also add some error handling to the find_apksigner() method.
2020-09-10 18:59:39 +02:00
Marcus Hoffmann
89f63b3e1c tests: use yaml.Loader on older yaml versions 2020-09-10 18:38:43 +02:00
Marcus Hoffmann
709f4c9b18 pickle -> yaml rename 2020-09-10 18:38:43 +02:00
Marcus Hoffmann
e613b65098 we need FullLoader for one test, we are dumping custom objects 2020-09-10 18:38:43 +02:00
Marcus Hoffmann
9bf0758f19 make update.Testcase tests work standalone 2020-09-10 18:38:43 +02:00
Marcus Hoffmann
4cd96d4a9f use yaml safeloader in tests
Try to use CSafeLoader when possible because its significantly faster.
Use the normal Safeloader otherwise. (This mirrors the non-test code
behaviour)
2020-09-10 14:18:36 +02:00
Marcus Hoffmann
768a91370c publish: use apksigner for signing apks with targetSDK>=30
This makes apksigner a hard requirement of the signing procedure.
We'll first try to find a globally installed version from PATH and if
that's not available fall back to using a version from build-tools.

Future TODO: always sign with apksigner, blocked on signature transplant
support for apksigv2/v3

Closes fdroid/fdroidserver#634
Closes fdroid/fdroidserver#827
2020-09-09 18:09:28 +02:00
Marcus Hoffmann
bc6fa986d1 fix vcs test
It doesn't need a valid sdk_path, so just set it to something.
The test was failing when ANDROID_HOME wasn't set in the env.
2020-08-25 23:09:28 +02:00
Marcus Hoffmann
34717fe88a fix typoes 2020-08-25 21:32:54 +02:00
Marcus Hoffmann
059ebd4bc9 tests: add check that we trigger a scanner error without setting it as extlib 2020-08-25 21:32:29 +02:00
Marcus Hoffmann
a301a1ba93 add test for correct added date for apps 2020-08-25 21:32:29 +02:00
Marcus Hoffmann
fce4721695 add a vcs.TestCase 2020-08-25 21:32:29 +02:00
Marcus Hoffmann
61736f3f50 scanner: add test for #759 2020-08-25 21:32:29 +02:00
Marcus Hoffmann
f46e99a5c4 test for #796
The extlib test is in build because it tests the interaction between
prepare_source with a later scan as it is run from build.py
2020-08-25 21:31:07 +02:00
Marcus
8801d37649 Merge branch 'publish_json' into 'master'
Some publish.py improvements

Closes #820

See merge request fdroid/fdroidserver!787
2020-08-25 12:33:39 +00:00
Marcus Hoffmann
882f8cfe19 test_check_for_key_collisions: test with an actual collision
Genrated with this script:
https://gitlab.com/fdroid/fdroidserver/-/merge_requests/787#note_401275883
2020-08-24 21:19:59 +02:00
Marcus Hoffmann
d9a6bfb0a9 CI: install pyjks as dependency for tests 2020-08-24 21:11:55 +02:00
Marcus Hoffmann
a114c73c2d publish: factor out the signing key creation into a method 2020-08-24 19:34:08 +02:00
Marcus Hoffmann
7813a17cf8 publish: extract a few functions out of main
publish is currently not reusable from other modules as everything is
happening in main. It's also not testable from python unittests.

There's already a function for getting the key_alias, so we can use
that.

Introduce tests for the split out functions.
2020-08-24 19:34:08 +02:00
Michael Pöhn
cb368a674c remove txt srclib support 2020-08-20 20:40:15 +02:00
Michael Pöhn
2ec90bb490 remove support for rewriting to txt 2020-08-20 20:40:15 +02:00
Hans-Christoph Steiner
29eea3f739 Merge branch 'androguard_minsdk' into 'master'
common: use androguard to figure out minSDK version

Closes #816

See merge request fdroid/fdroidserver!785
2020-08-20 12:45:28 +00:00
Marcus Hoffmann
ba28b44ae7 common: use androguard to figure out minSDK version
Closes: #816
2020-08-19 19:49:39 +02:00
Hans-Christoph Steiner
f8a1b45df5 gradlew-fdroid: fix random failures due to empty $line, closes #815 2020-08-19 16:28:58 +02:00
Hans-Christoph Steiner
10fa912c16 deploy: remove git remotes not enabled in servergitmirrors
If the repo operator removes an item from servergitmirrors, it should auto-
matically remove the remote.
2020-07-30 14:57:19 +02:00
Marcus
1902bfb6fc Merge branch 'fix_760' into 'master'
update.py (and index.py) refactoring

Closes #524 and #760

See merge request fdroid/fdroidserver!756
2020-06-24 22:37:51 +00:00
Marcus Hoffmann
3ebc44c54f fix tests after they switched to yaml 2020-06-25 00:07:42 +02:00
Marcus Hoffmann
07caa88920 don't include the localized metadata things for /archive
We haven't done this so far and it's a potential big change in archive
size and update performance.
2020-06-25 00:07:42 +02:00
Marcus Hoffmann
8c71637d43 update: make copy_triple_t_store_metadata and insert_localized_app_metadata not assume /repo
This will enable copying the localized metadata to the archive as well.
2020-06-24 23:53:36 +02:00
Marcus Hoffmann
03881154c6 metadata: make linkresolver an actual object
Previously this was magically capturing the apps dict when passing it around as a
function. This also moved the code to the metadata module.

Add a test doing read_metadata where the linkresolver is used. This
happens when the apps we read have a [[app.id]] link to another app.
2020-06-24 23:53:35 +02:00
Marcus Hoffmann
a4177e5ec3 add test for correct whatsnew handling without CVC 2020-06-24 23:53:35 +02:00
Hans-Christoph Steiner
d0f426e076
replace $$srclib$$ with an absolute path
closes #725
2020-06-24 22:40:26 +02:00
Hans-Christoph Steiner
238f048257 update: fix crash when liberapay: or open_collective: not in FUNDING.yml
closes #799
2020-06-24 21:39:43 +02:00
Hans-Christoph Steiner
82eceebd13 Merge branch 'parse-donation-links-from-funding.yml' into 'master'
update: insert donation links based on FUNDING.yml

See merge request fdroid/fdroidserver!754
2020-06-18 04:44:08 +00:00
Hans-Christoph Steiner
2c4e9beacb scanner: add test with abs/rel paths as run from fdroid build
https://gitlab.com/fdroid/fdroidserver/-/issues/791#note_361018050
!767
2020-06-17 10:33:55 +02:00
Hans-Christoph Steiner
62c8fd5999
add Liberapay: field with username as data
Liberapay was originally included using a numeric ID, since they had
not yet finalized the public URLs.  Now it is a username.  So this
logic prefers the username in Liberapay: field, and keeps the old
LiberapayID: to ease migration.  LiberapayID: will not override
Liberapay:.  Clients are expected to prefer Liberapay: over LiberapayID:
2020-06-16 15:35:28 +02:00
Hans-Christoph Steiner
0183592526
update: insert donation links based on FUNDING.yml
GitHub has specified FUNDING.yml, a file to include in a git repo for
pointing people to donation links.  Since F-Droid also points people
to donation links, this parses them to fill out Donate:
and OpenCollective:.  Specifying those in the metadata file takes
precedence over the FUNDING.yml.  This follows the same pattern as how
`fdroid update` includes Fastlane/Triple-T metadata.  This lets the
git repo maintain those specific donations links themselves.

https://help.github.com/en/articles/displaying-a-sponsor-button-in-your-repository#about-funding-files

The test file was generated using:

```python
import os, re, yaml

found = dict()
for root, dirs, files in os.walk('.'):
    for f in files:
        if f == 'FUNDING.yml':
            with open(os.path.join(root, f)) as fp:
                data = yaml.safe_load(fp)
            for k, v in data.items():
                if k not in found:
                    found[k] = set()
                if not v:
                    continue
                if isinstance(v, list):
                    for i in v:
                        found[k].add(i)
                else:
                    found[k].add(v)

            with open('gather-funding-names.yaml', 'w') as fp:
                output = dict()
                for k, v in found.items():
                    output[k] = sorted(v)
                yaml.dump(output, fp, default_flow_style=False)
```
2020-06-16 15:35:26 +02:00
Marcus
f84818c15e scanner: add a simple scan for blacklisted classes after build step
add com.android.billing to blacklist, see
https://gitlab.com/fdroid/fdroiddata/-/issues/2070#note_360611289
2020-06-15 18:03:19 +00:00
Hans-Christoph Steiner
0837289935 scanner: fix regex for matching URLs in gradle maven{} blocks
closes #465

This script generated gradle-maven-blocks.yaml:
```python
import os
import re
import yaml

pat = re.compile(r'\smaven\s*{[^}]+}')

finds = set()
for root, dirs, files in os.walk('.'):
    for f in files:
        if '.gradle' in f:
            with open(os.path.join(root, f), errors='surrogateescape') as fp:
                contents = fp.read()
            for m in pat.findall(contents):
                finds.add(m)

with open('finds.yaml', 'w') as fp:
    yaml.dump(sorted(finds), fp, default_flow_style=False)

```
2020-06-10 19:42:23 +02:00
Hans-Christoph Steiner
6590f3869e scanner: error/warn on dex/gz/zip, closes #394 2020-06-10 19:42:23 +02:00
Hans-Christoph Steiner
ee54dbce87 scanner: safely check options, for things using this API
https://gitlab.com/fdroid/fdroidserver/-/issues/771#note_353495799
2020-06-10 19:42:23 +02:00
Hans-Christoph Steiner
75acf63efa scanner: remove all gradle wrapper files 2020-06-10 19:42:23 +02:00
Hans-Christoph Steiner
d898ad0412 scanner: make AARs and JARs trigger an error
refs #491
2020-06-10 19:42:23 +02:00
Michael Pöhn
a97b3ca4dd implement plugin system review suggestsions 2020-06-10 18:43:23 +02:00
Michael Pöhn
77167e098e plugin system: regex instead of import bases plugin parsing 2020-06-10 18:43:11 +02:00
Michael Pöhn
b257a3411a stick to default python sys.path 2020-06-10 12:11:47 +02:00