Jonas Kalderstam
9e4459d7d1
Add test for when apk icon src could not be found
...
An APK (Netflix) was found to have the following icon filename:
\u2003\u2009\n.xml
This breaks the aapt dump parsing because it iterates line by line and
this filename goes across two lines. Consequently, icon_src will be
None (default value) when it is passed to the icons parser.
2018-03-27 18:39:59 +02:00
Hans-Christoph Steiner
d98d8c2e52
update: do not crash with androguard when working with apps without icons
...
For example, SpeedoMeterApp.main_1.apk
2018-03-12 17:55:45 +01:00
Hans-Christoph Steiner
7d79780df0
tests: checkupdates now requires a clean git repo state
2018-03-06 20:50:23 +01:00
Hans-Christoph Steiner
8f30c892c5
VercodeOperation: only allow simple math expresssions and %c
2018-03-05 09:45:58 +01:00
Hans-Christoph Steiner
498ea5d609
lint: ban all dangerous HTML tags
...
* https://en.wikipedia.org/wiki/HTML_sanitization
* https://asostack.com/enhance-your-google-play-store-description-with-rich-formatting-and-emojis-5f50ff354e5f
2018-02-27 12:09:54 +01:00
Hans-Christoph Steiner
e75bf70be6
signatures: future-proof fetching app ID info from APK
...
We're not using platformBuildVersionName and it might go away just like it
appeared: with no good reason or announcement.
2018-02-22 15:08:55 +01:00
Hans-Christoph Steiner
089712c012
tests: do not automatically run install.TestCase, its troublesome
...
`fdroid install` is rarely used, if at all, and the test frequently fails
for no reason in gitlab-ci, because it can't start the adb server.
2018-02-22 15:08:54 +01:00
Hans-Christoph Steiner
52b3436ff6
make is_apk_and_debuggable() default to using androguard before aapt
2018-02-22 15:08:53 +01:00
Hans-Christoph Steiner
40fac10ebc
update: extract and store XML icons
...
These can then be used by the client.
#344
#392
2018-02-15 14:28:45 +01:00
Hans-Christoph Steiner
05eef5a454
metadata: add new Translation app field for URL for contributions
...
This is a field requested from a bunch of translators so they can easily
find where to translate apps:
https://forum.f-droid.org/t/translation-field-for-app-entries/1403/5
2018-02-05 15:45:12 +01:00
relan
86f34ee70a
build: make per-build hard time limit customizable
...
Add "timeout=n" metadata field that overrides build timeout (in seconds).
The default is 7200, i.e. 2 hours.
2018-01-30 11:14:50 +03:00
Hans-Christoph Steiner
d3caf09421
use standard User-Agent in check-fdroid-apk
2018-01-26 10:18:41 +01:00
Hans-Christoph Steiner
513c95894c
build: remove unused, unmaintained Kivy build method
...
This code has never been used and contains some insecure uses of shell=True
Building Kivy apps should be done with the buildozer=yes method. The
buildozer method should probably be moved to a provisioner once that is in
place.
2018-01-23 23:16:05 +01:00
Hans-Christoph Steiner
dfbe114af3
new script to audit the FDroid.apk on https://f-droid.org
...
This makes sure its signed by the F-Droid key.
2018-01-05 14:08:14 +01:00
Hans-Christoph Steiner
6d32b68016
tests: correct package names in stats/known_apks.txt
...
This doesn't seem to affect the tests either way, but it is good to have
things correct there.
2017-12-28 23:07:26 +01:00
Hans-Christoph Steiner
e451ec0079
common: fix bug in new SHA-256 signatures for >= android-18
...
Luckily, this is only used in `fdroid nightly` so far.
2017-12-28 23:07:26 +01:00
Hans-Christoph Steiner
109eb928e8
aapt 26.0.0 is required to properly parse permissions and label
...
#236
closes #395
aapt 26.0.0 outputs the permissions correctly
closes #306
aapt 26.0.0 now outputs: application-label:'K-9 Mail'
2017-12-28 23:07:26 +01:00
mimi89999
455dcc34d8
Add Nextcloud and DavDroid test case
2017-12-27 11:51:57 +01:00
mimi89999
90c7dd29df
gradle file: use flavour specific versionCode/versionName, fall back to parsing line by line
2017-12-23 17:12:54 +01:00
mimi89999
918bd15c45
Revert: gradle file: use flavour specific versionCode/versionName, fall back to parsing line by line
2017-12-23 12:57:34 +01:00
Hans-Christoph Steiner
173a35b190
tests: only generate keystores when that is actually being tested
...
Generating a keystore is quite slow since it means a new RSA key is created.
That only needs to happen in the tests that check that it actually happened,
otherwise the test can just reuse the stored test keystore.
closes #432
2017-12-20 23:46:37 +01:00
Hans-Christoph Steiner
39b76b0eda
scanner: fix tests so they work on all tested platforms
...
The standard test configuration is needed to make the tests reliably. Also,
these tests used some odd yield logic. Who knows what exactly failed, but
these tests should be reliable.
* https://gitlab.com/fdroid/fdroidserver/-/jobs/44984595
* https://gitlab.com/fdroid/fdroidserver/-/jobs/44984596
* https://travis-ci.org/f-droid/fdroidserver/builds/318071369
2017-12-19 22:51:40 +01:00
Hans-Christoph Steiner
86c5598307
Merge branch 'whitelist-firebase' into 'master'
...
Whitelist some open-source firebase libs
See merge request fdroid/fdroidserver!411
2017-12-18 12:30:30 +00:00
Jan Berkel
365834d3ea
Convert to string
2017-12-15 00:58:46 +01:00
Jan Berkel
5dee23f7a6
Add a simple test for scanner
2017-12-15 00:37:52 +01:00
Hans-Christoph Steiner
bde0558d82
update: reject APKs with invalid file sig, probably Janus exploits
...
This just checks the first four bytes of the APK file, aka the "file
signature", to make sure it is the ZIP signature and not the DEX signature.
This was checked against the test APK, and I ran it against some known
malware and all of f-droid.org to make sure it works.
All valid ZIP files (therefore APK files) should start with the ZIP
Local File Header of four bytes.
https://www.guardsquare.com/en/blog/new-android-vulnerability-allows-attackers-modify-apps-without-affecting-their-signatures
2017-12-14 16:57:22 +01:00
Hans-Christoph Steiner
ca50adb2e5
update: switch tests to using standardized setUp() method
2017-12-14 16:57:22 +01:00
Hans-Christoph Steiner
61aac0503a
Merge branch 'fixFlavor' into 'master'
...
Regex only for flavor blocks: flavor { ... }
See merge request fdroid/fdroidserver!407
2017-12-14 16:56:01 +01:00
Hans-Christoph Steiner
bec4f7d547
add Conversations as gradle flavor test case
2017-12-14 16:52:02 +01:00
Andrea Scarpino
19a83b6219
Add Liberapay support
2017-12-12 11:53:31 +01:00
Hans-Christoph Steiner
6228162cbd
handle jarsigner/apksigner output cleanly for rational logging
...
These were both spamming the output with lots of confusing messages, even
when --verbose was not used. Jarsigner especially has confusing messages,
since it has warnings that do not pertain to APK signatures at all, like
the ones about timestamps and missing Certificate Authority.
closes #405
2017-12-07 17:32:14 +01:00
Hans-Christoph Steiner
05abbfbabd
gitlab-ci: move sdist test run to new fedora job
...
A full run of the test suite takes quite a bit of time. This removes one
of the 3 runs from the main 'tests' job, and puts it into the Fedora job.
That test run is mostly to make sure the setup.py and source tarball are
correctly, so that doesn't affect merge requests very often.
This also tests `pip install --user`, which was not really being tested
before.
2017-12-06 20:20:17 +01:00
Marcus Hoffmann
bfe2c00834
common.testCase: fix find_sdk_tools when aapt is installed in /usr/bin
...
The testlogic was broken when having both aapt in /usr/bin and also as
part of the android sdk.
2017-12-06 12:30:47 +01:00
Hans-Christoph Steiner
1c3a4479ab
add common.sign_apk() for nighly as test for using in publish
...
Since the MD5 migration was quite a bit of work, it makes sense to start
on moving away from SHA1 as much as possible while it is easy to do. SHA256
will only work in APK signatures on android-18 (4.3) or newer. So if an
APK has a minSdkVersion of 18 or newer, then sign with SHA256.
https://issuetracker.google.com/issues/36956587
https://android-review.googlesource.com/c/platform/libcore/+/44491
2017-12-04 22:52:41 +01:00
Hans-Christoph Steiner
3f4e93bf86
Merge branch 'metadataFlavours' into 'master'
...
Add flavour to metadata
See merge request fdroid/fdroidserver!370
2017-12-02 20:51:24 +00:00
tobiasKaminsky
d5ecb34df7
add flavour to metadata
2017-12-01 15:25:00 +01:00
Hans-Christoph Steiner
5b22ff7dc6
tests: use standard dir setup so all tests start in same dir
2017-11-30 17:32:53 +01:00
Hans-Christoph Steiner
3ff4b656c6
tests: name temp test dir after test function that used it
2017-11-30 17:32:53 +01:00
Hans-Christoph Steiner
30b2f5a48a
build: sort fdroid build --all by most recently changed first
2017-11-30 17:32:53 +01:00
Hans-Christoph Steiner
c7c40cb59f
PEP8 fixes
2017-11-30 13:42:37 +01:00
tobiasKaminsky
33aee96ed9
added test case
2017-11-30 11:12:18 +01:00
Hans-Christoph Steiner
743ef9ba00
rename test files to work round filename length limits
...
It seems a lot of things like tar, Windows, some filesystems, etc. do not
handle long filenames. #416
2017-11-27 16:57:30 +01:00
Hans-Christoph Steiner
e33684fc9b
tests: new tests for stacktraces caused by translated help strings
2017-11-22 11:40:16 +01:00
Hans-Christoph Steiner
cbd9669105
rename test file to fit within eCryptfs filename limits ( closes #416 )
2017-11-22 11:40:16 +01:00
Hans-Christoph Steiner
cd3e763953
tests: only test fdroid build if the right build-tools is installed
2017-11-09 14:07:40 +01:00
Hans-Christoph Steiner
a14124955a
tests: fix fdroid build test with Debian gradle
2017-11-09 13:11:49 +01:00
Hans-Christoph Steiner
2ee32f3524
fix tests to work when only Debian Android SDK is installed
...
jenkins.debian.net only had `apt install android-sdk`, it does not have any
of Google's packages installed.
2017-11-08 14:05:17 +01:00
Hans-Christoph Steiner
17efa13183
tests: pre-set failfast as reminder of a handy time saver
2017-10-25 23:01:25 +02:00
Hans-Christoph Steiner
e0df6d2479
choose the most recent available version of Java
...
This came about testing on OSX, where there are often multiple versions of
the JDK installed. This was choosing the oldest version. It should
choose the most recent version.
2017-10-25 23:01:25 +02:00
Hans-Christoph Steiner
49bd5a6462
use Babel to compile localizations and include in source tarball
...
python3-babel was also included in the gitlab-ci docker image:
ffc08dbc1d
2017-10-19 23:28:13 +02:00
Hans-Christoph Steiner
7d4e354f25
FDroidException handle Exceptions that return things other than str
...
This is related to a4c4a16ed9
2017-10-19 20:38:43 +02:00
Hans-Christoph Steiner
af0d8ab84c
tests: make pip3 install quieter for shorter CI logs
2017-10-19 20:23:57 +02:00
Michael Pöhn
788c8f97fd
better error message in publish when repo_key is not set
2017-10-17 22:10:39 +02:00
Michael Pöhn
89d9d0c45b
use OrderedDict for localization data in index-v1
2017-09-26 14:15:48 +02:00
Michael Pöhn
26192a00e2
make sure all timestamps in index-v1 are utc
2017-09-26 14:15:27 +02:00
Michael Pöhn
a2d361dec8
add test for index-v1.json
2017-09-26 14:12:59 +02:00
Michael Pöhn
b6b79edf11
store fdroid signing-key-fp-list in stats dir
2017-09-26 14:11:09 +02:00
Michael Pöhn
6930edf889
sort index-v1; publish now creates and stores a list of signature fingerprints
2017-09-26 14:11:09 +02:00
Michael Pöhn
bca07f794f
added functions for storing/loading signer fingerprints to stats
2017-09-26 14:11:09 +02:00
Michael Pöhn
5a524d4d0c
added some developer-signing key metadata to tests
2017-09-26 14:11:09 +02:00
Michael Pöhn
6a4a78429f
add signer to index v1
2017-09-26 14:11:09 +02:00
Michael Pöhn
7c4b8bcac4
added parse release filename function
2017-09-26 14:11:09 +02:00
Michael Pöhn
04daa7a03a
test for common.get_app_id_aapt
2017-09-26 14:11:09 +02:00
Michael Pöhn
efb0bf6ee5
add common functions for dealing with apk signatures
2017-09-26 14:11:09 +02:00
Michael Pöhn
c196f7dd7a
common function for fetching sha256 signing-key fingerprint
2017-09-26 14:11:09 +02:00
Hans-Christoph Steiner
eaca20675c
tests: test setting up app git repo like fdroid build does
...
I wrote this to try to find #379 but this test didn't trigger that bug.
2017-09-25 16:35:18 +02:00
Hans-Christoph Steiner
cbf7ba0414
tests: add setUp() method to common to handle standard stuff
2017-09-25 16:35:18 +02:00
Hans-Christoph Steiner
2764c43fa2
tests: use ci-test-app with current Gradle Android Plugin
...
This works around the gradle 2.x bug where versions newer than 2.9 cannot
run things. This also then specifies the version of Gradle Android Plugin
that is included in Debian/stretch.
2017-09-19 20:16:08 +02:00
Hans-Christoph Steiner
dc569b9c18
tests: support Java setups where MD5 is not disabled by default
...
For platforms using Java < 1.8.0_133, MD5 is still enabled for JAR
signatures. Its just too painful to manage all this, so support this
in the tests.
2017-09-19 20:16:08 +02:00
Hans-Christoph Steiner
373b46ab3f
tests: don't run gpgsign tests on Travis CI's OSX, gpg always fails
2017-09-19 20:16:08 +02:00
Hans-Christoph Steiner
d1f6126dea
tests: support Ubuntu/trusty's python3.4-venv
2017-09-19 20:13:36 +02:00
Hans-Christoph Steiner
bc192b6be0
tests: prefer GNU sed on BSD/OSX, BSD's sed has lame syntax
...
https://blog.remibergsma.com/2012/09/18/sed-inline-editing-different-on-mac-osx/
2017-09-19 20:13:36 +02:00
Hans-Christoph Steiner
176f539647
allow spaces in filenames
...
This fixes all the bugs I could find that prevented fdroid from
handling files with spaces in them. This is more important now that
fdroid supports random media files, and Repomaker
2017-09-19 20:13:36 +02:00
Hans-Christoph Steiner
6adf309bef
tests: move test APK with bad unicode filename to separate test
...
This filename has some messed up bytes related to bi-directional script
that is included (Left-to-Right and Right-to-Left). GNU/Linux always
interprets filenames as pure byte sequences. Windows and OSX store
filenames as Unicode strings. So on OSX, the invalid filename gets
converted to a valid name. That works fine, but the test fails because it
is compared to a file generated on Ubuntu, where it preserves the byte
sequence.
This includes an APK with a valid Unicode filename that includes
bi-directional script.
2017-09-19 20:13:36 +02:00
Hans-Christoph Steiner
5dcb48831f
index: always use jarsigner for verifying JAR signatures
...
apksigner v0.7 (build-tools 26.0.1), Google made it require that the
AndroidManifest.xml was present in the archive before it verifies the
signature. So this needs to stick with the jarsigner hack for JARs.
2017-09-19 20:13:36 +02:00
Hans-Christoph Steiner
acac15f74d
Merge branch 'newcomers' into 'master'
...
allow starting without config.py
See merge request fdroid/fdroidserver!335
2017-09-16 08:07:56 +00:00
Hans-Christoph Steiner
e4ca82d80c
tests: support running on platforms with old java
...
In order to test on travis-ci's OSX setup, its easiest to support the old
version of Java that is installed there by default. It is pretty old, so
it does not include the April 2017 update that disables MD5:
https://blogs.oracle.com/java-platform-group/oracle-jre-will-no-longer-trust-md5-signed-code-by-default
https://www.bleepingcomputer.com/news/security/oracle-to-block-jar-files-signed-with-md5-starting-with-april-2017
https://support.ca.com/us/knowledge-base-articles.TEC1691042.html
#292 #323
2017-09-15 11:37:24 +02:00
Marcus Hoffmann
34f79b876c
tests: don't touch config.py, no config is fine now
2017-09-14 02:15:57 +02:00
Michael Pöhn
3e6dfacf6c
added signatures subcommand
2017-09-07 00:03:35 +02:00
Hans-Christoph Steiner
c67f8e349c
metadata: handle empty YAML files without crashing
...
!311
2017-07-26 18:21:28 -07:00
Hans-Christoph Steiner
7bd171480f
update: test metadata creation using internal and external templates
...
There is a hardcoded template in update.py, and there is also the
possibility for the user to create a template.yml. This tests both of them
and cleans up the related code a bit.
#352 !310
2017-07-26 15:34:13 -07:00
Hans-Christoph Steiner
0a953814e2
update: always include name/summary/desc in index.xml if available
...
With the new localization support, the name/summary/description in the
metadata file becomes the global override. So most apps are not going to
have those fields present in their metadata file. This fixes the index.xml
generation to fall back to the localized versions of those fields when they
are not set in the metadata field.
https://forum.f-droid.org/t/what-has-happend-to-osmand
2017-07-19 15:07:50 +02:00
Hans-Christoph Steiner
523b5f4777
server: smooth out btlog transfer for offline signing setups
...
It turns out it is error prone to `git push` to a non-bare git repo. For
the offline signing machine, the git remote needs to be a regular git repo
in a directory on a thumbdrive so that once the thumbdrive is plugged into
an online machine, that git repo can be transferred to the online machine.
2017-07-17 11:38:15 +02:00
Hans-Christoph Steiner
25f96e1911
ensure that mirror URLs always include the repodir
...
Since the mirror URLs are per repo section (repo/archive), the mirror URLs
must include the repodir at the end. This was missing for servergitmirrors
found by @cde when working on fdroidclient#35
2017-07-07 15:50:11 +02:00
Hans-Christoph Steiner
dfb07808d3
support configing buildserver VM per-build with sudo=
...
This adds the 'sudo' build field, which is just a script that is run as
root. For more info, see the issue that this closes:
refs #318
closes #317
2017-07-06 17:29:23 +02:00
Hans-Christoph Steiner
4a15208b84
update: update openssl KnownVuln scan to handle all recent versions
...
Thanks to @bubu for reporting!
2017-07-06 13:31:42 +02:00
Hans-Christoph Steiner
df99c85ca6
support manually adding per-build antiFeatures in metadata
...
For cases like the OpenVPN vuln that was recently announced, it is useful
for fdroiddata maintainers to be able to mark builds that have known
vulnerabilities.
2017-07-06 12:25:48 +02:00
Michael Pöhn
071ad46b55
moved some yaml test files into a separate forlder to avoid conflicts with other test cases
2017-07-04 13:35:05 +02:00
Michael Pöhn
19d422da86
rewritemeta yaml: fixed boolen mapping for build flags
2017-07-04 11:51:08 +02:00
Michael Pöhn
fc070ee657
refactored yaml tests into metadata test case; fixed typo
2017-07-04 11:51:08 +02:00
Michael Pöhn
f1e0089931
prettify write_yaml metadata
2017-07-04 11:51:08 +02:00
Michael Pöhn
7aa92abebc
yaml rewrite version code as int
2017-07-04 11:51:08 +02:00
Michael Pöhn
f9d4007ea6
rewrite metadata builds list to yaml
2017-07-04 11:51:08 +02:00
Michael Pöhn
735b7d0f33
fixed pyflakes/pep warings
2017-07-04 11:51:08 +02:00
Michael Pöhn
0f90ab9aac
rewrite to yaml works for app data now (builds still missing)
2017-07-04 11:51:08 +02:00
Torsten Grote
0bbe0c5db0
Break up the scan_apk() method and rename it to process_apk()
2017-07-03 18:37:01 -03:00
Hans-Christoph Steiner
5f0817d7bf
tests: make sure apkcache gets created
2017-07-03 10:02:51 +02:00
Hans-Christoph Steiner
746d4bd4cf
update: allow_disabled_algorithms option to keep MD5 sigs in repo
...
The new policy is to move APKs with invalid signatures to the archive,
and only add those APKs to the archive's index if they have valid MD5
signatures.
closes #323
closes #292
2017-07-03 10:02:51 +02:00
Hans-Christoph Steiner
e1492148fa
fix "Archive Policy:" field, APKs can move in/out of archive
...
The original logic was checking keepversions against the len() of ALL the
APKs in the repo/archive. The correct thing is to check against the
number of APKs available for the given packageName/appid.
closes #166
2017-06-28 23:53:42 +02:00
Hans-Christoph Steiner
be2926ffc8
tests: test moving files to and from the archive
...
#166
2017-06-28 23:53:42 +02:00
