1
0
mirror of https://gitlab.com/fdroid/fdroidserver.git synced 2024-11-14 11:00:10 +01:00
Commit Graph

5800 Commits

Author SHA1 Message Date
Hans-Christoph Steiner
a76c5cc0b3 Merge branch 'release-test-overhaul' into 'master'
release test overhaul

See merge request fdroid/fdroidserver!499
2018-05-15 09:59:11 +00:00
Hans-Christoph Steiner
f0795cc092 tests: git repo mirror tests require git >= 2.3 2018-05-14 15:25:33 +02:00
Hans-Christoph Steiner
6570e85a2b gitlab-ci: new test case of pip installs on Ubuntu/trusty
Ubuntu/trusty is used as the base image for CI systems like Travis, as well
as Microsoft Subsystem for Linux.  SO we need to provide working options.
2018-05-14 15:25:33 +02:00
Hans-Christoph Steiner
2d3e7ef950 tests: allow tests to run without any Android SDK installed 2018-05-14 15:25:33 +02:00
Hans-Christoph Steiner
43c61aecee fix pylint Unused argument 2018-05-14 15:25:33 +02:00
Hans-Christoph Steiner
6db9ba8334 androguard >= 3.1.0 is in pypi, so add it to install_requires 2018-05-14 15:25:33 +02:00
Hans-Christoph Steiner
54b21a6d22 move pylint run to standalone gitlab-ci job 2018-05-14 15:25:33 +02:00
Hans-Christoph Steiner
3c9cc59c38 gitlab-ci: include fdroiddata yml files in metadata_v0 test 2018-05-14 15:25:33 +02:00
Hans-Christoph Steiner
0bd276de1c gitlab-ci: add new security scanner pyup.io/safety
https://pyup.io/safety/
2018-05-14 15:25:33 +02:00
Hans-Christoph Steiner
feac3fa067 jenkins: remove most troublesome test case to avoid false fails 2018-05-14 15:25:33 +02:00
Hans-Christoph Steiner
5610b3f0d7 set pyasn1 required version to prevent conflict with pyasn1-modules
pyasn1-modules v0.2.1 requires pyasn1 <0.5.0, >=0.4.1 but for whatever
reason, `pip3 install` is not picking it up properly.  Perhaps because pip3
finds an older version of pyasn1 installed but does not upgrade it.

* https://github.com/etingof/pyasn1-modules/issues/10
* https://gitlab.com/eighthave/fdroidserver/-/jobs/67781548
2018-05-14 15:25:33 +02:00
Hans-Christoph Steiner
b3f45ca024 split pip tests to run separately for Ubuntu/trusty and Debian/stretch
Trying to reuse the pip test run is now a lot more work than just
writing it cutstom for each target platform.

Ubuntu/trusty does not have androguard 3.1.x nor aapt >= 26.x, so
using trusty's aapt will always have errors scraping some names from
APKs.  This continues to use Google's binary.

`apt-get update` is now disabled by default by Travis, this adds it
back so that the latest files from the PPA are used.

Here's one example of an issue with pip on trusty:
https://github.com/requests/requests/issues/4006

    pip3 install --quiet -e /home/travis/build/fdroidtravis/fdroidserver
    Exception:
    Traceback (most recent call last):
      File "/home/travis/build/fdroidtravis/fdroidserver/env/lib/python-wheels/setuptools-3.3-py2.py3-none-any.whl/pkg_resources.py", line 2482, in _dep_map
        return self.__dep_map
      File "/home/travis/build/fdroidtravis/fdroidserver/env/lib/python-wheels/setuptools-3.3-py2.py3-none-any.whl/pkg_resources.py", line 2344, in __getattr__
        raise AttributeError(attr)
    AttributeError: _DistInfoDistribution__dep_map

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "/home/travis/build/fdroidtravis/fdroidserver/env/lib/python3.4/site-packages/pip/basecommand.py", line 122, in main
        status = self.run(options, args)
      File "/home/travis/build/fdroidtravis/fdroidserver/env/lib/python3.4/site-packages/pip/commands/install.py", line 278, in run
        requirement_set.prepare_files(finder, force_root_egg_info=self.bundle, bundle=self.bundle)
      File "/home/travis/build/fdroidtravis/fdroidserver/env/lib/python3.4/site-packages/pip/req.py", line 1266, in prepare_files
        req_to_install.extras):
      File "/home/travis/build/fdroidtravis/fdroidserver/env/lib/python-wheels/setuptools-3.3-py2.py3-none-any.whl/pkg_resources.py", line 2291, in requires
        dm = self._dep_map
      File "/home/travis/build/fdroidtravis/fdroidserver/env/lib/python-wheels/setuptools-3.3-py2.py3-none-any.whl/pkg_resources.py", line 2484, in _dep_map
        self.__dep_map = self._compute_dependencies()
      File "/home/travis/build/fdroidtravis/fdroidserver/env/lib/python-wheels/setuptools-3.3-py2.py3-none-any.whl/pkg_resources.py", line 2517, in _compute_dependencies
        common = frozenset(reqs_for_extra(None))
      File "/home/travis/build/fdroidtravis/fdroidserver/env/lib/python-wheels/setuptools-3.3-py2.py3-none-any.whl/pkg_resources.py", line 2514, in reqs_for_extra
        if req.marker_fn(override={'extra':extra}):
      File "/home/travis/build/fdroidtravis/fdroidserver/env/lib/python-wheels/setuptools-3.3-py2.py3-none-any.whl/_markerlib/markers.py", line 113, in marker_fn
        return eval(compiled_marker, environment)
      File "<environment marker>", line 1, in <module>
    NameError: name 'platform_system' is not defined
2018-05-14 15:25:33 +02:00
Marcus
3e0fa5c80e Merge branch 'master' into 'master'
allow running without aapt, support Java versions newer than 9

Closes #474

See merge request fdroid/fdroidserver!497
2018-05-08 18:37:38 +00:00
Hans-Christoph Steiner
040b8d8d5d prefer latest version of Java, its no longer necessary to prefer 7
closes #474
2018-05-04 21:38:06 +02:00
Hans-Christoph Steiner
7eb622930e update Java path searches to also match v10, v11, etc.
This is still pretty conversative, since its not easy to test on all those
platforms.

#474
2018-05-04 21:37:41 +02:00
Hans-Christoph Steiner
77f7ca2f25 tests: allow the full suite to run without aapt, only androguard 2018-05-04 15:11:25 +02:00
Hans-Christoph Steiner
cf1ce46e5c init: make work with only androguard, no Android SDK 2018-05-04 15:11:25 +02:00
Hans-Christoph Steiner
9cc8826a79 tests: allow running with only androguard, no Android SDK 2018-05-04 15:11:25 +02:00
Hans-Christoph Steiner
21a18cf26b gitlab-ci: fix ubuntu_lts test now that its bionic rather than xenial
The new ubuntu:latest image is not set up for non-interactive, so this
gitlab-ci job now needs to do that.
2018-05-04 15:11:25 +02:00
Hans-Christoph Steiner
d2af293bb7 travis-ci: update to only recent OSX versions, well supported by brew 2018-05-04 15:11:25 +02:00
Hans-Christoph Steiner
18a7dddd23 Merge branch 'master' into 'master'
Fix applicationIdSuffix / versionNameSuffix not understood #455

Closes #455

See merge request fdroid/fdroidserver!484
2018-05-04 12:58:29 +00:00
J-Jamet
d6f1de2649 Fix applicationIdSuffix / versionNameSuffix #455 2018-05-04 12:24:16 +02:00
Hans-Christoph Steiner
62a187607d Merge branch 'master' into 'master'
Three random fixes

See merge request fdroid/fdroidserver!496
2018-05-03 12:11:48 +00:00
Hans-Christoph Steiner
27a5cce832 implement common.get_apk_id() using androguard 2018-05-03 13:46:42 +02:00
Hans-Christoph Steiner
98a2f70e38 fix intermittent test failure
For some reason, the parser stopped working intermittently, even
though the format has been the same since aapt 23 or earlier.  Then
also, some of the test cases pointed to symlinks that were no longer
generated, and one test app now has a blank versionName.

Strange that this wasn't caught in the gitlab-ci runs.  !484

FAIL: test_get_api_id_aapt (__main__.CommonTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "./common.TestCase", line 578, in testA_get_api_id_aapt
    self.assertEqual(versionName, vn)
AssertionError: '0.1' != "0.1' platformBuildVersionName='4.3.1-1425645"
- 0.1
+ 0.1' platformBuildVersionName='4.3.1-1425645
2018-05-03 13:46:42 +02:00
Hans-Christoph Steiner
cc1e10a37a delete .java.security after checking MD5 signatures
This file is written freshly each time before use, so it does not need
to be ekpt around.  It was the only file making the fdroiddata.git
repo dirty on the f-droid.org infrastructure.

This also adds stricter file permissions to avoid an attacker changing
those settings during operation.
2018-05-03 13:46:36 +02:00
Hans-Christoph Steiner
03ad2578d4 Merge branch 'master' into 'master'
fix bad parsing of maxSdkVersion as a @string resource

See merge request fdroid/fdroidserver!494
2018-04-26 14:36:07 +00:00
Hans-Christoph Steiner
0e5ed1bb11 reusable method for checking if a value is a resId or not
I wrote this for the previous commit, but since aapt does not recognize
SDK Version values that are string resources, I made this do the same thing
2018-04-26 15:19:41 +02:00
Hans-Christoph Steiner
e17815e9f0 handle bad SDK Version values in APKs
Even though it is invalid to have *SdkVersion in AndroidManifest.xml set as
anything but an integer, sometimes people manage to get something in there.
`fdroid update` needs to be able to handle all that.

* https://developer.android.com/guide/topics/manifest/uses-sdk-element#min
* https://gitlab.com/souch/SMSbypass/blob/v0.9/app/src/main/AndroidManifest.xml#L29
* https://gitlab.com/souch/SMSbypass/blob/v0.9/app/src/main/res/values/strings.xml#L27

admin#65
2018-04-26 15:19:41 +02:00
Hans-Christoph Steiner
4ba5b8b5ec ignore index files generated as part of manual testing
it is quite useful to run `../fdroid update --pretty` directly in tests/
2018-04-26 12:33:44 +02:00
Hans-Christoph Steiner
74c6555c71 index: ensure index.xml timestamp is in UTC, closes #481
use verbose but safe syntax so this works with all versions of sed, e.g. osx
2018-04-25 16:35:22 +02:00
Marcus
8fbe0738ec Merge branch 'gradle_4.7' into 'master'
makebuildserver: add gradle 4.7

See merge request fdroid/fdroidserver!493
2018-04-22 18:00:33 +00:00
Marcus Hoffmann
a4fe2f711c
makebuildserver: add gradle 4.7 2018-04-20 17:55:51 +02:00
Hans-Christoph Steiner
afe0c6a167 fix scan_apk_androguard when parsing versionName without a dot (e.g. 5)
fixes de35f1b05b:

Traceback (most recent call last):
  File "/home/hans/code/fdroid/server/fdroid", line 164, in <module>
    main()
  File "/home/hans/code/fdroid/server/fdroid", line 138, in main
    mod.main()
  File "/export/share/code/fdroid/server/fdroidserver/update.py", line 1932, in main
    apks, cachechanged = process_apks(apkcache, repodirs[0], knownapks, options.use_date_from_apk)
  File "/export/share/code/fdroid/server/fdroidserver/update.py", line 1459, in process_apks
    use_date_from_apk, ada, True)
  File "/export/share/code/fdroid/server/fdroidserver/update.py", line 1332, in process_apk
    apk = scan_apk(apkfile)
  File "/export/share/code/fdroid/server/fdroidserver/update.py", line 1051, in scan_apk
    scan_apk_androguard(apk, apk_file)
  File "/export/share/code/fdroid/server/fdroidserver/update.py", line 1220, in scan_apk_androguard
    res_id = arsc.get_id(apk['packageName'], res_id)[1]
TypeError: 'NoneType' object is not subscriptable

For example https://f-droid.org/archive/com.abitsinc.andr_5.apk:
$ aapt dump badging archive/com.abitsinc.andr_5.apk |head -1
package: name='com.abitsinc.andr' versionCode='5' versionName='5' platformBuildVersionName='2.3.3'
2018-04-18 12:32:28 +02:00
Hans-Christoph Steiner
6cc26ad8c8 Merge branch 'blank-versionName' into 'master'
handle APKs with a blank versionName

Closes #477, #478, fdroidclient#1416, fdroidclient#1417, and fdroidclient#1418

See merge request fdroid/fdroidserver!492
2018-04-17 13:02:55 +00:00
Hans-Christoph Steiner
de35f1b05b update: handle APKs with a blank versionName
Instead of just crashing, first try to use the versionName as written in the
build metadata, otherwise just let it be blank.  A blank versionName will
cause fdroidclient < 1.3 to crash.  Blank versionNames are not allowed in
the .txt metadata format, only .yml.

closes #477
closes #478
closes fdroidclient#1416
closes fdroidclient#1417
closes fdroidclient#1418
fdroiddata!3061
2018-04-17 12:36:49 +02:00
Hans-Christoph Steiner
9bf9159a77 update: use only 7 chars of SHA256 for non-APK version name
#460
2018-04-17 12:36:49 +02:00
Hans-Christoph Steiner
74c0e98d82 update index metadata tests for 673a3517d3 2018-04-16 11:47:48 +02:00
Hans-Christoph Steiner
673a3517d3 add test of Android TV "activity banner" graphic
!491

https://developer.android.com/guide/topics/manifest/activity-element.html#banner
https://developer.android.com/training/tv/start/start.html#banner
2018-04-16 11:37:11 +02:00
Hans-Christoph Steiner
0cea73cd2f Merge branch 'patch-1' into 'master'
fdroidserver/update.py: APK_LABEL_ICON_PAT icon pattern should not be greedy

See merge request fdroid/fdroidserver!491
2018-04-16 09:37:42 +00:00
Phoenix09
ea89b25fd7 fdroidserver/update.py: APK_LABEL_ICON_PAT icon pattern should not be greedy 2018-04-14 23:30:23 +01:00
Marcus
eec005ed67 Merge branch 'wiki-page-name' into 'master'
Filter another invalid mediawiki page title character

See merge request fdroid/fdroidserver!490
2018-04-11 13:35:42 +00:00
Ciaran Gultnieks
bd00b87e4e Filter another invalid mediawiki page title character 2018-04-11 14:32:55 +01:00
Hans-Christoph Steiner
33c6d5570e jenkins-test: remove redundant fdroid update test
This test ran before config.py was setup, so it didn't respect the test
setup.  Things like "make_current_version_link = False" are needed so that
the checkupdates tests work like on the official infrastructure.  Also,
./tests/run-tests has many `fdroid update` tests, so having it here just
wastes time.  The tests that are directly in this script are meant to test
the separate build/sign setup.
2018-04-09 10:27:29 +02:00
Marcus
94a094323e Merge branch 'mwclient-dep' into 'master'
Replace deprecated mwclient method

See merge request fdroid/fdroidserver!489
2018-04-08 11:53:26 +00:00
Ciaran Gultnieks
31e8605864 Replace deprecated mwclient method
It's functionally the same, just a different name.
2018-04-08 09:37:48 +01:00
Hans-Christoph Steiner
ea3f9df473 Merge branch 'patch-1' into 'master'
Fix a typo in README.md

See merge request fdroid/fdroidserver!488
2018-04-06 20:24:54 +00:00
scde
2890b2feb2 Fix a typo in README.md 2018-04-06 17:16:56 +00:00
Hans-Christoph Steiner
45ff2231b0 jenkins-test: turn off link generation for checkupdates test
These links will make checkupdates fail since they will make the git repo
dirty.  There is also a mystery "cache/" subdir, but I don't know what is
making that.
2018-04-05 21:03:13 +02:00
relan
4a5c921aee Merge branch 'gradle_plugin_3.1' into 'master'
add gradle plugin 3.1 which requires gradle 4.4

See merge request fdroid/fdroidserver!487
2018-04-05 12:34:47 +00:00