1
0
mirror of https://gitlab.com/fdroid/fdroidserver.git synced 2024-09-11 23:23:27 +02:00
Commit Graph

5077 Commits

Author SHA1 Message Date
Hans-Christoph Steiner
27a5cce832 implement common.get_apk_id() using androguard 2018-05-03 13:46:42 +02:00
Hans-Christoph Steiner
98a2f70e38 fix intermittent test failure
For some reason, the parser stopped working intermittently, even
though the format has been the same since aapt 23 or earlier.  Then
also, some of the test cases pointed to symlinks that were no longer
generated, and one test app now has a blank versionName.

Strange that this wasn't caught in the gitlab-ci runs.  !484

FAIL: test_get_api_id_aapt (__main__.CommonTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "./common.TestCase", line 578, in testA_get_api_id_aapt
    self.assertEqual(versionName, vn)
AssertionError: '0.1' != "0.1' platformBuildVersionName='4.3.1-1425645"
- 0.1
+ 0.1' platformBuildVersionName='4.3.1-1425645
2018-05-03 13:46:42 +02:00
Hans-Christoph Steiner
cc1e10a37a delete .java.security after checking MD5 signatures
This file is written freshly each time before use, so it does not need
to be ekpt around.  It was the only file making the fdroiddata.git
repo dirty on the f-droid.org infrastructure.

This also adds stricter file permissions to avoid an attacker changing
those settings during operation.
2018-05-03 13:46:36 +02:00
Hans-Christoph Steiner
03ad2578d4 Merge branch 'master' into 'master'
fix bad parsing of maxSdkVersion as a @string resource

See merge request fdroid/fdroidserver!494
2018-04-26 14:36:07 +00:00
Hans-Christoph Steiner
0e5ed1bb11 reusable method for checking if a value is a resId or not
I wrote this for the previous commit, but since aapt does not recognize
SDK Version values that are string resources, I made this do the same thing
2018-04-26 15:19:41 +02:00
Hans-Christoph Steiner
e17815e9f0 handle bad SDK Version values in APKs
Even though it is invalid to have *SdkVersion in AndroidManifest.xml set as
anything but an integer, sometimes people manage to get something in there.
`fdroid update` needs to be able to handle all that.

* https://developer.android.com/guide/topics/manifest/uses-sdk-element#min
* https://gitlab.com/souch/SMSbypass/blob/v0.9/app/src/main/AndroidManifest.xml#L29
* https://gitlab.com/souch/SMSbypass/blob/v0.9/app/src/main/res/values/strings.xml#L27

admin#65
2018-04-26 15:19:41 +02:00
Hans-Christoph Steiner
4ba5b8b5ec ignore index files generated as part of manual testing
it is quite useful to run `../fdroid update --pretty` directly in tests/
2018-04-26 12:33:44 +02:00
Hans-Christoph Steiner
74c6555c71 index: ensure index.xml timestamp is in UTC, closes #481
use verbose but safe syntax so this works with all versions of sed, e.g. osx
2018-04-25 16:35:22 +02:00
Marcus
8fbe0738ec Merge branch 'gradle_4.7' into 'master'
makebuildserver: add gradle 4.7

See merge request fdroid/fdroidserver!493
2018-04-22 18:00:33 +00:00
Marcus Hoffmann
a4fe2f711c
makebuildserver: add gradle 4.7 2018-04-20 17:55:51 +02:00
Hans-Christoph Steiner
afe0c6a167 fix scan_apk_androguard when parsing versionName without a dot (e.g. 5)
fixes de35f1b05b:

Traceback (most recent call last):
  File "/home/hans/code/fdroid/server/fdroid", line 164, in <module>
    main()
  File "/home/hans/code/fdroid/server/fdroid", line 138, in main
    mod.main()
  File "/export/share/code/fdroid/server/fdroidserver/update.py", line 1932, in main
    apks, cachechanged = process_apks(apkcache, repodirs[0], knownapks, options.use_date_from_apk)
  File "/export/share/code/fdroid/server/fdroidserver/update.py", line 1459, in process_apks
    use_date_from_apk, ada, True)
  File "/export/share/code/fdroid/server/fdroidserver/update.py", line 1332, in process_apk
    apk = scan_apk(apkfile)
  File "/export/share/code/fdroid/server/fdroidserver/update.py", line 1051, in scan_apk
    scan_apk_androguard(apk, apk_file)
  File "/export/share/code/fdroid/server/fdroidserver/update.py", line 1220, in scan_apk_androguard
    res_id = arsc.get_id(apk['packageName'], res_id)[1]
TypeError: 'NoneType' object is not subscriptable

For example https://f-droid.org/archive/com.abitsinc.andr_5.apk:
$ aapt dump badging archive/com.abitsinc.andr_5.apk |head -1
package: name='com.abitsinc.andr' versionCode='5' versionName='5' platformBuildVersionName='2.3.3'
2018-04-18 12:32:28 +02:00
Hans-Christoph Steiner
6cc26ad8c8 Merge branch 'blank-versionName' into 'master'
handle APKs with a blank versionName

Closes #477, #478, fdroidclient#1416, fdroidclient#1417, and fdroidclient#1418

See merge request fdroid/fdroidserver!492
2018-04-17 13:02:55 +00:00
Hans-Christoph Steiner
de35f1b05b update: handle APKs with a blank versionName
Instead of just crashing, first try to use the versionName as written in the
build metadata, otherwise just let it be blank.  A blank versionName will
cause fdroidclient < 1.3 to crash.  Blank versionNames are not allowed in
the .txt metadata format, only .yml.

closes #477
closes #478
closes fdroidclient#1416
closes fdroidclient#1417
closes fdroidclient#1418
fdroiddata!3061
2018-04-17 12:36:49 +02:00
Hans-Christoph Steiner
9bf9159a77 update: use only 7 chars of SHA256 for non-APK version name
#460
2018-04-17 12:36:49 +02:00
Hans-Christoph Steiner
74c0e98d82 update index metadata tests for 673a3517d3 2018-04-16 11:47:48 +02:00
Hans-Christoph Steiner
673a3517d3 add test of Android TV "activity banner" graphic
!491

https://developer.android.com/guide/topics/manifest/activity-element.html#banner
https://developer.android.com/training/tv/start/start.html#banner
2018-04-16 11:37:11 +02:00
Hans-Christoph Steiner
0cea73cd2f Merge branch 'patch-1' into 'master'
fdroidserver/update.py: APK_LABEL_ICON_PAT icon pattern should not be greedy

See merge request fdroid/fdroidserver!491
2018-04-16 09:37:42 +00:00
Phoenix09
ea89b25fd7 fdroidserver/update.py: APK_LABEL_ICON_PAT icon pattern should not be greedy 2018-04-14 23:30:23 +01:00
Marcus
eec005ed67 Merge branch 'wiki-page-name' into 'master'
Filter another invalid mediawiki page title character

See merge request fdroid/fdroidserver!490
2018-04-11 13:35:42 +00:00
Ciaran Gultnieks
bd00b87e4e Filter another invalid mediawiki page title character 2018-04-11 14:32:55 +01:00
Hans-Christoph Steiner
33c6d5570e jenkins-test: remove redundant fdroid update test
This test ran before config.py was setup, so it didn't respect the test
setup.  Things like "make_current_version_link = False" are needed so that
the checkupdates tests work like on the official infrastructure.  Also,
./tests/run-tests has many `fdroid update` tests, so having it here just
wastes time.  The tests that are directly in this script are meant to test
the separate build/sign setup.
2018-04-09 10:27:29 +02:00
Marcus
94a094323e Merge branch 'mwclient-dep' into 'master'
Replace deprecated mwclient method

See merge request fdroid/fdroidserver!489
2018-04-08 11:53:26 +00:00
Ciaran Gultnieks
31e8605864 Replace deprecated mwclient method
It's functionally the same, just a different name.
2018-04-08 09:37:48 +01:00
Hans-Christoph Steiner
ea3f9df473 Merge branch 'patch-1' into 'master'
Fix a typo in README.md

See merge request fdroid/fdroidserver!488
2018-04-06 20:24:54 +00:00
scde
2890b2feb2 Fix a typo in README.md 2018-04-06 17:16:56 +00:00
Hans-Christoph Steiner
45ff2231b0 jenkins-test: turn off link generation for checkupdates test
These links will make checkupdates fail since they will make the git repo
dirty.  There is also a mystery "cache/" subdir, but I don't know what is
making that.
2018-04-05 21:03:13 +02:00
relan
4a5c921aee Merge branch 'gradle_plugin_3.1' into 'master'
add gradle plugin 3.1 which requires gradle 4.4

See merge request fdroid/fdroidserver!487
2018-04-05 12:34:47 +00:00
Marcus Hoffmann
970653bd92
add gradle plugin 3.1 which requires gradle 4.4 2018-04-05 14:12:35 +02:00
Hans-Christoph Steiner
47a930f9f3 Merge branch 'java-gentoo' into 'master'
added java selection criteria for gentoo linux

See merge request fdroid/fdroidserver!486
2018-04-04 17:04:40 +00:00
Hans-Christoph Steiner
c54e32a3b8 Merge branch 'fix-icon-crash' into 'master'
Fix crash when icon_name is None

See merge request fdroid/fdroidserver!485
2018-04-04 17:04:24 +00:00
lb@lb520
a30851ec98 added java selection criteria for gentoo linux 2018-04-04 16:39:25 +02:00
Jonas Kalderstam
9e4459d7d1 Add test for when apk icon src could not be found
An APK (Netflix) was found to have the following icon filename:

    \u2003\u2009\n.xml

This breaks the aapt dump parsing because it iterates line by line and
this filename goes across two lines. Consequently, icon_src will be
None (default value) when it is passed to the icons parser.
2018-03-27 18:39:59 +02:00
Jonas Kalderstam
1f706283ce Fix crash when icon_name is None
Fixes the following crash:

```
$ fdroid update --create-metadata --rename-apks
WARNING: Using Java's jarsigner, not recommended for verifying APKs! Use apksigner
CRITICAL: Unknown exception found!
Traceback (most recent call last):
  File "/home/jonas/miniconda3/bin/fdroid", line 164, in <module>
    main()
  File "/home/jonas/miniconda3/bin/fdroid", line 138, in main
    mod.main()
  File "/home/jonas/miniconda/lib/python3.6/site-packages/fdroidserver/update.py", line 1927, in main
    apks, cachechanged = process_apks(apkcache, repodirs[0], knownapks, options.use_date_from_apk)
  File "/home/jonas/miniconda/lib/python3.6/site-packages/fdroidserver/update.py", line 1454, in process_apks
    use_date_from_apk, ada, True)
  File "/home/jonas/miniconda/lib/python3.6/site-packages/fdroidserver/update.py", line 1327, in process_apk
    apk = scan_apk(apkfile)
  File "/home/jonas/miniconda/lib/python3.6/site-packages/fdroidserver/update.py", line 1056, in scan_apk
    scan_apk_aapt(apk, apk_file)
  File "/home/jonas/miniconda/lib/python3.6/site-packages/fdroidserver/update.py", line 1184, in scan_apk_aapt
    apk['icons_src'] = _get_apk_icons_src(apkfile, icon_name)
  File "/home/jonas/miniconda/lib/python3.6/site-packages/fdroidserver/update.py", line 1089, in _get_apk_icons_src
    density_re = re.compile('^res/(.*)/' + icon_name + '\.(png|xml)$')
TypeError: must be str, not NoneType
```
2018-03-27 14:17:08 +02:00
Hans-Christoph Steiner
b82f074919 bump to 1.0.3 2018-03-22 23:19:57 +01:00
Hans-Christoph Steiner
4ca77a4faf update: strip all whitespace from buildserverid 2018-03-22 23:19:54 +01:00
Hans-Christoph Steiner
8ab2deace2 Merge branch 'weblate' into 'master'
Weblate

See merge request fdroid/fdroidserver!483
2018-03-22 22:21:06 +00:00
Hans-Christoph Steiner
02203efe15 Weblate 2018-03-22 22:21:06 +00:00
Marcus
3ee4855945 Merge branch 'gradle-4.6' into 'master'
makebuildserver: add Gradle 4.6

See merge request fdroid/fdroidserver!482
2018-03-16 08:33:52 +00:00
relan
64fad95fa1 makebuildserver: add Gradle 4.6 2018-03-16 07:39:44 +03:00
Hans-Christoph Steiner
4b788b3b09 Merge branch 'master' into 'master'
update: handle AndroidManifest XML namespace named other than 'android'

See merge request fdroid/fdroidserver!481
2018-03-15 15:23:55 +00:00
Hans-Christoph Steiner
54242dee32 update: handle AndroidManifest XML namespace named other than 'android'
https://stackoverflow.com/questions/5856719/why-do-we-specify-namespace-in-android-xml-file

https://f-droid.org/repo/org.moire.ultrasonic_60.apk has:
xmlns:a="http://schemas.android.com/apk/res/android"
2018-03-15 15:56:30 +01:00
Hans-Christoph Steiner
d290eb96f9 Merge branch 'master' into 'master'
update: handle renameManifestPackage when extracting icons with androguard

See merge request fdroid/fdroidserver!480
2018-03-14 18:09:19 +00:00
Hans-Christoph Steiner
b5c1634320 update: handle renameManifestPackage when extracting icons with androguard
aapt --rename-manifest-package changes the applicationId for an app without
changing the packageName listed in AndroidManifest.xml under
<application android:package="">
2018-03-14 18:43:05 +01:00
Hans-Christoph Steiner
cd8b583db1 Merge branch 'master' into 'master'
update: support all official DPIs when extracting icons

See merge request fdroid/fdroidserver!479
2018-03-13 16:12:52 +00:00
Hans-Christoph Steiner
493c485755 update: support all official DPIs when extracting icons
https://developer.android.com/training/multiscreen/screendensities.html
2018-03-13 16:47:24 +01:00
Hans-Christoph Steiner
b445fc066b Merge branch 'master' into 'master'
two update parsing fixes  when using androguard

See merge request fdroid/fdroidserver!477
2018-03-13 11:36:43 +00:00
Hans-Christoph Steiner
3b09e5ee08 update: do not crash on android-26+ XML icon
For example:
res/drawable-v26/icon.xml

Here's the full range of possibilities, we have a lot of work to do:

$ for f in */*.apk; do unzip -l $f |grep -F icon.| grep -Eo 'res/drawable[^/]*'; done | sort -u
res/drawable
res/drawable-anydpi-v21
res/drawable-anydpi-v26
res/drawable-hdpi
res/drawable-hdpi-v11
res/drawable-hdpi-v12
res/drawable-hdpi-v4
res/drawable-hdpi-v5
res/drawable-hdpi-v6
res/drawable-hdpi-v9
res/drawable-large
res/drawable-large-hdpi-v11
res/drawable-large-hdpi-v4
res/drawable-large-ldpi-v4
res/drawable-large-mdpi
res/drawable-large-mdpi-v11
res/drawable-large-v4
res/drawable-large-xhdpi-v11
res/drawable-large-xhdpi-v4
res/drawable-large-xxhdpi-v11
res/drawable-large-xxhdpi-v4
res/drawable-ldpi
res/drawable-ldpi-v11
res/drawable-ldpi-v4
res/drawable-ldpi-v5
res/drawable-ldpi-v6
res/drawable-ldpi-v9
res/drawable-ldrtl-v17
res/drawable-mdpi
res/drawable-mdpi-v11
res/drawable-mdpi-v12
res/drawable-mdpi-v4
res/drawable-mdpi-v5
res/drawable-mdpi-v6
res/drawable-mdpi-v9
res/drawable-nodpi
res/drawable-nodpi-v4
res/drawable-tvdpi-v4
res/drawable-v11
res/drawable-v14
res/drawable-v21
res/drawable-xhdpi
res/drawable-xhdpi-v11
res/drawable-xhdpi-v12
res/drawable-xhdpi-v4
res/drawable-xhdpi-v9
res/drawable-xlarge-hdpi
res/drawable-xlarge-hdpi-v4
res/drawable-xlarge-mdpi
res/drawable-xlarge-mdpi-v4
res/drawable-xlarge-v4
res/drawable-xlarge-xhdpi-v4
res/drawable-xxhdpi
res/drawable-xxhdpi-v11
res/drawable-xxhdpi-v21
res/drawable-xxhdpi-v4
res/drawable-xxhdpi-v9
res/drawable-xxxhdpi
res/drawable-xxxhdpi-v11
res/drawable-xxxhdpi-v4
2018-03-13 12:13:23 +01:00
Hans-Christoph Steiner
e89478e529 update: do not crash on APKs without icons when using androguard 2018-03-13 11:19:19 +01:00
Hans-Christoph Steiner
2c8008356b update: handle bad XML when using androguard
repo/ch.swift.willi_417101.apk had a C/Java comment in the
AndroidManifest.xml rather than an XML comment:

<uses-sdk android:minSdkVersion="16" android:targetSdkVersion="26">
</uses-sdk>
// Remove permissions introduced by the appsflyer library
<uses-permission android:name="android.permission.ACCESS_COARSE_LOCATION">
</uses-permission>
2018-03-12 17:55:45 +01:00
Hans-Christoph Steiner
d98d8c2e52 update: do not crash with androguard when working with apps without icons
For example, SpeedoMeterApp.main_1.apk
2018-03-12 17:55:45 +01:00