1
0
mirror of https://gitlab.com/fdroid/fdroidserver.git synced 2024-11-15 03:20:10 +01:00
Commit Graph

2768 Commits

Author SHA1 Message Date
Michael Pöhn
4e69ff582f run yamllint when parsing failed and also on fdroid lint runs 2020-04-24 15:47:31 +02:00
Jochen Sprickerhof
30c654459f Use ImportError for Python < 3.6 (Closes: !734)
ModuleNotFoundError is a subclass of ImportError, so this should not
change anything.
2020-04-17 23:26:06 +02:00
Michael Pöhn
911e86fc87 support copying yml srclibs to buildserver vm during builds 2020-04-16 11:32:37 +02:00
Michael Pöhn
5741e6930b yml srclibs: support Subdir as list 2020-04-16 11:31:59 +02:00
Michael Pöhn
1ac7d612b1 yml srclibs: support multiline Prepare 2020-04-16 11:31:59 +02:00
Michael Pöhn
aa020af040 srclibs: forward yml parsing error cause 2020-04-16 11:31:59 +02:00
Michael Pöhn
ee3d8d2f18 srclib: yml parsing + tests for yml and txt 2020-04-16 11:31:59 +02:00
Michael Pöhn
2c86d38028 srclib: no spaces in dict key 2020-04-16 11:31:59 +02:00
Michael Pöhn
c6a97939f1 rename parse_srclib to parese_txt_srclib + test case 2020-04-16 11:31:59 +02:00
Hans-Christoph Steiner
c47f9ef123
index: xml.dom.minidom no longer sorts attribs
It seems now that xml.dom.minidom preserves the order of attributes, rather
than sorting them.  We assume alpha-sort, so this manually

This diff in the test suite running on Debian/testing pointed it out:
https://gitlab.com/fdroid/fdroidserver/-/jobs/486970383

```diff
--- /builds/fdroid/fdroidserver/tests/repo/index.xml	2020-04-11 13:36:57.000000000 +0000
+++ repo/index.xml	2020-04-11 13:41:44.000000000 +0000
@@ -1,6 +1,6 @@
 <?xml version="1.0" encoding="utf-8"?>
 <fdroid>
-	<repo icon="fdroid-icon.png" name="My First F-Droid Repo Demo" pubkey="308204e1308202c9a003020102020434597643300d06092a864886f70d01010b050030213110300e060355040b1307462d44726f6964310d300b06035504031304736f7661301e170d3136303931333230313930395a170d3434303133303230313930395a30213110300e060355040b1307462d44726f6964310d300b06035504031304736f766130820222300d06092a864886f70d01010105000382020f003082020a028202010086ef94b5aacf2ba4f38c875f4194b44f5644392e3715575d7c92828577e692c352b567172823851c8c72347fbc9d99684cd7ca3e1db3e4cca126382c53f2a5869fb4c19bdec989b2930501af3e758ff40588915fe96b10076ce3346a193a0277d79e83e30fd8657c20e35260dd085aa32eac7c4b85786ffefbf1555cafe2bc928443430cdbba48cfbe701e12ae86e676477932730d4fc7c00af820aef85038a5b4df084cf6470d110dc4c49ea1b749b80b34709d199b3db516b223625c5de4501e861f7d261b3838f8f616aa78831d618d41d25872dc810c9b2087b5a9e146ca95be740316dcdbcb77314e23ab87d4487913b800b1113c0603ea2294188b71d3e49875df097b56f9151211fc6832f9790c5c83d17481f14ad37915fd164f4fd713f6732a15f4245714b84cd665bdbd085660ea33ad7d7095dcc414f09e3903604a40facc2314a115c0045bb50e9df38efb57e1b8e7cc105f340a26eeb46aba0fa6672953eee7f1f92dcb408e561909bbd4bdf4a4948c4d57c467d21aa238c34ba43be050398be963191fa2b49828bc1e4eeed224b40dbe9dc3e570890a71a974a2f4527edb1b07105071755105edcb2af2f269facfb89180903a572a99b46456e80d4a01685a80b233278805f2c876678e731f4ec4f52075aeef6b2b023efbb8a3637ef507c4c37c27e428152ec1817fcba640ad601cb09f72f0fbe2d274a2410203010001a321301f301d0603551d0e04160414c28bf33dd5a9a17338e5b1d1a6edd8c7d141ed0b300d06092a864886f70d01010b0500038202010084e20458b2aafd7fc27146b0986f9324f4260f244920417a77c9bf15e2e2d22d2725bdd8093ec261c3779c3ca03312516506f9410075b90595b41345956d8eb2786fb5994f195611382c2b99dba13381b0100a30bc9e6e47248bf4325e2f6eec9d789216dc7536e753bf1f4be603d9fa2e6f5e192b4eb988b8cdb0bb1e8668a9225426f7d4636479f73ed24ad1d2657c31e63c93d9679b9080171b3bd1bf10a3b92b80bd790fbf62d3644900cd08eae8b9bf9c2567be98dc8cdd2ae19a8d57a3e3e2de899f81f1279f578989e6af906f80c8c2b67651730ee7e568c1af5bcb845b6d685dc55332a9984aeceaea3b7e883447edf1c76b155d95253e39b9710eaa22efa6c81468829702b5dce7126538f3ca70c2f0ad9a5795435fdb1f715f20d60359ef9a9926c7050116e802df651727447848827815f70bd82af3cedd08783156102d2d8ce995c4c43b8e47e91a3e6927f3505a5d395e6bebb84542c570903eeab4382a1c2151f1471c7a06a34dc4d268d8fa72e93bdcd2dccc4302ecac47b9e7e3d8bc9b46d21cd097874a24d529548018dc190ff568c6aa428f0a5eedff1a347730931c74f19277538e49647a4ad7254f4c1ec7d4da12cce9e1fad9607534e66ab40a56b473d9d7e3d563fd03cad2052bad365c5a29f8ae54f09b60dbca3ea768d7767cbe1c133ca08ce725c1c1370f4aab8e5b6e286f52dc0be8d0982b5a" timestamp="1480431575" url="https://MyFirstFDroidRepo.org/fdroid/repo" version="21">
+	<repo name="My First F-Droid Repo Demo" icon="fdroid-icon.png" url="https://MyFirstFDroidRepo.org/fdroid/repo" version="21" timestamp="1480431575" pubkey="308204e1308202c9a003020102020434597643300d06092a864886f70d01010b050030213110300e060355040b1307462d44726f6964310d300b06035504031304736f7661301e170d3136303931333230313930395a170d3434303133303230313930395a30213110300e060355040b1307462d44726f6964310d300b06035504031304736f766130820222300d06092a864886f70d01010105000382020f003082020a028202010086ef94b5aacf2ba4f38c875f4194b44f5644392e3715575d7c92828577e692c352b567172823851c8c72347fbc9d99684cd7ca3e1db3e4cca126382c53f2a5869fb4c19bdec989b2930501af3e758ff40588915fe96b10076ce3346a193a0277d79e83e30fd8657c20e35260dd085aa32eac7c4b85786ffefbf1555cafe2bc928443430cdbba48cfbe701e12ae86e676477932730d4fc7c00af820aef85038a5b4df084cf6470d110dc4c49ea1b749b80b34709d199b3db516b223625c5de4501e861f7d261b3838f8f616aa78831d618d41d25872dc810c9b2087b5a9e146ca95be740316dcdbcb77314e23ab87d4487913b800b1113c0603ea2294188b71d3e49875df097b56f9151211fc6832f9790c5c83d17481f14ad37915fd164f4fd713f6732a15f4245714b84cd665bdbd085660ea33ad7d7095dcc414f09e3903604a40facc2314a115c0045bb50e9df38efb57e1b8e7cc105f340a26eeb46aba0fa6672953eee7f1f92dcb408e561909bbd4bdf4a4948c4d57c467d21aa238c34ba43be050398be963191fa2b49828bc1e4eeed224b40dbe9dc3e570890a71a974a2f4527edb1b07105071755105edcb2af2f269facfb89180903a572a99b46456e80d4a01685a80b233278805f2c876678e731f4ec4f52075aeef6b2b023efbb8a3637ef507c4c37c27e428152ec1817fcba640ad601cb09f72f0fbe2d274a2410203010001a321301f301d0603551d0e04160414c28bf33dd5a9a17338e5b1d1a6edd8c7d141ed0b300d06092a864886f70d01010b0500038202010084e20458b2aafd7fc27146b0986f9324f4260f244920417a77c9bf15e2e2d22d2725bdd8093ec261c3779c3ca03312516506f9410075b90595b41345956d8eb2786fb5994f195611382c2b99dba13381b0100a30bc9e6e47248bf4325e2f6eec9d789216dc7536e753bf1f4be603d9fa2e6f5e192b4eb988b8cdb0bb1e8668a9225426f7d4636479f73ed24ad1d2657c31e63c93d9679b9080171b3bd1bf10a3b92b80bd790fbf62d3644900cd08eae8b9bf9c2567be98dc8cdd2ae19a8d57a3e3e2de899f81f1279f578989e6af906f80c8c2b67651730ee7e568c1af5bcb845b6d685dc55332a9984aeceaea3b7e883447edf1c76b155d95253e39b9710eaa22efa6c81468829702b5dce7126538f3ca70c2f0ad9a5795435fdb1f715f20d60359ef9a9926c7050116e802df651727447848827815f70bd82af3cedd08783156102d2d8ce995c4c43b8e47e91a3e6927f3505a5d395e6bebb84542c570903eeab4382a1c2151f1471c7a06a34dc4d268d8fa72e93bdcd2dccc4302ecac47b9e7e3d8bc9b46d21cd097874a24d529548018dc190ff568c6aa428f0a5eedff1a347730931c74f19277538e49647a4ad7254f4c1ec7d4da12cce9e1fad9607534e66ab40a56b473d9d7e3d563fd03cad2052bad365c5a29f8ae54f09b60dbca3ea768d7767cbe1c133ca08ce725c1c1370f4aab8e5b6e286f52dc0be8d0982b5a">
 		<description>This is a repository of apps to be used with F-Droid. Applications in this repository are either official binaries built by the original application developers, or are binaries built from source by the admin of f-droid.org using the tools on https://gitlab.com/u/fdroid. </description>
 		<mirror>http://foobarfoobarfoobar.onion/fdroid/repo</mirror>
 		<mirror>https://foo.bar/fdroid/repo</mirror>
@@ -94,9 +94,9 @@
 			<added>2017-12-22</added>
 			<sig>056c9f1554c40ba59a2103009c82b420</sig>
 			<permissions>ACCESS_NETWORK_STATE,ACCESS_WIFI_STATE,CHANGE_WIFI_MULTICAST_STATE,INTERNET,READ_EXTERNAL_STORAGE,WRITE_EXTERNAL_STORAGE</permissions>
-			<uses-permission maxSdkVersion="18" name="android.permission.READ_EXTERNAL_STORAGE"/>
-			<uses-permission maxSdkVersion="18" name="android.permission.WRITE_EXTERNAL_STORAGE"/>
-			<uses-permission-sdk-23 maxSdkVersion="27" name="android.permission.REQUEST_IGNORE_BATTERY_OPTIMIZATIONS"/>
+			<uses-permission name="android.permission.READ_EXTERNAL_STORAGE" maxSdkVersion="18"/>
+			<uses-permission name="android.permission.WRITE_EXTERNAL_STORAGE" maxSdkVersion="18"/>
+			<uses-permission-sdk-23 name="android.permission.REQUEST_IGNORE_BATTERY_OPTIMIZATIONS" maxSdkVersion="27"/>
 		</package>
 	</application>
 	<application id="fake.ota.update">
@@ -182,9 +182,9 @@
 			<added>2013-12-31</added>
 			<sig>eb41d4d6082bb3e81c3d58dbf7fc7332</sig>
 			<permissions>ACCESS_NETWORK_STATE,ACCESS_WIFI_STATE,BLUETOOTH,BLUETOOTH_ADMIN,CHANGE_NETWORK_STATE,CHANGE_WIFI_MULTICAST_STATE,CHANGE_WIFI_STATE,INTERNET,NFC,RECEIVE_BOOT_COMPLETED</permissions>
-			<uses-permission maxSdkVersion="22" name="android.permission.ACCESS_NETWORK_STATE"/>
-			<uses-permission maxSdkVersion="18" name="android.permission.BLUETOOTH_ADMIN"/>
-			<uses-permission-sdk-23 maxSdkVersion="25" name="android.permission.WRITE_SETTINGS"/>
+			<uses-permission name="android.permission.ACCESS_NETWORK_STATE" maxSdkVersion="22"/>
+			<uses-permission name="android.permission.BLUETOOTH_ADMIN" maxSdkVersion="18"/>
+			<uses-permission-sdk-23 name="android.permission.WRITE_SETTINGS" maxSdkVersion="25"/>
 		</package>
 	</application>
 	<application id="obb.main.twoversions">
```
2020-04-15 21:53:39 +02:00
Jochen Sprickerhof
86beac22e2 Use libarchive instead of the Python implementation 2020-04-15 18:27:13 +00:00
Hans-Christoph Steiner
e85573d0e1 Merge branch 'git-push-logging' into 'master'
Show debug messages to give clue why pushing git mirror failed

See merge request fdroid/fdroidserver!731
2020-04-15 12:56:04 +00:00
Hans-Christoph Steiner
f39732987e Merge branch 'patch-1' into 'master'
Import - sudo refresh repos too and use the newer npm

See merge request fdroid/fdroidserver!730
2020-04-15 12:53:56 +00:00
Izzy
46a464a74b
get_app_from_url: add codeberg.org as git url 2020-04-15 09:13:08 +02:00
Torsten Grote
7d69f04b94
Show debug messages to give clue why pushing git mirror failed 2020-04-10 13:34:46 -03:00
Licaon_Kter
463f6823de Import - sudo refresh repos too and use the newer npm 2020-04-08 10:25:53 +00:00
Hans-Christoph Steiner
138bc73668
import: --omit-disable option to easily work with generated files 2020-03-11 22:13:47 +01:00
Hans-Christoph Steiner
1bb9cf43e1
import: ensure gradle: yes is added for detected Gradle builds 2020-03-11 22:13:45 +01:00
Hans-Christoph Steiner
a1e17d2b2c
deploy: retry virustotal 30 seconds after rate limiting, and log 2020-03-11 22:13:42 +01:00
Hans-Christoph Steiner
b7901952a1 deploy: make androidobservatory and virustotal functions reusable
This should not change the logic at all, just make the loop runs into
standalone functions.
2020-03-11 22:13:39 +01:00
Hans-Christoph Steiner
733e7be1b3 set F-Droid HTTP Headers globally 2020-03-11 13:41:13 +01:00
Hans-Christoph Steiner
ab2291475b import: mv reusable functions to common.py to avoid import_proxy.py
import is a strict keyword in Python, so it is not possible to import a
module called 'import', even with things like:

* import fdroidserver.import
* from fdroidserver import import
2020-03-11 13:41:13 +01:00
Marcus Hoffmann
56ee7fe765 opencollecive: fix allowed characters
The url slug used by opencollective allows case variations, they all
lead back to the canonical lowercase variant though.

Apart from ascii characters and numbers only - and _ are allowed.
2020-03-10 16:26:34 +01:00
Marcus
9d24f2e4a7 add opencollective metadata and index field 2020-03-10 14:56:03 +00:00
fuwa
2d63ab66f5 libvirt: various fixes related to the makebuildserver and fdroid build commands 2020-03-09 15:06:19 +00:00
Torsten Grote
14208a6761
Add stats support to nightly repo
This will keep track of when APKs were added
and show these dates properly in the client
instead of always resetting all dates to the latest update.
2020-03-02 14:17:42 -03:00
Hans-Christoph Steiner
b5bce9f178
build: do not write/rsync status JSON when running --on-server
!717 !716
2020-02-20 16:40:19 +01:00
Hans-Christoph Steiner
b6e6ce87cb
build: has no pretty option (closes #742) 2020-02-19 20:29:54 +01:00
Hans-Christoph Steiner
21f5bd73a7
build: fix logging failed builds to the status JSON (closes #741) 2020-02-19 20:29:51 +01:00
Hans-Christoph Steiner
bf6004b08e
rsync status JSON as first and last steps of command runs 2020-02-19 14:50:35 +01:00
Hans-Christoph Steiner
c19ef45fd5
build: write status JSON 2020-02-19 14:50:32 +01:00
Hans-Christoph Steiner
55935d9a41
signindex: write status JSON 2020-02-19 14:50:30 +01:00
Hans-Christoph Steiner
c965e7d969
deploy: write status JSON 2020-02-19 14:50:29 +01:00
Hans-Christoph Steiner
ddbdff83bd
gpgsign: write status JSON 2020-02-19 14:50:26 +01:00
Hans-Christoph Steiner
4bb590b6e5
publish: write status JSON 2020-02-19 14:50:24 +01:00
Hans-Christoph Steiner
e8a6d0f813
checkupdates: write status JSON 2020-02-18 23:45:52 +01:00
Hans-Christoph Steiner
d16478b10b
update: write status in JSON repo file, using new internal API 2020-02-18 23:45:48 +01:00
Hans-Christoph Steiner
5459a461db
common.get_head_commit_id() to get string commit ID from HEAD 2020-02-18 23:16:53 +01:00
Hans-Christoph Steiner
fafaa8f81f
standardize on path.endswith() for file extension checks
Its already widely used, this just removes the exceptions.  It is also more
Pythonic
2020-02-18 23:16:51 +01:00
Hans-Christoph Steiner
202602937d
import: ignore results if build/appid dir already exists 2020-02-18 23:16:49 +01:00
Hans-Christoph Steiner
32a29b3304
import: get git commit ID from git repo, e.g. master instead of '?' 2020-02-18 23:16:44 +01:00
Hans-Christoph Steiner
271b74af7d fix remove_signing_keys() for Kotlin gradles files (*.gradle.kts) 2020-02-13 22:32:51 +01:00
Hans-Christoph Steiner
68b793e308 support kotlin "*.gradle.kts" files in more places
closes #613
2020-02-13 22:32:51 +01:00
Marcus Hoffmann
9ae41cc1ff add support for gradle kotlin scripts
https://guides.gradle.org/migrating-build-logic-from-groovy-to-kotlin/
Closes fdroid/fdroidserver#613

cherry-picked from Bubu/fdroidserver@7d2e9f9c
2020-02-13 22:32:51 +01:00
Hans-Christoph Steiner
83ffeb855f prefer build.gradle with Android Plugin as source of package/version/code
These days, the location that overrides all the others is in the android{}
block of the build.gradle file that loads the com.android.application
plugin.  So this should be the preferred place to read these values.

test files GPL licensed: https://github.com/Integreight/1Sheeld-Android-App
2020-02-13 22:32:51 +01:00
Hans-Christoph Steiner
ad92b4c678 scanner: add --force option for scanning disabled apps/builds 2020-02-13 13:58:13 +01:00
Hans-Christoph Steiner
e037ee5972 import: generate skeleton build fields for react-native and flutter 2020-02-13 13:51:52 +01:00
Hans-Christoph Steiner
1153ac24fd import: overhaul URL validation to use urllib.parse
Python provides us a lovely URL parser with some level of validation built
in.  The parsed URL is then much easier to validate.
2020-02-13 13:51:52 +01:00
Hans-Christoph Steiner
e9a6c84efd import: split URL parsing from code cloning
This makes things testable and easier to follow.
2020-02-13 13:51:52 +01:00
Hans-Christoph Steiner
bfe587979d import: make it work most of the time with git repos
This includes real tests too.
2020-02-13 13:51:52 +01:00