1
0
mirror of https://gitlab.com/fdroid/fdroidserver.git synced 2024-11-19 05:10:10 +01:00
Commit Graph

734 Commits

Author SHA1 Message Date
Daniel Martí
09a0aa2eb9 Also find usual suspects in gradle files 2015-07-05 14:26:41 -07:00
Daniel Martí
b848b99bab Always clean and reset all submodules 2015-06-28 19:37:28 -07:00
Daniel Martí
5aa9e2289d Improve xml string extracting support
* Shield ourselves from newlines that would break metadata files
* Properly use string resources for version names
2015-06-18 17:54:56 +02:00
Daniel Martí
52c2ca1fe7 Replace NDK r10d with r10e 2015-06-18 17:05:48 +02:00
Daniel Martí
e608975f78 Bump build-tools to 22.0.1 2015-06-18 16:56:56 +02:00
Daniel Martí
4db4a86420 Always make sure that we don't encode None
Apparently, we can have a non-nil element that has a nil text.
2015-06-07 14:02:50 +02:00
Daniel Martí
d0b5e60369 Properly support escaped strings
See:
https://developer.android.com/guide/topics/resources/string-resource.html
2015-06-04 15:56:20 +02:00
Daniel Martí
5cab5956e3 Don't accept non-integer vercodes 2015-06-03 19:40:43 +02:00
Daniel Martí
cd93d23062 Work around encoding issues when using xml files
The xml package returns "unicode" objects instead of strings in utf-8.
2015-06-03 18:42:24 +02:00
Daniel Martí
dd37061b41 Don't crash if an <application> has no android:label 2015-06-03 18:30:31 +02:00
Daniel Martí
1b2354858b Fix PEP8 E225 2015-06-03 18:05:17 +02:00
Daniel Martí
9be9b7e919 No need to walk the entire src/main subtree in gradle apps 2015-06-03 15:43:55 +02:00
Daniel Martí
50a3564989 Also use proper xml parsing when retrieving string resources - #58 2015-06-03 15:42:45 +02:00
Daniel Martí
7ab4712892 Also use xml parsing to obtain version from AM.xml - #58 2015-06-03 15:23:18 +02:00
Daniel Martí
7c2bd34c49 Use proper AndroidManifest.xml parsing for labels - #58 2015-06-03 14:35:50 +02:00
Daniel Martí
439cabb7f3 Remove "Srclibs:" from srclib metadata files
They are handled per build, this feature is of little help and practically
unused in master.
2015-06-03 13:52:47 +02:00
Daniel Martí
ac004264f0 Make comma-separated list parsing code common 2015-05-15 17:09:21 +02:00
Ciaran Gultnieks
afa7254a83 Merge branch 'master' into 'master'
make it really easy to upgrade unsigned repos to signed

As a key step to removing support for unsigned repos from fdroidclient (https://gitlab.com/fdroid/fdroidclient/issues/12), this merge request makes `fdroid update` require a signing key.  If there is no keystore, it'll prompt the user to create one using `fdroid update --create-key`.

This closes #13

See merge request !48
2015-05-14 16:09:40 +00:00
Daniel Martí
e916e5df4e Add COMMIT, VERSION and VERCODE recipe vars. Fixes #69 2015-05-10 13:55:16 +02:00
Hans-Christoph Steiner
cc0908bdbb common.write_to_config(): if key doesn't exist in config.py, append it
This lets `fdroid update --create-key` add key/value pairs to config.py
2015-05-08 16:13:15 -04:00
Hans-Christoph Steiner
276123856c move keystore gen logic to common function
This moves the method for generating a new keystore for the repo signing
key to a common function so that it can be used in more subcommands beyond
`fdroid init`.  The immediate idea is to make `fdroid update --create-key`
upgrade unsigned repos to signed ones so we can make fdroidclient only use
signed repos.
2015-05-08 16:13:15 -04:00
Hans-Christoph Steiner
af360199f1 move default keystore location to keystore.jks, i.e. in the fdroid repo
This makes `fdroid init` create a complete, self-contained repo in a single
folder.  That makes it easily transferable, backupable, etc.  It also means
that `fdroid update` can add a keystore to an existing unsigned repo
without having to worry about finding the right folder or overwriting any
other existing files.
2015-05-08 16:13:15 -04:00
Hans-Christoph Steiner
4861595288 move genkey() and genpassword() to common.py for use in multiple sections
genkey() and genpassword() are now going to be used by `fdroid update` as
well as `fdroid init`, so they should be in common.py
2015-05-08 16:13:15 -04:00
Daniel Martí
ce48e8f78d Don't match gradle plugin import lines which are commented 2015-05-08 00:58:52 +02:00
Daniel Martí
1af774811d Merge branch 'projectproperties' into 'master'
Fix project.properties dependency issue

Correctly place scrlib references also
in the case there is no project.properties file
present in the original repository.

This commit fixes a problem, which was introduced with
021df3c424: If no
project.properties file exists in the original
project's source repo, place_srclib will not place
a dependency --- even if there should be one:

>    if not os.path.isfile(proppath):
>        return

Example app: fdroiddata/metadata/org.berlin_vegan.bvapp.txt

See merge request !47
2015-04-17 17:28:35 +00:00
Daniel Martí
3f9d6b2947 Don't try reading lines from directories 2015-04-17 13:05:12 +02:00
micu
96e4062bcf Fix project.properties dependency issue
Correctly place scrlib references also
in the case there is no project.properties file
present in the original repository.

This commit fixes a problem, which was introduced with
021df3c424: If no
project.properties file exists in the original
project's source repo, place_srclib will not place
a dependency --- even if there should be one:

>    if not os.path.isfile(proppath):
>        return

Example app: fdroiddata/metadata/org.berlin_vegan.bvapp.txt
2015-04-17 01:10:23 +02:00
Daniel Martí
acd49c6e4d When patching or checking plugin versions, use *.gradle
Apparently all *.gradle files are valid, e.g. global.gradle.
2015-04-06 17:42:26 +02:00
Daniel Martí
021df3c424 Other minor file reading fixes 2015-04-03 00:14:32 +02:00
Daniel Martí
241a3f02ac Always check that files actually are files before reading
This fixes crashes with e.g. broken symlinks
2015-04-03 00:05:22 +02:00
Daniel Martí
45925c6816 Forgot to bump build_tools in the configs 2015-03-24 21:36:42 +01:00
Ciaran Gultnieks
d3f8a9f276 Signature file names are not fixed 2015-01-31 16:16:21 +00:00
Ciaran Gultnieks
94edef3b34 Need the information about what's different from verify_apks 2015-01-31 15:58:08 +00:00
Parménides GV
f26a908dab Add function to verify apks via jarsigner 2015-01-31 15:55:18 +00:00
Ciaran Gultnieks
e26ad377a6 Handle not-found package name gracefully 2015-01-31 12:07:42 +00:00
Ciaran Gultnieks
384f97e998 Ensure package names are valid 2015-01-26 18:30:01 +00:00
Ciaran Gultnieks
fe7f6cfe50 Remove shell option from FDroidPopen 2015-01-26 18:30:01 +00:00
Hans-Christoph Steiner
e3628b2d74 fix PEP8 E124 closing bracket missing visual indentation 2015-01-21 10:21:54 +01:00
Daniel Martí
3b3704710e Fix adding paths to the PATH env variable 2015-01-13 16:37:10 +01:00
Daniel Martí
ff2b061351 Fix issue introduced in 49549f4cad (fixes #62) 2015-01-13 16:15:11 +01:00
Ciaran Gultnieks
24aaa568da Fix PEP8 issues from latest pull request 2015-01-13 08:26:30 +00:00
Daniel Martí
c60cca57cf Don't limit gradle's forceversion to string literals 2015-01-13 00:28:19 +01:00
Hans-Christoph Steiner
4e68655437 if a build did verify against Binaries:, delete the comparison dirs
If the comparison between the provided APK and the built APK is verified,
then there is no need to keep the old comparison files around. On big build
servers like f-droid.org, there will be thousands of leftover dirs from all
the builds, so this is necessary.
2015-01-12 10:51:54 +01:00
Hans-Christoph Steiner
2458577209 if meld is available, use it to show differences on failed APK compare
meld is much easier to browse when comparing directory trees.
2015-01-12 10:46:04 +01:00
Hans-Christoph Steiner
fc4679cebb if apktool is available, use it to decompress APKs when verifying
apktool decompiles the binary XML to regular XML, so it is much easier to
read for differences.
2015-01-12 10:46:04 +01:00
Hans-Christoph Steiner
74a785a888 uncompress using jar xf in a folder with the same name
This makes it easier to compare the whole APK folders when there are
multiple methods of uncompressing the APK.
2015-01-12 10:46:04 +01:00
Hans-Christoph Steiner
b53746dc7b make resulting dirs of compare_apks() have clearer dir names
This makes it a lot easier to remember which APK is which when trying to
make sense of the differences.
2015-01-12 10:46:04 +01:00
Ciaran Gultnieks
49549f4cad Remove support for git-svn with authentication 2015-01-11 17:46:10 +00:00
Ciaran Gultnieks
1cc1ee32ee Extra debug logging when parsing manifests 2015-01-11 08:20:14 +00:00
Ciaran Gultnieks
1fa6ecc1eb Sanitise tags at point of reading 2015-01-11 08:20:14 +00:00
Daniel Martí
6819e3c30e Set up the ndk in $PATH before each build 2015-01-06 19:41:55 +01:00
Daniel Martí
347ff7b876 Error if any scanignore/delete paths are useless 2015-01-06 14:37:13 +01:00
Daniel Martí
24a8523c0b Gradle cleaner: support multilines with backslashes 2015-01-05 12:59:33 +01:00
Daniel Martí
371e64dfc2 Fix build passing madness (closes #55) 2015-01-05 00:29:27 +01:00
Daniel Martí
3645471a1c Never let build['ndk_path'] be None 2015-01-04 20:41:22 +01:00
Daniel Martí
f60f1bc59e Add NDK r10d, this time alongside r9b
Specified per-build with ndk=<version> defaulting to the oldest, r9b.
2015-01-03 00:07:01 +01:00
Daniel Martí
14f654fabc Apply some autopep8-python2 suggestions 2014-12-31 16:44:06 +01:00
Daniel Martí
e39602586f Make dump xmltree and dump badging silent again 2014-12-31 16:34:11 +01:00
Daniel Martí
e8fb3a143e Fix FDroidPopen refactor typo 2014-12-23 12:54:16 +01:00
Daniel Martí
52c3bf46a5 Catch new version of outputFile 2014-12-23 12:24:07 +01:00
Daniel Martí
1268da7ac6 Remove SilentPopen for consistency with SdkToolsPopen 2014-12-14 15:38:24 +01:00
Hans-Christoph Steiner
5f5bcd2e11 do not set sdk_path in config.py if using system-provided aapt
By not setting sdk_path when /usr/bin/aapt is found, sdk_path then defaults
to $ANDROID_HOME when its used.  Since in this case, aapt will be used from
the system path, using aapt entirely ignores sdk_path.  If the user runs
`fdroid build` in this setup, sdk_path will be $ANDROID_HOME, so it should
check the env vars for it, but maybe that doesn't actually work like that
yet.
2014-12-14 13:26:50 +01:00
Hans-Christoph Steiner
fa1cc48d57 run all SDK tools commands using SdkToolsPopen 2014-12-14 13:25:20 +01:00
Hans-Christoph Steiner
9244256461 find cmds from SDK build-tools in a more flexible way, on the fly
This is a more flexible approach than testing for the complete SDK and
build-tools up front.  This will only test for the commands that are
actually being run, so that if you only have `aapt` installed, you can do
`fdroid update` without errors, but other commands will still give
appropriate errors.

This also makes the build_tools item in config.py optional, it is only
needed if you want to force a specific version of the build-tools.
2014-12-14 13:25:20 +01:00
Hans-Christoph Steiner
f7c9eccc1f switched last aapt call to use central config['aapt'] 2014-12-14 13:25:20 +01:00
Hans-Christoph Steiner
0f71929e71 remove redundant test_sdk_exists() call, its called by test_build_tools_exists()
No need to call test_sdk_exists() before test_build_tools_exists() since
test_build_tools_exists() calls test_sdk_exists() as the first thing it
does.
2014-12-14 13:25:20 +01:00
Daniel Martí
16601a0be3 Bump build-tools again to 21.1.2 2014-12-12 12:57:12 +01:00
Daniel Martí
aaf36f654d Update build-tools to 21.1.1 2014-12-12 12:40:09 +01:00
Hans-Christoph Steiner
e826938734 static URLs to "Current Version" of each app
I wrote up the feature to automatically generate symlinks with a constant name
that points to the current release version. I have it on by default, with a
*config.py* option to turn it off. There is also an option to set where the
symlink name comes from which defaults to app['Name'] i.e. F-Droid.apk, but
can easily be set to app['id'], i.e. _org.fdroid.fdroid.apk_. I think the best
place for the symlinks is in the root of the repo, so like
https://f-droid.org/F-Droid.apk or https://guardianproject.info/fdroid/ChatSecure.apk

For the case of the current FDroid static link https://f-droid.org/FDroid.apk
it can just be a symlink to the generated one (https://f-droid.org/F-Droid.apk
or https://f-droid.org/org.fdroid.fdroid.apk). Right now, this feature is all
or nothing, meaning it generates symlinks for all apps in the repo, or none. I
can’t think of any problems that this might cause since its only symlinks, so
the amount of disk space is tiny. Also, I think it would be useful for having
an easy “Download this app” button on each app’s page on the “Browse” view. As
long as this button is less prominent than the “Download F-Droid” button, and
it is clear that it is better to use the FDroid app than doing direct
downloads. For the f-droid.org repo, the symlinks should probably be based on
app['id'] to prevent name conflicts.

more info here:
https://f-droid.org/forums/topic/static-urls-to-current-version-of-each-app/
2014-11-11 15:06:52 +01:00
Daniel Martí
3829896a8a Fix all pep8 warnings 2014-10-24 22:23:58 +02:00
Ciaran Gultnieks
8568805866 Support for publishing signed binaries from elsewhere
Done after verifying that they match ones built using a recipe.
Everything in the metadata should be the same as normal, with the
addition of the Binaries: directive to specify where (with pattern
substitution) to get the binaries from.

Publishing only takes place if there is a proper match. (Which seems
very unlikely to be the case unless the exact same toolchain is used, so
I would imagine that unless the person building and signing the incoming
binaries uses fdroidserver to build them, probably the exact same
buildserver id, they will not match. But at least we have the
functionality to support that.)
2014-10-24 21:04:15 +01:00
Daniel Martí
57efa16f3f Bump build-tools to 21.0.2 2014-10-22 15:01:01 +02:00
Daniel Martí
c77c9f06fc 'ndk_path' will always be in the config 2014-09-25 18:13:16 +02:00
Daniel Martí
c9daf93f69 Create local.properties if it doesn't exist 2014-09-25 18:11:56 +02:00
Daniel Martí
9327adfe74 Name config args thisconfig to avoid confusion with the global config 2014-09-17 00:12:24 +02:00
Daniel Martí
ce61186cd2 Fix issue where gradle=yes would be removed by rewritemeta 2014-09-15 17:03:45 +02:00
Daniel Martí
e038b4424c config['sdk_path'] will never be None, behave properly if ANDROID_HOME is unset 2014-09-15 12:49:36 +02:00
Daniel Martí
37aa3a7b99 Only expand paths on config items that can actually be paths 2014-09-15 12:39:42 +02:00
Daniel Martí
88dc0b2bb5 Keep consistency in config defaults and writes
* Default paths are not expanded
* When writing to config.py in "fdroid init", don't write expanded paths
  either
* Support changes in e.g. $ANDROID_HOME after a config.py is generated thanks
  to the fix above
2014-09-15 12:34:40 +02:00
Daniel Martí
34a3405208 Centralise handling of default gradle flavours 2014-09-13 13:04:24 +02:00
Daniel Martí
a195556378 Make gradle and antcommands (previously antcommand) proper lists 2014-09-13 13:01:08 +02:00
Daniel Martí
93a0d9918d Closes #34: Catch OSErrors when running Popen 2014-09-11 23:08:51 +02:00
Ciaran Gultnieks
b454ea3c6f Make stats retrieval more configurable 2014-09-02 18:53:36 +01:00
Daniel Martí
5e988eb84d common: discard wrapping quotes in strings
Fixes auto names that shouldn't have wrapping quotes like '"PassDroid"'
instead of just 'PassDroid'.
2014-08-25 08:45:46 +02:00
Daniel Martí
f7f8c80018 Fix "no such package" search logic 2014-08-25 00:28:18 +02:00
Ciaran Gultnieks
b43f7bea1a Add ability to filter asshattery from stats 2014-08-22 21:18:55 +01:00
Daniel Martí
58a88acd92 Make the scanner log scanignore as well 2014-08-21 20:02:52 +02:00
Daniel Martí
c5cab888e4 Revert some stuff from 94c29f9c37 that shouldn't have been commited 2014-08-18 11:25:39 +02:00
Daniel Martí
94c29f9c37 Map apps in memory from appid to appinfo
Instead of storing them in a list and doing linear searches by appinfo['id']
2014-08-16 12:46:02 +02:00
Daniel Martí
51c93c9576 Be more specific about when git svn is used 2014-08-16 10:55:21 +02:00
Daniel Martí
bf227948a9 Make use of Popen.wait() return value 2014-08-10 12:28:19 +02:00
Daniel Martí
94e2b120e4 Remove extra argument in magic warning call 2014-07-24 08:55:54 +02:00
Daniel Martí
a5f0f439c5 Use --before when using find-rev in git-svn 2014-07-23 17:26:33 +02:00
Daniel Martí
5167e3d771 Don't crash if a magic number can't be used
This should fix the anki build crashing.
2014-07-22 10:06:51 +02:00
Daniel Martí
78ff22d952 Drop svn support in favour of git-svn
Reasons:

* Cloning a svn repo via svn doesn't fetch the entire history
* Svn checkout is incredibly slow
* Svn doesn't have important features such as a 'clean' command

The only reason why we kept svn was for anonymous logins to repositories. This
is no longer a reason since git-svn also supports them.
2014-07-18 12:39:24 +02:00
Daniel Martí
13987b5c77 Use os.path.join instead of adding strings more often 2014-07-18 12:21:52 +02:00
Daniel Martí
d3ae163152 Fix Tags UCM on newer git-svn versions 2014-07-18 12:12:16 +02:00
Hans-Christoph Steiner
8c8fb8b156 support lists/tuples in 'serverwebroot' config item
This allows the user to specify multiple servers to put the repo to, and
`fdroid server update` will automatically push to them all.

fixes #22 https://gitlab.com/fdroid/fdroidserver/issues/22
2014-07-14 15:04:30 -04:00
Hans-Christoph Steiner
f34c842f55 auto-clean newlines and spaces in repo/archive descriptions
This gives us flexibility in how the blocks of text can be formatted in
config.py, but also provides a more useful format for displaying since the
client can decide where to wrap the text.
2014-07-14 15:01:57 -04:00
Daniel Martí
3d923a6718 Use p.wait() instead of p.communicate()
The latter calls the former anyway, and we don't need to fetch the output nor
send any stdin.
2014-07-09 19:22:39 +02:00