1
0
mirror of https://gitlab.com/fdroid/fdroidserver.git synced 2024-11-16 11:50:10 +01:00
Commit Graph

159 Commits

Author SHA1 Message Date
Hans-Christoph Steiner
a4e4310803 allow metadata to be embedded in source repos via .fdroid.yml
This allows a source repo to include a complete metadata file so that it
can be built directly in place using `fdroid build`.  If that app is then
included in fdroiddata, it will first load the source repo type and URL
from fdroiddata, then read .fdroid.yml if it exists, then include the rest
of the metadata as specified in fdroiddata, so that fdroiddata has
precedence over the metadata in the source code.

This lets `fdroid build` apps without having a whole fdroiddata setup, but
instead just directly in place in the source code.  This also lets devs
optionallu maintain the fdroid metadata as part of their app, rather than
in fdroiddata without loosing any control.  This should make it easier to
spread around the maintenance load.
2016-11-16 23:28:03 +01:00
Hans-Christoph Steiner
56d51fcd6b gpg-sign all valid files in the repo, including source tarballs
This makes sure there is a GPG signature on any file that is included in
the repo, including APKs, OBB, source tarballs, media files, OTA update
ZIPs, etc.  Having a GPG signature is more important on non-APK files since
they mostly do not have any signature mechanism of their own.

This also adds basic tests of adding non-APK/OBB files to a repo with
`fdroid update`.

closes #232
2016-11-07 14:53:01 +01:00
Hans-Christoph Steiner
6126b55136 rename server request from "delete" to "uninstall"
This matches the Android API's current Intent action for this, rather than
the deprecated one:

https://gitlab.com/fdroid/fdroidclient/blob/v0.101-alpha5/app/src/main/java/org/fdroid/fdroid/installer/DefaultInstallerActivity.java#L147
https://developer.android.com/reference/android/content/Intent.html#ACTION_UNINSTALL_PACKAGE
https://developer.android.com/reference/android/content/Intent.html#ACTION_DELETE
2016-08-24 23:05:34 +02:00
Hans-Christoph Steiner
85632ba00e set up install/delete lists for "push" commands from server
It is now possible for the server operator to specify lists of apps that
must be installed or deleted on the client (aka "push installs).  If
the user has opted in, or the device is already setup to respond to
these requests, then fdroidclient will automatically install/delete
the packageNames listed.  This is protected by the same signing key
as the app index metadata.

It generates single XML elements with the data set in the attributes. This
keeps the XML compact and easily extensible, e.g. for adding versionCode,
signingKey, etc as attributes:

    <install packageName="com.fsck.k9"/>
    <install packageName="at.bitfire.davdroid"/>
    <delete packageName="com.facebook.orca"/>

Copyright: 2016 Blue Jay Wireless
Signed-off-by: Hans-Christoph Steiner <hans@eds.org>

closes #177
2016-08-16 21:02:15 +02:00
Hans-Christoph Steiner
1be263e870 FDroidPopen must have a locale to support UTF-8 filenames
`fdroid update` should be able to handle any valid filename (hopefully
aapt doesn't barf on them).  To handle that, the environment where the
shell commands are run in needs to have a UTF-8 locale set. If LANG is
not set, things default to ASCII and UTF-8 filenames fail.

This also renames test APK with lots of Unicode chars as a test case.

closes #167
2016-06-27 20:30:32 +02:00
Hans-Christoph Steiner
9cd6b444f8 standardize on .yml as the file extension for YAML
Though the YAML people recommend .yaml for the file extension, in Android
land it seems clear that .yml has won out:

* .travis.yml
* .gitlab-ci.yml
* .circle.yml
* Ansible main.yml
2016-03-23 17:16:28 +01:00
Daniel Martí
24ad0418e3 tests: switch to python3 2016-03-11 13:27:00 +00:00
Hans-Christoph Steiner
3967a0068b added 'mirrors' option to config for giving official mirror URLs
serverwebroot has long supported uploading to multiple servers, this bit of
metadata communicates those official mirrors to the client so that it can
automatically do something useful with that information.

closes #14 https://gitlab.com/fdroid/fdroidserver/issues/14
2016-02-17 10:40:07 +01:00
Hans-Christoph Steiner
a7a053c4f5 use utf-8 as default encoding for config.py
This makes it easy to add all languages to the description.
2016-02-17 00:26:16 +01:00
Hans-Christoph Steiner
2983f3505d support BSD and GNU sed in tests 2016-02-15 23:06:31 +01:00
Hans-Christoph Steiner
c91b715a15 run-tests: use OSX/BSD compatible mktemp -d
-d requires an arg on BSD
2016-02-15 23:06:31 +01:00
Hans-Christoph Steiner
ff30bded21 only test ./gendocs on GNU/Linux
This helps with getting the tests running on OSX and other platforms.
2016-02-15 23:06:31 +01:00
Hans-Christoph Steiner
751fd3fb0a common: do not crash if the java_paths are not what is expected
Many commands work without the JDK installed, and it is also possible that
someone is using only JDK 8 or 9.
2016-02-12 08:33:21 +01:00
Hans-Christoph Steiner
25caa09420 run-tests: verbose output for fdroid update to aid debugging
I'm stumped by this stacktrace, hopefully debug logging will help:

Unknown exception found!
Traceback (most recent call last):
  File "./fdroidserver-0.4.0/fdroid", line 141, in <module>
    main()
  File "./fdroidserver-0.4.0/fdroid", line 119, in main
    mod.main()
  File "<http://localhost:18080/job/fdroidserver/ws/.testfiles/tmp.9hr9NiSmZs/fdroidserver-0.4.0/fdroidserver/update.py",> line 1146, in main
    apks, cachechanged = scan_apks(apps, apkcache, repodirs[0], knownapks)
  File "<http://localhost:18080/job/fdroidserver/ws/.testfiles/tmp.9hr9NiSmZs/fdroidserver-0.4.0/fdroidserver/update.py",> line 543, in scan_apks
    thisinfo['sig'] = getsig(os.path.join(os.getcwd(), apkfile))
  File "<http://localhost:18080/job/fdroidserver/ws/.testfiles/tmp.9hr9NiSmZs/fdroidserver-0.4.0/fdroidserver/update.py",> line 361, in getsig
    with zipfile.ZipFile(apkpath, 'r') as apk:
  File "/usr/lib/python2.7/zipfile.py", line 770, in __init__
    self._RealGetContents()
  File "/usr/lib/python2.7/zipfile.py", line 857, in _RealGetContents
    x._decodeExtra()
  File "/usr/lib/python2.7/zipfile.py", line 388, in _decodeExtra
    tp, ln = unpack('<HH', extra[:4])
struct.error: unpack requires a string argument of length 4
2016-01-21 12:26:50 +01:00
Daniel Martí
20470556f7 run-tests: Don't set -x until option parsing is done 2015-11-01 14:09:11 +01:00
Daniel Martí
c115f71640 run-tests: Don't show echo_header commands 2015-11-01 14:09:06 +01:00
Daniel Martí
27c717e764 tests: allow running ./tests/run-tests 2015-11-01 13:48:25 +01:00
Hans-Christoph Steiner
3fc2a99d71 fdroid --version for installed releases and running from git
This will report the version embedded in the module if it is installed, and
will report `git describe` if being run from git.  If someone installs from
git using pip, this will probably report the version in setup.py, which
will be wrong.  But that is not a documented install method, and I haven't
heard of anyone using it.  The recommended way is to run straight from git.
2015-09-01 19:45:00 +02:00
Hans-Christoph Steiner
8d6e0aebb9 exit with error if duplicate metadata file is found
In order to prevent confusion caused by multiple metadata files for a given
app, fdroid will exit with an error if it finds any app metadata file with
the same package ID as one that has already been parsed.
2015-09-01 11:39:51 +02:00
Hans-Christoph Steiner
6db6433e97 fix syntax error in manual generation and add ./gendocs.sh to the tests 2015-08-19 17:51:29 +02:00
Hans-Christoph Steiner
1bb14d2f82 run-tests: fix quoting to prevent spurious execution of fdroid update 2015-08-05 13:36:21 +02:00
Alex Burka
4a478528c2 tests: short args for mktemp to support BSD
*BSD and OSX do not have compatible long args
2015-08-01 00:38:40 +02:00
Hans-Christoph Steiner
b8a3bce430 run-tests: fix copying of urzip.apk into new repo 2015-08-01 00:38:40 +02:00
Hans-Christoph Steiner
d01e814e36 run-tests: run fdroid readmeta after each --create-metadata
Just another basic check, this time for `fdroid readmeta`.
2015-07-30 22:19:10 +02:00
Hans-Christoph Steiner
ff48c6b7f9 tests: no need to dump the index.xml to the build log
This should make the build log a lot shorter.
2015-05-08 16:13:15 -04:00
Hans-Christoph Steiner
de1c80f9b4 fdroid update --create-key to create a keystore/key
This provides the final option in this series, allowing the user to just
add --create-key to `fdroid update, and thereby upgrade an unsigned repo to
a proper signed repo.  It also might be useful

closes #13 https://gitlab.com/fdroid/fdroidserver/issues/13
2015-05-08 16:13:15 -04:00
Hans-Christoph Steiner
86865faa62 make fdroid update check that it can sign the repo, or exit with error
There is no good reason to run unsigned repos any more.  It is trivially
easy to create and use a signed repo, and having to support unsigned repos
in the client makes some security-critical parts of the code a lot more
complicated.

refs #13 https://gitlab.com/fdroid/fdroidserver/issues/13
https://gitlab.com/fdroid/fdroidclient/issues/12
2015-05-08 16:13:15 -04:00
Hans-Christoph Steiner
fa1cc48d57 run all SDK tools commands using SdkToolsPopen 2014-12-14 13:25:20 +01:00
Hans-Christoph Steiner
9244256461 find cmds from SDK build-tools in a more flexible way, on the fly
This is a more flexible approach than testing for the complete SDK and
build-tools up front.  This will only test for the commands that are
actually being run, so that if you only have `aapt` installed, you can do
`fdroid update` without errors, but other commands will still give
appropriate errors.

This also makes the build_tools item in config.py optional, it is only
needed if you want to force a specific version of the build-tools.
2014-12-14 13:25:20 +01:00
Hans-Christoph Steiner
298a88a498 added test case for common.isApkDebuggable()
Just getting into the habit of adding tests to everything that I change...
Also, it should be useful to have an unsigned APK in the test collection,
since `fdroid update` should handle it gracefully and give a warning of
some kind.
2014-12-14 13:25:20 +01:00
Hans-Christoph Steiner
14cd835929 run-tests: properly handle zero and multiple args
The test for the help flag threw an error if there were 0 args, or if arg 1
was set to a space-separated list.  The -z tests would fail if the arg was
set to a space-separated list.
2014-11-17 17:40:48 +01:00
Hans-Christoph Steiner
49b343d024 run-tests: make sure not to include the bad APKs in tests/
These APKs are meant for specific tests, not the tests that want just a
collection of good APKs.
2014-11-17 17:24:02 +01:00
Hans-Christoph Steiner
18db382c47 include test cases for python getsig replacement
This includes the old getsig.java since that is the canonical implementation
of that algorithm.

fixes #5 https://gitlab.com/fdroid/fdroidserver/issues/5
2014-11-07 09:20:33 +00:00
Daniel Martí
6e2d0a9e1e Replace getsig.java with a pure python implementation
Special thanks to deki for helping out with the certificate encodings:
https://gitlab.com/snippets/1842

fixes #5 https://gitlab.com/fdroid/fdroidserver/issues/5
2014-11-07 09:20:14 +00:00
Daniel Martí
c99f15ac86 Remove .egg-info dir when tests succeed 2014-09-17 08:54:02 +02:00
Daniel Martí
8b469a79f8 Fix the test suite 2014-07-03 22:33:40 +02:00
Hans-Christoph Steiner
a58a9bff18 run pre-commit hook as part of test suite 2014-06-30 23:01:48 -04:00
Hans-Christoph Steiner
4d913d646a replace redundant build_tools check that breaks fdroid init
This reverts b637568a62 since it added a
redundant check that broke `fdroid init` when the default version dir of
build_tools does not exist on the local system.  It then uses the function
that was already in place for checking the build_tools setup in a way that
does not break `fdroid init`.

Now that the fake android home version is not matching the default version,
the tests will catch this bug in the future.
2014-06-30 23:01:47 -04:00
Hans-Christoph Steiner
7a3992aa1a use 'python2' everywhere since fdroidserver has not been tested with 3.x 2014-06-30 23:01:47 -04:00
Hans-Christoph Steiner
40d4e30010 tests: create_fake_android_home should create old build-tools version
This is testing the build-tools version auto-detect in `fdroid init`, so it
should be kept as an older version.  This is not meant to test the current
version of the build tools.
2014-06-30 21:19:55 -04:00
Daniel Martí
cbdcdd1e78 Update build-tools to 20.0.0 2014-06-30 17:03:16 +02:00
Hans-Christoph Steiner
25a94dced2 tests: create a source tarball and use that to build a repo
This tests that setup.py is in working order and creating a functional
source tarball.
2014-06-27 20:31:28 -04:00
Hans-Christoph Steiner
0e47d62024 tests: by default, run on included urzip.apk
This means you can just do `cd tests/ && ./run-tests` to run the tests now.
You can still override the APK source with the first argument, like:

cd tests/ && ./run-tests /path/to/lots/of/apks/dir
2014-06-27 20:31:28 -04:00
Hans-Christoph Steiner
25f6b0c246 server: --sync-from-local-copy-dir for updating from offline signing repo
To support a fully offline build/signing machine, there is the "local copy
dir".  The repo is generated on the offline machine and then copied to a
local dir where a thumb drive or SD Card is mounted.  Then on the online
machine, using `fdroid server update --sync-from-local-copy-dir` allows
the whole server update process to happen in a single command:

0. read config.py on online machine's repo
1. rsync from the local_copy_dir to the current dir
2. copy to serverwebroot, awsbucket, etc.
2014-06-26 20:47:36 -04:00
Hans-Christoph Steiner
d73b43c5fc server: 'local_copy_dir' config/options to automate offline repo signing
This allows a dir to be specified in config.py that `fdroid server update`
will automatically rsync the repo to.  The idea is that the path would
point to an SD card on a fully offline machine that serves as the secure
repo signing machine.
2014-06-26 20:47:36 -04:00
Hans-Christoph Steiner
35ea01cbd6 run-tests: find current version of aapt in folder rather than in PATH
Not everyone adds the build-tools to their PATH, so this makes it so this
script will find aapt in the most recent build-tools version that is
installed on the local system.
2014-06-16 19:18:28 -04:00
Daniel Martí
1fcf81b1f4 Current build-tools is 19.1.0, not 19.1 2014-06-16 23:12:47 +02:00
Daniel Martí
1ca4bda4ef Fix create_fake_android_home with the new build-tools 2014-06-16 13:09:28 +02:00
Hans-Christoph Steiner
97334286ea tests: turn off unneeded debug logging during APK copying
The whole process of finding and copying APKs can be very verbose, so turn
of the bash verbose logging during that process.
2014-06-05 16:55:52 -04:00
Hans-Christoph Steiner
0272124248 tests: fix --android-home test to actually work
In this case, ANDROID_HOME is set to a fake, non-working version that will
be detected by fdroid as an Android SDK install.  It should use the path
set by --android-home over the one in ANDROID_HOME, therefore if it uses
the one in ANDROID_HOME, it won't work because it is a fake one.  Only
--android-home provides a working one.
2014-06-04 22:01:25 -04:00
Hans-Christoph Steiner
9b313e76bb tests: specify dir for APKs as cmd line arg, and set jenkins to ~
This lets people easily set whatever dir they want, while letting jenkins
search through its whole workspace for any APKs that have been built. Also,
only include the latest version of a given packageName+versionCode.
2014-06-04 22:01:25 -04:00
Hans-Christoph Steiner
78d5ebf26c use 'metadata' name throughout: --createmeta to --create-metadata
To keep the naming used within fdroid consistent, I renamed this long flag
to use the whole word 'metadata' since that is what is used everywhere else
2014-06-04 21:54:56 -04:00
Hans-Christoph Steiner
0e2f4d54d0 tests: try fdroid update after adding an APK to an existing repo
Yes, this includes a binary file, but it is only for the tests, and it is
free software since I wrote it.  The source is here:

https://github.com/eighthave/urzip
2014-06-04 21:54:55 -04:00
Hans-Christoph Steiner
ed7a0576c2 test: make sure fdroid update -c creates a full index.xml
Previously, `fdroid update -c` would only create the new metadata, but
would not add the new apps/apks to the repo.  That required a second run of
`fdroid update`.  This has been fixes, so this test makes sure it stays
fixed, in a very generic way.
2014-06-04 21:54:55 -04:00
Hans-Christoph Steiner
6ca060e10d add more SDK checks: build-tools/19.0.3 and presense of aapt
Make sure that fdroid can find aapt in the current config, otherwise exit
with an error.  Some users don't have build_tools set, and their SDK does
not include the build-tools in the default versioned dir, so this should
warn them of what is wrong.
2014-06-04 21:54:55 -04:00
Hans-Christoph Steiner
ef7c9d89d2 init: --no-prompt to skip sdk_path prompt
For running the tests and in other scripted setups, the user prompt is an
annoying. Using --no-prompt means the script can test for failure.
2014-04-23 20:10:02 -04:00
Hans-Christoph Steiner
66df02d5f8 init: --android-home for forcing the path to the Android SDK
This allows the user to set the path to their Android SDK from the command
line. This option is named after the standard env var ANDROID_HOME, as used
in the build.xml generated by `android update project`.  --android-home
takes precendence over the ANDROID_HOME env var if it is set.
2014-04-23 19:33:10 -04:00
Hans-Christoph Steiner
49387b7108 use FDroid-standard naming when copying APKs for tests
This makes sure that there are no APKs with duplicate version codes.
2014-04-10 12:37:07 -04:00
Hans-Christoph Steiner
234736f306 move run-tests to /bin/bash so we can use bashisms 2014-04-10 12:31:23 -04:00