1
0
mirror of https://gitlab.com/fdroid/fdroidserver.git synced 2024-11-14 11:00:10 +01:00
Commit Graph

274 Commits

Author SHA1 Message Date
Daniel Martí
bf8518ee8f Rework build into a class
This simplifies usage, goes from

    build['flag']
to
    build.flag

Also makes static analyzers able to detect invalid attributes as the set
is now limited in the class definition.

As a bonus, setting of the default field values is now done in the
constructor, not separately and manually.

While at it, unify "build", "thisbuild", "info", "thisinfo", etc into
just "build".
2015-12-02 20:03:11 +01:00
Daniel Martí
ab614ab442 Rework app into a class
This simplifies usage, goes from

	app['Foo']
to
	app.Foo

Also makes static analyzers able to detect invalid attributes as the set
is now limited in the class definition.

As a bonus, setting of the default field values is now done in the
constructor, not separately and manually.
2015-11-28 17:11:05 +01:00
Daniel Martí
de12cfdbe1 Handle duplicate apps in a much cleaner way
Don't log and exit in an inner metadata function. Handle it at a higher
level and do a proper exception. This also avoids unnecessary passing of
apps all around.
2015-11-28 14:04:21 +01:00
Daniel Martí
20470556f7 run-tests: Don't set -x until option parsing is done 2015-11-01 14:09:11 +01:00
Daniel Martí
c115f71640 run-tests: Don't show echo_header commands 2015-11-01 14:09:06 +01:00
Daniel Martí
27c717e764 tests: allow running ./tests/run-tests 2015-11-01 13:48:25 +01:00
Daniel Martí
16ca7f3be3 tests: we now replace "buildToolsVersion var" too
See d706cfa7d8
2015-10-24 18:47:43 +02:00
Daniel Martí
c877d4524a tests: Comment values are lists, not strings
Since there can be multiple lines.
2015-10-08 16:39:47 +02:00
Daniel Martí
5d180b8858 tests: adapt to new comments dict 2015-10-08 13:20:48 +02:00
Daniel Martí
00bd75aa8c tests: close files 2015-10-08 13:20:35 +02:00
Daniel Martí
e285a8ac31 Remove Dogecoin donation support
Not nearly as popular as bitcoin and litecoin. To the point where it hadn't
been used yet.

Bump index version to 14 to mark its removal.
2015-09-18 19:06:10 -07:00
Hans-Christoph Steiner
707930ce0a add a basic test of fdroid import 2015-09-10 11:08:40 +02:00
Hans-Christoph Steiner
e6c0be8898 move tests into common script for jenkins and gitlab-ci 2015-09-01 19:52:37 +02:00
Hans-Christoph Steiner
3fc2a99d71 fdroid --version for installed releases and running from git
This will report the version embedded in the module if it is installed, and
will report `git describe` if being run from git.  If someone installs from
git using pip, this will probably report the version in setup.py, which
will be wrong.  But that is not a documented install method, and I haven't
heard of anyone using it.  The recommended way is to run straight from git.
2015-09-01 19:45:00 +02:00
Hans-Christoph Steiner
325db90918 set "accepted formats" for metadata in config.py
For a bit repo like f-droid.org, it makes sense to standardize on a single
format for metadata files.  This adds support for enforcing a single data
format, or a reduced set of data formats.  So f-droid.org would run like
this if it changed to YAML:

  accepted_formats = ['txt', 'yaml']

Then once everything was converted to YAML, it could look like this:

  accepted_formats = ['yaml']
2015-09-01 11:53:37 +02:00
Hans-Christoph Steiner
beb217b6ae add 'gradleprops' to .pickle files for the metadata tests 2015-09-01 11:53:37 +02:00
Hans-Christoph Steiner
8d6e0aebb9 exit with error if duplicate metadata file is found
In order to prevent confusion caused by multiple metadata files for a given
app, fdroid will exit with an error if it finds any app metadata file with
the same package ID as one that has already been parsed.
2015-09-01 11:39:51 +02:00
Hans-Christoph Steiner
32e257d742 support app metadata in YAML format
YAML is a format that is quite similar to the .txt format, but is a
widespread standard that has editing modes in popular editors.  It is also
easily parsable in python.

The .pickle for testing is a lightly edited version of the real metadata
for org.videolan.vlc:

 * comments were removed
2015-09-01 11:39:51 +02:00
Hans-Christoph Steiner
0425788b7d consolidate all post-parse metadata checks
This puts all of the needed post parsing checks on the metadata into a
single method that is used by all parsing methods (.txt, JSON, XML, YAML).
This provides the single place to normalize the internal representation of
the metadata.

It would be good to also change the internal representation to use more Python
bool/int types so that less post parsing is needed for JSON, XML, and YAML.

The SMSSecure test .pickle was changed to account for the use of lstrip()
and rstrip() on all 'script' types.

This also changes the example JSON to use ints for versionCodes
2015-09-01 11:39:51 +02:00
Hans-Christoph Steiner
ab145de6bc support app metadata in XML format
While the current text metadata format is good for human readability and
editability, it is difficult to produce and parse using code.  XML is a
widespread standard format for easy automatic parsing and creating, while
having decent human readability.

The .pickle for testing is a lightly edited version of the real metadata
for net.osmand.plus:

* comments were removed
* "NonFreeNet" was added as an AntiFeature
2015-09-01 11:39:51 +02:00
Hans-Christoph Steiner
2831b3e93f convert internal representation of AntiFeatures to list
The AntiFeatures metadata is a comma-separated list of tags, like
Categories, so it should also be stored internally as a list.  This makes
parsing XML and JSON easier.

The test cases' .pickle files look like they change a lot, but they really
don't, its only the change of default AntiFeatures value from None to []
2015-09-01 11:39:51 +02:00
Hans-Christoph Steiner
dabb7e42b7 add test case for JSON parsing, based on parsed .txt metadata
The .pickle was created by dumping the output from parsing the current .txt
metadata for org.adaway.  The JSON started from that pickle dump, but was
then hand edited to be more proper JSON, e.g. using boolean values.
2015-09-01 11:39:51 +02:00
Hans-Christoph Steiner
1bbae4cd9e add test for metadata parsing based on a parse output to a pickle
This is a test to cover future modifications of the .txt metadata parsing.
The pickle file was generated by just dumping the current parsed metadata,
so this test will always succeed if the parsing is not changed.
2015-09-01 11:39:50 +02:00
Hans-Christoph Steiner
9e5dd19fc8 add test files to pre-commit hook and fix pep8 errors 2015-09-01 11:39:50 +02:00
Hans-Christoph Steiner
d53a5af715 build: improve regsub pattern for setting buildToolsVersion
This addresses the discussion in !64
https://gitlab.com/fdroid/fdroidserver/merge_requests/64

Sometimes, buildToolsVersion is a kind of gradle macro call, and other
times it is a variable assignment.  This regsub pattern now handles both of
those cases.
2015-08-19 17:55:17 +02:00
Hans-Christoph Steiner
6db6433e97 fix syntax error in manual generation and add ./gendocs.sh to the tests 2015-08-19 17:51:29 +02:00
Daniel Martí
90a926dfb0 Fix test that broke with c813a3cdbb 2015-08-13 12:44:35 -07:00
Hans-Christoph Steiner
1bb14d2f82 run-tests: fix quoting to prevent spurious execution of fdroid update 2015-08-05 13:36:21 +02:00
Hans-Christoph Steiner
64a9c93ce7 test new common.regsub_file() method that replaces sed calls 2015-08-01 00:38:41 +02:00
Alex Burka
4a478528c2 tests: short args for mktemp to support BSD
*BSD and OSX do not have compatible long args
2015-08-01 00:38:40 +02:00
Hans-Christoph Steiner
b8a3bce430 run-tests: fix copying of urzip.apk into new repo 2015-08-01 00:38:40 +02:00
Hans-Christoph Steiner
d01e814e36 run-tests: run fdroid readmeta after each --create-metadata
Just another basic check, this time for `fdroid readmeta`.
2015-07-30 22:19:10 +02:00
Hans-Christoph Steiner
ff48c6b7f9 tests: no need to dump the index.xml to the build log
This should make the build log a lot shorter.
2015-05-08 16:13:15 -04:00
Hans-Christoph Steiner
de1c80f9b4 fdroid update --create-key to create a keystore/key
This provides the final option in this series, allowing the user to just
add --create-key to `fdroid update, and thereby upgrade an unsigned repo to
a proper signed repo.  It also might be useful

closes #13 https://gitlab.com/fdroid/fdroidserver/issues/13
2015-05-08 16:13:15 -04:00
Hans-Christoph Steiner
86865faa62 make fdroid update check that it can sign the repo, or exit with error
There is no good reason to run unsigned repos any more.  It is trivially
easy to create and use a signed repo, and having to support unsigned repos
in the client makes some security-critical parts of the code a lot more
complicated.

refs #13 https://gitlab.com/fdroid/fdroidserver/issues/13
https://gitlab.com/fdroid/fdroidclient/issues/12
2015-05-08 16:13:15 -04:00
Ciaran Gultnieks
384f97e998 Ensure package names are valid 2015-01-26 18:30:01 +00:00
Hans-Christoph Steiner
2a87ae5127 fix update.TestCase: remove unused SilentPopen import, that class was removed 2015-01-21 10:21:54 +01:00
Hans-Christoph Steiner
3b3d026ee3 fix install test by making sure common.options is set 2015-01-21 10:21:54 +01:00
Hans-Christoph Steiner
fa1cc48d57 run all SDK tools commands using SdkToolsPopen 2014-12-14 13:25:20 +01:00
Hans-Christoph Steiner
9244256461 find cmds from SDK build-tools in a more flexible way, on the fly
This is a more flexible approach than testing for the complete SDK and
build-tools up front.  This will only test for the commands that are
actually being run, so that if you only have `aapt` installed, you can do
`fdroid update` without errors, but other commands will still give
appropriate errors.

This also makes the build_tools item in config.py optional, it is only
needed if you want to force a specific version of the build-tools.
2014-12-14 13:25:20 +01:00
Hans-Christoph Steiner
298a88a498 added test case for common.isApkDebuggable()
Just getting into the habit of adding tests to everything that I change...
Also, it should be useful to have an unsigned APK in the test collection,
since `fdroid update` should handle it gracefully and give a warning of
some kind.
2014-12-14 13:25:20 +01:00
Hans-Christoph Steiner
14cd835929 run-tests: properly handle zero and multiple args
The test for the help flag threw an error if there were 0 args, or if arg 1
was set to a space-separated list.  The -z tests would fail if the arg was
set to a space-separated list.
2014-11-17 17:40:48 +01:00
Hans-Christoph Steiner
49b343d024 run-tests: make sure not to include the bad APKs in tests/
These APKs are meant for specific tests, not the tests that want just a
collection of good APKs.
2014-11-17 17:24:02 +01:00
Hans-Christoph Steiner
18db382c47 include test cases for python getsig replacement
This includes the old getsig.java since that is the canonical implementation
of that algorithm.

fixes #5 https://gitlab.com/fdroid/fdroidserver/issues/5
2014-11-07 09:20:33 +00:00
Daniel Martí
6e2d0a9e1e Replace getsig.java with a pure python implementation
Special thanks to deki for helping out with the certificate encodings:
https://gitlab.com/snippets/1842

fixes #5 https://gitlab.com/fdroid/fdroidserver/issues/5
2014-11-07 09:20:14 +00:00
Daniel Martí
c99f15ac86 Remove .egg-info dir when tests succeed 2014-09-17 08:54:02 +02:00
Hans-Christoph Steiner
f34c842f55 auto-clean newlines and spaces in repo/archive descriptions
This gives us flexibility in how the blocks of text can be formatted in
config.py, but also provides a more useful format for displaying since the
client can decide where to wrap the text.
2014-07-14 15:01:57 -04:00
Daniel Martí
8b469a79f8 Fix the test suite 2014-07-03 22:33:40 +02:00
Hans-Christoph Steiner
a58a9bff18 run pre-commit hook as part of test suite 2014-06-30 23:01:48 -04:00
Hans-Christoph Steiner
4d913d646a replace redundant build_tools check that breaks fdroid init
This reverts b637568a62 since it added a
redundant check that broke `fdroid init` when the default version dir of
build_tools does not exist on the local system.  It then uses the function
that was already in place for checking the build_tools setup in a way that
does not break `fdroid init`.

Now that the fake android home version is not matching the default version,
the tests will catch this bug in the future.
2014-06-30 23:01:47 -04:00
Hans-Christoph Steiner
7a3992aa1a use 'python2' everywhere since fdroidserver has not been tested with 3.x 2014-06-30 23:01:47 -04:00
Hans-Christoph Steiner
40d4e30010 tests: create_fake_android_home should create old build-tools version
This is testing the build-tools version auto-detect in `fdroid init`, so it
should be kept as an older version.  This is not meant to test the current
version of the build tools.
2014-06-30 21:19:55 -04:00
Daniel Martí
cbdcdd1e78 Update build-tools to 20.0.0 2014-06-30 17:03:16 +02:00
Hans-Christoph Steiner
25a94dced2 tests: create a source tarball and use that to build a repo
This tests that setup.py is in working order and creating a functional
source tarball.
2014-06-27 20:31:28 -04:00
Hans-Christoph Steiner
0e47d62024 tests: by default, run on included urzip.apk
This means you can just do `cd tests/ && ./run-tests` to run the tests now.
You can still override the APK source with the first argument, like:

cd tests/ && ./run-tests /path/to/lots/of/apks/dir
2014-06-27 20:31:28 -04:00
Hans-Christoph Steiner
25f6b0c246 server: --sync-from-local-copy-dir for updating from offline signing repo
To support a fully offline build/signing machine, there is the "local copy
dir".  The repo is generated on the offline machine and then copied to a
local dir where a thumb drive or SD Card is mounted.  Then on the online
machine, using `fdroid server update --sync-from-local-copy-dir` allows
the whole server update process to happen in a single command:

0. read config.py on online machine's repo
1. rsync from the local_copy_dir to the current dir
2. copy to serverwebroot, awsbucket, etc.
2014-06-26 20:47:36 -04:00
Hans-Christoph Steiner
d73b43c5fc server: 'local_copy_dir' config/options to automate offline repo signing
This allows a dir to be specified in config.py that `fdroid server update`
will automatically rsync the repo to.  The idea is that the path would
point to an SD card on a fully offline machine that serves as the secure
repo signing machine.
2014-06-26 20:47:36 -04:00
Hans-Christoph Steiner
35ea01cbd6 run-tests: find current version of aapt in folder rather than in PATH
Not everyone adds the build-tools to their PATH, so this makes it so this
script will find aapt in the most recent build-tools version that is
installed on the local system.
2014-06-16 19:18:28 -04:00
Daniel Martí
1fcf81b1f4 Current build-tools is 19.1.0, not 19.1 2014-06-16 23:12:47 +02:00
Daniel Martí
1ca4bda4ef Fix create_fake_android_home with the new build-tools 2014-06-16 13:09:28 +02:00
Hans-Christoph Steiner
97334286ea tests: turn off unneeded debug logging during APK copying
The whole process of finding and copying APKs can be very verbose, so turn
of the bash verbose logging during that process.
2014-06-05 16:55:52 -04:00
Hans-Christoph Steiner
0272124248 tests: fix --android-home test to actually work
In this case, ANDROID_HOME is set to a fake, non-working version that will
be detected by fdroid as an Android SDK install.  It should use the path
set by --android-home over the one in ANDROID_HOME, therefore if it uses
the one in ANDROID_HOME, it won't work because it is a fake one.  Only
--android-home provides a working one.
2014-06-04 22:01:25 -04:00
Hans-Christoph Steiner
9b313e76bb tests: specify dir for APKs as cmd line arg, and set jenkins to ~
This lets people easily set whatever dir they want, while letting jenkins
search through its whole workspace for any APKs that have been built. Also,
only include the latest version of a given packageName+versionCode.
2014-06-04 22:01:25 -04:00
Hans-Christoph Steiner
78d5ebf26c use 'metadata' name throughout: --createmeta to --create-metadata
To keep the naming used within fdroid consistent, I renamed this long flag
to use the whole word 'metadata' since that is what is used everywhere else
2014-06-04 21:54:56 -04:00
Hans-Christoph Steiner
0e2f4d54d0 tests: try fdroid update after adding an APK to an existing repo
Yes, this includes a binary file, but it is only for the tests, and it is
free software since I wrote it.  The source is here:

https://github.com/eighthave/urzip
2014-06-04 21:54:55 -04:00
Hans-Christoph Steiner
ed7a0576c2 test: make sure fdroid update -c creates a full index.xml
Previously, `fdroid update -c` would only create the new metadata, but
would not add the new apps/apks to the repo.  That required a second run of
`fdroid update`.  This has been fixes, so this test makes sure it stays
fixed, in a very generic way.
2014-06-04 21:54:55 -04:00
Hans-Christoph Steiner
6ca060e10d add more SDK checks: build-tools/19.0.3 and presense of aapt
Make sure that fdroid can find aapt in the current config, otherwise exit
with an error.  Some users don't have build_tools set, and their SDK does
not include the build-tools in the default versioned dir, so this should
warn them of what is wrong.
2014-06-04 21:54:55 -04:00
Hans-Christoph Steiner
ef7c9d89d2 init: --no-prompt to skip sdk_path prompt
For running the tests and in other scripted setups, the user prompt is an
annoying. Using --no-prompt means the script can test for failure.
2014-04-23 20:10:02 -04:00
Hans-Christoph Steiner
66df02d5f8 init: --android-home for forcing the path to the Android SDK
This allows the user to set the path to their Android SDK from the command
line. This option is named after the standard env var ANDROID_HOME, as used
in the build.xml generated by `android update project`.  --android-home
takes precendence over the ANDROID_HOME env var if it is set.
2014-04-23 19:33:10 -04:00
Hans-Christoph Steiner
49387b7108 use FDroid-standard naming when copying APKs for tests
This makes sure that there are no APKs with duplicate version codes.
2014-04-10 12:37:07 -04:00
Hans-Christoph Steiner
234736f306 move run-tests to /bin/bash so we can use bashisms 2014-04-10 12:31:23 -04:00
Hans-Christoph Steiner
3829d37d34 support repo signing with a key on a smartcard
This assumes that the smartcard is already setup with a signing key.  init
does not generate a key on the smartcard, and skips genkey() if things are
configured to use a smartcard.

This also does not touch APK signing because that is a much more elaborate
question, since each app is signed by its own key.
2014-04-07 16:00:18 -04:00
Hans-Christoph Steiner
faf0c4381f add test case were init generates a keystore and uses it 2014-04-07 16:00:18 -04:00
Hans-Christoph Steiner
bfa21fb630 add script to do a test run of creating a new repo
This tests/ folder can then be used for all sorts of tests, including
standard python tests.
2014-04-02 17:49:07 -04:00