serverwebroot has long supported uploading to multiple servers, this bit of
metadata communicates those official mirrors to the client so that it can
automatically do something useful with that information.
closes#14https://gitlab.com/fdroid/fdroidserver/issues/14
Using the same JDK throughout should prevent weird bugs where a setup might
use Java8's jarsigner and Java7's keytool. This also allows the user to
set java_paths and have jarsigner and keytool used from that specified JDK.
This incorporates almost all of the patch that is in the Debian package
that forces fdroidserver to use the default JDK on that Debian release.
closes#93https://gitlab.com/fdroid/fdroidserver/issues/93
Only keep lists in metadata files in the json format, since they don't
support multiline strings that are readable.
This makes the internal code easier, and a bit faster.
This simplifies usage, goes from
build['flag']
to
build.flag
Also makes static analyzers able to detect invalid attributes as the set
is now limited in the class definition.
As a bonus, setting of the default field values is now done in the
constructor, not separately and manually.
While at it, unify "build", "thisbuild", "info", "thisinfo", etc into
just "build".
This simplifies usage, goes from
app['Foo']
to
app.Foo
Also makes static analyzers able to detect invalid attributes as the set
is now limited in the class definition.
As a bonus, setting of the default field values is now done in the
constructor, not separately and manually.
Don't log and exit in an inner metadata function. Handle it at a higher
level and do a proper exception. This also avoids unnecessary passing of
apps all around.
This will report the version embedded in the module if it is installed, and
will report `git describe` if being run from git. If someone installs from
git using pip, this will probably report the version in setup.py, which
will be wrong. But that is not a documented install method, and I haven't
heard of anyone using it. The recommended way is to run straight from git.
For a bit repo like f-droid.org, it makes sense to standardize on a single
format for metadata files. This adds support for enforcing a single data
format, or a reduced set of data formats. So f-droid.org would run like
this if it changed to YAML:
accepted_formats = ['txt', 'yaml']
Then once everything was converted to YAML, it could look like this:
accepted_formats = ['yaml']
In order to prevent confusion caused by multiple metadata files for a given
app, fdroid will exit with an error if it finds any app metadata file with
the same package ID as one that has already been parsed.
YAML is a format that is quite similar to the .txt format, but is a
widespread standard that has editing modes in popular editors. It is also
easily parsable in python.
The .pickle for testing is a lightly edited version of the real metadata
for org.videolan.vlc:
* comments were removed
This puts all of the needed post parsing checks on the metadata into a
single method that is used by all parsing methods (.txt, JSON, XML, YAML).
This provides the single place to normalize the internal representation of
the metadata.
It would be good to also change the internal representation to use more Python
bool/int types so that less post parsing is needed for JSON, XML, and YAML.
The SMSSecure test .pickle was changed to account for the use of lstrip()
and rstrip() on all 'script' types.
This also changes the example JSON to use ints for versionCodes
While the current text metadata format is good for human readability and
editability, it is difficult to produce and parse using code. XML is a
widespread standard format for easy automatic parsing and creating, while
having decent human readability.
The .pickle for testing is a lightly edited version of the real metadata
for net.osmand.plus:
* comments were removed
* "NonFreeNet" was added as an AntiFeature
The AntiFeatures metadata is a comma-separated list of tags, like
Categories, so it should also be stored internally as a list. This makes
parsing XML and JSON easier.
The test cases' .pickle files look like they change a lot, but they really
don't, its only the change of default AntiFeatures value from None to []
The .pickle was created by dumping the output from parsing the current .txt
metadata for org.adaway. The JSON started from that pickle dump, but was
then hand edited to be more proper JSON, e.g. using boolean values.
This is a test to cover future modifications of the .txt metadata parsing.
The pickle file was generated by just dumping the current parsed metadata,
so this test will always succeed if the parsing is not changed.
This addresses the discussion in !64https://gitlab.com/fdroid/fdroidserver/merge_requests/64
Sometimes, buildToolsVersion is a kind of gradle macro call, and other
times it is a variable assignment. This regsub pattern now handles both of
those cases.
This provides the final option in this series, allowing the user to just
add --create-key to `fdroid update, and thereby upgrade an unsigned repo to
a proper signed repo. It also might be useful
closes#13https://gitlab.com/fdroid/fdroidserver/issues/13
This is a more flexible approach than testing for the complete SDK and
build-tools up front. This will only test for the commands that are
actually being run, so that if you only have `aapt` installed, you can do
`fdroid update` without errors, but other commands will still give
appropriate errors.
This also makes the build_tools item in config.py optional, it is only
needed if you want to force a specific version of the build-tools.
Just getting into the habit of adding tests to everything that I change...
Also, it should be useful to have an unsigned APK in the test collection,
since `fdroid update` should handle it gracefully and give a warning of
some kind.
The test for the help flag threw an error if there were 0 args, or if arg 1
was set to a space-separated list. The -z tests would fail if the arg was
set to a space-separated list.
This gives us flexibility in how the blocks of text can be formatted in
config.py, but also provides a more useful format for displaying since the
client can decide where to wrap the text.
This reverts b637568a62 since it added a
redundant check that broke `fdroid init` when the default version dir of
build_tools does not exist on the local system. It then uses the function
that was already in place for checking the build_tools setup in a way that
does not break `fdroid init`.
Now that the fake android home version is not matching the default version,
the tests will catch this bug in the future.
This is testing the build-tools version auto-detect in `fdroid init`, so it
should be kept as an older version. This is not meant to test the current
version of the build tools.
This means you can just do `cd tests/ && ./run-tests` to run the tests now.
You can still override the APK source with the first argument, like:
cd tests/ && ./run-tests /path/to/lots/of/apks/dir
To support a fully offline build/signing machine, there is the "local copy
dir". The repo is generated on the offline machine and then copied to a
local dir where a thumb drive or SD Card is mounted. Then on the online
machine, using `fdroid server update --sync-from-local-copy-dir` allows
the whole server update process to happen in a single command:
0. read config.py on online machine's repo
1. rsync from the local_copy_dir to the current dir
2. copy to serverwebroot, awsbucket, etc.
This allows a dir to be specified in config.py that `fdroid server update`
will automatically rsync the repo to. The idea is that the path would
point to an SD card on a fully offline machine that serves as the secure
repo signing machine.
Not everyone adds the build-tools to their PATH, so this makes it so this
script will find aapt in the most recent build-tools version that is
installed on the local system.
In this case, ANDROID_HOME is set to a fake, non-working version that will
be detected by fdroid as an Android SDK install. It should use the path
set by --android-home over the one in ANDROID_HOME, therefore if it uses
the one in ANDROID_HOME, it won't work because it is a fake one. Only
--android-home provides a working one.
This lets people easily set whatever dir they want, while letting jenkins
search through its whole workspace for any APKs that have been built. Also,
only include the latest version of a given packageName+versionCode.
Yes, this includes a binary file, but it is only for the tests, and it is
free software since I wrote it. The source is here:
https://github.com/eighthave/urzip
Previously, `fdroid update -c` would only create the new metadata, but
would not add the new apps/apks to the repo. That required a second run of
`fdroid update`. This has been fixes, so this test makes sure it stays
fixed, in a very generic way.
Make sure that fdroid can find aapt in the current config, otherwise exit
with an error. Some users don't have build_tools set, and their SDK does
not include the build-tools in the default versioned dir, so this should
warn them of what is wrong.
This allows the user to set the path to their Android SDK from the command
line. This option is named after the standard env var ANDROID_HOME, as used
in the build.xml generated by `android update project`. --android-home
takes precendence over the ANDROID_HOME env var if it is set.
This assumes that the smartcard is already setup with a signing key. init
does not generate a key on the smartcard, and skips genkey() if things are
configured to use a smartcard.
This also does not touch APK signing because that is a much more elaborate
question, since each app is signed by its own key.
