The bug fixed in !1370 was due to a reuse of the per app anti features
dictionary in the per version one and adding the per version anti
features into it. This was not caught by the test suite as the app
versions are processed from newest to oldest and the test data had only
anti features in the oldest version. This patch adds an anti feature to
a newest version so only those in between should not have it.
I tried to get this to indent the .yaml files properly so yamllint defaults
work with tests/metadata/dump/*.yaml, but it didn't take for some reason:
yaml.indent(mapping=4, sequence=4, offset=2)
It turns out that the maven: field was originally declared as a TYPE_STRING,
given that it was not given a different type in metadata.py's flagtypes.
The code was confused because it was given a default value of `False` rather
than `None` as the rest of the TYPE_STRING fields have.
This construct in build.py means maven: should always be a string:
if '@' in build.maven:
maven_dir = os.path.join(root_dir, build.maven.split('@', 1)[1])
else:
maven_dir = root_dir
make_v0() modified the apps data structure to copy an anti feature for a
specific version to all versions resulting in index-v1 and -v2 to
contain wrong anti feature annotations. This patch fixes this and adds a
test that the data structure is no longer modified.
The bug shadowed bugs in the AF implementation of -v1 and -v2 resulting
in not coping the version specific data. This is corrected as well.
This is also tested now.
For -v2 the AF dict is now sorted to make the result reproducible.
Finally The NoSourceSince AF was added as a per version and overall AF
in -v1 and is now only applied as an overall AF and the test is updated
accordingly.
Make sudo, init prebuild, build and Prepare fields lists and only
concatenate them with '; ' before execution. This allows arbitrary
commands inside the fileds (even && and ';') as we don't need to split
the commands again for rewritemeta.
The .txt format was the last place where the lowercase "builds" was used,
this converts references everywhere to be "Builds". This makes it possible
to load metadata YAML files with any YAML parser, then have it possible to
use fdroidserver methods on that data, like metadata.write_metadata().
The test files in tests/metadata/dump/*.yaml were manually edited by cutting
the builds: block and putting it the sort order for Builds: so the contents
should be unchanged.
```
sed -i \
-e 's/app\.builds/app.get('Builds', \[\])/g' \
-e "s/app\.get(Builds, \[\]) =/app\['Builds'] =/g" \
-e "s/app\.get(Builds, \[\]) =/app\['Builds'] =/g" \
-e "s/app\.get(Builds, \[\])/app.get('Builds', \[\])/g" \
-e "s/app\.get('Builds', \[\])\.append/app\['Builds'\].append/g" \
-e "s/app\['builds'\]/app.get('Builds', [])/g" \
*/*.*
```
Liberapay was originally included using a numeric ID, since they had
not yet finalized the public URLs. Now it is a username. So this
logic prefers the username in Liberapay: field, and keeps the old
LiberapayID: to ease migration. LiberapayID: will not override
Liberapay:. Clients are expected to prefer Liberapay: over LiberapayID:
This code has never been used and contains some insecure uses of shell=True
Building Kivy apps should be done with the buildozer=yes method. The
buildozer method should probably be moved to a provisioner once that is in
place.
For cases like the OpenVPN vuln that was recently announced, it is useful
for fdroiddata maintainers to be able to mark builds that have known
vulnerabilities.
Fastlane Supply, Triple-T Gradle Play Publisher, and many app stores
include the possibility to specify a website for the author, as distinct
from the website for the app.
closes#204
This lets index-v1 be parsed directly into class instances because the
field/instance var names match exactly. The original index v0 element
must retain the 'lastupdated' name for backwards compatibility.
In the future, we should have better internal datatypes for this stuff,
i.e. instead of gradle: ['yes'] for True, actually use a boolean. For now,
make the YAML and JSON metadata produce the same internal data as .txt.
Like with the App class in the commit before, this makes it a lot
easier to work with this data when converting between the internal
formats and external formats like YAML, JSON, MsgPack, protobuf, etc.
The one unfortunate thing here is Build.update. It becomes
dict.update(), which is a method not an attribute.
build.get('update') or build['update'] could be used, but that would
be oddly inconsistent. So instead the field is renamed to
'androidupdate', except for in the .txt v0 metadata files. This better
describes what field does anyway, since it runs `android update`.
Build.update is only referenced in two places right next to each other
for the ant builds, so this change still seems worthwhile.
Python is heavily based on its core data types, and dict is one of the more
important ones. Even classes are basically a wrapper around a dict. This
converts metadata.App to be a subclass of dict so it can behave like a dict
when being dumped and loaded. This makes its drastically easier to use
different data formats for build metadata and for sending data to the
client. This approach will ultimately mean we no longer have to maintain
custom parsing and dumping code.
This also means then that the YAML/JSON field names will not have spaces in
them, and they will match exactly what it used as the dict keys once the
data is parsed, as well as matching exactly the instance attribute names:
* CurrentVersion: 1.2.6
* app['CurrentVersion'] == '1.2.6'
* app.CurrentVersion == '1.2.6'
Inspired by:
https://goodcode.io/articles/python-dict-object/
When making code changes related to the metadata parsing, it is useful to
see how the internal format has changed by seeing the differences in the
dump files. Those files are currently in the binary .pickle format. This
just straight converts them to YAML, which is a text format, so that normal
diff tools work to see changes.
The dump files are named .yaml instead of .yml since .yml is used for hand-
edited YAML files for fdroiddata/metadata, while these dump files here are
a human readable form of a Python pickle.
