1
0
mirror of https://gitlab.com/fdroid/fdroidserver.git synced 2024-11-15 03:20:10 +01:00
Commit Graph

3755 Commits

Author SHA1 Message Date
Hans-Christoph Steiner
beb217b6ae add 'gradleprops' to .pickle files for the metadata tests 2015-09-01 11:53:37 +02:00
Hans-Christoph Steiner
8d6e0aebb9 exit with error if duplicate metadata file is found
In order to prevent confusion caused by multiple metadata files for a given
app, fdroid will exit with an error if it finds any app metadata file with
the same package ID as one that has already been parsed.
2015-09-01 11:39:51 +02:00
Hans-Christoph Steiner
6f334558df metadata: rename metafile to metadatapath when it is a path
To keep the code clear, change the code to use separate variables for the
path to the metadata file, and for the file object of the opened metadata.
2015-09-01 11:39:51 +02:00
Hans-Christoph Steiner
285c06d156 metadata: use unified method for getting default app metadata dict
This puts process of setting up the defaults for the internal dict
that represents a parsed app into a single method that is reused for all
metadata formats.
2015-09-01 11:39:51 +02:00
Hans-Christoph Steiner
32e257d742 support app metadata in YAML format
YAML is a format that is quite similar to the .txt format, but is a
widespread standard that has editing modes in popular editors.  It is also
easily parsable in python.

The .pickle for testing is a lightly edited version of the real metadata
for org.videolan.vlc:

 * comments were removed
2015-09-01 11:39:51 +02:00
Hans-Christoph Steiner
0b62e7f22a set appid in get_default_app_info_list() 2015-09-01 11:39:51 +02:00
Hans-Christoph Steiner
0425788b7d consolidate all post-parse metadata checks
This puts all of the needed post parsing checks on the metadata into a
single method that is used by all parsing methods (.txt, JSON, XML, YAML).
This provides the single place to normalize the internal representation of
the metadata.

It would be good to also change the internal representation to use more Python
bool/int types so that less post parsing is needed for JSON, XML, and YAML.

The SMSSecure test .pickle was changed to account for the use of lstrip()
and rstrip() on all 'script' types.

This also changes the example JSON to use ints for versionCodes
2015-09-01 11:39:51 +02:00
Hans-Christoph Steiner
014df8426f eliminate Boolean metadata type, only 'bool' is needed
No need for a specific metadata type for 'Requires Root'.  Just use 'bool',
then convert on output.  This keeps the internal representation consistent
since all boolean-ish values will be Python bools.  This in turn makes
parsing metadata in various formats easier.  This also makes setting
booleans more tolerant since all these values are valid:

 Yes yes No no True true False false
2015-09-01 11:39:51 +02:00
Hans-Christoph Steiner
ab145de6bc support app metadata in XML format
While the current text metadata format is good for human readability and
editability, it is difficult to produce and parse using code.  XML is a
widespread standard format for easy automatic parsing and creating, while
having decent human readability.

The .pickle for testing is a lightly edited version of the real metadata
for net.osmand.plus:

* comments were removed
* "NonFreeNet" was added as an AntiFeature
2015-09-01 11:39:51 +02:00
Hans-Christoph Steiner
2831b3e93f convert internal representation of AntiFeatures to list
The AntiFeatures metadata is a comma-separated list of tags, like
Categories, so it should also be stored internally as a list.  This makes
parsing XML and JSON easier.

The test cases' .pickle files look like they change a lot, but they really
don't, its only the change of default AntiFeatures value from None to []
2015-09-01 11:39:51 +02:00
Hans-Christoph Steiner
cc29d39f0a metadata: store comments in lists since lists are used everywhere else
The only data produced by parsing metadata that was stored as a tuple was
the comments' key/values.  Everywhere else, lists are used.  This changes
the comments to also only use lists to keep the data consistent internally.
That makes it vastly easier to produce the exact same metadata dict when
parsing .txt, JSON, XML, etc.
2015-09-01 11:39:51 +02:00
Hans-Christoph Steiner
dabb7e42b7 add test case for JSON parsing, based on parsed .txt metadata
The .pickle was created by dumping the output from parsing the current .txt
metadata for org.adaway.  The JSON started from that pickle dump, but was
then hand edited to be more proper JSON, e.g. using boolean values.
2015-09-01 11:39:51 +02:00
Hans-Christoph Steiner
61f6be1a7b in JSON metadata, convert unicode to str to match the internal format
fdroidserver currently uses plain python 2.x strings rather than unicode
strings, so the parsed JSON should produce the same format as the parsed
.txt metadata.
2015-09-01 11:39:51 +02:00
Hans-Christoph Steiner
1bbae4cd9e add test for metadata parsing based on a parse output to a pickle
This is a test to cover future modifications of the .txt metadata parsing.
The pickle file was generated by just dumping the current parsed metadata,
so this test will always succeed if the parsing is not changed.
2015-09-01 11:39:50 +02:00
Hans-Christoph Steiner
eeb8ba02b0 move metadata post-parse processing to reusable function 2015-09-01 11:39:50 +02:00
Hans-Christoph Steiner
9e5dd19fc8 add test files to pre-commit hook and fix pep8 errors 2015-09-01 11:39:50 +02:00
Hans-Christoph Steiner
84c9777e9e support app metadata in JSON format
While the current text metadata format is good for human readability and
editability, it is difficult to produce and parse using code.  JSON is a
widespread standard format for easy automatic parsing and creating, while
having decent human readability.
2015-09-01 11:39:50 +02:00
Hans-Christoph Steiner
2b72634120 use common method to set up defaults in metadata dict
This is needed for the upcoming metadata formats: JSON, XML, etc.
2015-09-01 11:39:50 +02:00
Hans-Christoph Steiner
141e0e24c7 metadata: rename parse_metadata() to parse_txt_metadata()
This makes the code clear as support for other metadata formats is added.
2015-09-01 11:39:50 +02:00
Daniel Martí
04e6f2ebfd Move requests code out of common.py, closes #114 2015-08-31 17:05:08 -07:00
Daniel Martí
3bc0d43786 lint: fixed list false positives, no need for utf 2015-08-31 15:39:07 -07:00
Daniel Martí
19a281d7cb lint: warn about duplicate lines 2015-08-31 15:32:52 -07:00
Daniel Martí
23794446f7 lint: catch more incorrect lists 2015-08-31 15:25:41 -07:00
Daniel Martí
12926420a6 lint: stop accepting Office as valid category 2015-08-30 16:29:03 -07:00
Daniel Martí
18fe72bb0d lint: accept new category Sports & Health 2015-08-30 16:08:22 -07:00
Boris Kraut
28fd99f7ac Add liblzma-dev to buildserver 2015-08-30 12:50:30 +02:00
Daniel Martí
377199d241 Follow-up to 5a0129d8: also replace file mentions 2015-08-29 20:19:32 -07:00
Daniel Martí
9d75b130bc lint: accept all new categories 2015-08-29 18:37:08 -07:00
Daniel Martí
d83d0c2520 lint: new category 2015-08-29 18:24:18 -07:00
Daniel Martí
3a331c5be1 lint: Wallpaper is now called Theming 2015-08-29 17:56:45 -07:00
Daniel Martí
518ba10b9f lint: remove Children as valid category 2015-08-29 17:52:15 -07:00
Daniel Martí
120be4334d Move scan_source into scanner.py
Not really a common.py thing.
2015-08-28 19:20:39 -07:00
Daniel Martí
925fbee3b9 lint: complain about unwrapped descriptions 2015-08-28 19:05:47 -07:00
Daniel Martí
6fe8d96e85 Simplify some file logic with "with" 2015-08-28 18:37:23 -07:00
Daniel Martí
2894786ec9 stats: better exit message 2015-08-28 18:29:28 -07:00
Daniel Martí
05316e3292 stats: write known apks in natural order
By using natural instead of alphabetical sorting, we support sorting by
vercodes properly and thus app versions show in the correct order.
2015-08-28 18:28:39 -07:00
Daniel Martí
f4cbb6b3f8 Simlify known_apks writing 2015-08-28 18:26:23 -07:00
Daniel Martí
79749fe8b4 GitLab CI: Java is required to set up the SDK 2015-08-28 16:34:42 -07:00
Daniel Martí
9e99270680 Gitlab CI: install SDK 2015-08-28 16:11:54 -07:00
Daniel Martí
1eb83dcfea Gitlab CI: run tests too 2015-08-28 16:08:01 -07:00
Daniel Martí
bace7e912f Add missing -v and -q to readmeta 2015-08-28 15:53:59 -07:00
Daniel Martí
a4236d62cd lint: better footer output logic 2015-08-28 15:53:10 -07:00
Daniel Martí
ce8f74c98e lint: warn about unlinkified links 2015-08-28 15:50:34 -07:00
Daniel Martí
e0870f6e76 metadata: error on url titles that aren't 2015-08-28 15:48:53 -07:00
Daniel Martí
2c52f67cb1 lint: complain if link url isn't a url
Moving this check from metadata into lint is perhaps overkill, but at least it
doesn't break fdroid update because of the maintainer notes.
2015-08-28 14:23:30 -07:00
Daniel Martí
ecd0e07e93 Revert "metadata: complain if link url isn't a url"
I already tried this once. It had to be reverted because Maintainer Notes
often don't obey this. I had forgotten.

This reverts commit 97ba94903e.
2015-08-28 14:10:05 -07:00
Daniel Martí
e4219dd438 lint: format descriptions too
Like readmeta. This makes it error if any links are badly formatted or wrong.
2015-08-28 11:09:37 -07:00
Daniel Martí
97ba94903e metadata: complain if link url isn't a url 2015-08-28 11:08:59 -07:00
Daniel Martí
2253399190 Merge branch 'native' into 'master'
wp-fdroid: Don't imply 32bit code does not work on 64bit systems

Maybe someone could also check if the client lists app with 32bit code as incompatible with 64bit systems, even if they can run it (arm->arm64, i386->x64, mips->mips64 etc.).

Ref: https://f-droid.org/forums/topic/accordion-and-melodeon-2/

See merge request !68
2015-08-28 16:10:36 +00:00
Boris Kraut
52e67fa2a1 wp-fdroid: Don't imply 32bit code does not work on 64bit systems 2015-08-28 16:10:24 +02:00