1
0
mirror of https://gitlab.com/fdroid/fdroidserver.git synced 2024-09-19 11:30:15 +02:00
Commit Graph

2071 Commits

Author SHA1 Message Date
Michael Pöhn
1bd51966b8 vmtools debug logging for check_output calls 2017-05-23 20:06:06 +02:00
Hans-Christoph Steiner
5dbcd0e9bd added box handling to vmtools 2017-05-23 20:06:06 +02:00
Hans-Christoph Steiner
01b6473823 refactored kvm_package to vmtools 2017-05-23 20:06:06 +02:00
Michael Pöhn
d180aa2658 fix virsh destroy parameters 2017-05-23 20:06:06 +02:00
Michael Pöhn
4347c10d9d use configured vm provider in when calling destroy in makebuildserver 2017-05-23 20:06:06 +02:00
Michael Pöhn
a414aa00ff use overhauled mv destroy code in build.py 2017-05-23 20:06:06 +02:00
Michael Pöhn
92fada803e overhauled and moved destroying builder vm to vmtools.py 2017-05-23 20:04:08 +02:00
Michael Pöhn
fb03e17849 fdroid build: added sleep after destroy/undefine 2017-05-23 20:04:08 +02:00
Michael Pöhn
413c3836d5 auto-reset broken builder vm 2017-05-23 20:04:08 +02:00
Michael Pöhn
ce3c959ce5 scan vm provider again after re-creating builder vm 2017-05-23 20:04:08 +02:00
Michael Pöhn
34cddd3be8 delete associated libvirt domain/image when destroying builder vm 2017-05-23 20:04:08 +02:00
Michael Pöhn
fad98eeb7f build: fixed kvm snapshot support; makebuildserver: setup kvm ssh credentials 2017-05-23 20:04:08 +02:00
Michael Pöhn
fa2d44ee94 added libvirt vm-provider support to build.py 2017-05-23 20:04:08 +02:00
Hans-Christoph Steiner
7ef0d5dfd8 include class like UNIX tail -f for displaying logs
This allows fdroidserver to easily log activity while displaying it at the
same time.
2017-05-23 20:04:08 +02:00
Torsten Grote
1fcd8e63a3
Replace sys.exit() in non-main functions by exceptions
Also move all exceptions into one module
2017-05-22 16:51:11 -03:00
Hans-Christoph Steiner
0345d2c79f update: find aapt when it is not in the PATH
This is some very messy logic built up since 2010.  This will all go away
once we have a python3 version of androguard available.

The removed imports and `dir(APK)` is to silence pyflakes

closes #303
2017-05-18 17:21:05 +02:00
Hans-Christoph Steiner
c348186ad6 update: fix fastlane scraping, it uses a subdir called 'images'
https://commons.wikimedia.org/wiki/File:GetChromium_FeatureGraphic_1024x500.png
2017-05-18 16:13:17 +02:00
Hans-Christoph Steiner
77a65be9db update: skip packages without metadata in index V1
If a package does not have a matching "app" metadata file, then it should
not be added to the index. index V0 already does this properly.
2017-05-18 16:13:17 +02:00
Hans-Christoph Steiner
a17b95a860 update: improved logging
the aapt warnings were very verbose, and the other now includes the
file name type for screenshots, rather than the filename twice.
2017-05-18 16:13:17 +02:00
Hans-Christoph Steiner
5d705452f5 update: allow repo files to use _ in the file names
_ is a valid character for Java package names, so it should also work in
the repo file naming scheme. This makes it so it only splits the file
name based on the last _.
2017-05-18 16:13:17 +02:00
Hans-Christoph Steiner
84bb41a91f metadata: switch from deprecated cgi.escape to html.escape
cgi.escape is deprecated in Python 3.x and has security issues:
https://bugs.python.org/issue26398

html.escape() differs from cgi.escape() by its defaults to quote=True:

 s = html.escape( """& < " ' >""" )   # s = '&amp; &lt; &quot; &#x27; &gt;'
2017-05-18 16:13:17 +02:00
thez3ro
ec30d73f13
fix double pushing on remote 2017-05-17 23:08:25 +02:00
thez3ro
846f5422fd
Use gitlab pages as mirror 2017-05-17 21:08:34 +02:00
Hans-Christoph Steiner
eb49c9e8f3 Merge branch 'master' into 'master'
replace aapt with androguard

See merge request !234
2017-05-05 09:10:32 +00:00
Michael Pöhn
ff3baefbed Merge branch 'yaml' into 'master'
basic support for YAMLin `fdroid rewritemeta`

Closes #285 and #169

See merge request !260
2017-05-05 08:35:44 +00:00
thez3ro
06598ae406
use androguard if aapt isn't found 2017-05-04 23:35:17 +02:00
Torsten Grote
8d424f19ec
Support ETag when downloading repository index 2017-05-02 15:37:02 -03:00
Hans-Christoph Steiner
4f758f5c78 fix white space trimming when writing .txt format
Some of the internal data format changes somehow broke the white space
trimming when writing out .txt.

closes #285
2017-05-02 14:55:23 +02:00
Hans-Christoph Steiner
0d00705aed use pyyaml for rewriting .yml metadata files
This replaces the broken, custom code with the standard YAML lib.

In rewritemeta, do not call app.metadatapath since it will be deleted when
the dict is cleaned up for outputing. metadatapath is only used internally
and should not be written out.

closes #169
refs #290
2017-05-02 14:55:23 +02:00
Hans-Christoph Steiner
cdef5bcd92 downcase all 'localized' key names to match the rest of index-v1
This is a little omission.  keys that are used in metadata/*.yml all start
with an UpperCase letter, but in fdroidserver, index-v1.json, and
fdroidclient, it is all camelCase with lowercase first letter. The keys
from the 'localized' section are currently never in metadata/*.yml, so
these keys never get downcase.  This change will break fdroidclient
versions that do not also have this change, but since we're in alpha, that
should be fine.

If support for a 'localized' section is added to metadata/*.yml, then the
keys there should probably be UpperCase CamelCase to match the other keys.
2017-04-27 21:12:49 +02:00
Hans-Christoph Steiner
9f9f0d1a16 Merge branch 'git-identity-file' into 'master'
Make git server mirror upload honor config['identity_file'] option

See merge request !259
2017-04-24 21:24:07 +00:00
Torsten Grote
c137231d8b
Make git server mirror upload honor config['identity_file'] option 2017-04-24 16:22:33 -03:00
Michael Pöhn
fc4e40255d remove build output when not reproducible 2017-04-23 11:33:51 +02:00
Michael Pöhn
5b7209d882 unified method for naming build output 2017-04-22 12:04:32 +02:00
Michael Pöhn
62af9b8e3d check meld config for manual apk diffs 2017-04-22 10:48:50 +02:00
Michael Pöhn
4044e06e7b use verify_apks for checking build output 2017-04-22 10:48:50 +02:00
Michael Pöhn
a3e7eacc9b compare apk with Binaries from metadata right after building 2017-04-22 10:48:50 +02:00
Michael Pöhn
abe04272b1 throw proper error when download of apks fail 2017-04-20 17:48:38 +02:00
Hans-Christoph Steiner
fd21d68bc1 update: fix glob to actually match all locales
glob != regexp, [A-Z-.@] means one of those chars is required in that
position, so the glob pattern was requiring a least 3 chars.  Locales are
usually just two lower case letters, e.g. vi, de, ar.
2017-04-19 10:05:24 +02:00
Hans-Christoph Steiner
cb49f57c06 support "Author Web Site" as metadata field
Fastlane Supply, Triple-T Gradle Play Publisher, and many app stores
include the possibility to specify a website for the author, as distinct
from the website for the app.

closes #204
2017-04-19 10:05:24 +02:00
Hans-Christoph Steiner
18f949c62b include Author/Contact info from Triple-T Gradle Play Publisher
This is just the bare minimum, it adds it to the index, but AuthorPhone
and AuthorWebsite are not yet supported by fdroidclient.  AuthorName is.

#204
2017-04-19 10:05:24 +02:00
Hans-Christoph Steiner
9589d13ef2 update: include "What's New" texts when they are available
This uses the "What's New" entry for the CurrentVersionCode and includes it
as the current WhatsNew metadata for the App class.

Things like fastlane supply and Google Play support a "What's New" entry
per-APK, but fdroidclient does not current use anything but the current
version of this data.  Right now, it seems we probably only want to have
the latest WhatsNew in the index to save space.

In theory, we could make the WhatsNew data structure follow the structure
of fastlane/Play, but that would quite a bit of complexity for something
that might never be used.

fdroidclient#910
2017-04-19 10:05:24 +02:00
Hans-Christoph Steiner
822c2d3992 update: app store metadata from Triple-T gradle play publisher
Since f-droid.org has the source repos, `fdroid update` can just grab files
from the source repo.  This makes `fdroid update` use the metadata that is
laid out in the Triple-T gradle play publisher filesystem layout:

https://github.com/Triple-T/gradle-play-publisher#upload-images
https://github.com/Triple-T/gradle-play-publisher#play-store-metadata

closes #143
2017-04-19 10:05:24 +02:00
Hans-Christoph Steiner
89c480181c update: include localized app store metadata from fastlane
This includes all metadata from the app's source repo following the path
layout specified for the fastlane supply tool:
https://github.com/fastlane/fastlane/blob/1.109.0/supply/README.md#images-and-screenshots

refs #35
closes #143
2017-04-19 10:05:24 +02:00
Hans-Christoph Steiner
a7daec7c36 add char limits for text: Name=30, Video=256, WhatsNew=500
Since this info can come from automated sources, this prevents giant blobs
from being mistakenly stuck in these fields.  It also brings it inline with
the standard lengths other popular app stores use.

https://support.google.com/googleplay/android-developer/answer/113469?hl=en
https://stackoverflow.com/a/20276565
2017-04-19 10:05:24 +02:00
TheZ3ro
89b0dea2fa Merge branch 'support-new-features-with-offline' into 'master'
Support new features with offline signing

See merge request !250
2017-04-13 15:21:01 +00:00
Hans-Christoph Steiner
489b129dd8 PEP8 E305 expected 2 blank lines after class or function definition 2017-04-13 10:55:07 +02:00
Hans-Christoph Steiner
fcb0634eaa server: store virustotal JSON locally as reusable data cache
This helps with uploading faster when hitting the public API rating
limiting since it will not hit the API if there is already a matching
response on the local filesystem.  This also provides a source of data for
things like repomaker to diplay to users.
2017-04-13 00:49:36 +02:00
Hans-Christoph Steiner
d74d6160dd server: check virustotal has APK before uploading it
This restructures the virustotal uploading so that first checks whether
virustotal already has the file, and only if not does it upload it.  This
also handles the public API rate limiting, which returns an HTTP 204. This
will now try again until it succeeds, even when rate limited.   Instead of
just getting the list of files from the filesystem, this reads the index-v1
which also already has the SHA256 in there.  virustotal also uses SHA256 as
a unique ID for files.
2017-04-12 15:04:05 +02:00
Hans-Christoph Steiner
8dd9d63e8f server: show progress bar when pushing git mirrors
The git mirrors can be really slow to upload, so they need a progress bar.
2017-04-12 15:04:05 +02:00