1
0
mirror of https://gitlab.com/fdroid/fdroidserver.git synced 2024-11-09 00:40:11 +01:00
Commit Graph

4400 Commits

Author SHA1 Message Date
Hans-Christoph Steiner
39fd664778 update: create 'archive/' if needed when moving APKs
Normally, just 'repo/' is created by default, e.g. `fdroid init`. If APKs
are dumped into 'repo/', then have invalid signatures, then they'll be
automatically moved to 'archive/', which therefore needs to exist.
2017-07-03 10:02:51 +02:00
Hans-Christoph Steiner
746d4bd4cf update: allow_disabled_algorithms option to keep MD5 sigs in repo
The new policy is to move APKs with invalid signatures to the archive,
and only add those APKs to the archive's index if they have valid MD5
signatures.

closes #323
closes #292
2017-07-03 10:02:51 +02:00
Hans-Christoph Steiner
b7260ea854 update: allow deprecated signatures only in the archive
In April 2017, Oracle's jarsigner and Google's apksigner both switched to
considering any APK signature that uses MD5 as unsigned.  Any old build
is likely to have a MD5 signature.  This sets up the archive as the only
place where these "disabled algorithms" are allowed in the repo, and
marks any APK signed by a "disabled algorithm" as having a "known
vulnerability"

This also now automatically moves APKs with invalid signatures to the
archive section.

#323
2017-06-28 23:53:42 +02:00
Hans-Christoph Steiner
0047f19d56 update: move duplicated code into move_apk_between_sections() 2017-06-28 23:53:42 +02:00
Hans-Christoph Steiner
e1492148fa fix "Archive Policy:" field, APKs can move in/out of archive
The original logic was checking keepversions against the len() of ALL the
APKs in the repo/archive.  The correct thing is to check against the
number of APKs available for the given packageName/appid.

closes #166
2017-06-28 23:53:42 +02:00
Hans-Christoph Steiner
be2926ffc8 tests: test moving files to and from the archive
#166
2017-06-28 23:53:42 +02:00
Hans-Christoph Steiner
0d3f4547a9 Merge branch 'random-small-fixes' into 'master'
Random small fixes

Closes #222

See merge request !292
2017-06-28 21:53:18 +00:00
Hans-Christoph Steiner
7de6e4dfed init: update docs links for next steps 2017-06-28 23:14:57 +02:00
Hans-Christoph Steiner
214c9f7a2b lint: check file extension for metadata
This helps keep fdroiddata clean, on @krt's request.

closes #222
2017-06-28 23:14:57 +02:00
Hans-Christoph Steiner
4b99a505b3 locale: fix broken source string 2017-06-28 23:14:57 +02:00
Hans-Christoph Steiner
9886e539d3 scan APKs for signs of "Master Key" exploit
This exploit is old, and was fixed in 4.4.  But it was easy to exploit,
so it is still worth scanning for it.  It is also easy to scan for, since
valid APKs should not have files with duplicate names.  In theory, this
could look for duplicate file names for any file, but this limits the
false positives by only checking names of files related to executing code.

fdroidclient#40
2017-06-28 23:14:57 +02:00
Hans-Christoph Steiner
5fd014a852 update: move btlog import since btlog.py requires python3-git
To keep the dependencies limited to where they are needed.
2017-06-28 23:14:57 +02:00
Hans-Christoph Steiner
7f9f47496f gitlab-ci: fix metadata_v0 test to run on the right commits
Checking out master will often mean its testing the wrong commit, since
merge requests rarely are in master.
2017-06-28 23:14:57 +02:00
Hans-Christoph Steiner
6e72ffb043 gitlab-ci: fix metadata_v0 test for new buildozer build flag
Just remove it, since 0.7.0 does not know about that build flag.
2017-06-22 16:33:11 +02:00
Hans-Christoph Steiner
8b09ef46a0 Merge branch 'buildozer-build' into 'master'
Added a method to build python/kivy projects using buildozer.

See merge request !290
2017-06-21 12:04:45 +00:00
lb@lb520
bab8f8ea02 Test metadata files updated. 2017-06-21 08:26:52 +02:00
Hans-Christoph Steiner
5e7e17e800 gitlab-ci: properly quote colons 2017-06-20 20:27:23 +02:00
Hans-Christoph Steiner
4939b8a530 gitlab-ci: ignore blank Description in metadata_v0 test
We'll eventually be moving to having the internal representation use a
'' or None when there is no Description, so no use in fixing this.
2017-06-20 18:15:12 +02:00
lb@lb520
0672a6f1ec build.py corrected 2017-06-20 15:56:15 +02:00
lb@lb520
230286786f Added a method to build python/kivy projects using buildozer. 2017-06-20 14:16:31 +02:00
Hans-Christoph Steiner
26a4e33a7a Merge branch 'vagrantfile-box' into 'master'
Copy initial buildserver CPU/memory configuration to final box Vagrantfile

See merge request !279
2017-06-19 09:36:21 +00:00
Hans-Christoph Steiner
0f6dc31bce Merge branch 'gradle4' into 'master'
makebs: add gradle4.0

See merge request !289
2017-06-17 11:01:39 +00:00
Hans-Christoph Steiner
ee1b7ccea2 Merge branch 'googlemaven' into 'master'
Add Google's maven repo to allowed list

See merge request !275
2017-06-17 11:00:57 +00:00
Boris Kraut
1b88f886e7 makebs: add gradle4.0 2017-06-17 01:39:44 +02:00
Hans-Christoph Steiner
790c881000 set gettext template to UTF-8 2017-06-16 22:21:08 +02:00
Hans-Christoph Steiner
0dff73f29a move translation files to 'locale' like Repomaker does
The django gettext layout seems to the most standard approach with
Python projects.
2017-06-16 22:10:48 +02:00
Hans-Christoph Steiner
ca62f9c0b8 jenkins-build: point to new dir named after the script
This changes out the arbitrary names used before.
2017-06-15 17:21:35 +02:00
Hans-Christoph Steiner
55980a45cc ignore test files from d06f05e9c8 2017-06-15 17:21:35 +02:00
Hans-Christoph Steiner
7d6d827e0b jenkins-build: git clean before running fdroid cli tests
Otherwise, `fdroid init` will fail since the old repo files are present.
2017-06-15 17:21:35 +02:00
Hans-Christoph Steiner
d06f05e9c8 update: do not crash on screenshots without texts/graphics
`fdroid update` crashed for apps that only had screenshots but no graphics
or localized texts because destdir was not being set in that case.  This
fixes that and adds a test case.

closes #320
!286
2017-06-12 21:25:54 +02:00
Hans-Christoph Steiner
e1d316a4bc Merge branch 'fix' into 'master'
fix #320 due to unintentionally reusing the same variable name

Closes #320

See merge request !286
2017-06-12 19:26:04 +00:00
Rui Zhao (renyuneyun)
b5b463a5ec fix !320 due to unintentionally reusing the same variable name 2017-06-12 18:09:00 +01:00
Boris Kraut
03ef37cbde Merge branch 'master' into 'master'
makebuildserver: bump tools, add api26 and build-tools 26

See merge request !284
2017-06-10 09:55:47 +00:00
Sergey Eremin
eb64b0889d makebuildserver: bump tools, add api26 and build-tools 26 2017-06-09 20:35:04 +03:00
Boris Kraut
c687a5c65d Merge branch 'fdroiddata-localization-fixes' into 'master'
fdroiddata localization fixes

See merge request !283
2017-06-08 01:54:34 +00:00
Hans-Christoph Steiner
30ff777897 rewritemeta: do not include empty Summary: or Description:
Since the Summary: and Description: in the metadata file has the highest
priority of all the localized texts, adding blank versions means that
apps would always have blank Summary and Description even if the app has
those fields in the localized sections of fdroiddata and/or in the app's
source repo itself.

fdroiddata!2262
2017-06-07 22:10:25 +02:00
Hans-Christoph Steiner
18c3bfa5fb lint can no longer properly detect unset Summary/Description
Since the Summary/Description can now be set in the app's source code, or
in fdroiddata/metadata/<packageName>/<locale>/*.txt, this lint check is
no longer valid.  It is important to check whether these texts are empty,
but it'll require some thinking about how and where to best to that.
`fdroid update` will have access to all that data, but perhaps at that
point it is too late.

Also, the current text prioritization puts Summary/Description in the
.txt/.yml file at the highest priority, overriding every other copy,
including in fdroiddata/metadata/<packageName>/<locale> and in the app's
source code.
2017-06-07 22:10:25 +02:00
Hans-Christoph Steiner
37c3bf304c support fdroid names in filenames for localized texts
* "full description" is just "description"
* "short description" is "summary"
* "title" is "name"
2017-06-07 22:10:25 +02:00
Hans-Christoph Steiner
43d3a9c7a5 support fastlane simplified metadata dir
Running `fastlane init` gave me a much simpler directory layout, which
turns out to be the same as what is used for fdroiddata.
2017-06-07 22:10:25 +02:00
Hans-Christoph Steiner
e36864e4f0 Merge branch 'check-vmx-libvirt' into 'master'
Use Qemu instead of KVM when we don't have VMX/SVM

See merge request !282
2017-06-07 18:19:40 +00:00
Torsten Grote
061093d014 Merge branch 'localization-template' into 'master'
add core help strings to gettext source file

See merge request !281
2017-06-02 14:22:26 +00:00
Torsten Grote
78d4603090 Merge branch 'duplicate-apk-processing' into 'master'
APK processing for duplicate versionCodes and renaming

See merge request !280
2017-06-02 14:20:20 +00:00
Hans-Christoph Steiner
75f29dd690 add core help strings to gettext source file
This will allow us to put these up on Weblate and have people start
translating them.  Then we can figure out how to actually include and
deploy the translations later.  It is unfortunately non-trivial, since
we have to manually figure out the install paths.
2017-06-02 14:06:17 +02:00
Willem Mulder
60bb34ef1a Use Qemu instead of KVM when we don't have VMX/SVM 2017-06-02 11:37:15 +02:00
Hans-Christoph Steiner
8776221988 check signature and OpenSSL after APK has proven valid
If working with a random grabbag of APKs, there can be all sorts of
issues like corrupt entries in the ZIP, bad signatures, signatures that
are invalid since they use MD5, etc.  Moving these two checks later means
that the APKs can be renamed still.

This does change how common.getsig() works.  For years, it returned
None if the signature check failed.  Now that I've started working
with giant APK collections gathered from the wild, I can see that
`fdroid update` needs to be able to first index what's there, then
make decisions based on that information.  So that means separating
the getsig() fingerprint fetching from the APK signature verification.

This is not hugely security sensitive, since the APKs still have to
get past the Android checks, e.g. update signature checks.  Plus the
APK hash is already included in the signed index.
2017-06-01 17:45:29 +02:00
Hans-Christoph Steiner
372c8b418d strip file extension from generated name for non-APKs
With a generic file, the file name is the only guaranteed name metadata
field.  So if the name is not specified in the metadata, then the name
is set to the filename.  This changes that so that the file extension is
stripped from that generated name.
2017-06-01 16:01:05 +02:00
Hans-Christoph Steiner
6105f8a184 use var naming scheme in KnownApks (apk --> apkName)
Everywhere else, the file name of the APK is called apkName.
2017-06-01 16:01:05 +02:00
Hans-Christoph Steiner
9471bf2731 regexs for getting packageName and versionCode from filenames
This is useful for parsing APK files, which can include packageName,
versionCode, and optionally 7 char signing key ID (i.e. <sig>).
This also can set the packageName and versionCoe for non APK files, so
that it is easy to assign them to metadata files, and to allow for
upgrades by setting the versionCode in the filename.
2017-06-01 16:01:05 +02:00
Hans-Christoph Steiner
ceac6d25cb index.xml cannot handle APKs with the same packageName/versionCode
Really, it is the fdroidclient parser of index.xml that fails, due to the
hardcoded expectation that there will only ever be a single APK for any
given versionCode.  We keep index.xml backwards compatible for old
clients, and use index-v1.json to support new things.  Having multiple
APKs that have the same packageName and versionCode will break the client
v0.103.* since that version uses index-v1.json, but still has the hard-
coded database parsing stuff.

#153
2017-06-01 16:01:05 +02:00
Hans-Christoph Steiner
4053f03d77 update: add --rename-apks to force APK filenames to fdroid standard
uses the standard package.name_123.apk.  If that exists, it appends the
shasum.  If that exists, then its a duplicate, so its deleted. This should
help @SergeWinters with his 12,000 APKs.
2017-06-01 16:01:05 +02:00