1
0
mirror of https://gitlab.com/fdroid/fdroidserver.git synced 2024-11-14 11:00:10 +01:00
Commit Graph

4116 Commits

Author SHA1 Message Date
Jan Christian Grünhage
f637b461f0 Update support repository. 2017-03-29 10:33:38 +00:00
Hans-Christoph Steiner
8f96c9da3d Merge branch 'smaller-methods' into 'master'
Factor out code into smaller methods to be used by repomaker

See merge request !236
2017-03-27 19:59:51 +00:00
Hans-Christoph Steiner
c3492ab153 Merge branch 'notabug' into 'master'
import: add notabug.org, auto-detect gradle, add options for license and categories

See merge request !237
2017-03-27 19:55:44 +00:00
Hans-Christoph Steiner
916e8eb108 Merge branch 'build_tools_log' into 'master'
post correct build tool versions to wiki

Closes #267

See merge request !235
2017-03-27 19:54:48 +00:00
Hans-Christoph Steiner
7f08fad2c6 Merge branch 'sign-and-verify-update' into 'master'
sign and verify update

See merge request !230
2017-03-27 19:48:36 +00:00
Boris Kraut
13b276e02f import: add option to specify license and categories, auto-detect build.gradle 2017-03-26 19:00:03 +02:00
Boris Kraut
39f7429c9b import: add notabug.org 2017-03-26 18:05:37 +02:00
Torsten Grote
c9b76eb808
Return public key and fingerprint after generating repo signing key 2017-03-22 15:53:20 -03:00
Torsten Grote
c484dc33be
Factor out code from update.main() to individual functions 2017-03-22 15:17:04 -03:00
Michael Pöhn
843595eb33 moved version android tools version log header to corresponding function 2017-03-22 14:56:26 +01:00
Michael Pöhn
0fda23ef80 fix local var name overloading when iterating build flavours 2017-03-22 14:20:54 +01:00
Michael Pöhn
c15f02ed68 fix exception handling in dscanner and update.py 2017-03-22 14:07:54 +01:00
Michael Pöhn
c45c4041a5 Merge remote-tracking branch 'fdroid/master' into build_tools_log 2017-03-22 14:02:45 +01:00
Michael Pöhn
83fe2bda4d fix exception handling in build.py 2017-03-22 14:01:32 +01:00
Hans-Christoph Steiner
223c793201 prefer apksigner if installed, jarsigner sucks
Google has their own utility for verifying APK signatures on a desktop
machine since Java's jarsigner is bad for the task.  For example, it
acts as if an unsigned APK validates.  And to check whether an APK is
unsigned using jarsigner is difficult.

apksigner also does the v2 signatures, so it will have to be used
eventually anyway.  It is already in Debian/stretch and can be
available in jessie-backports if need be.

https://android.googlesource.com/platform/tools/apksig
https://packages.debian.org/apksigner
2017-03-22 10:51:12 +01:00
Hans-Christoph Steiner
ffd490d8da set_command_in_config() for finding CLI tools to run 2017-03-22 10:51:12 +01:00
Hans-Christoph Steiner
70915a7445 verify: fdroidserverid and buildserverid are part of the sig
There are two SHA1 git commit IDs that fdroidserver includes in the builds
it makes: fdroidserverid and buildserverid.  Originally, these were inserted
into AndroidManifest.xml, but that makes the build not reproducible. So
instead they are included as separate files in the APK's META-INF/ folder.
If those files exist in the signed APK, they will be part of the signature
and need to also be included in the unsigned APK for it to validate.
2017-03-22 10:51:12 +01:00
Hans-Christoph Steiner
364e609ebe make fdroid verify use common.verify_apks()
This makes the jarsigner the ultimate and only judge of whether two APKs
match.  This is the best tool since APK signatures are jar signatures. This
should be eventually updated to use the official Android APK signing tool
called apksigner.

https://android.googlesource.com/platform/tools/apksig/
2017-03-22 10:51:12 +01:00
Hans-Christoph Steiner
998b6245e9 verify: ensure only a single signature is in compared APK
The ZIP format allows multiple entries with the exact same filename, and on
top of that, it does not allow deleting or updating entries.  To make the
`fdroid verify` procedure failsafe, it needs to create a new temporary APK
that is made up on the contents of the "unsigned APK" and the signature
from the "signed APK".  Since it would be possible to give a signed APK as
in the unsigned one's position, `fdroid verify` was not able to update the
signature since it was just adding the new signature to the end of the ZIP
file.  When reading a ZIP, the first entry is used.
2017-03-22 10:51:12 +01:00
Hans-Christoph Steiner
3c9eeff7f3 Merge branch 'supportrepo' into 'master'
makebs: update support repo to r46

See merge request !233
2017-03-22 09:28:26 +00:00
Michael Pöhn
bb49123118 post correct build tool versions to wiki 2017-03-21 23:51:15 +01:00
Boris Kraut
53be33055b makebs: update support repo to r46 2017-03-21 21:50:29 +01:00
Hans-Christoph Steiner
71ff1d6257 Merge branch 'ndk14' into 'master'
makebs: add ndk r14

See merge request !231
2017-03-20 09:38:58 +00:00
Boris Kraut
ed310f0404 makebs: add ndk r14 2017-03-19 22:27:51 +01:00
Hans-Christoph Steiner
9b12aa6f71 Merge branch 'scan-single-apk' into 'master'
Allow for scanning single APKs

See merge request !227
2017-03-17 14:47:30 +00:00
Torsten Grote
04db6870d1
Factor code for scanning a single APK out into its own method.
This allows projects using fdroidserver to scan individual APKs without
needing to re-scan all APKs present in a repository.
2017-03-17 10:40:45 -03:00
Torsten Grote
60f166b2c6 Merge branch 'index-v1' into 'master'
app index V1 - support graphics, localization, and more

See merge request !221
2017-03-17 13:34:31 +00:00
Hans-Christoph Steiner
866528de5b signindex: support signing index-v1.jar
This is a bit different than index.jar: instead of their being index.xml
and index_unsigned.jar, the presense of index-v1.json means that there is
unsigned data.  That file is then stuck into a jar and signed by the
signing process.  index-v1.json is never published to the repo.  It is
included in the binary transparency log, if that is enabled.
2017-03-17 14:12:03 +01:00
Hans-Christoph Steiner
fa657ce720 move update.signjar() to common so it can also be used in signindex 2017-03-17 13:55:40 +01:00
Hans-Christoph Steiner
696bae4d6d warn and ignore graphics files that have no matching app metadata
If there are graphics files that are in repo/ but there is no metadata for
the associated app, then do not include the graphics in the index. Issue a
warning about this.
2017-03-17 13:55:40 +01:00
Hans-Christoph Steiner
21ba89a581 copy graphic assets from fdroiddata and app source into repo
This looks for standard graphics assets in two standardized locations, one
in metadata/ subdirs and another in fastlane's standard dirs in the app's
source repo.  When it finds them, it copies them into the repo, where they
will then be included in the index for use by fdroidclient.  The images in
the metadata/ folder of fdroiddata take precendence over the files in the
app's source repo.

So like this for including graphics in fdroiddata:
  metadata/packageName/locale/filename.(png|jpg|jpeg)
for example:
  metadata/org.videolan.vlc/en-US/featureGraphic.png
or
  metadata/info.guardianproject.ripple/zh-CN/phoneScreenshots/screenshot1.png

Including graphics in fdroiddata would be optional. The prefered way to get
graphics into the repo would be for the files to be in the git repo in a
standard location.  This fastlane layout is currently supported:

https://github.com/fastlane/fastlane/blob/1.109.0/supply/README.md#images-and-screenshots
2017-03-17 13:55:40 +01:00
Hans-Christoph Steiner
713d9195c3 ensure that app.Categories is always a list of strings
This just makes it easier for people writing build recipes.  Rewriting will
output a list of strings as well.

The test index.xml and categories.txt are updated to include the new number
categories, and the changed CurrentVersionCode to 2147483647 (MAX_VALUE)
2017-03-17 13:55:40 +01:00
Hans-Christoph Steiner
7044a909ca include graphics and screenshots from repo in V1 index
If the repo has store graphics and/or screenshots, then include those in
the metadata.  This follows the possible graphics for Google Play, and the
file naming scheme of the open source 'fastlane' tool for managing those
files.

https://github.com/fastlane/fastlane/blob/1.109.0/supply/README.md#images-and-screenshots
https://support.google.com/googleplay/android-developer/answer/1078870?
https://android-developers.blogspot.com/2011/10/android-market-featured-image.html

Signed-off-by: Hans-Christoph Steiner <hans@eds.org>
2017-03-17 13:55:40 +01:00
Hans-Christoph Steiner
ab7e85c951 include push install/uninstall requests in index-v1
Since the index-v1 is generated straight from the internal dict, this just
moves the generation earlier, and feeds it into the apps dict.
2017-03-17 13:55:40 +01:00
Hans-Christoph Steiner
6c2cf2ccdd convert internal dict keys to match fdroidclient's Apk class
This syncs up the field names between the fdroiddata .yml files, the keys
used in the implementation in fdroidserver, the index data format, and the
final data structures in fdroidclient.  This makes it easier for devs to
follow, and makes the Jackson parsing library automatically handle
converting the data from the index file to Java instances.

This bumps the metadata version since the apkcache will have to be
discarded.

Here are the name changes:
* apkname --> apkName
* id --> packageName
* sha256 --> hash
* version --> versionName
* versioncode --> versionCode

tests/repo/index.xml was changed only to bump the metadata version
from 17 to 18.
2017-03-17 13:55:40 +01:00
Hans-Christoph Steiner
fcb7a0feb4 fdroid update uses datetime instances for timestamps
Using datetime instances as the internal format makes it much easier to
convert between the formats needed for index.xml and index-v1.  apkcache
still uses time tuples and known_apks.txt still uses the ISO date.
2017-03-17 13:55:40 +01:00
Hans-Christoph Steiner
c9aa26d89e add index V1 format, a direct translation of internal dict
Python encode/decode libs work directly with dicts, so the internal dict
can just be passed directly to any of these libs (pyyaml, pyjson, msgpack,
simplejson, etc).  This still generates the exact same index.xml as before.

This converts the internal format for the repo timestamp to a datetime
instance, which can be easily converted to UNIX time in seconds for XML
and UNIX time in milliseconds for the new index formats.  UNIX time in
milliseconds is directly serialized into a java.util.Date instance by
Jackson.
2017-03-17 13:55:40 +01:00
Hans-Christoph Steiner
b3a5db52f7 break out jar signing into function so it can be reused
The new index format will also need to use jar signing
2017-03-17 13:55:40 +01:00
Hans-Christoph Steiner
3afd6ca684 only set 'No description available' in the old index format
The new index format aims to be a direct representation of the data, then
the clients/website will handle the rest.
2017-03-17 13:55:40 +01:00
Michael Pöhn
1bc0ec27c0 Merge branch 'buildserver-auto-install' into 'master'
buildserver auto install

Closes #247

See merge request !229
2017-03-17 12:51:15 +00:00
Hans-Christoph Steiner
b6d8916881 Merge branch 'metadata_output_vars' into 'master'
metadata output-lines now substitute $$VERSION$$/etc.

Closes #263

See merge request !228
2017-03-17 11:37:18 +00:00
Hans-Christoph Steiner
02f1a30ba8 Merge branch 'binary-transparency' into 'master'
new repo-wide config option for a "Binary Transparency" log based on git

See merge request !226
2017-03-16 19:57:56 +00:00
Hans-Christoph Steiner
a737255ce4 add a simple test of the binary transparency log
This makes sure it gets generated with two commits: README and first log.
2017-03-16 19:22:32 +01:00
Hans-Christoph Steiner
110914ebf3 update: include git config name/email in binary transparency log 2017-03-16 17:48:54 +01:00
Hans-Christoph Steiner
580a9eb058 buildserver: support HTTPS Debian mirrors
The ever troublesome gpjenkins box needs to use HTTPS mirrors.  Plus it
improves the security of the buildserver, since there have been CVEs that
HTTPS would protect against:
https://www.debian.org/security/2016/dsa-3733
2017-03-16 15:06:35 +01:00
Hans-Christoph Steiner
64ea4caac1 buildserver: allow gradle/sdkmanager to install into the new m2repository
Google is pushing gradle towards downloading all the SDK components that it
needs, rather than having a preconfigured SDK installed.  The buildserver
strongly supports the old model, with added checksum checking even. We can
still support the old model by pre-configuring the SDK and locking it down
as root. This can then also support the new model by setting the file perms
so that new packages can be auto-installed, but they cannot overwrite any
packages that come pre-installed and pre-verified.

fdroiddata!2096
closes #247
2017-03-16 13:23:42 +01:00
Michael Pöhn
d2570a493e metadata output-lines now substitute $$VERSION$$/etc. 2017-03-16 09:24:05 +01:00
Hans-Christoph Steiner
c5c244ccbd generate HTML diff from verify using diffoscope
If diffoscope is installed, then use it to generate HTML diffs of all
verification builds.
2017-03-15 22:12:04 +01:00
Hans-Christoph Steiner
03ec590989 new repo-wide config option for a Binary Transparency Log
A Binary Transparency Log is a append only log of all binaries published by
a repo.  This is useful for people to find whether the binary they have
matches what F-Droid has published, and also makes it more difficult for
the published history to be changed without notice, or for a server to give
specific users custom malware binaries.

https://www.eff.org/deeplinks/2014/02/open-letter-to-tech-companies
2017-03-15 22:12:04 +01:00
Ciaran Gultnieks
0ef818486d Remove whitespace from HTTP update check versions 2017-03-15 19:54:09 +00:00