1
0
mirror of https://gitlab.com/fdroid/fdroidserver.git synced 2024-11-18 12:40:11 +01:00
Commit Graph

8596 Commits

Author SHA1 Message Date
Hans-Christoph Steiner
03223ab9df jenkins-build-makebuildserver: test the whole build/publish workflow
For full deployments like f-droid.org, the building happens on a separate
machine from the signing.  This adds a basic test of that kind of setup.
2017-02-13 20:08:45 +01:00
Hans-Christoph Steiner
9c36894955 Merge branch 'supportlib' into 'master'
makebs: bump support repo to r43

See merge request !205
2017-02-13 19:05:44 +00:00
Boris Kraut
e382f09fd6 makebs: bump support repo to r43 2017-02-11 14:40:40 +01:00
Ciaran Gultnieks
70c347264c Merge branch 'fix-build-and-vlc' into 'master'
fix CI, makebuildserver, and VLC build

See merge request !204
2017-02-07 13:27:56 +00:00
Hans-Christoph Steiner
6debb3ebbf jenkins-build-makebuildserver: include VLC as a test build 2017-02-07 11:05:18 +01:00
Hans-Christoph Steiner
1eb65c7087 buildserver: use automake and cmake from jessie-backports
These should be reasonably backwards compatible, and there is already
automake1.11 for those that need a version that old.  As for cmake, there
are five apps that seem to it:

com.amaze.filemanager
org.dolphinemu.dolphinemu
org.navitproject.navit
org.yabause.android
org.videolan.vlc

It looks like VLC is the only app that is currently building and using
cmake in the most recent releases.  Some of them used to use cmake, but no
longer.
2017-02-07 10:47:20 +01:00
Hans-Christoph Steiner
36272656fc gitlab-ci: workaround "ImportError: No module named 'packaging'"
https://github.com/pypa/setuptools/issues/937

fdroid/ci-images#1
2017-02-06 17:28:07 +01:00
Hans-Christoph Steiner
4929349555 buildserver: add openjdk-8-jdk-headless depends from backports 2017-02-06 14:59:15 +01:00
Ciaran Gultnieks
c344c31271 Merge branch 'supportlib' into 'master'
makebs: upgrade support repo to r42

See merge request !203
2017-02-04 17:34:49 +00:00
Boris Kraut
f32f4a68d1 makebs: upgrade support repo to r42 2017-02-04 15:01:15 +01:00
Hans-Christoph Steiner
1993cc874d Merge branch 'collection-of-fixes' into 'master'
include fdroidserverid in build log post on wiki

See merge request !201
2017-01-21 21:34:33 +00:00
Hans-Christoph Steiner
69d39bb301 build: include fdroidserverid in build log post on wiki 2017-01-20 12:12:20 +01:00
Hans-Christoph Steiner
d08bd32a35 update URL for git repo of test app 2017-01-20 12:12:20 +01:00
Hans-Christoph Steiner
78a3e35ef7 break out categories.txt generation to standalone method
This is to clean up the code for the introduction of a new index format.
This also makes it so that only repo/categories.txt is generated, and not
archive/categories.txt.

Currently, the only thing that I can find that reads categories.txt is the
wordpress plugin, e.g. wp-fdroid.  And it only reads repo/categories.txt
not archive/categories.txt.
2017-01-20 10:47:35 +01:00
Hans-Christoph Steiner
d54ad41938 fix bad caching of non-APK files in the repo
Silly mistake in 07ce948809
2017-01-20 10:47:35 +01:00
Hans-Christoph Steiner
8147f92355 Merge branch 'fdroid-publish-ota-zip' into 'master'
`fdroid publish` now includes OTA ZIPs and related source

See merge request !193
2017-01-12 13:03:51 +00:00
Hans-Christoph Steiner
7a1b99f11a Merge branch 'libtool-from-testing' into 'master'
buildserver: support installing packages from Debian/testing

Closes #224

See merge request !200
2017-01-12 12:59:57 +00:00
Hans-Christoph Steiner
4741f75f03 buildserver: update android_m2repository to latest (r41) 2017-01-12 10:19:38 +01:00
Hans-Christoph Steiner
948689281a ensure fdroiddata metadata file overrides .fdroid.yml in source
If the already parsed App instance from metadata/*.* has a field, then the
value coming from .fdroid.yml should not override it.
2017-01-11 22:42:41 +01:00
Hans-Christoph Steiner
7af7297ca8 buildserver: support installing packages from Debian/testing
Sometimes, a build process requires newer versions of build tools than are
available in Debian/stable.  Oftentimes, using the package straight from
Debian/testing works fine when a package is not available as a backport.
libtool 2.4.6 is needed for building VLC, so it is one example of this.

The preferences file sets up the apt "pinning" so that all updates are not
installed from testing, only the packages that are requested by adding
"/testing" after then package name.

closes #224
2017-01-10 10:56:03 +01:00
Hans-Christoph Steiner
bc27dee950 fdroid publish now includes OTA ZIPs and related source
This adds support for publishing ZIP files which were built with
`fdroid build`.  This is for "Over-The-Air" (OTA) update ZIP files for
flashing to ROMs.  The first example of this is the Privileged Extension,
which must be installed by flashing an OTA ZIP on Android > 5.0.

!181
https://gitlab.com/fdroid/privileged-extension/issues/9
https://gitlab.com/fdroid/privileged-extension/issues/10
https://gitlab.com/fdroid/fdroiddata/merge_requests/1804

Also, "if app.Binaries:" is the same as "if app.Binaries is not None:", but
is the standard Python style.
2017-01-09 11:32:52 +01:00
Hans-Christoph Steiner
0eea26753a Merge branch 'gradle33' into 'master'
makebs: install gradle 3.3

See merge request !199
2017-01-09 10:31:57 +00:00
Boris Kraut
d63a1fde19 makebs: install gradle 3.3 2017-01-08 21:13:24 +01:00
Ciaran Gultnieks
3d103875ea Merge branch 'gettext-backport' into 'master'
buildserver: install gettext from jessie-backports

Closes #224

See merge request !198
2017-01-07 19:05:08 +00:00
Hans-Christoph Steiner
8caf98a00b buildserver: install gettext from jessie-backports
In order to install a package from jessie-backports, apt-get has to be told
to get it from there rather than the main archive.  It will not use
jessie-backports by default even if it is added as an apt source.

closes #224
2017-01-03 22:24:38 +01:00
Daniel Martí
fcba707c86 Merge branch 'import' into 'master'
import: fix raw git-over-https urls

See merge request !196
2017-01-01 23:25:30 +00:00
Daniel Martí
4c4ad2ebfc Merge branch 'spdx' into 'master'
docs: switch to spdx license list

See merge request !194
2017-01-01 23:24:37 +00:00
Daniel Martí
e1a1eada11 Merge branch 'ci-update' into 'master'
Bump CI image, now with build-tools 25.0.2

See merge request !197
2016-12-23 17:22:35 +00:00
Daniel Martí
926aafa58b Bump CI image, now with build-tools 25.0.2 2016-12-23 17:46:09 +01:00
Boris Kraut
e1dda886d7 import: fix raw git-over-https urls 2016-12-21 10:12:20 +01:00
Daniel Martí
87c326c401 Merge branch 'master' into 'master'
Makebs: add build tools 25.0.2

See merge request !195
2016-12-17 23:19:50 +00:00
est31
d6da0a283d Makebs: add build tools 25.0.2 2016-12-17 23:36:48 +01:00
Boris Kraut
fd51f7fab0 docs: switch to spdx license list 2016-12-17 11:00:40 +01:00
Hans-Christoph Steiner
7c8823e94e Merge branch 'feature/dscanner' into master
dscanner - drozer scanner work.

closes !187
2016-12-06 14:03:34 +01:00
Kevin C. Krinke
df27bae6a0 dscanner - Drozer based post-build dynamic vulnerability scanner command
* New command `dscanner`, enables one to scan signed APKs with Drozer
 * Drozer is a dynamic vulnerability scanner for Android
 * Drozer runs in a emulator or on-device, this new `dscanner` command...
  * starts a docker image with Drozer and the Android Emulator pre-installed,
  * loads the signed APK into the emulator
  * activates Drozer automated tests for the APK
  * gathers the report output and places it next to the original APK
 * The Drozer docker image can be:
  * cached locally for re-use (just don't run --clean*)
  * retrieved from dockerhub.com for more efficient runtime
  * or be built from scratch (in the new "./docker" directory)
 * New "Vulnerability Scanning" documentation section (run gendocs.sh)
2016-12-06 14:00:44 +01:00
Hans-Christoph Steiner
f439266303 Merge branch 'collection-fixes-and-KnownVuln' into 'master'
add KnownVuln anti-feature, and other fixes

This is a bit of a random collection of things that I have added in the process of working on the drozer/scanner support, YAML support, etc.  The only new things are:

* adding new AntiFeature for security issues: `KnownVuln`
* removing broken, incomplete XML metadata support

Everything else included are just code improvements.  This also includes the first check for `KnownVuln`, which is a scanner to check whether custom OpenSSL binaries in apps are not old with known vulnerabilities.  `KnownVuln` will then ultimately be used for things like the drozer scanner !187

See merge request !189
2016-12-06 12:37:11 +00:00
Hans-Christoph Steiner
e37be9b063 fix fdroid build with non-git repos
HEAD is really only in git.  This was introduced in
a4e4310803
2016-12-05 20:19:56 +01:00
Hans-Christoph Steiner
b1a943d562 convert metadata test dumps to YAML for easy comparison
When making code changes related to the metadata parsing, it is useful to
see how the internal format has changed by seeing the differences in the
dump files.  Those files are currently in the binary .pickle format.  This
just straight converts them to YAML, which is a text format, so that normal
diff tools work to see changes.

The dump files are named .yaml instead of .yml since .yml is used for hand-
edited YAML files for fdroiddata/metadata, while these dump files here are
a human readable form of a Python pickle.
2016-12-05 20:16:54 +01:00
Hans-Christoph Steiner
ce3efe4168 remove support for XML app metadata, its broken
JSON and YAML are very closely related, so supporting both of them is
basically almost no extra work.  Both are also closely related to how
Python works with dicts and pickles. XML is a very different beast, and its
not popular for this kind of thing anyway, so just purge it.
2016-12-05 20:16:54 +01:00
Hans-Christoph Steiner
b91cdffe17 App.get_last_build() method to replace duplicated code
Look @mvdan, I added a method to the App class!
2016-12-05 20:16:54 +01:00
Hans-Christoph Steiner
3db2c9869c do proper checking of versionCode value
versionCode is defined as a Java Integer, so any value between -2147483648
(Integer.MIN_VALUE) and 2147483647 (Integer.MIN_VALUE) is valid, including
0.

https://developer.android.com/guide/topics/manifest/manifest-element.html#vcode
2016-12-05 20:16:54 +01:00
Hans-Christoph Steiner
98297278bd check all APKs for old versions of OpenSSL
This scans all APKs for old versions of OpenSSL libraries that are known to
be vulnerable to issues, or fully unsupported.

This really should be implemented as a per-APK AntiFeature, so that it can
apply to any version that is vulnerable.  Since AntiFeatures are currently
only per-App, this instead sets the AntiFeature only if the latest APK is
vulnerable.

Google also enforces this:
https://support.google.com/faqs/answer/6376725?hl=en

apk['antiFeatures'] has the first letter small, since all build fields
start with a lowercase letter.  app.AntiFeatures has the first
uppercase since all App fields are that way.
2016-12-05 20:16:54 +01:00
Hans-Christoph Steiner
95656e6c61 check aapt version to make sure its new enough #236
Since `fdroid update` parses the output of aapt, and since aapt's output
format changes in non-compatible ways, test to make sure that the version
of aapt is new enough to prevent mystery stacktraces.  This only prints a
warning since in many cases, the old version will work just fine.
2016-12-05 20:16:12 +01:00
Ciaran Gultnieks
32182867ad Merge branch 'master' into 'master'
Add gradle 3.2.1

See merge request !192
2016-12-02 16:09:22 +00:00
est31
d39f54f04d Add gradle 3.2.1 2016-12-02 08:16:04 +01:00
Daniel Martí
6b2aa444d3 Merge branch 'buildtools-25.0.1' into 'master'
makebs: add buildtools 25.0.1

Please verify hash before merging.

See merge request !191
2016-12-01 12:40:25 +00:00
Boris Kraut
51083fdeb4 makebs: add buildtools 25.0.1 2016-12-01 08:44:47 +01:00
Hans-Christoph Steiner
a5cc4d74fc Merge branch 'master' into 'master'
Add ndk 13b

See merge request !190
2016-11-26 08:02:42 +00:00
est31
ddefec33cd Add ndk 13b 2016-11-25 21:18:37 +01:00
Hans-Christoph Steiner
d7ec321198 Merge branch 'fdroid-yml-builds' into 'master'
builds straight from source repo using .fdroid.yml

The overarching theme of the merge request is allowing _.fdroid.yml_ to be included in an app's source repo, then letting `fdroid build` build the app straight out of the git repo without requiring a setup like _fdroiddata_ (e.g. _config.py_, _metadata/packagename.txt_, etc.).  _fdroiddata_ repos can then include source repos with a _.fdroid.yml_ by having _metadata/packagename.txt_ that includes just:

```
Repo Type:git
Repo:https://gitlab.com/upstream/app.git
```

Any other metadata fields that are included in _metadata/packagename.txt_ will override what is in _.fdroid.yml_, giving the repo manager the final say about what is included in their repo.  This setup provides a number of benefits:

* CI systems like jenkins, travis, gitlab-ci can build from _.fdroid.yml_
* very easy to start building apps using `fdroid build`, no separate repo needed
* some maintenance can be offloaded to the upstream dev

See merge request !184
2016-11-24 14:04:09 +00:00