1
0
mirror of https://gitlab.com/fdroid/fdroidserver.git synced 2024-11-04 22:40:12 +01:00
Commit Graph

3845 Commits

Author SHA1 Message Date
Hans-Christoph Steiner
85632ba00e set up install/delete lists for "push" commands from server
It is now possible for the server operator to specify lists of apps that
must be installed or deleted on the client (aka "push installs).  If
the user has opted in, or the device is already setup to respond to
these requests, then fdroidclient will automatically install/delete
the packageNames listed.  This is protected by the same signing key
as the app index metadata.

It generates single XML elements with the data set in the attributes. This
keeps the XML compact and easily extensible, e.g. for adding versionCode,
signingKey, etc as attributes:

    <install packageName="com.fsck.k9"/>
    <install packageName="at.bitfire.davdroid"/>
    <delete packageName="com.facebook.orca"/>

Copyright: 2016 Blue Jay Wireless
Signed-off-by: Hans-Christoph Steiner <hans@eds.org>

closes #177
2016-08-16 21:02:15 +02:00
Daniel Martí
329e0247d5 Merge branch 'ndk11' into 'master'
Add ndk r11c

NDK11 is required by some apps (some versions of VLC) which will not build with r10e or r12b. As always, please test this before merging: I havent added a NDK before.

See merge request !155
2016-08-14 09:15:04 +00:00
Daniel Martí
76ecacc693 makebs: add automake
Version 1.14 is needed for k9 and some other apps.
2016-08-14 09:57:29 +02:00
Boris Kraut
815961acc5 Add ndk r11c 2016-08-14 00:59:31 +02:00
Daniel Martí
ec685342a4 Merge branch 'gradle-2.14.1' into 'master'
Add gradle 2.14.1

I hope verything is doen correctly, didnt add any gradle versions since moving from chef.. (or at all?)

See merge request !154
2016-08-12 20:36:35 +00:00
Boris Kraut
840cafd151 Add gradle 2.14.1 2016-08-12 20:25:38 +02:00
Hans-Christoph Steiner
abb4c5587f Merge branch 'java-home-lookup' into 'master'
Fix JDK lookup

Fix two cases when JDK path lookup could fail (found those issues on my Fedora 24 box).

See merge request !153
2016-08-09 15:44:06 +00:00
Hans-Christoph Steiner
911297272c Merge branch 'permission-parsing' into 'master'
Parse maxSdkVersion and -23 permission tags

Parse maxSdkVersion and uses-permission-sdk-23 tags and insert them into the index.

AndroidManifest:
```xml
    <uses-permission android:name="android.permission.READ_CALENDAR" />

    <uses-permission
        android:name="android.permission.WRITE_EXTERNAL_STORAGE"
        android:maxSdkVersion="18" />

    <uses-permission-sdk-23 android:name="android.permission.ACCESS_FINE_LOCATION" />
```

new index format:
```xml
<package>
            <version>1.0</version>
            <versioncode>1</versioncode>
            <apkname>maxmin.apk</apkname>
            <hash type="sha256">526eb6d643050c3fae42fb6c001c704006046db52e98998b21f7646ecae3dae5
            </hash>
            <sig>893fc1f22301c902d05c938b29d21648</sig>
            <size>8681</size>
            <sdkver>14</sdkver>
            <targetSdkVersion>24</targetSdkVersion>
            <added>2016-08-01</added>
            <permissions>READ_EXTERNAL_STORAGE,READ_CALENDAR,WRITE_EXTERNAL_STORAGE</permissions>
                <uses-permission name="android.permission.WRITE_EXTERNAL_STORAGE"
                    maxSdkVersion="18" />
                <uses-permission name="android.permission.READ_CALENDAR" />
                <uses-permission name="android.permission.READ_EXTERNAL_STORAGE" />
                <uses-permission-sdk-23 name="android.permission.ACCESS_FINE_LOCATION" />
</package>
```

old format:
```xml
<permissions>READ_CALENDAR,ACCESS_FINE_LOCATION,WRITE_EXTERNAL_STORAGE</permissions>
```

This generates both formats for backward compatibility with old clients.

Also increase version of index format.

This is a proposal for the issues discussed in https://gitlab.com/fdroid/fdroidclient/issues/704

@eighthave @mvdan 

See merge request !150
2016-08-09 15:33:19 +00:00
Dominik Schürmann
cb965dcb07 Parse maxSdkVersion and -23 permission tags
Parse maxSdkVersion and uses-permission-sdk-23
tags and insert them into the index, e.g.,
<package>
 <permissions>READ_EXTERNAL_STORAGE,READ_CALENDAR,WRITE_EXTERNAL_STORAGE</permissions>
  <uses-permission name="android.permission.WRITE_EXTERNAL_STORAGE"
   maxSdkVersion="18" />
  <uses-permission name="android.permission.READ_CALENDAR" />
  <uses-permission name="android.permission.READ_EXTERNAL_STORAGE" />
  <uses-permission-sdk-23 name="android.permission.ACCESS_FINE_LOCATION" />
</package>

Also increase version of index format
2016-08-09 10:14:57 +02:00
relan
162808a4cc Check javac existence when looking for JDK
Empty JDK directories can remain from previous JDK installations. For
example in RHEL/Fedora when RPM upgrades a package it can leave modified
files (usually configs) and, consequently, their directories. So we could
end up selecting a bad JDK path.
2016-08-08 07:25:26 +03:00
Ciaran Gultnieks
83951d0ed5 Merge branch 'fix-ndk' into 'master'
Re-add NDK r12b and fix build-tools 24.0.1



See merge request !152
2016-08-04 09:16:43 +00:00
relan
07660fdb13 Fix JAVA_HOME environment variable handling
Append JAVA_HOME to pathlist as a string, not as a list of characters.
2016-08-03 08:21:46 +03:00
Daniel Martí
4b5b1fca6f makebs: work around build-tools 24.0.1 version bug
The zip contains 24.0.0 as the version, which means our magic was
installing 24.0.0 twice and skipping 24.0.1.
2016-08-02 18:08:48 +02:00
Daniel Martí
662aa2c938 all: switch to build-tools 24.0.1 as default 2016-08-02 11:12:01 +02:00
Daniel Martí
059d216aff Re-add NDK r12b, now fixed. 2016-08-02 11:10:31 +02:00
Ciaran Gultnieks
bf153ec247 Revert "all: add NDK r12b and set it as default"
This reverts commit 82d09560c6.

It doesn't work - the setup scripts are expecting a ".bin" file (which
is apparently a 7z archive), but what's actually got is a ".zip".

Conflicts:
	buildserver/provision-android-ndk
2016-08-01 12:38:52 +01:00
Ciaran Gultnieks
e99454868a Catch another case of invalid wiki page names 2016-08-01 12:25:22 +01:00
Daniel Martí
5e6b986c40 Merge branch 'bump-ci' into 'master'
CI: bump docker image



See merge request !151
2016-08-01 06:07:26 +00:00
Daniel Martí
70777233d0 CI: bump docker image 2016-08-01 00:57:32 +02:00
Daniel Martí
690711e7f8 Merge branch 'pep' into 'master'
bs: add sqlite3 and asn1c

Ref: https://gitlab.com/fdroid/fdroiddata/merge_requests/1557

See merge request !149
2016-07-31 17:15:58 +00:00
Daniel Martí
e46debe12e common: extra checks for NDK in local.props
The previous commit fixed it in local setups, this fixes it in broken
buildservers too and makes it more reliable.
2016-07-28 08:26:51 +02:00
Daniel Martí
957f01f1d6 makebs: fix config crash
Without this, running makebuildserver from a clean master results in the
following:

	 $ ./makebuildserver
	Traceback (most recent call last):
	  File "./makebuildserver", line 74, in <module>
	    del(config['__builtins__'])  # added by compile/exec
	KeyError: '__builtins__'

This is because a clean checkout has no config, thus exec is never
actually ran.
2016-07-27 11:20:17 +02:00
Daniel Martí
7b36ffa2a6 wp-fdroid: add 7.0 (24) to the versions list 2016-07-25 14:56:54 +02:00
Daniel Martí
dee3e85098 common: don't insert $ANDROID_NDK to local.props
See the comment.

Updates #171.
2016-07-25 13:07:46 +02:00
Daniel Martí
11f30223d6 makebs: update android support repo 2016-07-25 12:45:56 +02:00
Daniel Martí
9a4af61a53 makebs: add build-tools-24.0.1
Fixes #196.
2016-07-25 12:40:44 +02:00
Daniel Martí
79a1f2919a makebs: "fix" platform v24 zip for r02
Google replaced the r01 zip instead of adding the r02 one. Oh well.
2016-07-25 12:40:17 +02:00
Boris Kraut
998c016925 bs: add sqlite3 and asn1c 2016-07-23 19:14:28 +02:00
Daniel Martí
8a8c139aa9 makebs: revert platform-tools change in 2227cc6d
Amusingly, the commit changed more than just this so a simple git revert
is not possible.

The problem is that the zip is replaced with each point release (24.0.1
at the time of writing) and there is no way to get a URL to a single,
non-changing version. Hence any caching or checksums are completely
worthless and will break every few weeks.
2016-07-19 20:38:09 +02:00
Hans-Christoph Steiner
9be1c75e58 Merge branch 'ndk-r12' into 'master'
NDK r12



See merge request !147
2016-07-18 14:50:51 +00:00
Hans-Christoph Steiner
ad966c5bd9 Merge branch 'ndk-error' into 'master'
build: error properly if an invalid ndk is used



See merge request !146
2016-07-18 14:47:45 +00:00
Daniel Martí
89eb7166fe Merge branch 'patch-2' into 'master'
Server manual phrasing matches client. "beta" -> "unstable"

Fixes #193 

See merge request !148
2016-07-13 20:48:33 +00:00
Stephen Michel
60dcc55273 Server manual phrasing matches client. "beta" -> "unstable" 2016-07-13 17:25:33 +00:00
Daniel Martí
d375318550 all: remove 32-bit buildserver code
We dropped support for 32-bit, so remove all the now unused code and
references.
2016-07-13 16:47:58 +01:00
Daniel Martí
82d09560c6 all: add NDK r12b and set it as default 2016-07-13 16:45:05 +01:00
Daniel Martí
a0e4245443 build: error properly if an invalid ndk is used
Currently, if buildjni is not used but ndk is set to an invalid value,
the build would start but $ANDROID_NDK would be empty. This is happening
in VLC, which results in very confusing errors.

If a build uses a ndk= value that is not set up, such as r11/r12 which
we do not have yet, it should error with "NDK version could not be
found". It does with this change.
2016-07-13 16:32:52 +01:00
Daniel Martí
0f708f367c Merge branch 'precommit-fixes' into 'master'
pre-commit script fixes



See merge request !145
2016-07-13 15:32:34 +00:00
Daniel Martí
b53e56916c pre-commit: pep8 is now pycodestyle
See https://github.com/PyCQA/pycodestyle/issues/466.
2016-07-13 14:42:23 +01:00
Ciaran Gultnieks
21404e8622 Merge branch 'use_date_from_apk_in_known_apks' into 'master'
Pass a date from APK to KnownApks.recordapk()

... if --use-date-from-apks option is used.

Essentially, it just expands influence of `--use-date-from-apks` option to `stats/known_apks.txt`.

See merge request !141
2016-07-13 11:43:33 +00:00
Daniel Martí
d8a8c24584 pre-commit: fix warnings and errors
Properly print warnings to stderr. Also, use : instead of 'echo' as a
fallback as the latter spits out garbage to stdout.

Examples without pep8 installed.

Before:

	 $ ./hooks/pre-commit
	./hooks/pre-commit: line 97: WARNING:: command not found
	ERROR: pep8 tests failed!

After:

	 $ ./hooks/pre-commit
	WARNING: pep8 is not installed, using dummy placeholder!
2016-07-13 12:15:22 +01:00
Daniel Martí
64d9eb3b13 Merge branch 'apk-extension-obb-support' into 'master'
support APK Extension OBB files

Google Play specifies OBB aka "APK Extension" files for apps that need more than 100 MBs, which is the Play APK size limit.  They also provide a mechanism to deliver large data blobs that do not need to be part of the APK.  For example, a game's assets do not need to change often, so they can be shipped as an OBB, then APK updates do not need to include all those assets for each update.
    
https://developer.android.com/google/play/expansion-files.html

See merge request !143
2016-07-13 11:01:42 +00:00
Hans-Christoph Steiner
5f8beaa2c6 tests for finding the proper OBB files for each APK 2016-07-07 22:34:41 +02:00
Hans-Christoph Steiner
2201df150f add SHA-256 hashes of each OBB file 2016-07-07 22:34:41 +02:00
Hans-Christoph Steiner
5074be224a support .obb files for Opaque Binary Blog APK Expansions
OBB files provide a mechanism to deliver large data blobs that do not need
to be part of the APK.  For example, a game's assets do not need to change
often, so they can be shipped as an OBB, then APK updates do not need to
include all those assets for each update.

https://developer.android.com/google/play/expansion-files.html
2016-07-07 22:34:41 +02:00
Ciaran Gultnieks
0c5725b5a9 Merge branch 'standalone-vagrantfile' into 'master'
Standalone Vagrantfile

@mvdan @CiaranG as a follow up on our work moving to a 64-bit build server VM, this moves the buildserver config to a standalone YAML file and commits a static _Vagrantfile_ to git.  This makes it a lot easier to work with, especially for people who normally use git.  The buildserver config is already a Python _dict_, and its trivial to export a _dict_ to a YAML file.  _Vagrantfile_ is a Ruby script, where its trivial to load a YAML file.  This moves some of the logic into _Vagrantfile_ itself.  This means that someone can mess with _Yagrantfile.yaml_ and/or _Vagrantfile_ to work on the buildserver setup, without having to run `./makebuildserver`.  Then once something is working, it can be ported to the current `./makebuildserver` setup that generates _Vagrantfile.yaml_.

This is important for working on getting this whole thing running in a KVM instance like jenkins.debian.net and elsewhere.  From what I read, VirtualBox in KVM is only possible if VirtualBox is running in 32-bit mode, so that's a dead end for us.  We need to be able to run the buildserver as KVM in KVM #190.  This merge request doesn't get us there yet, but it makes the process a lot easier.

This also moves everything but Kivy to provisioning shell scripts, since the existing chef scripts were really just shell scripts wrapped in Chef wrapped in Vagrant wrapped in `./makebuildserver`.

This passes the gpjenkins CI build that creates the buildserver from scratch, then builds F-Droid and AdAway:
http://qssio5fppcrojdh3.onion:8080/job/fdroidserver-makebuildserver-eighthave/602/

I tried adding Amaze and Retrolambda as test apps for the buildserver too, we'll see how that goes:
http://qssio5fppcrojdh3.onion:8080/job/fdroidserver-makebuildserver-eighthave

See merge request !144
2016-07-07 15:37:26 +00:00
Hans-Christoph Steiner
8b53ae0ad0 buildserver: delete corrupt files from the cache
In order to make CI and other automation easier, delete any corrupt files
from the cache if they fail the SHA-256 check.

https://jenkins.debian.net/view/reproducible/job/reproducible_setup_fdroid_build_environment_profitbricks3/112/console
2016-07-06 22:16:55 +02:00
Hans-Christoph Steiner
d4c6fffb30 buildserver: buildserver/Vagrantfile is configed by .yaml file
Vagrantfile is now committed and not changed between configurations. It is
configed by translating the python config file's dict to a YAML file, which
Vagrantfile now loads and uses.  This makes it a lot easier for vagrant
users and python programmers to understand, and hopefully makes it easier
to maintain and test with.
2016-07-04 23:54:52 +02:00
Hans-Christoph Steiner
2e1ec71404 buildserver: send config to vagrant via YAML file
Python can easily output dicts as YAML, and a Vagrantfile is a ruby script,
which can easily read YAML.  Going this route means that Vagrantfile can
ultimately be committed to git, and the configuration will happen all via
Python dicts output as YAML.  That makes it drastically easier to follow
the code, and to make modifications.
2016-07-04 23:54:52 +02:00
Hans-Christoph Steiner
4e787cc750 buildserver: make provision scripts output name to log 2016-07-04 23:54:52 +02:00
Hans-Christoph Steiner
0171dd60fd buildserver: use pip instead of easy_install for caching
easy_install does not provide any download caching, while pip does. This
also moves the python module installing a shell script that takes python
packages as args.  That will allow for future uses like allowing app
metadata to include pip modules that they need.
2016-07-04 23:54:52 +02:00