This replaces the current default behavior of always forcing the
build_tools version and allows the user to set build-tools forcing in
config.py.
closes#147
serverwebroot has long supported uploading to multiple servers, this bit of
metadata communicates those official mirrors to the client so that it can
automatically do something useful with that information.
closes#14https://gitlab.com/fdroid/fdroidserver/issues/14
4 gigs is still a common amount of RAM these days for laptops, if the VM
takes almost all of that, it makes the machine drag to almost a halt. Most
apps build fine in 1gig of RAM, indeed that's the default for most CI
instances, like travis-ci and gitlab-ci.
On slow machines or VMs like the Debian jenkins box, the VM boot timeout
needs to be a lot longer, otherwise vagrant times out before setting up
the VM.
This also provides a config option to override that default. ~/.cache is
a standard location on GNU/Linux machines for cached content. It is also
good to have the cache outside of the git repo in case `git clean -fdx` is
run, which would delete all files in the directory that are not part of the
git repo, including buildserver/cache/
This makes it so that ./makebuildserver will run without any config file,
using the defaults that are embedded in the script itself. This is like
how `fdroid` works.
This host automatically detects which is the closest mirror, then uses that
one. It does so dynamically, so it'll work on machines that move too. Now
that we are pushing more people to run F-Droid build servers, the defaults
should take those use cases into account.
This keeps the numbers of names down to a minimum, and since the config
is placed right next to the script, this keeps tab completion working
nicely when the config file is in place.
The old file name is still supported.
For a bit repo like f-droid.org, it makes sense to standardize on a single
format for metadata files. This adds support for enforcing a single data
format, or a reduced set of data formats. So f-droid.org would run like
this if it changed to YAML:
accepted_formats = ['txt', 'yaml']
Then once everything was converted to YAML, it could look like this:
accepted_formats = ['yaml']
config option to enable per-app repos for nightly builds
For Guardian Project, we've been running an fdroid repo for the nightly builds for each of our apps: https://dev.guardianproject.info/debug This is built using a big, hacked up shell script: [update-debug-fdroid-repo](https://github.com/guardianproject/fdroid-repo-tools/blob/master/update-debug-fdroid-repo). It has proven very useful to us to be able to subscribe to the nightly build for a single app, so this the first step of porting that horrid shell script to `fdroidserver`.
This also helps make the fdroidserver tool suite the single set of tools for all types of builds and releases. That will hopefully drive more free software developers to make f-droid.org the core channel for official releases.
See merge request !66
config.vm.box_url can be a list/tuple of URLs, which is useful to specific
a locally cached copy. This is needed on slow connections, so that if it
fails, the download of jessie32.box does not have to start from the
beginning of the file again.
For devs that want to build and distribute nightly builds of their apps
using the fdroid tools. The core idea here is to make the fdroidserver
tool suite the single set of tools for all types of builds and releases.
That will hopefully drive more free software developers to make f-droid.org
an core channel for official releases.
Update documentation re: testing32 image
Use different subsections for the Debian base box setup (which can be bypassed using the prebuilt image) and the F-Droid / SDK setup (which cannot be bypassed).
Side note: would it be possible to offer a torrent for testing32.box? The direct download is taking over 2 hours on a 50Mbps link.
See merge request !32
By not setting sdk_path when /usr/bin/aapt is found, sdk_path then defaults
to $ANDROID_HOME when its used. Since in this case, aapt will be used from
the system path, using aapt entirely ignores sdk_path. If the user runs
`fdroid build` in this setup, sdk_path will be $ANDROID_HOME, so it should
check the env vars for it, but maybe that doesn't actually work like that
yet.
I wrote up the feature to automatically generate symlinks with a constant name
that points to the current release version. I have it on by default, with a
*config.py* option to turn it off. There is also an option to set where the
symlink name comes from which defaults to app['Name'] i.e. F-Droid.apk, but
can easily be set to app['id'], i.e. _org.fdroid.fdroid.apk_. I think the best
place for the symlinks is in the root of the repo, so like
https://f-droid.org/F-Droid.apk or https://guardianproject.info/fdroid/ChatSecure.apk
For the case of the current FDroid static link https://f-droid.org/FDroid.apk
it can just be a symlink to the generated one (https://f-droid.org/F-Droid.apk
or https://f-droid.org/org.fdroid.fdroid.apk). Right now, this feature is all
or nothing, meaning it generates symlinks for all apps in the repo, or none. I
can’t think of any problems that this might cause since its only symlinks, so
the amount of disk space is tiny. Also, I think it would be useful for having
an easy “Download this app” button on each app’s page on the “Browse” view. As
long as this button is less prominent than the “Download F-Droid” button, and
it is clear that it is better to use the FDroid app than doing direct
downloads. For the f-droid.org repo, the symlinks should probably be based on
app['id'] to prevent name conflicts.
more info here:
https://f-droid.org/forums/topic/static-urls-to-current-version-of-each-app/
This gives us flexibility in how the blocks of text can be formatted in
config.py, but also provides a more useful format for displaying since the
client can decide where to wrap the text.
To support a fully offline build/signing machine, there is the "local copy
dir". The repo is generated on the offline machine and then copied to a
local dir where a thumb drive or SD Card is mounted. Then on the online
machine, using `fdroid server update --sync-from-local-copy-dir` allows
the whole server update process to happen in a single command:
0. read config.py on online machine's repo
1. rsync from the local_copy_dir to the current dir
2. copy to serverwebroot, awsbucket, etc.
This allows a dir to be specified in config.py that `fdroid server update`
will automatically rsync the repo to. The idea is that the path would
point to an SD card on a fully offline machine that serves as the secure
repo signing machine.
This allows the SSH key used to sync with the server to be specified via
the config.py or the command line. I need it for running automated tests
and setups.
apache-libcloud enables uploading to basically any cloud storage service.
This is the first implementation that allows `fdroid server` to push a repo
up to a AWS S3 'bucket'. Supporting other cloud storage services should
mostly be a matter of finding the libcloud "Provider" and setting the
access creditials.
fixes#3137https://dev.guardianproject.info/issues/3137
Having serverwebroot optional in `fdroid server` means that it can support
multiple methods of hosting, like cloud storage services. `fdroid server`
can also then support multiple repo hosting options at the same time.
This assumes that the smartcard is already setup with a signing key. init
does not generate a key on the smartcard, and skips genkey() if things are
configured to use a smartcard.
This also does not touch APK signing because that is a much more elaborate
question, since each app is signed by its own key.
These options are needed to configure Java's keytool and jarsigner to use
a Hardware Security Module aka HSM aka smartcard. The defaults provided
are meant to make things work as easily as possible.
This also makes the file layout in git basically the same as the installed
file layout, using an examples/ dir. I'm not sure if config.buildserver.py
is an example conf file, or a conf file that is actually in use, so I did
not move it.
