1
0
mirror of https://gitlab.com/fdroid/fdroidserver.git synced 2024-11-14 02:50:12 +01:00
Commit Graph

49 Commits

Author SHA1 Message Date
Licaon_Kter
f30dcf5069
Upgrade Buildserver VM to latest Debian (Bookworm) 2024-03-14 13:14:34 +01:00
Hans-Christoph Steiner
c1342ab9d6 stop pre-installing the ndk 2022-11-15 13:25:19 +00:00
Hans-Christoph Steiner
d6008cdb2a
convert examples/makebuildserver.config.py to Vagrantfile.yaml
Also, tests/androguard_test.py was removed long ago
2022-11-03 13:27:18 +01:00
Hans-Christoph Steiner
abf535aabe
buildserver: move config to buildserver/Vagrantfile.yaml 2022-11-03 13:27:16 +01:00
Hans-Christoph Steiner
e2fcd633fc
buildserver: hard code basebox name and version
This is not user-configurable, so it should not be setup to be.  This
process is only tested on the one basebox, and devs can just edit
Vagrantfile directly to test other base boxes.

# Conflicts:
#	makebuildserver
2022-11-03 13:27:15 +01:00
Hans-Christoph Steiner
21ea1c1c89
makebuildserver: purge apt_package_cache feature
This is unmaintained, lightly used, a tangled mess, and can be replaced by
things like the vagrant-cachier plugin or #418
2022-11-03 13:27:11 +01:00
Hans-Christoph Steiner
1dd480405e
buildserver: let vagrant package handle insecure private key
This script was overwriting the provisioning that `vagrant package` does
already, and breaking `vagrant ssh`.  It should have been removed in !1099

closes #990
    fdroid-bootstrap-buildserver#12
    !1012
    !1099
c6f5956537
2022-10-20 17:42:53 +02:00
Hans-Christoph Steiner
9089750277 buildserver: show informative message if Vagrant is not configed 2022-10-13 13:56:56 +00:00
Hans-Christoph Steiner
c6f5956537 install Vagrant insecure_private_key into buildserver box
Vagrant uses the "insecure private key" to establish the first SSH
connection to a new VM based on a clean public box.  In theory, the
`vagrant package` command should do that automatically.  This process
is still using custom code instead of `vagrant package`, hence this
script.

This public key can be generated on any Vagrant install using:
  ssh-keygen -y -f  ~/.vagrant.d/insecure_private_key

https://www.vagrantup.com/docs/vagrantfile/ssh_settings#config-ssh-private_key_path
fdroid-bootstrap-buildserver#12
!1012
!1099
2022-10-10 21:22:59 +02:00
linsui
5472cf88b8 Update preinstalled NDK to r23c 2022-06-14 11:03:42 +00:00
Licaon_Kter
e678df14ce
buildserver: remove old LTS NDK and only pre-install current LTS 2022-04-26 10:25:31 +02:00
Hans-Christoph Steiner
c6fddebecd buildserver: disable flutter dev/build analytics globally
The Flutter dev tools includes opt-out analytics, with a config option to
opt out.  We can at least try to disable them in the buildserver. That
config option is unfortunately not even guaranteed to work:
https://github.com/flutter/flutter/issues/19304
https://github.com/flutter/flutter/issues/45369

issuebot#21
issuebot!49
https://github.com/flutter/flutter/issues/73657
2022-04-21 15:38:48 +02:00
linsui
3f62e0ebde update preinstalled ndk to r23b 2022-02-10 16:47:03 +08:00
Hans-Christoph Steiner
5d09e70330
buildserver: give names to all provisioners 2022-01-13 21:22:23 +01:00
Hans-Christoph Steiner
57447f18e1
buildserver: move buildserverid to provisioner
In order to support Docker, this should be able to operate without ssh,
e.g. using vagrant-communicator-docker.  This removes the buildserverid
hack and makes it a provisioner shell script.
2022-01-11 11:23:36 +01:00
Hans-Christoph Steiner
28a4f9481a
provision-android-ndk: do all configuration via args
This makes it so it is easy to reuse this script in other contexts, like
in Docker, GitLab CI, etc.
2021-06-01 14:33:41 +02:00
Hans-Christoph Steiner
93145a43fb buildserver: use standardized paths for the NDKs
This should help with reproducibility since some tools like Python still
include the build paths in the binaries.
https://gitlab.com/fdroid/fdroidserver/-/merge_requests/919#note_578180986

The default ANDROID_SDK_ROOT base dir of /opt/android-sdk is hard-coded in
buildserver/Vagrantfile.  The $ANDROID_HOME/ndk subdir is where Android
Studio will install the NDK into versioned subdirs.
https://developer.android.com/studio/projects/configure-agp-ndk#agp_version_41
2021-05-26 09:21:52 +02:00
Hans-Christoph Steiner
bb77d7a6d2 buildserver: standardize SDK install location /opt/android-sdk
_/opt/android-sdk_ was chosen for a number of reasons:
* _/opt_ is [standardized][1] for packages like the Android SDK, which has its own directory layout.
* _android-sdk_ is used rather than the upstream directory name from the ZIP (e.g. _android-sdk-linux_) so that the path is the same on all platforms.
* On platforms without official _/opt_ support ([macOS][2], [Windows][3], [FreeBSD][4], etc.), it does not conflict with any existing system directory.

[1]: https://refspecs.linuxfoundation.org/FHS_3.0/fhs/ch03s13.html
[2]: https://developer.apple.com/library/archive/documentation/FileManagement/Conceptual/FileSystemProgrammingGuide/FileSystemOverview/FileSystemOverview.html#//apple_ref/doc/uid/TP40010672-CH2-SW7
[3]: https://en.wikipedia.org/wiki/Directory_structure#Windows_10
[4]: https://www.freebsd.org/cgi/man.cgi?query=hier&sektion=7&format=html
2021-05-25 17:30:32 +02:00
Hans-Christoph Steiner
ee84eb762c buildserver: SharedFoldersEnableSymlinksCreate: false 2021-03-23 11:01:40 +01:00
Marcus Hoffmann
e1c547cfdf Revert "Revert "Merge branch 'random-fixes' into 'master'""
This reverts commit f6f2fb0b89.

Only one of the included commit should have been reverted.
2019-01-10 14:48:29 +01:00
Marcus
f6f2fb0b89 Revert "Merge branch 'random-fixes' into 'master'"
This reverts merge request !605
2019-01-10 13:46:49 +00:00
Hans-Christoph Steiner
4c5864c975 buildserver: add config option to use 'nfs' instead of '9p'
'9p' is not possible when running KVM in VMware.  Also, 'nfs' might provide
more reliably operation on systems that are properly setup for it.
2019-01-03 21:00:01 +01:00
Marcus Hoffmann
e3a25ad3f2 buildserver: remove provision-pip
This was installing one python(2!) module. pip is installed in
provision-apt-get-install now.
2018-11-27 14:48:20 +01:00
Hans-Christoph Steiner
92bf9315e6 Merge branch 'verifiy-basebox' into 'master'
* verifiy-basebox:
  jenkins: switch to using auto-downloaded and verified basebox
  makebuildserver: verify stretch basebox

fdroid/fdroidserver!581
2018-10-17 17:04:16 +02:00
Hans-Christoph Steiner
3b0c8fe669 buildserver: only set disk_bus/nic_model_type if set in config file
This makes vagrant/KVM just use the default if these are not set in the
makebuildserver.config.py.

fdroid/fdroidserver!585
2018-10-11 12:11:15 +02:00
Michael Pöhn
d98d90ab90 makebuildserver: add config options for disk and network hardware emulation 2018-10-11 11:30:46 +02:00
Michael Pöhn
493a767d14 makebuildserver: verify stretch basebox 2018-10-11 09:56:46 +02:00
Marcus Hoffmann
9889a98dea
build: improve gradle experience
This expands the gradle wrapper shell script used by the buildserver for
usage outside the buildserver environment. It also allows downloading
whitelisted versions of gradle if they are not yet deployed to the
buildserver by simply upsating the copy of fdroidserver (in contrast to
having to reprovision the whole buildserver).

We first move the buildserver/gradle shell script to the repo root
as gradlew-fdroid, as it's an fdroid specific gradle wrapper.
We also now sync it inside the build VM before each build.

We then add a list of whitelisted gradle distributions taken from the
makebuildserver script.

The script additionally now reads two env vars which tell it where to
expect installed versions of gradle and where it might store downloaded
gradle .zip files. Both of those are configurable from config.py. As the
first should normally just be a subdir of the second it's not exposed in
the example config.py but only used by the buildserver config.py.

Default config now uses this internal gradle wrapper but a path to a
custom wrapper or specific gradle distribution can still be set from
config.py.

Closes fdroid/fdroidserver#98
Ref: fdroid/fdroidserver#370
2018-07-11 11:49:46 +02:00
Hans-Christoph Steiner
62ddab7edd buildserver: remove Qt installer, its huge, outdated, and being replaced
The currently included Qt has known security issues and is outdated.  This
can now be replaced by downloading and installing the Qt installer using
the sudo= build field.  @relan's provisioner system will also replace this
once that's done.  There are only two apps that currently use the Qt stuff:

* csd.qtproject.minesweeper
* org.openorienteering.mapper
2018-01-23 20:28:26 +01:00
relan
14f95f7750 buildserver: remove special sshd setup for Ubuntu 14.04
Now that fdroidserver requires Python 3, there's no need to tune sshd for
an old version of Paramiko because Ubuntu 14.04 does not have
python3-paramiko package in its repositories:

    https://packages.ubuntu.com/search?keywords=python3-paramiko

We can safely assume that Ubuntu 14.04 users have installed a more recent
version of Paramiko from PIP.
2017-11-17 21:15:18 +03:00
Willem Mulder
60bb34ef1a Use Qemu instead of KVM when we don't have VMX/SVM 2017-06-02 11:37:15 +02:00
Michael Pöhn
a8420817cb fix calling vagrant global-status 2017-05-22 17:40:19 +02:00
Hans-Christoph Steiner
764b99acd6 buildserver: only auto-detect KVM in ./makebuildserver
Having a second, different KVM auto-detect routine in Vagrantfile will only
confuse things.  This also removes the direct call to the systemd utility.
2016-09-28 10:16:30 +02:00
est31
91c5fb567a Add qt sdk support 2016-09-27 14:36:12 +02:00
Hans-Christoph Steiner
0a0ba6de84 buildserver: ensure dirs to mount cache exist in guest VM instance
It seems that the 9p synced folder setup is not as flexible and easy as the
VirtualBox one, so we have to do more little things like this.
2016-09-19 16:33:12 +02:00
Hans-Christoph Steiner
0ae2df6646 buildserver: support optionally using qemu+kvm
This makes it possible to run the full build process in a KVM virtual
machine, like jenkins.debian.net.

closes #190
2016-09-19 16:33:12 +02:00
Hans-Christoph Steiner
14de399bb7 buildserver: remove Kivy, unused since 2013 and out of date
This is the last thing using Chef, which adds a lot of time to the time it
takes to fully provision the buildserver.  This slows down development on
the things we are actually using, like running all builds on
jenkins.debian.net.

#210 #165
2016-09-12 14:58:08 +02:00
Hans-Christoph Steiner
b717271c61 buildserver: force a known-good version of chef
This forces the release channel and version of chef-solo to install on the
guest VM.  I was getting really massive, odd stacktraces without specifying
this, and chef is only used for Kivy now anyway.
2016-09-12 14:58:08 +02:00
Hans-Christoph Steiner
dc2f53b48c buildserver: vagrant-cachier conflicts with custom apt cache
The technique where /var/cache/apt is mounted as a shared folder conflicts
with vagrant-cachier's workings.  Therefore, ignore vagrant-cachier if the
user selects ./makebuildserver's custom apt cache.  The shared folder way
has the advantage for CI builds of storing the cache outside of
VAGRANT_HOME, which is set to be in the git project.  That gets wiped by
`git clean -fdx` on each CI build.
2016-09-12 14:58:08 +02:00
Hans-Christoph Steiner
d4c6fffb30 buildserver: buildserver/Vagrantfile is configed by .yaml file
Vagrantfile is now committed and not changed between configurations. It is
configed by translating the python config file's dict to a YAML file, which
Vagrantfile now loads and uses.  This makes it a lot easier for vagrant
users and python programmers to understand, and hopefully makes it easier
to maintain and test with.
2016-07-04 23:54:52 +02:00
Ciaran Gultnieks
cb16a63409 Much better build server provisioning
Including selection of correct ndk architecture, caching and performance
improvements, better configurability, etc.
2013-05-31 07:48:39 +01:00
Ciaran Gultnieks
f79debb132 More buildserver memory
Only needed for 64 bit really, 32 bit is fine.
2013-05-25 13:59:35 +01:00
Ciaran Gultnieks
03dcc07edb Install packages first 2013-05-25 13:56:00 +01:00
Ciaran Gultnieks
f33fb22360 Switch buildserver guest os 2013-05-23 11:05:29 +01:00
Ciaran Gultnieks
af391c7679 Be explicit about jarsigner algorithms
If these aren't specified, it works fine with the jdk6 version, but
produces an unusable (on Android) index.jar with the jdk7 version.
2013-04-11 17:30:51 +01:00
Ciaran Gultnieks
17c9e2f4e7 Increase RAM on build server 2013-01-21 11:09:36 +00:00
Ciaran Gultnieks
d9f404ba07 Integrated latest build server stuff (except snapshots) 2012-08-31 14:48:50 +01:00
Ciaran Gultnieks
b79ba3c630 Added Tags update check method 2012-08-23 14:25:39 +01:00
Ciaran Gultnieks
498e7d3c5f Work in progress on integrating build server 2012-02-03 16:01:35 +00:00