kamp/fdroidserver
1
0
Fork 0
mirror of https://gitlab.com/fdroid/fdroidserver.git synced 2024-11-19 21:30:10 +01:00
Code Activity
fc4f5a79a7
fdroidserver/makebuildserver

613 lines
32 KiB
Plaintext
Raw Normal View History

Switch all headers to python3
2016-01-04 16:33:20 +01:00
#!/usr/bin/env python3
Better build server creation script
2012-09-24 15:04:58 +02:00
import os
buildserver: use py3's pathlib to generate cache file URL This should handle edge cases better, like odd characters in the path, etc.
2016-09-26 13:17:31 +02:00
import pathlib
buildserver: add copy_caches_from_host config option For people using slow, expensive, and/or flaky internet, liberal use of caching can make a huge difference. The restricted environment of the gpjenkins box has been a good test environment for this (Tor-only, whitelist of allowed IPs to visit, home internet connection).
2016-06-23 17:12:06 +02:00
import re
makebuildserver: use original names for downloaded SDK components It will make it a lot easier to manage the cache if we use the original file names, which often include the file version. This also changes the download process to be resumable if there is a partial file in the cache, and switches from calling wget on the command line to using the python libs 'requests' and 'clint' to provide a similar experience. While its not so important for this particular bit of code to use those libraries, I think those two will allow us to provide a better user experience throughout the whole of fdroidserver. In this case, it is already doing special tricks fetching the file size from the server before trying to download it. I suppose this code could instead check if the file exists, and if so, check the hash sum. I think that would be slower for most people since checking the hash on large files takes a noticeable about of time, while a HTTP HEAD request is pretty tiny.
2016-06-10 11:42:42 +02:00
import requests
buildserver: make sure cachedir is accessible to VM instance It seems that KVM/QEMU's 9p sharing is more sensitive to file perms. If ~/.cache is 0700, then ~/.cache/fdroidserver cannot be mounted in the guest
2016-09-06 14:22:29 +02:00
import stat
Better build server creation script
2012-09-24 15:04:58 +02:00
import sys
makebuildserver deal with apt cache lock
2017-04-09 00:57:59 +02:00
import shutil
Better build server creation script
2012-09-24 15:04:58 +02:00
import subprocess
buildserver: replace custom code with python-vagrant I ran into some annoying issues with UTF-8 output in the vagrant logs, and it was hard to solve. So I switched to using python-vagrant, which handles it all for us. Its been around since 2012, has a number of contributors, and is still actively maintained, so it seems like a good bet. I also packaged it for Debian, including a backport in jessie-backports. On Debian/jessie, do `apt-get install python3-vagrant/jessie-backports`
2016-09-27 08:49:32 +02:00
import vagrant
Check sha256 sums via hashlib
2014-01-10 19:54:53 +01:00
import hashlib
buildserver: send config to vagrant via YAML file Python can easily output dicts as YAML, and a Vagrantfile is a ruby script, which can easily read YAML. Going this route means that Vagrantfile can ultimately be committed to git, and the configuration will happen all via Python dicts output as YAML. That makes it drastically easier to follow the code, and to make modifications.
2016-07-04 20:22:00 +02:00
import yaml
calculate correct size for buildserver-box in makebuildserver https://gitlab.com/fdroid/fdroidserver/issues/238#note_24000153 "Our hard-coded image size meta-data (1000) is for some interpreted as less than the size of the box-image by my kvm setup. This makes grub/initrd refuse to boot. So I've changed the metadata size to 9999 which resulted in an actually booting vm. I can log in on the builder-vm via virt-manager and virsh.
2017-05-22 16:57:47 +02:00
import json
makebuildserver debugging log details
2017-03-24 20:04:50 +01:00
import logging
makebuildserver: use original names for downloaded SDK components It will make it a lot easier to manage the cache if we use the original file names, which often include the file version. This also changes the download process to be resumable if there is a partial file in the cache, and switches from calling wget on the command line to using the python libs 'requests' and 'clint' to provide a similar experience. While its not so important for this particular bit of code to use those libraries, I think those two will allow us to provide a better user experience throughout the whole of fdroidserver. In this case, it is already doing special tricks fetching the file size from the server before trying to download it. I suppose this code could instead check if the file exists, and if so, check the hash sum. I think that would be slower for most people since checking the hash on large files takes a noticeable about of time, while a HTTP HEAD request is pretty tiny.
2016-06-10 11:42:42 +02:00
from clint.textui import progress
Add some makebuildserver options
2013-10-20 22:16:42 +02:00
from optparse import OptionParser
buildserver: display verbose logging in a background tail
2016-06-20 11:33:46 +02:00
import fdroidserver.tail
overhauled and moved destroying builder vm to vmtools.py
2017-03-26 01:41:39 +01:00
import fdroidserver.vmtools
Much better build server provisioning Including selection of correct ndk architecture, caching and performance improvements, better configurability, etc.
2013-05-31 08:48:39 +02:00
A few more PEP8 fixes
2014-05-07 16:13:22 +02:00
Add some makebuildserver options
2013-10-20 22:16:42 +02:00
parser = OptionParser()
makebuildserver debugging log details
2017-03-24 20:04:50 +01:00
parser.add_option('-v', '--verbose', action="count", dest='verbosity', default=1,
A few more PEP8 fixes
2014-05-07 16:13:22 +02:00
help="Spew out even more information than normal")
makebuildserver debugging log details
2017-03-24 20:04:50 +01:00
parser.add_option('-q', action='store_const', const=0, dest='verbosity')
Add some makebuildserver options
2013-10-20 22:16:42 +02:00
parser.add_option("-c", "--clean", action="store_true", default=False,
A few more PEP8 fixes
2014-05-07 16:13:22 +02:00
help="Build from scratch, rather than attempting to update the existing server")
makebuildserver debugging log details
2017-03-24 20:04:50 +01:00
parser.add_option('--skip-cache-update', action="store_true", default=False,
help="""Skip downloading and checking cache."""
"""This assumes that the cache is already downloaded completely.""")
added makebuildserver option for keeping vagrant box This is very useful for debugging this process, and also for people who might want to keep a working copy of the box.
2017-05-22 17:29:12 +02:00
parser.add_option('--keep-box-file', action="store_true", default=False,
leave VirtualBox `vagrant package` as it was originally We only need Vagrantfile hacks for KVM.
2017-05-22 15:50:32 +02:00
help="""Box file will not be deleted after adding it to box storage"""
""" (KVM-only).""")
Add some makebuildserver options
2013-10-20 22:16:42 +02:00
options, args = parser.parse_args()
overhauled and moved destroying builder vm to vmtools.py
2017-03-26 01:41:39 +01:00
logger = logging.getLogger('fdroidserver-makebuildserver')
makebuildserver debugging log details
2017-03-24 20:04:50 +01:00
if options.verbosity >= 2:
logging.basicConfig(format='%(message)s', level=logging.DEBUG)
logger.setLevel(logging.DEBUG)
elif options.verbosity == 1:
logging.basicConfig(format='%(message)s', level=logging.INFO)
logger.setLevel(logging.INFO)
elif options.verbosity <= 0:
logging.basicConfig(format='%(message)s', level=logging.WARNING)
logger.setLevel(logging.WARNING)
if not os.path.exists('makebuildserver') and not os.path.exists('buildserver'):
logger.critical('This must be run as ./makebuildserver in fdroidserver.git!')
sys.exit(1)
tail = None
makebuildserver: set defaults in script so it can run without config This makes it so that ./makebuildserver will run without any config file, using the defaults that are embedded in the script itself. This is like how `fdroid` works.
2016-01-20 21:00:16 +01:00
# set up default config
makebuildserver: use jessie32.box from cache, if present
2016-01-22 13:33:14 +01:00
cachedir = os.path.join(os.getenv('HOME'), '.cache', 'fdroidserver')
makebuildserver debugging log details
2017-03-24 20:04:50 +01:00
logger.debug('cachedir set to: %s', cachedir)
makebuildserver: set defaults in script so it can run without config This makes it so that ./makebuildserver will run without any config file, using the defaults that are embedded in the script itself. This is like how `fdroid` works.
2016-01-20 21:00:16 +01:00
config = {
switch to new 64-bit base box image: https://f-droid.org/jessie64.box build-tools and other SDK bits from android-24 require 64-bit GNU/Linux
2016-07-04 12:34:00 +02:00
'basebox': 'jessie64',
makebuildserver: use jessie32.box from cache, if present
2016-01-22 13:33:14 +01:00
'baseboxurl': [
makebuildserver: remove reference to f-droid.org/jessie64.box Also improve error message when the initial vagrant up fails, this is most likely due to the missing jessie64.box file.
2017-10-20 04:38:39 +02:00
pathlib.Path(os.path.join(cachedir, 'jessie64.box')).as_uri()
makebuildserver: use jessie32.box from cache, if present
2016-01-22 13:33:14 +01:00
],
makebuildserver: explicitly set debian_mirror by default There needs to be a default, otherwise it crashes. I should have included this in 8d66742dd7feec77aa30382bfb38da83f2b3e5a4
2016-01-26 13:33:53 +01:00
'debian_mirror': 'http://http.debian.net/debian/',
allow setting up an apt package cache for the build server setup This creates a cache folder which will store the apt cache from the VM.
2016-01-26 11:21:21 +01:00
'apt_package_cache': False,
buildserver: add copy_caches_from_host config option For people using slow, expensive, and/or flaky internet, liberal use of caching can make a huge difference. The restricted environment of the gpjenkins box has been a good test environment for this (Tor-only, whitelist of allowed IPs to visit, home internet connection).
2016-06-23 17:12:06 +02:00
'copy_caches_from_host': False,
makebuildserver: settable vm boot timeout, plus longer default On slow machines or VMs like the Debian jenkins box, the VM boot timeout needs to be a lot longer, otherwise vagrant times out before setting up the VM.
2016-01-26 17:35:25 +01:00
'boot_timeout': 600,
makebuildserver: use jessie32.box from cache, if present
2016-01-22 13:33:14 +01:00
'cachedir': cachedir,
makebuildserver: set defaults in script so it can run without config This makes it so that ./makebuildserver will run without any config file, using the defaults that are embedded in the script itself. This is like how `fdroid` works.
2016-01-20 21:00:16 +01:00
'cpus': 1,
makebuildserver: change mem default 2 GB 1 GB is not enough for a modern android toolchain, so we change it to something actually usable.
2017-10-20 15:20:51 +02:00
'memory': 2048,
buildserver: send config to vagrant via YAML file Python can easily output dicts as YAML, and a Vagrantfile is a ruby script, which can easily read YAML. Going this route means that Vagrantfile can ultimately be committed to git, and the configuration will happen all via Python dicts output as YAML. That makes it drastically easier to follow the code, and to make modifications.
2016-07-04 20:22:00 +02:00
'hwvirtex': 'off',
buildserver: auto-detect and use libvirt's QEMU+KVM For running in QEMU/KVM guests like on jenkins.debian.net, this sets up the whole process automatically. This only really covers the case where this is running in a KVM guest, and the original case of running VirtualBox on bare metal. It could be extended to cover more cases if someone wanted to.
2016-09-15 11:12:18 +02:00
'vm_provider': 'virtualbox',
makebuildserver: set defaults in script so it can run without config This makes it so that ./makebuildserver will run without any config file, using the defaults that are embedded in the script itself. This is like how `fdroid` works.
2016-01-20 21:00:16 +01:00
}
name makebuildserver config the same as the script This keeps the numbers of names down to a minimum, and since the config is placed right next to the script, this keeps tab completion working nicely when the config file is in place. The old file name is still supported.
2016-01-20 20:38:18 +01:00
buildserver: auto-detect and use libvirt's QEMU+KVM For running in QEMU/KVM guests like on jenkins.debian.net, this sets up the whole process automatically. This only really covers the case where this is running in a KVM guest, and the original case of running VirtualBox on bare metal. It could be extended to cover more cases if someone wanted to.
2016-09-15 11:12:18 +02:00
if os.path.isfile('/usr/bin/systemd-detect-virt'):
try:
virt = subprocess.check_output('/usr/bin/systemd-detect-virt').strip().decode('utf-8')
except subprocess.CalledProcessError as e:
virt = 'none'
buildserver: bochs can also be used with KVM jenkins.debian.net is being detected as 'bochs' rather than 'qemu'.
2016-09-23 17:40:23 +02:00
if virt == 'qemu' or virt == 'kvm' or virt == 'bochs':
makebuildserver debugging log details
2017-03-24 20:04:50 +01:00
logger.info('Running in a VM guest, defaulting to QEMU/KVM via libvirt')
buildserver: auto-detect and use libvirt's QEMU+KVM For running in QEMU/KVM guests like on jenkins.debian.net, this sets up the whole process automatically. This only really covers the case where this is running in a KVM guest, and the original case of running VirtualBox on bare metal. It could be extended to cover more cases if someone wanted to.
2016-09-15 11:12:18 +02:00
config['vm_provider'] = 'libvirt'
elif virt != 'none':
makebuildserver debugging log details
2017-03-24 20:04:50 +01:00
logger.info('Running in an unsupported VM guest (%s)!', virt)
makebuildserver vbox logmessage fix
2017-04-13 12:49:14 +02:00
logger.debug('detected virt: %s', virt)
buildserver: auto-detect and use libvirt's QEMU+KVM For running in QEMU/KVM guests like on jenkins.debian.net, this sets up the whole process automatically. This only really covers the case where this is running in a KVM guest, and the original case of running VirtualBox on bare metal. It could be extended to cover more cases if someone wanted to.
2016-09-15 11:12:18 +02:00
name makebuildserver config the same as the script This keeps the numbers of names down to a minimum, and since the config is placed right next to the script, this keeps tab completion working nicely when the config file is in place. The old file name is still supported.
2016-01-20 20:38:18 +01:00
# load config file, if present
if os.path.exists('makebuildserver.config.py'):
Run 2to3 on makebuildserver Mostly just print conversion.
2016-01-04 17:24:57 +01:00
exec(compile(open('makebuildserver.config.py').read(), 'makebuildserver.config.py', 'exec'), config)
name makebuildserver config the same as the script This keeps the numbers of names down to a minimum, and since the config is placed right next to the script, this keeps tab completion working nicely when the config file is in place. The old file name is still supported.
2016-01-20 20:38:18 +01:00
elif os.path.exists('makebs.config.py'):
# this is the old name for the config file
Run 2to3 on makebuildserver Mostly just print conversion.
2016-01-04 17:24:57 +01:00
exec(compile(open('makebs.config.py').read(), 'makebs.config.py', 'exec'), config)
makebs: fix config crash Without this, running makebuildserver from a clean master results in the following: $ ./makebuildserver Traceback (most recent call last): File "./makebuildserver", line 74, in <module> del(config['__builtins__']) # added by compile/exec KeyError: '__builtins__' This is because a clean checkout has no config, thus exec is never actually ran.
2016-07-27 11:20:17 +02:00
if '__builtins__' in config:
del(config['__builtins__']) # added by compile/exec
makebuildserver debugging log details
2017-03-24 20:04:50 +01:00
logger.debug("makebuildserver.config.py parsed -> %s", json.dumps(config, indent=4, sort_keys=True))
Add some makebuildserver options
2013-10-20 22:16:42 +02:00
Cache main sdk/ndk downloads
2013-05-25 14:55:16 +02:00
# Update cached files.
makebuildserver: cache defaults to ~/.cache/fdroidserver This also provides a config option to override that default. ~/.cache is a standard location on GNU/Linux machines for cached content. It is also good to have the cache outside of the git repo in case `git clean -fdx` is run, which would delete all files in the directory that are not part of the git repo, including buildserver/cache/
2016-01-21 14:09:12 +01:00
cachedir = config['cachedir']
Cache main sdk/ndk downloads
2013-05-25 14:55:16 +02:00
if not os.path.exists(cachedir):
Run 2to3 on makebuildserver Mostly just print conversion.
2016-01-04 17:24:57 +01:00
os.makedirs(cachedir, 0o755)
makebuildserver debugging log details
2017-03-24 20:04:50 +01:00
logger.debug('created cachedir %s because it did not exists.', cachedir)
Fix pep8 warnings in makebuildserver
2014-05-31 23:00:27 +02:00
buildserver: only check cache permissions when using libvirt VirtualBox runs as the same user as `fdroid`, so the cache does not need to be accessible by the world. On libvirt, libvirtd runs the VMs as its own user, so in that case, the cache dirs must have permissions to let that user access them.
2016-09-23 15:19:15 +02:00
if config['vm_provider'] == 'libvirt':
tmp = cachedir
while tmp != '/':
mode = os.stat(tmp).st_mode
if not (stat.S_IXUSR & mode and stat.S_IXGRP & mode and stat.S_IXOTH & mode):
makebuildserver debugging log details
2017-03-24 20:04:50 +01:00
logger.critical('ERROR: %s will not be accessible to the VM! To fix, run:', tmp)
logger.critical(' chmod a+X %s', tmp)
buildserver: only check cache permissions when using libvirt VirtualBox runs as the same user as `fdroid`, so the cache does not need to be accessible by the world. On libvirt, libvirtd runs the VMs as its own user, so in that case, the cache dirs must have permissions to let that user access them.
2016-09-23 15:19:15 +02:00
sys.exit(1)
tmp = os.path.dirname(tmp)
makebuildserver debugging log details
2017-03-24 20:04:50 +01:00
logger.debug('cache dir %s is accessible for libvirt vm.', cachedir)
buildserver: make sure cachedir is accessible to VM instance It seems that KVM/QEMU's 9p sharing is more sensitive to file perms. If ~/.cache is 0700, then ~/.cache/fdroidserver cannot be mounted in the guest
2016-09-06 14:22:29 +02:00
buildserver: send config to vagrant via YAML file Python can easily output dicts as YAML, and a Vagrantfile is a ruby script, which can easily read YAML. Going this route means that Vagrantfile can ultimately be committed to git, and the configuration will happen all via Python dicts output as YAML. That makes it drastically easier to follow the code, and to make modifications.
2016-07-04 20:22:00 +02:00
if config['apt_package_cache']:
config['aptcachedir'] = cachedir + '/apt/archives'
makebuildserver debugging log details
2017-03-24 20:04:50 +01:00
logger.debug('aptcachedir is set to %s', config['aptcachedir'])
makebuildserver deal with apt cache lock
2017-04-09 00:57:59 +02:00
aptcachelock = os.path.join(config['aptcachedir'], 'lock')
if os.path.isfile(aptcachelock):
logger.info('apt cache dir is locked, removing lock')
os.remove(aptcachelock)
aptcachepartial = os.path.join(config['aptcachedir'], 'partial')
if os.path.isdir(aptcachepartial):
logger.info('removing partial downloads from apt cache dir')
shutil.rmtree(aptcachepartial)
buildserver: send config to vagrant via YAML file Python can easily output dicts as YAML, and a Vagrantfile is a ruby script, which can easily read YAML. Going this route means that Vagrantfile can ultimately be committed to git, and the configuration will happen all via Python dicts output as YAML. That makes it drastically easier to follow the code, and to make modifications.
2016-07-04 20:22:00 +02:00
Cache main sdk/ndk downloads
2013-05-25 14:55:16 +02:00
cachefiles = [
remove trailing whitespace
2017-09-03 19:30:00 +02:00
# Don't update sdk tools beyond 25.2.5.
makebuildserver: add comment about not upgrading sdk-tools
2017-09-03 18:33:03 +02:00
# Support for android update project has been removed and there is no replacement.
# Until we find a solution for that we need to stay at this revision.
makebuildserver: bump tools, add api26 and build-tools 26
2017-06-09 19:35:04 +02:00
('https://dl.google.com/android/repository/tools_r25.2.5-linux.zip',
'577516819c8b5fae680f049d39014ff1ba4af870b687cab10595783e6f22d33e'),
Update support repository.
2017-03-29 12:33:38 +02:00
('https://dl.google.com/android/repository/android_m2repository_r47.zip',
'a3f91808dce50c1717737de90c18479ed3a78b147e06985247d138e7ab5123d0'),
makebuildserver: use original names for downloaded SDK components It will make it a lot easier to manage the cache if we use the original file names, which often include the file version. This also changes the download process to be resumable if there is a partial file in the cache, and switches from calling wget on the command line to using the python libs 'requests' and 'clint' to provide a similar experience. While its not so important for this particular bit of code to use those libraries, I think those two will allow us to provide a better user experience throughout the whole of fdroidserver. In this case, it is already doing special tricks fetching the file size from the server before trying to download it. I suppose this code could instead check if the file exists, and if so, check the hash sum. I think that would be slower for most people since checking the hash on large files takes a noticeable about of time, while a HTTP HEAD request is pretty tiny.
2016-06-10 11:42:42 +02:00
('https://dl.google.com/android/repository/android-1.5_r04-linux.zip',
Download platforms via makebuildserver
2015-11-20 15:43:47 +01:00
'85b6c8f9797e56aa415d3a282428bb640c96b0acb17c11d41621bb2a5302fe64'),
makebuildserver: use original names for downloaded SDK components It will make it a lot easier to manage the cache if we use the original file names, which often include the file version. This also changes the download process to be resumable if there is a partial file in the cache, and switches from calling wget on the command line to using the python libs 'requests' and 'clint' to provide a similar experience. While its not so important for this particular bit of code to use those libraries, I think those two will allow us to provide a better user experience throughout the whole of fdroidserver. In this case, it is already doing special tricks fetching the file size from the server before trying to download it. I suppose this code could instead check if the file exists, and if so, check the hash sum. I think that would be slower for most people since checking the hash on large files takes a noticeable about of time, while a HTTP HEAD request is pretty tiny.
2016-06-10 11:42:42 +02:00
('https://dl.google.com/android/repository/android-1.6_r03-linux.zip',
Download platforms via makebuildserver
2015-11-20 15:43:47 +01:00
'a8c4e3b32269c6b04c2adeabd112fce42f292dab1a40ef3b08ea7d4212be0df4'),
makebuildserver: use original names for downloaded SDK components It will make it a lot easier to manage the cache if we use the original file names, which often include the file version. This also changes the download process to be resumable if there is a partial file in the cache, and switches from calling wget on the command line to using the python libs 'requests' and 'clint' to provide a similar experience. While its not so important for this particular bit of code to use those libraries, I think those two will allow us to provide a better user experience throughout the whole of fdroidserver. In this case, it is already doing special tricks fetching the file size from the server before trying to download it. I suppose this code could instead check if the file exists, and if so, check the hash sum. I think that would be slower for most people since checking the hash on large files takes a noticeable about of time, while a HTTP HEAD request is pretty tiny.
2016-06-10 11:42:42 +02:00
('https://dl.google.com/android/repository/android-2.0_r01-linux.zip',
Download platforms via makebuildserver
2015-11-20 15:43:47 +01:00
'e70e2151b49613f23f40828c771ab85e241eed361cab037c6312df77f2612f0a'),
makebuildserver: use original names for downloaded SDK components It will make it a lot easier to manage the cache if we use the original file names, which often include the file version. This also changes the download process to be resumable if there is a partial file in the cache, and switches from calling wget on the command line to using the python libs 'requests' and 'clint' to provide a similar experience. While its not so important for this particular bit of code to use those libraries, I think those two will allow us to provide a better user experience throughout the whole of fdroidserver. In this case, it is already doing special tricks fetching the file size from the server before trying to download it. I suppose this code could instead check if the file exists, and if so, check the hash sum. I think that would be slower for most people since checking the hash on large files takes a noticeable about of time, while a HTTP HEAD request is pretty tiny.
2016-06-10 11:42:42 +02:00
('https://dl.google.com/android/repository/android-2.0.1_r01-linux.zip',
Download platforms via makebuildserver
2015-11-20 15:43:47 +01:00
'f47b46177b17f6368461f85bc2a27d0d2c437929f588ea27105712bc3185f664'),
buildserver: use `android update sdk` to install Android SDK `android update sdk --no-ui` is the standard command line tool for installing the Android SDK. By symlinking into the $ANDROID_HOME/temp dir, the cached files can still be used. This converts the chef recipe to a vagrant shell provisioning script since it was all bash anyway. Some file names no longer officially have a -linux in them, so those were changed to keep the cache working with the default filename.
2016-06-14 16:34:13 +02:00
('https://dl.google.com/android/repository/android-2.1_r03.zip',
Download platforms via makebuildserver
2015-11-20 15:43:47 +01:00
'b9cc140a9b879586181b22cfc7d4aa18b979251e16e9b17771c5d0acb71ba940'),
buildserver: use `android update sdk` to install Android SDK `android update sdk --no-ui` is the standard command line tool for installing the Android SDK. By symlinking into the $ANDROID_HOME/temp dir, the cached files can still be used. This converts the chef recipe to a vagrant shell provisioning script since it was all bash anyway. Some file names no longer officially have a -linux in them, so those were changed to keep the cache working with the default filename.
2016-06-14 16:34:13 +02:00
('https://dl.google.com/android/repository/android-2.2_r03.zip',
Download platforms via makebuildserver
2015-11-20 15:43:47 +01:00
'7c9ea1bd7cb225504bd085d7c93ae27d52bd88d29b621d28108f82fef68177c0'),
buildserver: use `android update sdk` to install Android SDK `android update sdk --no-ui` is the standard command line tool for installing the Android SDK. By symlinking into the $ANDROID_HOME/temp dir, the cached files can still be used. This converts the chef recipe to a vagrant shell provisioning script since it was all bash anyway. Some file names no longer officially have a -linux in them, so those were changed to keep the cache working with the default filename.
2016-06-14 16:34:13 +02:00
('https://dl.google.com/android/repository/android-2.3.1_r02.zip',
Download platforms via makebuildserver
2015-11-20 15:43:47 +01:00
'b2ab4896d0a4857e4f688f69eb08b0e1a8074709d4445a92a83ece7ec7cd198c'),
buildserver: use `android update sdk` to install Android SDK `android update sdk --no-ui` is the standard command line tool for installing the Android SDK. By symlinking into the $ANDROID_HOME/temp dir, the cached files can still be used. This converts the chef recipe to a vagrant shell provisioning script since it was all bash anyway. Some file names no longer officially have a -linux in them, so those were changed to keep the cache working with the default filename.
2016-06-14 16:34:13 +02:00
('https://dl.google.com/android/repository/android-2.3.3_r02.zip',
Download platforms via makebuildserver
2015-11-20 15:43:47 +01:00
'54bdb0f1ca06ba5747061ddeea20f431af72c448334fd4d3d7f84ea2ccd29fea'),
buildserver: use `android update sdk` to install Android SDK `android update sdk --no-ui` is the standard command line tool for installing the Android SDK. By symlinking into the $ANDROID_HOME/temp dir, the cached files can still be used. This converts the chef recipe to a vagrant shell provisioning script since it was all bash anyway. Some file names no longer officially have a -linux in them, so those were changed to keep the cache working with the default filename.
2016-06-14 16:34:13 +02:00
('https://dl.google.com/android/repository/android-3.0_r02.zip',
Download platforms via makebuildserver
2015-11-20 15:43:47 +01:00
'1cacae7b6e1b5a5d73c06f5d29d2ea92d16674df8fd5507681290e77d1647a1c'),
buildserver: use `android update sdk` to install Android SDK `android update sdk --no-ui` is the standard command line tool for installing the Android SDK. By symlinking into the $ANDROID_HOME/temp dir, the cached files can still be used. This converts the chef recipe to a vagrant shell provisioning script since it was all bash anyway. Some file names no longer officially have a -linux in them, so those were changed to keep the cache working with the default filename.
2016-06-14 16:34:13 +02:00
('https://dl.google.com/android/repository/android-3.1_r03.zip',
Download platforms via makebuildserver
2015-11-20 15:43:47 +01:00
'7570c86a86488a146aa2141a65a24d81800959c1907ff4f1d2c13bbafab230c5'),
buildserver: use `android update sdk` to install Android SDK `android update sdk --no-ui` is the standard command line tool for installing the Android SDK. By symlinking into the $ANDROID_HOME/temp dir, the cached files can still be used. This converts the chef recipe to a vagrant shell provisioning script since it was all bash anyway. Some file names no longer officially have a -linux in them, so those were changed to keep the cache working with the default filename.
2016-06-14 16:34:13 +02:00
('https://dl.google.com/android/repository/android-3.2_r01.zip',
Download platforms via makebuildserver
2015-11-20 15:43:47 +01:00
'ff6b26ad34d7060a72ba504b0314cef8ba3138005561705adec5ad470a073d9b'),
makebuildserver: use original names for downloaded SDK components It will make it a lot easier to manage the cache if we use the original file names, which often include the file version. This also changes the download process to be resumable if there is a partial file in the cache, and switches from calling wget on the command line to using the python libs 'requests' and 'clint' to provide a similar experience. While its not so important for this particular bit of code to use those libraries, I think those two will allow us to provide a better user experience throughout the whole of fdroidserver. In this case, it is already doing special tricks fetching the file size from the server before trying to download it. I suppose this code could instead check if the file exists, and if so, check the hash sum. I think that would be slower for most people since checking the hash on large files takes a noticeable about of time, while a HTTP HEAD request is pretty tiny.
2016-06-10 11:42:42 +02:00
('https://dl.google.com/android/repository/android-14_r04.zip',
Download platforms via makebuildserver
2015-11-20 15:43:47 +01:00
'da1af15c77ba41d062eb6d0ef5921cc424ab6167587033b830609d65f04802b6'),
makebuildserver: use original names for downloaded SDK components It will make it a lot easier to manage the cache if we use the original file names, which often include the file version. This also changes the download process to be resumable if there is a partial file in the cache, and switches from calling wget on the command line to using the python libs 'requests' and 'clint' to provide a similar experience. While its not so important for this particular bit of code to use those libraries, I think those two will allow us to provide a better user experience throughout the whole of fdroidserver. In this case, it is already doing special tricks fetching the file size from the server before trying to download it. I suppose this code could instead check if the file exists, and if so, check the hash sum. I think that would be slower for most people since checking the hash on large files takes a noticeable about of time, while a HTTP HEAD request is pretty tiny.
2016-06-10 11:42:42 +02:00
('https://dl.google.com/android/repository/android-15_r05.zip',
Download platforms via makebuildserver
2015-11-20 15:43:47 +01:00
'5bc1f93aae86b4336ffc4cae9eb8ec41a9a8fd677582dd86a9629798f019bed9'),
makebuildserver: use original names for downloaded SDK components It will make it a lot easier to manage the cache if we use the original file names, which often include the file version. This also changes the download process to be resumable if there is a partial file in the cache, and switches from calling wget on the command line to using the python libs 'requests' and 'clint' to provide a similar experience. While its not so important for this particular bit of code to use those libraries, I think those two will allow us to provide a better user experience throughout the whole of fdroidserver. In this case, it is already doing special tricks fetching the file size from the server before trying to download it. I suppose this code could instead check if the file exists, and if so, check the hash sum. I think that would be slower for most people since checking the hash on large files takes a noticeable about of time, while a HTTP HEAD request is pretty tiny.
2016-06-10 11:42:42 +02:00
('https://dl.google.com/android/repository/android-16_r05.zip',
Download platforms via makebuildserver
2015-11-20 15:43:47 +01:00
'fd7f269a423d1f1d079eabf9f918ceab49108702a1c6bb2589d57c23393503d3'),
makebuildserver: use original names for downloaded SDK components It will make it a lot easier to manage the cache if we use the original file names, which often include the file version. This also changes the download process to be resumable if there is a partial file in the cache, and switches from calling wget on the command line to using the python libs 'requests' and 'clint' to provide a similar experience. While its not so important for this particular bit of code to use those libraries, I think those two will allow us to provide a better user experience throughout the whole of fdroidserver. In this case, it is already doing special tricks fetching the file size from the server before trying to download it. I suppose this code could instead check if the file exists, and if so, check the hash sum. I think that would be slower for most people since checking the hash on large files takes a noticeable about of time, while a HTTP HEAD request is pretty tiny.
2016-06-10 11:42:42 +02:00
('https://dl.google.com/android/repository/android-17_r03.zip',
Download platforms via makebuildserver
2015-11-20 15:43:47 +01:00
'b66e73fb2639f8c916fde4369aa29012a5c531e156dbb205fe3788fe998fbbe8'),
makebuildserver: use original names for downloaded SDK components It will make it a lot easier to manage the cache if we use the original file names, which often include the file version. This also changes the download process to be resumable if there is a partial file in the cache, and switches from calling wget on the command line to using the python libs 'requests' and 'clint' to provide a similar experience. While its not so important for this particular bit of code to use those libraries, I think those two will allow us to provide a better user experience throughout the whole of fdroidserver. In this case, it is already doing special tricks fetching the file size from the server before trying to download it. I suppose this code could instead check if the file exists, and if so, check the hash sum. I think that would be slower for most people since checking the hash on large files takes a noticeable about of time, while a HTTP HEAD request is pretty tiny.
2016-06-10 11:42:42 +02:00
('https://dl.google.com/android/repository/android-18_r03.zip',
Download platforms via makebuildserver
2015-11-20 15:43:47 +01:00
'166ae9cf299747a5faa8f04168f0ee47cd7466a975d8b44acaaa62a43e767568'),
makebuildserver: use original names for downloaded SDK components It will make it a lot easier to manage the cache if we use the original file names, which often include the file version. This also changes the download process to be resumable if there is a partial file in the cache, and switches from calling wget on the command line to using the python libs 'requests' and 'clint' to provide a similar experience. While its not so important for this particular bit of code to use those libraries, I think those two will allow us to provide a better user experience throughout the whole of fdroidserver. In this case, it is already doing special tricks fetching the file size from the server before trying to download it. I suppose this code could instead check if the file exists, and if so, check the hash sum. I think that would be slower for most people since checking the hash on large files takes a noticeable about of time, while a HTTP HEAD request is pretty tiny.
2016-06-10 11:42:42 +02:00
('https://dl.google.com/android/repository/android-19_r04.zip',
Download platforms via makebuildserver
2015-11-20 15:43:47 +01:00
'5efc3a3a682c1d49128daddb6716c433edf16e63349f32959b6207524ac04039'),
makebuildserver: use original names for downloaded SDK components It will make it a lot easier to manage the cache if we use the original file names, which often include the file version. This also changes the download process to be resumable if there is a partial file in the cache, and switches from calling wget on the command line to using the python libs 'requests' and 'clint' to provide a similar experience. While its not so important for this particular bit of code to use those libraries, I think those two will allow us to provide a better user experience throughout the whole of fdroidserver. In this case, it is already doing special tricks fetching the file size from the server before trying to download it. I suppose this code could instead check if the file exists, and if so, check the hash sum. I think that would be slower for most people since checking the hash on large files takes a noticeable about of time, while a HTTP HEAD request is pretty tiny.
2016-06-10 11:42:42 +02:00
('https://dl.google.com/android/repository/android-20_r02.zip',
Download platforms via makebuildserver
2015-11-20 15:43:47 +01:00
'ef08c453e16ab6e656cf5d9413ef61cb8c650607d33b24ee4ce08dafdfe965a7'),
makebuildserver: use original names for downloaded SDK components It will make it a lot easier to manage the cache if we use the original file names, which often include the file version. This also changes the download process to be resumable if there is a partial file in the cache, and switches from calling wget on the command line to using the python libs 'requests' and 'clint' to provide a similar experience. While its not so important for this particular bit of code to use those libraries, I think those two will allow us to provide a better user experience throughout the whole of fdroidserver. In this case, it is already doing special tricks fetching the file size from the server before trying to download it. I suppose this code could instead check if the file exists, and if so, check the hash sum. I think that would be slower for most people since checking the hash on large files takes a noticeable about of time, while a HTTP HEAD request is pretty tiny.
2016-06-10 11:42:42 +02:00
('https://dl.google.com/android/repository/android-21_r02.zip',
Download platforms via makebuildserver
2015-11-20 15:43:47 +01:00
'a76cd7ad3080ac6ce9f037cb935b399a1bad396c0605d4ff42f693695f1dcefe'),
makebuildserver: use original names for downloaded SDK components It will make it a lot easier to manage the cache if we use the original file names, which often include the file version. This also changes the download process to be resumable if there is a partial file in the cache, and switches from calling wget on the command line to using the python libs 'requests' and 'clint' to provide a similar experience. While its not so important for this particular bit of code to use those libraries, I think those two will allow us to provide a better user experience throughout the whole of fdroidserver. In this case, it is already doing special tricks fetching the file size from the server before trying to download it. I suppose this code could instead check if the file exists, and if so, check the hash sum. I think that would be slower for most people since checking the hash on large files takes a noticeable about of time, while a HTTP HEAD request is pretty tiny.
2016-06-10 11:42:42 +02:00
('https://dl.google.com/android/repository/android-22_r02.zip',
Download platforms via makebuildserver
2015-11-20 15:43:47 +01:00
'45eb581bbe53c9256f34c26b2cea919543c0079140897ac721cf88c0b9f6789e'),
makebuildserver: use original names for downloaded SDK components It will make it a lot easier to manage the cache if we use the original file names, which often include the file version. This also changes the download process to be resumable if there is a partial file in the cache, and switches from calling wget on the command line to using the python libs 'requests' and 'clint' to provide a similar experience. While its not so important for this particular bit of code to use those libraries, I think those two will allow us to provide a better user experience throughout the whole of fdroidserver. In this case, it is already doing special tricks fetching the file size from the server before trying to download it. I suppose this code could instead check if the file exists, and if so, check the hash sum. I think that would be slower for most people since checking the hash on large files takes a noticeable about of time, while a HTTP HEAD request is pretty tiny.
2016-06-10 11:42:42 +02:00
('https://dl.google.com/android/repository/platform-23_r03.zip',
Bump platform-23 to r03
2016-05-26 17:05:39 +02:00
'4b4bcddead3319708275c54c76294707bfaa953d767e34f1a5b599f3edd0076c'),
update to platform-24_r02 in ./makebuildserver
2016-08-30 19:47:37 +02:00
('https://dl.google.com/android/repository/platform-24_r02.zip',
'f268f5945c6ece7ea95c1c252067280854d2a20da924e22ae4720287df8bdbc9'),
makebuildserver: update platform and build tool releases Update platform-25 to r03 Update platform-26 to r02 Add build-tools_r26.0.1 sha256 signatures locally computed after verifying the sha1 signature provided by the repository2-1.xml file. All these will be downloaded and installed in the buildserver through provision-android-sdk anyway but this adds them to the cache.
2017-08-17 00:41:23 +02:00
('https://dl.google.com/android/repository/platform-25_r03.zip',
'9b742d34590fe73fb7229e34835ecffb1846ca389d9f924f0b2a37de525dc6b8'),
('https://dl.google.com/android/repository/platform-26_r02.zip',
'2aafa7d19c5e9c4b643ee6ade3d85ef89dc2f79e8383efdb9baf7fddad74b52a'),
makebuildserver: use original names for downloaded SDK components It will make it a lot easier to manage the cache if we use the original file names, which often include the file version. This also changes the download process to be resumable if there is a partial file in the cache, and switches from calling wget on the command line to using the python libs 'requests' and 'clint' to provide a similar experience. While its not so important for this particular bit of code to use those libraries, I think those two will allow us to provide a better user experience throughout the whole of fdroidserver. In this case, it is already doing special tricks fetching the file size from the server before trying to download it. I suppose this code could instead check if the file exists, and if so, check the hash sum. I think that would be slower for most people since checking the hash on large files takes a noticeable about of time, while a HTTP HEAD request is pretty tiny.
2016-06-10 11:42:42 +02:00
('https://dl.google.com/android/repository/build-tools_r17-linux.zip',
Install build-tools from the cache too
2015-11-20 21:14:08 +01:00
'4c8444972343a19045236f6924bd7f12046287c70dace96ab88b2159c8ec0e74'),
makebuildserver: use original names for downloaded SDK components It will make it a lot easier to manage the cache if we use the original file names, which often include the file version. This also changes the download process to be resumable if there is a partial file in the cache, and switches from calling wget on the command line to using the python libs 'requests' and 'clint' to provide a similar experience. While its not so important for this particular bit of code to use those libraries, I think those two will allow us to provide a better user experience throughout the whole of fdroidserver. In this case, it is already doing special tricks fetching the file size from the server before trying to download it. I suppose this code could instead check if the file exists, and if so, check the hash sum. I think that would be slower for most people since checking the hash on large files takes a noticeable about of time, while a HTTP HEAD request is pretty tiny.
2016-06-10 11:42:42 +02:00
('https://dl.google.com/android/repository/build-tools_r18.0.1-linux.zip',
Install build-tools from the cache too
2015-11-20 21:14:08 +01:00
'a9b7b1bdfd864780fdd03fa1683f3fe712a4276cf200646833808cb9159bafc0'),
makebuildserver: use original names for downloaded SDK components It will make it a lot easier to manage the cache if we use the original file names, which often include the file version. This also changes the download process to be resumable if there is a partial file in the cache, and switches from calling wget on the command line to using the python libs 'requests' and 'clint' to provide a similar experience. While its not so important for this particular bit of code to use those libraries, I think those two will allow us to provide a better user experience throughout the whole of fdroidserver. In this case, it is already doing special tricks fetching the file size from the server before trying to download it. I suppose this code could instead check if the file exists, and if so, check the hash sum. I think that would be slower for most people since checking the hash on large files takes a noticeable about of time, while a HTTP HEAD request is pretty tiny.
2016-06-10 11:42:42 +02:00
('https://dl.google.com/android/repository/build-tools_r18.1-linux.zip',
Install build-tools from the cache too
2015-11-20 21:14:08 +01:00
'0753606738f31cc346426db1d46b7d021bc1bdaff63085f9ee9d278ee054d3c9'),
makebuildserver: use original names for downloaded SDK components It will make it a lot easier to manage the cache if we use the original file names, which often include the file version. This also changes the download process to be resumable if there is a partial file in the cache, and switches from calling wget on the command line to using the python libs 'requests' and 'clint' to provide a similar experience. While its not so important for this particular bit of code to use those libraries, I think those two will allow us to provide a better user experience throughout the whole of fdroidserver. In this case, it is already doing special tricks fetching the file size from the server before trying to download it. I suppose this code could instead check if the file exists, and if so, check the hash sum. I think that would be slower for most people since checking the hash on large files takes a noticeable about of time, while a HTTP HEAD request is pretty tiny.
2016-06-10 11:42:42 +02:00
('https://dl.google.com/android/repository/build-tools_r18.1.1-linux.zip',
Install build-tools from the cache too
2015-11-20 21:14:08 +01:00
'7e4ed326b53078f4f23276ddab52c400011f7593dfbb6508c0a6671954dba8b0'),
makebuildserver: use original names for downloaded SDK components It will make it a lot easier to manage the cache if we use the original file names, which often include the file version. This also changes the download process to be resumable if there is a partial file in the cache, and switches from calling wget on the command line to using the python libs 'requests' and 'clint' to provide a similar experience. While its not so important for this particular bit of code to use those libraries, I think those two will allow us to provide a better user experience throughout the whole of fdroidserver. In this case, it is already doing special tricks fetching the file size from the server before trying to download it. I suppose this code could instead check if the file exists, and if so, check the hash sum. I think that would be slower for most people since checking the hash on large files takes a noticeable about of time, while a HTTP HEAD request is pretty tiny.
2016-06-10 11:42:42 +02:00
('https://dl.google.com/android/repository/build-tools_r19-linux.zip',
Install build-tools from the cache too
2015-11-20 21:14:08 +01:00
'9442e1c5212ed594e344a231fa93e7a017a5ef8cc661117011f1d3142eca7acc'),
makebuildserver: use original names for downloaded SDK components It will make it a lot easier to manage the cache if we use the original file names, which often include the file version. This also changes the download process to be resumable if there is a partial file in the cache, and switches from calling wget on the command line to using the python libs 'requests' and 'clint' to provide a similar experience. While its not so important for this particular bit of code to use those libraries, I think those two will allow us to provide a better user experience throughout the whole of fdroidserver. In this case, it is already doing special tricks fetching the file size from the server before trying to download it. I suppose this code could instead check if the file exists, and if so, check the hash sum. I think that would be slower for most people since checking the hash on large files takes a noticeable about of time, while a HTTP HEAD request is pretty tiny.
2016-06-10 11:42:42 +02:00
('https://dl.google.com/android/repository/build-tools_r19.0.1-linux.zip',
Install build-tools from the cache too
2015-11-20 21:14:08 +01:00
'b068edaff05c3253a63e9c8f0e1786429799b7e4b01514a847a8b291beb9232e'),
makebuildserver: use original names for downloaded SDK components It will make it a lot easier to manage the cache if we use the original file names, which often include the file version. This also changes the download process to be resumable if there is a partial file in the cache, and switches from calling wget on the command line to using the python libs 'requests' and 'clint' to provide a similar experience. While its not so important for this particular bit of code to use those libraries, I think those two will allow us to provide a better user experience throughout the whole of fdroidserver. In this case, it is already doing special tricks fetching the file size from the server before trying to download it. I suppose this code could instead check if the file exists, and if so, check the hash sum. I think that would be slower for most people since checking the hash on large files takes a noticeable about of time, while a HTTP HEAD request is pretty tiny.
2016-06-10 11:42:42 +02:00
('https://dl.google.com/android/repository/build-tools_r19.0.2-linux.zip',
Install build-tools from the cache too
2015-11-20 21:14:08 +01:00
'06124fad0d4bde21191240d61df2059a8546c085064a9a57d024c36fa2c9bebb'),
makebuildserver: use original names for downloaded SDK components It will make it a lot easier to manage the cache if we use the original file names, which often include the file version. This also changes the download process to be resumable if there is a partial file in the cache, and switches from calling wget on the command line to using the python libs 'requests' and 'clint' to provide a similar experience. While its not so important for this particular bit of code to use those libraries, I think those two will allow us to provide a better user experience throughout the whole of fdroidserver. In this case, it is already doing special tricks fetching the file size from the server before trying to download it. I suppose this code could instead check if the file exists, and if so, check the hash sum. I think that would be slower for most people since checking the hash on large files takes a noticeable about of time, while a HTTP HEAD request is pretty tiny.
2016-06-10 11:42:42 +02:00
('https://dl.google.com/android/repository/build-tools_r19.0.3-linux.zip',
Install build-tools from the cache too
2015-11-20 21:14:08 +01:00
'bc9b3db0de4a3e233a170274293359051a758f1e3f0d0d852ff4ad6d90d0a794'),
makebuildserver: use original names for downloaded SDK components It will make it a lot easier to manage the cache if we use the original file names, which often include the file version. This also changes the download process to be resumable if there is a partial file in the cache, and switches from calling wget on the command line to using the python libs 'requests' and 'clint' to provide a similar experience. While its not so important for this particular bit of code to use those libraries, I think those two will allow us to provide a better user experience throughout the whole of fdroidserver. In this case, it is already doing special tricks fetching the file size from the server before trying to download it. I suppose this code could instead check if the file exists, and if so, check the hash sum. I think that would be slower for most people since checking the hash on large files takes a noticeable about of time, while a HTTP HEAD request is pretty tiny.
2016-06-10 11:42:42 +02:00
('https://dl.google.com/android/repository/build-tools_r19.1-linux.zip',
Install build-tools from the cache too
2015-11-20 21:14:08 +01:00
'3833b409f78c002a83244e220be380ea6fa44d604e0d47de4b7e5daefe7cd3f4'),
makebuildserver: use original names for downloaded SDK components It will make it a lot easier to manage the cache if we use the original file names, which often include the file version. This also changes the download process to be resumable if there is a partial file in the cache, and switches from calling wget on the command line to using the python libs 'requests' and 'clint' to provide a similar experience. While its not so important for this particular bit of code to use those libraries, I think those two will allow us to provide a better user experience throughout the whole of fdroidserver. In this case, it is already doing special tricks fetching the file size from the server before trying to download it. I suppose this code could instead check if the file exists, and if so, check the hash sum. I think that would be slower for most people since checking the hash on large files takes a noticeable about of time, while a HTTP HEAD request is pretty tiny.
2016-06-10 11:42:42 +02:00
('https://dl.google.com/android/repository/build-tools_r20-linux.zip',
Install build-tools from the cache too
2015-11-20 21:14:08 +01:00
'296e09d62095d80e6eaa06a64cfa4c6f9f317c2d67ad8da6514523ec66f5c871'),
makebuildserver: use original names for downloaded SDK components It will make it a lot easier to manage the cache if we use the original file names, which often include the file version. This also changes the download process to be resumable if there is a partial file in the cache, and switches from calling wget on the command line to using the python libs 'requests' and 'clint' to provide a similar experience. While its not so important for this particular bit of code to use those libraries, I think those two will allow us to provide a better user experience throughout the whole of fdroidserver. In this case, it is already doing special tricks fetching the file size from the server before trying to download it. I suppose this code could instead check if the file exists, and if so, check the hash sum. I think that would be slower for most people since checking the hash on large files takes a noticeable about of time, while a HTTP HEAD request is pretty tiny.
2016-06-10 11:42:42 +02:00
('https://dl.google.com/android/repository/build-tools_r21-linux.zip',
Install build-tools from the cache too
2015-11-20 21:14:08 +01:00
'12b818f38fe1b68091b94545988317438efbf41eb61fd36b72cd79f536044065'),
makebuildserver: use original names for downloaded SDK components It will make it a lot easier to manage the cache if we use the original file names, which often include the file version. This also changes the download process to be resumable if there is a partial file in the cache, and switches from calling wget on the command line to using the python libs 'requests' and 'clint' to provide a similar experience. While its not so important for this particular bit of code to use those libraries, I think those two will allow us to provide a better user experience throughout the whole of fdroidserver. In this case, it is already doing special tricks fetching the file size from the server before trying to download it. I suppose this code could instead check if the file exists, and if so, check the hash sum. I think that would be slower for most people since checking the hash on large files takes a noticeable about of time, while a HTTP HEAD request is pretty tiny.
2016-06-10 11:42:42 +02:00
('https://dl.google.com/android/repository/build-tools_r21.0.1-linux.zip',
Install build-tools from the cache too
2015-11-20 21:14:08 +01:00
'a8922e80d3dd0cf6df14b29a7862448fa111b48086c639168d4b18c92431f559'),
makebuildserver: use original names for downloaded SDK components It will make it a lot easier to manage the cache if we use the original file names, which often include the file version. This also changes the download process to be resumable if there is a partial file in the cache, and switches from calling wget on the command line to using the python libs 'requests' and 'clint' to provide a similar experience. While its not so important for this particular bit of code to use those libraries, I think those two will allow us to provide a better user experience throughout the whole of fdroidserver. In this case, it is already doing special tricks fetching the file size from the server before trying to download it. I suppose this code could instead check if the file exists, and if so, check the hash sum. I think that would be slower for most people since checking the hash on large files takes a noticeable about of time, while a HTTP HEAD request is pretty tiny.
2016-06-10 11:42:42 +02:00
('https://dl.google.com/android/repository/build-tools_r21.0.2-linux.zip',
Install build-tools from the cache too
2015-11-20 21:14:08 +01:00
'859b17a6b65d063dfd86c163489b736b12bdeecd9173fdddb3e9f32e0fe584b7'),
makebuildserver: use original names for downloaded SDK components It will make it a lot easier to manage the cache if we use the original file names, which often include the file version. This also changes the download process to be resumable if there is a partial file in the cache, and switches from calling wget on the command line to using the python libs 'requests' and 'clint' to provide a similar experience. While its not so important for this particular bit of code to use those libraries, I think those two will allow us to provide a better user experience throughout the whole of fdroidserver. In this case, it is already doing special tricks fetching the file size from the server before trying to download it. I suppose this code could instead check if the file exists, and if so, check the hash sum. I think that would be slower for most people since checking the hash on large files takes a noticeable about of time, while a HTTP HEAD request is pretty tiny.
2016-06-10 11:42:42 +02:00
('https://dl.google.com/android/repository/build-tools_r21.1-linux.zip',
Install build-tools from the cache too
2015-11-20 21:14:08 +01:00
'022a85b92360272379b2f04b8a4d727e754dbe7eb8ab5a9568190e33e480d8f1'),
makebuildserver: use original names for downloaded SDK components It will make it a lot easier to manage the cache if we use the original file names, which often include the file version. This also changes the download process to be resumable if there is a partial file in the cache, and switches from calling wget on the command line to using the python libs 'requests' and 'clint' to provide a similar experience. While its not so important for this particular bit of code to use those libraries, I think those two will allow us to provide a better user experience throughout the whole of fdroidserver. In this case, it is already doing special tricks fetching the file size from the server before trying to download it. I suppose this code could instead check if the file exists, and if so, check the hash sum. I think that would be slower for most people since checking the hash on large files takes a noticeable about of time, while a HTTP HEAD request is pretty tiny.
2016-06-10 11:42:42 +02:00
('https://dl.google.com/android/repository/build-tools_r21.1.1-linux.zip',
Install build-tools from the cache too
2015-11-20 21:14:08 +01:00
'29b612484de6b5cde0df6de655e413f7611b0557b440538397afa69b557e2f08'),
makebuildserver: use original names for downloaded SDK components It will make it a lot easier to manage the cache if we use the original file names, which often include the file version. This also changes the download process to be resumable if there is a partial file in the cache, and switches from calling wget on the command line to using the python libs 'requests' and 'clint' to provide a similar experience. While its not so important for this particular bit of code to use those libraries, I think those two will allow us to provide a better user experience throughout the whole of fdroidserver. In this case, it is already doing special tricks fetching the file size from the server before trying to download it. I suppose this code could instead check if the file exists, and if so, check the hash sum. I think that would be slower for most people since checking the hash on large files takes a noticeable about of time, while a HTTP HEAD request is pretty tiny.
2016-06-10 11:42:42 +02:00
('https://dl.google.com/android/repository/build-tools_r21.1.2-linux.zip',
Install build-tools from the cache too
2015-11-20 21:14:08 +01:00
'3f88efc2d5316fb73f547f35b472610eed5e6f3f56762750ddad1c7d1d81660d'),
makebuildserver: use original names for downloaded SDK components It will make it a lot easier to manage the cache if we use the original file names, which often include the file version. This also changes the download process to be resumable if there is a partial file in the cache, and switches from calling wget on the command line to using the python libs 'requests' and 'clint' to provide a similar experience. While its not so important for this particular bit of code to use those libraries, I think those two will allow us to provide a better user experience throughout the whole of fdroidserver. In this case, it is already doing special tricks fetching the file size from the server before trying to download it. I suppose this code could instead check if the file exists, and if so, check the hash sum. I think that would be slower for most people since checking the hash on large files takes a noticeable about of time, while a HTTP HEAD request is pretty tiny.
2016-06-10 11:42:42 +02:00
('https://dl.google.com/android/repository/build-tools_r22-linux.zip',
Install build-tools from the cache too
2015-11-20 21:14:08 +01:00
'061c021243f04c80c19568a6e3a027c00d8e269c9311d7bf07fced60fbde7bd5'),
makebuildserver: use original names for downloaded SDK components It will make it a lot easier to manage the cache if we use the original file names, which often include the file version. This also changes the download process to be resumable if there is a partial file in the cache, and switches from calling wget on the command line to using the python libs 'requests' and 'clint' to provide a similar experience. While its not so important for this particular bit of code to use those libraries, I think those two will allow us to provide a better user experience throughout the whole of fdroidserver. In this case, it is already doing special tricks fetching the file size from the server before trying to download it. I suppose this code could instead check if the file exists, and if so, check the hash sum. I think that would be slower for most people since checking the hash on large files takes a noticeable about of time, while a HTTP HEAD request is pretty tiny.
2016-06-10 11:42:42 +02:00
('https://dl.google.com/android/repository/build-tools_r22.0.1-linux.zip',
Install build-tools from the cache too
2015-11-20 21:14:08 +01:00
'91e5524bf227aad1135ddd10905518ac49f74797d33d48920dcf8364b9fde214'),
makebuildserver: use original names for downloaded SDK components It will make it a lot easier to manage the cache if we use the original file names, which often include the file version. This also changes the download process to be resumable if there is a partial file in the cache, and switches from calling wget on the command line to using the python libs 'requests' and 'clint' to provide a similar experience. While its not so important for this particular bit of code to use those libraries, I think those two will allow us to provide a better user experience throughout the whole of fdroidserver. In this case, it is already doing special tricks fetching the file size from the server before trying to download it. I suppose this code could instead check if the file exists, and if so, check the hash sum. I think that would be slower for most people since checking the hash on large files takes a noticeable about of time, while a HTTP HEAD request is pretty tiny.
2016-06-10 11:42:42 +02:00
('https://dl.google.com/android/repository/build-tools_r23-linux.zip',
Install build-tools from the cache too
2015-11-20 21:14:08 +01:00
'56bf4fc6c43638c55fef4a0937bad38281945725459841879b436c6922df786c'),
makebuildserver: use original names for downloaded SDK components It will make it a lot easier to manage the cache if we use the original file names, which often include the file version. This also changes the download process to be resumable if there is a partial file in the cache, and switches from calling wget on the command line to using the python libs 'requests' and 'clint' to provide a similar experience. While its not so important for this particular bit of code to use those libraries, I think those two will allow us to provide a better user experience throughout the whole of fdroidserver. In this case, it is already doing special tricks fetching the file size from the server before trying to download it. I suppose this code could instead check if the file exists, and if so, check the hash sum. I think that would be slower for most people since checking the hash on large files takes a noticeable about of time, while a HTTP HEAD request is pretty tiny.
2016-06-10 11:42:42 +02:00
('https://dl.google.com/android/repository/build-tools_r23.0.1-linux.zip',
Install build-tools from the cache too
2015-11-20 21:14:08 +01:00
'e56b3ef7b760ad06a7cee9b2d52ba7f43133dcecedfa5357f8845b3a80aeeecf'),
makebuildserver: use original names for downloaded SDK components It will make it a lot easier to manage the cache if we use the original file names, which often include the file version. This also changes the download process to be resumable if there is a partial file in the cache, and switches from calling wget on the command line to using the python libs 'requests' and 'clint' to provide a similar experience. While its not so important for this particular bit of code to use those libraries, I think those two will allow us to provide a better user experience throughout the whole of fdroidserver. In this case, it is already doing special tricks fetching the file size from the server before trying to download it. I suppose this code could instead check if the file exists, and if so, check the hash sum. I think that would be slower for most people since checking the hash on large files takes a noticeable about of time, while a HTTP HEAD request is pretty tiny.
2016-06-10 11:42:42 +02:00
('https://dl.google.com/android/repository/build-tools_r23.0.2-linux.zip',
Install build-tools from the cache too
2015-11-20 21:14:08 +01:00
'82754f551a6e36eaf516fbdd00c95ff0ccd19f81d1e134125b6ac4916f7ed9b6'),
makebuildserver: use original names for downloaded SDK components It will make it a lot easier to manage the cache if we use the original file names, which often include the file version. This also changes the download process to be resumable if there is a partial file in the cache, and switches from calling wget on the command line to using the python libs 'requests' and 'clint' to provide a similar experience. While its not so important for this particular bit of code to use those libraries, I think those two will allow us to provide a better user experience throughout the whole of fdroidserver. In this case, it is already doing special tricks fetching the file size from the server before trying to download it. I suppose this code could instead check if the file exists, and if so, check the hash sum. I think that would be slower for most people since checking the hash on large files takes a noticeable about of time, while a HTTP HEAD request is pretty tiny.
2016-06-10 11:42:42 +02:00
('https://dl.google.com/android/repository/build-tools_r23.0.3-linux.zip',
Add and switch to build-tools 23.0.3 Also bump to the CI image which contains it.
2016-04-05 11:49:30 +02:00
'd961663d4a9e128841751c0156548a347c882c081c83942e53788d8949bf34e1'),
all: add and switch to build-tools v24
2016-06-24 17:54:30 +02:00
('https://dl.google.com/android/repository/build-tools_r24-linux.zip',
'b4871f357224c5f660fd2bbee04d8c7d1c187eeddfd9702cc84503529e3b3724'),
makebs: add build-tools-24.0.1 Fixes #196.
2016-07-25 12:40:44 +02:00
('https://dl.google.com/android/repository/build-tools_r24.0.1-linux.zip',
new build-tools_r24.0.1.zip sha256, Google keeps changing it I validated this versus the Google repository XML downloaded from two separate paths from: https://dl.google.com/android/repository/repository-12.xml Using their SHA1: 84f18c392919a074fcbb9b1d967984e6b2fef8b4
2016-08-30 18:19:34 +02:00
'a38ac637db357a31e33e38248399cb0edcc15040dca041370da38b6daf50c84d'),
add build-tools_r24.0.2-linux.zip to ./makebuildserver
2016-08-30 18:19:49 +02:00
('https://dl.google.com/android/repository/build-tools_r24.0.2-linux.zip',
'924e29b8a189afbd119d44eae450fc0c9f197ed6f835df223931e45007987d95'),
makebs: add build-tools 24.0.3
2016-10-03 10:10:41 +02:00
('https://dl.google.com/android/repository/build-tools_r24.0.3-linux.zip',
'f2c02eb1d7e41ce314b5dac50440e7595380c4dd45b41ea1d7b0f86e49516927'),
makebs: bump m2repo, add api25 and build-tools 25
2016-10-23 17:47:19 +02:00
('https://dl.google.com/android/repository/build-tools_r25-linux.zip',
'74eb6931fd7a56859bd8e35d8d73ca8fe7ba6bfd4b7ffe560fe58b7354f2e3aa'),
makebs: add buildtools 25.0.1
2016-12-01 08:38:31 +01:00
('https://dl.google.com/android/repository/build-tools_r25.0.1-linux.zip',
'671b4e00f5b986c7355507c7024b725a4b4cadf11ca61fa5b1334ec6ea57d94f'),
Makebs: add build tools 25.0.2
2016-12-17 23:36:48 +01:00
('https://dl.google.com/android/repository/build-tools_r25.0.2-linux.zip',
'1d7ac9b6def16fb0254ec23c135c02dd9f6908073352a20315a017e4b2a904b0'),
makebs: update to ndk-r14b and build-tools-25.0.3
2017-05-26 16:54:06 +02:00
('https://dl.google.com/android/repository/build-tools_r25.0.3-linux.zip',
'152c1b187947edd10c65af8b279d40321ecc106106323e53df3608e578042d65'),
makebuildserver: bump tools, add api26 and build-tools 26
2017-06-09 19:35:04 +02:00
('https://dl.google.com/android/repository/build-tools_r26-linux.zip',
'7422682f92fb471d4aad4c053c9982a9a623377f9d5e4de7a73cd44ebf2f3c61'),
makebuildserver: update platform and build tool releases Update platform-25 to r03 Update platform-26 to r02 Add build-tools_r26.0.1 sha256 signatures locally computed after verifying the sha1 signature provided by the repository2-1.xml file. All these will be downloaded and installed in the buildserver through provision-android-sdk anyway but this adds them to the cache.
2017-08-17 00:41:23 +02:00
('https://dl.google.com/android/repository/build-tools_r26.0.1-linux.zip',
'c8617f25a7de2aeb9ddcacf1aeb413e053d5ed5ef4a3f31fe0ce21d4428ee0ea'),
add build-tools 26.0.2
2017-10-04 19:03:41 +02:00
('https://dl.google.com/android/repository/build-tools_r26.0.2-linux.zip',
'a752849fac85c4a7f9ea165ec8f367b0ebe8bbf6a1f33fc8605342be004231ce'),
makebuildserver: add build-tools_r26.0.3
2017-11-12 08:48:00 +01:00
('https://dl.google.com/android/repository/build-tools_r26.0.3-linux.zip',
'5c250c602b1657c4c70a6078925e9e01e5714526b707309bc1c708be6137a4db'),
makebuildserver: add build-tools_r27
2017-11-05 08:59:01 +01:00
('https://dl.google.com/android/repository/build-tools_r27-linux.zip',
'53d3322774a0bf229b372c0288108b4bfa27d74725fce8f0a3393e8df6b9ef22'),
makebuildserver: add build-tools_r27.0.1
2017-11-10 16:38:58 +01:00
('https://dl.google.com/android/repository/build-tools_r27.0.1-linux.zip',
'2e8e0946e93af50667ae02ef200e81c1ac2269b59f14955397245e9e441e8b1e'),
makebuildserver: add build-tools_r27.0.2
2017-12-15 12:34:40 +01:00
('https://dl.google.com/android/repository/build-tools_r27.0.2-linux.zip',
'e73674e065a93ffb05c30a15c8021c0d72ea7c3c206eb9020eb93e49e42ce851'),
name makebuildserver config the same as the script This keeps the numbers of names down to a minimum, and since the config is placed right next to the script, this keeps tab completion working nicely when the config file is in place. The old file name is still supported.
2016-01-20 20:38:18 +01:00
# the binaries that Google uses are here:
# https://android.googlesource.com/platform/tools/external/gradle/+/studio-1.5/
makebuildserver: use original names for downloaded SDK components It will make it a lot easier to manage the cache if we use the original file names, which often include the file version. This also changes the download process to be resumable if there is a partial file in the cache, and switches from calling wget on the command line to using the python libs 'requests' and 'clint' to provide a similar experience. While its not so important for this particular bit of code to use those libraries, I think those two will allow us to provide a better user experience throughout the whole of fdroidserver. In this case, it is already doing special tricks fetching the file size from the server before trying to download it. I suppose this code could instead check if the file exists, and if so, check the hash sum. I think that would be slower for most people since checking the hash on large files takes a noticeable about of time, while a HTTP HEAD request is pretty tiny.
2016-06-10 11:42:42 +02:00
('https://services.gradle.org/distributions/gradle-1.4-bin.zip',
Install gradle version 1.4, 1.6, 1.7, 1.8 and 1.9 to /opt/gradle
2014-01-09 15:44:58 +01:00
'cd99e85fbcd0ae8b99e81c9992a2f10cceb7b5f009c3720ef3a0078f4f92e94e'),
makebuildserver: use original names for downloaded SDK components It will make it a lot easier to manage the cache if we use the original file names, which often include the file version. This also changes the download process to be resumable if there is a partial file in the cache, and switches from calling wget on the command line to using the python libs 'requests' and 'clint' to provide a similar experience. While its not so important for this particular bit of code to use those libraries, I think those two will allow us to provide a better user experience throughout the whole of fdroidserver. In this case, it is already doing special tricks fetching the file size from the server before trying to download it. I suppose this code could instead check if the file exists, and if so, check the hash sum. I think that would be slower for most people since checking the hash on large files takes a noticeable about of time, while a HTTP HEAD request is pretty tiny.
2016-06-10 11:42:42 +02:00
('https://services.gradle.org/distributions/gradle-1.6-bin.zip',
Install gradle version 1.4, 1.6, 1.7, 1.8 and 1.9 to /opt/gradle
2014-01-09 15:44:58 +01:00
'de3e89d2113923dcc2e0def62d69be0947ceac910abd38b75ec333230183fac4'),
makebuildserver: use original names for downloaded SDK components It will make it a lot easier to manage the cache if we use the original file names, which often include the file version. This also changes the download process to be resumable if there is a partial file in the cache, and switches from calling wget on the command line to using the python libs 'requests' and 'clint' to provide a similar experience. While its not so important for this particular bit of code to use those libraries, I think those two will allow us to provide a better user experience throughout the whole of fdroidserver. In this case, it is already doing special tricks fetching the file size from the server before trying to download it. I suppose this code could instead check if the file exists, and if so, check the hash sum. I think that would be slower for most people since checking the hash on large files takes a noticeable about of time, while a HTTP HEAD request is pretty tiny.
2016-06-10 11:42:42 +02:00
('https://services.gradle.org/distributions/gradle-1.7-bin.zip',
Install gradle version 1.4, 1.6, 1.7, 1.8 and 1.9 to /opt/gradle
2014-01-09 15:44:58 +01:00
'360c97d51621b5a1ecf66748c718594e5f790ae4fbc1499543e0c006033c9d30'),
makebuildserver: use original names for downloaded SDK components It will make it a lot easier to manage the cache if we use the original file names, which often include the file version. This also changes the download process to be resumable if there is a partial file in the cache, and switches from calling wget on the command line to using the python libs 'requests' and 'clint' to provide a similar experience. While its not so important for this particular bit of code to use those libraries, I think those two will allow us to provide a better user experience throughout the whole of fdroidserver. In this case, it is already doing special tricks fetching the file size from the server before trying to download it. I suppose this code could instead check if the file exists, and if so, check the hash sum. I think that would be slower for most people since checking the hash on large files takes a noticeable about of time, while a HTTP HEAD request is pretty tiny.
2016-06-10 11:42:42 +02:00
('https://services.gradle.org/distributions/gradle-1.8-bin.zip',
Install gradle version 1.4, 1.6, 1.7, 1.8 and 1.9 to /opt/gradle
2014-01-09 15:44:58 +01:00
'a342bbfa15fd18e2482287da4959588f45a41b60910970a16e6d97959aea5703'),
makebuildserver: use original names for downloaded SDK components It will make it a lot easier to manage the cache if we use the original file names, which often include the file version. This also changes the download process to be resumable if there is a partial file in the cache, and switches from calling wget on the command line to using the python libs 'requests' and 'clint' to provide a similar experience. While its not so important for this particular bit of code to use those libraries, I think those two will allow us to provide a better user experience throughout the whole of fdroidserver. In this case, it is already doing special tricks fetching the file size from the server before trying to download it. I suppose this code could instead check if the file exists, and if so, check the hash sum. I think that would be slower for most people since checking the hash on large files takes a noticeable about of time, while a HTTP HEAD request is pretty tiny.
2016-06-10 11:42:42 +02:00
('https://services.gradle.org/distributions/gradle-1.9-bin.zip',
Update to gradle 1.9
2013-12-22 13:22:45 +01:00
'097ddc2bcbc9da2bb08cbf6bf8079585e35ad088bafd42e8716bc96405db98e9'),
makebuildserver: use original names for downloaded SDK components It will make it a lot easier to manage the cache if we use the original file names, which often include the file version. This also changes the download process to be resumable if there is a partial file in the cache, and switches from calling wget on the command line to using the python libs 'requests' and 'clint' to provide a similar experience. While its not so important for this particular bit of code to use those libraries, I think those two will allow us to provide a better user experience throughout the whole of fdroidserver. In this case, it is already doing special tricks fetching the file size from the server before trying to download it. I suppose this code could instead check if the file exists, and if so, check the hash sum. I think that would be slower for most people since checking the hash on large files takes a noticeable about of time, while a HTTP HEAD request is pretty tiny.
2016-06-10 11:42:42 +02:00
('https://services.gradle.org/distributions/gradle-1.10-bin.zip',
Add gradle 1.10 with the new plugin 0.8 release
2014-01-27 13:21:44 +01:00
'6e6db4fc595f27ceda059d23693b6f6848583950606112b37dfd0e97a0a0a4fe'),
makebuildserver: use original names for downloaded SDK components It will make it a lot easier to manage the cache if we use the original file names, which often include the file version. This also changes the download process to be resumable if there is a partial file in the cache, and switches from calling wget on the command line to using the python libs 'requests' and 'clint' to provide a similar experience. While its not so important for this particular bit of code to use those libraries, I think those two will allow us to provide a better user experience throughout the whole of fdroidserver. In this case, it is already doing special tricks fetching the file size from the server before trying to download it. I suppose this code could instead check if the file exists, and if so, check the hash sum. I think that would be slower for most people since checking the hash on large files takes a noticeable about of time, while a HTTP HEAD request is pretty tiny.
2016-06-10 11:42:42 +02:00
('https://services.gradle.org/distributions/gradle-1.11-bin.zip',
Add support for gradle 1.11 (and plugin ver 0.9)
2014-03-07 15:21:18 +01:00
'07e235df824964f0e19e73ea2327ce345c44bcd06d44a0123d29ab287fc34091'),
makebuildserver: use original names for downloaded SDK components It will make it a lot easier to manage the cache if we use the original file names, which often include the file version. This also changes the download process to be resumable if there is a partial file in the cache, and switches from calling wget on the command line to using the python libs 'requests' and 'clint' to provide a similar experience. While its not so important for this particular bit of code to use those libraries, I think those two will allow us to provide a better user experience throughout the whole of fdroidserver. In this case, it is already doing special tricks fetching the file size from the server before trying to download it. I suppose this code could instead check if the file exists, and if so, check the hash sum. I think that would be slower for most people since checking the hash on large files takes a noticeable about of time, while a HTTP HEAD request is pretty tiny.
2016-06-10 11:42:42 +02:00
('https://services.gradle.org/distributions/gradle-1.12-bin.zip',
Add gradle 1.12 for the plugin version 0.10 Also, use https for the gradle.org links
2014-05-20 16:40:43 +02:00
'8734b13a401f4311ee418173ed6ca8662d2b0a535be8ff2a43ecb1c13cd406ea'),
makebuildserver: use original names for downloaded SDK components It will make it a lot easier to manage the cache if we use the original file names, which often include the file version. This also changes the download process to be resumable if there is a partial file in the cache, and switches from calling wget on the command line to using the python libs 'requests' and 'clint' to provide a similar experience. While its not so important for this particular bit of code to use those libraries, I think those two will allow us to provide a better user experience throughout the whole of fdroidserver. In this case, it is already doing special tricks fetching the file size from the server before trying to download it. I suppose this code could instead check if the file exists, and if so, check the hash sum. I think that would be slower for most people since checking the hash on large files takes a noticeable about of time, while a HTTP HEAD request is pretty tiny.
2016-06-10 11:42:42 +02:00
('https://services.gradle.org/distributions/gradle-2.1-bin.zip',
Add support for gradle plugin version 0.13 via gradle 2.1
2014-10-07 16:36:10 +02:00
'3eee4f9ea2ab0221b89f8e4747a96d4554d00ae46d8d633f11cfda60988bf878'),
Add Gradle 2.2 and 2.14 to the buildserver 2.2 can be specifically requested by an app, and in some extreme cases those apps don't build with 2.2.1.
2016-06-24 18:03:48 +02:00
('https://services.gradle.org/distributions/gradle-2.2-bin.zip',
'91e5655fe11ef414449f218c4fa2985b3a49b7903c57556da109c84fa26e1dfb'),
makebuildserver: use original names for downloaded SDK components It will make it a lot easier to manage the cache if we use the original file names, which often include the file version. This also changes the download process to be resumable if there is a partial file in the cache, and switches from calling wget on the command line to using the python libs 'requests' and 'clint' to provide a similar experience. While its not so important for this particular bit of code to use those libraries, I think those two will allow us to provide a better user experience throughout the whole of fdroidserver. In this case, it is already doing special tricks fetching the file size from the server before trying to download it. I suppose this code could instead check if the file exists, and if so, check the hash sum. I think that would be slower for most people since checking the hash on large files takes a noticeable about of time, while a HTTP HEAD request is pretty tiny.
2016-06-10 11:42:42 +02:00
('https://services.gradle.org/distributions/gradle-2.2.1-bin.zip',
Add gradle 2.2.1 for plugin versions 1.0.X
2014-12-12 12:29:16 +01:00
'420aa50738299327b611c10b8304b749e8d3a579407ee9e755b15921d95ff418'),
makebuildserver: use original names for downloaded SDK components It will make it a lot easier to manage the cache if we use the original file names, which often include the file version. This also changes the download process to be resumable if there is a partial file in the cache, and switches from calling wget on the command line to using the python libs 'requests' and 'clint' to provide a similar experience. While its not so important for this particular bit of code to use those libraries, I think those two will allow us to provide a better user experience throughout the whole of fdroidserver. In this case, it is already doing special tricks fetching the file size from the server before trying to download it. I suppose this code could instead check if the file exists, and if so, check the hash sum. I think that would be slower for most people since checking the hash on large files takes a noticeable about of time, while a HTTP HEAD request is pretty tiny.
2016-06-10 11:42:42 +02:00
('https://services.gradle.org/distributions/gradle-2.3-bin.zip',
Add gradle 2.3 to the buildserver Fixes apps that explicitly require 2.3 via gradleVersion
2015-04-09 16:02:35 +02:00
'010dd9f31849abc3d5644e282943b1c1c355f8e2635c5789833979ce590a3774'),
makebuildserver: use original names for downloaded SDK components It will make it a lot easier to manage the cache if we use the original file names, which often include the file version. This also changes the download process to be resumable if there is a partial file in the cache, and switches from calling wget on the command line to using the python libs 'requests' and 'clint' to provide a similar experience. While its not so important for this particular bit of code to use those libraries, I think those two will allow us to provide a better user experience throughout the whole of fdroidserver. In this case, it is already doing special tricks fetching the file size from the server before trying to download it. I suppose this code could instead check if the file exists, and if so, check the hash sum. I think that would be slower for most people since checking the hash on large files takes a noticeable about of time, while a HTTP HEAD request is pretty tiny.
2016-06-10 11:42:42 +02:00
('https://services.gradle.org/distributions/gradle-2.4-bin.zip',
Add gradle 2.4
2015-06-18 16:57:00 +02:00
'c4eaecc621a81f567ded1aede4a5ddb281cc02a03a6a87c4f5502add8fc2f16f'),
makebuildserver: use original names for downloaded SDK components It will make it a lot easier to manage the cache if we use the original file names, which often include the file version. This also changes the download process to be resumable if there is a partial file in the cache, and switches from calling wget on the command line to using the python libs 'requests' and 'clint' to provide a similar experience. While its not so important for this particular bit of code to use those libraries, I think those two will allow us to provide a better user experience throughout the whole of fdroidserver. In this case, it is already doing special tricks fetching the file size from the server before trying to download it. I suppose this code could instead check if the file exists, and if so, check the hash sum. I think that would be slower for most people since checking the hash on large files takes a noticeable about of time, while a HTTP HEAD request is pretty tiny.
2016-06-10 11:42:42 +02:00
('https://services.gradle.org/distributions/gradle-2.5-bin.zip',
Add gradle 2.5, fixes #108
2015-08-13 20:28:01 +02:00
'3f953e0cb14bb3f9ebbe11946e84071547bf5dfd575d90cfe9cc4e788da38555'),
makebuildserver: use original names for downloaded SDK components It will make it a lot easier to manage the cache if we use the original file names, which often include the file version. This also changes the download process to be resumable if there is a partial file in the cache, and switches from calling wget on the command line to using the python libs 'requests' and 'clint' to provide a similar experience. While its not so important for this particular bit of code to use those libraries, I think those two will allow us to provide a better user experience throughout the whole of fdroidserver. In this case, it is already doing special tricks fetching the file size from the server before trying to download it. I suppose this code could instead check if the file exists, and if so, check the hash sum. I think that would be slower for most people since checking the hash on large files takes a noticeable about of time, while a HTTP HEAD request is pretty tiny.
2016-06-10 11:42:42 +02:00
('https://services.gradle.org/distributions/gradle-2.6-bin.zip',
Add gradle 2.6 to the buildserver
2015-08-18 22:40:21 +02:00
'18a98c560af231dfa0d3f8e0802c20103ae986f12428bb0a6f5396e8f14e9c83'),
makebuildserver: use original names for downloaded SDK components It will make it a lot easier to manage the cache if we use the original file names, which often include the file version. This also changes the download process to be resumable if there is a partial file in the cache, and switches from calling wget on the command line to using the python libs 'requests' and 'clint' to provide a similar experience. While its not so important for this particular bit of code to use those libraries, I think those two will allow us to provide a better user experience throughout the whole of fdroidserver. In this case, it is already doing special tricks fetching the file size from the server before trying to download it. I suppose this code could instead check if the file exists, and if so, check the hash sum. I think that would be slower for most people since checking the hash on large files takes a noticeable about of time, while a HTTP HEAD request is pretty tiny.
2016-06-10 11:42:42 +02:00
('https://services.gradle.org/distributions/gradle-2.7-bin.zip',
Fix shasum for gradle 2.7 (see a229afe1)
2015-10-08 15:16:16 +02:00
'cde43b90945b5304c43ee36e58aab4cc6fb3a3d5f9bd9449bb1709a68371cb06'),
makebuildserver: use original names for downloaded SDK components It will make it a lot easier to manage the cache if we use the original file names, which often include the file version. This also changes the download process to be resumable if there is a partial file in the cache, and switches from calling wget on the command line to using the python libs 'requests' and 'clint' to provide a similar experience. While its not so important for this particular bit of code to use those libraries, I think those two will allow us to provide a better user experience throughout the whole of fdroidserver. In this case, it is already doing special tricks fetching the file size from the server before trying to download it. I suppose this code could instead check if the file exists, and if so, check the hash sum. I think that would be slower for most people since checking the hash on large files takes a noticeable about of time, while a HTTP HEAD request is pretty tiny.
2016-06-10 11:42:42 +02:00
('https://services.gradle.org/distributions/gradle-2.8-bin.zip',
Fix gradle-2.8 checksum I once again did the -all.zip hash, not the -bin.zip one.
2015-10-25 10:48:17 +01:00
'a88db9c2f104defdaa8011c58cf6cda6c114298ae3695ecfb8beb30da3a903cb'),
makebuildserver: use original names for downloaded SDK components It will make it a lot easier to manage the cache if we use the original file names, which often include the file version. This also changes the download process to be resumable if there is a partial file in the cache, and switches from calling wget on the command line to using the python libs 'requests' and 'clint' to provide a similar experience. While its not so important for this particular bit of code to use those libraries, I think those two will allow us to provide a better user experience throughout the whole of fdroidserver. In this case, it is already doing special tricks fetching the file size from the server before trying to download it. I suppose this code could instead check if the file exists, and if so, check the hash sum. I think that would be slower for most people since checking the hash on large files takes a noticeable about of time, while a HTTP HEAD request is pretty tiny.
2016-06-10 11:42:42 +02:00
('https://services.gradle.org/distributions/gradle-2.9-bin.zip',
Add gradle 2.9 to the BS
2015-11-20 21:24:07 +01:00
'c9159ec4362284c0a38d73237e224deae6139cbde0db4f0f44e1c7691dd3de2f'),
makebuildserver: use original names for downloaded SDK components It will make it a lot easier to manage the cache if we use the original file names, which often include the file version. This also changes the download process to be resumable if there is a partial file in the cache, and switches from calling wget on the command line to using the python libs 'requests' and 'clint' to provide a similar experience. While its not so important for this particular bit of code to use those libraries, I think those two will allow us to provide a better user experience throughout the whole of fdroidserver. In this case, it is already doing special tricks fetching the file size from the server before trying to download it. I suppose this code could instead check if the file exists, and if so, check the hash sum. I think that would be slower for most people since checking the hash on large files takes a noticeable about of time, while a HTTP HEAD request is pretty tiny.
2016-06-10 11:42:42 +02:00
('https://services.gradle.org/distributions/gradle-2.10-bin.zip',
Add Gradle 2.10 to the BS
2016-01-05 09:15:54 +01:00
'66406247f745fc6f05ab382d3f8d3e120c339f34ef54b86f6dc5f6efc18fbb13'),
makebuildserver: use original names for downloaded SDK components It will make it a lot easier to manage the cache if we use the original file names, which often include the file version. This also changes the download process to be resumable if there is a partial file in the cache, and switches from calling wget on the command line to using the python libs 'requests' and 'clint' to provide a similar experience. While its not so important for this particular bit of code to use those libraries, I think those two will allow us to provide a better user experience throughout the whole of fdroidserver. In this case, it is already doing special tricks fetching the file size from the server before trying to download it. I suppose this code could instead check if the file exists, and if so, check the hash sum. I think that would be slower for most people since checking the hash on large files takes a noticeable about of time, while a HTTP HEAD request is pretty tiny.
2016-06-10 11:42:42 +02:00
('https://services.gradle.org/distributions/gradle-2.11-bin.zip',
Add gradle 2.11
2016-02-15 12:44:07 +01:00
'8d7437082356c9fd6309a4479c8db307673965546daea445c6c72759cd6b1ed6'),
makebuildserver: use original names for downloaded SDK components It will make it a lot easier to manage the cache if we use the original file names, which often include the file version. This also changes the download process to be resumable if there is a partial file in the cache, and switches from calling wget on the command line to using the python libs 'requests' and 'clint' to provide a similar experience. While its not so important for this particular bit of code to use those libraries, I think those two will allow us to provide a better user experience throughout the whole of fdroidserver. In this case, it is already doing special tricks fetching the file size from the server before trying to download it. I suppose this code could instead check if the file exists, and if so, check the hash sum. I think that would be slower for most people since checking the hash on large files takes a noticeable about of time, while a HTTP HEAD request is pretty tiny.
2016-06-10 11:42:42 +02:00
('https://services.gradle.org/distributions/gradle-2.12-bin.zip',
Add Gradle 2.12 to the buildserver
2016-04-05 11:49:15 +02:00
'e77064981906cd0476ff1e0de3e6fef747bd18e140960f1915cca8ff6c33ab5c'),
makebuildserver: use original names for downloaded SDK components It will make it a lot easier to manage the cache if we use the original file names, which often include the file version. This also changes the download process to be resumable if there is a partial file in the cache, and switches from calling wget on the command line to using the python libs 'requests' and 'clint' to provide a similar experience. While its not so important for this particular bit of code to use those libraries, I think those two will allow us to provide a better user experience throughout the whole of fdroidserver. In this case, it is already doing special tricks fetching the file size from the server before trying to download it. I suppose this code could instead check if the file exists, and if so, check the hash sum. I think that would be slower for most people since checking the hash on large files takes a noticeable about of time, while a HTTP HEAD request is pretty tiny.
2016-06-10 11:42:42 +02:00
('https://services.gradle.org/distributions/gradle-2.13-bin.zip',
Add gradle 2.13
2016-05-26 16:45:54 +02:00
'0f665ec6a5a67865faf7ba0d825afb19c26705ea0597cec80dd191b0f2cbb664'),
Add Gradle 2.2 and 2.14 to the buildserver 2.2 can be specifically requested by an app, and in some extreme cases those apps don't build with 2.2.1.
2016-06-24 18:03:48 +02:00
('https://services.gradle.org/distributions/gradle-2.14-bin.zip',
'993b4f33b652c689e9721917d8e021cab6bbd3eae81b39ab2fd46fdb19a928d5'),
Add gradle 2.14.1
2016-08-12 19:52:32 +02:00
('https://services.gradle.org/distributions/gradle-2.14.1-bin.zip',
'cfc61eda71f2d12a572822644ce13d2919407595c2aec3e3566d2aab6f97ef39'),
makebs: add gradle 3.0
2016-08-18 17:53:27 +02:00
('https://services.gradle.org/distributions/gradle-3.0-bin.zip',
'39c906941a474444afbddc38144ed44166825acb0a57b0551dddb04bbf157f80'),
makebs: add gradle 3.1
2016-09-23 07:50:52 +02:00
('https://services.gradle.org/distributions/gradle-3.1-bin.zip',
'c7de3442432253525902f7e8d7eac8b5fd6ce1623f96d76916af6d0e383010fc'),
makebs: add gradle 3.2
2016-11-15 21:56:15 +01:00
('https://services.gradle.org/distributions/gradle-3.2-bin.zip',
'5321b36837226dc0377047a328f12010f42c7bf88ee4a3b1cee0c11040082935'),
Add gradle 3.2.1
2016-12-02 06:51:38 +01:00
('https://services.gradle.org/distributions/gradle-3.2.1-bin.zip',
'9843a3654d3e57dce54db06d05f18b664b95c22bf90c6becccb61fc63ce60689'),
makebs: install gradle 3.3
2017-01-08 21:13:24 +01:00
('https://services.gradle.org/distributions/gradle-3.3-bin.zip',
'c58650c278d8cf0696cab65108ae3c8d95eea9c1938e0eb8b997095d5ca9a292'),
makebs: add gradle 3.4
2017-02-26 03:23:49 +01:00
('https://services.gradle.org/distributions/gradle-3.4-bin.zip',
'72d0cd4dcdd5e3be165eb7cd7bbd25cf8968baf400323d9ab1bba622c3f72205'),
makebs: add gradle 3.4.1
2017-03-10 14:29:36 +01:00
('https://services.gradle.org/distributions/gradle-3.4.1-bin.zip',
'db1db193d479cc1202be843f17e4526660cfb0b21b57d62f3a87f88c878af9b2'),
makebs: add gradle 3.5
2017-04-15 15:08:13 +02:00
('https://services.gradle.org/distributions/gradle-3.5-bin.zip',
'0b7450798c190ff76b9f9a3d02e18b33d94553f708ebc08ebe09bdf99111d110'),
makebuildserver: add Gradle 3.5.1, 4.0.1, 4.0.2 and 4.1 Android Gradle plugin 3.0.0 requires Gradle 4.1.
2017-08-07 13:46:08 +02:00
('https://services.gradle.org/distributions/gradle-3.5.1-bin.zip',
'8dce35f52d4c7b4a4946df73aa2830e76ba7148850753d8b5e94c5dc325ceef8'),
makebs: add gradle4.0
2017-06-17 01:39:44 +02:00
('https://services.gradle.org/distributions/gradle-4.0-bin.zip',
'56bd2dde29ba2a93903c557da1745cafd72cdd8b6b0b83c05a40ed7896b79dfe'),
makebuildserver: add Gradle 3.5.1, 4.0.1, 4.0.2 and 4.1 Android Gradle plugin 3.0.0 requires Gradle 4.1.
2017-08-07 13:46:08 +02:00
('https://services.gradle.org/distributions/gradle-4.0.1-bin.zip',
'd717e46200d1359893f891dab047fdab98784143ac76861b53c50dbd03b44fd4'),
('https://services.gradle.org/distributions/gradle-4.0.2-bin.zip',
'79ac421342bd11f6a4f404e0988baa9c1f5fabf07e3c6fa65b0c15c1c31dda22'),
('https://services.gradle.org/distributions/gradle-4.1-bin.zip',
'd55dfa9cfb5a3da86a1c9e75bb0b9507f9a8c8c100793ccec7beb6e259f9ed43'),
makebuildserver: add Gradle 4.2
2017-09-23 09:04:39 +02:00
('https://services.gradle.org/distributions/gradle-4.2-bin.zip',
'515dd63d32e55a9c05667809c5e40a947529de3054444ad274b3b75af5582eae'),
makebuildserver: add Gradle 4.2.1 N.B. Actual address of the distribution server is downloads.gradle.org; services.gradle.org redirects to it.
2017-10-08 20:10:43 +02:00
('https://downloads.gradle.org/distributions/gradle-4.2.1-bin.zip',
'b551cc04f2ca51c78dd14edb060621f0e5439bdfafa6fd167032a09ac708fbc0'),
makebuildserver: add Gradle 4.3
2017-10-31 15:19:57 +01:00
('https://downloads.gradle.org/distributions/gradle-4.3-bin.zip',
'8dcbf44eef92575b475dcb1ce12b5f19d38dc79e84c662670248dc8b8247654c'),
makebuildserver: add Gradle 4.3.1
2017-11-14 08:36:12 +01:00
('https://downloads.gradle.org/distributions/gradle-4.3.1-bin.zip',
'15ebe098ce0392a2d06d252bff24143cc88c4e963346582c8d88814758d93ac7'),
makebuildserver: add Gradle 4.4
2017-12-15 12:32:34 +01:00
('https://downloads.gradle.org/distributions/gradle-4.4-bin.zip',
'fa4873ae2c7f5e8c02ec6948ba95848cedced6134772a0169718eadcb39e0a2f'),
makebuildserver: add Gradle 4.4.1
2018-01-10 19:31:59 +01:00
('https://downloads.gradle.org/distributions/gradle-4.4.1-bin.zip',
'e7cf7d1853dfc30c1c44f571d3919eeeedef002823b66b6a988d27e919686389'),
buildserver: remove last vestige of 32-bit buildserver
2016-07-04 17:14:06 +02:00
('https://dl.google.com/android/ndk/android-ndk-r10e-linux-x86_64.bin',
'102d6723f67ff1384330d12c45854315d6452d6510286f4e5891e00a5a8f1d5a'),
('https://dl.google.com/android/ndk/android-ndk-r9b-linux-x86_64.tar.bz2',
'8956e9efeea95f49425ded8bb697013b66e162b064b0f66b5c75628f76e0f532'),
('https://dl.google.com/android/ndk/android-ndk-r9b-linux-x86_64-legacy-toolchains.tar.bz2',
'de93a394f7c8f3436db44568648f87738a8d09801a52f459dcad3fc047e045a1'),
Add ndk r11c
2016-08-12 20:24:54 +02:00
('https://dl.google.com/android/repository/android-ndk-r11c-linux-x86_64.zip',
'ba85dbe4d370e4de567222f73a3e034d85fc3011b3cbd90697f3e8dcace3ad94'),
Re-add NDK r12b, now fixed.
2016-08-02 11:09:32 +02:00
('https://dl.google.com/android/repository/android-ndk-r12b-linux-x86_64.zip',
'eafae2d614e5475a3bcfd7c5f201db5b963cc1290ee3e8ae791ff0c66757781e'),
Add ndk 13b
2016-11-25 21:10:44 +01:00
('https://dl.google.com/android/repository/android-ndk-r13b-linux-x86_64.zip',
'3524d7f8fca6dc0d8e7073a7ab7f76888780a22841a6641927123146c3ffd29c'),
makebs: update to ndk-r14b and build-tools-25.0.3
2017-05-26 16:54:06 +02:00
('https://dl.google.com/android/repository/android-ndk-r14b-linux-x86_64.zip',
'0ecc2017802924cf81fffc0f51d342e3e69de6343da892ac9fa1cd79bc106024'),
makebuildserver: upgrade NDK r15b to r15c
2017-08-11 09:19:40 +02:00
('https://dl.google.com/android/repository/android-ndk-r15c-linux-x86_64.zip',
'f01788946733bf6294a36727b99366a18369904eb068a599dde8cca2c1d2ba3c'),
makebuildserver: add ndk r16
2017-11-26 17:08:26 +01:00
('https://dl.google.com/android/repository/android-ndk-r16-linux-x86_64.zip',
'a8550b81771c67cc6ab7b479a6918d29aa78de3482901762b4f9e0132cd9672e'),
Add qt sdk support
2016-09-24 14:10:55 +02:00
('https://download.qt.io/official_releases/qt/5.7/5.7.0/qt-opensource-linux-x64-android-5.7.0.run',
'f7e55b7970e59bdaabb88cb7afc12e9061e933992bda2f076f52600358644586'),
Apply some autopep8-python2 suggestions
2014-12-31 16:42:26 +01:00
]
Fix pep8 warnings in makebuildserver
2014-05-31 23:00:27 +02:00
A few more PEP8 fixes
2014-05-07 16:13:22 +02:00
Check sha256 sums via hashlib
2014-01-10 19:54:53 +01:00
def sha256_for_file(path):
Fix python2/makebuildserver breakage
2016-03-13 21:49:38 +01:00
with open(path, 'rb') as f:
Check sha256 sums via hashlib
2014-01-10 19:54:53 +01:00
s = hashlib.sha256()
while True:
data = f.read(4096)
if not data:
break
s.update(data)
return s.hexdigest()
buildserver: remove my silly mistake Should have done this through a merge request... code review!
2017-03-14 20:14:06 +01:00
buildserver: run_via_vagrant_ssh() to run cmds via python-vagrant This moves the last vagrant call in a subprocess.
2016-06-23 16:38:39 +02:00
def run_via_vagrant_ssh(v, cmdlist):
if (isinstance(cmdlist, str) or isinstance(cmdlist, bytes)):
cmd = cmdlist
else:
cmd = ' '.join(cmdlist)
v._run_vagrant_command(['ssh', '-c', cmd])
makebuildserver debugging log details
2017-03-24 20:04:50 +01:00
def update_cache(cachedir, cachefiles):
buildserver: move code into main() method to always stop thread By running the whole program in a main() function, it can be wrapped in try/finally in order to stop the background display thread. This is also done in ./fdroid, its standard practice for Python CLI utilities.
2016-09-25 18:55:29 +02:00
for srcurl, shasum in cachefiles:
filename = os.path.basename(srcurl)
local_filename = os.path.join(cachedir, filename)
makebuildserver: use original names for downloaded SDK components It will make it a lot easier to manage the cache if we use the original file names, which often include the file version. This also changes the download process to be resumable if there is a partial file in the cache, and switches from calling wget on the command line to using the python libs 'requests' and 'clint' to provide a similar experience. While its not so important for this particular bit of code to use those libraries, I think those two will allow us to provide a better user experience throughout the whole of fdroidserver. In this case, it is already doing special tricks fetching the file size from the server before trying to download it. I suppose this code could instead check if the file exists, and if so, check the hash sum. I think that would be slower for most people since checking the hash on large files takes a noticeable about of time, while a HTTP HEAD request is pretty tiny.
2016-06-10 11:42:42 +02:00
buildserver: move code into main() method to always stop thread By running the whole program in a main() function, it can be wrapped in try/finally in order to stop the background display thread. This is also done in ./fdroid, its standard practice for Python CLI utilities.
2016-09-25 18:55:29 +02:00
if os.path.exists(local_filename):
local_length = os.path.getsize(local_filename)
else:
local_length = -1
resume_header = {}
download = True
try:
r = requests.head(srcurl, allow_redirects=True, timeout=60)
if r.status_code == 200:
content_length = int(r.headers.get('content-length'))
else:
content_length = local_length # skip the download
except requests.exceptions.RequestException as e:
content_length = local_length # skip the download
makebuildserver debugging log details
2017-03-24 20:04:50 +01:00
logger.warn('%s', e)
buildserver: move code into main() method to always stop thread By running the whole program in a main() function, it can be wrapped in try/finally in order to stop the background display thread. This is also done in ./fdroid, its standard practice for Python CLI utilities.
2016-09-25 18:55:29 +02:00
if local_length == content_length:
download = False
elif local_length > content_length:
makebuildserver debugging log details
2017-03-24 20:04:50 +01:00
logger.info('deleting corrupt file from cache: %s', local_filename)
buildserver: move code into main() method to always stop thread By running the whole program in a main() function, it can be wrapped in try/finally in order to stop the background display thread. This is also done in ./fdroid, its standard practice for Python CLI utilities.
2016-09-25 18:55:29 +02:00
os.remove(local_filename)
makebuildserver debugging log details
2017-03-24 20:04:50 +01:00
logger.info("Downloading %s to cache", filename)
buildserver: move code into main() method to always stop thread By running the whole program in a main() function, it can be wrapped in try/finally in order to stop the background display thread. This is also done in ./fdroid, its standard practice for Python CLI utilities.
2016-09-25 18:55:29 +02:00
elif local_length > -1 and local_length < content_length:
makebuildserver debugging log details
2017-03-24 20:04:50 +01:00
logger.info("Resuming download of %s", local_filename)
buildserver: move code into main() method to always stop thread By running the whole program in a main() function, it can be wrapped in try/finally in order to stop the background display thread. This is also done in ./fdroid, its standard practice for Python CLI utilities.
2016-09-25 18:55:29 +02:00
resume_header = {'Range': 'bytes=%d-%d' % (local_length, content_length)}
else:
makebuildserver debugging log details
2017-03-24 20:04:50 +01:00
logger.info("Downloading %s to cache", filename)
makebuildserver: use original names for downloaded SDK components It will make it a lot easier to manage the cache if we use the original file names, which often include the file version. This also changes the download process to be resumable if there is a partial file in the cache, and switches from calling wget on the command line to using the python libs 'requests' and 'clint' to provide a similar experience. While its not so important for this particular bit of code to use those libraries, I think those two will allow us to provide a better user experience throughout the whole of fdroidserver. In this case, it is already doing special tricks fetching the file size from the server before trying to download it. I suppose this code could instead check if the file exists, and if so, check the hash sum. I think that would be slower for most people since checking the hash on large files takes a noticeable about of time, while a HTTP HEAD request is pretty tiny.
2016-06-10 11:42:42 +02:00
buildserver: move code into main() method to always stop thread By running the whole program in a main() function, it can be wrapped in try/finally in order to stop the background display thread. This is also done in ./fdroid, its standard practice for Python CLI utilities.
2016-09-25 18:55:29 +02:00
if download:
r = requests.get(srcurl, headers=resume_header,
makebuildserver: verify https all requests to avoid urllib3 warnings According to http://docs.python-requests.org/en/master/user/advanced/#ca-certificates python-requests >= 2.4.0 will use the certifi provided certificates. We already recommend installing python3-certifi in the docs. On debian requests is patched to use the system trust store instead.
2017-08-17 00:49:53 +02:00
stream=True, allow_redirects=True)
makebuildserver: use original names for downloaded SDK components It will make it a lot easier to manage the cache if we use the original file names, which often include the file version. This also changes the download process to be resumable if there is a partial file in the cache, and switches from calling wget on the command line to using the python libs 'requests' and 'clint' to provide a similar experience. While its not so important for this particular bit of code to use those libraries, I think those two will allow us to provide a better user experience throughout the whole of fdroidserver. In this case, it is already doing special tricks fetching the file size from the server before trying to download it. I suppose this code could instead check if the file exists, and if so, check the hash sum. I think that would be slower for most people since checking the hash on large files takes a noticeable about of time, while a HTTP HEAD request is pretty tiny.
2016-06-10 11:42:42 +02:00
content_length = int(r.headers.get('content-length'))
buildserver: move code into main() method to always stop thread By running the whole program in a main() function, it can be wrapped in try/finally in order to stop the background display thread. This is also done in ./fdroid, its standard practice for Python CLI utilities.
2016-09-25 18:55:29 +02:00
with open(local_filename, 'ab') as f:
for chunk in progress.bar(r.iter_content(chunk_size=65536),
expected_size=(content_length / 65536) + 1):
if chunk: # filter out keep-alive new chunks
f.write(chunk)
v = sha256_for_file(local_filename)
if v == shasum:
makebuildserver debugging log details
2017-03-24 20:04:50 +01:00
logger.info("\t...shasum verified for %s", local_filename)
Install gradle 1.7 to buildserver (untested, may need some extra config)
2013-08-09 20:37:13 +02:00
else:
makebuildserver debugging log details
2017-03-24 20:04:50 +01:00
logger.critical("Invalid shasum of '%s' detected for %s", v, local_filename)
buildserver: move code into main() method to always stop thread By running the whole program in a main() function, it can be wrapped in try/finally in order to stop the background display thread. This is also done in ./fdroid, its standard practice for Python CLI utilities.
2016-09-25 18:55:29 +02:00
os.remove(local_filename)
sys.exit(1)
buildserver: send config to vagrant via YAML file Python can easily output dicts as YAML, and a Vagrantfile is a ruby script, which can easily read YAML. Going this route means that Vagrantfile can ultimately be committed to git, and the configuration will happen all via Python dicts output as YAML. That makes it drastically easier to follow the code, and to make modifications.
2016-07-04 20:22:00 +02:00
makebuildserver debugging log details
2017-03-24 20:04:50 +01:00
makebuildserver vbox logmessage fix
2017-04-13 12:49:14 +02:00
def debug_log_vagrant_vm(vm_dir, config):
makebuildserver debugging log details
2017-03-24 20:04:50 +01:00
if options.verbosity >= 3:
_vagrant_dir = os.path.join(vm_dir, '.vagrant')
logger.debug('check %s dir exists? -> %r', _vagrant_dir, os.path.isdir(_vagrant_dir))
logger.debug('> vagrant status')
subprocess.call(['vagrant', 'status'], cwd=vm_dir)
logger.debug('> vagrant box list')
subprocess.call(['vagrant', 'box', 'list'])
makebuildserver vbox logmessage fix
2017-04-13 12:49:14 +02:00
if config['vm_provider'] == 'libvirt':
logger.debug('> virsh -c qmeu:///system list --all')
subprocess.call(['virsh', '-c', 'qemu:///system', 'list', '--all'])
makebuildserver: make config['domain'] local var where its needed
2017-05-23 11:59:53 +02:00
domain = 'buildserver_default'
logger.debug('> virsh -c qemu:///system snapshot-list %s', domain)
subprocess.call(['virsh', '-c', 'qemu:///system', 'snapshot-list', domain])
makebuildserver debugging log details
2017-03-24 20:04:50 +01:00
def main():
global cachedir, cachefiles, config, tail
if options.skip_cache_update:
logger.info('skipping cache update and verification...')
else:
update_cache(cachedir, cachefiles)
buildserver: move code into main() method to always stop thread By running the whole program in a main() function, it can be wrapped in try/finally in order to stop the background display thread. This is also done in ./fdroid, its standard practice for Python CLI utilities.
2016-09-25 18:55:29 +02:00
local_qt_filename = os.path.join(cachedir, 'qt-opensource-linux-x64-android-5.7.0.run')
makebuildserver debugging log details
2017-03-24 20:04:50 +01:00
logger.info("Setting executable bit for %s", local_qt_filename)
buildserver: move code into main() method to always stop thread By running the whole program in a main() function, it can be wrapped in try/finally in order to stop the background display thread. This is also done in ./fdroid, its standard practice for Python CLI utilities.
2016-09-25 18:55:29 +02:00
os.chmod(local_qt_filename, 0o755)
buildserver: replace custom code with python-vagrant I ran into some annoying issues with UTF-8 output in the vagrant logs, and it was hard to solve. So I switched to using python-vagrant, which handles it all for us. Its been around since 2012, has a number of contributors, and is still actively maintained, so it seems like a good bet. I also packaged it for Debian, including a backport in jessie-backports. On Debian/jessie, do `apt-get install python3-vagrant/jessie-backports`
2016-09-27 08:49:32 +02:00
buildserver: move code into main() method to always stop thread By running the whole program in a main() function, it can be wrapped in try/finally in order to stop the background display thread. This is also done in ./fdroid, its standard practice for Python CLI utilities.
2016-09-25 18:55:29 +02:00
# use VirtualBox software virtualization if hardware is not available,
# like if this is being run in kvm or some other VM platform, like
# http://jenkins.debian.net, the values are 'on' or 'off'
if sys.platform.startswith('darwin'):
# all < 10 year old Macs work, and OSX servers as VM host are very
# rare, but this could also be auto-detected if someone codes it
config['hwvirtex'] = 'on'
makebuildserver debugging log details
2017-03-24 20:04:50 +01:00
logger.info('platform is darwnin -> hwvirtex = \'on\'')
buildserver: move code into main() method to always stop thread By running the whole program in a main() function, it can be wrapped in try/finally in order to stop the background display thread. This is also done in ./fdroid, its standard practice for Python CLI utilities.
2016-09-25 18:55:29 +02:00
elif os.path.exists('/proc/cpuinfo'):
with open('/proc/cpuinfo') as f:
contents = f.read()
if 'vmx' in contents or 'svm' in contents:
config['hwvirtex'] = 'on'
makebuildserver debugging log details
2017-03-24 20:04:50 +01:00
logger.info('found \'vmx\' or \'svm\' in /proc/cpuinfo -> hwvirtex = \'on\'')
buildserver: move code into main() method to always stop thread By running the whole program in a main() function, it can be wrapped in try/finally in order to stop the background display thread. This is also done in ./fdroid, its standard practice for Python CLI utilities.
2016-09-25 18:55:29 +02:00
serverdir = os.path.join(os.getcwd(), 'buildserver')
logfilename = os.path.join(serverdir, 'up.log')
if not os.path.exists(logfilename):
open(logfilename, 'a').close() # create blank file
log_cm = vagrant.make_file_cm(logfilename)
v = vagrant.Vagrant(root=serverdir, out_cm=log_cm, err_cm=log_cm)
makebuildserver debugging log details
2017-03-24 20:04:50 +01:00
if options.verbosity >= 2:
buildserver: move code into main() method to always stop thread By running the whole program in a main() function, it can be wrapped in try/finally in order to stop the background display thread. This is also done in ./fdroid, its standard practice for Python CLI utilities.
2016-09-25 18:55:29 +02:00
tail = fdroidserver.tail.Tail(logfilename)
tail.start()
use configured vm provider in when calling destroy in makebuildserver
2017-03-26 03:15:33 +02:00
vm = fdroidserver.vmtools.get_build_vm(serverdir, provider=config['vm_provider'])
buildserver: move code into main() method to always stop thread By running the whole program in a main() function, it can be wrapped in try/finally in order to stop the background display thread. This is also done in ./fdroid, its standard practice for Python CLI utilities.
2016-09-25 18:55:29 +02:00
if options.clean:
overhauled and moved destroying builder vm to vmtools.py
2017-03-26 01:41:39 +01:00
vm.destroy()
buildserver: move code into main() method to always stop thread By running the whole program in a main() function, it can be wrapped in try/finally in order to stop the background display thread. This is also done in ./fdroid, its standard practice for Python CLI utilities.
2016-09-25 18:55:29 +02:00
# Check against the existing Vagrantfile.yaml, and if they differ, we
# need to create a new box:
vf = os.path.join(serverdir, 'Vagrantfile.yaml')
writevf = True
if os.path.exists(vf):
makebuildserver debugging log details
2017-03-24 20:04:50 +01:00
logger.info('Halting %s', serverdir)
buildserver: move code into main() method to always stop thread By running the whole program in a main() function, it can be wrapped in try/finally in order to stop the background display thread. This is also done in ./fdroid, its standard practice for Python CLI utilities.
2016-09-25 18:55:29 +02:00
v.halt()
with open(vf, 'r', encoding='utf-8') as f:
oldconfig = yaml.load(f)
if config != oldconfig:
makebuildserver debugging log details
2017-03-24 20:04:50 +01:00
logger.info("Server configuration has changed, rebuild from scratch is required")
overhauled and moved destroying builder vm to vmtools.py
2017-03-26 01:41:39 +01:00
vm.destroy()
buildserver: move code into main() method to always stop thread By running the whole program in a main() function, it can be wrapped in try/finally in order to stop the background display thread. This is also done in ./fdroid, its standard practice for Python CLI utilities.
2016-09-25 18:55:29 +02:00
else:
makebuildserver debugging log details
2017-03-24 20:04:50 +01:00
logger.info("Re-provisioning existing server")
buildserver: move code into main() method to always stop thread By running the whole program in a main() function, it can be wrapped in try/finally in order to stop the background display thread. This is also done in ./fdroid, its standard practice for Python CLI utilities.
2016-09-25 18:55:29 +02:00
writevf = False
else:
makebuildserver debugging log details
2017-03-24 20:04:50 +01:00
logger.info("No existing server - building from scratch")
buildserver: move code into main() method to always stop thread By running the whole program in a main() function, it can be wrapped in try/finally in order to stop the background display thread. This is also done in ./fdroid, its standard practice for Python CLI utilities.
2016-09-25 18:55:29 +02:00
if writevf:
with open(vf, 'w', encoding='utf-8') as f:
yaml.dump(config, f)
buildserver: display verbose logging in a background tail
2016-06-20 11:33:46 +02:00
buildserver: replace custom code with python-vagrant I ran into some annoying issues with UTF-8 output in the vagrant logs, and it was hard to solve. So I switched to using python-vagrant, which handles it all for us. Its been around since 2012, has a number of contributors, and is still actively maintained, so it seems like a good bet. I also packaged it for Debian, including a backport in jessie-backports. On Debian/jessie, do `apt-get install python3-vagrant/jessie-backports`
2016-09-27 08:49:32 +02:00
if config['vm_provider'] == 'libvirt':
buildserver: move code into main() method to always stop thread By running the whole program in a main() function, it can be wrapped in try/finally in order to stop the background display thread. This is also done in ./fdroid, its standard practice for Python CLI utilities.
2016-09-25 18:55:29 +02:00
found_basebox = False
needs_mutate = False
for box in v.box_list():
if box.name == config['basebox']:
found_basebox = True
if box.provider != 'libvirt':
needs_mutate = True
continue
if not found_basebox:
if isinstance(config['baseboxurl'], str):
baseboxurl = config['baseboxurl']
else:
baseboxurl = config['baseboxurl'][0]
makebuildserver debugging log details
2017-03-24 20:04:50 +01:00
logger.info('Adding %s from %s', config['basebox'], baseboxurl)
buildserver: move code into main() method to always stop thread By running the whole program in a main() function, it can be wrapped in try/finally in order to stop the background display thread. This is also done in ./fdroid, its standard practice for Python CLI utilities.
2016-09-25 18:55:29 +02:00
v.box_add(config['basebox'], baseboxurl)
needs_mutate = True
if needs_mutate:
makebuildserver debugging log details
2017-03-24 20:04:50 +01:00
logger.info('Converting %s to libvirt format', config['basebox'])
buildserver: move code into main() method to always stop thread By running the whole program in a main() function, it can be wrapped in try/finally in order to stop the background display thread. This is also done in ./fdroid, its standard practice for Python CLI utilities.
2016-09-25 18:55:29 +02:00
v._call_vagrant_command(['mutate', config['basebox'], 'libvirt'])
makebuildserver debugging log details
2017-03-24 20:04:50 +01:00
logger.info('Removing virtualbox format copy of %s', config['basebox'])
buildserver: move code into main() method to always stop thread By running the whole program in a main() function, it can be wrapped in try/finally in order to stop the background display thread. This is also done in ./fdroid, its standard practice for Python CLI utilities.
2016-09-25 18:55:29 +02:00
v.box_remove(config['basebox'], 'virtualbox')
makebuildserver debugging log details
2017-03-24 20:04:50 +01:00
logger.info("Configuring build server VM")
makebuildserver vbox logmessage fix
2017-04-13 12:49:14 +02:00
debug_log_vagrant_vm(serverdir, config)
makebuildserver debug logging when initial provisioning fails
2017-03-25 01:56:20 +01:00
try:
refactored kvm_package to vmtools
2017-05-22 17:12:34 +02:00
v.up(provision=True)
makebuildserver: remove reference to f-droid.org/jessie64.box Also improve error message when the initial vagrant up fails, this is most likely due to the missing jessie64.box file.
2017-10-20 04:38:39 +02:00
except subprocess.CalledProcessError:
makebuildserver vbox logmessage fix
2017-04-13 12:49:14 +02:00
debug_log_vagrant_vm(serverdir, config)
makebuildserver: remove reference to f-droid.org/jessie64.box Also improve error message when the initial vagrant up fails, this is most likely due to the missing jessie64.box file.
2017-10-20 04:38:39 +02:00
logger.error("'vagrant up' failed, is the base box missing?")
makebuildserver debug logging when initial provisioning fails
2017-03-25 01:56:20 +01:00
sys.exit(1)
buildserver: move code into main() method to always stop thread By running the whole program in a main() function, it can be wrapped in try/finally in order to stop the background display thread. This is also done in ./fdroid, its standard practice for Python CLI utilities.
2016-09-25 18:55:29 +02:00
buildserver: add copy_caches_from_host config option For people using slow, expensive, and/or flaky internet, liberal use of caching can make a huge difference. The restricted environment of the gpjenkins box has been a good test environment for this (Tor-only, whitelist of allowed IPs to visit, home internet connection).
2016-06-23 17:12:06 +02:00
if config['copy_caches_from_host']:
ssh_config = v.ssh_config()
user = re.search(r'User ([^ \n]+)', ssh_config).group(1)
hostname = re.search(r'HostName ([^ \n]+)', ssh_config).group(1)
port = re.search(r'Port ([0-9]+)', ssh_config).group(1)
key = re.search(r'IdentityFile ([^ \n]+)', ssh_config).group(1)
for d in ('.m2', '.gradle/caches', '.gradle/wrapper', '.pip_download_cache'):
fullpath = os.path.join(os.getenv('HOME'), d)
if os.path.isdir(fullpath):
makebuildserver: fix join() syntax error, it needs a list/tuple If only there was a way to test this without taking hours to run... fixes 964ef996a01ceac8d6735668f2c9fd9119512346
2017-12-07 11:52:10 +01:00
ssh_command = ' '.join(('ssh -i {0} -p {1}'.format(key, port),
'-o StrictHostKeyChecking=no',
'-o UserKnownHostsFile=/dev/null',
'-o LogLevel=FATAL',
'-o IdentitiesOnly=yes',
'-o PasswordAuthentication=no'))
makebuildserver: make copy_caches_from_host do rsync like `fdroid build` This rsync hung because of an SSH unknown key prompt. Since this is just the vm host sshing to the vm guest, it is not essential to check the host keys.
2017-12-06 09:47:08 +01:00
# TODO vagrant 1.5+ provides `vagrant rsync`
buildserver: add copy_caches_from_host config option For people using slow, expensive, and/or flaky internet, liberal use of caching can make a huge difference. The restricted environment of the gpjenkins box has been a good test environment for this (Tor-only, whitelist of allowed IPs to visit, home internet connection).
2016-06-23 17:12:06 +02:00
run_via_vagrant_ssh(v, ['cd ~ && test -d', d, '|| mkdir -p', d])
makebuildserver: quiet rsync for copy_caches_from_host
2017-12-07 15:56:09 +01:00
subprocess.call(['rsync', '-ax', '--delete', '-e',
makebuildserver: make copy_caches_from_host do rsync like `fdroid build` This rsync hung because of an SSH unknown key prompt. Since this is just the vm host sshing to the vm guest, it is not essential to check the host keys.
2017-12-06 09:47:08 +01:00
ssh_command,
buildserver: add copy_caches_from_host config option For people using slow, expensive, and/or flaky internet, liberal use of caching can make a huge difference. The restricted environment of the gpjenkins box has been a good test environment for this (Tor-only, whitelist of allowed IPs to visit, home internet connection).
2016-06-23 17:12:06 +02:00
fullpath + '/',
user + '@' + hostname + ':~/' + d + '/'])
# this file changes every time but should not be cached
run_via_vagrant_ssh(v, ['rm', '-f', '~/.gradle/caches/modules-2/modules-2.lock'])
run_via_vagrant_ssh(v, ['rm', '-fr', '~/.gradle/caches/*/plugin-resolution/'])
buildserver: move code into main() method to always stop thread By running the whole program in a main() function, it can be wrapped in try/finally in order to stop the background display thread. This is also done in ./fdroid, its standard practice for Python CLI utilities.
2016-09-25 18:55:29 +02:00
p = subprocess.Popen(['git', 'rev-parse', 'HEAD'], stdout=subprocess.PIPE,
universal_newlines=True)
buildserverid = p.communicate()[0].strip()
makebuildserver debugging log details
2017-03-24 20:04:50 +01:00
logger.info("Writing buildserver ID ...ID is %s", buildserverid)
buildserver: run_via_vagrant_ssh() to run cmds via python-vagrant This moves the last vagrant call in a subprocess.
2016-06-23 16:38:39 +02:00
run_via_vagrant_ssh(v, 'sh -c "echo %s >/home/vagrant/buildserverid"' % buildserverid)
buildserver: move code into main() method to always stop thread By running the whole program in a main() function, it can be wrapped in try/finally in order to stop the background display thread. This is also done in ./fdroid, its standard practice for Python CLI utilities.
2016-09-25 18:55:29 +02:00
makebuildserver debugging log details
2017-03-24 20:04:50 +01:00
logger.info("Stopping build server VM")
buildserver: replace custom code with python-vagrant I ran into some annoying issues with UTF-8 output in the vagrant logs, and it was hard to solve. So I switched to using python-vagrant, which handles it all for us. Its been around since 2012, has a number of contributors, and is still actively maintained, so it seems like a good bet. I also packaged it for Debian, including a backport in jessie-backports. On Debian/jessie, do `apt-get install python3-vagrant/jessie-backports`
2016-09-27 08:49:32 +02:00
v.halt()
Much better build server provisioning Including selection of correct ndk architecture, caching and performance improvements, better configurability, etc.
2013-05-31 08:48:39 +02:00
makebuildserver debugging log details
2017-03-24 20:04:50 +01:00
logger.info("Packaging")
buildserver: consolidate boxfile export code into one block This is just for clarity, and moving more code into the main() function.
2016-09-26 11:22:05 +02:00
boxfile = os.path.join(os.getcwd(), 'buildserver.box')
if os.path.exists(boxfile):
os.remove(boxfile)
makebuildserver: package up KVM VM as a vagrant box `vagrant package` does not work with KVM, so we have to hack together our own until someone implements it (suppose we should do it). This is a hacked up version based on: https://github.com/vagrant-libvirt/vagrant-libvirt/blob/d7d440ea8f24f698a93a4c5b9a5149acef469579/tools/create_box.sh #238
2017-02-14 22:58:35 +01:00
leave VirtualBox `vagrant package` as it was originally We only need Vagrantfile hacks for KVM.
2017-05-22 15:50:32 +02:00
vm.package(output=boxfile)
buildserver: move code into main() method to always stop thread By running the whole program in a main() function, it can be wrapped in try/finally in order to stop the background display thread. This is also done in ./fdroid, its standard practice for Python CLI utilities.
2016-09-25 18:55:29 +02:00
makebuildserver debugging log details
2017-03-24 20:04:50 +01:00
logger.info("Adding box")
added box handling to vmtools
2017-05-22 17:14:48 +02:00
vm.box_add('buildserver', boxfile, force=True)
buildserver: move code into main() method to always stop thread By running the whole program in a main() function, it can be wrapped in try/finally in order to stop the background display thread. This is also done in ./fdroid, its standard practice for Python CLI utilities.
2016-09-25 18:55:29 +02:00
added box handling to vmtools
2017-05-22 17:14:48 +02:00
if 'buildserver' not in subprocess.check_output(['vagrant', 'box', 'list']).decode('utf-8'):
refactored kvm_package to vmtools
2017-05-22 17:12:34 +02:00
logger.critical('could not add box \'%s\' as \'buildserver\', terminating', boxfile)
sys.exit(1)
added makebuildserver option for keeping vagrant box This is very useful for debugging this process, and also for people who might want to keep a working copy of the box.
2017-05-22 17:29:12 +02:00
if not options.keep_box_file:
logger.debug('box added to vagrant, ' +
'removing generated box file \'%s\'',
boxfile)
os.remove(boxfile)
buildserver: move code into main() method to always stop thread By running the whole program in a main() function, it can be wrapped in try/finally in order to stop the background display thread. This is also done in ./fdroid, its standard practice for Python CLI utilities.
2016-09-25 18:55:29 +02:00
if __name__ == '__main__':
try:
main()
finally:
if tail is not None:
tail.stop()
Copy Permalink