kamp/fdroidserver
1
0
Fork 0
mirror of https://gitlab.com/fdroid/fdroidserver.git synced 2024-11-18 20:50:10 +01:00
Code Activity
83e16422a5
fdroidserver/buildserver/provision-android-sdk

139 lines
4.6 KiB
Plaintext
Raw Normal View History

buildserver: use `android update sdk` to install Android SDK `android update sdk --no-ui` is the standard command line tool for installing the Android SDK. By symlinking into the $ANDROID_HOME/temp dir, the cached files can still be used. This converts the chef recipe to a vagrant shell provisioning script since it was all bash anyway. Some file names no longer officially have a -linux in them, so those were changed to keep the cache working with the default filename.
2016-06-14 16:34:13 +02:00
#!/bin/bash
#
buildserver: make provision scripts output name to log
2016-07-04 14:20:42 +02:00
echo $0
buildserver: use `android update sdk` to install Android SDK `android update sdk --no-ui` is the standard command line tool for installing the Android SDK. By symlinking into the $ANDROID_HOME/temp dir, the cached files can still be used. This converts the chef recipe to a vagrant shell provisioning script since it was all bash anyway. Some file names no longer officially have a -linux in them, so those were changed to keep the cache working with the default filename.
2016-06-14 16:34:13 +02:00
set -e
buildserver: enable debug log for provision-android-sdk
2016-06-22 08:06:07 +02:00
set -x
buildserver: use `android update sdk` to install Android SDK `android update sdk --no-ui` is the standard command line tool for installing the Android SDK. By symlinking into the $ANDROID_HOME/temp dir, the cached files can still be used. This converts the chef recipe to a vagrant shell provisioning script since it was all bash anyway. Some file names no longer officially have a -linux in them, so those were changed to keep the cache working with the default filename.
2016-06-14 16:34:13 +02:00
if [ -z $ANDROID_HOME ]; then
echo "ANDROID_HOME env var must be set!"
exit 1
fi
# TODO remove the rm, this should work with an existing ANDROID_HOME
if [ ! -x $ANDROID_HOME/tools/android ]; then
rm -rf $ANDROID_HOME
mkdir ${ANDROID_HOME}
mkdir ${ANDROID_HOME}/temp
mkdir ${ANDROID_HOME}/platforms
mkdir ${ANDROID_HOME}/build-tools
cd $ANDROID_HOME
tools=`ls -1 /vagrant/cache/tools_*.zip | sort -n | tail -1`
unzip -qq $tools
fi
buildserver: ignore android sdk repos of proprietary bits The Android SDK by default includes some Google repositories of their proprietary SDKs. We of course do not want that stuff ever. We also do not need the emulator images since this process does not currently install or run an emulator.
2016-09-15 11:29:31 +02:00
# disable the repositories of proprietary stuff
disabled="
@version@=1
@disabled@https\://dl.google.com/android/repository/extras/intel/addon.xml=disabled
@disabled@https\://dl.google.com/android/repository/glass/addon.xml=disabled
@disabled@https\://dl.google.com/android/repository/sys-img/android/sys-img.xml=disabled
@disabled@https\://dl.google.com/android/repository/sys-img/android-tv/sys-img.xml=disabled
@disabled@https\://dl.google.com/android/repository/sys-img/android-wear/sys-img.xml=disabled
@disabled@https\://dl.google.com/android/repository/sys-img/google_apis/sys-img.xml=disabled
"
test -d ${HOME}/.android || mkdir ${HOME}/.android
buildserver: allow gradle/sdkmanager to install into the new m2repository Google is pushing gradle towards downloading all the SDK components that it needs, rather than having a preconfigured SDK installed. The buildserver strongly supports the old model, with added checksum checking even. We can still support the old model by pre-configuring the SDK and locking it down as root. This can then also support the new model by setting the file perms so that new packages can be auto-installed, but they cannot overwrite any packages that come pre-installed and pre-verified. fdroiddata!2096 closes #247
2017-03-16 11:43:46 +01:00
# there are currently zero user repos
echo 'count=0' > ${HOME}/.android/repositories.cfg
buildserver: ignore android sdk repos of proprietary bits The Android SDK by default includes some Google repositories of their proprietary SDKs. We of course do not want that stuff ever. We also do not need the emulator images since this process does not currently install or run an emulator.
2016-09-15 11:29:31 +02:00
for line in $disabled; do
echo $line >> ${HOME}/.android/sites-settings.cfg
done
buildserver: use `android update sdk` to install Android SDK `android update sdk --no-ui` is the standard command line tool for installing the Android SDK. By symlinking into the $ANDROID_HOME/temp dir, the cached files can still be used. This converts the chef recipe to a vagrant shell provisioning script since it was all bash anyway. Some file names no longer officially have a -linux in them, so those were changed to keep the cache working with the default filename.
2016-06-14 16:34:13 +02:00
cd /vagrant/cache
buildserver: android --silent hides errors, so remove --silent seem to prevent `android update sdk` from exiting with an error, so its kind of useless. I just wanted it to suppress the verbose logs.
2016-06-21 18:12:09 +02:00
# make links for `android update sdk` to use and delete
switch to allow/block list terminology throughout code base allowlist and blocklist are much clearer terms with no cultural baggage. This changes all "whitelist" references to "allowlist", and all "blacklist" references to "blocklist".
2021-06-17 14:46:51 +02:00
blocklist="build-tools_r17-linux.zip
provision-sdk: don't install obolete stuff This is probably still lying in the cache bu we explicitly don't want to install it anymore.
2018-11-28 20:47:09 +01:00
build-tools_r18.0.1-linux.zip
build-tools_r18.1-linux.zip
build-tools_r18.1.1-linux.zip
build-tools_r19-linux.zip
build-tools_r19.0.1-linux.zip
build-tools_r19.0.2-linux.zip
build-tools_r19.0.3-linux.zip
build-tools_r21-linux.zip
build-tools_r21.0.1-linux.zip
build-tools_r21.0.2-linux.zip
build-tools_r21.1-linux.zip
build-tools_r21.1.1-linux.zip
build-tools_r22-linux.zip
build-tools_r23-linux.zip
android-1.5_r04-linux.zip
android-1.6_r03-linux.zip
android-2.0_r01-linux.zip
android-2.0.1_r01-linux.zip"
buildserver: only include latest m2 when provisioning I think the `android update sdk` tool is installing all of the m2 files that are present in the temp cache, and it seems to do it in order of newest to oldest. Well done, and I thought that tool couldn't get any worse. So only include the latest version of android_m2repository*.zip in the temp cache.
2016-09-19 13:29:46 +02:00
latestm2=`ls -1 android_m2repository*.zip | sort -n | tail -1`
for f in $latestm2 android-[0-9]*.zip platform-[0-9]*.zip build-tools_r*-linux.zip; do
buildserver: use `android update sdk` to install Android SDK `android update sdk --no-ui` is the standard command line tool for installing the Android SDK. By symlinking into the $ANDROID_HOME/temp dir, the cached files can still be used. This converts the chef recipe to a vagrant shell provisioning script since it was all bash anyway. Some file names no longer officially have a -linux in them, so those were changed to keep the cache working with the default filename.
2016-06-14 16:34:13 +02:00
rm -f ${ANDROID_HOME}/temp/$f
switch to allow/block list terminology throughout code base allowlist and blocklist are much clearer terms with no cultural baggage. This changes all "whitelist" references to "allowlist", and all "blacklist" references to "blocklist".
2021-06-17 14:46:51 +02:00
if [[ $blocklist != *$f* ]]; then
provision-sdk: don't install obolete stuff This is probably still lying in the cache bu we explicitly don't want to install it anymore.
2018-11-28 20:47:09 +01:00
ln -s /vagrant/cache/$f ${ANDROID_HOME}/temp/
fi
buildserver: use `android update sdk` to install Android SDK `android update sdk --no-ui` is the standard command line tool for installing the Android SDK. By symlinking into the $ANDROID_HOME/temp dir, the cached files can still be used. This converts the chef recipe to a vagrant shell provisioning script since it was all bash anyway. Some file names no longer officially have a -linux in them, so those were changed to keep the cache working with the default filename.
2016-06-14 16:34:13 +02:00
done
# install all cached platforms
cached=""
for f in `ls -1 android-[0-9]*.zip platform-[0-9]*.zip`; do
sdk=`unzip -c $f "*/build.prop" | sed -n 's,^ro.build.version.sdk=,,p'`
cached=,android-${sdk}${cached}
done
# install all cached build-tools
for f in `ls -1 build-tools*.zip`; do
ver=`unzip -c $f "*/source.properties" | sed -n 's,^Pkg.Revision=,,p'`
makebs: work around build-tools 24.0.1 version bug The zip contains 24.0.0 as the version, which means our magic was installing 24.0.0 twice and skipping 24.0.1.
2016-08-02 18:08:48 +02:00
if [[ $ver == 24.0.0 ]] && [[ $f =~ .*r24\.0\.1.* ]]; then
# 24.0.1 has the wrong revision in the zip
ver=24.0.1
fi
buildserver: use `android update sdk` to install Android SDK `android update sdk --no-ui` is the standard command line tool for installing the Android SDK. By symlinking into the $ANDROID_HOME/temp dir, the cached files can still be used. This converts the chef recipe to a vagrant shell provisioning script since it was all bash anyway. Some file names no longer officially have a -linux in them, so those were changed to keep the cache working with the default filename.
2016-06-14 16:34:13 +02:00
cached=,build-tools-${ver}${cached}
done
buildserver: android --silent hides errors, so remove --silent seem to prevent `android update sdk` from exiting with an error, so its kind of useless. I just wanted it to suppress the verbose logs.
2016-06-21 18:12:09 +02:00
${ANDROID_HOME}/tools/android update sdk --no-ui --all \
buildserver: enable debug log for provision-android-sdk
2016-06-22 08:06:07 +02:00
--filter platform-tools,extra-android-m2repository${cached} <<EOH
buildserver: use `android update sdk` to install Android SDK `android update sdk --no-ui` is the standard command line tool for installing the Android SDK. By symlinking into the $ANDROID_HOME/temp dir, the cached files can still be used. This converts the chef recipe to a vagrant shell provisioning script since it was all bash anyway. Some file names no longer officially have a -linux in them, so those were changed to keep the cache working with the default filename.
2016-06-14 16:34:13 +02:00
y
EOH
buildserver: support new ConstraintLayout license bullshit The new ConstraintLayout library in Android Support has some new custom way of handling the license. I suspect that they are going to use this new way with all of the bits that gradle downloads. We also have to support it for apps that use it, including soon fdroidclient. fdroiddata!2094 ci-images!1
2017-03-13 16:31:27 +01:00
mkdir -p $ANDROID_HOME/licenses/
buildserver: include all Android SDK licenses in their exact format I tried to clone the files I got from `sdkmanager --licenses`, byte for byte.
2017-11-24 09:21:28 +01:00
buildserver: add new android-sdk-license hash The old hash does not work anymore, at least for platform-27. Where this new one comes from: 1. Download command line tools from https://developer.android.com/studio/index.html#command-tools 2. Unzip the package 3. Run ./tools/bin/sdkmanager --licenses 4. Read and accept licenses 5. Find the hash in ./licenses/android-sdk-license
2017-11-14 10:38:48 +01:00
cat << EOF > $ANDROID_HOME/licenses/android-sdk-license
8933bad161af4178b1185d1a37fbf41ea5269c55
buildserver: include all Android SDK licenses in their exact format I tried to clone the files I got from `sdkmanager --licenses`, byte for byte.
2017-11-24 09:21:28 +01:00
buildserver: add new android-sdk-license hash The old hash does not work anymore, at least for platform-27. Where this new one comes from: 1. Download command line tools from https://developer.android.com/studio/index.html#command-tools 2. Unzip the package 3. Run ./tools/bin/sdkmanager --licenses 4. Read and accept licenses 5. Find the hash in ./licenses/android-sdk-license
2017-11-14 10:38:48 +01:00
d56f5187479451eabf01fb78af6dfcb131a6481e
Added newer ndks, gradles, latest sdk-license, and update java 1.8 version
2019-03-30 22:10:21 +01:00
24333f8a63b6825ea9c5514f83c2829b004d1fee
buildserver: add new android-sdk-license hash The old hash does not work anymore, at least for platform-27. Where this new one comes from: 1. Download command line tools from https://developer.android.com/studio/index.html#command-tools 2. Unzip the package 3. Run ./tools/bin/sdkmanager --licenses 4. Read and accept licenses 5. Find the hash in ./licenses/android-sdk-license
2017-11-14 10:38:48 +01:00
EOF
buildserver: include all Android SDK licenses in their exact format I tried to clone the files I got from `sdkmanager --licenses`, byte for byte.
2017-11-24 09:21:28 +01:00
cat <<EOF > $ANDROID_HOME/licenses/android-sdk-preview-license
84831b9409646a918e30573bab4c9c91346d8abd
EOF
cat <<EOF > $ANDROID_HOME/licenses/android-sdk-preview-license-old
Added newer ndks, gradles, latest sdk-license, and update java 1.8 version
2019-03-30 22:10:21 +01:00
buildserver: include all Android SDK licenses in their exact format I tried to clone the files I got from `sdkmanager --licenses`, byte for byte.
2017-11-24 09:21:28 +01:00
79120722343a6f314e0719f863036c702b0e6b2a
84831b9409646a918e30573bab4c9c91346d8abd
EOF
buildserver: support new ConstraintLayout license bullshit The new ConstraintLayout library in Android Support has some new custom way of handling the license. I suspect that they are going to use this new way with all of the bits that gradle downloads. We also have to support it for apps that use it, including soon fdroidclient. fdroiddata!2094 ci-images!1
2017-03-13 16:31:27 +01:00
echo y | $ANDROID_HOME/tools/bin/sdkmanager "extras;m2repository;com;android;support;constraint;constraint-layout;1.0.1"
echo y | $ANDROID_HOME/tools/bin/sdkmanager "extras;m2repository;com;android;support;constraint;constraint-layout-solver;1.0.1"
echo y | $ANDROID_HOME/tools/bin/sdkmanager "extras;m2repository;com;android;support;constraint;constraint-layout;1.0.2"
echo y | $ANDROID_HOME/tools/bin/sdkmanager "extras;m2repository;com;android;support;constraint;constraint-layout-solver;1.0.2"
buildserver: use `android update sdk` to install Android SDK `android update sdk --no-ui` is the standard command line tool for installing the Android SDK. By symlinking into the $ANDROID_HOME/temp dir, the cached files can still be used. This converts the chef recipe to a vagrant shell provisioning script since it was all bash anyway. Some file names no longer officially have a -linux in them, so those were changed to keep the cache working with the default filename.
2016-06-14 16:34:13 +02:00
provision-android-sdk: ensure ANDROID_HOME parent dir is accessible by all https://gitlab.com/fdroid/fdroidserver/-/merge_requests/926#note_586778883
2021-05-28 09:02:19 +02:00
chmod a+X $(dirname $ANDROID_HOME/)
buildserver: use `android update sdk` to install Android SDK `android update sdk --no-ui` is the standard command line tool for installing the Android SDK. By symlinking into the $ANDROID_HOME/temp dir, the cached files can still be used. This converts the chef recipe to a vagrant shell provisioning script since it was all bash anyway. Some file names no longer officially have a -linux in them, so those were changed to keep the cache working with the default filename.
2016-06-14 16:34:13 +02:00
chmod -R a+rX $ANDROID_HOME/
buildserver: allow gradle to install new components Google is making gradle automatically download Android SDK components that are needed by the build, but not already present. We need to support that since it would be a lot of work to fight it. Plus, since each build starts from the fresh snapshot, it should not be such a big deal to let each build install stuff during the process. closes #268
2017-03-22 10:59:44 +01:00
chgrp vagrant $ANDROID_HOME
chmod g+w $ANDROID_HOME
buildserver: use `android update sdk` to install Android SDK `android update sdk --no-ui` is the standard command line tool for installing the Android SDK. By symlinking into the $ANDROID_HOME/temp dir, the cached files can still be used. This converts the chef recipe to a vagrant shell provisioning script since it was all bash anyway. Some file names no longer officially have a -linux in them, so those were changed to keep the cache working with the default filename.
2016-06-14 16:34:13 +02:00
find $ANDROID_HOME/ -type f -executable -print0 | xargs -0 chmod a+x
buildserver: allow gradle to install newer build-tools versions gradle will now automatically download and install missing bits of the Android SDK. While we prefer to have the SDK packages fully verified, we should allow this behavior on the buildserver to ensure that builds work even when the buildserver can't be updated. Since each build starts from a clean snapshot, this auto-installed build-tools will only be used for the single build, so it won't affect other apps.
2017-03-14 12:39:03 +01:00
buildserver: allow gradle/sdkmanager to install new platforms Google releases new API SDKs quite rarely (about 2 times a year), but when they do this, many apps quickly start using it. Allow downloading new API SDKs to avoid waiting for the next buildserver rebuild.
2017-11-14 10:29:21 +01:00
# allow gradle to install newer build-tools and platforms
chgrp vagrant $ANDROID_HOME/{build-tools,platforms}
chmod g+w $ANDROID_HOME/{build-tools,platforms}
buildserver: allow gradle/sdkmanager to install into the new m2repository Google is pushing gradle towards downloading all the SDK components that it needs, rather than having a preconfigured SDK installed. The buildserver strongly supports the old model, with added checksum checking even. We can still support the old model by pre-configuring the SDK and locking it down as root. This can then also support the new model by setting the file perms so that new packages can be auto-installed, but they cannot overwrite any packages that come pre-installed and pre-verified. fdroiddata!2096 closes #247
2017-03-16 11:43:46 +01:00
# allow gradle/sdkmanager to install into the new m2repository
test -d $ANDROID_HOME/extras/m2repository || mkdir -p $ANDROID_HOME/extras/m2repository
find $ANDROID_HOME/extras/m2repository -type d | xargs chgrp vagrant
find $ANDROID_HOME/extras/m2repository -type d | xargs chmod g+w
Copy Permalink