1
0
mirror of https://gitlab.com/fdroid/fdroidserver.git synced 2024-11-09 17:00:12 +01:00
Commit Graph

191 Commits

Author SHA1 Message Date
Hans-Christoph Steiner
6d541e3ef6
establish config/mirrors.yml as a way to set up mirrors
The mirrors: entry in config.yml is great for quick access and shorter
mirror lists.  Now that we are adding a lot more metadata to the mirrors,
including potentially the full text of the privacy policy, having this
in its own file should make these cases easier to manage.
2023-12-07 14:38:26 +01:00
linsui
5ae15532f0 fetch_srclib: fetch source code 2023-11-17 10:45:53 +00:00
Hans-Christoph Steiner
ce77a33228
mirror-to-mirror.sh: set 1h timeout for rsync runs
This script is run every 10 minutes or so to push new files to the primary
mirrors.  It sets a lock to prevent multiple copies from running in
parallel.  Yesterday, one rsync process got stuck and never exited, thereby
preventing this script from running.  Adding a 1h timeout seems like a safe
way to deal with this kind of problem.  This would not deal with rsync
getting so stuck that it fails to heed the timeout, hopefully that's not an
issue.
2023-07-25 14:52:29 +02:00
Hans-Christoph Steiner
d7214a7f1c hooks/pre-commit: run yamllint on more YAML files
I keep messing up YAML syntax with values that have : in them...

This doesn't work for all YAML files:
* tests/metadata/dump/*.yaml have bad indenting
* tests/metadata/*.yml have bad indenting, which is useful for tests
2023-04-28 10:55:21 +02:00
Hans-Christoph Steiner
7c692a4532 index-v2 'mirrors' fully settable from config
This lets mirrors: in config.yml be the same list-of-dicts format as it is
in index-v2.  This also includes a data format conversion to maintain the
right format for the old, unchanging index v0 and v1 formats.

#928
#1107
2023-04-27 07:53:02 +00:00
Hans-Christoph Steiner
b8f59097f7 purge all vestiges of the unused fdroid stats, closes #839
* for f in locale/*/LC_MESSAGES/fdroidserver.po; do msgattrib --set-obsolete --no-wrap --ignore-file=locale/fdroidserver.pot -o $f $f; done
* sed -i 's, \.\./fdroidserver/stats\.py,,' locale/*/LC_MESSAGES/fdroidserver.po
2023-02-19 22:54:38 +01:00
Jochen Sprickerhof
46e5fa3714 clean_repos plugin: also clean submodules 2022-11-29 10:30:10 +00:00
Hans-Christoph Steiner
e70ef18d52
examples/fdroid_clean_repos.py: fix typo
!1227
2022-11-22 20:47:09 +01:00
Hans-Christoph Steiner
052c753075 examples/fdroid_clean_repos.py: black code format
!1227
2022-11-22 20:43:19 +01:00
Jochen Sprickerhof
1015225a00 Add plugin to cleanup app VCSs 2022-11-22 19:36:33 +00:00
Michael Pöhn
24d88705fa 🔍 add scanner_signature_sources config option
This adds the option to configure which set of signatures `fdroid
scanner` should use, by configuring it in `config.yml`. It allows
fetching signatures in our custom json format. It also adds 3 additional
sources: 'suss', 'exodus', 'etip'
2022-11-15 09:20:26 +00:00
Hans-Christoph Steiner
d6008cdb2a
convert examples/makebuildserver.config.py to Vagrantfile.yaml
Also, tests/androguard_test.py was removed long ago
2022-11-03 13:27:18 +01:00
Hans-Christoph Steiner
e2fcd633fc
buildserver: hard code basebox name and version
This is not user-configurable, so it should not be setup to be.  This
process is only tested on the one basebox, and devs can just edit
Vagrantfile directly to test other base boxes.

# Conflicts:
#	makebuildserver
2022-11-03 13:27:15 +01:00
Hans-Christoph Steiner
abdd02f33a
makebuildserver: move copy_caches_from_host to command line flag 2022-11-03 13:27:13 +01:00
Hans-Christoph Steiner
21ea1c1c89
makebuildserver: purge apt_package_cache feature
This is unmaintained, lightly used, a tangled mess, and can be replaced by
things like the vagrant-cachier plugin or #418
2022-11-03 13:27:11 +01:00
Hans-Christoph Steiner
23d9f0ed50
mirror-to-mirror.sh: replace mirror.f-droid.org with ftp.agdsn.de
* ftp.agdsn.de is a new official mirror that receives push updates.
* mirror.f-droid.org is no more, it is now a DNS round robin redirector.

admin#247
admin#298

[skip ci]
2022-05-02 14:21:35 +02:00
Jochen Sprickerhof
5f3eb601df Add check for repo/archive_url 2022-04-21 11:16:14 +02:00
Jochen Sprickerhof
b6f1f4231a Drop code for the old wiki 2021-11-24 18:16:07 +01:00
Christian Stadelmann
a0a86aa5ec Clarify wording example for config.yml
Note: The old wording was applicable to the python version of this config file only.
2021-10-02 08:13:48 +00:00
Hans-Christoph Steiner
adbd7ba757
mirror-to-mirror.sh: also sync to plug-mirror.rcac.purdue.edu
This is the deployed script for pushing to the primary mirrors.  It starts
with the 'repo' section and runs all the rsyncs in parallel to each primary
mirror.  Once the 'repo' syncs are done, it does the same process for the
'archive' syncs. This adds a new primary push mirror at PLUG. #163

!800 !792
[skip ci]
2021-09-16 10:11:27 +02:00
Hans-Christoph Steiner
9f77044d0d
auto-detect NDKs installed in standard paths
'ndk_paths' will be automatically filled out from well known sources
like $ANDROID_HOME/ndk-bundle and $ANDROID_HOME/ndk/*.  If a required
version is missing in the buildserver VM, it will be automatically
downloaded and installed into the standard $ANDROID_HOME/ndk/
directory.  Manually setting it here will override the auto-detected
values.  The keys can either be the "release" (e.g. r21e) or the
"revision" (e.g. 21.4.7075529).

https://developer.android.com/studio/projects/configure-agp-ndk#agp_version_41
* sdkmanager installs "ndk;12.3.4567890" into $ANDROID_SDK_ROOT/ndk/
* sdkmanager installs "ndk-bundle" into $ANDROID_SDK_ROOT/ndk-bundle/
2021-05-28 09:13:34 +02:00
Hans-Christoph Steiner
9fc2a23713
build: remove default NDK, closes #717 2021-05-25 17:06:26 +02:00
FestplattenSchnitzel
e6c186c566
fetchsrclibs: disable preparation 2021-04-12 22:11:07 +02:00
relan
6d9410597b Upgrade NDK r22 to r22b 2021-04-07 12:12:04 +03:00
proletarius101
c70cc33d1d Fix common.config is None in fetchlibs 2021-03-26 14:52:29 +00:00
Hans-Christoph Steiner
dbf80ad771
metadata: remove git repo handling from metadata parsing
This is a vestige of implementing builds from a .fdroid.yml file directly
in the app's source repo.  It was never fully complete and seems to not be
used in any apps in fdroiddata.  This makes `fdroid build --all` runs much
faster since it does not need to do any git handling for apps that do not
have any new builds to run.

4e8e29794
948689281
a4e431080
e37be9b06
e775d330c
2021-03-17 13:44:24 +01:00
Licaon_Kter
d683a40f96 Add NDK 21e 2021-03-10 16:36:14 +02:00
Hans-Christoph Steiner
8bb2c7d4b1 metadata: stop setting up source repo when running lint/rewritemeta
broken in 54e8867a7e
2021-03-09 17:55:07 +01:00
Hans-Christoph Steiner
63be6f6347 standardize naming in strings and comments on all caps APK 2021-01-29 15:40:59 +01:00
Hans-Christoph Steiner
f527b17715 sync repo examples with defaults and comment out example repo settings
!359
2021-01-24 09:26:11 +01:00
Hans-Christoph Steiner
5ae2703477 init: no error if examples/ dir cannot be found; overhaul repo_icon
There are so many possible installation paths for Python modules, it has
been very hard to even find and test them all.  This adds a fallback option
if the examples dir cannot be found.  A repo can work without an icon or
the example config.py.

This removes the fake assumption that the icon can be a full path in the
config.py.  While the path was being properly passed through to the index
file, the file was never copied properly into place nor rsynced to the web
server.
2021-01-18 14:07:20 +01:00
Hans-Christoph Steiner
c8f21bf0e0 add some example plugins for repo key extraction and migration 2021-01-15 15:25:50 +01:00
Hans-Christoph Steiner
d36f71e048 make config optional when using common.set_FDroidPopen_env()
common.set_FDroidPopen_env() is used to call git and other tools that
are not part of the Android SDK nor require Java, so the items that
are being set from the config are optional.  This lets plugins do
quite a bit without ever setting up a config.
2021-01-11 20:58:50 +01:00
Hans-Christoph Steiner
4b7d29097d gitlab-ci: new job to test plugins API + fdroid fetchsrclibs 2021-01-11 20:58:50 +01:00
TacoTheDank
c88a3f56a1 Update NDK to r22 and add buildtools 30.0.3 2020-12-26 19:51:22 -05:00
Hans-Christoph Steiner
330ca2319c
example plugin for fetching srclibs
This is used in _fdroiddata_'s `fdroid build` CI job, and _issuebot_.
This uses @uniqx's awesome new 'plugin' feature to create a 'fetchsrclibs`
command.  The 'fdroid build' gitlab-ci job uses --on-server, which does not
set up the srclibs.  This plugin does the missing setup.

This is moved from _fdroiddata_ where it was _tools/fdroid_fetchsrclibs.py_.
2020-12-22 16:59:02 +01:00
Hans-Christoph Steiner
c98b15e72a purge code that modifies the repo descriptions
YAML now provides nice ways of handling blocks of text, including removing
indents and newlines.  Remove the custom hack.
2020-12-08 09:37:49 +01:00
Hans-Christoph Steiner
1c7df94e76 purge unneeded 'build_tools' config option
Back when fdroidserver was built around aapt, that was needed to
guarantee that a compatible version of aapt was used. Now, aapt is
only optionally used for getting the APK ID, so this was just
complicating maintenance.
2020-11-29 10:06:39 +01:00
Hans-Christoph Steiner
f430c58d01 make examples/template.yml pass yamllint 2020-10-23 14:20:49 +02:00
Hans-Christoph Steiner
a9fdb5b401 init: switch to config.yml as the default format 2020-10-23 14:20:49 +02:00
Hans-Christoph Steiner
17f6a778ba deploy: move server.py to deploy.py and remove aliases
closes #832
2020-10-21 11:38:49 +02:00
Izzy
a216c0487e
accepted_formats has been removed from fdroidserver (only yml remains anyway) 2020-10-20 22:55:52 +02:00
Marcus Hoffmann
0d5fde334d fix keytool not working with default smartcardoptions
This broke in 74af61f255.
Keytool has still a different opinion from both apksigner and jarsigner
about the providerName argument. apksigner doesn't support it at all,
jarsigner ignores it but keytool fails without it. :-/

So we add it back to the default argument list but filter it out before
calling apksigner.
2020-10-07 16:12:21 +02:00
Hans-Christoph Steiner
465ec2e84e
mirror-to-mirror.sh: also sync to ftp.lysator.liu.se, in parallel
This is the deployed script for pushing to the primary mirrors.  It starts
with the 'repo' section and runs all the rsyncs in parallel to each primary
mirror.  Once the 'repo' syncs are done, it does the same process for the
'archive' syncs.

[skip ci]
2020-10-01 14:49:04 +02:00
Hans-Christoph Steiner
08f724651e remove "force_build_tools" config option, closes #738
The `force_build_tools` config option was added a long time ago to
brute force the _build-tools_ version by trying to replace the value
in `build.gradle` files. This is never something that should be used
in production, since the app's build metadata should specify this kind
of thing. And now that we're moving towards _androguard_ for
everything except fdroid build and fdroid publish, _build-tools_ will
no longer even be used in the other commands.
2020-09-16 16:30:49 +02:00
Hans-Christoph Steiner
8b406ace08 mirror-to-mirror.sh: update script to handle multiple primary mirrors 2020-09-16 16:06:23 +02:00
Hans-Christoph Steiner
94cd2a960c add script used to rsync repo to primary mirrors 2020-09-16 16:01:13 +02:00
Marcus Hoffmann
74af61f255 remove providerName from default smartcardoptions
apksigner doesn't recognize the SunPKCS11-OpenSC set via providerName
Neither jarsigner nor apksigner need this to work.
2020-09-09 22:50:58 +02:00
Marcus
f84818c15e scanner: add a simple scan for blacklisted classes after build step
add com.android.billing to blacklist, see
https://gitlab.com/fdroid/fdroiddata/-/issues/2070#note_360611289
2020-06-15 18:03:19 +00:00
TacoTheDank
0216513da6 Upgrade NDK r21c to r21d 2020-06-10 15:55:53 -04:00