os.rename() only works if source and destination are on the same file
system, shutil.move() works across file systems.
OSError: [Errno 18] Invalid cross-device link: '/builds/eighthave/fdroidclient/app/build/outputs/apk/app-debug.apk' -> '/tmp/tmp966vh75f/tmp.apk'
This needs to use the config loading routine to find Java `keytool`, but
since it doesn't need to fully load the config, isolate that usage in the
function. Then read_config() is only ever called once, as is it meant to
be used, once the config.py is generated.
Using `from . import common; common.config = foo` will not always work,
due to some oddities to how the `from` imports work. So the full module
has to be imported in order to make sure its always properly set.
Rather than needing to run a command before and after the build, in order
to first install the debug.keystore, then after to fetch and publish the
APK, this makes `fdroid nightly` just resign the APK with the provided
debug.keystore. Then `fdroid nightly` can be run as the final step in a CI
build, and still ensure that the APKs are always signed by the provided
debug.keystore.
If we allow SSH, then we'd have to manage known_hosts.
All VCS and submodule URLs should use HTTPS. SSH URLs have security vulns:
https://blogs.msdn.microsoft.com/devops/2017/08/15/git-vulnerability-with-submodules/https://www.theregister.co.uk/2017/08/13/ssh_flaw_in_git_mercurial_svn/
CVE-2017-1000117
I did a manual scan of the setup on jenkins.debian.net to see if I could
find any suspicious URLs. Looks good so far. This is what I used:
find . -type f -print0 |xargs -0 grep -Eo 'ssh[:+][svn/]+...................'
find . -type f -print0 |xargs -0 grep -Eo 'ssh://-[^ "]+'
Also, some ssh://_ URLs in submodules might still work, because of the URL
rewriting in fdbfb4d1. But https://-oProxyCommand=pwnme does not really do
anything, unlike ssh://-oProxyCommand=pwnme
Later revisions might have removed the submodules so we want to keep
going when there are no submodules present.
We still abort when there is an error initializing submodules.
Fixesfdroid/fdroidserver#231
Something is preventing `fdroid build --all` from exiting after a long
run. @bubu, @uniqx and I think it is because of the use of
AsynchronousFileReader, somehow it's thread does not exit. So the
workaround for now is to just try a hard exit instead of waiting for
things to finish cleanly with `sys.exit(0)`.
https://jenkins.debian.net/job/reproducible_fdroid_build_apps/94/console
When running `fdroid build --all` on a buildserver with thousands of apps,
it was frequently hitting the open file limit. This increases the open
file limit based on how many apps are being process. It is doubled to
provide a margin of safety.
There are probably open file leaks which ideally would be fixed, but this
is also useful to make things more resilient to all the random stuff apps
include in their build systems.
This creates a mirror of a full repo by downloading all files listed in
the index, and the ones that are generated based on that data, e.g. icons
of different resolutions. This could be useful for setting up mirrors of
small repositories, instead of having to learn and manage rsync or
something else for mirroring. This just needs a working repo.
It uses wget in a batch mode with the aim as being as efficient as
possible. wget mirroring over HTTP is always going to be less efficient
than rsync, but it shouldn't be so bad since it uses --continue to check
whether it has already downloaded a file. I suppose it could be extended
to use ETags for a little more efficiency.
I developed this creating a test mirror of f-droid.org, which is now a bit
ironic, since I added a specific check to prevent people from using this
on f-droid.org.
