There are all sorts of unfiltered user inputs like tag and branch names in
source repos. If those names are fed into popen calls that use shell=True,
that opens up a wide range of exploits. All core operations should never
use shell=True.
This is a quick and very incomplete addition of '--' to command line calls
to source VCSs like git and hg that could manipulated by malicious
tag/branch names or other vectors.
These were all manually tested by calling the command lines on my own
machine.
This lets `fdroid scanner my.package.name` run without requiring that the
versionCode is also specified. It also allows scanner.scan_source() to be
called as a function in the public API of fdroidserver.
We should be replacing all our custom git shell commands with python3-git,
since it is a common library for doing that. It will receive a lot more
attention and maintenance than our code for doing it. For example, we
should not ever use shell=True, since that opens up a lot of security
risks.
This code has never been used and contains some insecure uses of shell=True
Building Kivy apps should be done with the buildozer=yes method. The
buildozer method should probably be moved to a provisioner once that is in
place.
The currently included Qt has known security issues and is outdated. This
can now be replaced by downloading and installing the Qt installer using
the sudo= build field. @relan's provisioner system will also replace this
once that's done. There are only two apps that currently use the Qt stuff:
* csd.qtproject.minesweeper
* org.openorienteering.mapper
Only start new builds for 12 hours. This ensures we publish new builds
often enough even on long backlogs.
This could be made configurable at a later point.
This uses the commit ID of the release tags, rather than the release tag
itself so that contributor forks do not need to include the tags in them
for this test to work.
The COMMIT_ID should be bumped after each release, so that the list of sed
hacks needed does not continuously grow.
GitLab storage provides two mirrors by default:
* https://gitlab.com/user/repo/raw/master/fdroid/repo
* https://user.gitlab.io/repo/fdroid/repo
While the F-Droid client will happily fetch the index*.jar files and
parse them from either of these two mirrors, only the GitLab Pages
mirror will serve files with the correct mime type. Many repos
tend to put index.html files (and associated .css/.js/image files) in
the root of a repository to provide information about that repo.
One example is RepoMaker. The way in which RepoMaker decides the public
URL of a repo, is to take the first mirror in the list. This means that
the URL which RepoMaker directs people to for GitLab storage returns a
.html document in text/plain, which means that it is not rendered.
We could change RepoMaker so that it takes the last mirror, and then it
woruld work. However there is something nice about the first mirror in a
list being the most authoritative (even though the mirror order doesn't
- and perhaps shouldn't have any specific meaning).
There have been frequent failures on import, some bugs suggest that it
might be because these dirs are missing. They would get wiped by a
`git clean -fdx`.
ERROR: Could not build app org.fdroid.fdroid due to unknown error: Traceback (most recent call last):
File "/var/lib/jenkins/userContent/reproducible/reproducible_setup_fdroid_build_environment/fdroidserver/build.py", line 1202, in main
options.onserver, options.refresh):
File "/var/lib/jenkins/userContent/reproducible/reproducible_setup_fdroid_build_environment/fdroidserver/build.py", line 972, in trybuild
build_server(app, build, vcs, build_dir, output_dir, log_dir, force)
File "/var/lib/jenkins/userContent/reproducible/reproducible_setup_fdroid_build_environment/fdroidserver/build.py", line 82, in build_server
logging.debug(_('Fetched buildserverid from VM: ') + buildserverid)
TypeError: Can't convert 'bytes' object to str implicitly
We remove the whole "build" directory while cleaning source code tree
because Gradle can leave there files even after "gradle clean". But some
projects (Mozilla Fennec) actually have useful stuff checked into VCS
under the "build" directory.
Remove only those subdirectories that we known for sure are leftovers
from Gradle.
Fixesfdroid/fdroidserver#438.
