Run the whole test suite in an environment that is like the
buildserver guest VM. This installs python3-babel because that is
only used by the test suite, and not needed in the buildserver.
Some extra packages are required for this test run that are not
provided by the buildserver since they are not needed there:
* gnupg-agent for the full signing setup
* python3-babel for compiling localization files
* python3-clint for fancy progress bars for users
fdroid/ci-images-base is deprecated and archived.
* https://gitlab.com/eighthave/fdroidserver/-/jobs/1961701458
Collecting pynacl>=1.0.1 (from paramiko->fdroidserver==2.1a0)
Downloading 27582568be/PyNaCl-1.5.0.tar.gz (3.4MB)
Installing build dependencies: started
Installing build dependencies: finished with status 'error'
Complete output from command /builds/eighthave/fdroidserver/env/bin/python3 -m pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-xokvr6uk --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- setuptools>=40.8.0 wheel "cffi>=1.4.1; python_implementation != 'PyPy'":
Collecting setuptools>=40.8.0
Using cached 0dd4c79605/setuptools-60.5.0-py3-none-any.whl
Collecting wheel
Using cached 003e593296/wheel-0.37.1-py2.py3-none-any.whl
Collecting cffi>=1.4.1
Downloading 92de7e1217/cffi-1.15.0.tar.gz (484kB)
Collecting pycparser (from cffi>=1.4.1)
Downloading 5f610ebe42/pycparser-2.21-py2.py3-none-any.whl (118kB)
Building wheels for collected packages: cffi
Running setup.py bdist_wheel for cffi: started
Running setup.py bdist_wheel for cffi: finished with status 'error'
Complete output from command /builds/eighthave/fdroidserver/env/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-djg9jc8p/cffi/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/pip-wheel-d1knhl7l --python-tag cp37:
unable to execute 'x86_64-linux-gnu-gcc': No such file or directory
unable to execute 'x86_64-linux-gnu-gcc': No such file or directory
No working compiler found, or bogus compiler options passed to
the compiler from Python's standard "distutils" module. See
the error messages above. Likely, the problem is not related
to CFFI but generic to the setup.py of any Python package that
tries to compile C code. (Hints: on OS/X 10.8, for errors about
-mno-fused-madd see http://stackoverflow.com/questions/22313407/
Otherwise, see https://wiki.python.org/moin/CompLangPython or
the IRC channel #python on irc.libera.chat.)
Trying to continue anyway. If you are trying to install CFFI from
a build done in a different context, you can ignore this warning.
running bdist_wheel
running build
running build_py
creating build
creating build/lib.linux-x86_64-3.7
creating build/lib.linux-x86_64-3.7/cffi
copying cffi/backend_ctypes.py -> build/lib.linux-x86_64-3.7/cffi
copying cffi/vengine_gen.py -> build/lib.linux-x86_64-3.7/cffi
copying cffi/vengine_cpy.py -> build/lib.linux-x86_64-3.7/cffi
copying cffi/pkgconfig.py -> build/lib.linux-x86_64-3.7/cffi
copying cffi/lock.py -> build/lib.linux-x86_64-3.7/cffi
copying cffi/recompiler.py -> build/lib.linux-x86_64-3.7/cffi
copying cffi/__init__.py -> build/lib.linux-x86_64-3.7/cffi
copying cffi/cparser.py -> build/lib.linux-x86_64-3.7/cffi
copying cffi/ffiplatform.py -> build/lib.linux-x86_64-3.7/cffi
copying cffi/verifier.py -> build/lib.linux-x86_64-3.7/cffi
copying cffi/cffi_opcode.py -> build/lib.linux-x86_64-3.7/cffi
copying cffi/setuptools_ext.py -> build/lib.linux-x86_64-3.7/cffi
copying cffi/api.py -> build/lib.linux-x86_64-3.7/cffi
copying cffi/commontypes.py -> build/lib.linux-x86_64-3.7/cffi
copying cffi/error.py -> build/lib.linux-x86_64-3.7/cffi
copying cffi/model.py -> build/lib.linux-x86_64-3.7/cffi
copying cffi/_cffi_include.h -> build/lib.linux-x86_64-3.7/cffi
copying cffi/parse_c_type.h -> build/lib.linux-x86_64-3.7/cffi
copying cffi/_embedding.h -> build/lib.linux-x86_64-3.7/cffi
copying cffi/_cffi_errors.h -> build/lib.linux-x86_64-3.7/cffi
running build_ext
building '_cffi_backend' extension
creating build/temp.linux-x86_64-3.7
creating build/temp.linux-x86_64-3.7/c
x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/usr/include/ffi -I/usr/include/libffi -I/builds/eighthave/fdroidserver/env/include -I/usr/include/python3.7m -c c/_cffi_backend.c -o build/temp.linux-x86_64-3.7/c/_cffi_backend.o
unable to execute 'x86_64-linux-gnu-gcc': No such file or directory
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
----------------------------------------
Failed building wheel for cffi
Running setup.py clean for cffi
Failed to build cffi
Installing collected packages: setuptools, wheel, pycparser, cffi
Running setup.py install for cffi: started
Running setup.py install for cffi: finished with status 'error'
Complete output from command /builds/eighthave/fdroidserver/env/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-djg9jc8p/cffi/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-ek80c81s/install-record.txt --single-version-externally-managed --prefix /tmp/pip-build-env-xokvr6uk --compile --install-headers /builds/eighthave/fdroidserver/env/include/site/python3.7/cffi:
unable to execute 'x86_64-linux-gnu-gcc': No such file or directory
unable to execute 'x86_64-linux-gnu-gcc': No such file or directory
No working compiler found, or bogus compiler options passed to
the compiler from Python's standard "distutils" module. See
the error messages above. Likely, the problem is not related
to CFFI but generic to the setup.py of any Python package that
tries to compile C code. (Hints: on OS/X 10.8, for errors about
-mno-fused-madd see http://stackoverflow.com/questions/22313407/
Otherwise, see https://wiki.python.org/moin/CompLangPython or
the IRC channel #python on irc.libera.chat.)
Trying to continue anyway. If you are trying to install CFFI from
a build done in a different context, you can ignore this warning.
running install
running build
running build_py
creating build
creating build/lib.linux-x86_64-3.7
creating build/lib.linux-x86_64-3.7/cffi
copying cffi/backend_ctypes.py -> build/lib.linux-x86_64-3.7/cffi
copying cffi/vengine_gen.py -> build/lib.linux-x86_64-3.7/cffi
copying cffi/vengine_cpy.py -> build/lib.linux-x86_64-3.7/cffi
copying cffi/pkgconfig.py -> build/lib.linux-x86_64-3.7/cffi
copying cffi/lock.py -> build/lib.linux-x86_64-3.7/cffi
copying cffi/recompiler.py -> build/lib.linux-x86_64-3.7/cffi
copying cffi/__init__.py -> build/lib.linux-x86_64-3.7/cffi
copying cffi/cparser.py -> build/lib.linux-x86_64-3.7/cffi
copying cffi/ffiplatform.py -> build/lib.linux-x86_64-3.7/cffi
copying cffi/verifier.py -> build/lib.linux-x86_64-3.7/cffi
copying cffi/cffi_opcode.py -> build/lib.linux-x86_64-3.7/cffi
copying cffi/setuptools_ext.py -> build/lib.linux-x86_64-3.7/cffi
copying cffi/api.py -> build/lib.linux-x86_64-3.7/cffi
copying cffi/commontypes.py -> build/lib.linux-x86_64-3.7/cffi
copying cffi/error.py -> build/lib.linux-x86_64-3.7/cffi
copying cffi/model.py -> build/lib.linux-x86_64-3.7/cffi
copying cffi/_cffi_include.h -> build/lib.linux-x86_64-3.7/cffi
copying cffi/parse_c_type.h -> build/lib.linux-x86_64-3.7/cffi
copying cffi/_embedding.h -> build/lib.linux-x86_64-3.7/cffi
copying cffi/_cffi_errors.h -> build/lib.linux-x86_64-3.7/cffi
running build_ext
building '_cffi_backend' extension
creating build/temp.linux-x86_64-3.7
creating build/temp.linux-x86_64-3.7/c
x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/usr/include/ffi -I/usr/include/libffi -I/builds/eighthave/fdroidserver/env/include -I/usr/include/python3.7m -c c/_cffi_backend.c -o build/temp.linux-x86_64-3.7/c/_cffi_backend.o
unable to execute 'x86_64-linux-gnu-gcc': No such file or directory
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
----------------------------------------
Command "/builds/eighthave/fdroidserver/env/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-djg9jc8p/cffi/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-ek80c81s/install-record.txt --single-version-externally-managed --prefix /tmp/pip-build-env-xokvr6uk --compile --install-headers /builds/eighthave/fdroidserver/env/include/site/python3.7/cffi" failed with error code 1 in /tmp/pip-install-djg9jc8p/cffi/
----------------------------------------
Command "/builds/eighthave/fdroidserver/env/bin/python3 -m pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-xokvr6uk --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- setuptools>=40.8.0 wheel "cffi>=1.4.1; python_implementation != 'PyPy'"" failed with error code 1 in None
mypy 0.901 was released and everything broke:
https://gitlab.com/linsui/fdroidserver/-/jobs/1330206567
My point is to reduce the number of false job fails like this one. We
have a lot of checkers running, I really think they need to prove they are
adding value before we invest any time maintaining them. mypy is still
"wait and see" in terms of the adding any value.
!951
Following the pattern of the gradle bot, this will check the transparency
log for any new NDK release. If there are any, it will make a merge
request from @fdroid-bot.
`Builds:` is directly in the .yml metadata, and the internal representation
now uses the same "Builds" name. Reading in .txt metadata files used the
name "builds" for that. So this test now uses "Builds" as canonical.
ci-images-server was created separately from ci-images-base back in the day
when we had to support Debian/stretch without stretch-backports. Those
days are gone now, so ci-images-server has been archived and should no
longer be used.
GIT_DEPTH sets how many commits of history to clone in CI Jobs.
gitlab.com defaults to 50 with a max of 1000. The metadata_v0 job is
the only job that needs history, and it needs more than 50. So this
sets the default to 1, then metadata_v0 to 1000.
https://docs.gitlab.com/ee/ci/pipelines/settings.html#git-shallow-clone
Liberapay was originally included using a numeric ID, since they had
not yet finalized the public URLs. Now it is a username. So this
logic prefers the username in Liberapay: field, and keeps the old
LiberapayID: to ease migration. LiberapayID: will not override
Liberapay:. Clients are expected to prefer Liberapay: over LiberapayID:
This should make it easier to accept merge requests where there are only
cosmetic problems with them. pep8/pylint/pyflakes runs can then be disabled
in the 'test' job by not installing the in the ci-images-server base image.
Now that androguard is working, there should be no need for a specific aapt
version. The aapt included in Ubuntu LTS should always work fine when
androguard handles the bulk of the work.
This code has never been used and contains some insecure uses of shell=True
Building Kivy apps should be done with the buildozer=yes method. The
buildozer method should probably be moved to a provisioner once that is in
place.
This uses the commit ID of the release tags, rather than the release tag
itself so that contributor forks do not need to include the tags in them
for this test to work.
The COMMIT_ID should be bumped after each release, so that the list of sed
hacks needed does not continuously grow.
A full run of the test suite takes quite a bit of time. This removes one
of the 3 runs from the main 'tests' job, and puts it into the Fedora job.
That test run is mostly to make sure the setup.py and source tarball are
correctly, so that doesn't affect merge requests very often.
This also tests `pip install --user`, which was not really being tested
before.
For cases like the OpenVPN vuln that was recently announced, it is useful
for fdroiddata maintainers to be able to mark builds that have known
vulnerabilities.
