This strips metadata and optimizes the compression of all PNGs copied
from the app's source repo as well as all the icons extracted from the
APKs. There have been exploits delivered via image metadata, and
F-Droid isn't using it all, so its best to just remove it.
This unfortunately uncompresses and recompresses the files. Luckily,
that's a lossless procedure with PNGs, and we might end up with
smaller files. The only tool I could find that strips without
changing the image data is exiftool, but that is written in Perl.
EXIF data can be abused to exploit systems a lot easier than the JPEG image
data can. The F-Droid ecosystem does not use the EXIF data, so keep things
safe and strip it all away. There is a chance that some images might rely
on the rotation to be set by EXIF, but I think having a safe system is more
important.
If needed, only the rotation data could be saved. But that then makes it
hard to tell which images have been stripped. This way, if there is no
EXIF, it has been stripped. And if there is EXIF data, then it is suspect.
https://securityaffairs.co/wordpress/51043/mobile-2/android-cve-2016-3862-flaw.htmlhttps://threatpost.com/google-shuts-down-potentially-massive-android-bug/120393/https://blog.sucuri.net/2013/07/malware-hidden-inside-jpg-exif-headers.html
The big downside of this is that it decompresses and recompresses the
image data. That should be replaced by a technique from jhead,
exiftool, ObscuraCam, etc. that only strips the metadata.
If I manually run some steps of the process, not all of the normal cruft
might be left behind. I'm not really sure which of the multiple copies of
the images are actually required, but these both seem to work when present.
Once `sudo` has been used to execute the commands in sudo=, then it should
be removed from the build VM. That prevents any other part of the build
from using sudo. That means that all commands run with `sudo` must be
committed to fdroiddata.git, making them very visible.
closes#424
The SSH key fingerprint is used in the filename. The base64 used for SSH
key fingerprints includes /. Not all keys will end up having a / in them.
For those that do, this will crash since the ssh key filename ends up being
non-existent dirs:
$ fdroid nightly
Importing keystore /home/mhoffmann/.android/debug.keystore to /tmp/.cqswaeo8/.keystore.p12...
MAC verified OK
writing RSA key
CRITICAL: Unknown exception found!
Traceback (most recent call last):
File "/usr/lib/python3.6/shutil.py", line 544, in move
os.rename(src, real_dst)
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/.cqswaeo8/.privkey' -> '/tmp/.cqswaeo8/debug_keystore_PZtS/4Tzk4dpzKiX9AAf1GrhAVi9U7UE1aYEHr6evKo_id_rsa'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/mhoffmann/projects/oss/fdroidserver/fdroid", line 156, in <module>
main()
File "/home/mhoffmann/projects/oss/fdroidserver/fdroid", line 132, in main
mod.main()
File "/home/mhoffmann/projects/oss/fdroidserver/fdroidserver/nightly.py", line 284, in main
privkey = _ssh_key_from_debug_keystore()
File "/home/mhoffmann/projects/oss/fdroidserver/fdroidserver/nightly.py", line 73, in _ssh_key_from_debug_keystore
shutil.move(privkey, ssh_private_key_file)
File "/usr/lib/python3.6/shutil.py", line 558, in move
copy_function(src, real_dst)
File "/usr/lib/python3.6/shutil.py", line 257, in copy2
copyfile(src, dst, follow_symlinks=follow_symlinks)
File "/usr/lib/python3.6/shutil.py", line 121, in copyfile
with open(dst, 'wb') as fdst:
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/.cqswaeo8/debug_keystore_PZtS/4Tzk4dpzKiX9AAf1GrhAVi9U7UE1aYEHr6evKo_id_rsa'
These were both spamming the output with lots of confusing messages, even
when --verbose was not used. Jarsigner especially has confusing messages,
since it has warnings that do not pertain to APK signatures at all, like
the ones about timestamps and missing Certificate Authority.
closes#405
fd-commit and checkupdates both require that there are two name fields,
AutoName: and Name:. This is only used for the commit messages. Since the
current devs do it manually, we can remove the fd-commit shell script, then
focus on checkupdates when revamping AutoName/Name.
https://botbot.me/freenode/fdroid-dev/msg/82539152
A full run of the test suite takes quite a bit of time. This removes one
of the 3 runs from the main 'tests' job, and puts it into the Fedora job.
That test run is mostly to make sure the setup.py and source tarball are
correctly, so that doesn't affect merge requests very often.
This also tests `pip install --user`, which was not really being tested
before.
Just give a proper error message rather than this stack trace:
Traceback (most recent call last):
File "/home/hans/code/fdroid/server/fdroid", line 156, in <module>
main()
File "/home/hans/code/fdroid/server/fdroid", line 132, in main
mod.main()
File "/export/share/code/fdroid/server/fdroidserver/init.py", line 148, in main
if os.path.isfile(aapt):
File "/usr/lib/python3.5/genericpath.py", line 30, in isfile
st = os.stat(path)
This rsync hung because of an SSH unknown key prompt. Since this is just
the vm host sshing to the vm guest, it is not essential to check the host
keys.
These are only used for checking syntax in buildserver/Vagrantfile.
Not requiring ruby makes doing CI tests on lots of distros easier
and faster. dash is an 'essential' package on Debian derivs, so
those tests will always be run somewhere.
