1
0
mirror of https://gitlab.com/fdroid/fdroidserver.git synced 2024-11-09 00:40:11 +01:00
Commit Graph

5100 Commits

Author SHA1 Message Date
Hans-Christoph Steiner
67b9514c5a update: strip EXIF data from all JPEGs
EXIF data can be abused to exploit systems a lot easier than the JPEG image
data can.  The F-Droid ecosystem does not use the EXIF data, so keep things
safe and strip it all away.  There is a chance that some images might rely
on the rotation to be set by EXIF, but I think having a safe system is more
important.

If needed, only the rotation data could be saved.  But that then makes it
hard to tell which images have been stripped.  This way, if there is no
EXIF, it has been stripped.  And if there is EXIF data, then it is suspect.

https://securityaffairs.co/wordpress/51043/mobile-2/android-cve-2016-3862-flaw.html
https://threatpost.com/google-shuts-down-potentially-massive-android-bug/120393/
https://blog.sucuri.net/2013/07/malware-hidden-inside-jpg-exif-headers.html

The big downside of this is that it decompresses and recompresses the
image data.  That should be replaced by a technique from jhead,
exiftool, ObscuraCam, etc. that only strips the metadata.
2017-12-14 16:57:22 +01:00
Hans-Christoph Steiner
bde0558d82 update: reject APKs with invalid file sig, probably Janus exploits
This just checks the first four bytes of the APK file, aka the "file
signature", to make sure it is the ZIP signature and not the DEX signature.
This was checked against the test APK, and I ran it against some known
malware and all of f-droid.org to make sure it works.

All valid ZIP files (therefore APK files) should start with the ZIP
Local File Header of four bytes.

https://www.guardsquare.com/en/blog/new-android-vulnerability-allows-attackers-modify-apps-without-affecting-their-signatures
2017-12-14 16:57:22 +01:00
Hans-Christoph Steiner
5ce950e748 update: print warnings for all KnownVulns found
Some baby steps towards making the KnownVuln stuff more visible.
2017-12-14 16:57:22 +01:00
Hans-Christoph Steiner
ca50adb2e5 update: switch tests to using standardized setUp() method 2017-12-14 16:57:22 +01:00
Hans-Christoph Steiner
61aac0503a Merge branch 'fixFlavor' into 'master'
Regex only for flavor blocks: flavor { ... }

See merge request fdroid/fdroidserver!407
2017-12-14 16:56:01 +01:00
Hans-Christoph Steiner
bec4f7d547 add Conversations as gradle flavor test case 2017-12-14 16:52:02 +01:00
Hans-Christoph Steiner
538d7155b4 jenkins-build-all: improve detection of working buildserver VM
If I manually run some steps of the process, not all of the normal cruft
might be left behind.  I'm not really sure which of the multiple copies of
the images are actually required, but these both seem to work when present.
2017-12-14 10:41:11 +01:00
Hans-Christoph Steiner
cc0399e740 Merge branch '428-Signature-key-fingerprint-of-file-stats-publishsigkeys-jar-does-not-match-repo_key_sha256-in-config-py' into 'master'
jenkins test: clear singing-key-fingerpring from previous run

Closes #428

See merge request fdroid/fdroidserver!410
2017-12-14 08:50:10 +00:00
Michael Pöhn
93c1c23a4b jenkins test: clear singing-key-fingerpring from previous run 2017-12-13 17:39:55 +01:00
Hans-Christoph Steiner
1f5818020d Merge branch 'fdroidserver-liberapay' into 'master'
Add Liberapay support

See merge request fdroid/fdroidserver!408
2017-12-12 16:55:05 +01:00
Hans-Christoph Steiner
8b9b18ceed fix metadata_v0 tests
The old metadata format didn't know anything of LiberapayID, so no need to
check for it.
2017-12-12 16:54:35 +01:00
Andrea Scarpino
19a83b6219 Add Liberapay support 2017-12-12 11:53:31 +01:00
Hans-Christoph Steiner
858b398927 Merge branch '1.0-polish' into 'master'
1.0 polish

Closes #424

See merge request fdroid/fdroidserver!405
2017-12-11 21:44:16 +00:00
Hans-Christoph Steiner
5c9d46b5b2 remove XML files from bash completion, they are not supported anymore 2017-12-11 22:11:16 +01:00
Hans-Christoph Steiner
bb758d3f00 build: apt-get purge sudo after using it for sudo= build field
Once `sudo` has been used to execute the commands in sudo=, then it should
be removed from the build VM.  That prevents any other part of the build
from using sudo.  That means that all commands run with `sudo` must be
committed to fdroiddata.git, making them very visible.

closes #424
2017-12-11 22:11:16 +01:00
Hans-Christoph Steiner
4af8f196e3 metadata: error if .fdroid.txt exists, it is unsupported
Using .txt format has bugs when used embedded in an app's git repo, so
tell the user to use YAML or JSON.

#364
2017-12-11 22:11:16 +01:00
Hans-Christoph Steiner
1d83d4396c makebuildserver: quiet rsync for copy_caches_from_host 2017-12-11 22:11:16 +01:00
Hans-Christoph Steiner
4835d7c20c Merge branch 'rational-jarsigner-logging' into 'master'
handle jarsigner/apksigner output cleanly for rational logging

Closes #405

See merge request fdroid/fdroidserver!404
2017-12-11 20:27:06 +00:00
tobiasKaminsky
03f301470e
regex only for flavor blocks: flavor { ... } and nothing else 2017-12-11 14:29:32 +01:00
Hans-Christoph Steiner
6640f276b0 Revert "makebuildserver: update SHA-256 for platform-27_r01.zip"
Looks like Google switched back to the old binary, which I guess is
good news?
https://issuetracker.google.com/issues/70292819

This reverts commit 956660085a.

!401
2017-12-08 09:44:10 +01:00
Hans-Christoph Steiner
5ac943a3f2 Merge branch 'yml_completion' into 'master'
bash completion: use correct yml suffix

See merge request fdroid/fdroidserver!406
2017-12-07 23:10:18 +00:00
Marcus Hoffmann
9270e68fe2 bash completion: use correct yml suffix 2017-12-07 23:29:34 +01:00
Hans-Christoph Steiner
4f43099c88 Merge branch 'more-nightly' into 'master'
More `fdroid nightly` polish

Closes #423

See merge request fdroid/fdroidserver!402
2017-12-07 22:11:40 +00:00
Hans-Christoph Steiner
7b52722d12 nightly: replace / from fingerprint in SSH key filename, fixes #423
The SSH key fingerprint is used in the filename.  The base64 used for SSH
key fingerprints includes /.  Not all keys will end up having a / in them.
For those that do, this will crash since the ssh key filename ends up being
non-existent dirs:

$ fdroid nightly
Importing keystore /home/mhoffmann/.android/debug.keystore to /tmp/.cqswaeo8/.keystore.p12...
MAC verified OK
writing RSA key
CRITICAL: Unknown exception found!
Traceback (most recent call last):
  File "/usr/lib/python3.6/shutil.py", line 544, in move
    os.rename(src, real_dst)
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/.cqswaeo8/.privkey' -> '/tmp/.cqswaeo8/debug_keystore_PZtS/4Tzk4dpzKiX9AAf1GrhAVi9U7UE1aYEHr6evKo_id_rsa'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/mhoffmann/projects/oss/fdroidserver/fdroid", line 156, in <module>
    main()
  File "/home/mhoffmann/projects/oss/fdroidserver/fdroid", line 132, in main
    mod.main()
  File "/home/mhoffmann/projects/oss/fdroidserver/fdroidserver/nightly.py", line 284, in main
    privkey = _ssh_key_from_debug_keystore()
  File "/home/mhoffmann/projects/oss/fdroidserver/fdroidserver/nightly.py", line 73, in _ssh_key_from_debug_keystore
    shutil.move(privkey, ssh_private_key_file)
  File "/usr/lib/python3.6/shutil.py", line 558, in move
    copy_function(src, real_dst)
  File "/usr/lib/python3.6/shutil.py", line 257, in copy2
    copyfile(src, dst, follow_symlinks=follow_symlinks)
  File "/usr/lib/python3.6/shutil.py", line 121, in copyfile
    with open(dst, 'wb') as fdst:
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/.cqswaeo8/debug_keystore_PZtS/4Tzk4dpzKiX9AAf1GrhAVi9U7UE1aYEHr6evKo_id_rsa'
2017-12-07 22:39:32 +01:00
Hans-Christoph Steiner
5d56841f8a nightly: prompt user to create a debug.keystore if its not there
#423
2017-12-07 22:39:32 +01:00
Hans-Christoph Steiner
c823d4c4a8 nightly: fix QR icon.png generation 2017-12-07 22:39:32 +01:00
Hans-Christoph Steiner
5d54446efc update: do not replace Name/Summary from template unless blank
`fdroid nightly` needs this change so it can set the Summary using the
template.
2017-12-07 22:39:32 +01:00
Hans-Christoph Steiner
4afe5aefd5 nightly: create app metadata using template of parsed data 2017-12-07 22:39:32 +01:00
Marcus
87018d45e2 Merge branch 'remove-fd-commit' into 'master'
remove fd-commit, no active devs use it, and requires Auto Name/Name

See merge request fdroid/fdroidserver!392
2017-12-07 20:31:09 +00:00
Hans-Christoph Steiner
adc0c23db4 Merge branch 'help_message_fix' into 'master'
metadata: make help for common -W option clearer

See merge request fdroid/fdroidserver!350
2017-12-07 20:03:17 +00:00
Marcus Hoffmann
0f18005104 metadata: make help for metadata -W option clearer
specify possible choices and provide clearer help text
2017-12-07 20:28:27 +01:00
Hans-Christoph Steiner
6228162cbd handle jarsigner/apksigner output cleanly for rational logging
These were both spamming the output with lots of confusing messages, even
when --verbose was not used.  Jarsigner especially has confusing messages,
since it has warnings that do not pertain to APK signatures at all, like
the ones about timestamps and missing Certificate Authority.

closes #405
2017-12-07 17:32:14 +01:00
Hans-Christoph Steiner
7f4d84684e Merge branch 'syntaxfix' into 'master'
correct "usage" output (--help; see #405)

See merge request fdroid/fdroidserver!403
2017-12-07 14:17:25 +00:00
Izzy
0a1fe3dc7e correct "usage" output (--help; see #405) 2017-12-07 14:51:27 +01:00
Hans-Christoph Steiner
0e37a18d83 makebuildserver: fix join() syntax error, it needs a list/tuple
If only there was a way to test this without taking hours to run...

fixes 964ef996a0
2017-12-07 11:52:52 +01:00
relan
8ffed8750c Merge branch 'update-platform-27_r01' into 'master'
makebuildserver: update SHA-256 for platform-27_r01.zip

See merge request fdroid/fdroidserver!401
2017-12-07 06:05:46 +00:00
Hans-Christoph Steiner
6902160e89 remove fd-commit, no active devs use it, and requires Auto Name/Name
fd-commit and checkupdates both require that there are two name fields,
AutoName: and Name:.  This is only used for the commit messages.  Since the
current devs do it manually, we can remove the fd-commit shell script, then
focus on checkupdates when revamping AutoName/Name.

https://botbot.me/freenode/fdroid-dev/msg/82539152
2017-12-06 22:48:08 +01:00
Hans-Christoph Steiner
956660085a makebuildserver: update SHA-256 for platform-27_r01.zip
Someone forgot to call this _r02.zip:
-ro.​build.​version.​incremental=4402310
+ro.​build.​version.​incremental=4458339

https://verification.f-droid.org/build-metadata/platform-27_r01.html

!364
2017-12-06 22:05:09 +01:00
Hans-Christoph Steiner
6fc507da39 Merge branch 'gitlab-ci-testing-xenial-fedora' into 'master'
gitlab CI runs on  Debian/testing,  Ubuntu/xenial,  Fedora

See merge request fdroid/fdroidserver!398
2017-12-06 19:44:58 +00:00
Hans-Christoph Steiner
05abbfbabd gitlab-ci: move sdist test run to new fedora job
A full run of the test suite takes quite a bit of time.  This removes one
of the 3 runs from the main 'tests' job, and puts it into the Fedora job.
That test run is mostly to make sure the setup.py and source tarball are
correctly, so that doesn't affect merge requests very often.

This also tests `pip install --user`, which was not really being tested
before.
2017-12-06 20:20:17 +01:00
Hans-Christoph Steiner
1a77c6af38 init: fix test for aapt when no aapt has been found
Just give a proper error message rather than this stack trace:

Traceback (most recent call last):
  File "/home/hans/code/fdroid/server/fdroid", line 156, in <module>
    main()
  File "/home/hans/code/fdroid/server/fdroid", line 132, in main
    mod.main()
  File "/export/share/code/fdroid/server/fdroidserver/init.py", line 148, in main
    if os.path.isfile(aapt):
  File "/usr/lib/python3.5/genericpath.py", line 30, in isfile
    st = os.stat(path)
2017-12-06 20:20:17 +01:00
Hans-Christoph Steiner
964ef996a0 makebuildserver: make copy_caches_from_host do rsync like fdroid build
This rsync hung because of an SSH unknown key prompt.  Since this is just
the vm host sshing to the vm guest, it is not essential to check the host
keys.
2017-12-06 20:20:17 +01:00
Hans-Christoph Steiner
a0a68c7a13 setup requires Babel aka python3-babel to compile translations
https://forum.f-droid.org/t/f-droid-server-building-error/1670
2017-12-06 20:20:17 +01:00
Hans-Christoph Steiner
dda9c8b774 gitlab-ci: add pip install test on Arch Linux 2017-12-06 20:20:17 +01:00
Hans-Christoph Steiner
1b1475c982 gitlab-ci: add test runs on Ubuntu, Debian/testing, Fedora 2017-12-06 12:30:47 +01:00
Marcus Hoffmann
bfe2c00834 common.testCase: fix find_sdk_tools when aapt is installed in /usr/bin
The testlogic was broken when having both aapt in /usr/bin and also as
part of the android sdk.
2017-12-06 12:30:47 +01:00
Hans-Christoph Steiner
67e6cbe793 hooks/pre-commit: make ruby and dash tests optional
These are only used for checking syntax in buildserver/Vagrantfile.
Not requiring ruby makes doing CI tests on lots of distros easier
and faster. dash is an 'essential' package on Debian derivs, so
those tests will always be run somewhere.
2017-12-06 12:30:47 +01:00
Hans-Christoph Steiner
a2978a5526 common: aapt 24.0.0 (v0.2-2964546) is now required
Without a recent aapt, the <uses-permission-sdk-23> tag will not be found.
2017-12-06 12:30:47 +01:00
Hans-Christoph Steiner
657b64f6ed Merge branch 'lint-fixes' into 'master'
Lint fixes, plus changing the standard link format in descriptions

See merge request fdroid/fdroidserver!397
2017-12-06 09:41:39 +00:00
Hans-Christoph Steiner
8e1c39f791 Merge branch 'rsync_improvements' into 'master'
Rsync improvements

See merge request fdroid/fdroidserver!400
2017-12-06 08:55:19 +00:00