Initially, the scanner used libmagic which used magic numbers in the file's
content to detect what kind of file it appears to be. Since that library isn't
available on all systems, we added support for two other libraries, mimetypes
amongst them.
The issue with mimetypes is that it only uses the file's extension, not its
actual content. So this ends in variable behaviour depending on what system
you're using fdroidserver on. For example, an executable binary without
extension would be ignored if mimetypes was being used.
We now drop all libraries - mimetypes too as it depends on the system's
mime.types file - and instead check extensions ourselves. On top of that, do
a simple binary content check to find binary executables that don't have an
extension.
The new in-house code without any dependencies doesn't add any new checks, so
no builds should break. The current checks still work:
% fdroid scanner app.openconnect:1029
[...]
Found executable binary at assets/raw/armeabi/curl
Found executable binary at assets/raw/mips/curl
Found executable binary at assets/raw/x86/curl
Found JAR file at lib/XposedBridgeApi-54.jar
Found JAR file at libs/acra-4.5.0.jar
Found JAR file at libs/openconnect-wrapper.jar
Found JAR file at libs/stoken-wrapper.jar
Found shared library at libs/armeabi/libopenconnect.so
Found shared library at libs/armeabi/libstoken.so
Found shared library at libs/mips/libopenconnect.so
Found shared library at libs/mips/libstoken.so
Found shared library at libs/x86/libopenconnect.so
Found shared library at libs/x86/libstoken.so
Import tweaks and tests
This makes the code for `fdroid import` a bit more flexible and adds tests for the first time. It also comments out options in `examples/config.py` that just mirror the defaults to make it clear that they are defaults, and help illustrate other options (this is standard procedure in default config files).
More details in the commit messages.
See merge request !76
replace deprecated optparse with argparse
squashed and rebased merge request fdroid/fdroidserver!74
following guidelines from:
https://docs.python.org/2/library/argparse.html#upgrading-optparse-code
except, still using option = parse.parse_args() instead of args = ...
- using the following script in folder fdroidserver:
```
for i in *.py; do
sed -i -e 's/optparse/argparse/' \
-e 's/OptionParser/ArgumentParser/' \
-e 's/OptionError/ArgumentError/' \
-e 's/add_option/add_argument/' \
-e 's/(options, args) = parser/options = parser/' \
-e 's/options, args = parser/options = parser/' \
-e 's/Usage: %prog/%(prog)s/' $i;
done
```
- use ArgumentParser argument to replace (option, args) = parser.parse()
call
- use parser.error(msg) instead of raise ArgumentException as suggested
in https://docs.python.org/2/library/argparse.html#exiting-methods
- in fdroid catch ArgumentError instead of OptionError
See merge request !75
If for whatever reason the update check results in an older version that we
didn't package, don't "update" to that version if we already packaged newer
versions.
following guidelines from:
https://docs.python.org/2/library/argparse.html#upgrading-optparse-code
except, still using option = parse.parse_args() instead of args = ...
- using the following script in folder fdroidserver:
for i in *.py; do
sed -i -e 's/optparse/argparse/' \
-e 's/OptionParser/ArgumentParser/' \
-e 's/OptionError/ArgumentError/' \
-e 's/add_option/add_argument/' \
-e 's/(options, args) = parser/options = parser/' \
-e 's/options, args = parser/options = parser/' \
-e 's/Usage: %prog/%(prog)s/' $i;
done
- use ArgumentParser argument to replace (option, args) = parser.parse()
call
- use parser.error(msg) instead of raise ArgumentException as suggested
in https://docs.python.org/2/library/argparse.html#exiting-methods
- in fdroid catch ArgumentError instead of OptionError
CI: Split up package installing
Advantages:
* Easier to tell why we need each package
* apt-get install output is less scary/huge
* CI job is split in more, smaller steps easier to debug
See merge request !72
Advantages:
* Easier to tell why we need each package
* apt-get install output is less scary/huge
* CI job is split in more, smaller steps easier to debug
install build dependencies for Pillow
When Pillow is installed with pip, it needs to compile the C code against a few libraries, like libjpeg. Pillow is included in Debian/Ubuntu as 'python-imaging', so just install the build
dependencies of that, and it should all be golden.
See merge request !71
fdroid --version and shared tests between gitlab and jenkins
This adds a `fdroid --version` flag for people to easily see the exact version of fdroidserver in use. It'll also report the version using `git describe` when running from git.
The other moves the extended tests out of `./jenkins-build` into a common script for both gitlab-ci and jenkins.
See merge request !70
