1
0
mirror of https://gitlab.com/fdroid/fdroidserver.git synced 2024-09-11 23:23:27 +02:00
Commit Graph

3405 Commits

Author SHA1 Message Date
Daniel Martí
d8e1f296e0 scanner: don't error on partially used globs
This meant that using something like `scanignore=*` would error if there were
ignores happening in some directories/files, but not all.

Fixes #110
2015-10-03 17:00:22 -07:00
Daniel Martí
59f5d19dfe Rework path glob expansion
Slightly simplifies the whole thing and lets us map what each resulting path
comes from. This will be useful to fix #110 later on.
2015-10-03 16:52:23 -07:00
Daniel Martí
75bde83fb8 import: fix issue tracker links 2015-10-03 16:28:48 -07:00
Daniel Martí
712deb4396 Don't use exists() on symlinks
If they are broken, exists() will return false. islink() will return true in
both cases.
2015-09-30 17:05:44 -07:00
Daniel Martí
1ed33bfa96 Smarter archive policy apk moving
* Move apks back from the archive repo (e.g. if builds are disabled thus the
  number of apks on the main repo decreases)
* Also move apk signatures back and forth
2015-09-30 17:05:44 -07:00
Daniel Martí
4ea5ce8889 checkupdates: Ignore xml tags in string content
This allows us to fetch strings like the following:

<string name="app_name">foo <xliff:g>bar</xliff:g></string>

Up until now, using .text would only return "foo ", but if we use .tostring()
with the text method, it converts everything into plain text for us resulting
in "foo bar".
2015-09-30 16:35:41 -07:00
Daniel Martí
79475d055f Fetch string contents in a safer way 2015-09-30 16:35:41 -07:00
Daniel Martí
cc017d70d2 Fix for bb2cbd527b 2015-09-29 20:39:36 -07:00
Daniel Martí
bb2cbd527b Increase wiki error detail length limit
8k is too small, e.g. when make -jN errors or when there are a lot of scanner
errors and warnings. 16k should be better. Also, use "[...]" to make it
obvious that the output is truncated.
2015-09-29 12:51:11 -07:00
Daniel Martí
9c3fd97530 fdroid: don't treat MetaDataException as unknown
Up until now, if it was thrown, fdroid would crash with a full stacktrace as
if the program had stopped unexpectedly. Now it just prints the metadata
error, as intended.
2015-09-26 16:53:08 -07:00
Daniel Martí
811fcd320f rewritemeta: add -l option to list files 2015-09-24 22:54:52 -07:00
Daniel Martí
911994fc99 Make write_metadata take a writer
This will let rewritemeta report format issues without writing to disk.
2015-09-24 22:38:33 -07:00
Daniel Martí
d88914b466 Replace MD5withRSA with SHA1withRSA. Fixes #26.
Looks to me like we were using SHA1withRSA all along. Tested that everything
still works with a test repo.

As Hans reports, SHA1withRSA has been the default on Android tools for a long
time and it's supported on all Android versions.
2015-09-24 18:53:04 -07:00
Ciaran Gultnieks
84573e4e5c Push a more complete/standard fdroidserver layout to buildserver 2015-09-24 09:19:17 +00:00
Daniel Martí
27a5dc8429 checkupdates: ignore gradle comments 2015-09-23 18:52:59 -07:00
Daniel Martí
80a8a39f6f build: make sure .gradle/ is removed at clean 2015-09-23 13:02:22 -07:00
Daniel Martí
1b8e33f714 Update build status badge to new CI url 2015-09-22 21:46:14 -07:00
Daniel Martí
b589595e45 scanner: skip symlinks to avoid issues
If they resolve to a missing file, we don't care about them. If they resolve
to an existing file, we'll scan that file anyway.
2015-09-22 13:03:42 -07:00
Daniel Martí
b9b5b5d1b0 scanner: don't crash on unused gradle files 2015-09-22 13:02:26 -07:00
Daniel Martí
83daf71338 gradle: don't strip out comments
This was never intended. Comments can help sed usage, and can still be useful
in source tarballs.
2015-09-22 11:52:16 -07:00
Daniel Martí
21bdd2941c Merge branch 'asynchronous_reader' into 'master'
Add asynchronous filereader, fix python3 lockups

with the current implementation of AsynchronousFileReader with Python 3 FDroidPopen deadlocks with 100% CPU-utilization

The code is from https://github.com/soxofaan/asynchronousfilereader

I hope the MIT-Licence makes no problems




See merge request !80
2015-09-21 22:09:23 +00:00
Daniel Martí
39e3bfa0ce Merge branch 'docs' into 'master'
docs: Add list of trusted maven repos



See merge request !81
2015-09-21 19:58:50 +00:00
Boris Kraut
22bdcb7acf docs: Add list of trusted maven repos 2015-09-21 21:51:35 +02:00
Daniel Martí
6c5bbb2431 Merge branch 'master' into 'master'
Added support for gradle flavor specific dependencies in usual suspects check

When you have flavor specific dependencies in your project, you specify them in the build.gradle file with a prefixed *compile* command:

```
compile 'normal dependency'
myflavorCompile 'only for myflavor'
```

The usual suspects check just searches for some often used library names in the whole build.gradle file. I added another condition, so dependencies for not build flavors are not checked.

See merge request !77
2015-09-21 17:06:38 +00:00
NeroBurner
694a4ce7da Add asynchronous filereader, fix python3 lockups 2015-09-20 20:57:53 +02:00
Jan Kühle
450409ef14 Added support for gradle flavor specific dependencies in usual suspects check. 2015-09-19 14:43:42 +02:00
Daniel Martí
e285a8ac31 Remove Dogecoin donation support
Not nearly as popular as bitcoin and litecoin. To the point where it hadn't
been used yet.

Bump index version to 14 to mark its removal.
2015-09-18 19:06:10 -07:00
Daniel Martí
d07bf0754a metadata: fix html paragraph tag regression
Spotted in fdroid/fdroidclient#422
2015-09-17 14:14:54 -07:00
Daniel Martí
d7d9e38ef7 scanner: use a blacklist instead of a whitelist
This is much closer to what we did before with mimetypes. Using a whitelist
turns out to be a bad idea since repositories seem to be randomly filled with
executable images and documents, which trigger the scanner.

In an ideal world the scanner would complain about all of those. For now, just
warn about the possibility of them being hidden binaries.
2015-09-16 17:23:34 -07:00
Daniel Martí
feaf3d04e5 scanner: also ignore ttf and otf files 2015-09-16 17:14:06 -07:00
Daniel Martí
da2a4f2cf5 scanner: improve usual suspect output 2015-09-16 17:13:54 -07:00
Daniel Martí
d155648a01 Merge branch 'fix_licence_setup' into 'master'
Update Licence in setup.py to AGPLv3

See merge request !79
2015-09-16 21:29:41 +00:00
NeroBurner
9e8d9bebfa Update Licence in setup.py to AGPLv3 2015-09-16 22:55:22 +02:00
Daniel Martí
2bf04f8054 import: fix import -u from fdroiddata
Have -u take precedence over .git checking to fix import -u being run from
fdroiddata.
2015-09-15 14:06:31 -07:00
Daniel Martí
41370a12b0 Deduplicate statsdir 2015-09-14 18:13:03 -07:00
Daniel Martí
5cb47203b3 Replace some hard-coded paths with os.path.join 2015-09-14 18:12:15 -07:00
Daniel Martí
2c0be49124 import: do set root_dir if running from a repo 2015-09-14 17:40:44 -07:00
Daniel Martí
eda0c965e9 Merge branch 'dont-use-sudo-brew' into 'master'
Don't recommend using sudo with Homebrew

Homebrew [doesn't recommend using `sudo`][1], so update the README to reflect that.

 [1]: https://github.com/Homebrew/homebrew/blob/master/share/doc/homebrew/FAQ.md#why-does-homebrew-say-sudo-is-bad-

See merge request !78
2015-09-14 16:40:41 +00:00
Daniel Martí
15e14075cf scanner: avoid checking if usual files are binary 2015-09-13 22:22:03 -07:00
Daniel Martí
055759cf76 Rewrite scanner logic
Initially, the scanner used libmagic which used magic numbers in the file's
content to detect what kind of file it appears to be. Since that library isn't
available on all systems, we added support for two other libraries, mimetypes
amongst them.

The issue with mimetypes is that it only uses the file's extension, not its
actual content. So this ends in variable behaviour depending on what system
you're using fdroidserver on. For example, an executable binary without
extension would be ignored if mimetypes was being used.

We now drop all libraries - mimetypes too as it depends on the system's
mime.types file - and instead check extensions ourselves. On top of that, do
a simple binary content check to find binary executables that don't have an
extension.

The new in-house code without any dependencies doesn't add any new checks, so
no builds should break. The current checks still work:

 % fdroid scanner app.openconnect:1029
[...]
Found executable binary at assets/raw/armeabi/curl
Found executable binary at assets/raw/mips/curl
Found executable binary at assets/raw/x86/curl
Found JAR file at lib/XposedBridgeApi-54.jar
Found JAR file at libs/acra-4.5.0.jar
Found JAR file at libs/openconnect-wrapper.jar
Found JAR file at libs/stoken-wrapper.jar
Found shared library at libs/armeabi/libopenconnect.so
Found shared library at libs/armeabi/libstoken.so
Found shared library at libs/mips/libopenconnect.so
Found shared library at libs/mips/libstoken.so
Found shared library at libs/x86/libopenconnect.so
Found shared library at libs/x86/libstoken.so
2015-09-13 22:17:37 -07:00
Daniel Martí
1a3816acfb scanner: support apps using srclib repo type 2015-09-13 22:05:41 -07:00
Alex Jordan
e62a9c5cbb Don't recommend using sudo with Homebrew 2015-09-13 14:53:34 -07:00
Daniel Martí
576da1d048 all: deduplicate -v/-q setup 2015-09-11 23:42:50 -07:00
Daniel Martí
b42b45f762 lint: small cleanup 2015-09-11 23:35:32 -07:00
Daniel Martí
ecad89dc05 lint: catch more duplicate links 2015-09-11 23:30:59 -07:00
Daniel Martí
3638acddc4 checkupdates: Avoid variable/string names as CV 2015-09-11 23:17:46 -07:00
Daniel Martí
7a79b000bf Allow slashes in tags
The wikipedia app uses them, for example.
2015-09-11 23:17:36 -07:00
Daniel Martí
fb23b2cff4 metadata: don't break words nor on hyphens 2015-09-11 22:52:50 -07:00
Daniel Martí
df9b044a3b lint: add dumb support for multiline links 2015-09-11 22:40:16 -07:00
Daniel Martí
b9b4880b49 lint: move regex closer to usage 2015-09-11 22:07:51 -07:00