1
0
mirror of https://gitlab.com/fdroid/fdroidserver.git synced 2024-11-18 04:30:12 +01:00
Commit Graph

4918 Commits

Author SHA1 Message Date
Daniel Martí
88d369e293 Merge branch 'makebuildserver-caching-fixes' into 'master'
./makebuildserver CI fixes

These are fixes for caching and flexible config related to the work of getting `./makebuildserver` running on Debian's and Guardian Project's Jenkins boxes.  @mvdan and I have been working with h01lger to get that going, here's the latest build log: https://jenkins.debian.net/view/reproducible/job/reproducible_setup_fdroid_build_environment_profitbricks3/lastBuild/console

See merge request !91
2016-01-26 11:00:58 +00:00
Hans-Christoph Steiner
980339d2f5 make all apt repos in build server use the configured debian mirror 2016-01-26 11:31:40 +01:00
Hans-Christoph Steiner
8d66742dd7 makebuildserver: set debian mirror in config file only
Setting it in the config file was not working, and right now, all of the
options are in the config file and not as command line flags, so remove
--debian-mirror to keep that consistent.
2016-01-26 11:31:40 +01:00
Hans-Christoph Steiner
82e26cf59e makebuildserver: use jessie32.box from cache, if present 2016-01-26 11:31:40 +01:00
Daniel Martí
1ff4f63de3 scanner: Allow clojars.org maven repository
See https://github.com/inorichi/tachiyomi/issues/46.
2016-01-24 18:13:45 +00:00
Boris Kraut
c773fdf633 wp-fdroid: Linkify more licenses 2016-01-24 13:27:15 +01:00
Hans-Christoph Steiner
16c220517d Merge branch 'master' into 'master'
Reproducibly build documentation

The two commits fix reproducibility problems detected by the Debian CI:

https://reproducible.debian.net/rb-pkg/unstable/amd64/fdroidserver.html

See merge request !84
2016-01-22 10:27:54 +00:00
Daniel Martí
0003e4c742 Merge branch 'makebuildserver-for-debian-jenkins' into 'master'
Makebuildserver for debian jenkins

@mvdan and I worked out a plan with Holger, the Debian Developer running the reproducible builds jenkins build farm.  The idea is for that to just run `./makebuildserver` in the jenkins script.  Then we'll make sure that it works.  This also makes it easy for anyone to set up and run an FDroid build instance, which is important for things like the verification server and for people to be able to run reproducible builds on their own machine.

This does change some of the defaults, but they can all be overridden in the `makebs.config.py` for f-droid.org.

More comments in the commits.


See merge request !89
2016-01-21 15:49:37 +00:00
Hans-Christoph Steiner
24a9f3b816 makebuildserver: delete cached files with zero size
If download fails to connect, it'll make a zero size file.  Then the next
run on ./makebuildserver will fail because the checksum will not be right.
2016-01-21 15:41:26 +01:00
Hans-Christoph Steiner
cda8fa00f5 makebuildserver: cache defaults to ~/.cache/fdroidserver
This also provides a config option to override that default.  ~/.cache is
a standard location on GNU/Linux machines for cached content.  It is also
good to have the cache outside of the git repo in case `git clean -fdx` is
run, which would delete all files in the directory that are not part of the
git repo, including buildserver/cache/
2016-01-21 15:34:55 +01:00
Hans-Christoph Steiner
3bd8ef2c8c makebuildserver: set defaults in script so it can run without config
This makes it so that ./makebuildserver will run without any config file,
using the defaults that are embedded in the script itself.  This is like
how `fdroid` works.
2016-01-21 15:34:55 +01:00
Hans-Christoph Steiner
50713c7ce9 makebuildserver: set default Debian mirror to http.debian.net
This host automatically detects which is the closest mirror, then uses that
one.  It does so dynamically, so it'll work on machines that move too. Now
that we are pushing more people to run F-Droid build servers, the defaults
should take those use cases into account.
2016-01-21 15:34:55 +01:00
Hans-Christoph Steiner
b33add9ce7 name makebuildserver config the same as the script
This keeps the numbers of names down to a minimum, and since the config
is placed right next to the script, this keeps tab completion working
nicely when the config file is in place.

The old file name is still supported.
2016-01-21 15:34:55 +01:00
Hans-Christoph Steiner
25caa09420 run-tests: verbose output for fdroid update to aid debugging
I'm stumped by this stacktrace, hopefully debug logging will help:

Unknown exception found!
Traceback (most recent call last):
  File "./fdroidserver-0.4.0/fdroid", line 141, in <module>
    main()
  File "./fdroidserver-0.4.0/fdroid", line 119, in main
    mod.main()
  File "<http://localhost:18080/job/fdroidserver/ws/.testfiles/tmp.9hr9NiSmZs/fdroidserver-0.4.0/fdroidserver/update.py",> line 1146, in main
    apks, cachechanged = scan_apks(apps, apkcache, repodirs[0], knownapks)
  File "<http://localhost:18080/job/fdroidserver/ws/.testfiles/tmp.9hr9NiSmZs/fdroidserver-0.4.0/fdroidserver/update.py",> line 543, in scan_apks
    thisinfo['sig'] = getsig(os.path.join(os.getcwd(), apkfile))
  File "<http://localhost:18080/job/fdroidserver/ws/.testfiles/tmp.9hr9NiSmZs/fdroidserver-0.4.0/fdroidserver/update.py",> line 361, in getsig
    with zipfile.ZipFile(apkpath, 'r') as apk:
  File "/usr/lib/python2.7/zipfile.py", line 770, in __init__
    self._RealGetContents()
  File "/usr/lib/python2.7/zipfile.py", line 857, in _RealGetContents
    x._decodeExtra()
  File "/usr/lib/python2.7/zipfile.py", line 388, in _decodeExtra
    tp, ln = unpack('<HH', extra[:4])
struct.error: unpack requires a string argument of length 4
2016-01-21 12:26:50 +01:00
Daniel Martí
0be343273d Apparently Python cannot handle Unicode
I really hope I can revert this in the near future. Having to mutilate
my name just so that pip will work is a terrible workaround.

For better or worse, this only affects scripts defined in setup.py.
2016-01-14 23:15:05 +01:00
Daniel Martí
fff658ba19 stats: use a stable sort that doesn't randomize
Since we only sorted by count, ignoring the string, it meant that items
with the same count might be arranged in different manners. Hence the
`stats` behaviour was not predictable at all. Now it sorts first by
count, then by string.
2016-01-11 02:43:06 +01:00
Daniel Martí
68e2b46cb0 build: don't add extra clean tasks
It is a good idea in theory, especially for extra tasks specified by the
user. But in practice, tasks specified by developers often don't have a
clean counterpart. So this just breaks the build.
2016-01-11 02:05:43 +01:00
Daniel Martí
0cf2539c89 metadata: fix text wrapping on unicode characters
We were passing the utf-8 encoded string to textwrap, which took the
bytes as characters. Hence multi-byte unicode characters (in utf-8)
would count as multiple columns, which is clearly wrong.
2016-01-10 17:54:38 +01:00
Daniel Martí
309660423c fdroid: python2 doesn't like print()
It just prints "()", which is stupid.
2016-01-09 13:24:13 +01:00
Daniel Martí
bd29fce13b Add Gradle 2.10 to the BS 2016-01-05 09:15:54 +01:00
Daniel Martí
e8eda55d9d metadata: Always use regex to verify values
Lists can be made into regexes very easily. Major code simplification.
Also no need to construct sets to avoid iterating over lists.
2016-01-04 13:44:00 +01:00
Daniel Martí
7ff833b948 Remove redundant AntiFeatures bool check 2016-01-04 13:26:48 +01:00
Daniel Martí
2cd3958526 Merge branch 'master' into 'master'
Add Author Name and Author Email fields. (Closes: #90)

Adds the two new fields as discussed in issue #90. Both are optional.

Mind that the Wiki template needs to be extended before merging.

See merge request !87
2016-01-04 08:29:17 +00:00
Dominik George
7f3434ea50
Add Author Name and Author Email fields. (Closes: #90) 2016-01-03 23:00:03 +01:00
Daniel Martí
622195bf16 lint: warn about spaces in summaries too 2015-12-31 11:21:47 +01:00
Daniel Martí
3ac535108a README: Note that Python 2.7.7+ is supported 2015-12-31 10:11:41 +01:00
Daniel Martí
4482ba971d gradle: stop trying to patch outputFile
This often breaks since we're just removing a single line. Instead, use
a much more reliable way of finding output apks.
2015-12-31 09:47:14 +01:00
NeroBurner
1683df0cf8 handle Queue 2015-12-29 20:11:47 +01:00
NeroBurner
0e0767492e Octal numbers 0600 to 0o600 2015-12-29 20:10:04 +01:00
NeroBurner
5ca182a20d except named exception handling 2015-12-29 20:09:27 +01:00
NeroBurner
af38f151a2 Change print-function to have brackets 2015-12-29 20:06:46 +01:00
Daniel Martí
967472408b Add NonFreeAssets anti-feature 2015-12-29 12:39:48 +01:00
Daniel Martí
20ced3f303 Small gradle wrapper fix
It broke on cases like:

	if (gradle.gradleVersion >= "2.2") {
2015-12-24 23:08:31 +01:00
Daniel Martí
5b8d593275 gradle: Completely get rid of gradlepluginver
We can support multiple output dirs without having to detect/infer the
plugin version. Much simpler and more robust too.
2015-12-24 16:41:39 +01:00
Daniel Martí
138618937f gradle: Never add lintOptions workaround
This breaks on newer plugin versions. Instead of complicating this logic
even further to support them, upstream devs should instead use gradle
configs that actually work.

If their build fails because of lint, it's theirs to fix, not ours.
2015-12-24 16:40:36 +01:00
Daniel Martí
adff5615f5 Fix issues spotted by older pyflakes 2015-12-16 16:42:50 +01:00
Daniel Martí
03b6662324 Merge branch 'docs' into 'master'
docs: Add information on binary verification

I added a section about `Binaries:` metadata field. However, my information might be wrong or outdated, since I barely used it by now. Maybe even this field was left out on purpose...

See merge request !83
2015-12-10 11:50:36 +00:00
Daniel Martí
020160f1e8 Merge branch 'summarylength' into 'master'
docs: Adjust summary length



See merge request !85
2015-12-10 11:47:48 +00:00
Boris Kraut
531a165daf docs: Adjust summary length 2015-12-10 12:42:05 +01:00
Daniel Martí
013315bf10 Open metadata files in only one place
Also get rid of the useless get_default_app_info func.
2015-12-07 20:31:23 +01:00
Daniel Martí
c2304b3315 lint: optionally check format 2015-12-07 20:12:26 +01:00
Daniel Martí
fa0dcf5449 lint: fix issue found by older version of pyflakes 2015-12-07 13:58:29 +01:00
Daniel Martí
5541cb83ac checkupdates: fix --auto build copy
Also, remove origlines which hasn't been used in some time.
2015-12-07 12:39:04 +01:00
Daniel Martí
02c59a327b stats: fix a few dict accesses left over 2015-12-07 12:33:24 +01:00
Daniel Martí
ae5b8ed496 import: fix get_default_app_info usage
Closes #129
2015-12-05 19:20:41 +01:00
Daniel Martí
7a7d36feef scanner: remove jni check
buildjni=no is the default. Moreover, some apps now run ndk-build via
gradle, so this was just in the way.
2015-12-04 19:32:01 +01:00
Daniel Martí
3101ad6c39 Set all list defaults to an actual list
Iterating over the defaults will crash otherwise. Update pickle files.
2015-12-04 00:08:42 +01:00
Daniel Martí
cb8d93e443 Fix tests by not checking private attributes
All attrs starting with '_' (e.g. _foo or __bar) are used internally, so
not very relevant for the metadata test anyway.
2015-12-03 19:31:20 +01:00
Daniel Martí
2711f1c868 Also skip unmodified fields in the regex checks
Now it's close to 1s.
2015-12-03 17:02:24 +01:00
Daniel Martí
ea99add3da Keep track of what attrs were modified in apps/builds
This allows us to run checks and fix types only on those that were
really modified. On our fdroiddata repo, with 2k apps with many unset
fields and flags, this reduces readmeta runtime from ~1.3s to ~1.1s.
2015-12-03 16:46:21 +01:00