1
0
mirror of https://gitlab.com/fdroid/fdroidserver.git synced 2024-07-02 07:20:37 +02:00
Commit Graph

4976 Commits

Author SHA1 Message Date
Hans-Christoph Steiner
5d56841f8a nightly: prompt user to create a debug.keystore if its not there
#423
2017-12-07 22:39:32 +01:00
Hans-Christoph Steiner
c823d4c4a8 nightly: fix QR icon.png generation 2017-12-07 22:39:32 +01:00
Hans-Christoph Steiner
5d54446efc update: do not replace Name/Summary from template unless blank
`fdroid nightly` needs this change so it can set the Summary using the
template.
2017-12-07 22:39:32 +01:00
Hans-Christoph Steiner
4afe5aefd5 nightly: create app metadata using template of parsed data 2017-12-07 22:39:32 +01:00
Marcus
87018d45e2 Merge branch 'remove-fd-commit' into 'master'
remove fd-commit, no active devs use it, and requires Auto Name/Name

See merge request fdroid/fdroidserver!392
2017-12-07 20:31:09 +00:00
Hans-Christoph Steiner
adc0c23db4 Merge branch 'help_message_fix' into 'master'
metadata: make help for common -W option clearer

See merge request fdroid/fdroidserver!350
2017-12-07 20:03:17 +00:00
Marcus Hoffmann
0f18005104 metadata: make help for metadata -W option clearer
specify possible choices and provide clearer help text
2017-12-07 20:28:27 +01:00
Hans-Christoph Steiner
6228162cbd handle jarsigner/apksigner output cleanly for rational logging
These were both spamming the output with lots of confusing messages, even
when --verbose was not used.  Jarsigner especially has confusing messages,
since it has warnings that do not pertain to APK signatures at all, like
the ones about timestamps and missing Certificate Authority.

closes #405
2017-12-07 17:32:14 +01:00
Hans-Christoph Steiner
7f4d84684e Merge branch 'syntaxfix' into 'master'
correct "usage" output (--help; see #405)

See merge request fdroid/fdroidserver!403
2017-12-07 14:17:25 +00:00
Izzy
0a1fe3dc7e correct "usage" output (--help; see #405) 2017-12-07 14:51:27 +01:00
Hans-Christoph Steiner
0e37a18d83 makebuildserver: fix join() syntax error, it needs a list/tuple
If only there was a way to test this without taking hours to run...

fixes 964ef996a0
2017-12-07 11:52:52 +01:00
relan
8ffed8750c Merge branch 'update-platform-27_r01' into 'master'
makebuildserver: update SHA-256 for platform-27_r01.zip

See merge request fdroid/fdroidserver!401
2017-12-07 06:05:46 +00:00
Hans-Christoph Steiner
6902160e89 remove fd-commit, no active devs use it, and requires Auto Name/Name
fd-commit and checkupdates both require that there are two name fields,
AutoName: and Name:.  This is only used for the commit messages.  Since the
current devs do it manually, we can remove the fd-commit shell script, then
focus on checkupdates when revamping AutoName/Name.

https://botbot.me/freenode/fdroid-dev/msg/82539152
2017-12-06 22:48:08 +01:00
Hans-Christoph Steiner
956660085a makebuildserver: update SHA-256 for platform-27_r01.zip
Someone forgot to call this _r02.zip:
-ro.​build.​version.​incremental=4402310
+ro.​build.​version.​incremental=4458339

https://verification.f-droid.org/build-metadata/platform-27_r01.html

!364
2017-12-06 22:05:09 +01:00
Hans-Christoph Steiner
6fc507da39 Merge branch 'gitlab-ci-testing-xenial-fedora' into 'master'
gitlab CI runs on  Debian/testing,  Ubuntu/xenial,  Fedora

See merge request fdroid/fdroidserver!398
2017-12-06 19:44:58 +00:00
Hans-Christoph Steiner
05abbfbabd gitlab-ci: move sdist test run to new fedora job
A full run of the test suite takes quite a bit of time.  This removes one
of the 3 runs from the main 'tests' job, and puts it into the Fedora job.
That test run is mostly to make sure the setup.py and source tarball are
correctly, so that doesn't affect merge requests very often.

This also tests `pip install --user`, which was not really being tested
before.
2017-12-06 20:20:17 +01:00
Hans-Christoph Steiner
1a77c6af38 init: fix test for aapt when no aapt has been found
Just give a proper error message rather than this stack trace:

Traceback (most recent call last):
  File "/home/hans/code/fdroid/server/fdroid", line 156, in <module>
    main()
  File "/home/hans/code/fdroid/server/fdroid", line 132, in main
    mod.main()
  File "/export/share/code/fdroid/server/fdroidserver/init.py", line 148, in main
    if os.path.isfile(aapt):
  File "/usr/lib/python3.5/genericpath.py", line 30, in isfile
    st = os.stat(path)
2017-12-06 20:20:17 +01:00
Hans-Christoph Steiner
964ef996a0 makebuildserver: make copy_caches_from_host do rsync like fdroid build
This rsync hung because of an SSH unknown key prompt.  Since this is just
the vm host sshing to the vm guest, it is not essential to check the host
keys.
2017-12-06 20:20:17 +01:00
Hans-Christoph Steiner
a0a68c7a13 setup requires Babel aka python3-babel to compile translations
https://forum.f-droid.org/t/f-droid-server-building-error/1670
2017-12-06 20:20:17 +01:00
Hans-Christoph Steiner
dda9c8b774 gitlab-ci: add pip install test on Arch Linux 2017-12-06 20:20:17 +01:00
Hans-Christoph Steiner
1b1475c982 gitlab-ci: add test runs on Ubuntu, Debian/testing, Fedora 2017-12-06 12:30:47 +01:00
Marcus Hoffmann
bfe2c00834 common.testCase: fix find_sdk_tools when aapt is installed in /usr/bin
The testlogic was broken when having both aapt in /usr/bin and also as
part of the android sdk.
2017-12-06 12:30:47 +01:00
Hans-Christoph Steiner
67e6cbe793 hooks/pre-commit: make ruby and dash tests optional
These are only used for checking syntax in buildserver/Vagrantfile.
Not requiring ruby makes doing CI tests on lots of distros easier
and faster. dash is an 'essential' package on Debian derivs, so
those tests will always be run somewhere.
2017-12-06 12:30:47 +01:00
Hans-Christoph Steiner
a2978a5526 common: aapt 24.0.0 (v0.2-2964546) is now required
Without a recent aapt, the <uses-permission-sdk-23> tag will not be found.
2017-12-06 12:30:47 +01:00
Hans-Christoph Steiner
657b64f6ed Merge branch 'lint-fixes' into 'master'
Lint fixes, plus changing the standard link format in descriptions

See merge request fdroid/fdroidserver!397
2017-12-06 09:41:39 +00:00
Hans-Christoph Steiner
8e1c39f791 Merge branch 'rsync_improvements' into 'master'
Rsync improvements

See merge request fdroid/fdroidserver!400
2017-12-06 08:55:19 +00:00
Hans-Christoph Steiner
b31239803a lint: greatly expand the list of link shorteners to ban
Since we are now getting credit for fighting trackers, might as well step
up the fight!

gleaned from these sources:
* https://bit.do/list-of-url-shorteners.php
* https://www.hashtags.org/featured/list-of-url-shorteners/
* http://l-lists.com/en/lists/gvaoif.html
2017-12-06 09:54:25 +01:00
Hans-Christoph Steiner
05616b33a7 lint: enforce HTTPS and shortener ban in descriptions as well 2017-12-06 09:54:25 +01:00
Hans-Christoph Steiner
42a9833536 lint: switch links to plain URLS rather than mediawiki syntax
fdroidclient#1000
2017-12-06 09:54:25 +01:00
Hans-Christoph Steiner
8588b89eff lint: add more VCS HTTPS checks
I manually checked that these work with HTTPS. fdroiddata!2710 should fix
all of these issues.
2017-12-06 09:54:25 +01:00
Marcus Hoffmann
1bfba12124 build: write out full rsync options
Also put target host:dir on one line to make it more readable
2017-12-05 21:52:22 +01:00
Marcus Hoffmann
e12e1b6a5c build: better logging output on rsync failures
Save rsync error output and combine that with the command invocation
into an FDroidException which can be logged to the wiki.

This additionally sets -q for rsync to only print errors.
2017-12-05 21:52:11 +01:00
Torsten Grote
2bb1445cd6 Merge branch 'nightly-fixes' into 'master'
more `fdroid nightly` polishing

See merge request fdroid/fdroidserver!399
2017-12-05 17:42:57 +00:00
Hans-Christoph Steiner
c33a71a945 fix hg pull, was stupid mistake in 7bba20c662
fdroid/fdroidserver!396
2017-12-05 16:55:58 +01:00
Hans-Christoph Steiner
bb643eddcf jenkins-setup-build-environment: delete libvirt images before test run
profitbricks-build7-amd64 was running out of disk space when running this
job...
2017-12-05 12:31:13 +01:00
Hans-Christoph Steiner
4561ea59a6 nightly: use shutil.move() only so all ops work across filesystems
https://gitlab.com/fdroid/fdroidserver/merge_requests/377#note_49998712
2017-12-05 09:13:19 +01:00
Hans-Christoph Steiner
2983c35361 shutil.move() in apk_strip_signature() to work across filesystems
os.rename() only works if source and destination are on the same file
system, shutil.move() works across file systems.

OSError: [Errno 18] Invalid cross-device link: '/builds/eighthave/fdroidclient/app/build/outputs/apk/app-debug.apk' -> '/tmp/tmp966vh75f/tmp.apk'
2017-12-04 22:52:41 +01:00
Hans-Christoph Steiner
bf913703c5 nightly: only use read_config to load final, generated config.py
This needs to use the config loading routine to find Java `keytool`, but
since it doesn't need to fully load the config, isolate that usage in the
function.  Then read_config() is only ever called once, as is it meant to
be used, once the config.py is generated.

Using `from . import common; common.config = foo` will not always work,
due to some oddities to how the `from` imports work. So the full module
has to be imported in order to make sure its always properly set.
2017-12-04 22:52:41 +01:00
Hans-Christoph Steiner
8a61b0b945 nightly: resign APKs with provided debug.keystore
Rather than needing to run a command before and after the build, in order
to first install the debug.keystore, then after to fetch and publish the
APK, this makes `fdroid nightly` just resign the APK with the provided
debug.keystore.  Then `fdroid nightly` can be run as the final step in a CI
build, and still ensure that the APKs are always signed by the provided
debug.keystore.
2017-12-04 22:52:41 +01:00
Hans-Christoph Steiner
1c3a4479ab add common.sign_apk() for nighly as test for using in publish
Since the MD5 migration was quite a bit of work, it makes sense to start
on moving away from SHA1 as much as possible while it is easy to do. SHA256
will only work in APK signatures on android-18 (4.3) or newer.  So if an
APK has a minSdkVersion of 18 or newer, then sign with SHA256.

https://issuetracker.google.com/issues/36956587
https://android-review.googlesource.com/c/platform/libcore/+/44491
2017-12-04 22:52:41 +01:00
Hans-Christoph Steiner
7da0854fa1 Merge branch 'CVE-2017-1000117' into 'master'
block all SSH connections for VCS, for usabililty and security

See merge request fdroid/fdroidserver!396
2017-12-04 18:48:24 +00:00
Hans-Christoph Steiner
7bba20c662 block all SSH connections for VCS, for usabililty and security
If we allow SSH, then we'd have to manage known_hosts.

All VCS and submodule URLs should use HTTPS.  SSH URLs have security vulns:
https://blogs.msdn.microsoft.com/devops/2017/08/15/git-vulnerability-with-submodules/
https://www.theregister.co.uk/2017/08/13/ssh_flaw_in_git_mercurial_svn/
CVE-2017-1000117

I did a manual scan of the setup on jenkins.debian.net to see if I could
find any suspicious URLs.  Looks good so far.  This is what I used:

find . -type f -print0 |xargs -0 grep -Eo 'ssh[:+][svn/]+...................'
find . -type f -print0 |xargs -0 grep -Eo 'ssh://-[^ "]+'

Also, some ssh://_ URLs in submodules might still work, because of the URL
rewriting in fdbfb4d1.  But https://-oProxyCommand=pwnme does not really do
anything, unlike ssh://-oProxyCommand=pwnme
2017-12-04 17:49:59 +01:00
Marcus
5ae14fab18 Merge branch 'submodules_ucm' into 'master'
checkupdates: don't fail when we can't init submodules

Closes #231

See merge request fdroid/fdroidserver!395
2017-12-04 16:29:11 +00:00
Marcus Hoffmann
db0a97e8e7 checkupdates: don't fail when we can't init submodules
Later revisions might have removed the submodules so we want to keep
going when there are no submodules present.
We still abort when there is an error initializing submodules.

Fixes fdroid/fdroidserver#231
2017-12-04 16:30:37 +01:00
Michael Pöhn
2dcb19d392 Merge branch 'git_clone_fix' into 'master'
GitFetchFDroidPopen: don't change cwd when cloning

See merge request fdroid/fdroidserver!393
2017-12-03 16:49:42 +00:00
Marcus Hoffmann
30b3f41a75 GitFetchFDroidPopen: don't change cwd per default
Fix for ca24aa4ca8.
For git clone we don't want to change cwd because clone actually
creates the repo dir.
2017-12-03 17:07:36 +01:00
Hans-Christoph Steiner
ca24aa4ca8 stop git clone from hanging at prompts
Forgot this in fdbfb4d1a2 !378

reviewed in person with @bubu @uniqx
2017-12-03 13:13:07 +01:00
relan
24e1da1e91 Merge branch 'log_git' into 'master'
build: log vcs tools version on every build attempt

See merge request fdroid/fdroidserver!391
2017-12-03 06:03:03 +00:00
Marcus Hoffmann
bb591e236d build: log vcs tools version on every build attempt 2017-12-02 22:41:08 +01:00
Hans-Christoph Steiner
3f4e93bf86 Merge branch 'metadataFlavours' into 'master'
Add flavour to metadata

See merge request fdroid/fdroidserver!370
2017-12-02 20:51:24 +00:00