Hans-Christoph Steiner
183ce9541b
server: support btlog on offline signing setups
...
This is necessary to shuttle the binary transparency log from the offline
machine to the thumb drive, then from the thumb drive to the online box.
2017-04-12 00:45:22 +02:00
Hans-Christoph Steiner
f75b547795
server: remove unused --sync-from-local-copy-dir
...
This option was not hooked up at all, and does not make sense as a command
line argument. It should just be a config.py item. In that case, the
presence of config.py marks the current dir as a repo, so there is no
longer a need to test for a dir called repo/ as a safety. This makes the
setup easier, since sync_from_localcopy() now creates repo/ for the user.
2017-04-12 00:45:22 +02:00
Hans-Christoph Steiner
0c19d4566d
Merge branch 'elimitate-password-files' into 'master'
...
Eliminate the need for password files
See merge request !249
2017-04-11 21:37:46 +00:00
Torsten Grote
4d25113fa0
Eliminate the need for password files
...
The passwords are now passed as private environment variables to the
processes that need them.
2017-04-11 16:34:49 -03:00
Torsten Grote
cb942168d5
Merge branch 'nail-down-btlog-and-bugfix' into 'master'
...
Nail down btlog and key bugfix
See merge request !248
2017-04-10 21:21:52 +00:00
Hans-Christoph Steiner
299ba86948
server: git push for binary transparency in fdroid server update
...
Since `fdroid server update` is the place where all uploads to servers
happens, it makes sense to also handle the git push for the binary
transparency log here instead of `fdroid btlog`
2017-04-10 23:02:12 +02:00
Hans-Christoph Steiner
e3152e9ef0
simplify Android Observatory config
...
This just keeps the config key name simple and direct.
2017-04-10 23:02:12 +02:00
Hans-Christoph Steiner
f3b55ab0f0
fix mistake: lists use .append() not .add()
...
This was a mistake in 5ad3486741
2017-04-10 23:02:12 +02:00
Hans-Christoph Steiner
5020f70997
btlog: fix bug with first run, the etag variable needs to exist
2017-04-10 20:30:13 +02:00
Torsten Grote
4e4c6d9acf
Merge branch 'morebintrans' into 'master'
...
`fdroid btlog` for anyone to setup a binary transparency log for any repo
See merge request !243
2017-04-05 12:45:40 +00:00
Torsten Grote
255addc53c
Merge branch 'miscfixes' into 'master'
...
misc fixes
Closes #268
See merge request !245
2017-04-04 21:28:08 +00:00
Hans-Christoph Steiner
136d58dbae
btlog: test with fdroid update without requiring ssh access
...
Yay git! Just use a filesystem git remote instead of ssh.
2017-04-03 23:17:03 +02:00
Hans-Christoph Steiner
4b7084f779
btlog: if git remote is specified, push commits to that remote
...
This makes fdroid automatically push the new binary transparency commits if
there is a git_remote specified in either config.py or from a CLI arg.
2017-04-03 20:31:42 +02:00
Hans-Christoph Steiner
53b69f507e
allow index.TestCase to work when run using any path
2017-04-03 20:24:21 +02:00
Hans-Christoph Steiner
40290fc5e0
move make_binary_transparency_log to btlog
...
This keeps the code more organized, and reduces the number of things that
are loaded for every command via common.
2017-04-03 20:24:21 +02:00
Hans-Christoph Steiner
896c6496b4
remove setting git name/email for binary transparency logs
...
The machine running the steps should include the git config, so that it is
specific to that machines.
2017-04-03 20:24:21 +02:00
Hans-Christoph Steiner
6f71465ec1
support pretty output in JSON and for binary transparency logs
...
This makes make_index_v1() support `fdroid update --pretty`, then also uses
pretty output for the binary transparency logs, so that the git history has
nice, readable diffs between commits.
2017-04-03 20:24:21 +02:00
Hans-Christoph Steiner
c591a4cd89
fdroid btlog external binary transparency logger
...
This complements the binary transparency logging that happens on the
server side !226 . Anyone can set up an efficient tracker of any
F-Droid repo which stores all index files that it sees. It uses HEAD
requests and ETag checking to be as efficient as possible, so that
this can be automatically run at a frequent pace.
2017-04-03 20:24:21 +02:00
Hans-Christoph Steiner
eadcd13723
move make_binary_transparency_log to common for easy reuse
2017-04-03 20:24:21 +02:00
Hans-Christoph Steiner
e58ad330f4
encode filenames as bytes to handle all locale setups
...
This was failing on environments that did not have any LANG or LC_* locale
variables set. This is a valid setup, and is common in headless setups, so
it needs to be handled.
This also adds a new pass of the test suite without the locale env vars set
so that this situation is also tests on gitlab-ci, not only gpjenkins.
The error this caused was:
UnicodeEncodeError: 'ascii' codec can't encode characters in position 6-18: ordinal not in range(128)
2017-04-03 20:24:00 +02:00
Hans-Christoph Steiner
4d50ab9bad
ignore cache on key name change, instead of crashing
...
In 6c2cf2ccdd
2017-04-03 20:20:25 +02:00
Hans-Christoph Steiner
5ad3486741
make sure indexes are sorted to minimize diffs between copies
...
With the binary transparency log stored in git, it makes that more readable
and as small as possible if the index files are fully sorted. That will
reduce the differences between two copies of an index file to a bare
minimum.
If in the future we implement some kind of binary diff transfer, this will
also help there.
2017-04-03 20:18:54 +02:00
Hans-Christoph Steiner
7d4176dd50
metadata: don't crash if metadata includes blank Categories list
...
If someone includes just 'Categories:', it shouldn't crash.
2017-04-03 20:18:54 +02:00
Hans-Christoph Steiner
baeae862ad
buildserver: allow gradle to install new components
...
Google is making gradle automatically download Android SDK components that
are needed by the build, but not already present. We need to support that
since it would be a lot of work to fight it. Plus, since each build starts
from the fresh snapshot, it should not be such a big deal to let each
build install stuff during the process.
closes #268
2017-04-03 20:18:54 +02:00
Hans-Christoph Steiner
d6310ed52c
Merge branch 'invalid-apk-fix' into 'master'
...
Fix bug when scanning invalid APKs and add a test for it
See merge request !246
2017-04-03 17:38:51 +00:00
Torsten Grote
8f6fa2955d
Fix bug when scanning invalid APKs and add a test for it
2017-04-03 13:07:49 -03:00
Hans-Christoph Steiner
2ba757e6c9
Merge branch 'index-parsing' into 'master'
...
Download and return repository index
See merge request !240
2017-04-03 16:04:35 +00:00
Hans-Christoph Steiner
159ca824b7
Merge branch 'local-install' into 'master'
...
Fix installs with pip --user
See merge request !244
2017-04-03 14:45:45 +00:00
Torsten Grote
6552c1fa49
Fix installs with pip --user
2017-04-03 09:55:20 -03:00
Torsten Grote
7c34dd96f4
Reduce code duplication
...
by re-using methods for extracting and verifying certificate
2017-04-03 09:23:06 -03:00
Torsten Grote
a23da47118
Add method for downloading (and verifying) a repository index
...
This includes some test cases to test the new code.
2017-04-03 09:02:07 -03:00
Hans-Christoph Steiner
d8ad1c78c2
Merge branch '265-fdroid-init-generates-double-entries-in-config-py' into 'master'
...
avoid duplicate value assignments in config files
Closes #265
See merge request !241
2017-04-03 10:13:20 +00:00
Michael Pöhn
8b51e40d63
avoid duplicate value assignments when updating config files
2017-04-02 12:08:01 +02:00
Hans-Christoph Steiner
a6b3ffeeea
Merge branch 'move-index-methods' into 'master'
...
Move index methods into dedicated modules
See merge request !239
2017-03-31 17:42:22 +00:00
Torsten Grote
389fd7e6c9
Re-use config and options from common module in index
2017-03-31 11:00:49 -03:00
Torsten Grote
f9321f6032
Move index related methods to new index module
2017-03-29 13:10:04 -03:00
Torsten Grote
9f765ed6f7
Move index signing methods into signindex.py
2017-03-29 12:11:40 -03:00
Hans-Christoph Steiner
256ef003fc
Merge branch 'patch-1' into 'master'
...
Update support repository.
See merge request !238
2017-03-29 14:56:37 +00:00
Jan Christian Grünhage
f637b461f0
Update support repository.
2017-03-29 10:33:38 +00:00
Hans-Christoph Steiner
8f96c9da3d
Merge branch 'smaller-methods' into 'master'
...
Factor out code into smaller methods to be used by repomaker
See merge request !236
2017-03-27 19:59:51 +00:00
Hans-Christoph Steiner
c3492ab153
Merge branch 'notabug' into 'master'
...
import: add notabug.org, auto-detect gradle, add options for license and categories
See merge request !237
2017-03-27 19:55:44 +00:00
Hans-Christoph Steiner
916e8eb108
Merge branch 'build_tools_log' into 'master'
...
post correct build tool versions to wiki
Closes #267
See merge request !235
2017-03-27 19:54:48 +00:00
Hans-Christoph Steiner
7f08fad2c6
Merge branch 'sign-and-verify-update' into 'master'
...
sign and verify update
See merge request !230
2017-03-27 19:48:36 +00:00
Boris Kraut
13b276e02f
import: add option to specify license and categories, auto-detect build.gradle
2017-03-26 19:00:03 +02:00
Boris Kraut
39f7429c9b
import: add notabug.org
2017-03-26 18:05:37 +02:00
Torsten Grote
c9b76eb808
Return public key and fingerprint after generating repo signing key
2017-03-22 15:53:20 -03:00
Torsten Grote
c484dc33be
Factor out code from update.main() to individual functions
2017-03-22 15:17:04 -03:00
Michael Pöhn
843595eb33
moved version android tools version log header to corresponding function
2017-03-22 14:56:26 +01:00
Michael Pöhn
0fda23ef80
fix local var name overloading when iterating build flavours
2017-03-22 14:20:54 +01:00
Michael Pöhn
c15f02ed68
fix exception handling in dscanner and update.py
2017-03-22 14:07:54 +01:00
