Hans-Christoph Steiner 2b6d692f06 use UTF8 as default instead of ASCII for .java .gradle pom.xml ... .java .gradle and XML files all can use any encoding. Most code is ASCII, but authors' names, etc. can easily be non-ASCII. UTF-8 is by far the most common file encoding. While UTF-8 is the default encoding inside the code in Python 3, it still has to deal with the real world, so the encoding needs to be explicitly set when reading and writing files. So this switches fdroidserver to expect UTF-8 instead of ASCII when parsing these files. For now, this commit means that we only support UTF-8 encoded *.java, pom.xml or *.gradle files. Ideally, the code would detect the encoding and use the actual one, but that's a lot more work, and its something that will not happen often. We can cross that bridge when we come to it. One approach, which is taken in the commit when possible, is to keep the data as `bytes`, in which case the encoding doesn't matter. This also fixes this crash when parsing gradle and maven files with non-ASCII chars: ERROR: test_adapt_gradle (__main__.BuildTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/var/lib/jenkins/workspace/fdroidserver-eighthave/tests/build.TestCase", line 59, in test_adapt_gradle fdroidserver.build.adapt_gradle(testsdir) File "/var/lib/jenkins/workspace/fdroidserver-eighthave/fdroidserver/build.py", line 445, in adapt_gradle path) File "/var/lib/jenkins/workspace/fdroidserver-eighthave/fdroidserver/common.py", line 188, in regsub_file text = f.read() File "/usr/lib/python3.4/encodings/ascii.py", line 26, in decode return codecs.ascii_decode(input, self.errors)[0] UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 9460: ordinal not in range(128)		2016-06-07 20:13:54 +02:00
..
asynchronousfilereader	Add asynchronous filereader, fix python3 lockups	2015-09-20 20:57:53 +02:00
__init__.py	Make the server tools an installable package (with distutils) - wip	2012-03-11 11:59:19 +00:00
build.py	Fix error in build.py	2016-03-27 21:36:16 +02:00
checkupdates.py	rename metadata.write_metadata() to match metadata.parse_*_metadata()	2016-03-23 17:16:28 +01:00
common.py	use UTF8 as default instead of ASCII for .java .gradle pom.xml	2016-06-07 20:13:54 +02:00
gpgsign.py	Port all imports to python3	2016-03-10 16:43:37 +00:00
import.py	standardize on .yml as the file extension for YAML	2016-03-23 17:16:28 +01:00
init.py	read/write F-Droid files always as UTF-8	2016-06-07 14:10:04 +02:00
install.py	Port all imports to python3	2016-03-10 16:43:37 +00:00
lint.py	lint: Add check for missing and unused extlibs	2016-05-05 13:42:07 +01:00
metadata.py	use UTF8 as default instead of ASCII for .java .gradle pom.xml	2016-06-07 20:13:54 +02:00
net.py	Fix python headers in some source files	2016-03-11 13:27:00 +00:00
publish.py	publish: fix hashing of strings	2016-03-10 16:43:37 +00:00
readmeta.py	Port all imports to python3	2016-03-10 16:43:37 +00:00
rewritemeta.py	read/write F-Droid files always as UTF-8	2016-06-07 14:10:04 +02:00
scanner.py	use UTF8 as default instead of ASCII for .java .gradle pom.xml	2016-06-07 20:13:54 +02:00
server.py	Port all imports to python3	2016-03-10 16:43:37 +00:00
signindex.py	Port all imports to python3	2016-03-10 16:43:37 +00:00
stats.py	Replace itervalues() with values()	2016-03-10 16:43:37 +00:00
update.py	read/write F-Droid files always as UTF-8	2016-06-07 14:10:04 +02:00
verify.py	Port all imports to python3	2016-03-10 16:43:37 +00:00