kamp/searxng
1
0
Fork 0
mirror of https://github.com/searxng/searxng.git synced 2024-08-22 22:20:42 +02:00
Code
4,773 Commits 6 Branches 17 Tags 149 MiB
7dc370efe9
Commit Graph

3055 Commits

Author SHA1 Message Date
Alexandre Flament
7dc370efe9 [enh] new issue: include information from searx.version 2021-08-31 11:45:33 +02:00
Markus Heiser
1e942435be
Merge pull request #273 from searxng/dependabot/pip/master/pylint-2.10.2 
Bump pylint from 2.9.6 to 2.10.2
 2021-08-31 09:08:59 +00:00
Markus Heiser
03e7d423be [pylint] Pylint 2.10 - unused-variable 
Pylint 2.10 fixed [1]:

  Fixed bug with cell-var-from-loop checker: it no longer has false negatives
  when both unused-variable and used-before-assignment are disabled.

[1] https://pylint.pycqa.org/en/latest/whatsnew/2.10.html

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2021-08-31 10:51:50 +02:00
Markus Heiser
e856b563dd [pylint] Pylint 2.10 - unspecified-encoding 
Pylint 2.10 added new default checks [1]:

unspecified-encoding:
  Emitted when open() is called without specifying an encoding [2]

[1] https://pylint.pycqa.org/en/latest/whatsnew/2.10.html
[2] https://github.com/PyCQA/pylint/issues/3826

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2021-08-31 10:49:36 +02:00
Markus Heiser
b83c14cf6b [pylint] Pylint 2.10 - fix use-list-literal & use-dict-literal 
Pylint 2.10 added new default checks [1]:

use-list-literal
  Emitted when list() is called with no arguments instead of using []

use-dict-literal
  Emitted when dict() is called with no arguments instead of using {}

[1] https://pylint.pycqa.org/en/latest/whatsnew/2.10.html

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2021-08-31 10:40:29 +02:00
searxng-bot
9e63e20c36 [translations] update 
18fae620 - 2021-08-25 - Markus Heiser <markus.heiser@darmarit.de>
2e76843c - 2021-08-25 - ewm <gnu.ewm@protonmail.com>
172a1a88 - 2021-08-25 - Alexandre Flament <alex@al-f.net>
4567065b - 2021-08-25 - Full Name <cheez.1337@mail.ru>
59d9d549 - 2021-08-25 - Markus Heiser <markus.heiser@darmarit.de>
5f135b41 - 2021-08-23 - Markus Heiser <markus.heiser@darmarit.de>
91b3a32e - 2021-08-24 - Full Name <cheez.1337@mail.ru>
0198ea11 - 2021-08-22 - Allan Nordhøy <epost@anotheragency.no>
6efc4b9f - 2021-08-22 - Markus Heiser <markus.heiser@darmarit.de>
a263af1e - 2021-08-23 - Alexandre Flament <alex@al-f.net>
713b07f7 - 2021-08-22 - Markus Heiser <markus.heiser@darmarit.de>
0c7b7859 - 2021-08-22 - Markus Heiser <markus.heiser@darmarit.de>
408c85f9 - 2021-08-20 - Alexandre Flament <alex@al-f.net>
 2021-08-27 07:16:35 +00:00
Alexandre Flament
697ebeddcc [mod] searx.utils.dict_subset: rewrite with comprehension 2021-08-24 15:28:08 +02:00
Alexandre Flament
3b0f70ed0f [mod] /image_proxy: use HTTP/1 instead of HTTP/2 
httpx: HTTP/2 is slow when a lot data is downloaded.
https://github.com/dalf/pyhttp-benchmark

also, the usage of HTTP/1 decreases the load average
 2021-08-24 14:51:20 +02:00
Alexandre Flament
43fcaa642a [fix] image_proxy: always close the httpx respone 
previously, when the content type was not an image and some other error,
the httpx response was not closed
 2021-08-24 14:51:20 +02:00
Alexandre Flament
df15c655f7 [mod] /image_proxy: don't decompress images 2021-08-24 14:51:20 +02:00
Markus Heiser
29b40338d6
Merge pull request #260 from dalf/mod_locale 
[mod] searx.webapp.get_locale: read locale from the preferences
 2021-08-22 16:07:21 +00:00
Noémi Ványi
3d5e6e0abb [enh] google: add filter=0 to Google engine for more results 
backport from searx ( 23b3b56a06ef831af0a1b30a12c26ebd50e329bb )
 2021-08-21 17:46:16 +02:00
searxng-bot
eafa373cff [translations] update 
b25c9d58 - 2021-08-17 - Allan Nordhøy <epost@anotheragency.no>
 2021-08-20 07:16:15 +00:00
Markus Heiser
8290f70e24 [fix] /config add missing GIT_BRANCH value 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2021-08-18 19:51:07 +02:00
searxng-bot
02b6f0c370 [translations] update 
fc3041a0 - 2021-08-10 - Allan Nordhøy <epost@anotheragency.no>
fcc3732d - 2021-08-06 - Allan Nordhøy <epost@anotheragency.no>
894bd655 - 2021-08-06 - Allan Nordhøy <epost@anotheragency.no>
 2021-08-17 08:47:59 +00:00
Alexandre Flament
28ccf47ca4 [mod] searx.webapp.get_locale: read locale from the preferences 
pre_request already set the locale.
 2021-08-17 08:18:30 +02:00
Alexandre Flament
a925254c3d
Merge pull request #258 from dalf/plugin_minor_change 
[mod] plugins: minor change
 2021-08-16 17:10:28 +02:00
Alexandre Flament
2f363858b8 [fix] searx.search.checker.get_result() always return a dict 
So checker_results['status'] == 'ok' is enough to check the checker result.
See searx/webapp.py, /preferences endpoint
 2021-08-16 08:29:16 +02:00
Alexandre Flament
3f3b5d6181 [mod] plugins: minor change 
required attributes: display a different message
when the attribute has the wrong type
 2021-08-14 18:03:31 +02:00
Markus Heiser
15bf9fc850 [fix] settings.yml - replace http:// URLs by https:// 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2021-08-07 16:43:19 +02:00
Alexandre Flament
0d20e5dfe3 [mod] searx/locales.py: language names based on Unicode CLDR 
rename "oc" to "Occitan":
* https://github.com/unicode-org/cldr/blob/35.1/seed/main/oc.xml#L115
* https://oc.wikipedia.org/wiki/Occitan

see https://github.com/searxng/searxng/pull/247#issuecomment-892382001
 2021-08-04 09:50:34 +02:00
Markus Heiser
809bf1a105 [mod] pylint & document searx.locales (settings.yml: remove locales) 
- Add ``# lint: pylint`` header to pylint this python file.
- Fix issues reported by pylint.
- Add source code documentation of modul searx.locales

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2021-08-04 07:54:53 +02:00
Alexandre Flament
f30d01ffab [mod] settings.yml: remove locales 
There are detected from the searx/translations directory
 2021-08-03 15:44:45 +02:00
Alexandre Flament
bd17544f82 [fix] make test.robot: firefox profile: intl.accept_languages=en 
make sure Firefox in test.robot asks for English pages
 2021-08-03 15:11:51 +02:00
Alexandre Flament
7fa7177759 Update translations (pot, po) 2021-08-02 09:44:13 +02:00
Markus Heiser
e0ca749ff5
Merge pull request #242 from searxng/update_data_update_currencies.py 
Update searx.data - update_currencies.py
 2021-08-02 07:40:19 +00:00
Alexandre Flament
b6711ba0ad
Merge pull request #241 from searxng/update_data_update_languages.py 
Update searx.data - update_languages.py
 2021-08-02 09:01:57 +02:00
Markus Heiser
274edd523c [fix] get common used languages back (min_engines_per_lang = 13) 
In commit 4d3f2f48d common used languages has been droped.  By reducing the
number of `min_engines_per_lang` from 15 to 13 we get theses languages back.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2021-08-01 13:04:24 +02:00
Markus Heiser
0fc1dcbfa5
Merge pull request #243 from searxng/update_data_update_wikidata_units.py 
Update searx.data - update_wikidata_units.py
 2021-08-01 09:42:01 +00:00
dalf
7fae5647d3 Update searx.data - update_wikidata_units.py 2021-08-01 06:24:23 +00:00
dalf
2963c83daa Update searx.data - update_currencies.py 2021-08-01 06:24:13 +00:00
dalf
4d3f2f48d2 Update searx.data - update_languages.py 2021-08-01 06:24:12 +00:00
dalf
f5389fd722 Update searx.data - update_firefox_version.py 2021-08-01 06:24:01 +00:00
Alexandre Flament
38ee88b390
Merge pull request #229 from dalf/version-from-git 
version based on git
 2021-07-30 18:57:00 +02:00
Alexandre Flament
4b43775c91 version based on the git repository 
This commit remove the need to update the brand for GIT_URL and GIT_BRANCH:
there are read from the git repository.

It is possible to call python -m searx.version freeze to freeze the current version.
Useful when the code is installed outside git (distro package, docker, etc...)
 2021-07-30 14:40:09 +02:00
Alexandre Flament
feef577f99
Merge pull request #230 from dalf/remove-zh-hant-tw 
Remove zh_Hant_TW translation
 2021-07-28 08:34:09 +02:00
Alexandre Flament
0b2dca088f Remove zh_Hant_TW translation 
See https://github.com/searxng/searxng/issues/195#issuecomment-887518808
 2021-07-27 18:54:25 +02:00
Alexandre Flament
ed0db4d61a
Merge pull request #226 from return42/hardening 
[mod] hardening SearXNG instances by default (formats)
 2021-07-27 11:22:18 +02:00
Alexandre Flament
24fcd7475a Revert "[brand] !!DO NOT MERGE!! this patch uses branch fix-searx.sh @return42" 
This reverts commit e946c3462c.
 2021-07-27 08:40:53 +02:00
Markus Heiser
beff764d74
Merge pull request #216 from return42/fix-searx.sh 
normalize environment of installation tasks (shell) with YAML settings
 2021-07-27 05:32:57 +00:00
Markus Heiser
80f5241387
Merge pull request #225 from return42/docs-nosql 
[docs] document NoSLQ engines MongoDB and redis
 2021-07-27 05:32:26 +00:00
Samuel Dudik
7a7ef9cea6 [fix] Seznam engine - some XPath selectors has been changed 
Merged from https://github.com/dudik/searx/commit/5a4207759

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2021-07-27 07:13:41 +02:00
Markus Heiser
7b40c736e6 [mod] hardening SearXNG instances by default (formats) 
Deny formats has been implemented in 6ed4616d.

To harden SearXNG instances by default, other formats than HTML should be
denied.  Most of JSON, RSS and CSV requests are bots [1]::

    Bots are the only users of this feature on a public instance, and they abuse
    it too much that the engines rate limit pretty quickly the IP address of the
    instance.

[1] https://github.com/searxng/searxng/issues/95

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2021-07-25 12:59:09 +02:00
Noémi Ványi
5e648e77ab [docs] document NoSLQ engines MongoDB and redis 
Slightly modified merge of commit [3b192e638]

[3b192e638] https://github.com/searx/searx/commit/3b192e638

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2021-07-25 11:39:11 +02:00
Alexandre Flament
48fe83b901
Merge pull request #221 from dalf/fix-peertube_fetch_supported_languages 
[fix] peertube: update _fetch_supported_languages
 2021-07-25 10:30:53 +02:00
Markus Heiser
e946c3462c [brand] !!DO NOT MERGE!! this patch uses branch fix-searx.sh @return42 
Do not merge this patch in master branch of SearXNG!  This branch exists only
for testing the feature branch fix-searx.sh @return42.

This patch changes the buildenv to::

    GIT_URL='https://github.com/return42/searxng'
    GIT_BRANCH='fix-searx.sh'
    SEARX_PORT='7777'
    SEARX_BIND_ADDRESS='127.0.0.12'

To test installation procedure, clone feature branch (fix-searx.sh)::

    $ cd ~/Downloads
    $ git clone --branch fix-searx.sh https://github.com/return42/searxng searxng
    $ cd searxng

    $ ./utils/searx.sh install all
    ...

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2021-07-24 16:28:15 +02:00
Markus Heiser
b6a55e223c [mod] reduce enviroment variables in shell scripts to what is needed 
Not all settings from the 'brand:' section of the YAML files are needed in the
shell scripts.  This patch reduce the variables in ./utils/brand.env to what is
needed.  The following ('brand:' settings) can be removed from this file:

- ISSUE_URL
- DOCS_URL
- PUBLIC_INSTANCES
- WIKI_URL

Tasks running outside of an *installed instance*, need the following settings
from the YAML configuration:

- GIT_URL            <--> brand.git_url
- GIT_BRANCH         <--> brand.git_branch
- SEARX_URL          <--> server.base_url  (aka PUBLIC_URL)
- SEARX_PORT         <--> server.port
- SEARX_BIND_ADDRESS <--> server.bind_address

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2021-07-24 16:28:15 +02:00
Markus Heiser
fe67f1478f [fix] qwant engine - prevent API locale exception on lang 'all' 
Has been reported in [1], error message::

    Error
        Error: searx.exceptions.SearxEngineAPIException
        Percentage: 0
        Parameters: ('API error::locale must be a string,locale must be one of
        the following values: en_gb, en_ie, en_us, en_ca, en_in, en_my, en_au,
        en_nz, cy_gb, gd_gb, de_de, de_ch, de_at, fr_fr, br_fr, fr_be, fr_ch,
        fr_ca, fr_ad, fc_ca, ec_ca, co_fr, es_es, es_ar, es_cl, es_co, es_mx,
        es_pe, es_ad, ca_es, ca_ad, ca_fr, eu_es, eu_fr, it_it, it_ch, pt_br,
        pt_pt, pt_ad, nl_be, nl_nl, pl_pl, zh_hk, zh_cn, fi_fi, bg_bg, et_ee,
        hu_hu, da_dk, nb_no, sv_se, ko_kr, th_th, cs_cz, ro_ro, el_gr',)
        File name: searx/engines/qwant.py:114
        Function: response
        Code: raise SearxEngineAPIException('API error::' + msg)

[1] https://github.com/searxng/searxng/issues/222

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2021-07-24 14:48:27 +02:00
Markus Heiser
ca57c7421b [fix] qwant engine - prevent exception on date/time value is None 
Has been reported in [1], error messages::

  Error
       Error: ValueError
       Percentage: 0
       Parameters: ()
       File name: searx/engines/qwant.py:159
       Function: response
       Code: pub_date = datetime.fromtimestamp(item['date'], None)

    Error
        Error: TypeError
        Percentage: 0
        Parameters: ('an integer is required (got type NoneType)',)
        File name: searx/engines/qwant.py:196
        Function: response
       Code: pub_date = datetime.fromtimestamp(item['date'])

Fix timedelta from seconds to milliseconds [1], error message::

    Error
        Error: TypeError
        Percentage: 0
        Parameters: ('unsupported type for timedelta seconds component: NoneType',)
        File name: searx/engines/qwant.py:195
        Function: response
        Code: length = timedelta(seconds=item['duration'])

[1] https://github.com/searxng/searxng/issues/222

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2021-07-24 14:48:14 +02:00
Alexandre Flament
b0a12924a0 [fix] peertube: update _fetch_supported_languages 
update the regex to match the changes in peertube source code
fix "make data.languages"
 2021-07-23 12:03:16 +02:00