kamp/searxng
1
0
Fork 0
mirror of https://github.com/searxng/searxng.git synced 2024-11-16 01:20:32 +01:00
Code
8,192 Commits 6 Branches 17 Tags 132 MiB
fa4dfd4efe
Commit Graph

440 Commits

Author SHA1 Message Date
Markus Heiser
d0f465e6fa [mod] bing: add time_range support & upgrade to data_type: traits_v1 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-03-24 10:37:42 +01:00
Markus Heiser
7daf4f95ef [mod] Wikipedia: fetch engine traits (data_type: supported_languages) 
Implements a fetch_traits function for the Wikipedia engines.

.. note::

   Does not include migration of the request methode from 'supported_languages'
   to 'traits' (EngineTraits) object!

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-03-24 10:37:42 +01:00
Markus Heiser
f78f908383 [mod] Google: fetch engine traits (data_type: supported_languages) 
Implements a fetch_traits function for the Google engines.

.. note::

   Does not include migration of the request methode from 'supported_languages'
   to 'traits' (EngineTraits) object!

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-03-24 10:37:42 +01:00
Markus Heiser
dba8977b09 [mod] DuckDuckGo: fetch engine traits (data_type: supported_languages) 
Implements a fetch_traits function for the DuckDuckGo engines.

.. note::

   Does not include migration of the request methode from 'supported_languages'
   to 'traits' (EngineTraits) object!

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-03-24 10:37:42 +01:00
Markus Heiser
ef143729a0 [mod] yahoo: fetch engine traits (data_type: traits_v1) 
Implements a fetch_traits function for the Yahoo engine.

.. note::

   Includes migration of the request methode from 'supported_languages' to
   'traits' (EngineTraits) object!

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-03-24 10:37:42 +01:00
Markus Heiser
c1ae2ef57c [mod] qwant: fetch engine traits (data_type: traits_v1) 
Implements a fetch_traits function for the Qwant engines.

.. note::

   Includes migration of the request methode from 'supported_languages' to
   'traits' (EngineTraits) object!

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-03-24 10:37:42 +01:00
Markus Heiser
fc0c775030 [mod] Dailymotion: fetch engine traits (data_type: supported_languages) 
Implements a fetch_traits function for the Dailymotion engine.

.. note::

   Does not include migration of the request methode from 'supported_languages'
   to 'traits' (EngineTraits) object!

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-03-24 10:37:42 +01:00
Markus Heiser
61383edb27 [mod] Startpage: fetch engine traits (data_type: supported_languages) 
Implements a fetch_traits function for the Startpage engine.

.. note::

   Does not include migration of the request methode from 'supported_languages'
   to 'traits' (EngineTraits) object!

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-03-24 10:37:42 +01:00
Markus Heiser
d3aa690a7a [mod] bing: fetch engine traits (data_type: supported_languages) 
Implements a fetch_traits function for the Bing engines.

.. note::

   Does not include migration of the request methode from 'supported_languages'
   to 'traits' (EngineTraits) object!

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-03-24 10:37:42 +01:00
Markus Heiser
a7fe22770a [mod] Peertube: re-engineered & upgrade to data_type: traits_v1 
- fetch_traits(): Fetch languages from peertube's search-index source code.

  [mod] Include migration of the request methode from 'supported_languages'
        to 'traits' (EngineTraits) object.
  [fix] old supported_languages_url is no longer valid since the sources
        has been moved to a different path.

- fixed code to pass pylint
- request(): complete re-implementation based on the API docs [1]
- response(): complete re-implementation, adds serveral fields missed before
- add source code documentation

[1] https://docs.joinpeertube.org/api-rest-reference.html#tag/Search/operation/searchVideos

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-03-24 10:37:42 +01:00
Markus Heiser
6e5f22e558 [mod] replace engines_languages.json by engines_traits.json 
Implementations of the *traits* of the engines.

Engine's traits are fetched from the origin engine and stored in a JSON file in
the *data folder*.  Most often traits are languages and region codes and their
mapping from SearXNG's representation to the representation in the origin search
engine.

To load traits from the persistence::

    searx.enginelib.traits.EngineTraitsMap.from_data()

For new traits new properties can be added to the class::

    searx.enginelib.traits.EngineTraits

.. hint::

   Implementation is downward compatible to the deprecated *supported_languages
   method* from the vintage implementation.

   The vintage code is tagged as *deprecated* an can be removed when all engines
   has been ported to the *traits method*.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-03-24 10:37:42 +01:00
Markus Heiser
097d092a7f
Merge pull request #2224 from searxng/update_data_update_currencies.py 
Update searx.data - update_currencies.py
 2023-03-15 18:36:29 +01:00
Markus Heiser
85ef1af343
Merge pull request #2222 from searxng/update_data_update_wikidata_units.py 
Update searx.data - update_wikidata_units.py
 2023-03-15 18:35:36 +01:00
Alexandre Flament
1632f18631
Merge pull request #2227 from searxng/update_data_update_engine_descriptions.py 
Update searx.data - update_engine_descriptions.py
 2023-03-01 17:52:57 +01:00
Alexandre Flament
5bbbb14b62
Merge pull request #2226 from searxng/update_data_update_ahmia_blacklist.py 
Update searx.data - update_ahmia_blacklist.py
 2023-03-01 17:52:02 +01:00
dalf
5042d94dea Update searx.data - update_engine_descriptions.py 2023-03-01 01:48:22 +00:00
dalf
e30a45812f Update searx.data - update_ahmia_blacklist.py 2023-03-01 01:38:07 +00:00
dalf
935415bfcf Update searx.data - update_currencies.py 2023-03-01 01:37:46 +00:00
dalf
ccd00518fd Update searx.data - update_firefox_version.py 2023-03-01 01:37:42 +00:00
dalf
d7f10909fa Update searx.data - update_wikidata_units.py 2023-03-01 01:37:25 +00:00
Markus Heiser
d989c9429f
Merge pull request #2134 from searxng/update_data_update_wikidata_units.py 
Update searx.data - update_wikidata_units.py
 2023-01-29 17:59:40 +01:00
Markus Heiser
5c2667b956
Merge pull request #2135 from searxng/update_data_update_ahmia_blacklist.py 
Update searx.data - update_ahmia_blacklist.py
 2023-01-29 17:59:03 +01:00
Markus Heiser
e6a17f2fae
Merge pull request #2136 from searxng/update_data_update_firefox_version.py 
Update searx.data - update_firefox_version.py
 2023-01-29 17:58:22 +01:00
Markus Heiser
c4cbf8829a
Merge pull request #2137 from searxng/update_data_update_currencies.py 
Update searx.data - update_currencies.py
 2023-01-29 17:57:56 +01:00
dalf
f900e33092 Update searx.data - update_engine_descriptions.py 2023-01-29 01:33:29 +00:00
dalf
dafe026815 Update searx.data - update_currencies.py 2023-01-29 01:24:14 +00:00
dalf
47435be70b Update searx.data - update_firefox_version.py 2023-01-29 01:23:43 +00:00
dalf
53a4a90d0c Update searx.data - update_wikidata_units.py 2023-01-29 01:23:42 +00:00
dalf
9dd95457a9 Update searx.data - update_ahmia_blacklist.py 2023-01-29 01:23:42 +00:00
dalf
0639769274 Update searx.data - update_firefox_version.py 2022-12-29 09:17:56 +01:00
Markus Heiser
174e6851b3
Merge pull request #2054 from searxng/update_data_update_currencies.py 
Update searx.data - update_currencies.py
 2022-12-29 08:35:51 +01:00
Markus Heiser
d531f89316
Merge pull request #2057 from searxng/update_data_update_engine_descriptions.py 
Update searx.data - update_engine_descriptions.py
 2022-12-29 08:31:19 +01:00
Markus Heiser
76cd808a49
Merge pull request #2053 from searxng/update_data_update_ahmia_blacklist.py 
Update searx.data - update_ahmia_blacklist.py
 2022-12-29 08:28:19 +01:00
dalf
000734cb93 Update searx.data - update_engine_descriptions.py 2022-12-29 01:25:31 +00:00
dalf
36eea5b7cc Update searx.data - update_currencies.py 2022-12-29 01:16:59 +00:00
dalf
66780efda7 Update searx.data - update_wikidata_units.py 2022-12-29 01:16:59 +00:00
dalf
de793fb71e Update searx.data - update_ahmia_blacklist.py 2022-12-29 01:16:52 +00:00
ArtikusHG
9925a20950 [mod] new plugin: Autodetect search language 2022-12-10 13:11:47 +01:00
Markus Heiser
a8359dd4d3
Merge pull request #1981 from searxng/update_data_update_firefox_version.py 
Update searx.data - update_firefox_version.py
 2022-11-29 07:48:26 +01:00
Markus Heiser
1b2f1c17f6
Merge pull request #1982 from searxng/update_data_update_wikidata_units.py 
Update searx.data - update_wikidata_units.py
 2022-11-29 07:47:52 +01:00
Markus Heiser
3579a38a2d
Merge pull request #1983 from searxng/update_data_update_currencies.py 
Update searx.data - update_currencies.py
 2022-11-29 07:46:49 +01:00
Markus Heiser
82af2f4460
Merge pull request #1984 from searxng/update_data_update_ahmia_blacklist.py 
Update searx.data - update_ahmia_blacklist.py
 2022-11-29 07:45:51 +01:00
dalf
a508cdbd4b Update searx.data - update_engine_descriptions.py 2022-11-29 01:34:42 +00:00
dalf
c4512cf64d Update searx.data - update_ahmia_blacklist.py 2022-11-29 01:24:48 +00:00
dalf
b0b45fd2d0 Update searx.data - update_currencies.py 2022-11-29 01:24:39 +00:00
dalf
a9be73b4da Update searx.data - update_wikidata_units.py 2022-11-29 01:24:38 +00:00
dalf
9fc72419e1 Update searx.data - update_firefox_version.py 2022-11-29 01:24:35 +00:00
Alexandre Flament
e473addaff User agent: don't include the patch number in the Firefox version 
The Firefox version in the user agent doesn't include the patch version: 106.0 not 106.0.2

Close #1914
 2022-11-05 22:04:37 +01:00
Markus Heiser
fc9986de0a
Merge pull request #1905 from searxng/update_data_update_ahmia_blacklist.py 
Update searx.data - update_ahmia_blacklist.py
 2022-10-29 08:22:43 +02:00
Markus Heiser
3f1d594c23
Merge pull request #1906 from searxng/update_data_update_external_bangs.py 
Update searx.data - update_external_bangs.py
 2022-10-29 08:21:33 +02:00
Markus Heiser
a9deead102
Merge pull request #1907 from searxng/update_data_update_firefox_version.py 
Update searx.data - update_firefox_version.py
 2022-10-29 08:20:53 +02:00
Markus Heiser
d49ccb54ea
Merge pull request #1908 from searxng/update_data_update_wikidata_units.py 
Update searx.data - update_wikidata_units.py
 2022-10-29 08:20:27 +02:00
Markus Heiser
fa59ff9b78
Merge pull request #1909 from searxng/update_data_update_currencies.py 
Update searx.data - update_currencies.py
 2022-10-29 08:19:56 +02:00
dalf
e70589c598 Update searx.data - update_engine_descriptions.py 2022-10-29 01:56:57 +00:00
dalf
985a2a87a6 Update searx.data - update_currencies.py 2022-10-29 01:46:40 +00:00
dalf
c0ecc2f865 Update searx.data - update_wikidata_units.py 2022-10-29 01:46:38 +00:00
dalf
c24abc2f8c Update searx.data - update_firefox_version.py 2022-10-29 01:46:34 +00:00
dalf
f0c7898445 Update searx.data - update_external_bangs.py 2022-10-29 01:46:27 +00:00
dalf
799c5270ae Update searx.data - update_ahmia_blacklist.py 2022-10-29 01:46:24 +00:00
Markus Heiser
23d797ee6f Update searx.data - update_osm_keys_tags.py 
Patch was generated by::

  ./manage pyenv.cmd searxng_extra/update/update_osm_keys_tags.py

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2022-10-11 11:49:06 +02:00
Alexandre Flament
a7d693231c
Merge pull request #1819 from searxng/update_data_update_currencies.py 
Update searx.data - update_currencies.py
 2022-09-29 16:03:34 +02:00
Alexandre Flament
f3d25f9cef
Merge pull request #1821 from searxng/update_data_update_engine_descriptions.py 
Update searx.data - update_engine_descriptions.py
 2022-09-29 16:02:57 +02:00
Alexandre Flament
5060732486
Merge pull request #1818 from searxng/update_data_update_ahmia_blacklist.py 
Update searx.data - update_ahmia_blacklist.py
 2022-09-29 16:02:02 +02:00
Alexandre Flament
520a873ab4
Merge pull request #1820 from searxng/update_data_update_wikidata_units.py 
Update searx.data - update_wikidata_units.py
 2022-09-29 16:01:31 +02:00
dalf
9cb792c0d6 Update searx.data - update_engine_descriptions.py 2022-09-29 02:22:00 +00:00
dalf
2d429ebd75 Update searx.data - update_wikidata_units.py 2022-09-29 02:13:02 +00:00
dalf
8fc3940448 Update searx.data - update_currencies.py 2022-09-29 02:12:54 +00:00
dalf
b7f4a3a207 Update searx.data - update_ahmia_blacklist.py 2022-09-29 02:12:48 +00:00
dalf
3153f0a795 Update searx.data - update_firefox_version.py 2022-09-29 02:12:48 +00:00
Markus Heiser
13ef9cc125
Merge pull request #1720 from searxng/update_data_update_ahmia_blacklist.py 
Update searx.data - update_ahmia_blacklist.py
 2022-08-29 07:09:04 +02:00
Markus Heiser
e9b564d066
Merge pull request #1722 from searxng/update_data_update_languages.py 
Update searx.data - update_languages.py
 2022-08-29 07:08:26 +02:00
Markus Heiser
2b65502388
Merge pull request #1723 from searxng/update_data_update_firefox_version.py 
Update searx.data - update_firefox_version.py
 2022-08-29 07:07:54 +02:00
Markus Heiser
55d04a089d
Merge pull request #1724 from searxng/update_data_update_engine_descriptions.py 
Update searx.data - update_engine_descriptions.py
 2022-08-29 07:07:14 +02:00
Markus Heiser
4a96480bd5
Merge pull request #1721 from searxng/update_data_update_wikidata_units.py 
Update searx.data - update_wikidata_units.py
 2022-08-29 07:06:39 +02:00
dalf
c2400a8677 Update searx.data - update_engine_descriptions.py 2022-08-29 02:17:55 +00:00
dalf
e8bf907eef Update searx.data - update_firefox_version.py 2022-08-29 02:09:34 +00:00
dalf
915c0a2bc6 Update searx.data - update_languages.py 2022-08-29 02:09:27 +00:00
dalf
b1ccecbeb3 Update searx.data - update_wikidata_units.py 2022-08-29 02:09:17 +00:00
dalf
2e6d41fa24 Update searx.data - update_ahmia_blacklist.py 2022-08-29 02:09:09 +00:00
dalf
83fbc16908 Update searx.data - update_currencies.py 2022-08-29 02:09:09 +00:00
Markus Heiser
6579d6d558 [fix] qwant - API error::locale must be one .. 
The request function should not request a language (aka locale) that is not
supported by qwant. Select a locale like zh-TW ends in qwant's API error:

  ERROR searx.engines.qwant news: exception : \
  API error::locale must be one of the following values: \
    en_gb, en_ie, en_us, en_ca, en_my, en_au, en_nz, de_de, de_ch, de_at, fr_fr, \
    fr_be, fr_ch, fr_ca, fr_ad, fc_ca, co_fr, es_es, es_ar, es_cl, es_co, es_mx, \
    es_pe, es_ad, ca_es, ca_ad, ca_fr, eu_es, eu_fr, it_it, it_ch, pt_pt, pt_ad, \
    nl_be, nl_nl

The existing searx.utils.match_language function is unsuitable for this purpose,
it is replaced by function searx.locales.get_engine_locale that is based on the
methods from the babel package.

The quant's _fetch_supported_languages function has been revised to filter out
languages 8aka locales) not supported by qwant.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2022-08-14 10:36:14 +02:00
Markus Heiser
73ffa0c070 Update searx.data - update_engine_descriptions.py 
./manage pyenv.cmd python ./searxng_extra/update/update_engine_descriptions.py

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2022-08-10 21:48:34 +02:00
Markus Heiser
14085cc0a7
Merge pull request #1580 from searxng/update_data_update_engine_descriptions.py 
Update searx.data - update_engine_descriptions.py
 2022-07-29 07:23:02 +02:00
Markus Heiser
7602c23d71
Merge pull request #1579 from searxng/update_data_update_currencies.py 
Update searx.data - update_currencies.py
 2022-07-29 07:22:12 +02:00
Markus Heiser
0bad4fb931
Merge pull request #1578 from searxng/update_data_update_firefox_version.py 
Update searx.data - update_firefox_version.py
 2022-07-29 07:21:06 +02:00
Markus Heiser
7dd75ae080
Merge pull request #1576 from searxng/update_data_update_languages.py 
Update searx.data - update_languages.py
 2022-07-29 07:20:28 +02:00
dalf
2bdb367e8c Update searx.data - update_engine_descriptions.py 2022-07-29 02:13:07 +00:00
dalf
fc019fb296 Update searx.data - update_currencies.py 2022-07-29 02:01:30 +00:00
dalf
c2662ec3c8 Update searx.data - update_firefox_version.py 2022-07-29 02:01:08 +00:00
dalf
385542e6c5 Update searx.data - update_ahmia_blacklist.py 2022-07-29 02:01:04 +00:00
dalf
90068b3d0b Update searx.data - update_languages.py 2022-07-29 02:00:53 +00:00
Markus Heiser
445cba7c02 Update searx.data - update_engine_descriptions.py 
./manage pyenv.cmd python ./searxng_extra/update/update_engine_descriptions.py

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2022-07-15 09:26:44 +02:00
Markus Heiser
81e8c7370f Update searx.data - update_engine_descriptions.py 
./manage pyenv.cmd python ./searxng_extra/update/update_engine_descriptions.py

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2022-07-07 17:16:41 +02:00
Markus Heiser
1416559f22 Update searx.data - update_engine_descriptions.py 
./manage pyenv.cmd python ./searxng_extra/update/update_engine_descriptions.py

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2022-07-02 12:16:00 +02:00
Markus Heiser
2bf800a4b1 Update searx.data - update_currencies.py 
./manage pyenv.cmd python ./searxng_extra/update/update_currencies.py

Closes: https://github.com/searxng/searxng/pull/1375
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2022-07-02 12:16:00 +02:00
Markus Heiser
eb3123e164
Merge pull request #1379 from searxng/update_data_update_ahmia_blacklist.py 
Update searx.data - update_ahmia_blacklist.py
 2022-06-29 15:40:51 +02:00
dalf
a89fca9d08 Update searx.data - update_ahmia_blacklist.py 2022-06-29 07:21:09 +00:00
Markus Heiser
3a75d3c1cc
Merge pull request #1376 from searxng/update_data_update_firefox_version.py 
Update searx.data - update_firefox_version.py
 2022-06-29 08:45:29 +02:00
dalf
11def5155e Update searx.data - update_wikidata_units.py 2022-06-29 01:52:03 +00:00
dalf
21ab024469 Update searx.data - update_firefox_version.py 2022-06-29 01:51:46 +00:00
Markus Heiser
e568da2314
Merge pull request #1266 from searxng/update_data_update_firefox_version.py 
Update searx.data - update_firefox_version.py
 2022-06-01 13:49:48 +02:00
Markus Heiser
84a5e48a90
Merge pull request #1265 from searxng/update_data_update_ahmia_blacklist.py 
Update searx.data - update_ahmia_blacklist.py
 2022-06-01 11:29:41 +02:00
Markus Heiser
78968843a8
Merge pull request #1267 from searxng/update_data_update_currencies.py 
Update searx.data - update_currencies.py
 2022-06-01 11:24:35 +02:00
Markus Heiser
a9251cfeb4
Merge pull request #1268 from searxng/update_data_update_engine_descriptions.py 
Update searx.data - update_engine_descriptions.py
 2022-06-01 11:24:00 +02:00
Markus Heiser
61ee915064
Merge pull request #1264 from searxng/update_data_update_languages.py 
Update searx.data - update_languages.py
 2022-06-01 10:34:19 +02:00
dalf
fda176144e Update searx.data - update_engine_descriptions.py 2022-06-01 06:40:37 +00:00
dalf
f37b9978cb Update searx.data - update_currencies.py 2022-06-01 06:32:10 +00:00
dalf
a93170725f Update searx.data - update_firefox_version.py 2022-06-01 06:31:57 +00:00
dalf
8d6d32bf6e Update searx.data - update_ahmia_blacklist.py 2022-06-01 06:31:54 +00:00
dalf
012d7fcaa0 Update searx.data - update_languages.py 2022-06-01 06:31:54 +00:00
dalf
ab9616c2cc Update searx.data - update_wikidata_units.py 2022-06-01 06:31:52 +00:00
Markus Heiser
8a79eb4d7b
Merge pull request #1158 from searxng/update_data_update_currencies.py 
Update searx.data - update_currencies.py
 2022-05-01 09:51:56 +02:00
Markus Heiser
6bd6e09e14
Merge pull request #1159 from searxng/update_data_update_engine_descriptions.py 
Update searx.data - update_engine_descriptions.py
 2022-05-01 09:51:21 +02:00
Markus Heiser
9c7e9bf1d9
Merge pull request #1157 from searxng/update_data_update_wikidata_units.py 
Update searx.data - update_wikidata_units.py
 2022-05-01 09:50:28 +02:00
Markus Heiser
9e6000407a
Merge pull request #1156 from searxng/update_data_update_ahmia_blacklist.py 
Update searx.data - update_ahmia_blacklist.py
 2022-05-01 09:49:55 +02:00
dalf
6f0a717dce Update searx.data - update_engine_descriptions.py 2022-05-01 06:33:42 +00:00
dalf
bdfbc0ec02 Update searx.data - update_currencies.py 2022-05-01 06:26:53 +00:00
dalf
6cb08a568a Update searx.data - update_wikidata_units.py 2022-05-01 06:26:44 +00:00
dalf
10023344a5 Update searx.data - update_ahmia_blacklist.py 2022-05-01 06:26:33 +00:00
dalf
e36d7067a9 Update searx.data - update_firefox_version.py 2022-05-01 06:26:31 +00:00
Markus Heiser
62982c8812 [fix] add back missing languages & regions (followup of PR #1071) 
In PR #1071 the language catalog of dailymotion has been cleaned up, before
there had been over 7000 "languages" in the catalog.

As a side effect of this clean-up the language & region catalog in SearXNG has
been reduced [1].

This patch reduce the ``min_engines_per_lang`` from 13 to 12 to get the missed
languages back in language & region catalog of SearXNG.

[1] 3bb62823ec (diff-f3f00db0f87f95b882624a192e0aac21525638af0b18c9514e765fcf1991678d)

Requested-by: @tiekoetter in a Matrix chat
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2022-04-22 12:09:42 +02:00
Alexandre Flament
7bf9e12db9 Update searx.data - update_engine_descriptions.py 2022-04-16 13:17:39 +02:00
Markus Heiser
3bb62823ec [fix] dailymotion engine: filter by language & country 
- fix the issue of fetching more the 7000 *languages*
- improve the request function and filter by language & country
- implement time_range_support & safesearch
- add more fields to the response from dailymotion (allow_embed, length)
- better clean up of HTML tags in the 'content' field.

This is more or less a complete rework based on the '/videos' API from [1].
This patch cleans up the language list in SearXNG that has been polluted by the
ISO-639-3 2 and 3 letter codes from dailymotion languages which have never been
used.

[1] https://developers.dailymotion.com/tools/

Closes: https://github.com/searxng/searxng/issues/1065
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2022-04-16 09:27:34 +02:00
Alexandre Flament
765504ceca
Merge pull request #1031 from dalf/fix-bing-_fetch_supported_languages 
Fix bing  fetch supported languages
 2022-04-02 08:23:58 +02:00
Markus Heiser
099840f4d6
Merge pull request #1035 from searxng/update_data_update_wikidata_units.py 
Update searx.data - update_wikidata_units.py
 2022-04-01 09:02:19 +02:00
Markus Heiser
b853ab717e
Merge pull request #1034 from searxng/update_data_update_ahmia_blacklist.py 
Update searx.data - update_ahmia_blacklist.py
 2022-04-01 09:01:48 +02:00
Markus Heiser
c2de356497
Merge pull request #1033 from searxng/update_data_update_currencies.py 
Update searx.data - update_currencies.py
 2022-04-01 09:00:51 +02:00
dalf
086517d269 Update searx.data - update_wikidata_units.py 2022-04-01 06:27:23 +00:00
dalf
d245ca97ca Update searx.data - update_ahmia_blacklist.py 2022-04-01 06:27:22 +00:00
dalf
084d9ab4e4 Update searx.data - update_currencies.py 2022-04-01 06:27:18 +00:00
dalf
4d55523247 Update searx.data - update_firefox_version.py 2022-04-01 06:27:13 +00:00
Alexandre FLAMENT
7d31b2c0af Update languages 2022-03-31 20:04:46 +00:00
Markus Heiser
0ddcc12474
Merge pull request #934 from searxng/update_data_update_ahmia_blacklist.py 
Update searx.data - update_ahmia_blacklist.py
 2022-03-01 07:54:58 +01:00
Markus Heiser
69f84d941f
Merge pull request #935 from searxng/update_data_update_firefox_version.py 
Update searx.data - update_firefox_version.py
 2022-03-01 07:53:33 +01:00
Markus Heiser
5508155252
Merge pull request #936 from searxng/update_data_update_wikidata_units.py 
Update searx.data - update_wikidata_units.py
 2022-03-01 07:52:54 +01:00
dalf
f4c6eba227 Update searx.data - update_currencies.py 2022-03-01 06:25:18 +00:00
dalf
6946c4566f Update searx.data - update_wikidata_units.py 2022-03-01 06:24:57 +00:00
dalf
49bc3971ae Update searx.data - update_firefox_version.py 2022-03-01 06:24:55 +00:00
dalf
6add9dd6d3 Update searx.data - update_ahmia_blacklist.py 2022-03-01 06:24:53 +00:00
Markus Heiser
314ea2dc61
Merge pull request #852 from searxng/update_data_update_ahmia_blacklist.py 
Update searx.data - update_ahmia_blacklist.py
 2022-02-01 06:52:57 +00:00
Markus Heiser
4d0bd47ff0
Merge pull request #853 from searxng/update_data_update_firefox_version.py 
Update searx.data - update_firefox_version.py
 2022-02-01 06:52:19 +00:00
Markus Heiser
20cbabff96
Merge pull request #851 from searxng/update_data_update_currencies.py 
Update searx.data - update_currencies.py
 2022-02-01 06:48:54 +00:00
dalf
f4f90aa3a4 Update searx.data - update_firefox_version.py 2022-02-01 06:23:16 +00:00
dalf
7b402e1aa0 Update searx.data - update_ahmia_blacklist.py 2022-02-01 06:23:01 +00:00
dalf
1c47f4b6e0 Update searx.data - update_currencies.py 2022-02-01 06:23:00 +00:00
dalf
e78dc45d1e Update searx.data - update_wikidata_units.py 2022-02-01 06:22:58 +00:00
Alexandre Flament
f3f61df6a0 [mod] remove deprecate code 
remove code to support Python 3.5 and Python 3.6
 2022-01-29 08:54:12 +01:00
Markus Heiser
a7c3c352d8 [upd] searx/data/external_bangs.json 
Updated external bangs by::

    ./manage pyenv.cmd ./searxng_extra/update/update_external_bangs.py

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2022-01-12 19:37:13 +01:00
Markus Heiser
1461d6a07e [upd] engine_descriptions.json 
Updated by:

    ./manage pyenv.cmd python3 \
        ./searxng_extra/update/update_engine_descriptions.py

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2022-01-07 13:59:03 +01:00
Markus Heiser
3cdf2d6570 Update searx.data - update_languages.py 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2022-01-01 18:37:35 +01:00
Alexandre Flament
6b49f32d3e
Merge pull request #670 from searxng/update_data_update_firefox_version.py 
Update searx.data - update_firefox_version.py
 2022-01-01 12:16:50 +01:00
Markus Heiser
3dcba2f67c
Merge pull request #671 from searxng/update_data_update_ahmia_blacklist.py 
Update searx.data - update_ahmia_blacklist.py
 2022-01-01 12:08:26 +01:00
Markus Heiser
877293c470
Merge pull request #673 from searxng/update_data_update_currencies.py 
Update searx.data - update_currencies.py
 2022-01-01 12:08:02 +01:00
dalf
127da09fc9 Update searx.data - update_wikidata_units.py 2022-01-01 06:25:06 +00:00
dalf
291483af53 Update searx.data - update_currencies.py 2022-01-01 06:25:02 +00:00
dalf
a0016250ec Update searx.data - update_ahmia_blacklist.py 2022-01-01 06:24:59 +00:00
dalf
184242d36b Update searx.data - update_firefox_version.py 2022-01-01 06:24:49 +00:00
Markus Heiser
3d96a9839a [format.python] initial formatting of the python code 
This patch was generated by black [1]::

    make format.python

[1] https://github.com/psf/black

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2021-12-27 09:26:22 +01:00
Markus Heiser
c5d604f06f [upd] engine_descriptions.json 
Updated by:

    ./manage pyenv.cmd python3 \
        ./searxng_extra/update/update_engine_descriptions.py

Include documentation of new added engines:

- superuser
- stackoverflow
- askubuntu
- indb
- ...

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2021-12-20 14:39:10 +01:00
Markus Heiser
e62733fcf3
Merge pull request #571 from searxng/update_data_update_languages.py 
Update searx.data - update_languages.py
 2021-12-01 07:30:41 +01:00
Markus Heiser
24d04f2854
Merge pull request #570 from searxng/update_data_update_firefox_version.py 
Update searx.data - update_firefox_version.py
 2021-12-01 07:29:56 +01:00
Markus Heiser
a3fc1304cc
Merge pull request #569 from searxng/update_data_update_currencies.py 
Update searx.data - update_currencies.py
 2021-12-01 07:29:24 +01:00
dalf
c74b465ece Update searx.data - update_languages.py 2021-12-01 06:24:47 +00:00
dalf
56e653011f Update searx.data - update_firefox_version.py 2021-12-01 06:24:38 +00:00
dalf
b833e08fc3 Update searx.data - update_currencies.py 2021-12-01 06:24:29 +00:00
dalf
5fdf8f04b7 Update searx.data - update_ahmia_blacklist.py 2021-12-01 06:24:25 +00:00
Markus Heiser
07f849cd56
Merge pull request #468 from searxng/update_data_update_ahmia_blacklist.py 
Update searx.data - update_ahmia_blacklist.py
 2021-11-01 06:49:09 +00:00
Markus Heiser
853a144bb4
Merge pull request #467 from searxng/update_data_update_firefox_version.py 
Update searx.data - update_firefox_version.py
 2021-11-01 06:48:43 +00:00
Markus Heiser
82a0dfab61
Merge pull request #466 from searxng/update_data_update_wikidata_units.py 
Update searx.data - update_wikidata_units.py
 2021-11-01 06:48:15 +00:00
dalf
f3dfdad1d1 Update searx.data - update_ahmia_blacklist.py 2021-11-01 06:24:06 +00:00
dalf
2be4c35f76 Update searx.data - update_firefox_version.py 2021-11-01 06:23:59 +00:00
dalf
073d3edc0c Update searx.data - update_wikidata_units.py 2021-11-01 06:23:58 +00:00
dalf
3d9be86519 Update searx.data - update_currencies.py 2021-11-01 06:23:54 +00:00
Markus Heiser
f63ffbb22b [fix] engine - yahoo: rewrite and fix issues 
Languages are supported by mapping the language to a domain.  If domain is not
found in :py:obj:`lang2domain` URL ``<lang>.search.yahoo.com`` is used.

BTW: fix issue reported at https://github.com/searx/searx/issues/3020

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2021-10-16 20:05:26 +00:00
Alexandre Flament
1bb82a6b54 SearXNG: searxng_extra 2021-10-02 17:30:39 +02:00
Markus Heiser
fda5609a17
Merge pull request #360 from return42/ddg-lite 
Implement a DuckDuckGo Lite engine
 2021-10-02 05:16:29 +00:00
Markus Heiser
97419a770d [upd] make data.languages 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2021-10-01 20:01:41 +02:00
Markus Heiser
c74f45b1df
Merge pull request #365 from searxng/update_data_update_wikidata_units.py 
Update searx.data - update_wikidata_units.py
 2021-10-01 07:11:13 +00:00
Markus Heiser
62ea3c7074
Merge pull request #366 from searxng/update_data_update_languages.py 
Update searx.data - update_languages.py
 2021-10-01 06:41:09 +00:00
Markus Heiser
d814f1182f
Merge pull request #364 from searxng/update_data_update_firefox_version.py 
Update searx.data - update_firefox_version.py
 2021-10-01 06:34:20 +00:00
Markus Heiser
8479a83411
Merge pull request #363 from searxng/update_data_update_ahmia_blacklist.py 
Update searx.data - update_ahmia_blacklist.py
 2021-10-01 06:33:51 +00:00
dalf
db51a64390 Update searx.data - update_languages.py 2021-10-01 06:23:31 +00:00
dalf
a582e55af0 Update searx.data - update_wikidata_units.py 2021-10-01 06:23:25 +00:00
dalf
56c68beb42 Update searx.data - update_firefox_version.py 2021-10-01 06:23:14 +00:00
dalf
453114fbe4 Update searx.data - update_ahmia_blacklist.py 2021-10-01 06:23:12 +00:00
dalf
0c24a03f05 Update searx.data - update_currencies.py 2021-10-01 06:23:04 +00:00
Alexandre Flament
bfd24d1226 [mod] add /engine_descriptions.json endpoint 
returns engine descriptions (JSON):
* key: engine name
* value: description in the user locale, use English description as a fallback
 2021-09-24 20:20:30 +02:00
Markus Heiser
065b4dab56
Merge pull request #278 from searxng/update_data_update_ahmia_blacklist.py 
Update searx.data - update_ahmia_blacklist.py
 2021-09-01 06:48:55 +00:00
Markus Heiser
a7a879181d
Merge pull request #279 from searxng/update_data_update_currencies.py 
Update searx.data - update_currencies.py
 2021-09-01 06:46:29 +00:00
Markus Heiser
00a51fa17f
Merge pull request #280 from searxng/update_data_update_firefox_version.py 
Update searx.data - update_firefox_version.py
 2021-09-01 06:44:22 +00:00
dalf
9e3113865d Update searx.data - update_firefox_version.py 2021-09-01 06:23:44 +00:00
dalf
a45fc66e83 Update searx.data - update_currencies.py 2021-09-01 06:23:35 +00:00
dalf
44f7e1fcaa Update searx.data - update_ahmia_blacklist.py 2021-09-01 06:23:20 +00:00
dalf
a556d0c1aa Update searx.data - update_wikidata_units.py 2021-09-01 06:23:11 +00:00
Markus Heiser
e0ca749ff5
Merge pull request #242 from searxng/update_data_update_currencies.py 
Update searx.data - update_currencies.py
 2021-08-02 07:40:19 +00:00
Alexandre Flament
b6711ba0ad
Merge pull request #241 from searxng/update_data_update_languages.py 
Update searx.data - update_languages.py
 2021-08-02 09:01:57 +02:00
Markus Heiser
0fc1dcbfa5
Merge pull request #243 from searxng/update_data_update_wikidata_units.py 
Update searx.data - update_wikidata_units.py
 2021-08-01 09:42:01 +00:00
dalf
7fae5647d3 Update searx.data - update_wikidata_units.py 2021-08-01 06:24:23 +00:00
dalf
2963c83daa Update searx.data - update_currencies.py 2021-08-01 06:24:13 +00:00
dalf
4d3f2f48d2 Update searx.data - update_languages.py 2021-08-01 06:24:12 +00:00