kamp/searxng
1
0
Fork 0
mirror of https://github.com/searxng/searxng.git synced 2024-11-15 09:10:12 +01:00
Code
8,151 Commits 8 Branches 17 Tags 132 MiB
2fbedc4316
Commit Graph

5165 Commits

Author SHA1 Message Date
allixx
6d31ca28b5 [fix] remove missing link icon macro 2024-02-01 08:13:43 +01:00
Hermógenes Oliveira
100a150a33 Remove rtl exception 
This right-to-left exception doesn't seem to be called for.
 2024-01-31 20:03:10 +01:00
Hermógenes Oliveira
f9f0c2ec03 Fix media exposure toggle 
Make initially hidden and the toggle button consistent.
 2024-01-31 20:03:10 +01:00
Hermógenes Oliveira
fa7fef7d87 Provide template for results in file category 
This is used, for instance, by the recoll engine.
 2024-01-31 20:03:10 +01:00
Markus Heiser
ab8e5383fb [mod] remove X-XSS-Protection headers 
Deprecated header not used by browsers nowadays[1]:

"""In modern browsers, X-XSS-Protection has been deprecated in favor of the
Content-Security-Policy to disable the use of inline JavaScript. Its use can
introduce XSS vulnerabilities in otherwise safe websites. This should not be
used unless you need to support older web browsers that don’t yet support CSP.
It is thus recommended to set the header as X-XSS-Protection: 0."""[2]

[1] https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/X-XSS-Protection
[2] https://infosec.mozilla.org/guidelines/web_security#x-xss-protection

Closes: https://github.com/searxng/searxng/issues/3171
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2024-01-31 17:23:41 +01:00
allixx
dca78f920f [build] /static 2024-01-31 17:22:03 +01:00
allixx
c85f706601 [fix] wrong engines padding 2024-01-31 17:22:03 +01:00
allixx
30dcaf1db6 [fix] equalize cached/proxied links left margin 2024-01-31 17:22:03 +01:00
allixx
1e03094dee [fix] justify engines/cached/proxied vertically 2024-01-31 17:22:03 +01:00
Markus Heiser
c14b117121 [fix] brave.search - update result selectors to fit to new layout 
Closes: https://github.com/searxng/searxng/issues/3167
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2024-01-31 16:21:56 +01:00
Bnyro
efbee96b1d [feat] brave: fix empty content and parse published dates 2024-01-30 07:41:45 +01:00
Bnyro
90072eb6ff [build] /static 2024-01-29 18:09:22 +01:00
Bnyro
a5decfb838 [feat] preferences: button to copy input to restore preferences hash 2024-01-29 18:09:22 +01:00
Bnyro
559b860618 [feat] engine: implementation of goodreads 2024-01-29 16:07:24 +01:00
searxng-bot
a962aab9b0 [translations] update from Weblate 
c1a3532a2 - 2024-01-23 - return42 <markus.heiser@darmarit.de>
603c6894b - 2024-01-23 - return42 <markus.heiser@darmarit.de>
2920c7342 - 2024-01-22 - Eshan-K-I <2027eiyer@tjhsst.edu>
64cc1168a - 2024-01-20 - return42 <markus.heiser@darmarit.de>
21ec41823 - 2024-01-20 - return42 <markus.heiser@darmarit.de>
 2024-01-29 15:01:37 +01:00
dalf
62fa85287a Update searx.data - update_wikidata_units.py 2024-01-29 14:04:24 +01:00
dalf
0ed7548d83 Update searx.data - update_firefox_version.py 2024-01-29 14:04:03 +01:00
dalf
d763934a7a Update searx.data - update_ahmia_blacklist.py 2024-01-29 14:03:03 +01:00
dalf
14f73ef3d9 Update searx.data - update_engine_traits.py 2024-01-29 14:02:30 +01:00
dalf
a393341403 Update searx.data - update_engine_descriptions.py 2024-01-29 13:34:12 +01:00
allixx
e4cf0a7d4f [fix] do highlight replacement at once 
Highlights all search queries in search result in one go.

Fixes the case where search query contains word from highlight HTML code,
which causes broken HTML to appear in search results.

Closes #3057
 2024-01-29 13:15:37 +01:00
Bnyro
8c73aa772b [fix] semantic scholar: bad request 2024-01-25 06:12:08 +01:00
searxng-bot
ab5d2f32bb [translations] update from Weblate 
0f8246b81 - 2024-01-17 - return42 <markus.heiser@darmarit.de>
ce23e8ce9 - 2024-01-16 - return42 <markus.heiser@darmarit.de>
1112731cb - 2024-01-17 - return42 <markus.heiser@darmarit.de>
 2024-01-19 08:29:29 +01:00
Émilien (perso)
7c80807bb8
Block HeadlessChrome (#3116) 2024-01-17 09:14:36 +01:00
Jinyuan Huang
9c5f9735cf [fix] engine: safesearch parameter in Google Videos engine (#2762) 
Closes: https://github.com/searxng/searxng/issues/2762
 2024-01-16 15:29:09 +01:00
Markus Heiser
8762863ebf [mod] presearch: set WEB timeout to 4sec & single network for all request 
timeout: 4.0
  The timeout of presearch-WEB is left up from the default of 3sec to 4sec.  The
  engine has to send two HTTP requests, they often exceed the default timeout of
  3sec. Since all other presearch categories (images, videos, news) also have a
  timeout of 4 sec, the WEB search should also have the same timeout.

network: presearch
  Place all HTTP requests in the same network, named ``presearch``.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2024-01-15 19:23:26 +01:00
Markus Heiser
e560d7e373 [mod] presearch: add language & region support 
In Presearch there are languages for the UI and regions for narrowing down the
search.  With this change the SearXNG engine supports a search by region.  The
details can be found in the documentation of the source code.

To test, you can search terms like::

   !presearch bmw :zh-TW
   !presearch bmw :en-CA

1. You should get results corresponding to the region (Taiwan, Canada)
2. and in the language (Chinese, Englisch).
3. The context in info box content is in the same language.

Exceptions:

1. Region or language is not supported by Presearch or
2. SearXNG user did not selected a region tag, example::

    !presearch bmw :en

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2024-01-15 19:23:26 +01:00
Markus Heiser
a2c269bbac [mod] presearch: hardening engine's response against KeyErrors 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2024-01-15 19:23:26 +01:00
Bnyro
0a78f59aba [fix] presearch: safesearch, time ranges, crash when no result found 2024-01-15 19:23:26 +01:00
Markus Heiser
87f18b98ec [fix] SyntaxWarning: invalid escape sequence '\>' 
This patch fixes issue reported by ``make test.unit``::

   searx/search/checker/impl.py:39: SyntaxWarning: invalid escape sequence '\>'
      rep = ['<' + tag + '[^\>]*>' for tag in HTML_TAGS]

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2024-01-15 18:27:21 +01:00
searxng-bot
b0b2efdc8d [translations] update from Weblate 
3dafa37f3 - 2024-01-10 - return42 <markus.heiser@darmarit.de>
a641db83f - 2024-01-10 - Rick1029 <yoshibear1029@gmail.com>
a3d2b083f - 2024-01-08 - johnmartzbuntia <johnmartibuntia@jmtb.xyz>
b6dec1f75 - 2024-01-08 - return42 <markus.heiser@darmarit.de>
eb76d8b8d - 2024-01-09 - Rick1029 <yoshibear1029@gmail.com>
 2024-01-13 09:33:41 +01:00
Markus Heiser
f9c5727ddc [mod] get rid of ./utils/brand.env and its workflow 
All the environments defined in ./utils/brand.env are generated on the fly, so
there is no longer a need to define the brand environment in this file and all
the workflows to handle this file.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2024-01-09 16:31:19 +01:00
Bnyro
3dea7e609b [feat] autocompleter: implementation of stract (beta) 2024-01-07 11:18:16 +01:00
Bnyro
621e1313af [feat] engine: implementation of stract (beta) 2024-01-07 11:18:16 +01:00
searxng-bot
c90dc201fc [translations] update from Weblate 
a04e3b59c - 2024-01-02 - return42 <markus.heiser@darmarit.de>
a4d387847 - 2024-01-02 - return42 <markus.heiser@darmarit.de>
80521c1b3 - 2024-01-02 - return42 <markus.heiser@darmarit.de>
e90952815 - 2024-01-02 - return42 <markus.heiser@darmarit.de>
2debd5992 - 2023-12-29 - alextecplayz <alextec70@outlook.com>
c4d54c3e5 - 2023-12-29 - alexgabi <alexgabi@disroot.org>
1a0c9c025 - 2023-12-29 - SecularSteve <fairfull.playing@gmail.com>
dc99d1390 - 2023-12-29 - ETRB <codeberg-cm58mk@r.acmrb.uk>
477e01149 - 2023-12-29 - SecularSteve <fairfull.playing@gmail.com>
65dacdd30 - 2023-12-29 - staram <gritty.year0043@fastmail.com>
6a20c8853 - 2023-12-29 - SecularSteve <fairfull.playing@gmail.com>
 2024-01-06 16:20:13 +01:00
Émilien (perso)
3535377c9a Searx to SearXNG change error message 2023-12-31 16:19:14 +01:00
searxng-bot
27e26b3d65 [translations] update from Weblate 
966bbd849 - 2023-12-29 - staram <gritty.year0043@fastmail.com>
d4680cb39 - 2023-12-29 - 1024mb <aberetta@protonmail.com>
b2cf5fbaf - 2023-12-27 - Linuks <linuks@skiff.com>
fa8654087 - 2023-12-27 - tentsbet <remendne@pentrens.jp>
fc759fe0b - 2023-12-27 - Linerly <linerly@protonmail.com>
cf9b03bfd - 2023-12-27 - return42 <markus.heiser@darmarit.de>
03bd73400 - 2023-12-27 - Linerly <linerly@protonmail.com>
88ffa01ec - 2023-12-27 - return42 <markus.heiser@darmarit.de>
736c44d3f - 2023-12-25 - err_connection_closed <err_connection_closed@users.noreply.translate.codeberg.org>
dd3a15279 - 2023-12-26 - ghose <correo@xmgz.eu>
fd68404ee - 2023-12-26 - return42 <markus.heiser@darmarit.de>
941e60232 - 2023-12-23 - SomeTr <SomeTr@users.noreply.translate.codeberg.org>
41dfd42d7 - 2023-12-23 - mittwerk <w0o0y8jt@duck.com>
1ca048b0b - 2023-12-23 - Fjuro <ifjuro@proton.me>
 2023-12-29 08:48:24 +01:00
dalf
4647981419 Update searx.data - update_wikidata_units.py 2023-12-29 06:09:04 +01:00
dalf
a8cc101c35 Update searx.data - update_ahmia_blacklist.py 2023-12-29 06:08:35 +01:00
dalf
59a67f30c8 Update searx.data - update_firefox_version.py 2023-12-29 06:07:46 +01:00
dalf
81e8638de0 Update searx.data - update_currencies.py 2023-12-29 06:06:51 +01:00
Markus Heiser
d676ae8ab3 [build] /static 2023-12-28 19:15:04 +01:00
Jinyuan Huang
f6d797ef43 [fix] fix a display problem in the image detail preview. 2023-12-28 19:15:04 +01:00
Markus Heiser
0d47322ac8 [build] /static 2023-12-28 18:59:14 +01:00
ahmad-alkadri
07e846e91a Prevent default browser behavior on click event 2023-12-28 18:59:14 +01:00
Markus Heiser
8dfc1dbc5c [fix] engine - offical URL moved to https://lingva.thedaviddelta.com/ 
The ML top-level domain has been removed from Freenom ... moved the official
instance to https://lingva.thedaviddelta.com  [1]

To test use translation syntax:

    !lv en-de Alternative front-end for Google Translate

[1] https://fosstodon.org/@thedaviddelta/111376540936289966

Closes: https://github.com/searxng/searxng/issues/2718
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-12-25 09:57:35 +01:00
Markus Heiser
2863b0f6da Update searx.data - update_engine_traits.py 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-12-24 08:28:32 +01:00
Markus Heiser
6df922b9da [fix] fetch_traits of google and zlibrary 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-12-24 08:28:32 +01:00
Bnyro
d6ac2b5a55 [feat] engine: implementation of bahnhof.de 2023-12-23 17:38:38 +01:00
Markus Heiser
6129b16165 [mod] simplify no results on pages > 1 to "There are no more results." 
BTW: remove not needed whitespaces in the generated HTML

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-12-23 09:50:39 +01:00
obfuscated-loop
2179a418c4 [mod] Change no results error message based on page number 2023-12-23 09:50:39 +01:00
Jay
b4de72a96f [mod] Capitalize theme names/styles in theme.html view file 2023-12-23 09:07:08 +01:00
dependabot[bot]
26737fda1b Bump sharp from 0.32.6 to 0.33.0 in /searx/static/themes/simple 
Bumps [sharp](https://github.com/lovell/sharp) from 0.32.6 to 0.33.0.
- [Release notes](https://github.com/lovell/sharp/releases)
- [Changelog](https://github.com/lovell/sharp/blob/main/docs/changelog.md)
- [Commits](https://github.com/lovell/sharp/compare/v0.32.6...v0.33.0)

---
updated-dependencies:
- dependency-name: sharp
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-12-23 07:49:03 +01:00
Markus Heiser
3665b32aff Revert "[fix] update user agent" 
This reverts commit 3c6549a17f.

Related:

- https://github.com/searxng/searxng/pull/2826
 2023-12-23 07:48:38 +01:00
searxng-bot
518dd97c89 [translations] update from Weblate 
fc2dfc108 - 2023-12-21 - return42 <markus.heiser@darmarit.de>
3938cfbac - 2023-12-21 - return42 <markus.heiser@darmarit.de>
7b7286fba - 2023-12-21 - return42 <markus.heiser@darmarit.de>
1d066d218 - 2023-12-20 - sriramsekar <srirams_24@yahoo.com>
 2023-12-23 07:40:02 +01:00
Bnyro
8a4104b992 [feat] engine: implementation of rotten tomatoes 2023-12-11 18:59:36 +01:00
Bnyro
dda74fc9c9 [feat] engine: implementation of mdn 2023-12-11 18:57:40 +01:00
Bnyro
b5b81c9806 [feat] engine: implementation of fyyd 2023-12-11 18:56:32 +01:00
Bnyro
527e13ab46 [feat] engine: implementation of podcastindex.org 2023-12-11 18:55:16 +01:00
Bnyro
3bc85c511c [mod] tootfinder: implement python module 2023-12-11 18:54:24 +01:00
Brock Vojkovic
0bcff32dd7 [fix] update check.searx.space IP addresses 2023-12-11 15:21:02 +01:00
searxng-bot
574ed29d3e [translations] update from Weblate 
fa2970d07 - 2023-12-08 - return42 <markus.heiser@darmarit.de>
f19144327 - 2023-12-07 - artnay <jiri.gronroos@iki.fi>
110f1b4db - 2023-12-05 - Ivan Gabaldon <admin@inetol.net>
c791e0c81 - 2023-12-07 - APoniatowski <adam@poniatowski.dev>
1f45cc83a - 2023-12-04 - akwceles <abadcopovisamuel@gmail.com>
5f93c2a39 - 2023-12-04 - return42 <markus.heiser@darmarit.de>
0d735b7aa - 2023-12-04 - return42 <markus.heiser@darmarit.de>
9cad06b09 - 2023-12-04 - akwceles <abadcopovisamuel@gmail.com>
97554eaa2 - 2023-12-03 - kirisameSoup <kirisoup@skiff.com>
2f4e19946 - 2023-12-03 - kirisameSoup <kirisoup@skiff.com>
d1dac8b5b - 2023-12-01 - Luna <github@luwuna.gay>
5c57f6eb5 - 2023-12-01 - return42 <markus.heiser@darmarit.de>
4e3908a20 - 2023-12-01 - return42 <markus.heiser@darmarit.de>
4d2b58e40 - 2023-12-01 - return42 <markus.heiser@darmarit.de>
43a40da4d - 2023-12-01 - return42 <markus.heiser@darmarit.de>
89f08502e - 2023-12-01 - return42 <markus.heiser@darmarit.de>
 2023-12-08 08:42:24 +01:00
Markus Heiser
3829c253ff [mod] add option max_page to bing, brave, qwant, startpage & mojeek 
[1] https://github.com/searxng/searxng/issues/2982#issuecomment-1808975780

Reported-by: @Damaj301damaj-lol [1]
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-12-03 13:47:17 +01:00
Markus Heiser
2274d55d5a [mod] add option max_page 
Related: https://github.com/searxng/searxng/issues/2982
Closes: https://github.com/searxng/searxng/issues/2972

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-12-03 13:47:17 +01:00
dalf
5a72977efa Update searx.data - update_currencies.py 2023-11-29 06:34:25 +01:00
dalf
b18bef17a7 Update searx.data - update_engine_descriptions.py 2023-11-29 06:24:03 +01:00
dalf
a136b0b44d Update searx.data - update_ahmia_blacklist.py 2023-11-29 06:18:21 +01:00
dalf
3ca84cb8d7 Update searx.data - update_wikidata_units.py 2023-11-29 06:17:32 +01:00
Bnyro
bf75a8c2a0 [feat] engine: implementation of bpb 2023-11-27 16:46:41 +01:00
Bnyro
04cfce2eb8 [feat] yep: support for images and news (and safesearch) 2023-11-27 15:07:26 +01:00
Markus Heiser
ee2675c999 [update] make pygments.less 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-11-27 14:43:43 +01:00
Markus Heiser
44392bd436 [mod] improve implementation of presearch engine 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-11-27 14:16:42 +01:00
Bnyro
23582aac5c [feat] implementation of presearch engine 2023-11-27 14:16:42 +01:00
obfuscated-loop
99fb565b39 [mod] Retain page numbers even when there are no results 2023-11-27 14:07:04 +01:00
Bnyro
c3cc24be12 [feat] engine: implementation of destatis 2023-11-27 13:54:48 +01:00
searxng-bot
a36eae1d68 [translations] update from Weblate 
d0511fe50 - 2023-11-22 - return42 <markus.heiser@darmarit.de>
e5d09a011 - 2023-11-21 - 0que <0que@users.noreply.translate.codeberg.org>
111634b33 - 2023-11-21 - return42 <markus.heiser@darmarit.de>
29df37609 - 2023-11-21 - return42 <markus.heiser@darmarit.de>
bb415aca1 - 2023-11-21 - RTRedreovic <RTRedreovic@users.noreply.translate.codeberg.org>
4b05dce9f - 2023-11-21 - return42 <markus.heiser@darmarit.de>
2f730019b - 2023-11-20 - return42 <markus.heiser@darmarit.de>
389a46a05 - 2023-11-19 - alextecplayz <alextec70@outlook.com>
2620f6911 - 2023-11-20 - return42 <markus.heiser@darmarit.de>
716d7001d - 2023-11-20 - return42 <markus.heiser@darmarit.de>
30f17f24c - 2023-11-20 - return42 <markus.heiser@darmarit.de>
b7c4062e2 - 2023-11-19 - RTRedreovic <RTRedreovic@users.noreply.translate.codeberg.org>
7ba200bc5 - 2023-11-18 - Azharjan <alexander.um.edu@gmail.com>
 2023-11-27 07:32:05 +01:00
obfuscated-loop
cc6058057d [build] /static 2023-11-23 07:30:56 +01:00
obfuscated-loop
355d2dc678 [fix] Remove anchor tag outline in image results 2023-11-23 07:30:56 +01:00
searxng-bot
e99ebb3dec [translations] update from Weblate 
e245d1bd0 - 2023-11-16 - return42 <markus.heiser@darmarit.de>
8ee33fca3 - 2023-11-16 - return42 <markus.heiser@darmarit.de>
1c8c1557c - 2023-11-16 - return42 <markus.heiser@darmarit.de>
925c63f41 - 2023-11-16 - return42 <markus.heiser@darmarit.de>
15908183b - 2023-11-14 - return42 <markus.heiser@darmarit.de>
dd57115ef - 2023-11-12 - return42 <markus.heiser@darmarit.de>
9db7544d8 - 2023-11-11 - SomeTr <SomeTr@users.noreply.translate.codeberg.org>
23ce4d1f2 - 2023-11-12 - return42 <markus.heiser@darmarit.de>
902b8bafd - 2023-11-13 - tentsbet <remendne@pentrens.jp>
2350f4a05 - 2023-11-13 - ghose <correo@xmgz.eu>
10e6293fd - 2023-11-11 - gallegonovato <fran-carro@hotmail.es>
477881443 - 2023-11-13 - return42 <markus.heiser@darmarit.de>
556b2f611 - 2023-11-13 - Fjuro <ifjuro@proton.me>
3133818f8 - 2023-11-12 - return42 <markus.heiser@darmarit.de>
 2023-11-22 09:43:45 +01:00
Jinyuan Huang
3601badd84 [build] /static 2023-11-14 13:34:44 +01:00
Jinyuan Huang
903449e0b5 [fix] Images cannot be clicked anymore #3006 
URL in sidedbar only exists in HTTP POST requests.  On HTTP GET requests the
selector ``#search_url button#copy_url`` results in a ``null`` type and a
``.style.display`` raises::

   Uncaught TypeError: d.querySelector(...) is null

As a result, the initialization of the event handler is no longer carried out.

Closes: https://github.com/searxng/searxng/issues/3003
Suggested-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-11-14 13:28:50 +01:00
searxng-bot
79db70f2b7 [translations] update from Weblate 
2b7f195db - 2023-11-09 - proton-penguin <cgshyuankai@proton.me>
411951c72 - 2023-11-09 - SomeTr <SomeTr@users.noreply.translate.codeberg.org>
878199f48 - 2023-11-06 - RTRedreovic <RTRedreovic@users.noreply.translate.codeberg.org>
512f9981a - 2023-11-05 - return42 <markus.heiser@darmarit.de>
adcaeb0c6 - 2023-11-05 - return42 <markus.heiser@darmarit.de>
acc01d46d - 2023-11-05 - return42 <markus.heiser@darmarit.de>
32f1dd13c - 2023-11-05 - RTRedreovic <RTRedreovic@users.noreply.translate.codeberg.org>
aafe161aa - 2023-11-03 - quenty_occitania <quentinantonin@free.fr>
c910601c9 - 2023-11-03 - return42 <markus.heiser@darmarit.de>
 2023-11-12 20:54:35 +01:00
Jinyuan Huang
727de1c4ef [build] /static 2023-11-10 13:28:28 +00:00
Jinyuan Huang
8adc7b0433 add a "copy" button to the "search-url" sidebar 2023-11-10 13:28:04 +00:00
searxng-bot
96ccec1aa6 [translations] update from Weblate 
acdcdace8 - 2023-11-01 - Ivan Gabaldon <admin@inetol.net>
31a1602a9 - 2023-10-29 - return42 <markus.heiser@darmarit.de>
a1d1070d3 - 2023-10-29 - return42 <markus.heiser@darmarit.de>
 2023-11-03 10:01:28 +01:00
Markus Heiser
b5a8ddfec8
Revert "[fix] limit maximum page number of a search query to page 50." (#2979) 
This reverts commit 7e2e335dd1.
 2023-11-01 16:55:20 +00:00
Markus Heiser
d13a8f6453 [mod] document server:public_instance & remove it out of the botdetection 
- the option server:public_instance lacks some documentation
- the processing of this option belongs in the limiter and not
  in botdetection module

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-11-01 06:44:56 +01:00
Markus Heiser
fd814aac86 [mod] isolation of botdetection from the limiter 
This patch was inspired by the discussion around PR-2882 [2].  The goals of this
patch are:

1. Convert plugin searx.plugin.limiter to normal code [1]
2. isolation of botdetection from the limiter [2]
3. searx/{tools => botdetection}/config.py and drop searx.tools
4. in URL /config, 'limiter.enabled' is true only if the limiter is really
   enabled (Redis is available).

This patch moves all the code that belongs to botdetection into namespace
searx.botdetection and code that belongs to limiter is placed in namespace
searx.limiter.

Tthe limiter used to be a plugin at some point botdetection was added, it was
not a plugin.  The modularization of these two components was long overdue.
With the clear modularization, the documentation could then also be organized
according to the architecture.

[1] https://github.com/searxng/searxng/pull/2882
[2] https://github.com/searxng/searxng/pull/2882#issuecomment-1741716891

To test:

- check the app works without the limiter, check `/config`
- check the app works with the limiter and with the token, check `/config`
- make docs.live .. and read
  - http://0.0.0.0:8000/admin/searx.limiter.html
  - http://0.0.0.0:8000/src/searx.botdetection.html#botdetection

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-11-01 06:44:56 +01:00
sev
b05a15540e Check public_instance in simple theme 
Fix #2975
 2023-10-31 10:40:19 +01:00
Markus Heiser
7e2e335dd1 [fix] limit maximum page number of a search query to page 50. 
To test this PR run a local instance and try to query page 51:

    http://127.0.0.1:8888/search?q=foo&pageno=51

A parameter exception will be raised:

    searx.exceptions.SearxParameterException: Invalid value "51" for parameter pageno

And the client will receive a HTTP 400 (Bad request).

Closes https://github.com/searxng/searxng/issues/2972

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-10-29 16:17:25 +01:00
dalf
b63841f0a1 Update searx.data - update_engine_traits.py 2023-10-29 09:04:24 +01:00
dalf
a9d778ee0f Update searx.data - update_wikidata_units.py 2023-10-29 08:45:15 +01:00
dalf
92efb941d8 Update searx.data - update_firefox_version.py 2023-10-29 08:44:59 +01:00
dalf
fa494b9be3 Update searx.data - update_currencies.py 2023-10-29 08:44:35 +01:00
dalf
1b92ea4e70 Update searx.data - update_engine_descriptions.py 2023-10-29 08:44:17 +01:00
dalf
55e7cb3e33 Update searx.data - update_ahmia_blacklist.py 2023-10-29 08:17:29 +01:00
searxng-bot
6af8644ead [translations] update from Weblate 
4e5e5db44 - 2023-10-26 - quenty_occitania <quentinantonin@free.fr>
e1a8d3508 - 2023-10-26 - quenty_occitania <quentinantonin@free.fr>
84bddfb89 - 2023-10-26 - return42 <markus.heiser@darmarit.de>
d67a4114d - 2023-10-26 - quenty_occitania <quentinantonin@free.fr>
62fe8e328 - 2023-10-26 - return42 <markus.heiser@darmarit.de>
6e37ab975 - 2023-10-26 - quenty_occitania <quentinantonin@free.fr>
2cdab3247 - 2023-10-25 - SomeTr <SomeTr@users.noreply.translate.codeberg.org>
cf7ea7234 - 2023-10-23 - clsty <celestial.y@outlook.com>
0ea313893 - 2023-10-23 - return42 <markus.heiser@darmarit.de>
22151e440 - 2023-10-23 - return42 <markus.heiser@darmarit.de>
e4eaf42b6 - 2023-10-22 - clsty <celestial.y@outlook.com>
 2023-10-27 09:23:23 +02:00
Markus Heiser
ef56e1d684 [fix] HTMLParser: undocumented not implemented method 
In python versions <py3.10 there is an issue with an undocumented method
HTMLParser.error() [1][2] that was deprecated in Python 3.4 and removed
in Python 3.5.

To be compatible to higher versions (>=py3.10) an error method is implemented
which throws an AssertionError exception like the higher Python versions do [3].

[1] https://github.com/python/cpython/issues/76025
[2] https://bugs.python.org/issue31844
[3] https://github.com/python/cpython/pull/8562

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-10-22 10:35:02 +02:00
searxng-bot
01b5b9cb8e [translations] update from Weblate 
2325f1583 - 2023-10-20 - return42 <markus.heiser@darmarit.de>
5090c6a8e - 2023-10-18 - return42 <markus.heiser@darmarit.de>
3a38219d8 - 2023-10-18 - return42 <markus.heiser@darmarit.de>
94a9f4164 - 2023-10-16 - return42 <markus.heiser@darmarit.de>
bdbeb4b30 - 2023-10-16 - return42 <markus.heiser@darmarit.de>
f9b483f48 - 2023-10-16 - return42 <markus.heiser@darmarit.de>
7f1ca1997 - 2023-10-16 - return42 <markus.heiser@darmarit.de>
c5a701dc4 - 2023-10-14 - alexgabi <alexgabi@disroot.org>
 2023-10-20 10:44:11 +02:00
Markus Heiser
b013cbb4cc [build] /static 2023-10-13 12:56:29 +02:00
Bnyro
df2e41c150 [fix] search.js: crash on homepage when setting form listeners 2023-10-13 12:56:29 +02:00
Émilien (perso)
4280318fc5 fixing results parsing brave 2023-10-13 11:47:30 +02:00
searxng-bot
ac82daf58d [translations] update from Weblate 
74e401e68 - 2023-10-09 - return42 <markus.heiser@darmarit.de>
897dd8db1 - 2023-10-09 - return42 <markus.heiser@darmarit.de>
6ed046a90 - 2023-10-09 - tentsbet <remendne@pentrens.jp>
815ecb336 - 2023-10-09 - return42 <markus.heiser@darmarit.de>
65d9a0c2f - 2023-10-07 - return42 <markus.heiser@darmarit.de>
3ec249ef9 - 2023-10-07 - return42 <markus.heiser@darmarit.de>
53dc6c108 - 2023-10-07 - return42 <markus.heiser@darmarit.de>
 2023-10-13 09:20:54 +02:00
Hackurei
efada7cba2 [fix] hackernews keyerror problem 2023-10-13 08:16:47 +02:00
Hackurei
af071121de [fix] imgur - incorrect wikidata id 2023-10-12 09:14:00 +02:00
Markus Heiser
14323d683f [fix] ddg-lite & ddg-extra: don't send empty vqd value 
DDG's bot detection is sensitive to the vqd value.  For some search terms (such
as extremely long search terms that are often sent by bots), no vqd value can be
determined.

If SearXNG cannot determine a vqd value, then no request should go out to
DDG (WEB): a request with a wrong vqd value leads to DDG temporarily putting
SearXNG's IP on a block list.

Requests from IPs in this block list run into timeouts.

Not sure, but it seems the block list is a sliding window: to get my IP rid from
the bot list I had to cool down my IP for 1h (send no requests from that IP to
DDG).

Since such issues can't reproduce in a local instance I tested this patch 24h on
my public SearXNG instance: There are still errors (rare), but the reliability
is still 100%.

Related:

- https://github.com/searxng/searxng/pull/2922
- https://github.com/searxng/searxng/pull/2923

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-10-12 08:52:28 +02:00
Markus Heiser
3388441917 [fix] ddg-lite vqd value: some search terms do not have a vqd value 
Some search terms do not have results and therefore no vqd value

BTW: remove a leftover from 9197efa

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-10-10 09:12:30 +02:00
Markus Heiser
9197efa2a7 [fix] duckduckgo lite engine: set HTTP header 'Referer' 
We have had problems with this before, the bot protection from ddg-lite seems to
have included this referer in the rating [1][2].

From reverse engineering:

- The Referer ``https://google.com/`` was set in commt 257dc7d6c4 --> DDG lite
  does not like this referer anymore!

- The 'Referer' header is only set on second and follow up pages but not on the
  first page

- The vqd value is not needed on the first page, the ddg-lite client sets this
  value only on follow up pages / this can help to reduce the vqd requests from
  SearXNG.

Related to 'Referer' header & ddg requests:

[1] https://github.com/searxng/searxng/pull/2161
[2] https://github.com/searxng/searxng/pull/2081

Closes: https://github.com/searxng/searxng/issues/2796
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-10-10 08:40:53 +02:00
Bnyro
fa5b2a7948 [mod] yacy: use official instance by default and fix crashes 2023-10-09 20:50:24 +02:00
Alex Balgavy
5d53aac20f [mod] add hotkeys option to settings.yml 
The change in the hotkey mechanism introduced in 317db5b04 does not allow
configuration via `settings.yml`.  This commit adds that functionality.

Closes: #2898
 2023-10-09 18:13:00 +02:00
Hackurei
ff78b1a902 [feat] implement hackernews engine - news.ycombinator.com 2023-10-09 14:00:04 +02:00
Aine
213cb74378 [fix] matrixrooms add proper MRS integration 
Related:

- https://github.com/searxng/searxng/issues/2918
 2023-10-09 13:25:13 +02:00
Bnyro
48cb58bd2e [feat] duckduckgo: support for videos and news 2023-10-09 06:53:43 +02:00
Bnyro
c3ab49cd90 [fix] kickass: crash when no results 2023-10-07 11:48:23 +02:00
Bnyro
f22daf8b47 [mod] piped: always show video length if available 2023-10-07 11:45:46 +02:00
Bnyro
ce270961e8 [feat] engine: implementation of mastodon 2023-10-06 10:58:23 +02:00
searxng-bot
48f8ac48d1 [translations] update from Weblate 
68d743281 - 2023-10-05 - return42 <markus.heiser@darmarit.de>
42f091b7f - 2023-10-05 - return42 <markus.heiser@darmarit.de>
2479c0d7b - 2023-10-05 - ghose <correo@xmgz.eu>
a4e6cd592 - 2023-10-05 - return42 <markus.heiser@darmarit.de>
9d4e5f5c3 - 2023-10-05 - return42 <markus.heiser@darmarit.de>
b79d44775 - 2023-10-05 - gallegonovato <fran-carro@hotmail.es>
746291184 - 2023-10-06 - return42 <markus.heiser@darmarit.de>
f24d7e8b1 - 2023-10-05 - return42 <markus.heiser@darmarit.de>
6140911f9 - 2023-10-05 - Fjuro <ifjuro@proton.me>
 2023-10-06 09:19:00 +02:00
Markus Heiser
fd1422a670 [mod] engine - simplify region & lang handling, make filters configurable 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-10-05 10:55:08 +02:00
Bnyro
3e2ae756f0 [feat] engine: implementation of radio-browser.info 2023-10-05 10:55:08 +02:00
Markus Heiser
d56b31d674 [fix] limiter / botdetection: remove http_connection method 
Related:

- https://github.com/searxng/searxng/issues/2892#issuecomment-1742153932

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-10-01 20:24:22 +02:00
Jinyuan Huang
e509cb7c45 [typo] solved a typo in yahoo error message. 2023-10-01 08:29:06 +02:00
Jinyuan Huang
d4d9f2073e [fix] Bug: Yahoo results for simplified Chinese search sometimes have the first character cut off #2866 
Co-authored-by: Blair Noctis <n@sail.ng>
 2023-10-01 08:29:06 +02:00
Bnyro
fe9386b58d [fix] emojipedia: fix engine 2023-10-01 08:19:45 +02:00
Markus Heiser
32a4ea350e [fix] Revision of the Bing engines 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-10-01 08:01:38 +02:00
jazzzooo
079636c079 [fix] engine - bing fix search, pagination, remove safesearch 2023-10-01 08:01:38 +02:00
Bnyro
5ce1792432 [feat] engine: implementation of pinterest 2023-09-30 15:01:45 +02:00
Bnyro
6096457e4d [fix] matrixrooms.info: pagination not working properly 2023-09-30 14:51:07 +02:00
Markus Heiser
e1a8b8189f [fix] engine - moviepilot instead of thumbnail use img_src 
Instead of thumbnail use img_src in the result item, otherwise the "movies"
categories looks clunky.

Related:

- b4e0d2eedc (r128785388)

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-09-30 11:29:19 +02:00
Bnyro
159629c588 [mod] tagesschau: add option to only use tagesschau urls 2023-09-30 11:00:11 +02:00
Bnyro
2ca60a19fc [feat] engine: implementation of matrixrooms.info 2023-09-30 09:09:23 +02:00
Bnyro
d1a3fc5be6 [feat] engine: implementation of tootfinder 2023-09-29 23:00:30 +02:00
Bnyro
fc4a20f734 [mod] add movies category for tmdb, imdb and moviepilot 2023-09-29 22:37:51 +02:00
Émilien (perso)
0330648b98 [mod] warn to info for failed ping limiter 2023-09-29 22:20:56 +02:00
Bnyro
26fed56d51 [mod] settings.yml: remove plugin settings for plugins that don't exist anymore 2023-09-29 11:26:49 +02:00
searxng-bot
9a065832e0 [translations] update from Weblate 
80bbf74ef - 2023-09-27 - return42 <markus.heiser@darmarit.de>
5d554ef71 - 2023-09-27 - BernieHuang2008 <berniehuang2008@163.com>
771fd1b90 - 2023-09-27 - return42 <markus.heiser@darmarit.de>
d412012c8 - 2023-09-27 - y0nei <y0nei@proton.me>
1ac171d0a - 2023-09-27 - return42 <markus.heiser@darmarit.de>
a004c40cc - 2023-09-27 - return42 <markus.heiser@darmarit.de>
47e729f89 - 2023-09-27 - return42 <markus.heiser@darmarit.de>
f85b28cc4 - 2023-09-26 - Fjuro <ifjuro@proton.me>
75ce5d581 - 2023-09-27 - return42 <markus.heiser@darmarit.de>
40ce06009 - 2023-09-24 - nonepork <ptohrek@gmail.com>
21f343f71 - 2023-09-25 - tentsbet <remendne@pentrens.jp>
4c80f80fd - 2023-09-22 - return42 <markus.heiser@darmarit.de>
ef3c859e4 - 2023-09-22 - artnay <jiri.gronroos@iki.fi>
 2023-09-29 09:12:12 +02:00
dalf
ae2762eb78 Update searx.data - update_firefox_version.py 2023-09-29 06:58:11 +02:00
dalf
18da3d65c5 Update searx.data - update_ahmia_blacklist.py 2023-09-29 06:57:40 +02:00
dalf
4a66fa3e7d Update searx.data - update_wikidata_units.py 2023-09-29 06:56:50 +02:00
dalf
9be6e20256 Update searx.data - update_engine_traits.py 2023-09-29 06:56:10 +02:00
dalf
d821ec44dd Update searx.data - update_currencies.py 2023-09-29 06:55:04 +02:00
dalf
cdee5aa3b9 Update searx.data - update_engine_descriptions.py 2023-09-29 06:53:05 +02:00
Bnyro
75587d03a6 [fix] categories.html: can't select social media category using search on category select 2023-09-28 10:45:49 +02:00
jazzzooo
e37d775fa2 [fix] engine - currency fix and simplify 2023-09-28 08:29:38 +02:00
Jinyuan Huang
ae28d429c9 [fix] bilibili new api used 2023-09-28 08:24:51 +02:00
jazzzooo
1a66d74673 [fix] engine - kickass update url, fix parsing, use multiple mirrors 2023-09-27 10:19:41 +02:00
Markus Heiser
0c39064d60 Update searx.data - update_engine_traits.py 
Update searx.data - update_engine_traits.py

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-09-27 09:12:38 +02:00
Markus Heiser
b428ccc5a0 [fix] engine brave - fetch traits (modified settings menu) 
Brave has changed it settings menu fundamental.  Region codes are no longer in
the HTML page, we have to read the regional codes from a JS:

  https://cdn.search.brave.com/serp/v2/_app/immutable/chunks/parameters.734c106a.js

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-09-27 09:12:38 +02:00
Markus Heiser
3a456b1282 [fix] engine annas archive - fetch traits (modified xpath selectors) 
Anna’s Archive has cleaned up their languages, available file extensions and
changed the HTML form.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-09-27 09:12:38 +02:00
Jinyuan Huang
1f835a9f1f solve #2839 2023-09-25 23:01:14 +02:00
jazzzooo
3c6549a17f [fix] update user agent 2023-09-25 22:46:22 +02:00
Emilien Devos
ffec8d1b4c add warning about enabling public_instance setting 2023-09-25 22:31:14 +02:00
Emilien Devos
47721a3485 add new parameter called server.public_instance 
for enabling by default advanced limiter functions
in the future allow us to add features just for the public instances
 2023-09-25 22:31:14 +02:00
Jinyuan Huang
1df4588279 [build] /static 2023-09-25 17:45:21 +02:00
Jinyuan Huang
98366bcf90 [fix] settings icon invisble on mobile devices 2023-09-25 17:45:21 +02:00
Alexandre Flament
19383921d7 /config: expose limiter configuration 2023-09-24 11:01:11 +02:00
Émilien (perso)
1851f27154
[mod] remove twitter (#2843) 2023-09-24 08:32:53 +00:00
Markus Heiser
597c68b4aa [doc] move dosc of botdetection from developer to admin section 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-09-23 14:00:03 +02:00
Bnyro
75c9de02d1 [feat] engine: implementation of imgur 2023-09-22 20:50:53 +02:00
Bnyro
fb72f71f0a [fix] internet archive scholar: crash when there's no title 2023-09-22 18:49:39 +02:00
searxng-bot
0660d20e1d [translations] update from Weblate 
f27a3bbd4 - 2023-09-21 - return42 <markus.heiser@darmarit.de>
a97336d0a - 2023-09-21 - return42 <markus.heiser@darmarit.de>
91e8946b3 - 2023-09-19 - return42 <markus.heiser@darmarit.de>
19f53731e - 2023-09-20 - ghose <correo@xmgz.eu>
daf10bd75 - 2023-09-19 - gallegonovato <fran-carro@hotmail.es>
ab004b8b8 - 2023-09-19 - return42 <markus.heiser@darmarit.de>
cdedd99c9 - 2023-09-19 - return42 <markus.heiser@darmarit.de>
 2023-09-22 18:05:02 +02:00
Bnyro
95885e7f9f [build] /static 2023-09-22 17:54:55 +02:00
Bnyro
8b0b6731fc [fix] search.less: start page search view on mobile devices 2023-09-22 17:54:55 +02:00
Markus Heiser
71358e9c67 Revert "[fix] engine - duckduckgo vqd edge-case" 
This reverts commit 102502a4f0.
 2023-09-22 09:31:25 +02:00
Bnyro
51236ae47a [feat] engine: implementation of chefkoch.de 2023-09-21 17:23:59 +02:00
jazzzooo
8bcca0e620 [fix] engine - brave don't show ads 2023-09-21 16:55:39 +02:00
jazzzooo
b729542a66 [fix] engine - google images error when no results 2023-09-21 16:38:37 +02:00
Bnyro
f0df53673e [mod] search-syntax.md: add tutorial about the feeling lucky feature 2023-09-21 15:17:23 +02:00
Bnyro
cc2e0537a3 [feat] engine: implementation of google icons/material design icons 2023-09-21 15:16:49 +02:00
Bnyro
c999cfb422 [feat] engine: implementation of wallhaven 2023-09-21 14:25:43 +02:00
jazzzooo
102502a4f0 [fix] engine - duckduckgo vqd edge-case 2023-09-20 20:05:06 +02:00
Markus Heiser
d013f51a25 [fix] Search on category select without JavaScript PR #2740 
Small addendum to #2740; search_on_category_select is now no longer a plugin.

Related: https://github.com/searxng/searxng/pull/2740#issuecomment-1725437584
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-09-19 21:03:02 +02:00
Markus Heiser
0ce10046c4 [build] /static 2023-09-19 20:15:51 +02:00
ormai
a76c8e4101 Improve the layout of the search page on mobile 2023-09-19 20:15:51 +02:00
Émilien (perso)
33496e7f51 Switch back to qwant web normal API 2023-09-19 17:07:22 +02:00
Markus Heiser
043dcbf7c5 [fix] engine qwant (web-lite) - ignore advertising adds 
Closes: https://github.com/searxng/searxng/issues/2812
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-09-19 17:06:56 +02:00
Markus Heiser
3ac7c40b6a [build] /static 2023-09-19 11:15:25 +02:00
Emilien Devos
33e722f83b better error message when no results found 2023-09-19 11:10:48 +02:00
Émilien (perso)
ad725ce7d7
wikipedia wikidata infobox + disable wikisource (#2806) 
Co-authored-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-09-19 10:31:02 +02:00
Bnyro
dcee823345 [feat] implement feeling lucky feature 2023-09-19 09:40:57 +02:00
Markus Heiser
71508abcbf [build] /static 2023-09-19 08:06:06 +02:00
Jinyuan Huang
2ec77ef813 [mod] reformatted keyBindingLayouts in JS client (default, vim) 2023-09-19 08:06:06 +02:00
Markus Heiser
317db5b04f [mod] preferences: implement drop-down menu for hotkeys (default, vim) 
Replace the on/off checkbox of the vim-hotkeys in the preferences by a drop-down
menu.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-09-19 08:06:06 +02:00
Markus Heiser
09935e2897 [build] /static 2023-09-18 21:29:11 +02:00
Bnyro
a55e0ac553 [feat] search on category select without JS 
Co-authored-by: Alexandre Flament <alex@al-f.net>
 2023-09-18 21:29:11 +02:00
Bnyro
efd3a2d6d1 [feat] engine: implementation of internet archive scholar 2023-09-18 18:12:00 +02:00
jazzzooo
223b3487c3 [fix] spelling 2023-09-18 16:20:27 +02:00
Markus Heiser
a9b6963971 [fix] engine - qwant delivers only 5 pages maximum 
all qwant engines (incl qwant-lite) delivers only 5 pages maximum

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-09-18 01:43:43 +02:00
jazzzooo
da1446c5ed [fix] engine - qwant wrong error type 2023-09-18 01:43:43 +02:00
Markus Heiser
7398d525c8 [fix] qwant: subsequent fix of commit d9dbcedeb 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-09-17 17:52:56 +02:00
Markus Heiser
d9dbcedeb6 [feat] implementation of qwant lite for web search 
Related: https://github.com/searxng/searxng/issues/2719
Replace: https://github.com/searxng/searxng/pull/2748
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-09-17 16:53:25 +02:00
Bnyro
b4e0d2eedc [feat] engine: implemenation of moviepilot (de) 2023-09-17 14:30:56 +02:00
jazzzooo
7dfcc3386e [fix] tagesschau videos 2023-09-16 18:40:26 +02:00
Markus Heiser
89fbac5763 [build] /static 2023-09-16 13:45:43 +02:00
Jinyuan Huang
92d0c378e0 [fix] keyboard navigation / simple theme (UI) 
- avoid loop select
- fix select next item in mixed result lists

Replaces: https://github.com/searxng/searxng/pull/2789
Closes: https://github.com/searxng/searxng/issues/2751
Closes: https://github.com/searxng/searxng/issues/2788
 2023-09-16 13:45:43 +02:00
jazzzooo
ec540a967a [fix] brave.videos 2023-09-15 22:00:09 +02:00
jazzzooo
27477f51fd [fix] brave.news 2023-09-15 22:00:09 +02:00
Bnyro
140125e641 [feat] implementation of the movie database 2023-09-15 19:41:37 +02:00
searxng-bot
4eebb25882 [translations] update from Weblate 
4f055ccb8 - 2023-09-11 - return42 <markus.heiser@darmarit.de>
9edaacb09 - 2023-09-11 - return42 <markus.heiser@darmarit.de>
ccb3e8934 - 2023-09-11 - return42 <markus.heiser@darmarit.de>
206640415 - 2023-09-11 - return42 <markus.heiser@darmarit.de>
d8db608c2 - 2023-09-11 - return42 <markus.heiser@darmarit.de>
6db95be68 - 2023-09-11 - return42 <markus.heiser@darmarit.de>
0792cd970 - 2023-09-11 - return42 <markus.heiser@darmarit.de>
949ad5db8 - 2023-09-11 - return42 <markus.heiser@darmarit.de>
0921f948e - 2023-09-11 - return42 <markus.heiser@darmarit.de>
3677f0ccb - 2023-09-11 - return42 <markus.heiser@darmarit.de>
2dea9cd3a - 2023-09-11 - return42 <markus.heiser@darmarit.de>
5038588e8 - 2023-09-11 - return42 <markus.heiser@darmarit.de>
6b63e980b - 2023-09-11 - return42 <markus.heiser@darmarit.de>
858a8660b - 2023-09-11 - return42 <markus.heiser@darmarit.de>
5852633d2 - 2023-09-11 - return42 <markus.heiser@darmarit.de>
92d25097b - 2023-09-11 - return42 <markus.heiser@darmarit.de>
8ae9d7a20 - 2023-09-11 - return42 <markus.heiser@darmarit.de>
989ccd21a - 2023-09-11 - return42 <markus.heiser@darmarit.de>
1c309f3fc - 2023-09-11 - return42 <markus.heiser@darmarit.de>
3ee5e679e - 2023-09-11 - return42 <markus.heiser@darmarit.de>
7420db0cf - 2023-09-11 - return42 <markus.heiser@darmarit.de>
7327ffb1f - 2023-09-11 - return42 <markus.heiser@darmarit.de>
542814e22 - 2023-09-11 - return42 <markus.heiser@darmarit.de>
5f2623600 - 2023-09-11 - return42 <markus.heiser@darmarit.de>
7bea3da45 - 2023-09-11 - return42 <markus.heiser@darmarit.de>
68a854d93 - 2023-09-11 - ghose <correo@xmgz.eu>
1781efe60 - 2023-09-11 - return42 <markus.heiser@darmarit.de>
016a60442 - 2023-09-11 - return42 <markus.heiser@darmarit.de>
00f143119 - 2023-09-11 - return42 <markus.heiser@darmarit.de>
8e61bb7ac - 2023-09-11 - return42 <markus.heiser@darmarit.de>
d04327a41 - 2023-09-11 - return42 <markus.heiser@darmarit.de>
841227415 - 2023-09-11 - return42 <markus.heiser@darmarit.de>
9ed21b385 - 2023-09-11 - return42 <markus.heiser@darmarit.de>
914b774da - 2023-09-11 - return42 <markus.heiser@darmarit.de>
ff7b45ec8 - 2023-09-11 - return42 <markus.heiser@darmarit.de>
4146f8707 - 2023-09-11 - return42 <markus.heiser@darmarit.de>
383602051 - 2023-09-11 - return42 <markus.heiser@darmarit.de>
b050d9ee4 - 2023-09-11 - return42 <markus.heiser@darmarit.de>
8d6e7a20c - 2023-09-11 - return42 <markus.heiser@darmarit.de>
ee77e2072 - 2023-09-11 - return42 <markus.heiser@darmarit.de>
0420f44b6 - 2023-09-11 - return42 <markus.heiser@darmarit.de>
 2023-09-15 09:12:13 +02:00
Justas Zabulionis
41ef73ca3a [fix] rumble redirect 2023-09-14 19:21:21 +02:00
Justas Zabulionis
b701ee257b [fix] habrahabr 2023-09-14 18:43:14 +02:00
Justas Zabulionis
be888810ba [fix] pubmed content being None 2023-09-14 18:40:15 +02:00
Justas Zabulionis
cf734e6008 [fix] engine - pub.dev 2023-09-14 18:07:55 +02:00
Justas Zabulionis
92d39de410 [fix] solidtorrents redirects 2023-09-14 18:03:21 +02:00
Justas Zabulionis
cf8a6cf6db [fix] solidtorrents pagination 2023-09-14 18:03:21 +02:00
Justas Zabulionis
8172f89075 [fix] solidtorrents 2023-09-14 18:03:21 +02:00
jazzzooo
74600c028d [fix] engine - Crossref 
Crossref was broken on result types journal-issue and component .. The old code
had lots of assumptions, and broke during parsing.  Now the assumptions are more
explicit and checked them with the API.
 2023-09-14 17:39:23 +02:00
Markus Heiser
ed6a5a01bb [build] /static 2023-09-14 16:26:10 +02:00
Jinyuan Huang
0984b00b7e [mod] UI smooth scroll added / CSS scroll-behavior 
This PR improves the UX by making auto-scroll more smoother.  The css is changed
so all the auto-scroll will be smoother but User-scroll will not be influenced.

  The scroll-behavior CSS property sets the behavior for a scrolling box when
  scrolling is triggered by the navigation or CSSOM scrolling APIs.[1]

[1] https://developer.mozilla.org/en-US/docs/Web/CSS/scroll-behavior
 2023-09-14 16:26:10 +02:00
Bnyro
bf7af99ef2 [mod] make data.traits: add Odysee 2023-09-13 21:41:33 +02:00
Bnyro
3568a3cafb [feat] odysee: implement fetch_traits for language support 2023-09-13 21:41:33 +02:00
Bnyro
09c61dabc9 [mod] odysee: time range support 2023-09-13 21:41:33 +02:00
jazzzooo
b98907e91f [fix] engine - piped.music incorrect timestamps 2023-09-13 21:39:37 +02:00
jazzzooo
6039dbf211 [fix] engine - invidious thumbnails 2023-09-13 11:37:42 +02:00
jazzzooo
ed3819bdd5 add inv.tux.pizza 2023-09-13 11:12:01 +02:00
jazzzooo
122b4342eb remove invidious.snopyta.org 2023-09-13 11:12:01 +02:00
jazzzooo
e2a5e183cc [fix] settings.yml - change default invidious instances 2023-09-13 11:12:01 +02:00
jazzzooo
b2fd6304bf [fix] engine - openstreetmap currency rendering 2023-09-13 10:56:52 +02:00
jazzzooo
976b3e21c4 [fix] engine - pypi redirect 2023-09-12 21:14:48 +02:00
jazzzooo
54a3e03b45 [fix] engine - openstreetmap currency matching 2023-09-12 20:57:05 +02:00
Bnyro
64d9587ac8 [feat] new engine: svgrepo 2023-09-12 20:38:36 +02:00
Markus Heiser
aa1453db0c [build] /static 2023-09-12 20:30:50 +02:00
Markus Heiser
a987672340 [mod] improve handling of the hotkeys 
- KeyboardEvent: keyCode property is depricated, replaced by key property [2]

- the check for ifDetailOpened is not necessary, because the hotkeys are not
  only applicable to image-results, by example:

   `!goi !go !scc hello`

- Key bindings like h for help are to be used in general (not only in vim-mode)

[1] https://developer.mozilla.org/en-US/docs/Web/API/KeyboardEvent/keyCode
[2] https://developer.mozilla.org/en-US/docs/Web/API/KeyboardEvent/key

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-09-12 20:30:50 +02:00
Jinyuan Huang
e6b160da62 [feature] key bindings: left & right arrow are always on 2023-09-12 20:30:50 +02:00
jazzzooo
b189578b6b [fix] engine - brave 2023-09-12 11:31:43 +02:00
Bnyro
f182abd6f8 [mod] library of congress: fix engine 2023-09-11 19:42:31 +02:00
Bnyro
e73a6f5d14 [fix] engine deviantart: review of the result-scrapper 
The deviantart site changed and hence deviantart is currently unusable.
 2023-09-11 13:22:36 +02:00
Markus Heiser
1f7366060e [build] /static 2023-09-11 12:27:56 +02:00
Markus Heiser
935aed7ca4 [feature] dark theme for code highlighter in the result list 
Closes: https://github.com/searxng/searxng/issues/1354

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-09-11 12:27:56 +02:00
Alexandre Flament
d07c006aed Replace chompjs with pure Python code 
The new implementation is good enough for the current usage (brave)
 2023-09-09 13:02:36 +02:00
Markus Heiser
8e45ac4271 [build] /static 2023-09-09 00:39:51 +02:00
Markus Heiser
15a3dbf7a0 [fix] preferences - layout of the "special queries" tab 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-09-09 00:39:51 +02:00
Bnyro
9e83c0dedc [feat] engine: implementation of Yummly 
Co-authored-by: Markus Heiser <markus.heiser@damarit.de>
 2023-09-08 11:47:13 +02:00
Bnyro
a3d7e9c285 [mod] utils.py: add markdown_to_text helper function 2023-09-08 11:47:13 +02:00
searxng-bot
c62605b3e3 [translations] update from Weblate 
304f6af5c - 2023-09-04 - return42 <markus.heiser@darmarit.de>
76a4b6496 - 2023-09-03 - alexgabi <alexgabi@disroot.org>
 2023-09-08 09:20:19 +02:00
Markus Heiser
019bd97f4b [fix] settings.yml - fix docs-links in the comments 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-09-06 11:35:39 +02:00
Hackurei
1f21ac7d62 [feat] engine: implementation of bilibili https://www.bilibili.com 2023-09-05 22:53:03 +02:00
Markus Heiser
696c35d2c3 [fix] engine - duckduckgo_images / determination of vqd value incorrect 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-09-05 22:24:51 +02:00
bonswouar
4fb26cd96d [fix] engine duckduckgo weather api changes 2023-09-05 16:55:00 +02:00
Markus Heiser
01be9e0e20 [fix] engine: wikicommons - don't quoute ':|' in URL parameters 
From [1]: It seems to be because of [2] For some reason it gets url encoded
twice, resulting in

- ``filetype%253Abitmap%257Cdrawing+birds`` instead of
- ``filetype:bitmap%7Cdrawing+birds``

[1] https://github.com/searxng/searxng/issues/2707
[2] https://github.com/searxng/searxng/blob/master/searx/engines/wikicommons.py#L43

Closes: #2707
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-09-03 18:45:39 +02:00
Markus Heiser
4f8895c6de [fix] follow-up of 4da7003ae / add missing review from @Bnyro 
[1] https://github.com/searxng/searxng/pull/2656#pullrequestreview-1607956209

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-09-02 09:26:45 +02:00
Hackurei
4da7003ae0 [feat] engine: implementation of odysee 2023-09-02 09:14:12 +02:00
Bnyro
9c4e9d3814 [feat] implementation of Wikimedia commons for images 2023-09-01 18:39:24 +02:00
searxng-bot
9fce459c7f [translations] update from Weblate 
979909acc - 2023-09-01 - return42 <markus.heiser@darmarit.de>
f40af3403 - 2023-08-28 - return42 <markus.heiser@darmarit.de>
 2023-09-01 09:29:07 +02:00
Jafar Farganlooj
7af66736c0 Create search-syntax.md fa 2023-08-30 16:39:10 +02:00
Jafar Farganlooj
26cd770cf7 [mod] Persian translation of the about page (fa_IR) 2023-08-30 16:13:44 +02:00
Alexandre Flament
faa4280e1a [mod] bing: resolve redirect without additional requests 
Remove the usage of searx.network.multi_requests
The results from Bing contains the target URL encoded in base64
See the u parameter, remove the first two character "a1", and done.

Also add a comment the check of the result_len / pageno
( from https://github.com/searx/searx/pull/1387 )
 2023-08-29 07:39:06 +02:00
dalf
f57842b05d Update searx.data - update_firefox_version.py 2023-08-29 07:21:46 +02:00
dalf
a5cd777350 Update searx.data - update_ahmia_blacklist.py 2023-08-29 07:21:21 +02:00
dalf
f361a10af6 Update searx.data - update_currencies.py 2023-08-29 07:20:42 +02:00
dalf
978062bdbe Update searx.data - update_engine_descriptions.py 2023-08-29 07:20:06 +02:00
dalf
f59fd8dc9b Update searx.data - update_wikidata_units.py 2023-08-29 07:17:44 +02:00
Markus Heiser
3294e7cfe4 [fix] reduce error logs related to X-Forwarded-For & X-Real-IP headers 
To test start a `make run` instance and query multiple times for `ip` (the
answerer), you should see only once the error message(s):

    ERROR   searx.botdetection            : X-Forwarded-For header is not set!
    ERROR   searx.botdetection            : X-Real-IP header is not set!

[1] https://github.com/searxng/searxng-docker/discussions/174
[2] https://github.com/searxng/searxng-docker/pull/162#issuecomment-1692834457

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-08-28 08:05:17 +02:00
Bnyro
1428385d1c [mod] preferences.py: add and use BooleanSetting for checkbox preferences 2023-08-27 21:23:43 +02:00
Bnyro
e5535ec078 [feat] settings: replace boolean select preferences with checkboxes 2023-08-27 21:23:43 +02:00
Bnyro
5a5cfc1930 [mod] random answerer: add random hex color generator 2023-08-27 18:59:54 +02:00
Markus Heiser
b0d2cd5ca9 [doc] add documentation of Mwmbl engine & autocompleter 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-08-27 17:25:26 +02:00
Bnyro
df71c24b20 [mod] autocomplete.py: add support for mwmbl completions 2023-08-27 17:25:26 +02:00
Bnyro
19300a5659 [mod] engine mwmbl: add link to official api docs 2023-08-27 17:25:26 +02:00
Alexandre Flament
97b1df1629 [mod] searx.network: memory optimization 
Avoid to create a SSLContext in AsyncHTTPTransportNoHttp

See:
* 0f61aa58d6/httpx/_transports/default.py (L271)
* https://github.com/encode/httpx/issues/2298
 2023-08-27 11:49:40 +02:00
Alexandre Flament
e16c007c22 [fix] openstreetmap engine 
It seems there is an API change:
extratags can be either a dictionnary or None.

This commit avoid crash when extratags is None

Test query "!osm gare du nord"
 2023-08-27 11:49:16 +02:00
Markus Heiser
0647f83a3e [fix] google engine: don't overspecify the search query to Google 
The method EngineTraits.get_region(..) returns engine's region string
that **best fits** to SearXNG's locale.  This means it returns a
region (country) if only a language is set in the locale.  By example the method
returns for a locale tag `es` a region `ES`.

Google's search parameter `cr` restricts search results to documents originating
in a particular country / in case of a locale tag (language) as described above,
this argument should be unset in the query send to Google.

Closes: https://github.com/searxng/searxng/issues/2672
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-08-26 07:47:07 +02:00
searxng-bot
7d806d1020 [translations] update from Weblate 
a6c2d1988 - 2023-08-22 - shoko <nickskorohod@outlook.com>
38bc33a71 - 2023-08-20 - return42 <markus.heiser@darmarit.de>
0ff21aa3c - 2023-08-19 - MonsoonFire <re1qnb5mq@mozmail.com>
 2023-08-25 09:36:49 +02:00
Paul Braeuning
9069da051a
Merge pull request #2644 from dalf/update_httpx 
Bump httpx 0.21.2 from to 0.24.1
 2023-08-24 23:12:06 +02:00
Markus Heiser
4b42644579 [fix] engine google_video: google has changed the layout of the rsponse 
Closes: https://github.com/searxng/searxng/issues/2664
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-08-22 08:34:04 +02:00
Alexandre Flament
b4e4cfc026 Bump httpx 0.21.2 from to 0.24.1 2023-08-21 22:05:12 +02:00
Markus Heiser
d52919cc3e [mod] decrease the weighting of wikivoyage, wikibooks, wikisource and wikiversity 
The search engines deliver hits for many search terms [1], but these are usually
not the focus of the user. In order to arrange these hits further down in the
list, their weighting is reduced.

[1] https://github.com/searxng/searxng/pull/2589#issuecomment-1670915089

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-08-21 21:52:40 +02:00
PlayDay
018b0a9322 [feat] engine: XPath configuration for pkg.go.dev 2023-08-19 19:19:31 +02:00
Bnyro
c59ae91b76 [feat] engine: implementation of mwmbl 2023-08-19 18:23:42 +02:00
searxng-bot
bb9e9ab773 [translations] update from Weblate 
d8e4100b6 - 2023-08-16 - return42 <markus.heiser@darmarit.de>
b8e03f3bc - 2023-08-17 - ghose <correo@xmgz.eu>
 2023-08-18 19:22:13 +02:00
Markus Heiser
c741fc6f00 [mod] currency_convert: support for showing the answer source url 
Show URL of the ddg-search page, not the URL of a (generic) Javascript.  The
latter one is not usefull for the user.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-08-18 19:07:14 +02:00
Markus Heiser
e2744520f8 [mod] google: support for showing the answer source url 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-08-18 19:07:14 +02:00
Bnyro
5ec7df3480 [mod] engine duckduckgo definitions: support for answer source 2023-08-18 19:07:14 +02:00
Bnyro
64bc98b5fb [mod] brave: support for showing the answer source url 2023-08-18 19:07:14 +02:00
Markus Heiser
6fc2ee514d [build] /static 2023-08-18 19:07:14 +02:00
Bnyro
13d801b75d [feat] results: show source of suggested answer 2023-08-18 19:07:14 +02:00
Markus Heiser
9100a48541 [mod] improve seekr engines and add documentation 
Tis patch adds some more fields to the result items and changed paging to the
``nextResultSet`` given in seekr's JSON response.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-08-15 16:17:42 +02:00
Bnyro
2bab658d39 [feat] engine: implementation of seekr for news, images and videos 2023-08-15 16:17:42 +02:00
Markus Heiser
7052a1a7cf [drop] engine archive.is / blocked by CAPTCHA [1] 
Sadly archive.is is blocked by a CAPTCHA that can't be avoid (at least in a
XPath engine).

[1] https://github.com/searxng/searxng/issues/2643

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-08-14 08:50:13 +02:00
mrpaulblack
621b09b9b9 [fix] preferences: change engine table category colspan to 8 
* this is a small fix to increase the colspan of the category in engine preferences from 7 to 8, since there was a column added
=> fixing a small fallout from 4731290317
 2023-08-14 07:11:06 +02:00
ToxyFlog1627
f175574f37 [fix] typos in documentation & messages 2023-08-13 08:50:29 +02:00
Markus Heiser
c348427f07 [update] make pygments.less 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-08-11 10:54:50 +02:00
searxng-bot
99df7b84d1 [translations] update from Weblate 
12c527bcd - 2023-08-10 - maxch <maxletters@tutanota.com>
ea46b66fc - 2023-08-10 - return42 <markus.heiser@darmarit.de>
ae3d8dcdf - 2023-08-10 - return42 <markus.heiser@darmarit.de>
ab6a59e35 - 2023-08-10 - return42 <markus.heiser@darmarit.de>
1db1affcc - 2023-08-10 - gallegonovato <fran-carro@hotmail.es>
94b682aa0 - 2023-08-10 - Fjuro <ifjuro@proton.me>
07c49bb6c - 2023-08-10 - return42 <markus.heiser@darmarit.de>
 2023-08-11 09:17:56 +02:00
Bnyro
c76c999a1e [feat] engine: implementation of anaconda 2023-08-10 23:51:07 +02:00
Bnyro
e25d1c7288 [feat] engine: implementation of German news, Tagesschau 
Co-authored-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-08-10 20:27:54 +02:00
GeckoLinux
2256ba2ffb [mod] settings.yml: move wolframalpha to categ general & disable by default 
As suggested in comment [1] as a good compromise between [2] and [3].

This will help users that want Wolfram Alpha instant answers by default with
every query and don't mind the slowness.  It will be disabled by default so it's
fast for all users, and those that need it can enable it as a user-configurable
option without requiring setting up their own instance to change the Wolfram
Alpha categorization.

[1] https://github.com/searxng/searxng/pull/813#issuecomment-1574851572
[2] https://github.com/searxng/searxng/pull/813#issuecomment-1020581765
[3] https://github.com/searxng/searxng/issues/966#issue-1171107451
 2023-08-10 14:31:09 +02:00
Bnyro
834e1c3f12 [mod] engine lemmy: increase thumbnail quality to align with theme 2023-08-10 12:58:40 +02:00
Markus Heiser
c381fc001f [mod] settings: remove lemmy from categ 'general' & enable by default 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-08-10 12:58:40 +02:00
Markus Heiser
fda111c0c9 [mod] engine lemmy: add more info fields to the result items 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-08-10 12:58:40 +02:00
Bnyro
224f2250ae [feat] engine: support for lemmy communities, posts, comments and users 2023-08-10 12:58:40 +02:00
Bnyro
9f82c39610 [mod] engine google_play: raise error on unsupported category 2023-08-10 12:35:24 +02:00
Bnyro
0a99dc85b9 [mod] engine brave: raise error on unsupported category 2023-08-10 12:35:24 +02:00
Bnyro
4731290317 [mod] settings: show weight of each instance 2023-08-10 10:50:14 +02:00
Markus Heiser
733b795d53 [fix] make flask_babel.gettext() work in engine modules (L10n & threads) 
incident:
  flask_babel.gettext() does not work in the engine modules.

cause:
  the request() and response() functions of the engine modules run in the
  processor, whose search() method runs in a thread and in the threads the
  context of the Flask app does not exist. The context of the Flask app is
  needed by the gettext() function for the L10n.

Solution:
  copy context of the Flask app into the threads. [1]

special case:
  We cannot equip the search() method of the processors with the decorator [1],
  because the decorator requires a context (Flask app) that does not yet exist
  at the time of the initialization of the processors (the initialization of the
  processors is part of the initialization of the Flask app).

[1] https://flask.palletsprojects.com/en/2.3.x/api/#flask.copy_current_request_context

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-08-09 13:27:43 +02:00
allendema_searxng_pi
c00c0c5434 [mod] remove discontinued petalsearch engines 2023-08-09 07:17:40 +02:00
Markus Heiser
b8352eca0c [mod] brave engines: add fetch_traits() / improve language support 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-08-08 16:21:45 +02:00
Markus Heiser
460bbe5b81 [mod] implement brave (WEB) engine to replace XPath configuration 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-08-08 16:21:45 +02:00
Bnyro
d151497db3 [feat] engine: brave - support for news 2023-08-08 16:21:45 +02:00
Bnyro
cae06f2781 [feat] engine: brave - support for videos 2023-08-08 16:21:45 +02:00
Bnyro
73364e158e [feat] engine: brave - support for images 2023-08-08 16:21:45 +02:00
Markus Heiser
bcaaae699f [doc] fix some minor typos in the doc-strings of class EngineTraits 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-08-08 08:15:53 +02:00
Markus Heiser
a2108432f7 [fix] settings.yml: typo in engine name wikipecies -> wikispecies 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-08-07 10:35:44 +02:00
Markus Heiser
1d0abb7157 [doc] engine bt4g: add documentation to docs/dev/engines/online/ 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-08-06 09:30:48 +02:00
Emilien Devos
0fc8f99ecc [feat] new engine: bt4g added & enabled and disable by default btdigg 
Disable btdigg because on most SearXNG instances, SearXNG is blocked by btdigg
due to cloudflare too many requests.

This impementation did not parse the HTML page because there is an API in
XML (RSS).  The RSS feed provides fewer data like amount of seeders/leechers and
the files in the torrent file. It's a tradeoff for a "stable" engine as the XML
from RSS content will change way less than the HTML page.

Closes: https://github.com/searxng/searxng/issues/2553
 2023-08-06 09:30:48 +02:00
Markus Heiser
dad16c0949 [mod] settings.yml: enable engines from Wikimedia Foundation 
The Wikimedia wikis [1] engines provide good answers and have short response
times --> no reason to disable these enhgines by default.  BTW: this patch adds
a (sub-) category ``wikimedia`` for the engines [1].

[1] https://meta.wikimedia.org/wiki/Wikimedia_wikis

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-08-04 19:06:50 +02:00
Markus Heiser
db522cf76d [mod] engine: wikimedia - improve results, add addition settings & doc 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-08-04 19:06:50 +02:00
Bnyro
7d8c20c80d [feat] new engine: wikispecies 2023-08-04 19:06:50 +02:00
searxng-bot
9a5e91ce84 [translations] update from Weblate 
310daeb72 - 2023-08-03 - return42 <markus.heiser@darmarit.de>
01be8a264 - 2023-08-01 - return42 <markus.heiser@darmarit.de>
 2023-08-04 09:30:11 +02:00
Markus Heiser
1b030d4b41 [doc] engine: Yacy 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-08-03 19:58:51 +02:00
zutto
ca518c6803 add option to change yacy search mode 2023-08-03 19:58:51 +02:00
Markus Heiser
203f1f0928 [fix] engine piped: 'invalid content' 
SearXNG does not allow a None value in the content field of a result item.

If the key (shortDescription, uploaderName) in the JSON response from piped
exists but is set to None, SearXNG ignores this result item::

  DEBUG   searx    : result: invalid content: { ..,  'content': None,  ..}

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-08-03 16:23:36 +02:00
Markus Heiser
207fcc0c8c [mod] engine piped: add paging support 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-08-03 16:23:36 +02:00
Markus Heiser
ef5831cd84 [mod] engine piped: split into two dedicated engiens for video & music 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-08-03 16:23:36 +02:00
Markus Heiser
7aa95d2d52 [doc] engine piped: add documentation to docs/dev/engines/online/ 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-08-03 16:23:36 +02:00
Bnyro
636bfdac68 [feat] engine: implementation of Piped 2023-08-03 16:23:36 +02:00
dalf
8b4ba204b7 Update searx.data - update_engine_descriptions.py 2023-07-29 11:02:54 +02:00
dalf
d421385dc6 Update searx.data - update_wikidata_units.py 2023-07-29 11:02:23 +02:00
dalf
290f16207b Update searx.data - update_currencies.py 2023-07-29 11:02:14 +02:00
dalf
fcd2013511 Update searx.data - update_firefox_version.py 2023-07-29 11:01:28 +02:00
dalf
cfa8169509 Update searx.data - update_ahmia_blacklist.py 2023-07-29 09:25:24 +02:00
searxng-bot
f45d1920d9 [translations] update from Weblate 
ba4888c96 - 2023-07-26 - return42 <markus.heiser@darmarit.de>
6ec8a8a28 - 2023-07-22 - return42 <markus.heiser@darmarit.de>
0a7b701b3 - 2023-07-24 - artnay <jiri.gronroos@iki.fi>
c0b34cbdb - 2023-07-23 - MonsoonFire <re1qnb5mq@mozmail.com>
37cbd41c2 - 2023-07-22 - return42 <markus.heiser@darmarit.de>
 2023-07-29 08:15:21 +02:00
mrpaulblack
65d8b1a310 [fix] remove disabled: false from engine definitions in settings.yml 
* setting disabled: false is not needed, since it is by default enabled
 2023-07-22 18:19:01 +02:00
searxng-bot
51c531d450 [translations] update from Weblate 
b7f1e9ae - 2023-07-17 - Hudobni Volk <hudobni.volk@tuta.io>
3c7c821e - 2023-07-16 - alextecplayz <alextec70@outlook.com>
0e305f84 - 2023-07-17 - return42 <markus.heiser@darmarit.de>
80745a22 - 2023-07-15 - tentsbet <remendne@pentrens.jp>
afef0e2e - 2023-07-16 - Salif Mehmed <mail@salif.eu>
4a7687ac - 2023-07-14 - Ivan Gabaldon <admin@inetol.net>
 2023-07-22 17:47:05 +02:00
mrpaulblack
b477349824 [build] /static 2023-07-19 15:07:45 +02:00
Kiru
de5c1cedca fix "#backToTop" button always being clickable 
`pointer-events` never gets set to "none" when the button is hidden,
allowing you to click the button. And your mouse further changes it's
cursor to the pointer style.
 2023-07-19 15:06:09 +02:00
searxng-bot
b7b184244d [translations] update from Weblate 
01350cf1 - 2023-07-13 - return42 <markus.heiser@darmarit.de>
5f037a4d - 2023-07-12 - return42 <markus.heiser@darmarit.de>
820a78ad - 2023-07-12 - return42 <markus.heiser@darmarit.de>
73037743 - 2023-07-12 - return42 <markus.heiser@darmarit.de>
e656795c - 2023-07-09 - Linerly <linerly@protonmail.com>
0ee18285 - 2023-07-08 - return42 <markus.heiser@darmarit.de>
c087c7fb - 2023-07-08 - return42 <markus.heiser@darmarit.de>
6eb318c5 - 2023-07-08 - return42 <markus.heiser@darmarit.de>
3b4a3d1f - 2023-07-08 - return42 <markus.heiser@darmarit.de>
b3187499 - 2023-07-09 - return42 <markus.heiser@darmarit.de>
c1226646 - 2023-07-08 - return42 <markus.heiser@darmarit.de>
2356a402 - 2023-07-08 - return42 <markus.heiser@darmarit.de>
c9a74b52 - 2023-07-08 - return42 <markus.heiser@darmarit.de>
3d9f2938 - 2023-07-08 - return42 <markus.heiser@darmarit.de>
68af8585 - 2023-07-08 - return42 <markus.heiser@darmarit.de>
261a2a72 - 2023-07-08 - return42 <markus.heiser@darmarit.de>
fcea15cf - 2023-07-08 - return42 <markus.heiser@darmarit.de>
7685385e - 2023-07-08 - return42 <markus.heiser@darmarit.de>
ec0a3727 - 2023-07-08 - return42 <markus.heiser@darmarit.de>
0130ddf7 - 2023-07-08 - return42 <markus.heiser@darmarit.de>
b93f9609 - 2023-07-08 - return42 <markus.heiser@darmarit.de>
4a5cdcb3 - 2023-07-08 - return42 <markus.heiser@darmarit.de>
9cba3939 - 2023-07-08 - return42 <markus.heiser@darmarit.de>
d973d937 - 2023-07-08 - return42 <markus.heiser@darmarit.de>
ce076245 - 2023-07-08 - return42 <markus.heiser@darmarit.de>
5c36ccab - 2023-07-08 - return42 <markus.heiser@darmarit.de>
226ff7d4 - 2023-07-08 - return42 <markus.heiser@darmarit.de>
8148a9ed - 2023-07-08 - return42 <markus.heiser@darmarit.de>
840bc189 - 2023-07-08 - return42 <markus.heiser@darmarit.de>
51ffc22e - 2023-07-08 - return42 <markus.heiser@darmarit.de>
394ec63e - 2023-07-08 - return42 <markus.heiser@darmarit.de>
428c16a8 - 2023-07-08 - return42 <markus.heiser@darmarit.de>
218cf51e - 2023-07-08 - return42 <markus.heiser@darmarit.de>
70260934 - 2023-07-09 - ghose <correo@xmgz.eu>
c6244c2b - 2023-07-08 - return42 <markus.heiser@darmarit.de>
b92dc5c1 - 2023-07-08 - return42 <markus.heiser@darmarit.de>
43917957 - 2023-07-08 - return42 <markus.heiser@darmarit.de>
df1bf630 - 2023-07-08 - return42 <markus.heiser@darmarit.de>
d1c00dff - 2023-07-08 - return42 <markus.heiser@darmarit.de>
0a6da54f - 2023-07-08 - return42 <markus.heiser@darmarit.de>
12377e28 - 2023-07-08 - return42 <markus.heiser@darmarit.de>
b5b8ea78 - 2023-07-07 - gallegonovato <fran-carro@hotmail.es>
ec31e65f - 2023-07-08 - return42 <markus.heiser@darmarit.de>
6c33b1fe - 2023-07-08 - return42 <markus.heiser@darmarit.de>
393d390c - 2023-07-08 - return42 <markus.heiser@darmarit.de>
a4f6b353 - 2023-07-08 - return42 <markus.heiser@darmarit.de>
0f8d6b6b - 2023-07-08 - return42 <markus.heiser@darmarit.de>
67f2fc96 - 2023-07-08 - Fjuro <ifjuro@proton.me>
5f2d3f02 - 2023-07-08 - return42 <markus.heiser@darmarit.de>
5ae2b8dc - 2023-07-08 - return42 <markus.heiser@darmarit.de>
0bd4fb1e - 2023-07-08 - return42 <markus.heiser@darmarit.de>
ce768726 - 2023-07-08 - return42 <markus.heiser@darmarit.de>
a22ae2f2 - 2023-07-08 - return42 <markus.heiser@darmarit.de>
b5b8774f - 2023-07-08 - return42 <markus.heiser@darmarit.de>
 2023-07-14 10:21:27 +02:00
Paolo Basso
cada89ee36 [feat] engine: re-enables z-library (zlibrary-global.se) 
- re-enables z-library as the new domain zlibrary-global.se is now available
  from the open web.   The announcement of the domain:

    https://www.reddit.com/r/zlibrary/comments/13whe08/mod_note_zlibraryglobalse_domain_is_officially/

  It is an official domain, it requires to log in to the "personal" subdomain
  only to download files, but the search works.

- changes the result template of zlibrary to paper.html, filling the appropriate fields
- implements language filtering for zlibrary
- implement zlibrary custom filters (engine traits)
- refactor and document the zlibrary engine
 2023-07-07 21:36:51 +02:00
Hackurei
cb92767f19 [feat] enigine: add CrowdView forum search engine 2023-07-07 21:36:11 +02:00
searxng-bot
4a2f310da3 [translations] update from Weblate 
152f2008 - 2023-07-05 - return42 <markus.heiser@darmarit.de>
9dbf6b22 - 2023-07-01 - return42 <markus.heiser@darmarit.de>
4ad4c00f - 2023-07-01 - Bananhylsa <thayer@hjemmeserver.net>
 2023-07-07 21:13:47 +02:00
Markus Heiser
5720844fcd [doc] rearranges Settings & Engines docs for better readability 
We have built up detailed documentation of the *settings* and the *engines* over
the past few years.  However, this documentation was still spread over various
chapters and was difficult to navigate in its entirety.

This patch rearranges the Settings & Engines documentation for better
readability.

To review new ordered docs::

   make docs.clean docs.live

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-07-01 22:45:19 +02:00
searxng-bot
81c9a18456 [translations] update from Weblate 
2238e87b - 2023-06-28 - jenishngl <jenishngl+codeberg@gmail.com>
c70d228a - 2023-06-24 - nogb <u8cn71wq@yogibo.anonaddy.me>
389c0c62 - 2023-06-24 - return42 <markus.heiser@darmarit.de>
656d9fcb - 2023-06-23 - return42 <markus.heiser@darmarit.de>
a9c9b116 - 2023-06-25 - alma <alma@users.noreply.translate.codeberg.org>
528b845f - 2023-06-24 - nogb <u8cn71wq@yogibo.anonaddy.me>
b8c50f23 - 2023-06-23 - return42 <markus.heiser@darmarit.de>
39f47c0f - 2023-06-23 - return42 <markus.heiser@darmarit.de>
ae0aa811 - 2023-06-24 - Fjuro <ifjuro@proton.me>
c8216259 - 2023-06-26 - lemonadeforlife <nahianlabiblimon44@gmail.com>
 2023-06-30 11:49:07 +02:00
dalf
fbb72fc1f4 Update searx.data - update_engine_descriptions.py 2023-06-29 13:59:25 +02:00
Markus Heiser
87e7926ae9 [fix] engine: Anna's Archive - grep results from '.js-scroll-hidden' elements 
The renderuing of the WEB page is very strange; except the firts position all
other positions of Anna's result page are enclosed in SGML comments.  These
cooments are *uncommented* by some JS code, see query of the class
'.js-scroll-hidden' in Anna's HTML template [1].

[1] https://annas-software.org/AnnaArchivist/annas-archive/-/blob/main/allthethings/templates/macros/md5_list.html

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-06-29 09:32:57 +02:00
Markus Heiser
e2df6b77a3 [mod] engine: Anna's Archive - additionl settings (content, sort, ext) 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-06-29 09:32:57 +02:00
Markus Heiser
eafc2906f1 [mod] engine: Anna's Archive - fetch search arguments from search form 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-06-29 09:32:57 +02:00
Paolo Basso
7adb9090e5 [mod] engine: Anna's Archive - add language support 2023-06-29 09:32:57 +02:00
Paolo Basso
e5637fe7b9 [feat] engine: implementation of Anna's Archive 
Anna's Archive [1] is a free non-profit online shadow library metasearch engine
providing access to a variety of book resources (also via IPFS), created by a
team of anonymous archivists [2].

[1] https://annas-archive.org/
[2] https://annas-software.org/AnnaArchivist/annas-archive
 2023-06-29 09:32:57 +02:00
Markus Heiser
fd26f37073 [upd] make data.all 
- ahmia_blacklist.txt
- currencies.json
- engine_descriptions.json
- engine_traits.json
- osm_keys_tags.json
- useragents.json

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-06-28 21:21:53 +02:00
Markus Heiser
efea962504 [fix] simple template: preferences - add missing icon_smal import 
Related: https://github.com/searxng/searxng/commit/2149e88bdd64#r119535272
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-06-28 18:36:52 +02:00
Paolo Basso
401561cb58 [mod] engine torznab - refactor & option to hide links 
- torznab engine using types and clearer code
- torznab option to hide torrent and magnet links.
- document the torznab engine
- add myself to authors

Closes: https://github.com/searxng/searxng/issues/1124
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-06-28 10:03:44 +02:00
Markus Heiser
da7c30291d [fix] Google API changed 
It seems that Google is rolling out a modified WEB API [1][2].

In the past there was only the UI language in the `hl` argument but nowadays it
seems a combination of the UI language and the "search region" is mixed in this
argument and the `gl` argument has been removed.  I'm very surprised that google
is starting to mix the parameters of the UI with the parameters of the search
index.

This patch modifies the get_google_info(..) function.  Beside Google-WEB this
function is also used by other Google services, here are some examples to test
region & language of ..

- Google-WEB:    `!go dragon boat :en-CA`
- Google-News:   `!gon dragon boat :en-CA`
- Google-Videos: `!gov bmw :en-CA`
- Goolge-Images  `!goi bmw :en-CA`

- [1] https://github.com/searxng/searxng/issues/2515#issuecomment-1606294635
- [2] https://github.com/searxng/searxng/issues/2515#issuecomment-1607150817

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-06-26 18:28:09 +02:00
Markus Heiser
e8706fb738 [fix] engine & network issues / documentation and type annotations 
This patch fixes some quirks and issues related to the engines and the network.
Each engine has its own network and this network was broken for the following
engines[1]:

- archlinux
- bing
- dailymotion
- duckduckgo
- google
- peertube
- startpage
- wikipedia

Since the files have been touched anyway, the type annotaions of the engine
modules has also been completed so that error messages from the type checker are
no longer reported.

Related and (partial) fixed issue:

- [1] https://github.com/searxng/searxng/issues/762#issuecomment-1605323861
- [2] https://github.com/searxng/searxng/issues/2513
- [3] https://github.com/searxng/searxng/issues/2515

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-06-25 13:58:26 +02:00
searxng-bot
2e4a435134 [translations] update from Weblate 
9512b92a - 2023-06-23 - Coccocoas_Helper <coccocoahelper@gmail.com>
ca08c51e - 2023-06-23 - Coccocoas_Helper <coccocoahelper@gmail.com>
56ad4f21 - 2023-06-21 - return42 <markus.heiser@darmarit.de>
3ee419d6 - 2023-06-21 - return42 <markus.heiser@darmarit.de>
 2023-06-23 09:34:46 +02:00
Markus Heiser
86db08793b [fix] implement a JSONEncoder for the json format 
This patch implements a simple JSONEncoder just to fix #2502 / on the long term
SearXNG needs a data schema for the result items and a json generator for the
result list.

Closes: https://github.com/searxng/searxng/issues/2505
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-06-19 19:49:44 +02:00
Markus Heiser
fa1ef9a07b [mod] move some code from webapp module to webutils module (no functional change) 
Over the years the webapp module became more and more a mess.  To improve the
modulaization a little this patch moves some implementations from the webapp
module to webutils module.

HINT: this patch brings non functional change

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-06-19 19:49:44 +02:00
searxng-bot
71b6ff07ca [translations] update from Weblate 
98f61c70 - 2023-06-15 - alexgabi <alexgabi@disroot.org>
a1679b93 - 2023-06-13 - return42 <markus.heiser@darmarit.de>
ebd1d574 - 2023-06-13 - return42 <markus.heiser@darmarit.de>
b28a1da3 - 2023-06-13 - return42 <markus.heiser@darmarit.de>
56409bf0 - 2023-06-11 - return42 <markus.heiser@darmarit.de>
abc4916c - 2023-06-10 - return42 <markus.heiser@darmarit.de>
b1900abe - 2023-06-10 - return42 <markus.heiser@darmarit.de>
b48e84c4 - 2023-06-10 - return42 <markus.heiser@darmarit.de>
bf395e32 - 2023-06-10 - return42 <markus.heiser@darmarit.de>
c9c0a3c9 - 2023-06-10 - return42 <markus.heiser@darmarit.de>
3f50d31e - 2023-06-10 - return42 <markus.heiser@darmarit.de>
9da1c142 - 2023-06-09 - artnay <jiri.gronroos@iki.fi>
 2023-06-16 09:20:43 +02:00
searxng-bot
1be27d5d83 [translations] update from Weblate 
b40da1a3 - 2023-06-06 - return42 <markus.heiser@darmarit.de>
666ee7d4 - 2023-06-06 - return42 <markus.heiser@darmarit.de>
1e0e8ead - 2023-06-06 - return42 <markus.heiser@darmarit.de>
404b9937 - 2023-06-07 - Ivan Gabaldon <admin@inetol.net>
a627f9a1 - 2023-06-04 - return42 <markus.heiser@darmarit.de>
a234d2f8 - 2023-06-04 - gallegonovato <fran-carro@hotmail.es>
cc41f9b5 - 2023-06-02 - return42 <markus.heiser@darmarit.de>
24651eac - 2023-06-02 - return42 <markus.heiser@darmarit.de>
c37b0627 - 2023-06-02 - return42 <markus.heiser@darmarit.de>
9a435ea1 - 2023-06-02 - return42 <markus.heiser@darmarit.de>
40e0adad - 2023-06-02 - return42 <markus.heiser@darmarit.de>
6833b142 - 2023-06-02 - return42 <markus.heiser@darmarit.de>
00f397ad - 2023-06-02 - tentsbet <remendne@pentrens.jp>
7d3d4a97 - 2023-06-02 - return42 <markus.heiser@darmarit.de>
f7d713a4 - 2023-06-02 - return42 <markus.heiser@darmarit.de>
b1ec3160 - 2023-06-03 - ghose <correo@xmgz.eu>
04591a3a - 2023-06-02 - return42 <markus.heiser@darmarit.de>
cb3ac67c - 2023-06-02 - return42 <markus.heiser@darmarit.de>
fe81dbc7 - 2023-06-02 - return42 <markus.heiser@darmarit.de>
7882670f - 2023-06-02 - return42 <markus.heiser@darmarit.de>
38882f3b - 2023-06-02 - return42 <markus.heiser@darmarit.de>
c6df5047 - 2023-06-02 - return42 <markus.heiser@darmarit.de>
6ca23c3b - 2023-06-02 - return42 <markus.heiser@darmarit.de>
72f1ee09 - 2023-06-02 - return42 <markus.heiser@darmarit.de>
 2023-06-09 07:07:51 +00:00
Markus Heiser
22b13f4fa5 [mod] tools.Config.get(): add missing type annotations 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-06-05 14:07:19 +02:00
Markus Heiser
f3763d73ad [mod] limiter: blocklist and passlist (ip_lists) 
A blocklist and a passlist can be configured in /etc/searxng/limiter.toml::

    [botdetection.ip_lists]
    pass_ip = [
      '51.15.252.168',  # IPv4 of check.searx.space
    ]

    block_ip = [
      '93.184.216.34',  # IPv4 of example.org
    ]

Closes: https://github.com/searxng/searxng/issues/2127
Closes: https://github.com/searxng/searxng/pull/2129
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-06-05 14:07:19 +02:00
Markus Heiser
f77807257b [fix] engines: don't spam marginalia.nu with default settings 
The engine configuration of marginalia [2][3][4][5] spams marginalia.nu with
requests from SearXNG instances [1].  It is not in the interest of SearXNG to
disturb other FOSS projects, so the engine will be removed::

    - name: marginalia
      engine: json_engine
      shortcut: mar
      categories: general
      paging: false
      # Key and license: https://www.marginalia.nu/marginalia-search/api/
      # index: 0 popular, 1 blogs, 2 big_sites, 3 default, 4 experimental
      search_url: https://api.marginalia.nu/<insert your key here>/search/{query}?index=4&count=20
      results_query: results
      url_query: url
      title_query: title
      content_query: description
      timeout: 1.5
      disabled: true
      about:
        website: https://www.marginalia.nu/
        official_api_documentation: https://api.marginalia.nu/
        use_official_api: true
        require_api_key: true
        results: JSON

[1] https://github.com/searxng/searxng/issues/1673
[2] https://github.com/searxng/searxng/pull/1627
[3] https://github.com/searxng/searxng/issues/1620
[4] https://news.ycombinator.com/item?id=35874640
[5] d82a858491/code/services-satellite/api-service/src/main/java/nu/marginalia/api/svc/ResponseCache.java (L12-L20)

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2023-06-05 08:23:17 +02:00
Markus Heiser
80aaef6c95
Merge pull request #2357 / limiter -> botdetection 
The monolithic implementation of the limiter was divided into methods and
implemented in the Python package searx.botdetection.  Detailed documentation on
the methods has been added.

The methods are divided into two groups:

1. Probe HTTP headers

- Method http_accept
- Method http_accept_encoding
- Method http_accept_language
- Method http_connection
- Method http_user_agent

2. Rate limit:

- Method ip_limit
- Method link_token (new)

The (reduced) implementation of the limiter is now in the module
searx.botdetection.limiter.  The first group was transferred unchanged to this
module.  The ip_limit contains the sliding windows implemented by the limiter so
far.

This merge also fixes some long outstandig issue:

- limiter does not evaluate the Accept-Language correct [1]
- limiter needs a IPv6 prefix to block networks instead of IPs [2]

Without additional configuration the limiter works as before (apart from the
bugfixes).  For the commissioning of additional methods (link_toke), a
configuration must be made in an additional configuration file.  Without this
configuration, the limiter runs as before (zero configuration).

The ip_limit Method implements the sliding windows of the vanilla limiter,
additionally the link_token method can be used in this method.  The link_token
method can be used to investigate whether a request is suspicious. To activate
the link_token method in the ip_limit method add the following to your
/etc/searxng/limiter.toml::

    [botdetection.ip_limit]
    link_token = true


[1] https://github.com/searxng/searxng/issues/2455
[2] https://github.com/searxng/searxng/issues/2477
 2023-06-03 06:00:15 +02:00