mirror of
https://github.com/searxng/searxng.git
synced 2024-11-22 04:01:40 +01:00
[doc] rearranges Settings & Engines docs for better readability
We have built up detailed documentation of the *settings* and the *engines* over the past few years. However, this documentation was still spread over various chapters and was difficult to navigate in its entirety. This patch rearranges the Settings & Engines documentation for better readability. To review new ordered docs:: make docs.clean docs.live Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
This commit is contained in:
Markus Heiser
Markus Heiser
parent
8e8d8dabe9
commit
5720844fcd
@ -9,7 +9,7 @@ Buildhosts
|
||||
If you have any contribution send us your :pull:`PR <../pulls>`, see
|
||||
:ref:`how to contribute`.
|
||||
|
||||
.. contents:: Contents
|
||||
.. contents::
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
@ -1,79 +0,0 @@
|
||||
.. _engine command:
|
||||
|
||||
====================
|
||||
Command Line Engines
|
||||
====================
|
||||
|
||||
.. sidebar:: info
|
||||
|
||||
- :origin:`command.py <searx/engines/command.py>`
|
||||
- :ref:`offline engines`
|
||||
|
||||
With *command engines* administrators can run engines to integrate arbitrary
|
||||
shell commands.
|
||||
|
||||
When creating and enabling a ``command`` engine on a public instance, you must
|
||||
be careful to avoid leaking private data. The easiest solution is to limit the
|
||||
access by setting ``tokens`` as described in section :ref:`private engines`.
|
||||
|
||||
The engine base is flexible. Only your imagination can limit the power of this
|
||||
engine (and maybe security concerns). The following options are available:
|
||||
|
||||
``command``:
|
||||
A comma separated list of the elements of the command. A special token
|
||||
``{{QUERY}}`` tells where to put the search terms of the user. Example:
|
||||
|
||||
.. code:: yaml
|
||||
|
||||
['ls', '-l', '-h', '{{QUERY}}']
|
||||
|
||||
``delimiter``:
|
||||
A mapping containing a delimiter ``char`` and the *titles* of each element in
|
||||
``keys``.
|
||||
|
||||
``parse_regex``:
|
||||
A dict containing the regular expressions for each result key.
|
||||
|
||||
``query_type``:
|
||||
|
||||
The expected type of user search terms. Possible values: ``path`` and
|
||||
``enum``.
|
||||
|
||||
``path``:
|
||||
Checks if the user provided path is inside the working directory. If not,
|
||||
the query is not executed.
|
||||
|
||||
``enum``:
|
||||
Is a list of allowed search terms. If the user submits something which is
|
||||
not included in the list, the query returns an error.
|
||||
|
||||
``query_enum``:
|
||||
A list containing allowed search terms if ``query_type`` is set to ``enum``.
|
||||
|
||||
``working_dir``:
|
||||
|
||||
The directory where the command has to be executed. Default: ``./``
|
||||
|
||||
``result_separator``:
|
||||
The character that separates results. Default: ``\n``
|
||||
|
||||
The example engine below can be used to find files with a specific name in the
|
||||
configured working directory:
|
||||
|
||||
.. code:: yaml
|
||||
|
||||
- name: find
|
||||
engine: command
|
||||
command: ['find', '.', '-name', '{{QUERY}}']
|
||||
query_type: path
|
||||
shortcut: fnd
|
||||
delimiter:
|
||||
chars: ' '
|
||||
keys: ['line']
|
||||
|
||||
|
||||
Acknowledgment
|
||||
==============
|
||||
|
||||
This development was sponsored by `Search and Discovery Fund
|
||||
<https://nlnet.nl/discovery>`_ of `NLnet Foundation <https://nlnet.nl/>`_.
|
||||
@ -1,26 +0,0 @@
|
||||
.. _engines and settings:
|
||||
|
||||
==================
|
||||
Engines & Settings
|
||||
==================
|
||||
|
||||
.. sidebar:: Further reading ..
|
||||
|
||||
- :ref:`settings engine`
|
||||
- :ref:`engine settings` & :ref:`engine file`
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 3
|
||||
|
||||
settings
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 1
|
||||
|
||||
private-engines
|
||||
recoll
|
||||
sql-engines
|
||||
nosql-engines
|
||||
search-indexer-engines
|
||||
command-line-engines
|
||||
searx.engines.xpath
|
||||
@ -1,49 +0,0 @@
|
||||
.. _private engines:
|
||||
|
||||
============================
|
||||
Private Engines (``tokens``)
|
||||
============================
|
||||
|
||||
Administrators might find themselves wanting to limit access to some of the
|
||||
enabled engines on their instances. It might be because they do not want to
|
||||
expose some private information through :ref:`offline engines`. Or they would
|
||||
rather share engines only with their trusted friends or colleagues.
|
||||
|
||||
To solve this issue the concept of *private engines* exists.
|
||||
|
||||
|
||||
A new option was added to engines named `tokens`. It expects a list of
|
||||
strings. If the user making a request presents one of the tokens of an engine,
|
||||
they can access information about the engine and make search requests.
|
||||
|
||||
Example configuration to restrict access to the Arch Linux Wiki engine:
|
||||
|
||||
.. code:: yaml
|
||||
|
||||
- name: arch linux wiki
|
||||
engine: archlinux
|
||||
shortcut: al
|
||||
tokens: [ 'my-secret-token' ]
|
||||
|
||||
|
||||
Unless a user has configured the right token, the engine is going
|
||||
to be hidden from him/her. It is not going to be included in the
|
||||
list of engines on the Preferences page and in the output of
|
||||
`/config` REST API call.
|
||||
|
||||
Tokens can be added to one's configuration on the Preferences page
|
||||
under "Engine tokens". The input expects a comma separated list of
|
||||
strings.
|
||||
|
||||
The distribution of the tokens from the administrator to the users
|
||||
is not carved in stone. As providing access to such engines
|
||||
implies that the admin knows and trusts the user, we do not see
|
||||
necessary to come up with a strict process. Instead,
|
||||
we would like to add guidelines to the documentation of the feature.
|
||||
|
||||
|
||||
Acknowledgment
|
||||
==============
|
||||
|
||||
This development was sponsored by `Search and Discovery Fund
|
||||
<https://nlnet.nl/discovery>`_ of `NLnet Foundation <https://nlnet.nl/>`_.
|
||||
@ -1,50 +0,0 @@
|
||||
.. _engine recoll:
|
||||
|
||||
=============
|
||||
Recoll Engine
|
||||
=============
|
||||
|
||||
.. sidebar:: info
|
||||
|
||||
- `Recoll <https://www.lesbonscomptes.com/recoll/>`_
|
||||
- `recoll-webui <https://framagit.org/medoc92/recollwebui.git>`_
|
||||
- :origin:`searx/engines/recoll.py`
|
||||
|
||||
Recoll_ is a desktop full-text search tool based on Xapian. By itself Recoll_
|
||||
does not offer WEB or API access, this can be achieved using recoll-webui_
|
||||
|
||||
|
||||
Configuration
|
||||
=============
|
||||
|
||||
You must configure the following settings:
|
||||
|
||||
``base_url``:
|
||||
Location where recoll-webui can be reached.
|
||||
|
||||
``mount_prefix``:
|
||||
Location where the file hierarchy is mounted on your *local* filesystem.
|
||||
|
||||
``dl_prefix``:
|
||||
Location where the file hierarchy as indexed by recoll can be reached.
|
||||
|
||||
``search_dir``:
|
||||
Part of the indexed file hierarchy to be search, if empty the full domain is
|
||||
searched.
|
||||
|
||||
|
||||
Example
|
||||
=======
|
||||
|
||||
Scenario:
|
||||
|
||||
#. Recoll indexes a local filesystem mounted in ``/export/documents/reference``,
|
||||
#. the Recoll search interface can be reached at https://recoll.example.org/ and
|
||||
#. the contents of this filesystem can be reached though https://download.example.org/reference
|
||||
|
||||
.. code:: yaml
|
||||
|
||||
base_url: https://recoll.example.org/
|
||||
mount_prefix: /export/documents
|
||||
dl_prefix: https://download.example.org
|
||||
search_dir: ''
|
||||
@ -1,136 +0,0 @@
|
||||
====================
|
||||
Local Search Engines
|
||||
====================
|
||||
|
||||
.. sidebar:: further read
|
||||
|
||||
- `Comparison to alternatives
|
||||
<https://docs.meilisearch.com/learn/what_is_meilisearch/comparison_to_alternatives.html>`_
|
||||
|
||||
Administrators might find themselves wanting to integrate locally running search
|
||||
engines. The following ones are supported for now:
|
||||
|
||||
* `Elasticsearch`_
|
||||
* `Meilisearch`_
|
||||
* `Solr`_
|
||||
|
||||
Each search engine is powerful, capable of full-text search. All of the engines
|
||||
above are added to ``settings.yml`` just commented out, as you have to
|
||||
``base_url`` for all them.
|
||||
|
||||
Please note that if you are not using HTTPS to access these engines, you have to enable
|
||||
HTTP requests by setting ``enable_http`` to ``True``.
|
||||
|
||||
Furthermore, if you do not want to expose these engines on a public instance, you
|
||||
can still add them and limit the access by setting ``tokens`` as described in
|
||||
section :ref:`private engines`.
|
||||
|
||||
.. _engine meilisearch:
|
||||
|
||||
MeiliSearch
|
||||
===========
|
||||
|
||||
.. sidebar:: info
|
||||
|
||||
- :origin:`meilisearch.py <searx/engines/meilisearch.py>`
|
||||
- `MeiliSearch <https://www.meilisearch.com>`_
|
||||
- `MeiliSearch Documentation <https://docs.meilisearch.com/>`_
|
||||
- `Install MeiliSearch
|
||||
<https://docs.meilisearch.com/learn/getting_started/installation.html>`_
|
||||
|
||||
MeiliSearch_ is aimed at individuals and small companies. It is designed for
|
||||
small-scale (less than 10 million documents) data collections. E.g. it is great
|
||||
for storing web pages you have visited and searching in the contents later.
|
||||
|
||||
The engine supports faceted search, so you can search in a subset of documents
|
||||
of the collection. Furthermore, you can search in MeiliSearch_ instances that
|
||||
require authentication by setting ``auth_token``.
|
||||
|
||||
Here is a simple example to query a Meilisearch instance:
|
||||
|
||||
.. code:: yaml
|
||||
|
||||
- name: meilisearch
|
||||
engine: meilisearch
|
||||
shortcut: mes
|
||||
base_url: http://localhost:7700
|
||||
index: my-index
|
||||
enable_http: true
|
||||
|
||||
|
||||
.. _engine elasticsearch:
|
||||
|
||||
Elasticsearch
|
||||
=============
|
||||
|
||||
.. sidebar:: info
|
||||
|
||||
- :origin:`elasticsearch.py <searx/engines/elasticsearch.py>`
|
||||
- `Elasticsearch <https://www.elastic.co/elasticsearch/>`_
|
||||
- `Elasticsearch Guide
|
||||
<https://www.elastic.co/guide/en/elasticsearch/reference/current/index.html>`_
|
||||
- `Install Elasticsearch
|
||||
<https://www.elastic.co/guide/en/elasticsearch/reference/current/install-elasticsearch.html>`_
|
||||
|
||||
Elasticsearch_ supports numerous ways to query the data it is storing. At the
|
||||
moment the engine supports the most popular search methods (``query_type``):
|
||||
|
||||
- ``match``,
|
||||
- ``simple_query_string``,
|
||||
- ``term`` and
|
||||
- ``terms``.
|
||||
|
||||
If none of the methods fit your use case, you can select ``custom`` query type
|
||||
and provide the JSON payload to submit to Elasticsearch in
|
||||
``custom_query_json``.
|
||||
|
||||
The following is an example configuration for an Elasticsearch_ instance with
|
||||
authentication configured to read from ``my-index`` index.
|
||||
|
||||
.. code:: yaml
|
||||
|
||||
- name: elasticsearch
|
||||
shortcut: es
|
||||
engine: elasticsearch
|
||||
base_url: http://localhost:9200
|
||||
username: elastic
|
||||
password: changeme
|
||||
index: my-index
|
||||
query_type: match
|
||||
# custom_query_json: '{ ... }'
|
||||
enable_http: true
|
||||
|
||||
.. _engine solr:
|
||||
|
||||
Solr
|
||||
====
|
||||
|
||||
.. sidebar:: info
|
||||
|
||||
- :origin:`solr.py <searx/engines/solr.py>`
|
||||
- `Solr <https://solr.apache.org>`_
|
||||
- `Solr Resources <https://solr.apache.org/resources.html>`_
|
||||
- `Install Solr <https://solr.apache.org/guide/installing-solr.html>`_
|
||||
|
||||
Solr_ is a popular search engine based on Lucene, just like Elasticsearch_. But
|
||||
instead of searching in indices, you can search in collections.
|
||||
|
||||
This is an example configuration for searching in the collection
|
||||
``my-collection`` and get the results in ascending order.
|
||||
|
||||
.. code:: yaml
|
||||
|
||||
- name: solr
|
||||
engine: solr
|
||||
shortcut: slr
|
||||
base_url: http://localhost:8983
|
||||
collection: my-collection
|
||||
sort: asc
|
||||
enable_http: true
|
||||
|
||||
|
||||
Acknowledgment
|
||||
==============
|
||||
|
||||
This development was sponsored by `Search and Discovery Fund
|
||||
<https://nlnet.nl/discovery>`_ of `NLnet Foundation <https://nlnet.nl/>`_.
|
||||
@ -1,778 +0,0 @@
|
||||
.. _settings.yml:
|
||||
|
||||
================
|
||||
``settings.yml``
|
||||
================
|
||||
|
||||
This page describe the options possibilities of the :origin:`searx/settings.yml`
|
||||
file.
|
||||
|
||||
.. sidebar:: Further reading ..
|
||||
|
||||
- :ref:`use_default_settings.yml`
|
||||
- :ref:`search API`
|
||||
|
||||
.. contents:: Contents
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
.. _settings location:
|
||||
|
||||
settings.yml location
|
||||
=====================
|
||||
|
||||
The initial ``settings.yml`` we be load from these locations:
|
||||
|
||||
1. the full path specified in the ``SEARXNG_SETTINGS_PATH`` environment variable.
|
||||
2. ``/etc/searxng/settings.yml``
|
||||
|
||||
If these files don't exist (or are empty or can't be read), SearXNG uses the
|
||||
:origin:`searx/settings.yml` file. Read :ref:`settings use_default_settings` to
|
||||
see how you can simplify your *user defined* ``settings.yml``.
|
||||
|
||||
|
||||
.. _settings global:
|
||||
|
||||
Global Settings
|
||||
===============
|
||||
|
||||
.. _settings brand:
|
||||
|
||||
``brand:``
|
||||
----------
|
||||
|
||||
.. code:: yaml
|
||||
|
||||
brand:
|
||||
issue_url: https://github.com/searxng/searxng/issues
|
||||
docs_url: https://docs.searxng.org
|
||||
public_instances: https://searx.space
|
||||
wiki_url: https://github.com/searxng/searxng/wiki
|
||||
|
||||
``issue_url`` :
|
||||
If you host your own issue tracker change this URL.
|
||||
|
||||
``docs_url`` :
|
||||
If you host your own documentation change this URL.
|
||||
|
||||
``public_instances`` :
|
||||
If you host your own https://searx.space change this URL.
|
||||
|
||||
``wiki_url`` :
|
||||
Link to your wiki (or ``false``)
|
||||
|
||||
.. _settings general:
|
||||
|
||||
``general:``
|
||||
------------
|
||||
|
||||
.. code:: yaml
|
||||
|
||||
general:
|
||||
debug: false
|
||||
instance_name: "SearXNG"
|
||||
privacypolicy_url: false
|
||||
donation_url: false
|
||||
contact_url: false
|
||||
enable_metrics: true
|
||||
|
||||
``debug`` : ``$SEARXNG_DEBUG``
|
||||
Allow a more detailed log if you run SearXNG directly. Display *detailed* error
|
||||
messages in the browser too, so this must be deactivated in production.
|
||||
|
||||
``donation_url`` :
|
||||
Set value to ``true`` to use your own donation page written in the
|
||||
:ref:`searx/info/en/donate.md <searx.infopage>` and use ``false`` to disable
|
||||
the donation link altogether.
|
||||
|
||||
``privacypolicy_url``:
|
||||
Link to privacy policy.
|
||||
|
||||
``contact_url``:
|
||||
Contact ``mailto:`` address or WEB form.
|
||||
|
||||
``enable_metrics``:
|
||||
Enabled by default. Record various anonymous metrics availabled at ``/stats``,
|
||||
``/stats/errors`` and ``/preferences``.
|
||||
|
||||
.. _settings search:
|
||||
|
||||
``search:``
|
||||
-----------
|
||||
|
||||
.. code:: yaml
|
||||
|
||||
search:
|
||||
safe_search: 0
|
||||
autocomplete: ""
|
||||
default_lang: ""
|
||||
ban_time_on_fail: 5
|
||||
max_ban_time_on_fail: 120
|
||||
suspended_times:
|
||||
SearxEngineAccessDenied: 86400
|
||||
SearxEngineCaptcha: 86400
|
||||
SearxEngineTooManyRequests: 3600
|
||||
cf_SearxEngineCaptcha: 1296000
|
||||
cf_SearxEngineAccessDenied: 86400
|
||||
recaptcha_SearxEngineCaptcha: 604800
|
||||
formats:
|
||||
- html
|
||||
|
||||
``safe_search``:
|
||||
Filter results.
|
||||
|
||||
- ``0``: None
|
||||
- ``1``: Moderate
|
||||
- ``2``: Strict
|
||||
|
||||
``autocomplete``:
|
||||
Existing autocomplete backends, leave blank to turn it off.
|
||||
|
||||
- ``dbpedia``
|
||||
- ``duckduckgo``
|
||||
- ``google``
|
||||
- ``startpage``
|
||||
- ``swisscows``
|
||||
- ``qwant``
|
||||
- ``wikipedia``
|
||||
|
||||
``default_lang``:
|
||||
Default search language - leave blank to detect from browser information or
|
||||
use codes from :origin:`searx/languages.py`.
|
||||
|
||||
``languages``:
|
||||
List of available languages - leave unset to use all codes from
|
||||
:origin:`searx/languages.py`. Otherwise list codes of available languages.
|
||||
The ``all`` value is shown as the ``Default language`` in the user interface
|
||||
(in most cases, it is meant to send the query without a language parameter ;
|
||||
in some cases, it means the English language) Example:
|
||||
|
||||
.. code:: yaml
|
||||
|
||||
languages:
|
||||
- all
|
||||
- en
|
||||
- en-US
|
||||
- de
|
||||
- it-IT
|
||||
- fr
|
||||
- fr-BE
|
||||
|
||||
``ban_time_on_fail``:
|
||||
Ban time in seconds after engine errors.
|
||||
|
||||
``max_ban_time_on_fail``:
|
||||
Max ban time in seconds after engine errors.
|
||||
|
||||
``suspended_times``:
|
||||
Engine suspension time after error (in seconds; set to 0 to disable)
|
||||
|
||||
``SearxEngineAccessDenied``: 86400
|
||||
For error "Access denied" and "HTTP error [402, 403]"
|
||||
|
||||
``SearxEngineCaptcha``: 86400
|
||||
For error "CAPTCHA"
|
||||
|
||||
``SearxEngineTooManyRequests``: 3600
|
||||
For error "Too many request" and "HTTP error 429"
|
||||
|
||||
Cloudflare CAPTCHA:
|
||||
- ``cf_SearxEngineCaptcha``: 1296000
|
||||
- ``cf_SearxEngineAccessDenied``: 86400
|
||||
|
||||
Google CAPTCHA:
|
||||
- ``recaptcha_SearxEngineCaptcha``: 604800
|
||||
|
||||
``formats``:
|
||||
Result formats available from web, remove format to deny access (use lower
|
||||
case).
|
||||
|
||||
- ``html``
|
||||
- ``csv``
|
||||
- ``json``
|
||||
- ``rss``
|
||||
|
||||
|
||||
.. _settings server:
|
||||
|
||||
``server:``
|
||||
-----------
|
||||
|
||||
.. code:: yaml
|
||||
|
||||
server:
|
||||
base_url: http://example.org/location # change this!
|
||||
port: 8888
|
||||
bind_address: "127.0.0.1"
|
||||
secret_key: "ultrasecretkey" # change this!
|
||||
limiter: false
|
||||
image_proxy: false
|
||||
default_http_headers:
|
||||
X-Content-Type-Options : nosniff
|
||||
X-XSS-Protection : 1; mode=block
|
||||
X-Download-Options : noopen
|
||||
X-Robots-Tag : noindex, nofollow
|
||||
Referrer-Policy : no-referrer
|
||||
|
||||
|
||||
``base_url`` : ``$SEARXNG_URL`` :ref:`buildenv <make buildenv>`
|
||||
The base URL where SearXNG is deployed. Used to create correct inbound links.
|
||||
If you change the value, don't forget to rebuild instance's environment
|
||||
(:ref:`utils/brand.env <make buildenv>`)
|
||||
|
||||
``port`` & ``bind_address``: ``$SEARXNG_PORT`` & ``$SEARXNG_BIND_ADDRESS`` :ref:`buildenv <make buildenv>`
|
||||
Port number and *bind address* of the SearXNG web application if you run it
|
||||
directly using ``python searx/webapp.py``. Doesn't apply to a SearXNG
|
||||
services running behind a proxy and using socket communications. If you
|
||||
change the value, don't forget to rebuild instance's environment
|
||||
(:ref:`utils/brand.env <make buildenv>`)
|
||||
|
||||
``secret_key`` : ``$SEARXNG_SECRET``
|
||||
Used for cryptography purpose.
|
||||
|
||||
.. _limiter:
|
||||
|
||||
``limiter`` :
|
||||
Rate limit the number of request on the instance, block some bots. The
|
||||
:ref:`limiter src` requires a :ref:`settings redis` database.
|
||||
|
||||
.. _image_proxy:
|
||||
|
||||
``image_proxy`` :
|
||||
Allow your instance of SearXNG of being able to proxy images. Uses memory space.
|
||||
|
||||
.. _HTTP headers: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers
|
||||
|
||||
``default_http_headers`` :
|
||||
Set additional HTTP headers, see `#755 <https://github.com/searx/searx/issues/715>`__
|
||||
|
||||
|
||||
.. _settings ui:
|
||||
|
||||
``ui:``
|
||||
-------
|
||||
|
||||
.. _cache busting:
|
||||
https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Cache-Control#caching_static_assets_with_cache_busting
|
||||
|
||||
.. code:: yaml
|
||||
|
||||
ui:
|
||||
static_use_hash: false
|
||||
default_locale: ""
|
||||
query_in_title: false
|
||||
infinite_scroll: false
|
||||
center_alignment: false
|
||||
cache_url: https://web.archive.org/web/
|
||||
default_theme: simple
|
||||
theme_args:
|
||||
simple_style: auto
|
||||
|
||||
.. _static_use_hash:
|
||||
|
||||
``static_use_hash`` :
|
||||
Enables `cache busting`_ of static files.
|
||||
|
||||
``default_locale`` :
|
||||
SearXNG interface language. If blank, the locale is detected by using the
|
||||
browser language. If it doesn't work, or you are deploying a language
|
||||
specific instance of searx, a locale can be defined using an ISO language
|
||||
code, like ``fr``, ``en``, ``de``.
|
||||
|
||||
``query_in_title`` :
|
||||
When true, the result page's titles contains the query it decreases the
|
||||
privacy, since the browser can records the page titles.
|
||||
|
||||
``infinite_scroll``:
|
||||
When true, automatically loads the next page when scrolling to bottom of the current page.
|
||||
|
||||
``center_alignment`` : default ``false``
|
||||
When enabled, the results are centered instead of being in the left (or RTL)
|
||||
side of the screen. This setting only affects the *desktop layout*
|
||||
(:origin:`min-width: @tablet <searx/static/themes/simple/src/less/definitions.less>`)
|
||||
|
||||
.. cache_url:
|
||||
|
||||
``cache_url`` : ``https://web.archive.org/web/``
|
||||
URL prefix of the internet archive or cache, don't forgett trailing slash (if
|
||||
needed). The default is https://web.archive.org/web/ alternatives are:
|
||||
|
||||
- https://webcache.googleusercontent.com/search?q=cache:
|
||||
- https://archive.today/
|
||||
|
||||
``default_theme`` :
|
||||
Name of the theme you want to use by default on your SearXNG instance.
|
||||
|
||||
``theme_args.simple_style``:
|
||||
Style of simple theme: ``auto``, ``light``, ``dark``
|
||||
|
||||
``results_on_new_tab``:
|
||||
Open result links in a new tab by default.
|
||||
|
||||
.. _settings redis:
|
||||
|
||||
``redis:``
|
||||
----------
|
||||
|
||||
.. _Redis.from_url(url): https://redis-py.readthedocs.io/en/stable/connections.html#redis.client.Redis.from_url
|
||||
|
||||
A redis DB can be connected by an URL, in :py:obj:`searx.redisdb` you
|
||||
will find a description to test your redis connection in SerXNG. When using
|
||||
sockets, don't forget to check the access rights on the socket::
|
||||
|
||||
ls -la /usr/local/searxng-redis/run/redis.sock
|
||||
srwxrwx--- 1 searxng-redis searxng-redis ... /usr/local/searxng-redis/run/redis.sock
|
||||
|
||||
In this example read/write access is given to the *searxng-redis* group. To get
|
||||
access rights to redis instance (the socket), your SearXNG (or even your
|
||||
developer) account needs to be added to the *searxng-redis* group.
|
||||
|
||||
``url`` : ``$SEARXNG_REDIS_URL``
|
||||
URL to connect redis database, see `Redis.from_url(url)`_ & :ref:`redis db`::
|
||||
|
||||
redis://[[username]:[password]]@localhost:6379/0
|
||||
rediss://[[username]:[password]]@localhost:6379/0
|
||||
unix://[[username]:[password]]@/path/to/socket.sock?db=0
|
||||
|
||||
.. admonition:: Tip for developers
|
||||
|
||||
To set up a local redis instance, first set the socket path of the Redis DB
|
||||
in your YAML setting:
|
||||
|
||||
.. code:: yaml
|
||||
|
||||
redis:
|
||||
url: unix:///usr/local/searxng-redis/run/redis.sock?db=0
|
||||
|
||||
Then use the following commands to install the redis instance ::
|
||||
|
||||
$ ./manage redis.build
|
||||
$ sudo -H ./manage redis.install
|
||||
$ sudo -H ./manage redis.addgrp "${USER}"
|
||||
# don't forget to logout & login to get member of group
|
||||
|
||||
|
||||
.. _settings outgoing:
|
||||
|
||||
``outgoing:``
|
||||
-------------
|
||||
|
||||
Communication with search engines.
|
||||
|
||||
.. code:: yaml
|
||||
|
||||
outgoing:
|
||||
request_timeout: 2.0 # default timeout in seconds, can be override by engine
|
||||
max_request_timeout: 10.0 # the maximum timeout in seconds
|
||||
useragent_suffix: "" # information like an email address to the administrator
|
||||
pool_connections: 100 # Maximum number of allowable connections, or null
|
||||
# for no limits. The default is 100.
|
||||
pool_maxsize: 10 # Number of allowable keep-alive connections, or null
|
||||
# to always allow. The default is 10.
|
||||
enable_http2: true # See https://www.python-httpx.org/http2/
|
||||
# uncomment below section if you want to use a custom server certificate
|
||||
# see https://www.python-httpx.org/advanced/#changing-the-verification-defaults
|
||||
# and https://www.python-httpx.org/compatibility/#ssl-configuration
|
||||
# verify: ~/.mitmproxy/mitmproxy-ca-cert.cer
|
||||
#
|
||||
# uncomment below section if you want to use a proxyq see: SOCKS proxies
|
||||
# https://2.python-requests.org/en/latest/user/advanced/#proxies
|
||||
# are also supported: see
|
||||
# https://2.python-requests.org/en/latest/user/advanced/#socks
|
||||
#
|
||||
# proxies:
|
||||
# all://:
|
||||
# - http://proxy1:8080
|
||||
# - http://proxy2:8080
|
||||
#
|
||||
# using_tor_proxy: true
|
||||
#
|
||||
# Extra seconds to add in order to account for the time taken by the proxy
|
||||
#
|
||||
# extra_proxy_timeout: 10.0
|
||||
#
|
||||
|
||||
``request_timeout`` :
|
||||
Global timeout of the requests made to others engines in seconds. A bigger
|
||||
timeout will allow to wait for answers from slow engines, but in consequence
|
||||
will slow SearXNG reactivity (the result page may take the time specified in the
|
||||
timeout to load). Can be override by ``timeout`` in the :ref:`settings engine`.
|
||||
|
||||
``useragent_suffix`` :
|
||||
Suffix to the user-agent SearXNG uses to send requests to others engines. If an
|
||||
engine wish to block you, a contact info here may be useful to avoid that.
|
||||
|
||||
.. _Pool limit configuration: https://www.python-httpx.org/advanced/#pool-limit-configuration
|
||||
|
||||
``pool_maxsize``:
|
||||
Number of allowable keep-alive connections, or ``null`` to always allow. The
|
||||
default is 10. See ``max_keepalive_connections`` `Pool limit configuration`_.
|
||||
|
||||
``pool_connections`` :
|
||||
Maximum number of allowable connections, or ``null`` # for no limits. The
|
||||
default is 100. See ``max_connections`` `Pool limit configuration`_.
|
||||
|
||||
``keepalive_expiry`` :
|
||||
Number of seconds to keep a connection in the pool. By default 5.0 seconds.
|
||||
See ``keepalive_expiry`` `Pool limit configuration`_.
|
||||
|
||||
|
||||
.. _httpx proxies: https://www.python-httpx.org/advanced/#http-proxying
|
||||
|
||||
``proxies`` :
|
||||
Define one or more proxies you wish to use, see `httpx proxies`_.
|
||||
If there are more than one proxy for one protocol (http, https),
|
||||
requests to the engines are distributed in a round-robin fashion.
|
||||
|
||||
``source_ips`` :
|
||||
If you use multiple network interfaces, define from which IP the requests must
|
||||
be made. Example:
|
||||
|
||||
* ``0.0.0.0`` any local IPv4 address.
|
||||
* ``::`` any local IPv6 address.
|
||||
* ``192.168.0.1``
|
||||
* ``[ 192.168.0.1, 192.168.0.2 ]`` these two specific IP addresses
|
||||
* ``fe80::60a2:1691:e5a2:ee1f``
|
||||
* ``fe80::60a2:1691:e5a2:ee1f/126`` all IP addresses in this network.
|
||||
* ``[ 192.168.0.1, fe80::/126 ]``
|
||||
|
||||
``retries`` :
|
||||
Number of retry in case of an HTTP error. On each retry, SearXNG uses an
|
||||
different proxy and source ip.
|
||||
|
||||
``enable_http2`` :
|
||||
Enable by default. Set to ``false`` to disable HTTP/2.
|
||||
|
||||
.. _httpx verification defaults: https://www.python-httpx.org/advanced/#changing-the-verification-defaults
|
||||
.. _httpx ssl configuration: https://www.python-httpx.org/compatibility/#ssl-configuration
|
||||
|
||||
``verify``: : ``$SSL_CERT_FILE``, ``$SSL_CERT_DIR``
|
||||
Allow to specify a path to certificate.
|
||||
see `httpx verification defaults`_.
|
||||
|
||||
In addition to ``verify``, SearXNG supports the ``$SSL_CERT_FILE`` (for a file) and
|
||||
``$SSL_CERT_DIR`` (for a directory) OpenSSL variables.
|
||||
see `httpx ssl configuration`_.
|
||||
|
||||
``max_redirects`` :
|
||||
30 by default. Maximum redirect before it is an error.
|
||||
|
||||
``using_tor_proxy`` :
|
||||
Using tor proxy (``true``) or not (``false``) for all engines. The default is
|
||||
``false`` and can be overwritten in the :ref:`settings engine`
|
||||
|
||||
|
||||
|
||||
.. _settings categories_as_tabs:
|
||||
|
||||
``categories_as_tabs:``
|
||||
-----------------------
|
||||
|
||||
A list of the categories that are displayed as tabs in the user interface.
|
||||
Categories not listed here can still be searched with the :ref:`search-syntax`.
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
categories_as_tabs:
|
||||
general:
|
||||
images:
|
||||
videos:
|
||||
news:
|
||||
map:
|
||||
music:
|
||||
it:
|
||||
science:
|
||||
files:
|
||||
social media:
|
||||
|
||||
Engines are added to ``categories:`` (compare :ref:`engine categories`), the
|
||||
categories listed in ``categories_as_tabs`` are shown as tabs in the UI. If
|
||||
there are no active engines in a category, the tab is not displayed (e.g. if a
|
||||
user disables all engines in a category).
|
||||
|
||||
On the preferences page (``/preferences``) -- under *engines* -- there is an
|
||||
additional tab, called *other*. In this tab are all engines listed that are not
|
||||
in one of the UI tabs (not included in ``categories_as_tabs``).
|
||||
|
||||
.. _settings engine:
|
||||
|
||||
Engine settings
|
||||
===============
|
||||
|
||||
.. sidebar:: Further reading ..
|
||||
|
||||
- :ref:`configured engines`
|
||||
- :ref:`engines-dev`
|
||||
|
||||
In the code example below a *full fledged* example of a YAML setup from a dummy
|
||||
engine is shown. Most of the options have a default value or even are optional.
|
||||
|
||||
.. code:: yaml
|
||||
|
||||
- name: example engine
|
||||
engine: example
|
||||
shortcut: demo
|
||||
base_url: 'https://{language}.example.com/'
|
||||
send_accept_language_header: false
|
||||
categories: general
|
||||
timeout: 3.0
|
||||
api_key: 'apikey'
|
||||
disabled: false
|
||||
language: en_US
|
||||
tokens: [ 'my-secret-token' ]
|
||||
weight: 1
|
||||
display_error_messages: true
|
||||
about:
|
||||
website: https://example.com
|
||||
wikidata_id: Q306656
|
||||
official_api_documentation: https://example.com/api-doc
|
||||
use_official_api: true
|
||||
require_api_key: true
|
||||
results: HTML
|
||||
|
||||
# overwrite values from section 'outgoing:'
|
||||
enable_http2: false
|
||||
retries: 1
|
||||
max_connections: 100
|
||||
max_keepalive_connections: 10
|
||||
keepalive_expiry: 5.0
|
||||
using_tor_proxy: false
|
||||
proxies:
|
||||
http:
|
||||
- http://proxy1:8080
|
||||
- http://proxy2:8080
|
||||
https:
|
||||
- http://proxy1:8080
|
||||
- http://proxy2:8080
|
||||
- socks5://user:password@proxy3:1080
|
||||
- socks5h://user:password@proxy4:1080
|
||||
|
||||
# other network settings
|
||||
enable_http: false
|
||||
retry_on_http_error: true # or 403 or [404, 429]
|
||||
|
||||
|
||||
``name`` :
|
||||
Name that will be used across SearXNG to define this engine. In settings, on
|
||||
the result page...
|
||||
|
||||
``engine`` :
|
||||
Name of the python file used to handle requests and responses to and from this
|
||||
search engine.
|
||||
|
||||
``shortcut`` :
|
||||
Code used to execute bang requests (in this case using ``!bi``)
|
||||
|
||||
``base_url`` : optional
|
||||
Part of the URL that should be stable across every request. Can be useful to
|
||||
use multiple sites using only one engine, or updating the site URL without
|
||||
touching at the code.
|
||||
|
||||
``send_accept_language_header`` :
|
||||
Several engines that support languages (or regions) deal with the HTTP header
|
||||
``Accept-Language`` to build a response that fits to the locale. When this
|
||||
option is activated, the language (locale) that is selected by the user is used
|
||||
to build and send a ``Accept-Language`` header in the request to the origin
|
||||
search engine.
|
||||
|
||||
.. _engine categories:
|
||||
|
||||
``categories`` : optional
|
||||
Specifies to which categories the engine should be added. Engines can be
|
||||
assigned to multiple categories.
|
||||
|
||||
Categories can be shown as tabs (:ref:`settings categories_as_tabs`) in the
|
||||
UI. A search in a tab (in the UI) will query all engines that are active in
|
||||
this tab. In the preferences page (``/preferences``) -- under *engines* --
|
||||
users can select what engine should be active when querying in this tab.
|
||||
|
||||
Alternatively, :ref:`\!bang <search-syntax>` can be used to search all engines
|
||||
in a category, regardless of whether they are active or not, or whether they
|
||||
are in a tab of the UI or not. For example, ``!dictionaries`` can be used to
|
||||
query all search engines in that category (group).
|
||||
|
||||
``timeout`` : optional
|
||||
Timeout of the search with the current search engine. Overwrites
|
||||
``request_timeout`` from :ref:`settings outgoing`. **Be careful, it will
|
||||
modify the global timeout of SearXNG.**
|
||||
|
||||
``api_key`` : optional
|
||||
In a few cases, using an API needs the use of a secret key. How to obtain them
|
||||
is described in the file.
|
||||
|
||||
``disabled`` : optional
|
||||
To disable by default the engine, but not deleting it. It will allow the user
|
||||
to manually activate it in the settings.
|
||||
|
||||
``inactive``: optional
|
||||
Remove the engine from the settings (*disabled & removed*).
|
||||
|
||||
``language`` : optional
|
||||
If you want to use another language for a specific engine, you can define it
|
||||
by using the ISO code of language (and region), like ``fr``, ``en-US``,
|
||||
``de-DE``.
|
||||
|
||||
``tokens`` : optional
|
||||
A list of secret tokens to make this engine *private*, more details see
|
||||
:ref:`private engines`.
|
||||
|
||||
``weight`` : default ``1``
|
||||
Weighting of the results of this engine.
|
||||
|
||||
``display_error_messages`` : default ``true``
|
||||
When an engine returns an error, the message is displayed on the user interface.
|
||||
|
||||
``network`` : optional
|
||||
Use the network configuration from another engine.
|
||||
In addition, there are two default networks:
|
||||
|
||||
- ``ipv4`` set ``local_addresses`` to ``0.0.0.0`` (use only IPv4 local addresses)
|
||||
- ``ipv6`` set ``local_addresses`` to ``::`` (use only IPv6 local addresses)
|
||||
|
||||
``enable_http`` : optional
|
||||
Enable HTTP for this engine (by default only HTTPS is enabled).
|
||||
|
||||
``retry_on_http_error`` : optional
|
||||
Retry request on some HTTP status code.
|
||||
|
||||
Example:
|
||||
|
||||
* ``true`` : on HTTP status code between 400 and 599.
|
||||
* ``403`` : on HTTP status code 403.
|
||||
* ``[403, 429]``: on HTTP status code 403 and 429.
|
||||
|
||||
``proxies`` :
|
||||
Overwrites proxy settings from :ref:`settings outgoing`.
|
||||
|
||||
``using_tor_proxy`` :
|
||||
Using tor proxy (``true``) or not (``false``) for this engine. The default is
|
||||
taken from ``using_tor_proxy`` of the :ref:`settings outgoing`.
|
||||
|
||||
``max_keepalive_connection#s`` :
|
||||
`Pool limit configuration`_, overwrites value ``pool_maxsize`` from
|
||||
:ref:`settings outgoing` for this engine.
|
||||
|
||||
``max_connections`` :
|
||||
`Pool limit configuration`_, overwrites value ``pool_connections`` from
|
||||
:ref:`settings outgoing` for this engine.
|
||||
|
||||
``keepalive_expiry`` :
|
||||
`Pool limit configuration`_, overwrites value ``keepalive_expiry`` from
|
||||
:ref:`settings outgoing` for this engine.
|
||||
|
||||
.. note::
|
||||
|
||||
A few more options are possible, but they are pretty specific to some
|
||||
engines, and so won't be described here.
|
||||
|
||||
|
||||
Example: Multilingual Search
|
||||
----------------------------
|
||||
|
||||
SearXNG does not support true multilingual search. You have to use the language
|
||||
prefix in your search query when searching in a different language.
|
||||
|
||||
But there is a workaround: By adding a new search engine with a different
|
||||
language, SearXNG will search in your default and other language.
|
||||
|
||||
Example configuration in settings.yml for a German and English speaker:
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
search:
|
||||
default_lang : "de"
|
||||
...
|
||||
|
||||
engines:
|
||||
- name : google english
|
||||
engine : google
|
||||
language : en
|
||||
...
|
||||
|
||||
When searching, the default google engine will return German results and
|
||||
"google english" will return English results.
|
||||
|
||||
|
||||
.. _settings use_default_settings:
|
||||
|
||||
use_default_settings
|
||||
====================
|
||||
|
||||
.. sidebar:: ``use_default_settings: true``
|
||||
|
||||
- :ref:`settings location`
|
||||
- :ref:`use_default_settings.yml`
|
||||
- :origin:`/etc/searxng/settings.yml <utils/templates/etc/searxng/settings.yml>`
|
||||
|
||||
The user defined ``settings.yml`` is loaded from the :ref:`settings location`
|
||||
and can relied on the default configuration :origin:`searx/settings.yml` using:
|
||||
|
||||
``use_default_settings: true``
|
||||
|
||||
``server:``
|
||||
In the following example, the actual settings are the default settings defined
|
||||
in :origin:`searx/settings.yml` with the exception of the ``secret_key`` and
|
||||
the ``bind_address``:
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
use_default_settings: true
|
||||
server:
|
||||
secret_key: "ultrasecretkey" # change this!
|
||||
bind_address: "0.0.0.0"
|
||||
|
||||
``engines:``
|
||||
With ``use_default_settings: true``, each settings can be override in a
|
||||
similar way, the ``engines`` section is merged according to the engine
|
||||
``name``. In this example, SearXNG will load all the default engines, will
|
||||
enable the ``bing`` engine and define a :ref:`token <private engines>` for
|
||||
the arch linux engine:
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
use_default_settings: true
|
||||
server:
|
||||
secret_key: "ultrasecretkey" # change this!
|
||||
engines:
|
||||
- name: arch linux wiki
|
||||
tokens: ['$ecretValue']
|
||||
- name: bing
|
||||
disabled: false
|
||||
|
||||
|
||||
``engines:`` / ``remove:``
|
||||
It is possible to remove some engines from the default settings. The following
|
||||
example is similar to the above one, but SearXNG doesn't load the the google
|
||||
engine:
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
use_default_settings:
|
||||
engines:
|
||||
remove:
|
||||
- google
|
||||
server:
|
||||
secret_key: "ultrasecretkey" # change this!
|
||||
engines:
|
||||
- name: arch linux wiki
|
||||
tokens: ['$ecretValue']
|
||||
|
||||
``engines:`` / ``keep_only:``
|
||||
As an alternative, it is possible to specify the engines to keep. In the
|
||||
following example, SearXNG has only two engines:
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
use_default_settings:
|
||||
engines:
|
||||
keep_only:
|
||||
- google
|
||||
- duckduckgo
|
||||
server:
|
||||
secret_key: "ultrasecretkey" # change this!
|
||||
engines:
|
||||
- name: google
|
||||
tokens: ['$ecretValue']
|
||||
- name: duckduckgo
|
||||
tokens: ['$ecretValue']
|
||||
@ -4,8 +4,8 @@ Administrator documentation
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
:caption: Contents
|
||||
|
||||
settings/index
|
||||
installation
|
||||
installation-docker
|
||||
installation-scripts
|
||||
@ -15,7 +15,6 @@ Administrator documentation
|
||||
installation-apache
|
||||
update-searxng
|
||||
answer-captcha
|
||||
engines/index
|
||||
api
|
||||
architecture
|
||||
plugins
|
||||
|
||||
@ -61,7 +61,7 @@ section might give you some guidance.
|
||||
- `Apache Fedora`_
|
||||
- `Apache directives`_
|
||||
|
||||
.. contents:: Contents
|
||||
.. contents::
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
@ -41,7 +41,7 @@ section might give you some guidance.
|
||||
- `uWSGI support from nginx`_
|
||||
|
||||
|
||||
.. contents:: Contents
|
||||
.. contents::
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
@ -4,7 +4,7 @@
|
||||
Step by step installation
|
||||
=========================
|
||||
|
||||
.. contents:: Contents
|
||||
.. contents::
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
@ -73,7 +73,7 @@ Configuration
|
||||
|
||||
.. sidebar:: ``use_default_settings: True``
|
||||
|
||||
- :ref:`settings global`
|
||||
- :ref:`settings.yml`
|
||||
- :ref:`settings location`
|
||||
- :ref:`settings use_default_settings`
|
||||
- :origin:`/etc/searxng/settings.yml <utils/templates/etc/searxng/settings.yml>`
|
||||
|
||||
@ -9,7 +9,7 @@ uWSGI
|
||||
- `systemd.unit`_
|
||||
- `uWSGI Emperor`_
|
||||
|
||||
.. contents:: Contents
|
||||
.. contents::
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
25
docs/admin/settings/index.rst
Normal file
25
docs/admin/settings/index.rst
Normal file
@ -0,0 +1,25 @@
|
||||
========
|
||||
Settings
|
||||
========
|
||||
|
||||
.. sidebar:: Further reading ..
|
||||
|
||||
- :ref:`engine settings`
|
||||
- :ref:`engine file`
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
|
||||
settings
|
||||
settings_engine
|
||||
settings_brand
|
||||
settings_general
|
||||
settings_search
|
||||
settings_server
|
||||
settings_ui
|
||||
settings_redis
|
||||
settings_outgoing
|
||||
settings_categories_as_tabs
|
||||
|
||||
|
||||
|
||||
117
docs/admin/settings/settings.rst
Normal file
117
docs/admin/settings/settings.rst
Normal file
@ -0,0 +1,117 @@
|
||||
.. _settings.yml:
|
||||
|
||||
================
|
||||
``settings.yml``
|
||||
================
|
||||
|
||||
This page describe the options possibilities of the :origin:`searx/settings.yml`
|
||||
file.
|
||||
|
||||
.. sidebar:: Further reading ..
|
||||
|
||||
- :ref:`use_default_settings.yml`
|
||||
- :ref:`search API`
|
||||
|
||||
.. contents::
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
.. _settings location:
|
||||
|
||||
settings.yml location
|
||||
=====================
|
||||
|
||||
The initial ``settings.yml`` we be load from these locations:
|
||||
|
||||
1. the full path specified in the ``SEARXNG_SETTINGS_PATH`` environment variable.
|
||||
2. ``/etc/searxng/settings.yml``
|
||||
|
||||
If these files don't exist (or are empty or can't be read), SearXNG uses the
|
||||
:origin:`searx/settings.yml` file. Read :ref:`settings use_default_settings` to
|
||||
see how you can simplify your *user defined* ``settings.yml``.
|
||||
|
||||
|
||||
|
||||
.. _settings use_default_settings:
|
||||
|
||||
use_default_settings
|
||||
====================
|
||||
|
||||
.. sidebar:: ``use_default_settings: true``
|
||||
|
||||
- :ref:`settings location`
|
||||
- :ref:`use_default_settings.yml`
|
||||
- :origin:`/etc/searxng/settings.yml <utils/templates/etc/searxng/settings.yml>`
|
||||
|
||||
The user defined ``settings.yml`` is loaded from the :ref:`settings location`
|
||||
and can relied on the default configuration :origin:`searx/settings.yml` using:
|
||||
|
||||
``use_default_settings: true``
|
||||
|
||||
``server:``
|
||||
In the following example, the actual settings are the default settings defined
|
||||
in :origin:`searx/settings.yml` with the exception of the ``secret_key`` and
|
||||
the ``bind_address``:
|
||||
|
||||
.. code:: yaml
|
||||
|
||||
use_default_settings: true
|
||||
server:
|
||||
secret_key: "ultrasecretkey" # change this!
|
||||
bind_address: "0.0.0.0"
|
||||
|
||||
``engines:``
|
||||
With ``use_default_settings: true``, each settings can be override in a
|
||||
similar way, the ``engines`` section is merged according to the engine
|
||||
``name``. In this example, SearXNG will load all the default engines, will
|
||||
enable the ``bing`` engine and define a :ref:`token <private engines>` for
|
||||
the arch linux engine:
|
||||
|
||||
.. code:: yaml
|
||||
|
||||
use_default_settings: true
|
||||
server:
|
||||
secret_key: "ultrasecretkey" # change this!
|
||||
engines:
|
||||
- name: arch linux wiki
|
||||
tokens: ['$ecretValue']
|
||||
- name: bing
|
||||
disabled: false
|
||||
|
||||
|
||||
``engines:`` / ``remove:``
|
||||
It is possible to remove some engines from the default settings. The following
|
||||
example is similar to the above one, but SearXNG doesn't load the the google
|
||||
engine:
|
||||
|
||||
.. code:: yaml
|
||||
|
||||
use_default_settings:
|
||||
engines:
|
||||
remove:
|
||||
- google
|
||||
server:
|
||||
secret_key: "ultrasecretkey" # change this!
|
||||
engines:
|
||||
- name: arch linux wiki
|
||||
tokens: ['$ecretValue']
|
||||
|
||||
``engines:`` / ``keep_only:``
|
||||
As an alternative, it is possible to specify the engines to keep. In the
|
||||
following example, SearXNG has only two engines:
|
||||
|
||||
.. code:: yaml
|
||||
|
||||
use_default_settings:
|
||||
engines:
|
||||
keep_only:
|
||||
- google
|
||||
- duckduckgo
|
||||
server:
|
||||
secret_key: "ultrasecretkey" # change this!
|
||||
engines:
|
||||
- name: google
|
||||
tokens: ['$ecretValue']
|
||||
- name: duckduckgo
|
||||
tokens: ['$ecretValue']
|
||||
25
docs/admin/settings/settings_brand.rst
Normal file
25
docs/admin/settings/settings_brand.rst
Normal file
@ -0,0 +1,25 @@
|
||||
.. _settings brand:
|
||||
|
||||
==========
|
||||
``brand:``
|
||||
==========
|
||||
|
||||
.. code:: yaml
|
||||
|
||||
brand:
|
||||
issue_url: https://github.com/searxng/searxng/issues
|
||||
docs_url: https://docs.searxng.org
|
||||
public_instances: https://searx.space
|
||||
wiki_url: https://github.com/searxng/searxng/wiki
|
||||
|
||||
``issue_url`` :
|
||||
If you host your own issue tracker change this URL.
|
||||
|
||||
``docs_url`` :
|
||||
If you host your own documentation change this URL.
|
||||
|
||||
``public_instances`` :
|
||||
If you host your own https://searx.space change this URL.
|
||||
|
||||
``wiki_url`` :
|
||||
Link to your wiki (or ``false``)
|
||||
31
docs/admin/settings/settings_categories_as_tabs.rst
Normal file
31
docs/admin/settings/settings_categories_as_tabs.rst
Normal file
@ -0,0 +1,31 @@
|
||||
.. _settings categories_as_tabs:
|
||||
|
||||
=======================
|
||||
``categories_as_tabs:``
|
||||
=======================
|
||||
|
||||
A list of the categories that are displayed as tabs in the user interface.
|
||||
Categories not listed here can still be searched with the :ref:`search-syntax`.
|
||||
|
||||
.. code:: yaml
|
||||
|
||||
categories_as_tabs:
|
||||
general:
|
||||
images:
|
||||
videos:
|
||||
news:
|
||||
map:
|
||||
music:
|
||||
it:
|
||||
science:
|
||||
files:
|
||||
social media:
|
||||
|
||||
Engines are added to ``categories:`` (compare :ref:`engine categories`), the
|
||||
categories listed in ``categories_as_tabs`` are shown as tabs in the UI. If
|
||||
there are no active engines in a category, the tab is not displayed (e.g. if a
|
||||
user disables all engines in a category).
|
||||
|
||||
On the preferences page (``/preferences``) -- under *engines* -- there is an
|
||||
additional tab, called *other*. In this tab are all engines listed that are not
|
||||
in one of the UI tabs (not included in ``categories_as_tabs``).
|
||||
244
docs/admin/settings/settings_engine.rst
Normal file
244
docs/admin/settings/settings_engine.rst
Normal file
@ -0,0 +1,244 @@
|
||||
.. _settings engine:
|
||||
|
||||
===========
|
||||
``engine:``
|
||||
===========
|
||||
|
||||
.. sidebar:: Further reading ..
|
||||
|
||||
- :ref:`configured engines`
|
||||
- :ref:`engines-dev`
|
||||
|
||||
In the code example below a *full fledged* example of a YAML setup from a dummy
|
||||
engine is shown. Most of the options have a default value or even are optional.
|
||||
|
||||
.. hint::
|
||||
|
||||
A few more options are possible, but they are pretty specific to some
|
||||
engines (:ref:`engine implementations`).
|
||||
|
||||
.. code:: yaml
|
||||
|
||||
- name: example engine
|
||||
engine: example
|
||||
shortcut: demo
|
||||
base_url: 'https://{language}.example.com/'
|
||||
send_accept_language_header: false
|
||||
categories: general
|
||||
timeout: 3.0
|
||||
api_key: 'apikey'
|
||||
disabled: false
|
||||
language: en_US
|
||||
tokens: [ 'my-secret-token' ]
|
||||
weight: 1
|
||||
display_error_messages: true
|
||||
about:
|
||||
website: https://example.com
|
||||
wikidata_id: Q306656
|
||||
official_api_documentation: https://example.com/api-doc
|
||||
use_official_api: true
|
||||
require_api_key: true
|
||||
results: HTML
|
||||
|
||||
# overwrite values from section 'outgoing:'
|
||||
enable_http2: false
|
||||
retries: 1
|
||||
max_connections: 100
|
||||
max_keepalive_connections: 10
|
||||
keepalive_expiry: 5.0
|
||||
using_tor_proxy: false
|
||||
proxies:
|
||||
http:
|
||||
- http://proxy1:8080
|
||||
- http://proxy2:8080
|
||||
https:
|
||||
- http://proxy1:8080
|
||||
- http://proxy2:8080
|
||||
- socks5://user:password@proxy3:1080
|
||||
- socks5h://user:password@proxy4:1080
|
||||
|
||||
# other network settings
|
||||
enable_http: false
|
||||
retry_on_http_error: true # or 403 or [404, 429]
|
||||
|
||||
|
||||
``name`` :
|
||||
Name that will be used across SearXNG to define this engine. In settings, on
|
||||
the result page...
|
||||
|
||||
``engine`` :
|
||||
Name of the python file used to handle requests and responses to and from this
|
||||
search engine.
|
||||
|
||||
``shortcut`` :
|
||||
Code used to execute bang requests (in this case using ``!bi``)
|
||||
|
||||
``base_url`` : optional
|
||||
Part of the URL that should be stable across every request. Can be useful to
|
||||
use multiple sites using only one engine, or updating the site URL without
|
||||
touching at the code.
|
||||
|
||||
``send_accept_language_header`` :
|
||||
Several engines that support languages (or regions) deal with the HTTP header
|
||||
``Accept-Language`` to build a response that fits to the locale. When this
|
||||
option is activated, the language (locale) that is selected by the user is used
|
||||
to build and send a ``Accept-Language`` header in the request to the origin
|
||||
search engine.
|
||||
|
||||
.. _engine categories:
|
||||
|
||||
``categories`` : optional
|
||||
Specifies to which categories the engine should be added. Engines can be
|
||||
assigned to multiple categories.
|
||||
|
||||
Categories can be shown as tabs (:ref:`settings categories_as_tabs`) in the
|
||||
UI. A search in a tab (in the UI) will query all engines that are active in
|
||||
this tab. In the preferences page (``/preferences``) -- under *engines* --
|
||||
users can select what engine should be active when querying in this tab.
|
||||
|
||||
Alternatively, :ref:`\!bang <search-syntax>` can be used to search all engines
|
||||
in a category, regardless of whether they are active or not, or whether they
|
||||
are in a tab of the UI or not. For example, ``!dictionaries`` can be used to
|
||||
query all search engines in that category (group).
|
||||
|
||||
``timeout`` : optional
|
||||
Timeout of the search with the current search engine. Overwrites
|
||||
``request_timeout`` from :ref:`settings outgoing`. **Be careful, it will
|
||||
modify the global timeout of SearXNG.**
|
||||
|
||||
``api_key`` : optional
|
||||
In a few cases, using an API needs the use of a secret key. How to obtain them
|
||||
is described in the file.
|
||||
|
||||
``disabled`` : optional
|
||||
To disable by default the engine, but not deleting it. It will allow the user
|
||||
to manually activate it in the settings.
|
||||
|
||||
``inactive``: optional
|
||||
Remove the engine from the settings (*disabled & removed*).
|
||||
|
||||
``language`` : optional
|
||||
If you want to use another language for a specific engine, you can define it
|
||||
by using the ISO code of language (and region), like ``fr``, ``en-US``,
|
||||
``de-DE``.
|
||||
|
||||
``tokens`` : optional
|
||||
A list of secret tokens to make this engine *private*, more details see
|
||||
:ref:`private engines`.
|
||||
|
||||
``weight`` : default ``1``
|
||||
Weighting of the results of this engine.
|
||||
|
||||
``display_error_messages`` : default ``true``
|
||||
When an engine returns an error, the message is displayed on the user interface.
|
||||
|
||||
``network`` : optional
|
||||
Use the network configuration from another engine.
|
||||
In addition, there are two default networks:
|
||||
|
||||
- ``ipv4`` set ``local_addresses`` to ``0.0.0.0`` (use only IPv4 local addresses)
|
||||
- ``ipv6`` set ``local_addresses`` to ``::`` (use only IPv6 local addresses)
|
||||
|
||||
``enable_http`` : optional
|
||||
Enable HTTP for this engine (by default only HTTPS is enabled).
|
||||
|
||||
``retry_on_http_error`` : optional
|
||||
Retry request on some HTTP status code.
|
||||
|
||||
Example:
|
||||
|
||||
* ``true`` : on HTTP status code between 400 and 599.
|
||||
* ``403`` : on HTTP status code 403.
|
||||
* ``[403, 429]``: on HTTP status code 403 and 429.
|
||||
|
||||
``proxies`` :
|
||||
Overwrites proxy settings from :ref:`settings outgoing`.
|
||||
|
||||
``using_tor_proxy`` :
|
||||
Using tor proxy (``true``) or not (``false``) for this engine. The default is
|
||||
taken from ``using_tor_proxy`` of the :ref:`settings outgoing`.
|
||||
|
||||
.. _Pool limit configuration: https://www.python-httpx.org/advanced/#pool-limit-configuration
|
||||
|
||||
``max_keepalive_connection#s`` :
|
||||
`Pool limit configuration`_, overwrites value ``pool_maxsize`` from
|
||||
:ref:`settings outgoing` for this engine.
|
||||
|
||||
``max_connections`` :
|
||||
`Pool limit configuration`_, overwrites value ``pool_connections`` from
|
||||
:ref:`settings outgoing` for this engine.
|
||||
|
||||
``keepalive_expiry`` :
|
||||
`Pool limit configuration`_, overwrites value ``keepalive_expiry`` from
|
||||
:ref:`settings outgoing` for this engine.
|
||||
|
||||
|
||||
.. _private engines:
|
||||
|
||||
Private Engines (``tokens``)
|
||||
============================
|
||||
|
||||
Administrators might find themselves wanting to limit access to some of the
|
||||
enabled engines on their instances. It might be because they do not want to
|
||||
expose some private information through :ref:`offline engines`. Or they would
|
||||
rather share engines only with their trusted friends or colleagues.
|
||||
|
||||
.. sidebar:: info
|
||||
|
||||
Initial sponsored by `Search and Discovery Fund
|
||||
<https://nlnet.nl/discovery>`_ of `NLnet Foundation <https://nlnet.nl/>`_.
|
||||
|
||||
To solve this issue the concept of *private engines* exists.
|
||||
|
||||
A new option was added to engines named `tokens`. It expects a list of strings.
|
||||
If the user making a request presents one of the tokens of an engine, they can
|
||||
access information about the engine and make search requests.
|
||||
|
||||
Example configuration to restrict access to the Arch Linux Wiki engine:
|
||||
|
||||
.. code:: yaml
|
||||
|
||||
- name: arch linux wiki
|
||||
engine: archlinux
|
||||
shortcut: al
|
||||
tokens: [ 'my-secret-token' ]
|
||||
|
||||
Unless a user has configured the right token, the engine is going to be hidden
|
||||
from him/her. It is not going to be included in the list of engines on the
|
||||
Preferences page and in the output of `/config` REST API call.
|
||||
|
||||
Tokens can be added to one's configuration on the Preferences page under "Engine
|
||||
tokens". The input expects a comma separated list of strings.
|
||||
|
||||
The distribution of the tokens from the administrator to the users is not carved
|
||||
in stone. As providing access to such engines implies that the admin knows and
|
||||
trusts the user, we do not see necessary to come up with a strict process.
|
||||
Instead, we would like to add guidelines to the documentation of the feature.
|
||||
|
||||
|
||||
Example: Multilingual Search
|
||||
============================
|
||||
|
||||
SearXNG does not support true multilingual search. You have to use the language
|
||||
prefix in your search query when searching in a different language.
|
||||
|
||||
But there is a workaround: By adding a new search engine with a different
|
||||
language, SearXNG will search in your default and other language.
|
||||
|
||||
Example configuration in settings.yml for a German and English speaker:
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
search:
|
||||
default_lang : "de"
|
||||
...
|
||||
|
||||
engines:
|
||||
- name : google english
|
||||
engine : google
|
||||
language : en
|
||||
...
|
||||
|
||||
When searching, the default google engine will return German results and
|
||||
"google english" will return English results.
|
||||
|
||||
34
docs/admin/settings/settings_general.rst
Normal file
34
docs/admin/settings/settings_general.rst
Normal file
@ -0,0 +1,34 @@
|
||||
.. _settings general:
|
||||
|
||||
============
|
||||
``general:``
|
||||
============
|
||||
|
||||
.. code:: yaml
|
||||
|
||||
general:
|
||||
debug: false
|
||||
instance_name: "SearXNG"
|
||||
privacypolicy_url: false
|
||||
donation_url: false
|
||||
contact_url: false
|
||||
enable_metrics: true
|
||||
|
||||
``debug`` : ``$SEARXNG_DEBUG``
|
||||
Allow a more detailed log if you run SearXNG directly. Display *detailed* error
|
||||
messages in the browser too, so this must be deactivated in production.
|
||||
|
||||
``donation_url`` :
|
||||
Set value to ``true`` to use your own donation page written in the
|
||||
:ref:`searx/info/en/donate.md <searx.infopage>` and use ``false`` to disable
|
||||
the donation link altogether.
|
||||
|
||||
``privacypolicy_url``:
|
||||
Link to privacy policy.
|
||||
|
||||
``contact_url``:
|
||||
Contact ``mailto:`` address or WEB form.
|
||||
|
||||
``enable_metrics``:
|
||||
Enabled by default. Record various anonymous metrics availabled at ``/stats``,
|
||||
``/stats/errors`` and ``/preferences``.
|
||||
110
docs/admin/settings/settings_outgoing.rst
Normal file
110
docs/admin/settings/settings_outgoing.rst
Normal file
@ -0,0 +1,110 @@
|
||||
.. _settings outgoing:
|
||||
|
||||
=============
|
||||
``outgoing:``
|
||||
=============
|
||||
|
||||
Communication with search engines.
|
||||
|
||||
.. code:: yaml
|
||||
|
||||
outgoing:
|
||||
request_timeout: 2.0 # default timeout in seconds, can be override by engine
|
||||
max_request_timeout: 10.0 # the maximum timeout in seconds
|
||||
useragent_suffix: "" # information like an email address to the administrator
|
||||
pool_connections: 100 # Maximum number of allowable connections, or null
|
||||
# for no limits. The default is 100.
|
||||
pool_maxsize: 10 # Number of allowable keep-alive connections, or null
|
||||
# to always allow. The default is 10.
|
||||
enable_http2: true # See https://www.python-httpx.org/http2/
|
||||
# uncomment below section if you want to use a custom server certificate
|
||||
# see https://www.python-httpx.org/advanced/#changing-the-verification-defaults
|
||||
# and https://www.python-httpx.org/compatibility/#ssl-configuration
|
||||
# verify: ~/.mitmproxy/mitmproxy-ca-cert.cer
|
||||
#
|
||||
# uncomment below section if you want to use a proxyq see: SOCKS proxies
|
||||
# https://2.python-requests.org/en/latest/user/advanced/#proxies
|
||||
# are also supported: see
|
||||
# https://2.python-requests.org/en/latest/user/advanced/#socks
|
||||
#
|
||||
# proxies:
|
||||
# all://:
|
||||
# - http://proxy1:8080
|
||||
# - http://proxy2:8080
|
||||
#
|
||||
# using_tor_proxy: true
|
||||
#
|
||||
# Extra seconds to add in order to account for the time taken by the proxy
|
||||
#
|
||||
# extra_proxy_timeout: 10.0
|
||||
#
|
||||
|
||||
``request_timeout`` :
|
||||
Global timeout of the requests made to others engines in seconds. A bigger
|
||||
timeout will allow to wait for answers from slow engines, but in consequence
|
||||
will slow SearXNG reactivity (the result page may take the time specified in the
|
||||
timeout to load). Can be override by ``timeout`` in the :ref:`settings engine`.
|
||||
|
||||
``useragent_suffix`` :
|
||||
Suffix to the user-agent SearXNG uses to send requests to others engines. If an
|
||||
engine wish to block you, a contact info here may be useful to avoid that.
|
||||
|
||||
.. _Pool limit configuration: https://www.python-httpx.org/advanced/#pool-limit-configuration
|
||||
|
||||
``pool_maxsize``:
|
||||
Number of allowable keep-alive connections, or ``null`` to always allow. The
|
||||
default is 10. See ``max_keepalive_connections`` `Pool limit configuration`_.
|
||||
|
||||
``pool_connections`` :
|
||||
Maximum number of allowable connections, or ``null`` # for no limits. The
|
||||
default is 100. See ``max_connections`` `Pool limit configuration`_.
|
||||
|
||||
``keepalive_expiry`` :
|
||||
Number of seconds to keep a connection in the pool. By default 5.0 seconds.
|
||||
See ``keepalive_expiry`` `Pool limit configuration`_.
|
||||
|
||||
.. _httpx proxies: https://www.python-httpx.org/advanced/#http-proxying
|
||||
|
||||
``proxies`` :
|
||||
Define one or more proxies you wish to use, see `httpx proxies`_.
|
||||
If there are more than one proxy for one protocol (http, https),
|
||||
requests to the engines are distributed in a round-robin fashion.
|
||||
|
||||
``source_ips`` :
|
||||
If you use multiple network interfaces, define from which IP the requests must
|
||||
be made. Example:
|
||||
|
||||
* ``0.0.0.0`` any local IPv4 address.
|
||||
* ``::`` any local IPv6 address.
|
||||
* ``192.168.0.1``
|
||||
* ``[ 192.168.0.1, 192.168.0.2 ]`` these two specific IP addresses
|
||||
* ``fe80::60a2:1691:e5a2:ee1f``
|
||||
* ``fe80::60a2:1691:e5a2:ee1f/126`` all IP addresses in this network.
|
||||
* ``[ 192.168.0.1, fe80::/126 ]``
|
||||
|
||||
``retries`` :
|
||||
Number of retry in case of an HTTP error. On each retry, SearXNG uses an
|
||||
different proxy and source ip.
|
||||
|
||||
``enable_http2`` :
|
||||
Enable by default. Set to ``false`` to disable HTTP/2.
|
||||
|
||||
.. _httpx verification defaults: https://www.python-httpx.org/advanced/#changing-the-verification-defaults
|
||||
.. _httpx ssl configuration: https://www.python-httpx.org/compatibility/#ssl-configuration
|
||||
|
||||
``verify``: : ``$SSL_CERT_FILE``, ``$SSL_CERT_DIR``
|
||||
Allow to specify a path to certificate.
|
||||
see `httpx verification defaults`_.
|
||||
|
||||
In addition to ``verify``, SearXNG supports the ``$SSL_CERT_FILE`` (for a file) and
|
||||
``$SSL_CERT_DIR`` (for a directory) OpenSSL variables.
|
||||
see `httpx ssl configuration`_.
|
||||
|
||||
``max_redirects`` :
|
||||
30 by default. Maximum redirect before it is an error.
|
||||
|
||||
``using_tor_proxy`` :
|
||||
Using tor proxy (``true``) or not (``false``) for all engines. The default is
|
||||
``false`` and can be overwritten in the :ref:`settings engine`
|
||||
|
||||
|
||||
43
docs/admin/settings/settings_redis.rst
Normal file
43
docs/admin/settings/settings_redis.rst
Normal file
@ -0,0 +1,43 @@
|
||||
.. _settings redis:
|
||||
|
||||
==========
|
||||
``redis:``
|
||||
==========
|
||||
|
||||
.. _Redis.from_url(url): https://redis-py.readthedocs.io/en/stable/connections.html#redis.client.Redis.from_url
|
||||
|
||||
A redis DB can be connected by an URL, in :py:obj:`searx.redisdb` you
|
||||
will find a description to test your redis connection in SerXNG. When using
|
||||
sockets, don't forget to check the access rights on the socket::
|
||||
|
||||
ls -la /usr/local/searxng-redis/run/redis.sock
|
||||
srwxrwx--- 1 searxng-redis searxng-redis ... /usr/local/searxng-redis/run/redis.sock
|
||||
|
||||
In this example read/write access is given to the *searxng-redis* group. To get
|
||||
access rights to redis instance (the socket), your SearXNG (or even your
|
||||
developer) account needs to be added to the *searxng-redis* group.
|
||||
|
||||
``url`` : ``$SEARXNG_REDIS_URL``
|
||||
URL to connect redis database, see `Redis.from_url(url)`_ & :ref:`redis db`::
|
||||
|
||||
redis://[[username]:[password]]@localhost:6379/0
|
||||
rediss://[[username]:[password]]@localhost:6379/0
|
||||
unix://[[username]:[password]]@/path/to/socket.sock?db=0
|
||||
|
||||
.. admonition:: Tip for developers
|
||||
|
||||
To set up a local redis instance, first set the socket path of the Redis DB
|
||||
in your YAML setting:
|
||||
|
||||
.. code:: yaml
|
||||
|
||||
redis:
|
||||
url: unix:///usr/local/searxng-redis/run/redis.sock?db=0
|
||||
|
||||
Then use the following commands to install the redis instance ::
|
||||
|
||||
$ ./manage redis.build
|
||||
$ sudo -H ./manage redis.install
|
||||
$ sudo -H ./manage redis.addgrp "${USER}"
|
||||
# don't forget to logout & login to get member of group
|
||||
|
||||
97
docs/admin/settings/settings_search.rst
Normal file
97
docs/admin/settings/settings_search.rst
Normal file
@ -0,0 +1,97 @@
|
||||
.. _settings search:
|
||||
|
||||
===========
|
||||
``search:``
|
||||
===========
|
||||
|
||||
.. code:: yaml
|
||||
|
||||
search:
|
||||
safe_search: 0
|
||||
autocomplete: ""
|
||||
default_lang: ""
|
||||
ban_time_on_fail: 5
|
||||
max_ban_time_on_fail: 120
|
||||
suspended_times:
|
||||
SearxEngineAccessDenied: 86400
|
||||
SearxEngineCaptcha: 86400
|
||||
SearxEngineTooManyRequests: 3600
|
||||
cf_SearxEngineCaptcha: 1296000
|
||||
cf_SearxEngineAccessDenied: 86400
|
||||
recaptcha_SearxEngineCaptcha: 604800
|
||||
formats:
|
||||
- html
|
||||
|
||||
``safe_search``:
|
||||
Filter results.
|
||||
|
||||
- ``0``: None
|
||||
- ``1``: Moderate
|
||||
- ``2``: Strict
|
||||
|
||||
``autocomplete``:
|
||||
Existing autocomplete backends, leave blank to turn it off.
|
||||
|
||||
- ``dbpedia``
|
||||
- ``duckduckgo``
|
||||
- ``google``
|
||||
- ``startpage``
|
||||
- ``swisscows``
|
||||
- ``qwant``
|
||||
- ``wikipedia``
|
||||
|
||||
``default_lang``:
|
||||
Default search language - leave blank to detect from browser information or
|
||||
use codes from :origin:`searx/languages.py`.
|
||||
|
||||
``languages``:
|
||||
List of available languages - leave unset to use all codes from
|
||||
:origin:`searx/languages.py`. Otherwise list codes of available languages.
|
||||
The ``all`` value is shown as the ``Default language`` in the user interface
|
||||
(in most cases, it is meant to send the query without a language parameter ;
|
||||
in some cases, it means the English language) Example:
|
||||
|
||||
.. code:: yaml
|
||||
|
||||
languages:
|
||||
- all
|
||||
- en
|
||||
- en-US
|
||||
- de
|
||||
- it-IT
|
||||
- fr
|
||||
- fr-BE
|
||||
|
||||
``ban_time_on_fail``:
|
||||
Ban time in seconds after engine errors.
|
||||
|
||||
``max_ban_time_on_fail``:
|
||||
Max ban time in seconds after engine errors.
|
||||
|
||||
``suspended_times``:
|
||||
Engine suspension time after error (in seconds; set to 0 to disable)
|
||||
|
||||
``SearxEngineAccessDenied``: 86400
|
||||
For error "Access denied" and "HTTP error [402, 403]"
|
||||
|
||||
``SearxEngineCaptcha``: 86400
|
||||
For error "CAPTCHA"
|
||||
|
||||
``SearxEngineTooManyRequests``: 3600
|
||||
For error "Too many request" and "HTTP error 429"
|
||||
|
||||
Cloudflare CAPTCHA:
|
||||
- ``cf_SearxEngineCaptcha``: 1296000
|
||||
- ``cf_SearxEngineAccessDenied``: 86400
|
||||
|
||||
Google CAPTCHA:
|
||||
- ``recaptcha_SearxEngineCaptcha``: 604800
|
||||
|
||||
``formats``:
|
||||
Result formats available from web, remove format to deny access (use lower
|
||||
case).
|
||||
|
||||
- ``html``
|
||||
- ``csv``
|
||||
- ``json``
|
||||
- ``rss``
|
||||
54
docs/admin/settings/settings_server.rst
Normal file
54
docs/admin/settings/settings_server.rst
Normal file
@ -0,0 +1,54 @@
|
||||
.. _settings server:
|
||||
|
||||
===========
|
||||
``server:``
|
||||
===========
|
||||
|
||||
.. code:: yaml
|
||||
|
||||
server:
|
||||
base_url: http://example.org/location # change this!
|
||||
port: 8888
|
||||
bind_address: "127.0.0.1"
|
||||
secret_key: "ultrasecretkey" # change this!
|
||||
limiter: false
|
||||
image_proxy: false
|
||||
default_http_headers:
|
||||
X-Content-Type-Options : nosniff
|
||||
X-XSS-Protection : 1; mode=block
|
||||
X-Download-Options : noopen
|
||||
X-Robots-Tag : noindex, nofollow
|
||||
Referrer-Policy : no-referrer
|
||||
|
||||
|
||||
``base_url`` : ``$SEARXNG_URL`` :ref:`buildenv <make buildenv>`
|
||||
The base URL where SearXNG is deployed. Used to create correct inbound links.
|
||||
If you change the value, don't forget to rebuild instance's environment
|
||||
(:ref:`utils/brand.env <make buildenv>`)
|
||||
|
||||
``port`` & ``bind_address``: ``$SEARXNG_PORT`` & ``$SEARXNG_BIND_ADDRESS`` :ref:`buildenv <make buildenv>`
|
||||
Port number and *bind address* of the SearXNG web application if you run it
|
||||
directly using ``python searx/webapp.py``. Doesn't apply to a SearXNG
|
||||
services running behind a proxy and using socket communications. If you
|
||||
change the value, don't forget to rebuild instance's environment
|
||||
(:ref:`utils/brand.env <make buildenv>`)
|
||||
|
||||
``secret_key`` : ``$SEARXNG_SECRET``
|
||||
Used for cryptography purpose.
|
||||
|
||||
.. _limiter:
|
||||
|
||||
``limiter`` :
|
||||
Rate limit the number of request on the instance, block some bots. The
|
||||
:ref:`limiter src` requires a :ref:`settings redis` database.
|
||||
|
||||
.. _image_proxy:
|
||||
|
||||
``image_proxy`` :
|
||||
Allow your instance of SearXNG of being able to proxy images. Uses memory space.
|
||||
|
||||
.. _HTTP headers: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers
|
||||
|
||||
``default_http_headers`` :
|
||||
Set additional HTTP headers, see `#755 <https://github.com/searx/searx/issues/715>`__
|
||||
|
||||
62
docs/admin/settings/settings_ui.rst
Normal file
62
docs/admin/settings/settings_ui.rst
Normal file
@ -0,0 +1,62 @@
|
||||
.. _settings ui:
|
||||
|
||||
=======
|
||||
``ui:``
|
||||
=======
|
||||
|
||||
.. _cache busting:
|
||||
https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Cache-Control#caching_static_assets_with_cache_busting
|
||||
|
||||
.. code:: yaml
|
||||
|
||||
ui:
|
||||
static_use_hash: false
|
||||
default_locale: ""
|
||||
query_in_title: false
|
||||
infinite_scroll: false
|
||||
center_alignment: false
|
||||
cache_url: https://web.archive.org/web/
|
||||
default_theme: simple
|
||||
theme_args:
|
||||
simple_style: auto
|
||||
|
||||
.. _static_use_hash:
|
||||
|
||||
``static_use_hash`` :
|
||||
Enables `cache busting`_ of static files.
|
||||
|
||||
``default_locale`` :
|
||||
SearXNG interface language. If blank, the locale is detected by using the
|
||||
browser language. If it doesn't work, or you are deploying a language
|
||||
specific instance of searx, a locale can be defined using an ISO language
|
||||
code, like ``fr``, ``en``, ``de``.
|
||||
|
||||
``query_in_title`` :
|
||||
When true, the result page's titles contains the query it decreases the
|
||||
privacy, since the browser can records the page titles.
|
||||
|
||||
``infinite_scroll``:
|
||||
When true, automatically loads the next page when scrolling to bottom of the current page.
|
||||
|
||||
``center_alignment`` : default ``false``
|
||||
When enabled, the results are centered instead of being in the left (or RTL)
|
||||
side of the screen. This setting only affects the *desktop layout*
|
||||
(:origin:`min-width: @tablet <searx/static/themes/simple/src/less/definitions.less>`)
|
||||
|
||||
.. cache_url:
|
||||
|
||||
``cache_url`` : ``https://web.archive.org/web/``
|
||||
URL prefix of the internet archive or cache, don't forgett trailing slash (if
|
||||
needed). The default is https://web.archive.org/web/ alternatives are:
|
||||
|
||||
- https://webcache.googleusercontent.com/search?q=cache:
|
||||
- https://archive.today/
|
||||
|
||||
``default_theme`` :
|
||||
Name of the theme you want to use by default on your SearXNG instance.
|
||||
|
||||
``theme_args.simple_style``:
|
||||
Style of simple theme: ``auto``, ``light``, ``dark``
|
||||
|
||||
``results_on_new_tab``:
|
||||
Open result links in a new tab by default.
|
||||
@ -9,7 +9,7 @@ SearXNG maintenance
|
||||
- :ref:`toolboxing`
|
||||
- :ref:`uWSGI maintenance`
|
||||
|
||||
.. contents:: Contents
|
||||
.. contents::
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
@ -4,7 +4,7 @@
|
||||
How to contribute
|
||||
=================
|
||||
|
||||
.. contents:: Contents
|
||||
.. contents::
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
@ -4,6 +4,11 @@
|
||||
Demo Offline Engine
|
||||
===================
|
||||
|
||||
.. contents::
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
.. automodule:: searx.engines.demo_offline
|
||||
:members:
|
||||
|
||||
@ -4,6 +4,11 @@
|
||||
Demo Online Engine
|
||||
==================
|
||||
|
||||
.. contents::
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
.. automodule:: searx.engines.demo_online
|
||||
:members:
|
||||
|
||||
@ -4,6 +4,11 @@
|
||||
Engine Overview
|
||||
===============
|
||||
|
||||
.. contents::
|
||||
:depth: 3
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
.. _metasearch-engine: https://en.wikipedia.org/wiki/Metasearch_engine
|
||||
|
||||
.. sidebar:: Further reading ..
|
||||
@ -11,10 +16,6 @@ Engine Overview
|
||||
- :ref:`configured engines`
|
||||
- :ref:`settings engine`
|
||||
|
||||
.. contents::
|
||||
:depth: 3
|
||||
:backlinks: entry
|
||||
|
||||
SearXNG is a metasearch-engine_, so it uses different search engines to provide
|
||||
better results.
|
||||
|
||||
@ -49,12 +50,12 @@ Engine File
|
||||
categories list categories, in which the engine is working
|
||||
paging boolean support multiple pages
|
||||
time_range_support boolean support search time range
|
||||
engine_type str - ``online`` :ref:`[ref] <demo online engine>` by
|
||||
engine_type str - ``online`` :ref:`[ref] <online engines>` by
|
||||
default, other possibles values are:
|
||||
- ``offline`` :ref:`[ref] <offline engines>`
|
||||
- ``online_dictionary``
|
||||
- ``online_currency``
|
||||
- ``online_url_search``
|
||||
- ``online_dictionary`` :ref:`[ref] <online dictionary>`
|
||||
- ``online_currency`` :ref:`[ref] <online currency>`
|
||||
- ``online_url_search`` :ref:`[ref] <online url search>`
|
||||
======================= =========== ========================================================
|
||||
|
||||
.. _engine settings:
|
||||
@ -239,12 +240,18 @@ following parameters can be used to specify a search request:
|
||||
.. _engine results:
|
||||
.. _engine media types:
|
||||
|
||||
Media Types
|
||||
===========
|
||||
Result Types (``template``)
|
||||
===========================
|
||||
|
||||
Each result item of an engine can be of different media-types. Currently the
|
||||
following media-types are supported. To set another media-type as ``default``,
|
||||
the parameter ``template`` must be set to the desired type.
|
||||
following media-types are supported. To set another media-type as
|
||||
:ref:`template default`, the parameter ``template`` must be set to the desired
|
||||
type.
|
||||
|
||||
.. _template default:
|
||||
|
||||
``default``
|
||||
-----------
|
||||
|
||||
.. table:: Parameter of the **default** media type:
|
||||
:width: 100%
|
||||
@ -259,6 +266,11 @@ the parameter ``template`` must be set to the desired type.
|
||||
========================= =====================================================
|
||||
|
||||
|
||||
.. _template images:
|
||||
|
||||
``images``
|
||||
----------
|
||||
|
||||
.. table:: Parameter of the **images** media type:
|
||||
:width: 100%
|
||||
|
||||
@ -277,6 +289,11 @@ the parameter ``template`` must be set to the desired type.
|
||||
========================= =====================================================
|
||||
|
||||
|
||||
.. _template videos:
|
||||
|
||||
``videos``
|
||||
----------
|
||||
|
||||
.. table:: Parameter of the **videos** media type:
|
||||
:width: 100%
|
||||
|
||||
@ -292,6 +309,12 @@ the parameter ``template`` must be set to the desired type.
|
||||
thumbnail string, url to a small-preview image
|
||||
========================= =====================================================
|
||||
|
||||
|
||||
.. _template torrent:
|
||||
|
||||
``torrent``
|
||||
-----------
|
||||
|
||||
.. _magnetlink: https://en.wikipedia.org/wiki/Magnet_URI_scheme
|
||||
|
||||
.. table:: Parameter of the **torrent** media type:
|
||||
@ -315,6 +338,12 @@ the parameter ``template`` must be set to the desired type.
|
||||
torrentfile string, torrentfile of the result
|
||||
========================= =====================================================
|
||||
|
||||
|
||||
.. _template map:
|
||||
|
||||
``map``
|
||||
-------
|
||||
|
||||
.. table:: Parameter of the **map** media type:
|
||||
:width: 100%
|
||||
|
||||
@ -342,6 +371,12 @@ the parameter ``template`` must be set to the desired type.
|
||||
address.country country of object
|
||||
========================= =====================================================
|
||||
|
||||
|
||||
.. _template paper:
|
||||
|
||||
``paper``
|
||||
---------
|
||||
|
||||
.. _BibTeX format: https://www.bibtex.com/g/bibtex-format/
|
||||
.. _BibTeX field types: https://en.wikipedia.org/wiki/BibTeX#Field_types
|
||||
|
||||
@ -430,3 +465,4 @@ the parameter ``template`` must be set to the desired type.
|
||||
* - html_url
|
||||
- :py:class:`str`
|
||||
- URL to full article, HTML version
|
||||
|
||||
@ -1,15 +1,20 @@
|
||||
.. _searx.enginelib:
|
||||
|
||||
============
|
||||
Engine model
|
||||
============
|
||||
==============
|
||||
Engine Library
|
||||
==============
|
||||
|
||||
.. contents::
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
.. automodule:: searx.enginelib
|
||||
:members:
|
||||
|
||||
.. _searx.enginelib.traits:
|
||||
|
||||
=============
|
||||
|
||||
Engine traits
|
||||
=============
|
||||
|
||||
9
docs/dev/engines/engines.rst
Normal file
9
docs/dev/engines/engines.rst
Normal file
@ -0,0 +1,9 @@
|
||||
.. _searx.engines loader:
|
||||
|
||||
========================
|
||||
SearXNG's engines loader
|
||||
========================
|
||||
|
||||
.. automodule:: searx.engines
|
||||
:members:
|
||||
|
||||
97
docs/dev/engines/index.rst
Normal file
97
docs/dev/engines/index.rst
Normal file
@ -0,0 +1,97 @@
|
||||
.. _engine implementations:
|
||||
|
||||
======================
|
||||
Engine Implementations
|
||||
======================
|
||||
|
||||
Framework Components
|
||||
====================
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
|
||||
enginelib
|
||||
engines
|
||||
engine_overview
|
||||
|
||||
|
||||
Engine Types
|
||||
============
|
||||
|
||||
The :py:obj:`engine_type <searx.enginelib.Engine.engine_type>` of an engine
|
||||
determines which :ref:`search processor <searx.search.processors>` is used by
|
||||
the engine.
|
||||
|
||||
In this section a list of the enignes that are documented is given, a complete
|
||||
list of the engines can be found in the source under: :origin:`searx/engines`.
|
||||
|
||||
.. _online engines:
|
||||
|
||||
Online Engines
|
||||
--------------
|
||||
|
||||
.. sidebar:: info
|
||||
|
||||
- :py:obj:`processors.online <searx.search.processors.online>`
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 1
|
||||
:glob:
|
||||
|
||||
demo/demo_online
|
||||
xpath
|
||||
online/*
|
||||
|
||||
.. _offline engines:
|
||||
|
||||
Offline Engines
|
||||
---------------
|
||||
|
||||
.. sidebar:: info
|
||||
|
||||
- :py:obj:`processors.offline <searx.search.processors.offline>`
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 1
|
||||
:glob:
|
||||
|
||||
offline_concept
|
||||
demo/demo_offline
|
||||
offline/*
|
||||
|
||||
.. _online url search:
|
||||
|
||||
Online URL Search
|
||||
-----------------
|
||||
|
||||
.. sidebar:: info
|
||||
|
||||
- :py:obj:`processors.online_url_search <searx.search.processors.online_url_search>`
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 1
|
||||
:glob:
|
||||
|
||||
online_url_search/*
|
||||
|
||||
.. _online currency:
|
||||
|
||||
Online Currency
|
||||
---------------
|
||||
|
||||
.. sidebar:: info
|
||||
|
||||
- :py:obj:`processors.online_currency <searx.search.processors.online_currency>`
|
||||
|
||||
*no engine of this type is documented yet / comming soon*
|
||||
|
||||
.. _online dictionary:
|
||||
|
||||
Online Dictionary
|
||||
-----------------
|
||||
|
||||
.. sidebar:: info
|
||||
|
||||
- :py:obj:`processors.online_dictionary <searx.search.processors.online_dictionary>`
|
||||
|
||||
*no engine of this type is documented yet / comming soon*
|
||||
23
docs/dev/engines/offline/command-line-engines.rst
Normal file
23
docs/dev/engines/offline/command-line-engines.rst
Normal file
@ -0,0 +1,23 @@
|
||||
.. _engine command:
|
||||
|
||||
====================
|
||||
Command Line Engines
|
||||
====================
|
||||
|
||||
.. sidebar:: info
|
||||
|
||||
- :origin:`command.py <searx/engines/command.py>`
|
||||
- :ref:`offline engines`
|
||||
|
||||
.. contents::
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
.. sidebar:: info
|
||||
|
||||
Initial sponsored by `Search and Discovery Fund
|
||||
<https://nlnet.nl/discovery>`_ of `NLnet Foundation <https://nlnet.nl/>`_.
|
||||
|
||||
.. automodule:: searx.engines.command
|
||||
:members:
|
||||
@ -1,3 +1,5 @@
|
||||
.. _nosql engines:
|
||||
|
||||
===============
|
||||
NoSQL databases
|
||||
===============
|
||||
@ -8,6 +10,16 @@ NoSQL databases
|
||||
- `redis.io <https://redis.io/>`_
|
||||
- `MongoDB <https://www.mongodb.com>`_
|
||||
|
||||
.. contents::
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
.. sidebar:: info
|
||||
|
||||
Initial sponsored by `Search and Discovery Fund
|
||||
<https://nlnet.nl/discovery>`_ of `NLnet Foundation <https://nlnet.nl/>`_.
|
||||
|
||||
The following `NoSQL databases`_ are supported:
|
||||
|
||||
- :ref:`engine redis_server`
|
||||
@ -30,15 +42,8 @@ can still add them and limit the access by setting ``tokens`` as described in
|
||||
section :ref:`private engines`.
|
||||
|
||||
|
||||
Configure the engines
|
||||
=====================
|
||||
|
||||
`NoSQL databases`_ are used for storing arbitrary data without first defining
|
||||
their structure.
|
||||
|
||||
|
||||
Extra Dependencies
|
||||
------------------
|
||||
==================
|
||||
|
||||
For using :ref:`engine redis_server` or :ref:`engine mongodb` you need to
|
||||
install additional packages in Python's Virtual Environment of your SearXNG
|
||||
@ -49,6 +54,13 @@ instance. To switch into the environment (:ref:`searxng-src`) you can use
|
||||
(searxng-pyenv)$ pip install ...
|
||||
|
||||
|
||||
Configure the engines
|
||||
=====================
|
||||
|
||||
`NoSQL databases`_ are used for storing arbitrary data without first defining
|
||||
their structure.
|
||||
|
||||
|
||||
.. _engine redis_server:
|
||||
|
||||
Redis Server
|
||||
@ -62,29 +74,9 @@ Redis Server
|
||||
- redis.io_
|
||||
- :origin:`redis_server.py <searx/engines/redis_server.py>`
|
||||
|
||||
.. automodule:: searx.engines.redis_server
|
||||
:members:
|
||||
|
||||
Redis is an open source (BSD licensed), in-memory data structure (key value
|
||||
based) store. Before configuring the ``redis_server`` engine, you must install
|
||||
the dependency redis_.
|
||||
|
||||
Select a database to search in and set its index in the option ``db``. You can
|
||||
either look for exact matches or use partial keywords to find what you are
|
||||
looking for by configuring ``exact_match_only``. You find an example
|
||||
configuration below:
|
||||
|
||||
.. code:: yaml
|
||||
|
||||
# Required dependency: redis
|
||||
|
||||
- name: myredis
|
||||
shortcut : rds
|
||||
engine: redis_server
|
||||
exact_match_only: false
|
||||
host: '127.0.0.1'
|
||||
port: 6379
|
||||
enable_http: true
|
||||
password: ''
|
||||
db: 0
|
||||
|
||||
.. _engine mongodb:
|
||||
|
||||
@ -99,37 +91,7 @@ MongoDB
|
||||
- MongoDB_
|
||||
- :origin:`mongodb.py <searx/engines/mongodb.py>`
|
||||
|
||||
MongoDB_ is a document based database program that handles JSON like data.
|
||||
Before configuring the ``mongodb`` engine, you must install the dependency
|
||||
redis_.
|
||||
|
||||
In order to query MongoDB_, you have to select a ``database`` and a
|
||||
``collection``. Furthermore, you have to select a ``key`` that is going to be
|
||||
searched. MongoDB_ also supports the option ``exact_match_only``, so configure
|
||||
it as you wish. Below is an example configuration for using a MongoDB
|
||||
collection:
|
||||
|
||||
.. code:: yaml
|
||||
|
||||
# MongoDB engine
|
||||
# Required dependency: pymongo
|
||||
|
||||
- name: mymongo
|
||||
engine: mongodb
|
||||
shortcut: md
|
||||
exact_match_only: false
|
||||
host: '127.0.0.1'
|
||||
port: 27017
|
||||
enable_http: true
|
||||
results_per_page: 20
|
||||
database: 'business'
|
||||
collection: 'reviews' # name of the db collection
|
||||
key: 'name' # key in the collection to search for
|
||||
|
||||
|
||||
Acknowledgment
|
||||
==============
|
||||
|
||||
This development was sponsored by `Search and Discovery Fund
|
||||
<https://nlnet.nl/discovery>`_ of `NLnet Foundation <https://nlnet.nl/>`_.
|
||||
.. automodule:: searx.engines.mongodb
|
||||
:members:
|
||||
|
||||
62
docs/dev/engines/offline/search-indexer-engines.rst
Normal file
62
docs/dev/engines/offline/search-indexer-engines.rst
Normal file
@ -0,0 +1,62 @@
|
||||
=================
|
||||
Local Search APIs
|
||||
=================
|
||||
|
||||
.. sidebar:: further read
|
||||
|
||||
- `Comparison to alternatives
|
||||
<https://docs.meilisearch.com/learn/what_is_meilisearch/comparison_to_alternatives.html>`_
|
||||
|
||||
.. contents::
|
||||
:depth: 1
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
.. sidebar:: info
|
||||
|
||||
Initial sponsored by `Search and Discovery Fund
|
||||
<https://nlnet.nl/discovery>`_ of `NLnet Foundation <https://nlnet.nl/>`_.
|
||||
|
||||
Administrators might find themselves wanting to integrate locally running search
|
||||
engines. The following ones are supported for now:
|
||||
|
||||
* `Elasticsearch`_
|
||||
* `Meilisearch`_
|
||||
* `Solr`_
|
||||
|
||||
Each search engine is powerful, capable of full-text search. All of the engines
|
||||
above are added to ``settings.yml`` just commented out, as you have to
|
||||
``base_url`` for all them.
|
||||
|
||||
Please note that if you are not using HTTPS to access these engines, you have to
|
||||
enable HTTP requests by setting ``enable_http`` to ``True``.
|
||||
|
||||
Furthermore, if you do not want to expose these engines on a public instance,
|
||||
you can still add them and limit the access by setting ``tokens`` as described
|
||||
in section :ref:`private engines`.
|
||||
|
||||
.. _engine meilisearch:
|
||||
|
||||
MeiliSearch
|
||||
===========
|
||||
|
||||
.. automodule:: searx.engines.meilisearch
|
||||
:members:
|
||||
|
||||
|
||||
.. _engine elasticsearch:
|
||||
|
||||
Elasticsearch
|
||||
=============
|
||||
|
||||
.. automodule:: searx.engines.elasticsearch
|
||||
:members:
|
||||
|
||||
.. _engine solr:
|
||||
|
||||
Solr
|
||||
====
|
||||
|
||||
.. automodule:: searx.engines.solr
|
||||
:members:
|
||||
|
||||
@ -10,6 +10,16 @@ SQL Engines
|
||||
- `PostgreSQL <https://www.postgresql.org>`_
|
||||
- `MySQL <https://www.mysql.com>`_
|
||||
|
||||
.. contents::
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
.. sidebar:: info
|
||||
|
||||
Initial sponsored by `Search and Discovery Fund
|
||||
<https://nlnet.nl/discovery>`_ of `NLnet Foundation <https://nlnet.nl/>`_.
|
||||
|
||||
With the *SQL engines* you can bind SQL databases into SearXNG. The following
|
||||
Relational Database Management System (RDBMS) are supported:
|
||||
|
||||
@ -42,6 +52,18 @@ add them and limit the access by setting ``tokens`` as described in section
|
||||
:ref:`private engines`.
|
||||
|
||||
|
||||
Extra Dependencies
|
||||
==================
|
||||
|
||||
For using :ref:`engine postgresql` or :ref:`engine mysql_server` you need to
|
||||
install additional packages in Python's Virtual Environment of your SearXNG
|
||||
instance. To switch into the environment (:ref:`searxng-src`) you can use
|
||||
:ref:`searxng.sh`::
|
||||
|
||||
$ sudo utils/searxng.sh instance cmd bash
|
||||
(searxng-pyenv)$ pip install ...
|
||||
|
||||
|
||||
Configure the engines
|
||||
=====================
|
||||
|
||||
@ -64,45 +86,8 @@ SQLite
|
||||
|
||||
- :origin:`sqlite.py <searx/engines/sqlite.py>`
|
||||
|
||||
.. _MediathekView: https://mediathekview.de/
|
||||
|
||||
SQLite is a small, fast and reliable SQL database engine. It does not require
|
||||
any extra dependency. To demonstrate the power of database engines, here is a
|
||||
more complex example which reads from a MediathekView_ (DE) movie database. For
|
||||
this example of the SQlite engine download the database:
|
||||
|
||||
- https://liste.mediathekview.de/filmliste-v2.db.bz2
|
||||
|
||||
and unpack into ``searx/data/filmliste-v2.db``. To search the database use e.g
|
||||
Query to test: ``!mediathekview concert``
|
||||
|
||||
.. code:: yaml
|
||||
|
||||
- name: mediathekview
|
||||
engine: sqlite
|
||||
disabled: False
|
||||
categories: general
|
||||
result_template: default.html
|
||||
database: searx/data/filmliste-v2.db
|
||||
query_str: >-
|
||||
SELECT title || ' (' || time(duration, 'unixepoch') || ')' AS title,
|
||||
COALESCE( NULLIF(url_video_hd,''), NULLIF(url_video_sd,''), url_video) AS url,
|
||||
description AS content
|
||||
FROM film
|
||||
WHERE title LIKE :wildcard OR description LIKE :wildcard
|
||||
ORDER BY duration DESC
|
||||
|
||||
|
||||
Extra Dependencies
|
||||
------------------
|
||||
|
||||
For using :ref:`engine postgresql` or :ref:`engine mysql_server` you need to
|
||||
install additional packages in Python's Virtual Environment of your SearXNG
|
||||
instance. To switch into the environment (:ref:`searxng-src`) you can use
|
||||
:ref:`searxng.sh`::
|
||||
|
||||
$ sudo utils/searxng.sh instance cmd bash
|
||||
(searxng-pyenv)$ pip install ...
|
||||
.. automodule:: searx.engines.sqlite
|
||||
:members:
|
||||
|
||||
|
||||
.. _engine postgresql:
|
||||
@ -115,20 +100,10 @@ PostgreSQL
|
||||
.. sidebar:: info
|
||||
|
||||
- :origin:`postgresql.py <searx/engines/postgresql.py>`
|
||||
- ``pip install`` psycopg2_
|
||||
- ``pip install`` `psycopg2-binary <psycopg2>`_
|
||||
|
||||
PostgreSQL is a powerful and robust open source database. Before configuring
|
||||
the PostgreSQL engine, you must install the dependency ``psychopg2``. You can
|
||||
find an example configuration below:
|
||||
|
||||
.. code:: yaml
|
||||
|
||||
- name: my_database
|
||||
engine: postgresql
|
||||
database: my_database
|
||||
username: searxng
|
||||
password: password
|
||||
query_str: 'SELECT * from my_table WHERE my_column = %(query)s'
|
||||
.. automodule:: searx.engines.postgresql
|
||||
:members:
|
||||
|
||||
.. _engine mysql_server:
|
||||
|
||||
@ -140,27 +115,7 @@ MySQL
|
||||
- :origin:`mysql_server.py <searx/engines/mysql_server.py>`
|
||||
- ``pip install`` :pypi:`mysql-connector-python <mysql-connector-python>`
|
||||
|
||||
MySQL is said to be the most popular open source database. Before enabling MySQL
|
||||
engine, you must install the package ``mysql-connector-python``.
|
||||
|
||||
The authentication plugin is configurable by setting ``auth_plugin`` in the
|
||||
attributes. By default it is set to ``caching_sha2_password``. This is an
|
||||
example configuration for querying a MySQL server:
|
||||
|
||||
.. code:: yaml
|
||||
|
||||
- name: my_database
|
||||
engine: mysql_server
|
||||
database: my_database
|
||||
username: searxng
|
||||
password: password
|
||||
limit: 5
|
||||
query_str: 'SELECT * from my_table WHERE my_column=%(query)s'
|
||||
|
||||
|
||||
Acknowledgment
|
||||
==============
|
||||
|
||||
This development was sponsored by `Search and Discovery Fund
|
||||
<https://nlnet.nl/discovery>`_ of `NLnet Foundation <https://nlnet.nl/>`_.
|
||||
.. automodule:: searx.engines.mysql_server
|
||||
:members:
|
||||
|
||||
@ -1,15 +1,14 @@
|
||||
.. _offline engines:
|
||||
|
||||
===============
|
||||
Offline Engines
|
||||
Offline Concept
|
||||
===============
|
||||
|
||||
.. sidebar:: offline engines
|
||||
|
||||
- :ref:`demo offline engine`
|
||||
- :ref:`sql engines`
|
||||
- :ref:`engine command`
|
||||
- :origin:`Redis <searx/engines/redis_server.py>`
|
||||
- :ref:`sql engines`
|
||||
- :ref:`nosql engines`
|
||||
- :py:obj:`searx.search.processors.offline`
|
||||
|
||||
To extend the functionality of SearXNG, offline engines are going to be
|
||||
introduced. An offline engine is an engine which does not need Internet
|
||||
@ -31,7 +30,6 @@ Programming Interface
|
||||
in advance.
|
||||
|
||||
:py:func:`search(query, params) <searx.engines.demo_offline.searc>`
|
||||
|
||||
Each offline engine has a function named ``search``. This function is
|
||||
responsible to perform a search and return the results in a presentable
|
||||
format. (Where *presentable* means presentable by the selected result
|
||||
@ -69,10 +67,3 @@ administrators can set token(s) for each of the :ref:`private engines`. If a
|
||||
query contains a valid token, then SearXNG performs the requested private
|
||||
search. If not, requests from an offline engines return errors.
|
||||
|
||||
|
||||
Acknowledgement
|
||||
===============
|
||||
|
||||
This development was sponsored by `Search and Discovery Fund
|
||||
<https://nlnet.nl/discovery>`_ of `NLnet Foundation <https://nlnet.nl/>`_ .
|
||||
|
||||
13
docs/dev/engines/online/annas_archive.rst
Normal file
13
docs/dev/engines/online/annas_archive.rst
Normal file
@ -0,0 +1,13 @@
|
||||
.. _annas_archive engine:
|
||||
|
||||
==============
|
||||
Anna's Archive
|
||||
==============
|
||||
|
||||
.. contents::
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
.. automodule:: searx.engines.annas_archive
|
||||
:members:
|
||||
@ -4,6 +4,11 @@
|
||||
Arch Linux
|
||||
==========
|
||||
|
||||
.. contents::
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
.. automodule:: searx.engines.archlinux
|
||||
:members:
|
||||
|
||||
@ -4,7 +4,7 @@
|
||||
Bing Engines
|
||||
============
|
||||
|
||||
.. contents:: Contents
|
||||
.. contents::
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
@ -4,5 +4,10 @@
|
||||
Dailymotion
|
||||
===========
|
||||
|
||||
.. contents::
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
.. automodule:: searx.engines.dailymotion
|
||||
:members:
|
||||
@ -1,10 +1,10 @@
|
||||
.. _duckduckgo engines:
|
||||
|
||||
=================
|
||||
DukcDukGo engines
|
||||
DukcDukGo Engines
|
||||
=================
|
||||
|
||||
.. contents:: Contents
|
||||
.. contents::
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
@ -4,7 +4,7 @@
|
||||
Google Engines
|
||||
==============
|
||||
|
||||
.. contents:: Contents
|
||||
.. contents::
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
@ -4,7 +4,7 @@
|
||||
Peertube Engines
|
||||
================
|
||||
|
||||
.. contents:: Contents
|
||||
.. contents::
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
13
docs/dev/engines/online/recoll.rst
Normal file
13
docs/dev/engines/online/recoll.rst
Normal file
@ -0,0 +1,13 @@
|
||||
.. _engine recoll:
|
||||
|
||||
=============
|
||||
Recoll Engine
|
||||
=============
|
||||
|
||||
.. contents::
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
.. automodule:: searx.engines.recoll
|
||||
:members:
|
||||
@ -1,10 +1,10 @@
|
||||
.. _startpage engines:
|
||||
|
||||
=================
|
||||
Startpage engines
|
||||
Startpage Engines
|
||||
=================
|
||||
|
||||
.. contents:: Contents
|
||||
.. contents::
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
13
docs/dev/engines/online/torznab.rst
Normal file
13
docs/dev/engines/online/torznab.rst
Normal file
@ -0,0 +1,13 @@
|
||||
.. _torznab engine:
|
||||
|
||||
==============
|
||||
Torznab WebAPI
|
||||
==============
|
||||
|
||||
.. contents::
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
.. automodule:: searx.engines.torznab
|
||||
:members:
|
||||
@ -4,7 +4,7 @@
|
||||
Wikimedia
|
||||
=========
|
||||
|
||||
.. contents:: Contents
|
||||
.. contents::
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
@ -4,5 +4,10 @@
|
||||
Yahoo Engine
|
||||
============
|
||||
|
||||
.. contents::
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
.. automodule:: searx.engines.yahoo
|
||||
:members:
|
||||
@ -4,6 +4,11 @@
|
||||
Tineye
|
||||
======
|
||||
|
||||
.. contents::
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
.. automodule:: searx.engines.tineye
|
||||
:members:
|
||||
|
||||
@ -4,6 +4,11 @@
|
||||
XPath Engine
|
||||
============
|
||||
|
||||
.. contents::
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
.. automodule:: searx.engines.xpath
|
||||
:members:
|
||||
|
||||
@ -4,12 +4,10 @@ Developer documentation
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
:caption: Contents
|
||||
|
||||
quickstart
|
||||
contribution_guide
|
||||
engine_overview
|
||||
offline_engines
|
||||
engines/index
|
||||
search_api
|
||||
plugins
|
||||
translation
|
||||
|
||||
@ -22,7 +22,7 @@ In this article we will show, how you can make use of Linux Containers (LXC_) in
|
||||
section :ref:`internet connectivity docker`.
|
||||
|
||||
|
||||
.. contents:: Contents
|
||||
.. contents::
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
@ -22,7 +22,7 @@ Calling the ``help`` target gives a first overview (``make help``):
|
||||
|
||||
.. program-output:: bash -c "cd ..; make --no-print-directory help"
|
||||
|
||||
.. contents:: Contents
|
||||
.. contents::
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
@ -243,14 +243,14 @@ calling ``make clean`` stop all processes using the :ref:`make install` or
|
||||
We describe the usage of the ``doc.*`` targets in the :ref:`How to contribute /
|
||||
Documentation <contrib docs>` section. If you want to edit the documentation
|
||||
read our :ref:`make docs.live` section. If you are working in your own brand,
|
||||
adjust your :ref:`settings global`.
|
||||
adjust your :ref:`settings brand`.
|
||||
|
||||
.. _make docs.gh-pages:
|
||||
|
||||
``make docs.gh-pages``
|
||||
======================
|
||||
|
||||
To deploy on github.io first adjust your :ref:`settings global`. For any
|
||||
To deploy on github.io first adjust your :ref:`settings brand`. For any
|
||||
further read :ref:`deploy on github.io`.
|
||||
|
||||
.. _make test:
|
||||
|
||||
@ -37,7 +37,7 @@ docs.live <make docs.live>` to build HTML while editing.
|
||||
- DOT_, `Graphviz's dot`_, Graphviz_
|
||||
|
||||
|
||||
.. contents:: Contents
|
||||
.. contents::
|
||||
:depth: 3
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
@ -9,7 +9,6 @@ developers.
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
:caption: Contents
|
||||
|
||||
update
|
||||
standalone_searx.py
|
||||
|
||||
@ -43,7 +43,6 @@ If you don't trust anyone, you can set up your own, see :ref:`installation`.
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
:caption: Contents
|
||||
|
||||
user/index
|
||||
own-instance
|
||||
|
||||
@ -7,7 +7,7 @@ Why use a private instance?
|
||||
\.\. is a common question among SearXNG users. Before answering this
|
||||
question, see what options a SearXNG user has.
|
||||
|
||||
.. contents:: Contents
|
||||
.. contents::
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
@ -8,7 +8,6 @@ every item from the source code, but we will add documentation when requested.
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
:caption: Contents
|
||||
:glob:
|
||||
|
||||
searx.*
|
||||
|
||||
@ -4,7 +4,7 @@
|
||||
Bot Detection
|
||||
=============
|
||||
|
||||
.. contents:: Contents
|
||||
.. contents::
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
@ -1,2 +0,0 @@
|
||||
.. automodule:: searx.engines.annas_archive
|
||||
:members:
|
||||
@ -1,8 +0,0 @@
|
||||
.. _searx.engines:
|
||||
|
||||
=================
|
||||
SearXNG's engines
|
||||
=================
|
||||
|
||||
.. automodule:: searx.engines
|
||||
:members:
|
||||
@ -1,2 +0,0 @@
|
||||
.. automodule:: searx.engines.torznab
|
||||
:members:
|
||||
@ -4,7 +4,7 @@
|
||||
Locales
|
||||
=======
|
||||
|
||||
.. contents:: Contents
|
||||
.. contents::
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
@ -4,7 +4,7 @@
|
||||
Search processors
|
||||
=================
|
||||
|
||||
.. contents:: Contents
|
||||
.. contents::
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
@ -34,7 +34,7 @@ Online currency processor
|
||||
.. automodule:: searx.search.processors.online_currency
|
||||
:members:
|
||||
|
||||
Online Dictionary processor
|
||||
Online dictionary processor
|
||||
===========================
|
||||
|
||||
.. automodule:: searx.search.processors.online_dictionary
|
||||
|
||||
@ -22,7 +22,7 @@ Configured Engines
|
||||
called *tabs*), engines can be queried by their name or the categories they
|
||||
belong to, by using a :ref:`\!bing syntax <search-syntax>`.
|
||||
|
||||
.. contents:: Contents
|
||||
.. contents::
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
@ -2,7 +2,7 @@
|
||||
User information
|
||||
================
|
||||
|
||||
.. contents:: Contents
|
||||
.. contents::
|
||||
:depth: 3
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
@ -10,7 +10,6 @@ and developers.
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
:caption: Contents
|
||||
|
||||
searxng.sh
|
||||
lxc.sh
|
||||
|
||||
@ -26,7 +26,7 @@ to care about*).
|
||||
- `LXC/LXD Image Server`_
|
||||
- `LXD@github`_
|
||||
|
||||
.. contents:: Contents
|
||||
.. contents::
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
@ -15,7 +15,7 @@ script :origin:`utils/searxng.sh`.
|
||||
- :ref:`installation nginx`
|
||||
- :ref:`installation apache`
|
||||
|
||||
.. contents:: Contents
|
||||
.. contents::
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
@ -1,18 +1,15 @@
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
# lint: pylint
|
||||
"""Engine related implementations
|
||||
"""Implementations of the framework for the SearXNG engines.
|
||||
|
||||
.. note::
|
||||
.. hint::
|
||||
|
||||
The long term goal is to modularize all relevant implementations to the
|
||||
engines here in this Python package. In addition to improved modularization,
|
||||
this will also be necessary in part because the probability of circular
|
||||
imports will increase due to the increased typification of implementations in
|
||||
the future.
|
||||
The long term goal is to modularize all implementations of the engine
|
||||
framework here in this Python package. ToDo:
|
||||
|
||||
ToDo:
|
||||
- move implementations of the :ref:`searx.engines loader` to a new module in
|
||||
the :py:obj:`searx.enginelib` namespace.
|
||||
|
||||
- move :py:obj:`searx.engines.load_engine` to a new module `searx.enginelib`.
|
||||
"""
|
||||
|
||||
|
||||
@ -36,7 +33,7 @@ class Engine: # pylint: disable=too-few-public-methods
|
||||
# Common options in the engine module
|
||||
|
||||
engine_type: str
|
||||
"""Type of the engine (:origin:`searx/search/processors`)"""
|
||||
"""Type of the engine (:ref:`searx.search.processors`)"""
|
||||
|
||||
paging: bool
|
||||
"""Engine supports multiple pages."""
|
||||
|
||||
@ -1,8 +1,6 @@
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
# lint: pylint
|
||||
"""This module implements the engine loader.
|
||||
|
||||
Load and initialize the ``engines``, see :py:func:`load_engines` and register
|
||||
"""Load and initialize the ``engines``, see :py:func:`load_engines` and register
|
||||
:py:obj:`engine_shortcuts`.
|
||||
|
||||
usage::
|
||||
|
||||
@ -1,24 +1,12 @@
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
# lint: pylint
|
||||
""".. _annas_archive engine:
|
||||
|
||||
==============
|
||||
Anna's Archive
|
||||
==============
|
||||
"""`Anna's Archive`_ is a free non-profit online shadow library metasearch
|
||||
engine providing access to a variety of book resources (also via IPFS), created
|
||||
by a team of anonymous archivists (AnnaArchivist_).
|
||||
|
||||
.. _Anna's Archive: https://annas-archive.org/
|
||||
.. _AnnaArchivist: https://annas-software.org/AnnaArchivist/annas-archive
|
||||
|
||||
`Anna's Archive`_ is a free non-profit online shadow library metasearch engine
|
||||
providing access to a variety of book resources (also via IPFS), created by a
|
||||
team of anonymous archivists (AnnaArchivist_).
|
||||
|
||||
.. contents:: Contents
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
|
||||
Configuration
|
||||
=============
|
||||
|
||||
@ -41,7 +29,6 @@ for *newest* articles and journals (PDF) / by shortcut ``!aaa <search-term>``.
|
||||
aa_ext: 'pdf'
|
||||
aa_sort: 'newest'
|
||||
|
||||
|
||||
Implementations
|
||||
===============
|
||||
|
||||
|
||||
@ -1,6 +1,77 @@
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
"""
|
||||
Command (offline)
|
||||
"""With *command engines* administrators can run engines to integrate arbitrary
|
||||
shell commands.
|
||||
|
||||
.. attention::
|
||||
|
||||
When creating and enabling a ``command`` engine on a public instance, you
|
||||
must be careful to avoid leaking private data.
|
||||
|
||||
The easiest solution is to limit the access by setting ``tokens`` as described
|
||||
in section :ref:`private engines`. The engine base is flexible. Only your
|
||||
imagination can limit the power of this engine (and maybe security concerns).
|
||||
|
||||
Configuration
|
||||
=============
|
||||
|
||||
The following options are available:
|
||||
|
||||
``command``:
|
||||
A comma separated list of the elements of the command. A special token
|
||||
``{{QUERY}}`` tells where to put the search terms of the user. Example:
|
||||
|
||||
.. code:: yaml
|
||||
|
||||
['ls', '-l', '-h', '{{QUERY}}']
|
||||
|
||||
``delimiter``:
|
||||
A mapping containing a delimiter ``char`` and the *titles* of each element in
|
||||
``keys``.
|
||||
|
||||
``parse_regex``:
|
||||
A dict containing the regular expressions for each result key.
|
||||
|
||||
``query_type``:
|
||||
|
||||
The expected type of user search terms. Possible values: ``path`` and
|
||||
``enum``.
|
||||
|
||||
``path``:
|
||||
Checks if the user provided path is inside the working directory. If not,
|
||||
the query is not executed.
|
||||
|
||||
``enum``:
|
||||
Is a list of allowed search terms. If the user submits something which is
|
||||
not included in the list, the query returns an error.
|
||||
|
||||
``query_enum``:
|
||||
A list containing allowed search terms if ``query_type`` is set to ``enum``.
|
||||
|
||||
``working_dir``:
|
||||
The directory where the command has to be executed. Default: ``./``.
|
||||
|
||||
``result_separator``:
|
||||
The character that separates results. Default: ``\\n``.
|
||||
|
||||
Example
|
||||
=======
|
||||
|
||||
The example engine below can be used to find files with a specific name in the
|
||||
configured working directory:
|
||||
|
||||
.. code:: yaml
|
||||
|
||||
- name: find
|
||||
engine: command
|
||||
command: ['find', '.', '-name', '{{QUERY}}']
|
||||
query_type: path
|
||||
shortcut: fnd
|
||||
delimiter:
|
||||
chars: ' '
|
||||
keys: ['line']
|
||||
|
||||
Implementations
|
||||
===============
|
||||
"""
|
||||
|
||||
import re
|
||||
|
||||
@ -1,6 +1,44 @@
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
"""
|
||||
Elasticsearch
|
||||
""".. sidebar:: info
|
||||
|
||||
- :origin:`elasticsearch.py <searx/engines/elasticsearch.py>`
|
||||
- `Elasticsearch <https://www.elastic.co/elasticsearch/>`_
|
||||
- `Elasticsearch Guide
|
||||
<https://www.elastic.co/guide/en/elasticsearch/reference/current/index.html>`_
|
||||
- `Install Elasticsearch
|
||||
<https://www.elastic.co/guide/en/elasticsearch/reference/current/install-elasticsearch.html>`_
|
||||
|
||||
Elasticsearch_ supports numerous ways to query the data it is storing. At the
|
||||
moment the engine supports the most popular search methods (``query_type``):
|
||||
|
||||
- ``match``,
|
||||
- ``simple_query_string``,
|
||||
- ``term`` and
|
||||
- ``terms``.
|
||||
|
||||
If none of the methods fit your use case, you can select ``custom`` query type
|
||||
and provide the JSON payload to submit to Elasticsearch in
|
||||
``custom_query_json``.
|
||||
|
||||
Example
|
||||
=======
|
||||
|
||||
The following is an example configuration for an Elasticsearch_ instance with
|
||||
authentication configured to read from ``my-index`` index.
|
||||
|
||||
.. code:: yaml
|
||||
|
||||
- name: elasticsearch
|
||||
shortcut: es
|
||||
engine: elasticsearch
|
||||
base_url: http://localhost:9200
|
||||
username: elastic
|
||||
password: changeme
|
||||
index: my-index
|
||||
query_type: match
|
||||
# custom_query_json: '{ ... }'
|
||||
enable_http: true
|
||||
|
||||
"""
|
||||
|
||||
from json import loads, dumps
|
||||
|
||||
@ -1,7 +1,35 @@
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
# lint: pylint
|
||||
"""
|
||||
Meilisearch
|
||||
""".. sidebar:: info
|
||||
|
||||
- :origin:`meilisearch.py <searx/engines/meilisearch.py>`
|
||||
- `MeiliSearch <https://www.meilisearch.com>`_
|
||||
- `MeiliSearch Documentation <https://docs.meilisearch.com/>`_
|
||||
- `Install MeiliSearch
|
||||
<https://docs.meilisearch.com/learn/getting_started/installation.html>`_
|
||||
|
||||
MeiliSearch_ is aimed at individuals and small companies. It is designed for
|
||||
small-scale (less than 10 million documents) data collections. E.g. it is great
|
||||
for storing web pages you have visited and searching in the contents later.
|
||||
|
||||
The engine supports faceted search, so you can search in a subset of documents
|
||||
of the collection. Furthermore, you can search in MeiliSearch_ instances that
|
||||
require authentication by setting ``auth_token``.
|
||||
|
||||
Example
|
||||
=======
|
||||
|
||||
Here is a simple example to query a Meilisearch instance:
|
||||
|
||||
.. code:: yaml
|
||||
|
||||
- name: meilisearch
|
||||
engine: meilisearch
|
||||
shortcut: mes
|
||||
base_url: http://localhost:7700
|
||||
index: my-index
|
||||
enable_http: true
|
||||
|
||||
"""
|
||||
|
||||
# pylint: disable=global-statement
|
||||
|
||||
@ -1,11 +1,53 @@
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
# lint: pylint
|
||||
"""MongoDB engine (Offline)
|
||||
"""MongoDB_ is a document based database program that handles JSON like data.
|
||||
Before configuring the ``mongodb`` engine, you must install the dependency
|
||||
pymongo_.
|
||||
|
||||
Configuration
|
||||
=============
|
||||
|
||||
In order to query MongoDB_, you have to select a ``database`` and a
|
||||
``collection``. Furthermore, you have to select a ``key`` that is going to be
|
||||
searched. MongoDB_ also supports the option ``exact_match_only``, so configure
|
||||
it as you wish.
|
||||
|
||||
Example
|
||||
=======
|
||||
|
||||
Below is an example configuration for using a MongoDB collection:
|
||||
|
||||
.. code:: yaml
|
||||
|
||||
# MongoDB engine
|
||||
# Required dependency: pymongo
|
||||
|
||||
- name: mymongo
|
||||
engine: mongodb
|
||||
shortcut: md
|
||||
exact_match_only: false
|
||||
host: '127.0.0.1'
|
||||
port: 27017
|
||||
enable_http: true
|
||||
results_per_page: 20
|
||||
database: 'business'
|
||||
collection: 'reviews' # name of the db collection
|
||||
key: 'name' # key in the collection to search for
|
||||
|
||||
Implementations
|
||||
===============
|
||||
|
||||
"""
|
||||
|
||||
import re
|
||||
from pymongo import MongoClient # pyright: ignore # pylint: disable=import-error
|
||||
|
||||
try:
|
||||
from pymongo import MongoClient # type: ignore
|
||||
except ImportError:
|
||||
# import error is ignored because the admin has to install pymongo manually
|
||||
# to use the engine
|
||||
pass
|
||||
|
||||
|
||||
engine_type = 'offline'
|
||||
|
||||
|
||||
@ -1,12 +1,37 @@
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
# lint: pylint
|
||||
"""MySQL database (offline)
|
||||
"""MySQL is said to be the most popular open source database. Before enabling
|
||||
MySQL engine, you must install the package ``mysql-connector-python``.
|
||||
|
||||
The authentication plugin is configurable by setting ``auth_plugin`` in the
|
||||
attributes. By default it is set to ``caching_sha2_password``.
|
||||
|
||||
Example
|
||||
=======
|
||||
|
||||
This is an example configuration for querying a MySQL server:
|
||||
|
||||
.. code:: yaml
|
||||
|
||||
- name: my_database
|
||||
engine: mysql_server
|
||||
database: my_database
|
||||
username: searxng
|
||||
password: password
|
||||
limit: 5
|
||||
query_str: 'SELECT * from my_table WHERE my_column=%(query)s'
|
||||
|
||||
Implementations
|
||||
===============
|
||||
|
||||
"""
|
||||
|
||||
# import error is ignored because the admin has to install mysql manually to use
|
||||
# the engine
|
||||
import mysql.connector # pyright: ignore # pylint: disable=import-error
|
||||
try:
|
||||
import mysql.connector # type: ignore
|
||||
except ImportError:
|
||||
# import error is ignored because the admin has to install mysql manually to use
|
||||
# the engine
|
||||
pass
|
||||
|
||||
engine_type = 'offline'
|
||||
auth_plugin = 'caching_sha2_password'
|
||||
|
||||
@ -1,12 +1,33 @@
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
# lint: pylint
|
||||
"""PostgreSQL database (offline)
|
||||
"""PostgreSQL is a powerful and robust open source database. Before configuring
|
||||
the PostgreSQL engine, you must install the dependency ``psychopg2``.
|
||||
|
||||
Example
|
||||
=======
|
||||
|
||||
Below is an example configuration:
|
||||
|
||||
.. code:: yaml
|
||||
|
||||
- name: my_database
|
||||
engine: postgresql
|
||||
database: my_database
|
||||
username: searxng
|
||||
password: password
|
||||
query_str: 'SELECT * from my_table WHERE my_column = %(query)s'
|
||||
|
||||
Implementations
|
||||
===============
|
||||
|
||||
"""
|
||||
|
||||
# import error is ignored because the admin has to install mysql manually to use
|
||||
# the engine
|
||||
import psycopg2 # pyright: ignore # pylint: disable=import-error
|
||||
try:
|
||||
import psycopg2 # type: ignore
|
||||
except ImportError:
|
||||
# import error is ignored because the admin has to install postgresql
|
||||
# manually to use the engine.
|
||||
pass
|
||||
|
||||
engine_type = 'offline'
|
||||
host = "127.0.0.1"
|
||||
|
||||
@ -1,6 +1,51 @@
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
"""
|
||||
Recoll (local search engine)
|
||||
# lint: pylint
|
||||
""".. sidebar:: info
|
||||
|
||||
- `Recoll <https://www.lesbonscomptes.com/recoll/>`_
|
||||
- `recoll-webui <https://framagit.org/medoc92/recollwebui.git>`_
|
||||
- :origin:`searx/engines/recoll.py`
|
||||
|
||||
Recoll_ is a desktop full-text search tool based on Xapian. By itself Recoll_
|
||||
does not offer WEB or API access, this can be achieved using recoll-webui_
|
||||
|
||||
Configuration
|
||||
=============
|
||||
|
||||
You must configure the following settings:
|
||||
|
||||
``base_url``:
|
||||
Location where recoll-webui can be reached.
|
||||
|
||||
``mount_prefix``:
|
||||
Location where the file hierarchy is mounted on your *local* filesystem.
|
||||
|
||||
``dl_prefix``:
|
||||
Location where the file hierarchy as indexed by recoll can be reached.
|
||||
|
||||
``search_dir``:
|
||||
Part of the indexed file hierarchy to be search, if empty the full domain is
|
||||
searched.
|
||||
|
||||
Example
|
||||
=======
|
||||
|
||||
Scenario:
|
||||
|
||||
#. Recoll indexes a local filesystem mounted in ``/export/documents/reference``,
|
||||
#. the Recoll search interface can be reached at https://recoll.example.org/ and
|
||||
#. the contents of this filesystem can be reached though https://download.example.org/reference
|
||||
|
||||
.. code:: yaml
|
||||
|
||||
base_url: https://recoll.example.org/
|
||||
mount_prefix: /export/documents
|
||||
dl_prefix: https://download.example.org
|
||||
search_dir: ''
|
||||
|
||||
Implementations
|
||||
===============
|
||||
|
||||
"""
|
||||
|
||||
from datetime import date, timedelta
|
||||
@ -33,7 +78,7 @@ embedded_url = '<{ttype} controls height="166px" ' + 'src="{url}" type="{mtype}"
|
||||
|
||||
# helper functions
|
||||
def get_time_range(time_range):
|
||||
sw = {'day': 1, 'week': 7, 'month': 30, 'year': 365}
|
||||
sw = {'day': 1, 'week': 7, 'month': 30, 'year': 365} # pylint: disable=invalid-name
|
||||
|
||||
offset = sw.get(time_range, 0)
|
||||
if not offset:
|
||||
|
||||
@ -1,6 +1,37 @@
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
# lint: pylint
|
||||
"""Redis engine (offline)
|
||||
"""Redis is an open source (BSD licensed), in-memory data structure (key value
|
||||
based) store. Before configuring the ``redis_server`` engine, you must install
|
||||
the dependency redis_.
|
||||
|
||||
Configuration
|
||||
=============
|
||||
|
||||
Select a database to search in and set its index in the option ``db``. You can
|
||||
either look for exact matches or use partial keywords to find what you are
|
||||
looking for by configuring ``exact_match_only``.
|
||||
|
||||
Example
|
||||
=======
|
||||
|
||||
Below is an example configuration:
|
||||
|
||||
.. code:: yaml
|
||||
|
||||
# Required dependency: redis
|
||||
|
||||
- name: myredis
|
||||
shortcut : rds
|
||||
engine: redis_server
|
||||
exact_match_only: false
|
||||
host: '127.0.0.1'
|
||||
port: 6379
|
||||
enable_http: true
|
||||
password: ''
|
||||
db: 0
|
||||
|
||||
Implementations
|
||||
===============
|
||||
|
||||
"""
|
||||
|
||||
|
||||
@ -1,7 +1,31 @@
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
# lint: pylint
|
||||
"""
|
||||
Solr
|
||||
""".. sidebar:: info
|
||||
|
||||
- :origin:`solr.py <searx/engines/solr.py>`
|
||||
- `Solr <https://solr.apache.org>`_
|
||||
- `Solr Resources <https://solr.apache.org/resources.html>`_
|
||||
- `Install Solr <https://solr.apache.org/guide/installing-solr.html>`_
|
||||
|
||||
Solr_ is a popular search engine based on Lucene, just like Elasticsearch_. But
|
||||
instead of searching in indices, you can search in collections.
|
||||
|
||||
Example
|
||||
=======
|
||||
|
||||
This is an example configuration for searching in the collection
|
||||
``my-collection`` and get the results in ascending order.
|
||||
|
||||
.. code:: yaml
|
||||
|
||||
- name: solr
|
||||
engine: solr
|
||||
shortcut: slr
|
||||
base_url: http://localhost:8983
|
||||
collection: my-collection
|
||||
sort: asc
|
||||
enable_http: true
|
||||
|
||||
"""
|
||||
|
||||
# pylint: disable=global-statement
|
||||
|
||||
@ -1,7 +1,40 @@
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
# lint: pylint
|
||||
"""SQLite is a small, fast and reliable SQL database engine. It does not require
|
||||
any extra dependency.
|
||||
|
||||
"""SQLite database (Offline)
|
||||
Example
|
||||
=======
|
||||
|
||||
.. _MediathekView: https://mediathekview.de/
|
||||
|
||||
To demonstrate the power of database engines, here is a more complex example
|
||||
which reads from a MediathekView_ (DE) movie database. For this example of the
|
||||
SQlite engine download the database:
|
||||
|
||||
- https://liste.mediathekview.de/filmliste-v2.db.bz2
|
||||
|
||||
and unpack into ``searx/data/filmliste-v2.db``. To search the database use e.g
|
||||
Query to test: ``!mediathekview concert``
|
||||
|
||||
.. code:: yaml
|
||||
|
||||
- name: mediathekview
|
||||
engine: sqlite
|
||||
disabled: False
|
||||
categories: general
|
||||
result_template: default.html
|
||||
database: searx/data/filmliste-v2.db
|
||||
query_str: >-
|
||||
SELECT title || ' (' || time(duration, 'unixepoch') || ')' AS title,
|
||||
COALESCE( NULLIF(url_video_hd,''), NULLIF(url_video_sd,''), url_video) AS url,
|
||||
description AS content
|
||||
FROM film
|
||||
WHERE title LIKE :wildcard OR description LIKE :wildcard
|
||||
ORDER BY duration DESC
|
||||
|
||||
Implementations
|
||||
===============
|
||||
|
||||
"""
|
||||
|
||||
@ -26,14 +59,15 @@ def init(engine_settings):
|
||||
|
||||
@contextlib.contextmanager
|
||||
def sqlite_cursor():
|
||||
"""Implements a `Context Manager`_ for a :py:obj:`sqlite3.Cursor`.
|
||||
"""Implements a :py:obj:`Context Manager <contextlib.contextmanager>` for a
|
||||
:py:obj:`sqlite3.Cursor`.
|
||||
|
||||
Open database in read only mode: if the database doesn't exist.
|
||||
The default mode creates an empty file on the file system.
|
||||
Open database in read only mode: if the database doesn't exist. The default
|
||||
mode creates an empty file on the file system. See:
|
||||
|
||||
see:
|
||||
* https://docs.python.org/3/library/sqlite3.html#sqlite3.connect
|
||||
* https://www.sqlite.org/uri.html
|
||||
|
||||
"""
|
||||
uri = 'file:' + database + '?mode=ro'
|
||||
with contextlib.closing(sqlite3.connect(uri, uri=True)) as connect:
|
||||
|
||||
@ -1,17 +1,6 @@
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
# lint: pylint
|
||||
""".. _torznab engine:
|
||||
|
||||
==============
|
||||
Torznab WebAPI
|
||||
==============
|
||||
|
||||
.. contents:: Contents
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
Torznab_ is an API specification that provides a standardized way to query
|
||||
"""Torznab_ is an API specification that provides a standardized way to query
|
||||
torrent site for content. It is used by a number of torrent applications,
|
||||
including Prowlarr_ and Jackett_.
|
||||
|
||||
@ -55,7 +44,6 @@ The engine has the following settings:
|
||||
.. _Jackett-categories:
|
||||
https://github.com/Jackett/Jackett/wiki/Jackett-Categories
|
||||
|
||||
|
||||
Implementations
|
||||
===============
|
||||
|
||||
|
||||
@ -3,8 +3,55 @@
|
||||
"""The XPath engine is a *generic* engine with which it is possible to configure
|
||||
engines in the settings.
|
||||
|
||||
Here is a simple example of a XPath engine configured in the
|
||||
:ref:`settings engine` section, further read :ref:`engines-dev`.
|
||||
.. _XPath selector: https://quickref.me/xpath.html#xpath-selectors
|
||||
|
||||
Configuration
|
||||
=============
|
||||
|
||||
Request:
|
||||
|
||||
- :py:obj:`search_url`
|
||||
- :py:obj:`lang_all`
|
||||
- :py:obj:`soft_max_redirects`
|
||||
- :py:obj:`cookies`
|
||||
- :py:obj:`headers`
|
||||
|
||||
Paging:
|
||||
|
||||
- :py:obj:`paging`
|
||||
- :py:obj:`page_size`
|
||||
- :py:obj:`first_page_num`
|
||||
|
||||
Time Range:
|
||||
|
||||
- :py:obj:`time_range_support`
|
||||
- :py:obj:`time_range_url`
|
||||
- :py:obj:`time_range_map`
|
||||
|
||||
Safe-Search:
|
||||
|
||||
- :py:obj:`safe_search_support`
|
||||
- :py:obj:`safe_search_map`
|
||||
|
||||
Response:
|
||||
|
||||
- :py:obj:`no_result_for_http_status`
|
||||
|
||||
`XPath selector`_:
|
||||
|
||||
- :py:obj:`results_xpath`
|
||||
- :py:obj:`url_xpath`
|
||||
- :py:obj:`title_xpath`
|
||||
- :py:obj:`content_xpath`
|
||||
- :py:obj:`thumbnail_xpath`
|
||||
- :py:obj:`suggestion_xpath`
|
||||
|
||||
|
||||
Example
|
||||
=======
|
||||
|
||||
Here is a simple example of a XPath engine configured in the :ref:`settings
|
||||
engine` section, further read :ref:`engines-dev`.
|
||||
|
||||
.. code:: yaml
|
||||
|
||||
@ -16,6 +63,9 @@ Here is a simple example of a XPath engine configured in the
|
||||
title_xpath : //article[@class="repo-summary"]//a[@class="repo-link"]
|
||||
content_xpath : //article[@class="repo-summary"]/p
|
||||
|
||||
Implementations
|
||||
===============
|
||||
|
||||
"""
|
||||
|
||||
from urllib.parse import urlencode
|
||||
@ -26,7 +76,7 @@ from searx.network import raise_for_httperror
|
||||
|
||||
search_url = None
|
||||
"""
|
||||
Search URL of the engine. Example::
|
||||
Search URL of the engine. Example::
|
||||
|
||||
https://example.org/?search={query}&page={pageno}{time_range}{safe_search}
|
||||
|
||||
@ -74,30 +124,33 @@ soft_max_redirects = 0
|
||||
'''Maximum redirects, soft limit. Record an error but don't stop the engine'''
|
||||
|
||||
results_xpath = ''
|
||||
'''XPath selector for the list of result items'''
|
||||
'''`XPath selector`_ for the list of result items'''
|
||||
|
||||
url_xpath = None
|
||||
'''XPath selector of result's ``url``.'''
|
||||
'''`XPath selector`_ of result's ``url``.'''
|
||||
|
||||
content_xpath = None
|
||||
'''XPath selector of result's ``content``.'''
|
||||
'''`XPath selector`_ of result's ``content``.'''
|
||||
|
||||
title_xpath = None
|
||||
'''XPath selector of result's ``title``.'''
|
||||
'''`XPath selector`_ of result's ``title``.'''
|
||||
|
||||
thumbnail_xpath = False
|
||||
'''XPath selector of result's ``img_src``.'''
|
||||
'''`XPath selector`_ of result's ``img_src``.'''
|
||||
|
||||
suggestion_xpath = ''
|
||||
'''XPath selector of result's ``suggestion``.'''
|
||||
'''`XPath selector`_ of result's ``suggestion``.'''
|
||||
|
||||
cached_xpath = ''
|
||||
cached_url = ''
|
||||
|
||||
cookies = {}
|
||||
'''Some engines might offer different result based on cookies.
|
||||
Possible use-case: To set safesearch cookie.'''
|
||||
|
||||
headers = {}
|
||||
'''Some engines might offer different result based on cookies or headers.
|
||||
Possible use-case: To set safesearch cookie or header to moderate.'''
|
||||
'''Some engines might offer different result based headers. Possible use-case:
|
||||
To set header to moderate.'''
|
||||
|
||||
paging = False
|
||||
'''Engine supports paging [True or False].'''
|
||||
|
||||
Loading…
Reference in New Issue
Block a user