1
0
mirror of https://github.com/searxng/searxng.git synced 2024-11-16 17:40:11 +01:00
Commit Graph

102 Commits

Author SHA1 Message Date
Markus Heiser
1c79c1a3d2 [LXC] fedora33 EOL [1], upgrade to fedora35 [2]
[1] https://docs.fedoraproject.org/en-US/releases/eol/
[2] https://docs.fedoraproject.org/en-US/releases/f35/

Tested by::

    # build the container ..
    $ sudo -H ./utils/lxc.sh build searx-fedora35

    # open a shell in the container
    $ sudo -H ./utils/lxc.sh cmd searx-fedora35 bash
    [root@searx-fedora35 SearXNG]#

    # install a complete SearXNG suite ..
    [root@searx-fedora35 SearXNG]# ./utils/searx.sh install all
    ...

    # install apache to export the SearXNG instance by HTTP
    [root@searx-fedora35 SearXNG]# ./utils/searx.sh apache install
    ...
    INFO:  got 200 from http://10.174.184.94/searx

To build wheel `python3-devel` needs to be added to SEARX_PACKAGES_fedora::

    |searx|   × Building wheel for setproctitle (pyproject.toml) did not run successfully.
    |searx|   │ exit code: 1
    ...
    |searx|       In file included from src/spt.h:15,
    |searx|                        from src/setproctitle.c:14:
    |searx|       src/spt_python.h:16:10: fatal error: Python.h: No such file or directory
    |searx|          16 | #include <Python.h>
    |searx|             |          ^~~~~~~~~~
    |searx|       compilation terminated.
    |searx|       error: command '/usr/bin/gcc' failed with exit code 1
    |searx|       [end of output]
    ...
    |searx|   ERROR: Failed building wheel for setproctitle

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-03-12 17:44:00 +01:00
Markus Heiser
5eedd5b72a [fix] socket in SearXNG's uWSGI app (searxng.ini)
Use SEARX_UWSGI_SOCKET in uWSGI systemd service

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-03-08 17:44:27 +01:00
Markus Heiser
e48e2a9914 [fix] APACHE_SEARX_SITE=searxng.conf renamed in 2b125214
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-03-08 17:18:02 +01:00
Markus Heiser
ee8b6aac36 [fix] searx.sh update: replace git pull by a hard reset
If the fetched branch has been rebased a 'git pull' will fails.  To get fetched
branch in the working tree, a git reset is needed.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-11-26 15:15:17 +01:00
Markus Heiser
1dae0c0be0 [brand] SearXNG - docs rename links and fix documentation
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-11-18 18:27:26 +01:00
Markus Heiser
dc1442a2d1 [mod] Tools to install and maintain NVM versions manager for Node.js
[1] https://github.com/nvm-sh/nvm

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-11-16 23:31:11 +01:00
Markus Heiser
db6a4d13e7 [mod] searx, filtron & morty scripts: add 'reinstall all' command
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-10-30 16:11:01 +02:00
Markus Heiser
4167bf15dc [mod] utils/searx.sh - apache, nginx & uwsgi check old 'searx' installations
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-10-30 16:11:01 +02:00
Markus Heiser
38482739de [mod] utils/searx.sh - check a SearXNG installation
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-10-30 16:11:01 +02:00
Markus Heiser
2b1252148d [brand] SearXNG - nginx & apache searxng.conf, uwsgi searxng.conf
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-10-30 16:11:01 +02:00
Markus Heiser
a9fc4885f2 [brand] SearXNG - bash env SEARXNG_URL
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-10-30 16:11:01 +02:00
Markus Heiser
f46508b35f [mod] utils/searxng_check.py - check a SearXNG installation
Impplement a script to check a SearXNG installation::

   ./utils/searx.sh install check

Related-to: https://github.com/searxng/searxng/issues/450
Related-to: https://github.com/searxng/searxng/pull/446#issuecomment-954599668
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-10-29 15:51:30 +00:00
Markus Heiser
60edf2623d [brand] SearXNG - reference /etc/searxng/settings.yml
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-10-11 12:28:55 +00:00
Alexandre Flament
253b850376 SearXNG: SEARXNG_SETTINGS_PATH 2021-10-02 17:18:05 +02:00
Alexandre Flament
9e266ecad3 SearXNG: SEARX_BIND_ADDRESS 2021-10-02 16:58:09 +02:00
Alexandre Flament
6443ed7562 SearXNG: SEARXNG_PORT 2021-10-02 16:56:24 +02:00
Markus Heiser
8ba4dc26d1 [fix] utils/searx.sh install settings - do not eval the template
The settings template in utils/templates/etc/searx/settings.yml
does not include any replacements (shell environment variables).

BTW: utils/filtron.sh & utils/lib.sh
  Add some more messages to installation procedures, to be more clear.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-08-20 14:21:54 +02:00
Markus Heiser
5b3eeb6e54 [mod] searx.sh - improve some messages prompted while installation
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-07-30 14:53:09 +02:00
Markus Heiser
85e0908e4f [fix] searx.sh - get_installed_version_variables()
Fix the quotation marks in the command line to run

    python -m searx.version

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-07-30 14:53:09 +02:00
Alexandre Flament
4b43775c91 version based on the git repository
This commit remove the need to update the brand for GIT_URL and GIT_BRANCH:
there are read from the git repository.

It is possible to call python -m searx.version freeze to freeze the current version.
Useful when the code is installed outside git (distro package, docker, etc...)
2021-07-30 14:40:09 +02:00
Markus Heiser
57b8f340a6 [mod] utils/searx.sh - add command 'inspect settings'
Inspect YAML setting <key> from SearXNG instance (${SEARX_SRC})::

  utils/searx.sh inspect settings server.base_url

utils/lib_install.sh
  should not log on stdout which is used for the output of function
  prompt_installation_setting().  Turned build_msg (stdout) into
  info_msg (stderr).

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-07-24 16:24:22 +02:00
Markus Heiser
28c874bf3f [brand] searx.sh & morty.sh - renamed 'searx' into 'SearXNG'
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-07-21 14:39:02 +02:00
Markus Heiser
7167a7ee36 [mod] utils/.searx.sh: add commands: install [dot-config|init-src]
This patch implements two new commands:

install dot-config :
  - copy ./config.sh to ${SEARX_SRC}
  - implemented in new function install_DOT_CONFIG

install init-src:
  - sync files (SEARX_SRC_INIT_FILES) with ${SEARX_SRC}
  - implemented in new function init_SEARX_SRC()

new functions: verify_continue_install() and prompt_installation_status()
  These functions are used in installation procedures to get the status of the
  installation procedure.  The status is based on the existing function:

    usage: install_searx_get_state

        Prompts a string indicating the status of the installation procedure

    missing-searx-clone:
       There is no clone at ${SEARX_SRC}
    missing-searx-pyenv:
       There is no pyenv in ${SEARX_PYENV}
    installer-modified:
       There are files modified locally in the installer (clone),
       see ${SEARX_SRC_INIT_FILES} description.
    python-installed:
       Scripts can be executed in instance's environment
       - user:  ${SERVICE_USER}
       - pyenv: ${SEARX_PYENV}

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-07-21 14:38:59 +02:00
Markus Heiser
f61c918dd4 [mod] normalize .config.sh with settings.yml
In commit 94851790 we have centralized all SearXNG setups in the settings.yml
file:

  94851790 [mod] move brand options from Makefile to settings.yml

This step has not yet been completed for the installation procedures!  Since all
SearXNG setups are done in the settings.yml these environment variables needs to
be removed from the ./conf.sh file.  Scripts and other tasks running outside of
an instance got the needed values from the ./utils/brand.env file.

By example: ATM the environment variables of the ./config.sh file are in
conflict with them from settings.yml:

  - PUBLIC_URL          --> {server:base_url}
  - SEARX_INTERNAL_HTTP --> {server:bind_address}.{server:port}
  - GIT_BRANCH          --> {brand:GIT_URL}

These environment variable of a SearXNG instance and additional

  - SEARX_SETTINGS_TEMPLATE

has been remove from the '.config.sh' file.  With this patch, the main focus of
./conf.sh resists on environment variables needed for the installation of morty,
filtron software.

modified  .config.sh:
  - removed no longer supported variables (see above)
  - add comment about: SearXNG setup in settings.yml

modified utils/searx.sh:
  - SEARX_INTERNAL_HTTP no longer take from .config.sh
  - SEARX_SETTINGS_PATH /etc/searx/settings.yml
  - SEARX_SETTINGS_TEMPLATE obsolete

modified utils/lib_install.sh:
  Initialize environment variables SEARX_PYENV, SEARX_SETTINGS_PATH and
  PUBLIC_URL.

modified:  utils/morty.sh
  Add missing hint about SEARX_SETTINGS_PATH and move PUBLIC_URL to
  utils/lib_install.sh

modified:  utils/morty.sh
  Move PUBLIC_URL to utils/lib_install.sh

Renamed utils/templates/etc/searx/use_default_settings.yml -> settings.yml
  - removed option which can't be modified after installation
  - add some comments with examples

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-07-21 14:38:57 +02:00
Markus Heiser
414a6105e7 [mod] load .config.sh from installation (utils/lib_install.sh)
**new** utils/lib_install.sh:
  Used to initialize installation procedures

  - Modified source_dot_config function that
    - loads .config.sh from an existing installation (at SEARX_SRC).
    - initialize **SEARX_SRC_INIT_FILES**
  - functions like:
    - install_log_searx_instance()
    - install_searx_get_state()

**modified** utils/searx.sh
    - obsolete environment SEARX_INSTANCE_NAME has been replaced
      by string 'SearXNG'.

**modified** utils/filtron.sh, utils/morty.sh, utils/searx.sh
  - source utils/lib_install.sh
  - normalize logging of environment variables using new function
    install_log_searx_instance()

**modified** utils/lib.sh
  - fix marginal typos

**Installation scripts**
  The utils/lib_install.sh is sourced by the installations scripts:

  - utils/searx.sh
  - utils/morty.sh
  - utils/filtron.sh

  If '${SEARX_SRC}/.config.sh' exists, the modified source_dot_config() function
  loads this configuration (instead of './.config.sh').

**SEARX_SRC_INIT_FILES**
  Array of file names to sync into a installation at $SEARX_SRC.  The file names
  are relative to the $REPO_ROOT.  Set by function init_SEARX_SRC_INIT_FILES().
  Most often theses are files like:

  - .config.sh
  - searx/settings.yml
  - utils/brand.env
  - ...

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-07-21 14:38:33 +02:00
Markus Heiser
b5f5ddd5a6 [fix] utils/searx.sh: signature of YAML settings has been changed
In 5a7b12ee we changed the signature of the YAML settings, this patch takes this
into account.

Related-to: 5a7b12ee [yamllint] searx/settings.yml
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-23 09:56:59 +02:00
Alexandre Flament
dca3bcca9e [mod] simple theme: include fonts
"npm run webfont" to build the fonts directory.
It requires fontforge and ttfautohint distro packages.

partial revert of commit 7137d2893f
2021-06-22 08:15:17 +02:00
Markus Heiser
ae677cb64b [enh] ./manage node.env - check build tools first
The node.env build environment require npm, ttfautohint and fontforge installed
in the OS.  These tools can be installed by::

    sudo -H ./utils/searx.sh install buildhost

If one of the tools is not installed, the script node.env stops with a
appropriate message.

BTW: We ignore CentOS-7 as developer & build platform

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-17 09:23:02 +02:00
Alexandre Flament
7137d2893f [mod] simple theme: remove src/less/ion.less
This file is generated by webfont.
* It is now generated as searx/static/themes/simple/ion.less
* It is generated before the .less compilation.
* .gitignore includes this file

Add two new package depedencies: fontforge ttfautohint
See utils/searx.sh
2021-06-16 14:30:09 +02:00
Markus Heiser
666bf1d32b [fix] typo: 'interactiv' --> 'interactive'
Reported-by: https://github.com/searxng/searxng/pull/38#pullrequestreview-646940531
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-04-28 16:29:03 +02:00
Markus Heiser
d0e371f474 [mod] replace makefile boilerplate by 'manage' script
Replaces the make targets with the bash scripts

Signed-off-by: Markus Heiser <markus@darmarit.de>
2021-04-23 11:26:22 +02:00
Markus Heiser
f9b05a6c44 [fix] utils/searx.sh - Double quote to prevent globbing
Fix issues reported by `make test.sh`

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-04-23 11:14:29 +02:00
3nprob
0fb423ea59 Allow overriding env vars SEARX_SETTINGS_PATH,UWSGI_SETTINGS_PATH 2021-04-08 12:56:15 +09:00
Alexandre Flament
4c2a8aea39
Merge pull request #2620 from return42/fix-git
[fix] support git versions <v2.22
2021-03-04 11:17:24 +01:00
Markus Heiser
2e58988191 [fix] utils/serax.sh create_pyenv() - drop duplicate 'pip install .'
The wrong and unnecessary `pip install .` is executed in /usr/local/searx and is
responsible for the error message:

    ERROR: File "setup.py" not found. Directory cannot be installed in editable mode: /usr/local/searx

The correct pip-install comes right after changing to `cd ${SEARX_SRC}`.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-03-03 18:57:20 +01:00
Markus Heiser
237b1cfdb3 [fix] support git versions <v2.22
LTS distros like Ubuntu 18.04 do not ship a up-to-date version of git.::

    $ sudo -H ./utils/lxc.sh cmd searx-ubu1804 git --version
    ...
    git version 2.17.1

The option `--show-current` was added in git v2.22, the alternative to this
option is::

    git rev-parse --abbrev-ref HEAD

Issue when using option `--show-current`::

    [searx-ubu1804] Clone searx sources
    [searx-ubu1804] -------------------
    [searx-ubu1804]
    [searx-ubu1804] error: unknown option `show-current'
    [searx-ubu1804] usage: git branch [<options>] [-r | -a] [--merged | --no-merged]
    ....

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-03-03 18:22:41 +01:00
Alexandre Flament
aa887eb375 [mod] checker : replace pycld3 by langdetect
pycld3 requires the native library cld3
langdetect is a pure python package
2021-01-19 21:26:04 +01:00
Markus Heiser
c4a98862bf [fix] setup.py requires pyyaml installed
pip install -e .
...
Obtaining file:///usr/local/searx/searx-src
    ERROR: Command errored out with exit status 1:
     command: /usr/local/searx/searx-pyenv/bin/python3 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/usr/local/searx/searx-src/setup.py'"'"'; __file__='"'"'/usr/local/searx/searx-src/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'rn'"'"', '"'"'n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-vzer91m2
         cwd: /usr/local/searx/searx-src/
    Complete output (9 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/usr/local/searx/searx-src/setup.py", line 10, in <module>
        from searx.version import VERSION_STRING
      File "/usr/local/searx/searx-src/searx/__init__.py", line 19, in <module>
        import searx.settings_loader
      File "/usr/local/searx/searx-src/searx/settings_loader.py", line 8, in <module>
        import yaml
    ModuleNotFoundError: No module named 'yaml'
    ----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-01-16 08:58:13 +01:00
Alexandre Flament
8cbc9f2d58 [enh] add checker 2021-01-12 11:47:17 +01:00
Markus Heiser
89e6ba5af7 [mod] add Ubuntu 20.10 image to the LXC suite
Ubuntu 20.10 image [1] is avialable from https://images.linuxcontainers.org

BTW: replace builhost package 'ttf-dejavu' by more common meta package
'fonts-dejavu' ('ttf-dejavu' is no longer available in 20.10).

[1] d161de601b

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-01-02 17:27:16 +01:00
Markus Heiser
5fc9273e23 [mod] utils/searx.sh install: create branch if not already exists
This patch keeps an eye on:

1. When installing searx, the branch to install needs to be a local branch.
2. The branch to install can be different from the current branch.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-12-23 16:06:06 +01:00
Markus Heiser
0fc3759d74 [fix] use SEARX_SETTINGS_TEMPLATE from .config environment
In commit a70b9b9f the SEARX_SETTINGS_TEMPLATE environment was added to the
.config file, but was not use in utils/searx.sh.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-12-23 14:10:07 +01:00
Markus Heiser
1a4524e03c [mod] remove obsolete virtualenv command
Installation and use of the command 'virtualenv' was only needed in py2 and py2
is no longer suported by searx.  In py3 the command is replaced by 'python -m
venv'.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-12-18 22:31:13 +01:00
Markus Heiser
a70b9b9f61 [doc] recommend to use 'use_default_settings=True'
Since #2291 is merged, it is recommend to use::

  use_default_settings=True

1. Add a template file use_default_settings.yml::

    SEARX_SETTINGS_TEMPLATE="${REPO_ROOT}/utils/templates/etc/searx/use_default_settings.yml"

2. In Chapter "Configuration" recommend to make use of
   'use_default_settings=True' and describe it

3. Rewrite of docs/admin/settings.rst
   - move chapter 'settings.yml location' to the top
   - update and split chapter 'Global Settings'

4. Add environment SEARX_SETTINGS_TEMPLATE to .config.sh

5. Use environment $SEARX_SETTINGS_TEMPLATE in the utils/searx.sh script

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-12-08 20:19:10 +01:00
Markus Heiser
69a61ce333 [fix] remove doc-build for centos-7 from './utils/searx.sh doc'
Since there is no centos-7 documentation 'searx.sh doc' produce an error
message::

    ERROR: centos-: uWSGI not yet implemented

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-09-20 11:11:14 +02:00
Eliesemoule
1986b5ecac
Utility scripts adapted to run on CentOS 7 (#2112) 2020-08-01 22:12:44 +02:00
Markus Heiser
18805ee352 [fix] markup of auto generated installation instruction (uWSGI part)
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-06-24 15:15:15 +02:00
Markus Heiser
51428de0fe [fix] add missing latexmk package to debian's BUILD_PACKAGES
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-06-18 18:35:59 +02:00
Markus Heiser
f576cadfba [fix] installation instructions: pip install -e searx (setup.py)
In the past we did not really install searx into the virtualenv.  This is a
problem, since entry-points and other python installation stuff is not trigger.

See discussion: https://github.com/asciimoo/searx/pull/1938#issuecomment-632056508

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-05-21 15:46:43 +02:00
Markus Heiser
0973255aff [fix] fatal: repository '/root/searx' does not exist
The $SERVICE_USER (searx) needs read permission to clone repository.  The
installation script stops if permission is not granted.

- https://github.com/asciimoo/searx/pull/1803#issuecomment-616406528

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-04-30 17:49:26 +02:00