mirror of
https://github.com/searxng/searxng.git
synced 2024-11-19 19:00:10 +01:00
Merge pull request #2450 from return42/lxc-doc
[doc] update & fix documentation of the "SearXNG LXC suite"
This commit is contained in:
Markus Heiser
GitHub
commit
ddbab4b3c9
@ -16,6 +16,12 @@ In this article we will show, how you can make use of Linux Containers (LXC_) in
|
||||
should have a serious meaning about the terms: *distributed*, *merge* and
|
||||
*linux container*.
|
||||
|
||||
**hint**
|
||||
|
||||
If you have issues with the internet connectivity of your containers read
|
||||
section :ref:`internet connectivity docker`.
|
||||
|
||||
|
||||
.. contents:: Contents
|
||||
:depth: 2
|
||||
:local:
|
||||
@ -25,38 +31,38 @@ In this article we will show, how you can make use of Linux Containers (LXC_) in
|
||||
Motivation
|
||||
==========
|
||||
|
||||
Usually in our development cycle, we edit the sources and run some test and/or
|
||||
builds by using ``make`` :ref:`[ref] <makefile>` before we commit. This cycle
|
||||
is simple and perfect but might fail in some aspects we should not overlook.
|
||||
Most often in our development cycle, we edit the sources and run some test
|
||||
and/or builds by using ``make`` :ref:`[ref] <makefile>` before we commit. This
|
||||
cycle is simple and perfect but might fail in some aspects we should not
|
||||
overlook.
|
||||
|
||||
**The environment in which we run all our development processes matters!**
|
||||
|
||||
The :ref:`makefile` and the :ref:`make install` encapsulate a lot for us, but
|
||||
they do not have access to all prerequisites. For example, there may have
|
||||
dependencies on packages that are installed on the developer's desktop, but
|
||||
these tools do not have access to all prerequisites. For example, there may
|
||||
have dependencies on packages that are installed on developer's desktop, but
|
||||
usually are not preinstalled on a server or client system. Another example is;
|
||||
settings have been made to the software on developer's desktop that would never
|
||||
be set on a *production* system.
|
||||
|
||||
**Linux Containers are isolate environments and not to mix up all the
|
||||
prerequisites from various projects on developer's desktop is always a good
|
||||
choice.**
|
||||
**Linux Containers are isolate environments**, we use them to not mix up all
|
||||
the prerequisites from various projects on developer's desktop.
|
||||
|
||||
The scripts from :ref:`searx_utils` can divide in those to install and maintain
|
||||
software:
|
||||
software
|
||||
|
||||
- :ref:`searxng.sh`
|
||||
|
||||
and the script :ref:`lxc.sh`, with we can scale our installation, maintenance or
|
||||
even development tasks over a stack of isolated containers / what we call the:
|
||||
and the script
|
||||
|
||||
**SearXNG LXC suite**
|
||||
- :ref:`lxc.sh`
|
||||
|
||||
.. hint::
|
||||
with we can scale our installation, maintenance or even development tasks over a
|
||||
stack of isolated containers / what we call the:
|
||||
|
||||
If you see any problems with the internet connectivity of your
|
||||
containers read section :ref:`internet connectivity docker`.
|
||||
- :ref:`searxng lxc suite`
|
||||
|
||||
.. _lxcdev install searxng:
|
||||
|
||||
Gentlemen, start your engines!
|
||||
==============================
|
||||
@ -69,7 +75,7 @@ once:
|
||||
|
||||
.. tabs::
|
||||
|
||||
.. group-tab:: desktop
|
||||
.. group-tab:: desktop (HOST)
|
||||
|
||||
.. code:: bash
|
||||
|
||||
@ -81,7 +87,7 @@ fork:
|
||||
|
||||
.. tabs::
|
||||
|
||||
.. group-tab:: desktop
|
||||
.. group-tab:: desktop (HOST)
|
||||
|
||||
.. code:: bash
|
||||
|
||||
@ -89,76 +95,61 @@ fork:
|
||||
$ git clone https://github.com/searxng/searxng.git searxng
|
||||
$ cd searxng
|
||||
|
||||
.. sidebar:: The ``searxng-archlinux`` container
|
||||
|
||||
is the base of all our exercises here.
|
||||
|
||||
The :ref:`lxc-searxng.env` consists of several images, see ``export
|
||||
LXC_SUITE=(...`` near by :origin:`utils/lxc-searxng.env#L19`. For this blog post
|
||||
we exercise on a archlinux_ image. The container of this image is named
|
||||
``searxng-archlinux``. Lets build the container, but be sure that this container
|
||||
does not already exists, so first lets remove possible old one:
|
||||
LXC_SUITE=(...`` near by :origin:`utils/lxc-searxng.env#L19`.
|
||||
For this blog post we exercise on a archlinux_ image. The container of this
|
||||
image is named ``searxng-archlinux``.
|
||||
|
||||
Lets build the container, but be sure that this container does not already
|
||||
exists, so first lets remove possible old one:
|
||||
|
||||
.. tabs::
|
||||
|
||||
.. group-tab:: desktop
|
||||
.. group-tab:: desktop (HOST)
|
||||
|
||||
.. code:: bash
|
||||
|
||||
$ sudo -H ./utils/lxc.sh remove searxng-archlinux
|
||||
$ sudo -H ./utils/lxc.sh build searxng-archlinux
|
||||
|
||||
.. sidebar:: The ``searxng-archlinux`` container
|
||||
|
||||
is the base of all our exercises here.
|
||||
.. sidebar:: further read
|
||||
|
||||
In this container we install all services :ref:`including searx, morty & filtron
|
||||
<lxc.sh install suite>` in once:
|
||||
- :ref:`lxc.sh install suite`
|
||||
- :ref:`installation nginx`
|
||||
|
||||
To install the complete :ref:`SearXNG suite <searxng lxc suite>` and the HTTP
|
||||
proxy :ref:`installation nginx` into the archlinux container run:
|
||||
|
||||
.. tabs::
|
||||
|
||||
.. group-tab:: desktop
|
||||
.. group-tab:: desktop (HOST)
|
||||
|
||||
.. code:: bash
|
||||
|
||||
$ sudo -H ./utils/lxc.sh install suite searxng-archlinux
|
||||
|
||||
To proxy HTTP from filtron and morty in the container to the outside of the
|
||||
container, install nginx into the container. Once for the bot blocker filtron:
|
||||
|
||||
.. tabs::
|
||||
|
||||
.. group-tab:: desktop
|
||||
|
||||
.. code:: bash
|
||||
|
||||
$ sudo -H ./utils/lxc.sh cmd searxng-archlinux \
|
||||
./utils/filtron.sh nginx install
|
||||
$ sudo -H ./utils/lxc.sh cmd -- FORCE_TIMEOUT=0 ./utils/searxng.sh install nginx
|
||||
$ sudo ./utils/lxc.sh show suite | grep SEARXNG_URL
|
||||
...
|
||||
INFO: got 429 from http://10.174.184.156/searx
|
||||
|
||||
and once for the content sanitizer (content proxy morty):
|
||||
|
||||
.. tabs::
|
||||
|
||||
.. group-tab:: desktop
|
||||
|
||||
.. code:: bash
|
||||
|
||||
$ sudo -H ./utils/lxc.sh cmd searxng-archlinux \
|
||||
./utils/morty.sh nginx install
|
||||
...
|
||||
INFO: got 200 from http://10.174.184.156/morty/
|
||||
[searxng-archlinux] SEARXNG_URL : http://n.n.n.140/searxng
|
||||
|
||||
.. sidebar:: Fully functional SearXNG suite
|
||||
|
||||
From here on you have a fully functional SearXNG suite running with bot
|
||||
blocker (filtron) and WEB content sanitizer (content proxy morty), both are
|
||||
needed for a *privacy protecting* search engine.
|
||||
From here on you have a fully functional SearXNG suite (including a
|
||||
:ref:`redis db`).
|
||||
|
||||
On your system, the IP of your ``searxng-archlinux`` container differs from
|
||||
http://10.174.184.156/searx, just open the URL reported in your installation
|
||||
protocol in your WEB browser from the desktop to test the instance from outside
|
||||
of the container.
|
||||
In such a SearXNG suite admins can maintain and access the debug log of the
|
||||
services quite easy.
|
||||
|
||||
In such a earXNG suite admins can maintain and access the debug log of the
|
||||
different services quite easy.
|
||||
In the example above the SearXNG instance in the container is wrapped to
|
||||
``http://n.n.n.140/searxng`` to the HOST system. Note, on your HOST system, the
|
||||
IP of your ``searxng-archlinux`` container is different to this example. To
|
||||
test the instance in the conatiner from outside of the container, in your WEB
|
||||
browser on your desktop just open the URL reported in your installation
|
||||
|
||||
.. _working in containers:
|
||||
|
||||
@ -166,77 +157,76 @@ In containers, work as usual
|
||||
============================
|
||||
|
||||
Usually you open a root-bash using ``sudo -H bash``. In case of LXC containers
|
||||
open the root-bash in the container using ``./utils/lxc.sh cmd
|
||||
searxng-archlinux``:
|
||||
open the root-bash in the container is done by the ``./utils/lxc.sh cmd
|
||||
searxng-archlinux`` command:
|
||||
|
||||
.. tabs::
|
||||
|
||||
.. group-tab:: desktop
|
||||
.. group-tab:: desktop (HOST)
|
||||
|
||||
.. code:: bash
|
||||
|
||||
$ sudo -H ./utils/lxc.sh cmd searxng-archlinux bash
|
||||
INFO: [searxng-archlinux] bash
|
||||
[root@searxng-archlinux searx]# pwd
|
||||
/share/searxng
|
||||
[root@searxng-archlinux SearXNG]$
|
||||
|
||||
The prompt ``[root@searxng-archlinux ...]`` signals, that you are the root user in
|
||||
the searxng-container. To debug the running SearXNG instance use:
|
||||
The prompt ``[root@searxng-archlinux ...]`` signals, that you are the root user
|
||||
in the container (GUEST). To debug the running SearXNG instance use:
|
||||
|
||||
.. tabs::
|
||||
|
||||
.. group-tab:: root@searxng-archlinux
|
||||
.. group-tab:: ``[root@searxng-archlinux SearXNG]`` (GUEST)
|
||||
|
||||
.. code:: bash
|
||||
|
||||
$ ./utils/searx.sh inspect service
|
||||
$ ./utils/searxng.sh instance inspect
|
||||
...
|
||||
use [CTRL-C] to stop monitoring the log
|
||||
...
|
||||
|
||||
Back in the browser on your desktop open the service http://10.174.184.156/searx
|
||||
.. group-tab:: desktop (HOST)
|
||||
|
||||
.. code:: bash
|
||||
|
||||
$ sudo -H ./utils/lxc.sh cmd searxng-archlinux ./utils/searxng.sh instance inspect
|
||||
...
|
||||
use [CTRL-C] to stop monitoring the log
|
||||
...
|
||||
|
||||
|
||||
Back in the browser on your desktop open the service http://n.n.n.140/searxng
|
||||
and run your application tests while the debug log is shown in the terminal from
|
||||
above. You can stop monitoring using ``CTRL-C``, this also disables the *"debug
|
||||
option"* in SearXNG's settings file and restarts the SearXNG uwsgi application.
|
||||
To debug services from filtron and morty analogous use:
|
||||
|
||||
Another point we have to notice is that the service (:ref:`SearXNG <searxng.sh>`
|
||||
Another point we have to notice is that the service :ref:`SearXNG <searxng.sh>`
|
||||
runs under dedicated system user account with the same name (compare
|
||||
:ref:`create searxng user`). To get a shell from these accounts, simply call:
|
||||
:ref:`create searxng user`). To get a login shell from these accounts, simply
|
||||
call:
|
||||
|
||||
.. tabs::
|
||||
|
||||
.. group-tab:: root@searxng-archlinux
|
||||
.. group-tab:: ``[root@searxng-archlinux SearXNG]`` (GUEST)
|
||||
|
||||
.. code:: bash
|
||||
|
||||
$ ./utils/searxng.sh instance cmd bash
|
||||
$ ./utils/searxng.sh instance cmd bash -l
|
||||
(searx-pyenv) [searxng@searxng-archlinux ~]$ pwd
|
||||
/usr/local/searxng
|
||||
|
||||
To get in touch, open a shell from the service user (searxng@searxng-archlinux):
|
||||
|
||||
.. tabs::
|
||||
|
||||
.. group-tab:: desktop
|
||||
.. group-tab:: desktop (HOST)
|
||||
|
||||
.. code:: bash
|
||||
|
||||
$ sudo -H ./utils/lxc.sh cmd searxng-archlinux ./utils/searxng.sh instance cmd bash
|
||||
INFO: [searxng-archlinux] ./utils/searxng.sh instance cmd bash
|
||||
[searxng@searxng-archlinux ~]$
|
||||
$ sudo -H ./utils/lxc.sh cmd searxng-archlinux ./utils/searxng.sh instance cmd bash -l
|
||||
INFO: [searxng-archlinux] ./utils/searxng.sh instance cmd bash -l
|
||||
(searx-pyenv) [searxng@searxng-archlinux ~]$ pwd
|
||||
/usr/local/searxng
|
||||
|
||||
The prompt ``[searxng@searxng-archlinux]`` signals that you are logged in as system
|
||||
user ``searx`` in the ``searxng-archlinux`` container and the python *virtualenv*
|
||||
user ``searxng`` in the ``searxng-archlinux`` container and the python *virtualenv*
|
||||
``(searxng-pyenv)`` environment is activated.
|
||||
|
||||
.. tabs::
|
||||
|
||||
.. group-tab:: searxng@searxng-archlinux
|
||||
|
||||
.. code:: bash
|
||||
|
||||
(searxng-pyenv) [searxng@searxng-archlinux ~]$ pwd
|
||||
/usr/local/searxng
|
||||
|
||||
|
||||
Wrap production into developer suite
|
||||
====================================
|
||||
@ -256,12 +246,11 @@ With the use of the :ref:`searxng.sh` the SearXNG service was installed as
|
||||
|
||||
.. tabs::
|
||||
|
||||
.. group-tab:: desktop
|
||||
.. group-tab:: uwsgi@searxng
|
||||
|
||||
.. code:: bash
|
||||
|
||||
$ sudo -H ./utils/lxc.sh cmd searxng-archlinux \
|
||||
systemctl stop uwsgi@searxng
|
||||
$ sudo -H ./utils/lxc.sh cmd searxng-archlinux systemctl stop uwsgi@searxng
|
||||
|
||||
With the command above, we stopped the SearXNG uWSGI-App in the archlinux
|
||||
container.
|
||||
@ -278,17 +267,25 @@ least you should attend the settings of ``uid``, ``chdir``, ``env`` and
|
||||
virtualenv = /usr/local/searxng/searxng-pyenv
|
||||
pythonpath = /usr/local/searxng/searxng-src
|
||||
|
||||
If you have read the :ref:`"Good to know section" <lxc.sh>` you remember, that
|
||||
each container shares the root folder of the repository and the command
|
||||
``utils/lxc.sh cmd`` handles relative path names **transparent**. To wrap the
|
||||
SearXNG installation into a developer one, we simple have to create a smylink to
|
||||
the **transparent** reposetory from the desktop. Now lets replace the
|
||||
repository at ``searxng-src`` in the container with the working tree from outside
|
||||
of the container:
|
||||
If you have read the :ref:`Good to know` you remember, that each container
|
||||
shares the root folder of the repository and the command ``utils/lxc.sh cmd``
|
||||
handles relative path names **transparent**.
|
||||
|
||||
To wrap the SearXNG installation in the container into a developer one, we
|
||||
simple have to create a smylink to the **transparent** reposetory from the
|
||||
desktop. Now lets replace the repository at ``searxng-src`` in the container
|
||||
with the working tree from outside of the container:
|
||||
|
||||
.. tabs::
|
||||
|
||||
.. group-tab:: container becomes a developer suite
|
||||
.. group-tab:: ``[root@searxng-archlinux SearXNG]`` (GUEST)
|
||||
|
||||
.. code:: bash
|
||||
|
||||
$ mv /usr/local/searxng/searxng-src /usr/local/searxng/searxng-src.old
|
||||
$ ln -s /share/SearXNG/ /usr/local/searxng/searxng-src
|
||||
|
||||
.. group-tab:: desktop (HOST)
|
||||
|
||||
.. code:: bash
|
||||
|
||||
@ -296,7 +293,7 @@ of the container:
|
||||
mv /usr/local/searxng/searxng-src /usr/local/searxng/searxng-src.old
|
||||
|
||||
$ sudo -H ./utils/lxc.sh cmd searxng-archlinux \
|
||||
ln -s /share/searx/ /usr/local/searxng/searxng-src
|
||||
ln -s /share/SearXNG/ /usr/local/searxng/searxng-src
|
||||
|
||||
Now we can develop as usual in the working tree of our desktop system. Every
|
||||
time the software was changed, you have to restart the SearXNG service (in the
|
||||
@ -304,48 +301,83 @@ container):
|
||||
|
||||
.. tabs::
|
||||
|
||||
.. group-tab:: desktop
|
||||
.. group-tab:: uwsgi@searxng
|
||||
|
||||
.. code:: bash
|
||||
|
||||
$ sudo -H ./utils/lxc.sh cmd searxng-archlinux \
|
||||
systemctl restart uwsgi@searx
|
||||
$ sudo -H ./utils/lxc.sh cmd searxng-archlinux systemctl restart uwsgi@searxng
|
||||
|
||||
|
||||
Remember: :ref:`working in containers` .. here are just some examples from my
|
||||
daily usage:
|
||||
|
||||
To *inspect* the SearXNG instance (already described above):
|
||||
|
||||
.. tabs::
|
||||
|
||||
.. group-tab:: desktop
|
||||
|
||||
To *inspect* the SearXNG instance (already described above):
|
||||
.. group-tab:: ``[root@searxng-archlinux SearXNG]`` (GUEST)
|
||||
|
||||
.. code:: bash
|
||||
|
||||
$ sudo -H ./utils/lxc.sh cmd searxng-archlinux \
|
||||
./utils/searx.sh inspect service
|
||||
$ ./utils/searx.sh inspect service
|
||||
|
||||
.. group-tab:: desktop (HOST)
|
||||
|
||||
.. code:: bash
|
||||
|
||||
$ sudo -H ./utils/lxc.sh cmd searxng-archlinux ./utils/searx.sh inspect service
|
||||
|
||||
Run :ref:`makefile`, e.g. to test inside the container:
|
||||
|
||||
.. tabs::
|
||||
|
||||
.. group-tab:: ``[root@searxng-archlinux SearXNG]`` (GUEST)
|
||||
|
||||
.. code:: bash
|
||||
|
||||
$ sudo -H ./utils/lxc.sh cmd searxng-archlinux \
|
||||
make test
|
||||
$ make test
|
||||
|
||||
.. group-tab:: desktop (HOST)
|
||||
|
||||
.. code:: bash
|
||||
|
||||
$ sudo -H ./utils/lxc.sh cmd searxng-archlinux make test
|
||||
|
||||
|
||||
|
||||
To install all prerequisites needed for a :ref:`buildhosts`:
|
||||
|
||||
.. tabs::
|
||||
|
||||
.. group-tab:: ``[root@searxng-archlinux SearXNG]`` (GUEST)
|
||||
|
||||
.. code:: bash
|
||||
|
||||
$ sudo -H ./utils/lxc.sh cmd searxng-archlinux \
|
||||
./utils/searxng.sh install buildhost
|
||||
$ ./utils/searxng.sh install buildhost
|
||||
|
||||
.. group-tab:: desktop (HOST)
|
||||
|
||||
.. code:: bash
|
||||
|
||||
$ sudo -H ./utils/lxc.sh cmd searxng-archlinux ./utils/searxng.sh install buildhost
|
||||
|
||||
|
||||
To build the docs on a buildhost :ref:`buildhosts`:
|
||||
|
||||
.. tabs::
|
||||
|
||||
.. group-tab:: ``[root@searxng-archlinux SearXNG]`` (GUEST)
|
||||
|
||||
.. code:: bash
|
||||
|
||||
$ sudo -H ./utils/lxc.sh cmd searxng-archlinux \
|
||||
make docs.html
|
||||
$ make docs.html
|
||||
|
||||
.. group-tab:: desktop (HOST)
|
||||
|
||||
.. code:: bash
|
||||
|
||||
$ sudo -H ./utils/lxc.sh cmd searxng-archlinux make docs.html
|
||||
|
||||
|
||||
.. _lxcdev summary:
|
||||
|
||||
@ -356,48 +388,51 @@ We build up a fully functional SearXNG suite in a archlinux container:
|
||||
|
||||
.. code:: bash
|
||||
|
||||
$ sudo -H ./utils/lxc.sh build searxng-archlinux
|
||||
$ sudo -H ./utils/lxc.sh install suite searxng-archlinux
|
||||
...
|
||||
Developer install? (wraps source from HOST into the running instance) [YES/no]
|
||||
|
||||
To wrap the suite into a developer one answer ``YES`` (or press Enter).
|
||||
|
||||
.. code:: text
|
||||
|
||||
link SearXNG's sources to: /share/SearXNG
|
||||
=========================================
|
||||
|
||||
mv -f "/usr/local/searxng/searxng-src" "/usr/local/searxng/searxng-src.backup"
|
||||
ln -s "/share/SearXNG" "/usr/local/searxng/searxng-src"
|
||||
ls -ld /usr/local/searxng/searxng-src
|
||||
|searxng| lrwxrwxrwx 1 searxng searxng ... /usr/local/searxng/searxng-src -> /share/SearXNG
|
||||
|
||||
On code modification the instance has to be restarted (see :ref:`uWSGI
|
||||
maintenance`):
|
||||
|
||||
.. code:: bash
|
||||
|
||||
$ sudo -H ./utils/lxc.sh cmd searxng-archlinux systemctl restart uwsgi@searxng
|
||||
|
||||
To access HTTP from the desktop we installed nginx for the services inside the
|
||||
container:
|
||||
|
||||
.. tabs::
|
||||
|
||||
.. group-tab:: [root@searxng-archlinux]
|
||||
|
||||
.. code:: bash
|
||||
|
||||
$ ./utils/filtron.sh nginx install
|
||||
$ ./utils/morty.sh nginx install
|
||||
$ sudo -H ./utils/lxc.sh cmd -- FORCE_TIMEOUT=0 ./utils/searxng.sh install nginx
|
||||
|
||||
To wrap the suite into a developer one, we created a symbolic link to the
|
||||
repository which is shared **transparent** from the desktop's file system into
|
||||
the container :
|
||||
|
||||
.. tabs::
|
||||
|
||||
.. group-tab:: [root@searxng-archlinux]
|
||||
|
||||
.. code:: bash
|
||||
|
||||
$ mv /usr/local/searxng/searxng-src /usr/local/searxng/searxng-src.old
|
||||
$ ln -s /share/searx/ /usr/local/searxng/searxng-src
|
||||
$ systemctl restart uwsgi@searx
|
||||
|
||||
To get information about the searxNG suite in the archlinux container we can
|
||||
To get information about the SearxNG suite in the archlinux container we can
|
||||
use:
|
||||
|
||||
.. tabs::
|
||||
|
||||
.. group-tab:: desktop
|
||||
|
||||
.. code:: bash
|
||||
.. code:: text
|
||||
|
||||
$ sudo -H ./utils/lxc.sh show suite searxng-archlinux
|
||||
...
|
||||
[searxng-archlinux] INFO: (eth0) filtron: http://10.174.184.156:4004/ http://10.174.184.156/searx
|
||||
[searxng-archlinux] INFO: (eth0) morty: http://10.174.184.156:3000/
|
||||
[searxng-archlinux] INFO: (eth0) docs.live: http://10.174.184.156:8080/
|
||||
[searxng-archlinux] INFO: (eth0) IPv6: http://[fd42:573b:e0b3:e97e:216:3eff:fea5:9b65]
|
||||
...
|
||||
[searxng-archlinux] INFO: (eth0) docs-live: http:///n.n.n.140:8080/
|
||||
[searxng-archlinux] INFO: (eth0) IPv6: http://[fd42:555b:2af9:e121:216:3eff:fe5b:1744]
|
||||
[searxng-archlinux] uWSGI:
|
||||
[searxng-archlinux] SEARXNG_UWSGI_SOCKET : /usr/local/searxng/run/socket
|
||||
[searxng-archlinux] environment /usr/local/searxng/searxng-src/utils/brand.env:
|
||||
[searxng-archlinux] GIT_URL : https://github.com/searxng/searxng
|
||||
[searxng-archlinux] GIT_BRANCH : master
|
||||
[searxng-archlinux] SEARXNG_URL : http:///n.n.n.140/searxng
|
||||
[searxng-archlinux] SEARXNG_PORT : 8888
|
||||
[searxng-archlinux] SEARXNG_BIND_ADDRESS : 127.0.0.1
|
||||
|
||||
|
||||
@ -20,6 +20,8 @@ Common command environments
|
||||
|
||||
The scripts in our tooling box often dispose of common environments:
|
||||
|
||||
.. _FORCE_TIMEOUT:
|
||||
|
||||
``FORCE_TIMEOUT`` : environment
|
||||
Sets timeout for interactive prompts. If you want to run a script in batch
|
||||
job, with defaults choices, set ``FORCE_TIMEOUT=0``. By example; to install a
|
||||
|
||||
@ -14,6 +14,11 @@
|
||||
``utils/lxc.sh``
|
||||
================
|
||||
|
||||
With the use of *Linux Containers* (LXC_) we can scale our tasks over a stack of
|
||||
containers, what we call the: *lxc suite*. The :ref:`lxc-searxng.env` is
|
||||
loaded by default, every time you start the ``lxc.sh`` script (*you do not need
|
||||
to care about*).
|
||||
|
||||
.. sidebar:: further reading
|
||||
|
||||
- snap_, `snapcraft LXD`_
|
||||
@ -21,10 +26,16 @@
|
||||
- `LXC/LXD Image Server`_
|
||||
- `LXD@github`_
|
||||
|
||||
With the use of *Linux Containers* (LXC_) we can scale our tasks over a stack of
|
||||
containers, what we call the: *lxc suite*. The *SearXNG suite*
|
||||
(:origin:`lxc-searxng.env <utils/lxc-searxng.env>`) is loaded by default, every time
|
||||
you start the ``lxc.sh`` script (*you do not need to care about*).
|
||||
.. contents:: Contents
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
|
||||
.. _lxd install:
|
||||
|
||||
Install LXD
|
||||
===========
|
||||
|
||||
Before you can start with containers, you need to install and initiate LXD_
|
||||
once::
|
||||
@ -38,44 +49,19 @@ take some time**::
|
||||
|
||||
$ sudo -H ./utils/lxc.sh build
|
||||
|
||||
.. sidebar:: hint
|
||||
|
||||
If you have issues with the internet connectivity of your containers read
|
||||
section :ref:`internet connectivity docker`.
|
||||
|
||||
A cup of coffee later, your LXC suite is build up and you can run whatever task
|
||||
you want / in a selected or even in all :ref:`LXC suite containers <lxc.sh
|
||||
help>`.
|
||||
|
||||
.. hint::
|
||||
|
||||
If you see any problems with the internet connectivity of your
|
||||
containers read section :ref:`internet connectivity docker`.
|
||||
|
||||
If you do not want to build all containers, **you can build just one**::
|
||||
|
||||
$ sudo -H ./utils/lxc.sh build searxng-archlinux
|
||||
|
||||
*Good to know ...*
|
||||
|
||||
Each container shares the root folder of the repository and the command
|
||||
``utils/lxc.sh cmd`` **handles relative path names transparent**, compare output
|
||||
of::
|
||||
|
||||
$ sudo -H ./utils/lxc.sh cmd -- ls -la Makefile
|
||||
...
|
||||
|
||||
In the containers, you can run what ever you want, e.g. to start a bash use::
|
||||
|
||||
$ sudo -H ./utils/lxc.sh cmd searxng-archlinux bash
|
||||
INFO: [searxng-archlinux] bash
|
||||
[root@searxng-archlinux SearXNG]#
|
||||
|
||||
If there comes the time you want to **get rid off all** the containers and
|
||||
**clean up local images** just type::
|
||||
|
||||
$ sudo -H ./utils/lxc.sh remove
|
||||
$ sudo -H ./utils/lxc.sh remove images
|
||||
|
||||
.. _internet connectivity docker:
|
||||
|
||||
Internet Connectivity & Docker
|
||||
==============================
|
||||
------------------------------
|
||||
|
||||
.. sidebar:: further read
|
||||
|
||||
@ -115,21 +101,132 @@ Reboot your system and check the iptables rules::
|
||||
:FORWARD ACCEPT [7048:7851230]
|
||||
|
||||
|
||||
.. _searxng lxc suite:
|
||||
|
||||
SearXNG LXC suite
|
||||
=================
|
||||
|
||||
The intention of the *SearXNG LXC suite* is to build up a suite of containers
|
||||
for development tasks or :ref:`buildhosts <Setup SearXNG buildhost>` with a very
|
||||
small set of simple commands. At the end of the ``--help`` output the SearXNG
|
||||
suite from the :ref:`lxc-searxng.env` is introduced::
|
||||
|
||||
$ sudo -H ./utils/lxc.sh --help
|
||||
...
|
||||
LXC suite: searxng
|
||||
Suite includes installation of SearXNG
|
||||
images: ubu2004 ubu2204 fedora35 archlinux
|
||||
containers: searxng-ubu2004 searxng-ubu2204 searxng-fedora35 searxng-archlinux
|
||||
|
||||
As shown above there are images and containers build up on this images. To show
|
||||
more info about the containers in the *SearXNG LXC suite* call ``show suite``.
|
||||
If this is the first time you make use of the SearXNG LXC suite, no containers
|
||||
are installed and the output is::
|
||||
|
||||
$ sudo -H ./utils/lxc.sh show suite
|
||||
|
||||
LXC suite (searxng-*)
|
||||
=====================
|
||||
|
||||
+------+-------+------+------+------+-----------+
|
||||
| NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS |
|
||||
+------+-------+------+------+------+-----------+
|
||||
|
||||
WARN: container searxng-ubu2004 does not yet exists
|
||||
WARN: container searxng-ubu2204 does not yet exists
|
||||
WARN: container searxng-fedora35 does not yet exists
|
||||
WARN: container searxng-archlinux does not yet exists
|
||||
|
||||
If you do not want to run a command or a build in all containers, **you can
|
||||
build just one**. Here by example in the container that is build upon the
|
||||
*archlinux* image::
|
||||
|
||||
$ sudo -H ./utils/lxc.sh build searxng-archlinux
|
||||
$ sudo -H ./utils/lxc.sh cmd searxng-archlinux pwd
|
||||
|
||||
Otherwise, to apply a command to all containers you can use::
|
||||
|
||||
$ sudo -H ./utils/lxc.sh build
|
||||
$ sudo -H ./utils/lxc.sh cmd -- ls -la .
|
||||
|
||||
Running commands
|
||||
----------------
|
||||
|
||||
**Inside containers, you can run scripts** from the :ref:`toolboxing` or run
|
||||
what ever command you need. By example, to start a bash use::
|
||||
|
||||
$ sudo -H ./utils/lxc.sh cmd searxng-archlinux bash
|
||||
INFO: [searxng-archlinux] bash
|
||||
[root@searxng-archlinux SearXNG]#
|
||||
|
||||
.. _Good to know:
|
||||
|
||||
Good to know
|
||||
------------
|
||||
|
||||
Each container shares the root folder of the repository and the command
|
||||
``utils/lxc.sh cmd`` **handle relative path names transparent**::
|
||||
|
||||
$ pwd
|
||||
/share/SearXNG
|
||||
|
||||
$ sudo -H ./utils/lxc.sh cmd searxng-archlinux pwd
|
||||
INFO: [searxng-archlinux] pwd
|
||||
/share/SearXNG
|
||||
|
||||
The path ``/share/SearXNG`` will be different on your HOST system. The commands
|
||||
in the conatiner are executed by the ``root`` inside of the container. Compare
|
||||
output of::
|
||||
|
||||
$ ls -li Makefile
|
||||
47712402 -rw-rw-r-- 1 markus markus 2923 Apr 19 13:52 Makefile
|
||||
|
||||
$ sudo -H ./utils/lxc.sh cmd searxng-archlinux ls -li Makefile
|
||||
INFO: [searxng-archlinux] ls -li Makefile
|
||||
47712402 -rw-rw-r-- 1 root root 2923 Apr 19 11:52 Makefile
|
||||
...
|
||||
|
||||
Since the path ``/share/SearXNG`` of the HOST system is wrapped into the
|
||||
container under the same name, the shown ``Makefile`` (inode ``47712402``) in
|
||||
the ouput is always the identical ``/share/SearXNG/Makefile`` from the HOST
|
||||
system. In the example shown above the owner of the path in the container is
|
||||
the ``root`` user of the conatiner (and the timezone in the container is
|
||||
different to HOST system).
|
||||
|
||||
|
||||
.. _lxc.sh install suite:
|
||||
|
||||
Install suite
|
||||
=============
|
||||
-------------
|
||||
|
||||
To install the complete :ref:`SearXNG suite (includes searx, morty & filtron)
|
||||
<lxc-searxng.env>` into all LXC_ use::
|
||||
.. sidebar:: further read
|
||||
|
||||
- :ref:`working in containers`
|
||||
- :ref:`FORCE_TIMEOUT <FORCE_TIMEOUT>`
|
||||
|
||||
To install the complete :ref:`SearXNG suite <lxc-searxng.env>` into **all** LXC_
|
||||
containers leave the container argument empty and run::
|
||||
|
||||
$ sudo -H ./utils/lxc.sh build
|
||||
$ sudo -H ./utils/lxc.sh install suite
|
||||
|
||||
To *build & install* suite only in one container you can use by example::
|
||||
|
||||
$ sudo -H ./utils/lxc.sh build searxng-archlinux
|
||||
$ sudo -H ./utils/lxc.sh install suite searxng-archlinux
|
||||
|
||||
The command above installs a SearXNG suite (see :ref:`installation scripts`).
|
||||
To :ref:`install a nginx <installation nginx>` reverse proxy (or alternatively
|
||||
use :ref:`apache <installation apache>`)::
|
||||
|
||||
sudo -H ./utils/lxc.sh cmd -- FORCE_TIMEOUT=0 ./utils/searxng.sh install nginx
|
||||
$ sudo -H ./utils/lxc.sh cmd -- FORCE_TIMEOUT=0 ./utils/searxng.sh install nginx
|
||||
|
||||
Same operation just in one container of the suite::
|
||||
|
||||
$ sudo -H ./utils/lxc.sh cmd searxng-archlinux FORCE_TIMEOUT=0 ./utils/searxng.sh install nginx
|
||||
|
||||
The :ref:`FORCE_TIMEOUT <FORCE_TIMEOUT>` environment is set to zero to run the
|
||||
script without user interaction.
|
||||
|
||||
To get the IP (URL) of the SearXNG service in the containers use ``show suite``
|
||||
command. To test instances from containers just open the URLs in your
|
||||
@ -137,22 +234,22 @@ WEB-Browser::
|
||||
|
||||
$ sudo ./utils/lxc.sh show suite | grep SEARXNG_URL
|
||||
|
||||
[searxng-ubu2110] SEARXNG_URL : http://n.n.n.147/searxng
|
||||
[searxng-ubu2004] SEARXNG_URL : http://n.n.n.246/searxng
|
||||
[searxnggfedora35] SEARXNG_URL : http://n.n.n.140/searxng
|
||||
[searxng-archlinux] SEARXNG_URL : http://n.n.n.165/searxng
|
||||
[searxng-ubu2110] SEARXNG_URL : http://n.n.n.170/searxng
|
||||
[searxng-ubu2004] SEARXNG_URL : http://n.n.n.160/searxng
|
||||
[searxnggfedora35] SEARXNG_URL : http://n.n.n.150/searxng
|
||||
[searxng-archlinux] SEARXNG_URL : http://n.n.n.140/searxng
|
||||
|
||||
Clean up
|
||||
--------
|
||||
|
||||
If there comes the time you want to **get rid off all** the containers and
|
||||
**clean up local images** just type::
|
||||
|
||||
$ sudo -H ./utils/lxc.sh remove
|
||||
$ sudo -H ./utils/lxc.sh remove images
|
||||
|
||||
|
||||
Running commands
|
||||
================
|
||||
|
||||
**Inside containers, you can use make or run scripts** from the
|
||||
:ref:`toolboxing`. By example: to setup a :ref:`buildhosts` and run the
|
||||
Makefile target ``test`` in the archlinux_ container::
|
||||
|
||||
sudo -H ./utils/lxc.sh cmd searxng-archlinux ./utils/searxng.sh install buildhost
|
||||
sudo -H ./utils/lxc.sh cmd searxng-archlinux make test
|
||||
|
||||
.. _Setup SearXNG buildhost:
|
||||
|
||||
Setup SearXNG buildhost
|
||||
=======================
|
||||
@ -174,13 +271,13 @@ To get IP of the container and the port number *live docs* is listening::
|
||||
|
||||
$ sudo ./utils/lxc.sh show suite | grep docs.live
|
||||
...
|
||||
[searxng-archlinux] INFO: (eth0) docs.live: http://n.n.n.12:8080/
|
||||
[searxng-archlinux] INFO: (eth0) docs.live: http://n.n.n.140:8080/
|
||||
|
||||
|
||||
.. _lxc.sh help:
|
||||
|
||||
Overview
|
||||
========
|
||||
Command Help
|
||||
============
|
||||
|
||||
The ``--help`` output of the script is largely self-explanatory:
|
||||
|
||||
@ -189,8 +286,10 @@ The ``--help`` output of the script is largely self-explanatory:
|
||||
|
||||
.. _lxc-searxng.env:
|
||||
|
||||
SearXNG suite
|
||||
=============
|
||||
SearXNG suite config
|
||||
====================
|
||||
|
||||
The SearXNG suite is defined in the file :origin:`utils/lxc-searxng.env`:
|
||||
|
||||
.. literalinclude:: ../../utils/lxc-searxng.env
|
||||
:language: bash
|
||||
|
||||
@ -5,6 +5,9 @@
|
||||
``utils/searxng.sh``
|
||||
====================
|
||||
|
||||
To simplify the installation and maintenance of a SearXNG instance you can use the
|
||||
script :origin:`utils/searxng.sh`.
|
||||
|
||||
.. sidebar:: further reading
|
||||
|
||||
- :ref:`architecture`
|
||||
@ -12,8 +15,11 @@
|
||||
- :ref:`installation nginx`
|
||||
- :ref:`installation apache`
|
||||
|
||||
To simplify the installation and maintenance of a SearXNG instance you can use the
|
||||
script :origin:`utils/searxng.sh`.
|
||||
.. contents:: Contents
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
|
||||
Install
|
||||
=======
|
||||
@ -28,8 +34,8 @@ The installation is described in chapter :ref:`installation basic`.
|
||||
|
||||
.. _searxng.sh overview:
|
||||
|
||||
Overview
|
||||
========
|
||||
Command Help
|
||||
============
|
||||
|
||||
The ``--help`` output of the script is largely self-explanatory:
|
||||
|
||||
|
||||
@ -42,10 +42,11 @@ EOF
|
||||
lxc_suite_install() {
|
||||
(
|
||||
lxc_set_suite_env
|
||||
FORCE_TIMEOUT=0
|
||||
export FORCE_TIMEOUT
|
||||
"${LXC_REPO_ROOT}/utils/searxng.sh" install all
|
||||
rst_title "suite installation finished ($(hostname))" part
|
||||
FORCE_TIMEOUT=0 "${LXC_REPO_ROOT}/utils/searxng.sh" install all
|
||||
rst_title "Suite installation finished ($(hostname))" part
|
||||
if ask_yn "Developer install? (wraps source from HOST into the running instance)" Yn; then
|
||||
"${LXC_REPO_ROOT}/utils/searxng.sh" searxng.install.link_src "$(pwd)"
|
||||
fi
|
||||
lxc_suite_info
|
||||
echo
|
||||
)
|
||||
|
||||
@ -467,6 +467,18 @@ EOF
|
||||
popd > /dev/null
|
||||
}
|
||||
|
||||
searxng.install.link_src() {
|
||||
rst_title "link SearXNG's sources to: $2" chapter
|
||||
echo
|
||||
tee_stderr 0.1 <<EOF | sudo -H -u "${SERVICE_USER}" -i 2>&1 | prefix_stdout "$_service_prefix"
|
||||
mv -f "${SEARXNG_SRC}" "${SEARXNG_SRC}.backup"
|
||||
ln -s "${2}" "${SEARXNG_SRC}"
|
||||
ls -ld /usr/local/searxng/searxng-src
|
||||
EOF
|
||||
echo
|
||||
uWSGI_restart "$SEARXNG_UWSGI_APP"
|
||||
}
|
||||
|
||||
searxng.install.pyenv() {
|
||||
rst_title "Create virtualenv (python)" section
|
||||
echo
|
||||
|
||||
Loading…
Reference in New Issue
Block a user