make test.shell reports an issue that has been added in [PR-500]::
In ./manage line 80:
yamllint : lint YAML files: $YAMLLINT_FILES
^-------------^
SC2128: Expanding an array without an
index only gives the first element.
[PR-500] https://github.com/searxng/searxng/pull/500
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
new bash function convert_if_newer() / usage::
convert_if_newer <origfile> <outfile> [<options>, ...]
convert_if_newer "path/to/origin.svg" "path/to/converted.png" -transparent white -resize 64x64
Run's ImageMagik' convert comand to generate <outfile> from <origfile>, if
<origfile> is newer than <outfile>. The command line is to convert is::
convert <origfile> [<options>, ...] <outfile>
PNG 'searx/static/themes/simple/img/favicon.png' has been created by::
$ make themes.simple
CONVERT searx/static/themes/simple/src/svg/searxng-wordmark.svg -transparent white -resize 64x64 searx/static/themes/simple/img/favicon.png
...
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
In commit 97355672c the functions named babel.* has been renamed to weblate.*
but it was forgotten to change it also in the help().
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Functions implemented to run *Weblate* workflows should use prefix 'weblate.':
- babel.setup.translations.worktree --> weblate.translations.worktree
Create git worktree ${TRANSLATIONS_WORKTREE} and checkout branch
'translations' from Weblate's counterpart (weblate) of the SearXNG
(origin)::
remote weblate https://weblate.bubu1.eu/git/searxng/searxng/
- babel.weblate.to.translations --> weblate.to.translations
Update 'translations' branch of SearXNG (origin) with last additions from
Weblate.
- babel.translations.to.master --> weblate.translations.commit
Update 'translations' branch of SearXNG (origin) with last additions from
Weblate. Copy the changes to the master branch, compile translations and
create a commit in the local branch (master)
- babel.master.to.translations --> weblate.push.translations
Push *translation changes* from SearXNG (origin) to Weblate's
counterpart (weblate).
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
In ./manage implement babel.*:
- extract : extract messages from source files and generate POT file
- update : update existing message catalogs from POT file
- compile : compile translation catalogs into binary MO files
Replace searx_extra/update/update_translations.sh by command:
- ci.babel.update
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
This commit remove the need to update the brand for GIT_URL and GIT_BRANCH:
there are read from the git repository.
It is possible to call python -m searx.version freeze to freeze the current version.
Useful when the code is installed outside git (distro package, docker, etc...)
Some defaults in the settings.yml are taken from the environment.
By example;
The manage scripts sources the ./utils/brand.env and sets SEARX_PORT
environment. This enviroment *wins over* any settings in a YAML file.
Whe we run a::
make test.robot
The searx/settings_robot.yml is used, in this file the server settings are::
server:
port: 11111
bind_address: 127.0.0.1
To get in use of the 'port: 11111' we have to unset the SEARX_PORT environment
which was sourced before.
The function buildenv.unset_env() can be called in all use cases where the
enviroment from ./utils/brand.env is not wanted. ATM it unset the enviroment
variables::
unset GIT_URL
unset GIT_BRANCH
unset SEARX_URL
unset SEARX_PORT
unset SEARX_BIND_ADDRESS
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
modified docs/admin/engines/settings.rst
- Fix documentation and add section 'brand'.
- Add remarks about **buildenv** variables.
- Add remarks about settings from environment variables $SEARX_DEBUG,
$SEARX_PORT, $SEARX_BIND_ADDRESS and $SEARX_SECRET
modified docs/admin/installation-searx.rst & docs/build-templates/searx.rst
Fix template location /templates/etc/searx/settings.yml
modified docs/dev/makefile.rst
Add description of the 'make buildenv' target and describe
- we have all SearXNG setups are centralized in the settings.yml file
- why some tasks need a utils/brand.env (aka instance's buildenv)
modified manage
Settings file from repository's working tree are used by default and
ask user if a /etc/searx/settings.yml file exists.
modified searx/settings.yml
Add comments about when it is needed to run 'make buildenv'
modified searx/settings_defaults.py
Default for server:port is taken from enviroment variable SEARX_PORT.
modified utils/build_env.py
- Some defaults in the settings.yml are taken from the environment,
e.g. SEARX_BIND_ADDRESS (searx.settings_defaults.SHEMA). When the
'brand.env' file is created these enviroment variables should be
unset first.
- The CONTACT_URL enviroment is not needed in the utils/brand.env
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
The environment variables ..
- SEARX_GIT_VERSION
- VERSION_GITCOMMIT
- SEARX_PYTHON_VERSION
- GITHUB_USER
- SEARX_IMAGE_NAME
- BUILD
are local to the docker.buildx() function, other variables like GIT_URL are
comming from the utils/brand.env file (rebuild by: make buildenv).
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
The functions:
- static.build.commit
- static.build.commit.drop
- static.build.restore
are imported into the ./manage script. To avoid name collisions some variables
and fucntions has been renamed by adding the prefix *static_*.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
The node.env build environment require npm, ttfautohint and fontforge installed
in the OS. These tools can be installed by::
sudo -H ./utils/searx.sh install buildhost
If one of the tools is not installed, the script node.env stops with a
appropriate message.
BTW: We ignore CentOS-7 as developer & build platform
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Before this commit, there are 3 node_modules directory:
* one in .
* two others in ./searx/statics/themes/*
This is no desirable:
* it declares the npm depdenencies in the shell script.
* dependabot can't updates theses dependencies.
* this is a not standard way to build a package (two different locations for the dependencies).
With this commit and the PR #150 there is one unique node_modules directory per theme.
This file is generated by webfont.
* It is now generated as searx/static/themes/simple/ion.less
* It is generated before the .less compilation.
* .gitignore includes this file
Add two new package depedencies: fontforge ttfautohint
See utils/searx.sh
Usage::
make test.yamllint
./manage test.yamllint
test.yamllint is also added to the `test` makefile target.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
make docker.buildx : build and push multiarch build.
(it can't be only build)
use buildx with the --cache-from and --cache-to options to cache the layers
(only the last built is cached)
The philosophy of set -e is typically that it only exits upon uncaught
errors. Here, the presence of || outside the subshell seems to tell the shell
that the error inside the subshell is 'caught' and therefore set -e does not
cause an exit after false [1].
The shell does not exit if the command that fails is ... part of any command
executed in a && or || list except the command following the final && or ||, any
command in a pipeline but the last, or if the command’s return status is being
inverted with ! [2]
[1] https://unix.stackexchange.com/questions/296526/set-e-in-a-subshell
[2] https://www.gnu.org/software/bash/manual/html_node/The-Set-Builtin.html#The-Set-Builtin
BTW: fix error reported by 'make test.shell'
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
These py files are linted by test.pylint(), all other files are linted by
test.pep8()
close: https://github.com/searxng/searxng/issues/21
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
- add to list of pylint scripts
- add debug log messages
- move API key int `settings.yml`
- improved readability
- add some metadata to results
Signed-off-by: Markus Heiser <markus@darmarit.de>
note: in further patches script 'manage' will replace 'manage.sh'
pyenv.* :
assert : build virtualenv if not exists
intsall : developer install of searx into virtualenv
unintsall : uninstall developer installation
cmd ... : run command ... in virtualenv
OK : test if virtualenv is OK
pypi.upload:
Upload python packages to PyPi (to test use pypi.upload.test)
pybuild :
Build python packages at ./${PYDIST}
pyclean :
delete virtualenv and intermediate py files
test.* :
pylint : lint PYLINT_FILES, searx/engines, searx & tests
pep8 : pycodestyle (pep8) for all files except PYLINT_FILES
unit : run unit tests
coverage : run unit tests with coverage
robot : run robot test
clean : clean intermediate test stuff
node.* :
env : download & install npm dependencies locally
clean : drop npm installations
buildenv :
rebuild ./utils/brand.env
data.* :
all : update searx/languages.py and ./data/*
languages : update searx/data/engines_languages.json & searx/languages.py
useragents: update searx/data/useragents.json with the most recent versions of Firefox.
themes.* :
all : build all themes
oscar : build oscar theme
simple : build simple theme
bootstrap : less compile bootstrap.min.css CSS
babel.compile :
pybabel compile ./searx/translations
docs.* :
html : build HTML documentation
gh-pages : deploy on gh-pages branch
autobuild : autobuild HTML documentation while editing
prebuild : build reST include files (./${DOCS_BUILD}/includes)
clean : clean documentation build
docker.build [push] :
build (and push) docker image
gecko.driver :
download & install geckodriver if not already installed (required for
robot_tests)
Signed-off-by: Markus Heiser <markus@darmarit.de>
