Markus Heiser
8cd544b2a6
[doc] add documentation about the XPath engine
...
- pylint searx/engines/xpath.py
- fix indentation of some long lines
- add logging
- add doc-strings
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-05-23 11:48:21 +02:00
Markus Heiser
ffcebf5e12
[enh] xpath engine - add request parameter 'soft_max_redirects'
...
Make 'soft_max_redirects' configurable per Xpath engine::
- name : <engine-name>
engine : xpath
soft_max_redirects: 1
...
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-05-17 15:04:55 +02:00
Alexandre Flament
a4dcfa025c
[enh] engines: add about variable
...
move meta information from comment to the about variable
so the preferences, the documentation can show these information
2021-01-14 20:57:17 +01:00
Alexandre Flament
d41cafd5f3
[fix] xpath, mojeek: fix commit 58d72f2692
...
before commit 58d72f258d72f2close #2368
2020-12-10 10:52:06 +01:00
Alexandre Flament
ad72803ed9
[mod] xpath, 1337x, acgsou, apkmirror, archlinux, arxiv: use eval_xpath_* functions
2020-12-03 10:22:48 +01:00
Alexandre Flament
58d72f2692
[mod] pylint: minor code change to allow pylint globally
...
This commit is only a step, it doesn't fix all the issues reported by pylint
2020-11-03 11:35:53 +01:00
a01200356
c3daa08537
[enh] Add onions category with Ahmia, Not Evil and Torch
...
Xpath engine and results template changed to account for the fact that
archive.org doesn't cache .onions, though some onion engines migth have
their own cache.
Disabled by default. Can be enabled by setting the SOCKS proxies to
wherever Tor is listening and setting using_tor_proxy as True.
Requires Tor and updating packages.
To avoid manually adding the timeout on each engine, you can set
extra_proxy_timeout to account for Tor's (or whatever proxy used) extra
time.
2020-10-25 17:59:05 -07:00
Alexandre Flament
2006eb4680
[mod] move extract_text, extract_url to searx.utils
2020-10-02 18:13:56 +02:00
Dalf
1022228d95
Drop Python 2 (1/n): remove unicode string and url_utils
2020-09-10 10:39:04 +02:00
xywei
1d4657b714
Fix relative urls that do not start with '/'
2020-07-23 11:12:19 -05:00
Dalf
85b3723345
[mod] speed optimization
...
compile XPath only once
avoid redundant call to urlparse
get_locale(webapp.py): avoid useless call to request.accept_languages.best_match
2019-11-15 09:33:15 +01:00
Alexandre Flament
f34b5cedb1
[fix] fixes google play engines ( #1651 )
...
update commit 87baa74a86
2019-07-25 09:31:47 +02:00
Venca24
87baa74a86
[fix] fixes google play engines and adds thumbnails to their results ( #1612 )
...
fix google play apps, google play apps, google play music engines
xpath engine: thumbnail_xpath can define an optional thumbnail
2019-07-25 07:46:41 +02:00
Marc Abonce Seguin
343e555ee9
[fix] append http if no scheme is provided in xpath's extact_url
...
This solves a bug with Yahoo where some results don't specify
a protocol.
2018-04-08 20:35:34 -05:00
Adam Tauber
1972a044a3
[fix] produce valid urls if scheme is missing
2017-05-22 15:48:37 +02:00
Adam Tauber
52e615dede
[enh] py3 compatibility
2017-05-15 12:02:30 +02:00
David A Roberts
7492997c51
[fix] allow empty content
2017-01-17 21:14:33 +10:00
Alexandre Flament
90e1db3e5c
[fix] extract_text: use html.tostring instead html_to_text. Fix #711
2016-12-31 13:56:09 +01:00
David A Roberts
1e9dab08e6
[fix] behaviour for page_size>1 and first_page_num>0
...
eg. pageno=1,21,41,... instead of 20,40,60,...
2016-08-14 22:10:25 +10:00
Kirill Isakov
bacc9a3df1
Add paging support to XPath & Erowid engines
2016-03-28 19:15:03 +06:00
Adam Tauber
bd22e9a336
[fix] pep8 compatibilty
2016-01-18 12:47:31 +01:00
Cqoicebordel
44c9216c49
Sanitize extract_text
2015-01-25 20:04:44 +01:00
potato
6f535b6fae
[fix] error when xpath_results in extraxt_text is _ElementUnicodeResult instead of _ElementStringResult
2014-03-04 19:43:41 +01:00
asciimoo
c1d7d30b8e
[mod] len() removed from conditions
2014-02-11 13:13:51 +01:00
asciimoo
b647244abf
[fix] function parameters
2014-01-30 03:10:20 +01:00
asciimoo
3dcb835910
[fix] function parameters
2014-01-30 02:36:05 +01:00
asciimoo
fe82637eac
[enh] importable url extractor
2014-01-30 02:32:58 +01:00
asciimoo
59eeeaab87
[fix] html tag removal
2014-01-23 11:08:08 +01:00
asciimoo
b2492c94f4
[fix] pep/flake8 compatibility
2014-01-20 02:31:20 +01:00
asciimoo
060ea4d2f5
[fix] whitespaces removed
2014-01-12 18:48:38 +01:00
Dalf
3dc3fc7770
[mod][fix] xpath engine simplified, yahoo engine never returns truncated urls
2014-01-05 14:06:52 +01:00
dalf
664c039b38
xpath engine: bug fix
2013-12-30 22:34:35 +01:00
asciimoo
e50a72b0e3
[enh] suggestion support for xpath engine
2013-11-13 19:33:09 +01:00
asciimoo
17bf00ee42
[enh] removing result html tags
2013-11-09 18:39:20 +01:00
asciimoo
7965da55a7
[fix] urlparsing fix
2013-10-27 12:01:03 +01:00
asciimoo
5d764f95cf
[enh] xpath engine absolute xpath support
2013-10-26 13:45:43 +02:00
asciimoo
badd988545
[enh] xpath engine added
2013-10-26 02:22:20 +02:00
