Commit Graph

848 Commits

Author SHA1 Message Date
Sean Hatfield
79656718b2
[FEAT] Create custom pdfloader (#1852)
* implement custom PDFLoader to remove LC dep

* remove unneeded comment

* remove pdfjs as dep and fix page splitting using pdf-parse

* linting + export rename for desktop compat

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-07-11 12:26:11 -07:00
Sean Hatfield
881c4bc4de
[FIX] Hide TTS and more actions on chat reset message (#1850)
hide TTS and more actions on chat reset message
2024-07-11 11:05:25 -07:00
Sean Hatfield
85e5866be6
[STYLE] Fix styles of general settings page (#1847)
fix styles of limit messages per day
2024-07-10 15:42:32 -07:00
Sean Hatfield
554f414cb8
[FIX] Embed chat widget message color bug fix (#1843)
* fix assistant bg color data prop

* publish modified embed changes
2024-07-10 14:02:43 -07:00
Sean Hatfield
69d67401ff
[STYLE] Implement new settings sidebar UI (#1829)
* implement new settings sidebar v2

* store state of settings menu in localstorage to improve ux

* add tailwind color

* add missed admin translation

* fix admin pages showing on single user

* perms fix for manager role

* refactor permissions for options on sidebar

* minor refactor of menuoption

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-07-10 12:20:06 -07:00
Sean Hatfield
e7fe35bda9
[STYLE] Implement new chat tools UI (#1835)
* implement new chat tools ui + bump phosphor icons package for new icons

* move TTS button below user image/fix styling

* Show tools on hover
update package deps

* patch styles for desktop

* fix more actions tooltip and disable hide/show on hover for mobile

* z-index on mobile patch

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-07-10 12:16:58 -07:00
植心
f6c61d0dd1
Update README.zh-CN.md (#1827) 2024-07-08 09:36:28 -07:00
bchoor
baa5f35264
Fixed typo "AAnything" (#1822) 2024-07-07 15:55:33 -07:00
timothycarambat
e952182de0 update embed openOnLoad attrib 2024-07-04 03:44:32 -07:00
timothycarambat
ddf268771d Merge branch 'master' of github.com:Mintplex-Labs/anything-llm 2024-07-03 18:29:29 -07:00
timothycarambat
d94720f31a remove mark as bad feedback 2024-07-03 18:29:20 -07:00
Sean Hatfield
ab56bae1bb
[FEAT] Ability to delete messages from conversation history (#1812)
* implement delete message feature

* add new delete chat endpoint that just hides messages from ui and fix perms

* refactor: add delete-event hooks for delete so we dont need to pass so much
refactor: implmentation of delete endpoint PUT

* update animation
linting
remove element on animiation to prevent popin

* linting and cleanup imports

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-07-03 18:28:08 -07:00
timothycarambat
8658b1e7c7 linting 2024-07-03 18:25:44 -07:00
Timothy Carambat
29c9eeaa5c
Add winston logging for production (#1811) 2024-07-03 16:39:33 -07:00
Timothy Carambat
06fb9bdd99
1738 generic open ai agent (#1809)
* enable support for generic openAI as Agent provider
2024-07-03 15:17:53 -07:00
Sean Hatfield
8b5d9ccdb3
[FEAT] Fork chat to new thread (#1788)
* implement thread forking feature

* rename thread based on forked message

* refactor bulk message create for thread fork + bump prisma version

* revert prisma version bump

* add todo to bulkCreate function in workspace chats

* cast user input to expected type to prevent prisma injection

* refactor: update order of ops for thread fork

---------

Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
2024-07-03 14:44:35 -07:00
Sean Hatfield
a87014822a
[REFACTOR] Improve asPDF collector processor with pdfjs (#1791)
* WIP replace langchain pdfloader with pdfjs and add more context to each page

* remove extras from pdfjs and just replace langchain library

* remove unneeded dep

* fix console log in docs

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-07-03 14:26:48 -07:00
timothycarambat
3fa00ad7b6 Merge branch 'master' of github.com:Mintplex-Labs/anything-llm 2024-07-03 14:02:42 -07:00
timothycarambat
89538c3614 fix typo in agent debug logs 2024-07-03 14:02:33 -07:00
Sean Hatfield
f205d51fe9
[FIX] Confluence code snippet blocks not being extracted (#1804)
implement custom confluence loader to extract code blocks properly from documents

Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
2024-07-03 14:00:44 -07:00
timothycarambat
fbc9c868c4 chore: silence getGitVersion for docker env runtime 2024-07-03 11:50:31 -07:00
Timothy Carambat
cb6b3340f0
Update bgworker to use fork instead of worker_thread (#1808) 2024-07-03 11:44:34 -07:00
Sean Hatfield
fc375f4036
[FIX] Bulk link scraper bug fix (#1800)
patch website depth data connector to work for other links that are not root url
2024-07-01 16:59:28 -07:00
apstrom
b105c6c969
Update index.jsx (#1794)
Max. number of context snippets increased from 12 to 200. 
TODO: May remove restraint entirely
2024-07-01 16:39:59 -07:00
Timothy Carambat
245d1b878e
Patch Embed styles with prefixing (#1799)
* Patch Embed styles with prefixing

* forgot files
2024-07-01 16:32:23 -07:00
Timothy Carambat
0b845fbb1c
Deprecate .isSafe moderation (#1790)
Add type defs to helpers
2024-06-28 15:32:30 -07:00
Sean Hatfield
910eb36cfe
[FIX] OpenAI compatible endpoints query mode developer API bug fix (#1789)
fix query mode always responding with refusal message on develop api openai compatible endpoints
2024-06-28 14:23:16 -07:00
Timothy Carambat
7a78ad3960
Apply workspace topN to agent call (#1779)
resolves #1777
2024-06-27 17:58:28 -07:00
timothycarambat
4afdf846e0 patch reveal hook 2024-06-27 12:51:28 -07:00
timothycarambat
834fb5f900 Merge branch 'master' of github.com:Mintplex-Labs/anything-llm 2024-06-26 23:48:46 -07:00
timothycarambat
d9cfcba9bb prevent holdToReveal from spamming toasts 2024-06-26 23:48:38 -07:00
Timothy Carambat
9bd65f1567
[CHORE] Migration from vectordb to @lancedb/lancedb NodeJS SDK (#1766)
WIP on migration to @lancedb/lancedb NodeJS SDK
2024-06-26 21:57:16 -07:00
timothycarambat
69cb71b1fc patch cached tag overflow of filename text
patch parseChunkHeader bug
2024-06-26 21:45:52 -07:00
timothycarambat
c341f7a298 Merge branch 'master' of github.com:Mintplex-Labs/anything-llm 2024-06-26 21:02:14 -07:00
timothycarambat
0c32b31f3a fix filerow overflow 2024-06-26 21:02:04 -07:00
Sean Hatfield
77916d920b
[FEAT] Remove users_can_delete_workspaces feature (#1742)
remove all references to users_can_delete_workspaces option
2024-06-25 16:46:06 -07:00
Sean Hatfield
b5ac62ff9f
[FEAT] Auto detection for Ollama and LMStudio (#1756)
* add ollama automatic url detection in llm and embedder prefrences

* implement auto detection for lmstudio llm and embedder/improve performance of checking common urls

* fix modal not clearing

* fix lmstudio url check

* improve ux for ollama llm provider option

* improve ux for lm studio llm provider option

* improve ux for ollama embedder option

* improve ux for lmstudio embedder option

* ux improvement lmstudio embedder options

* refactor implementation to hook and use native timeout
Swap to promise.any for resolving of available endpoints

* implement useProviderEndpointAutoDiscovery hook for lmstudio and ollama provider options

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-06-25 16:30:05 -07:00
Jason Zhang
fa4ab0f65f
fix: sanitize filename before writing (#1743)
* fix: sanitize filename before writing

Fixes: https://github.com/Mintplex-Labs/anything-llm/issues/1737

* fixup

* fixup
2024-06-25 15:45:09 -07:00
Sean Hatfield
dde8bc238b
[FIX] Remove Azure URL validation (#1758)
remove azure url validation
2024-06-25 12:10:51 -07:00
Timothy Carambat
88a0335fd2
Add OpenAI compatible API support (#1757) 2024-06-24 18:28:21 -07:00
timothycarambat
158ba83cab Merge branch 'master' of github.com:Mintplex-Labs/anything-llm 2024-06-24 09:32:53 -07:00
timothycarambat
dfff9070db fix typo 2024-06-24 09:32:42 -07:00
Sean Hatfield
e72fa8b370
[FEAT] Generic OpenAI embedding provider (#1664)
* implement generic openai embedding provider

* linting

* comment & description update for generic openai embedding provider

* fix privacy for generic

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-06-21 16:27:02 -07:00
Timothy Carambat
dc4ad6b5a9
[BETA] Live document sync (#1719)
* wip bg workers for live document sync

* Add ability to re-embed specific documents across many workspaces via background queue
bgworkser is gated behind expieremental system setting flag that needs to be explictly enabled
UI for watching/unwatching docments that are embedded.
TODO: UI to easily manage all bg tasks and see run results
TODO: UI to enable this feature and background endpoints to manage it

* create frontend views and paths
Move elements to correct experimental scope

* update migration to delete runs on removal of watched document

* Add watch support to YouTube transcripts (#1716)

* Add watch support to YouTube transcripts
refactor how sync is done for supported types

* Watch specific files in Confluence space (#1718)

Add failure-prune check for runs

* create tmp workflow modifications for beta image

* create tmp workflow modifications for beta image

* create tmp workflow modifications for beta image

* dual build
update copy of alert modals

* update job interval

* Add support for live-sync of Github files

* update copy for document sync feature

* hide Experimental features from UI

* update docs links

* [FEAT] Implement new settings menu for experimental features (#1735)

* implement new settings menu for experimental features

* remove unused context save bar

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>

* dont run job on boot

* unset workflow changes

* Add persistent encryption service
Relay key to collector so persistent encryption can be used
Encrypt any private data in chunkSources used for replay during resync jobs

* update jsDOC

* Linting and organization

* update modal copy for feature

---------

Co-authored-by: Sean Hatfield <seanhatfield5@gmail.com>
2024-06-21 13:38:50 -07:00
timothycarambat
c176fe38d2 remove async on dumpEnv 2024-06-20 20:50:14 -07:00
timothycarambat
ed53bbcf15 Merge branch 'master' of github.com:Mintplex-Labs/anything-llm 2024-06-20 18:03:33 -07:00
timothycarambat
9411791fe5 Simplify ENV management
auto write ENV on changes
Dont double-track envs that are already tracked
2024-06-20 18:03:21 -07:00
Sean Hatfield
12bf576e16
[FEAT] Workspace threads developer API endpoints (#1668)
* WIP workspace threads developer api endpoints

* fix swagger docs

* fix broken endpoints and test on single/multi user modes

* support for chatting with workspace thread by userId

* update swagger json
2024-06-20 16:27:35 -07:00
Sean Hatfield
c2523a9593
[FEAT] Persist query mode refusal responses as chat history (#1727)
* log query refusals to workspace chats but hide in ui

* linting

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-06-20 15:44:19 -07:00
Timothy Carambat
1d675d09fb
RU Transaltions (#1734)
* ru language added

* uncheck bad changes
Use chatGPT to create new dictionary for RU translation
turn off debug mode in produ for i18n
patch issue with null comparison in verifyTranslation.js

---------

Co-authored-by: UUSR <konstantik@gmail.com>
2024-06-20 15:36:39 -07:00