Commit Graph

886 Commits

Author SHA1 Message Date
timothycarambat
8658b1e7c7 linting 2024-07-03 18:25:44 -07:00
Timothy Carambat
29c9eeaa5c
Add winston logging for production (#1811) 2024-07-03 16:39:33 -07:00
Timothy Carambat
06fb9bdd99
1738 generic open ai agent (#1809)
* enable support for generic openAI as Agent provider
2024-07-03 15:17:53 -07:00
Sean Hatfield
8b5d9ccdb3
[FEAT] Fork chat to new thread (#1788)
* implement thread forking feature

* rename thread based on forked message

* refactor bulk message create for thread fork + bump prisma version

* revert prisma version bump

* add todo to bulkCreate function in workspace chats

* cast user input to expected type to prevent prisma injection

* refactor: update order of ops for thread fork

---------

Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
2024-07-03 14:44:35 -07:00
Sean Hatfield
a87014822a
[REFACTOR] Improve asPDF collector processor with pdfjs (#1791)
* WIP replace langchain pdfloader with pdfjs and add more context to each page

* remove extras from pdfjs and just replace langchain library

* remove unneeded dep

* fix console log in docs

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-07-03 14:26:48 -07:00
timothycarambat
3fa00ad7b6 Merge branch 'master' of github.com:Mintplex-Labs/anything-llm 2024-07-03 14:02:42 -07:00
timothycarambat
89538c3614 fix typo in agent debug logs 2024-07-03 14:02:33 -07:00
Sean Hatfield
f205d51fe9
[FIX] Confluence code snippet blocks not being extracted (#1804)
implement custom confluence loader to extract code blocks properly from documents

Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
2024-07-03 14:00:44 -07:00
timothycarambat
fbc9c868c4 chore: silence getGitVersion for docker env runtime 2024-07-03 11:50:31 -07:00
Timothy Carambat
cb6b3340f0
Update bgworker to use fork instead of worker_thread (#1808) 2024-07-03 11:44:34 -07:00
Sean Hatfield
fc375f4036
[FIX] Bulk link scraper bug fix (#1800)
patch website depth data connector to work for other links that are not root url
2024-07-01 16:59:28 -07:00
apstrom
b105c6c969
Update index.jsx (#1794)
Max. number of context snippets increased from 12 to 200. 
TODO: May remove restraint entirely
2024-07-01 16:39:59 -07:00
Timothy Carambat
245d1b878e
Patch Embed styles with prefixing (#1799)
* Patch Embed styles with prefixing

* forgot files
2024-07-01 16:32:23 -07:00
Timothy Carambat
0b845fbb1c
Deprecate .isSafe moderation (#1790)
Add type defs to helpers
2024-06-28 15:32:30 -07:00
Sean Hatfield
910eb36cfe
[FIX] OpenAI compatible endpoints query mode developer API bug fix (#1789)
fix query mode always responding with refusal message on develop api openai compatible endpoints
2024-06-28 14:23:16 -07:00
Timothy Carambat
7a78ad3960
Apply workspace topN to agent call (#1779)
resolves #1777
2024-06-27 17:58:28 -07:00
timothycarambat
4afdf846e0 patch reveal hook 2024-06-27 12:51:28 -07:00
timothycarambat
834fb5f900 Merge branch 'master' of github.com:Mintplex-Labs/anything-llm 2024-06-26 23:48:46 -07:00
timothycarambat
d9cfcba9bb prevent holdToReveal from spamming toasts 2024-06-26 23:48:38 -07:00
Timothy Carambat
9bd65f1567
[CHORE] Migration from vectordb to @lancedb/lancedb NodeJS SDK (#1766)
WIP on migration to @lancedb/lancedb NodeJS SDK
2024-06-26 21:57:16 -07:00
timothycarambat
69cb71b1fc patch cached tag overflow of filename text
patch parseChunkHeader bug
2024-06-26 21:45:52 -07:00
timothycarambat
c341f7a298 Merge branch 'master' of github.com:Mintplex-Labs/anything-llm 2024-06-26 21:02:14 -07:00
timothycarambat
0c32b31f3a fix filerow overflow 2024-06-26 21:02:04 -07:00
Sean Hatfield
77916d920b
[FEAT] Remove users_can_delete_workspaces feature (#1742)
remove all references to users_can_delete_workspaces option
2024-06-25 16:46:06 -07:00
Sean Hatfield
b5ac62ff9f
[FEAT] Auto detection for Ollama and LMStudio (#1756)
* add ollama automatic url detection in llm and embedder prefrences

* implement auto detection for lmstudio llm and embedder/improve performance of checking common urls

* fix modal not clearing

* fix lmstudio url check

* improve ux for ollama llm provider option

* improve ux for lm studio llm provider option

* improve ux for ollama embedder option

* improve ux for lmstudio embedder option

* ux improvement lmstudio embedder options

* refactor implementation to hook and use native timeout
Swap to promise.any for resolving of available endpoints

* implement useProviderEndpointAutoDiscovery hook for lmstudio and ollama provider options

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-06-25 16:30:05 -07:00
Jason Zhang
fa4ab0f65f
fix: sanitize filename before writing (#1743)
* fix: sanitize filename before writing

Fixes: https://github.com/Mintplex-Labs/anything-llm/issues/1737

* fixup

* fixup
2024-06-25 15:45:09 -07:00
Sean Hatfield
dde8bc238b
[FIX] Remove Azure URL validation (#1758)
remove azure url validation
2024-06-25 12:10:51 -07:00
Timothy Carambat
88a0335fd2
Add OpenAI compatible API support (#1757) 2024-06-24 18:28:21 -07:00
timothycarambat
158ba83cab Merge branch 'master' of github.com:Mintplex-Labs/anything-llm 2024-06-24 09:32:53 -07:00
timothycarambat
dfff9070db fix typo 2024-06-24 09:32:42 -07:00
Sean Hatfield
e72fa8b370
[FEAT] Generic OpenAI embedding provider (#1664)
* implement generic openai embedding provider

* linting

* comment & description update for generic openai embedding provider

* fix privacy for generic

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-06-21 16:27:02 -07:00
Timothy Carambat
dc4ad6b5a9
[BETA] Live document sync (#1719)
* wip bg workers for live document sync

* Add ability to re-embed specific documents across many workspaces via background queue
bgworkser is gated behind expieremental system setting flag that needs to be explictly enabled
UI for watching/unwatching docments that are embedded.
TODO: UI to easily manage all bg tasks and see run results
TODO: UI to enable this feature and background endpoints to manage it

* create frontend views and paths
Move elements to correct experimental scope

* update migration to delete runs on removal of watched document

* Add watch support to YouTube transcripts (#1716)

* Add watch support to YouTube transcripts
refactor how sync is done for supported types

* Watch specific files in Confluence space (#1718)

Add failure-prune check for runs

* create tmp workflow modifications for beta image

* create tmp workflow modifications for beta image

* create tmp workflow modifications for beta image

* dual build
update copy of alert modals

* update job interval

* Add support for live-sync of Github files

* update copy for document sync feature

* hide Experimental features from UI

* update docs links

* [FEAT] Implement new settings menu for experimental features (#1735)

* implement new settings menu for experimental features

* remove unused context save bar

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>

* dont run job on boot

* unset workflow changes

* Add persistent encryption service
Relay key to collector so persistent encryption can be used
Encrypt any private data in chunkSources used for replay during resync jobs

* update jsDOC

* Linting and organization

* update modal copy for feature

---------

Co-authored-by: Sean Hatfield <seanhatfield5@gmail.com>
2024-06-21 13:38:50 -07:00
timothycarambat
c176fe38d2 remove async on dumpEnv 2024-06-20 20:50:14 -07:00
timothycarambat
ed53bbcf15 Merge branch 'master' of github.com:Mintplex-Labs/anything-llm 2024-06-20 18:03:33 -07:00
timothycarambat
9411791fe5 Simplify ENV management
auto write ENV on changes
Dont double-track envs that are already tracked
2024-06-20 18:03:21 -07:00
Sean Hatfield
12bf576e16
[FEAT] Workspace threads developer API endpoints (#1668)
* WIP workspace threads developer api endpoints

* fix swagger docs

* fix broken endpoints and test on single/multi user modes

* support for chatting with workspace thread by userId

* update swagger json
2024-06-20 16:27:35 -07:00
Sean Hatfield
c2523a9593
[FEAT] Persist query mode refusal responses as chat history (#1727)
* log query refusals to workspace chats but hide in ui

* linting

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-06-20 15:44:19 -07:00
Timothy Carambat
1d675d09fb
RU Transaltions (#1734)
* ru language added

* uncheck bad changes
Use chatGPT to create new dictionary for RU translation
turn off debug mode in produ for i18n
patch issue with null comparison in verifyTranslation.js

---------

Co-authored-by: UUSR <konstantik@gmail.com>
2024-06-20 15:36:39 -07:00
Timothy Carambat
8b1ceb30c1
Add support for searXNG search for agents (#1733)
resolves #1367
2024-06-20 14:08:00 -07:00
Sean Hatfield
524edd6e69
[FEAT] Add support for Claude Sonnet 3.5 model (#1731)
add support for claude sonnet 3.5 model
2024-06-20 10:13:53 -07:00
timothycarambat
d043fdada6 Add FullScreenLoader as suspense fallback 2024-06-20 10:11:48 -07:00
Timothy Carambat
610c87ce19
Init support of i18n and English, Mandarin, Spanish, French (#1317)
* Init support of i18n and English and mandarin

* Update common.js (#1320)

* add General Appearance and Chat setting zh translate (#1414)

* add config zh translate (#1461)

* patch some translation pages

* Update locality fixes

* update: complete login page Mandarin translation. (#1709)

update: complete Mandarin translation.

* complete translation

* update github to run validator

* bump to test workflow failure

* bump to fix tests

* update workflow

* refactor lang selector support

* add Spanish and French

* add dictionaries

---------

Co-authored-by: GetOffer.help <13744916+getofferhelp@users.noreply.github.com>
Co-authored-by: AIR <129256286+KochabStar@users.noreply.github.com>
Co-authored-by: Ezio T <ezio5600@gmail.com>
2024-06-19 14:48:19 -07:00
timothycarambat
421c5c6b91 linting 2024-06-18 11:30:50 -07:00
lewismacnow
6c68bdbd51
1711 enhance workspace users api (#1712)
* Update workspace.js

adds userId to the workspaceUsers function

* Update openapi.json

Updates the workspace/users API docs to display correct 200 response example, including userId, username, role and last update datetime.
2024-06-18 09:35:40 -07:00
Sean Hatfield
1c473621aa
[FEAT] Add user management endpoints to list all users via developer API (#1708)
* add user management endpoints to list all users via developer api

* rename users to user management in swagger docs

* linting

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-06-17 16:24:41 -07:00
Timothy Carambat
a598c8e04c
1347 human readable confluence url (#1706)
* chore: confluence data connector can now handle custom urls, in addition to default {subdomain}.atlassian.net ones

* chore: formatting as per yarn lint

* chore: fixing the human readable confluence url fetch baseUrl

* chore: fixing the human readable confluence url fetch baseUrl

* chore: fixing the human readable confluence url fetch baseUrl

* chore: fixing the human readable confluence url fetch baseUrl

* chore: fixing the human readable confluence url fetch baseUrl

* refactor implementation of various types of Confluence URL patterns

---------

Co-authored-by: Predrag Stojadinovic <predrag@stojadinovic.net>
Co-authored-by: Predrag Stojadinović <cope@users.noreply.github.com>
Co-authored-by: Predrag Stojadinovic <predrags@nvidia.com>
2024-06-17 16:04:20 -07:00
Sean Hatfield
c8c618137f
[CHORE] Clean up text and bg hex colors (#1685)
clean up text and bg hex colors

Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
2024-06-17 14:55:56 -07:00
Sean Hatfield
c5ba2d73d0
[STYLE] Document picker UI improvements (#1700)
* rework document picker ui

* refactor tooltips to use react-tooltip lib with custom elements

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-06-17 14:26:44 -07:00
timothycarambat
0d5cc558c9 linting
patch UI exception with invalid return value from failed parse
resolves #1692
2024-06-17 13:55:10 -07:00
lewismacnow
38441f4b21
1693 get workspace users api (#1694)
* Enhance API - add GET for users with access to workspace

Adds GET request endpoint for retrieving a list of users with permissions to access the specified workspace

* Update Swagger for users with access to workspace

Adds swagger docs for the endpoint used to retrieve users with access to workspace. "v1/admin/workspaces/:workspaceId/users"
2024-06-17 13:52:01 -07:00