Commit Graph

509 Commits

Author SHA1 Message Date
timothycarambat
2d2e49bc00 fix missing export 2024-08-21 14:58:56 -07:00
Timothy Carambat
1f96b837b3
Refactor api endpoint chat handler to its own function (#2157)
remove legacy `chatWithWorkspace` and cleanup `index.js`
2024-08-21 14:47:06 -07:00
timothycarambat
17abbe97e5 remove #swagger.path from API docs 2024-08-20 12:48:08 -07:00
timothycarambat
cb7cb2d976 Add 405B to perplexity 2024-08-19 12:26:22 -07:00
Timothy Carambat
a8d25c7dd3
Allow readable username passed in script widget (#2131) 2024-08-16 16:55:42 -07:00
Sean Hatfield
1dad4d9409
Fix multi-user setup bug for invalid username/password (#2130)
* fix multiuser setup bug for invalid username/password

* handle user creation failed gracefully and delete user on fail
2024-08-16 12:55:14 -07:00
Timothy Carambat
c8fe254d45
Omit invalid response.text values and prompts (#2127)
* Omit invalid `response.text` values and `prompts`
resolves #2108

* remove import
2024-08-15 14:22:27 -07:00
Timothy Carambat
99f2c25b1c
Agent Context window + context window refactor. (#2126)
* Enable agent context windows to be accurate per provider:model

* Refactor model mapping to external file
Add token count to document length instead of char-count
refernce promptWindowLimit from AIProvider in central location

* remove unused imports
2024-08-15 12:13:28 -07:00
Shahar
4365d69359
Fix TypeError by replacing this.openai.createChatCompletion with the correct function call (#2117)
fixed new api syntax
2024-08-14 14:39:48 -07:00
Timothy Carambat
4430ddb059
Encryption in JWT for single-user password mode (#2111)
* wip encrypting jwt value

* Encrypt/Decrypt pass in JWT value for verification in single-user password mode
2024-08-13 17:54:12 -07:00
PyKen
a2571024a9
Add prompt window limits for gpt-4o-* models (#2104) 2024-08-13 09:13:36 -07:00
Timothy Carambat
f06ef6180d
add exp model to v1Beta (#2082) 2024-08-09 14:19:49 -07:00
timothycarambat
c9e19264eb fix validation issue 2024-08-07 14:24:24 -07:00
Sean Hatfield
665553af5a
Developer API access for embed chat widgets (#1999)
* developer api access for embedded chat widgets and embed chats

* lint

* uncheck files

* sanitize embed api inputs

---------

Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
2024-08-07 11:36:32 -07:00
Sean Hatfield
c97066526a
New user account validations (#2037)
* force lowercase and no space for new and
 edit user modals

* edit account modal validations

* use pattern for form validation + remove validations from edit user

* revert comment deletions

* comment fix

* update validation message

* update regex
allow updating by block name changes to invalid names

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-08-07 11:35:37 -07:00
Timothy Carambat
d072875e43
Add piperTTS in-browser text-to-speech (#2052)
* Add piperTTS in-browser text-to-speech

* update vite config

* Add voice default + change prod public URL

* uncheck file

* Error handling
bump package for better quality and voices

* bump package

* Remove pre-packed WASM - will not support offline first solution for docker

* attach TTSProvider telem
2024-08-07 11:09:51 -07:00
Sean Hatfield
6666fff0c2
Support multiple preset prompts in single message (#2036)
support multiple preset prompts in single message
2024-08-06 10:27:36 -07:00
Sean Hatfield
7273c892a1
Ollama performance mode option (#2014)
* ollama performance mode option

* Change ENV prop
Move perf setting to advanced

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-08-02 13:29:17 -07:00
Timothy Carambat
ba8e4e5d3e
handle OpenRouter exceptions on streaming (#2033) 2024-08-02 12:23:39 -07:00
RahSwe
c55ef33fce
Gemini Pro 1.5, API support for 2M context and new experimental model (#2031) 2024-08-02 10:24:31 -07:00
timothycarambat
6dc3642661 Patch Groq preview models maxed to 8K tokens due to warning 2024-08-01 09:24:57 -07:00
timothycarambat
466bf7dc9c Bump Perplexity and Together AI static model list 2024-07-31 10:58:34 -07:00
Timothy Carambat
38fc181238
Add multimodality support (#2001)
* Add multimodality support

* Add Bedrock, KoboldCpp,LocalAI,and TextWebGenUI multi-modal

* temp dev build

* patch bad import

* noscrolls for windows dnd

* noscrolls for windows dnd

* update README

* update README

* add multimodal check
2024-07-31 10:47:49 -07:00
Timothy Carambat
20cd6b7481
Added more char exceptions to mapping extension (#2010) 2024-07-31 09:41:12 -07:00
Timothy Carambat
d877d2b7ad
Add drag-and-drop to chat window (#1995)
* Add drag-and-drop to chat window

* add uploader icon and remove empty space text when attachments are present

* color theme

* color update
2024-07-30 10:26:16 -07:00
Timothy Carambat
5e73dce506
Enable editing of OpenRouter stream timeout for slower connections (#1994) 2024-07-29 11:49:14 -07:00
timothycarambat
296f041564 path perplexity model ids
closes #1990
2024-07-28 16:29:18 -07:00
Timothy Carambat
ae58a2cb0d
Return threads for API workspace calls (#1978)
* Return threads for API workspace calls

* remove import
2024-07-26 14:18:40 -07:00
timothycarambat
7a2ffefdc3 update case stmt for duplicate groq model 2024-07-25 17:39:29 -07:00
timothycarambat
6bab8b5bd4 ui adjustments for order flow
back nav on order flow
fix bad schema ref
2024-07-25 14:10:17 -07:00
Timothy Carambat
88e2209959
bump jsonwebtoken version (#1971)
* bump `jsonwebtoken` version

* update dev build branch
2024-07-25 11:03:39 -07:00
Timothy Carambat
04a0fc4ec9
Remove unused deps (#1938)
* Remove unused deps

* improve dependency
2024-07-25 10:21:03 -07:00
Timothy Carambat
61e214aa8c
Add support for Groq /models endpoint (#1957)
* Add support for Groq /models endpoint

* linting
2024-07-24 08:35:52 -07:00
Timothy Carambat
23de85a3bd
Revert odbc support (#1936)
* Revert "Patch ODBC support from missing binary/headers for node-odbc"

This reverts commit 9de6b1cc26.

* Revert "OBDC Support (#1933)"

This reverts commit cd597a361e.
2024-07-23 17:27:39 -07:00
Timothy Carambat
9366e69d88
Add AWS bedrock support for LLM + agents (#1935)
add AWS bedrock support for LLM + agents
2024-07-23 16:35:37 -07:00
Timothy Carambat
cd597a361e
OBDC Support (#1933)
* add possibility to connect to SQL Base by ODBC

---------

Co-authored-by: suchaudn <nicolas.suchaud@legrand.fr>
Co-authored-by: nicho2 <nicho2@laposte.net>
2024-07-23 12:42:53 -07:00
Timothy Carambat
42235fcd8a
GitLab Hosted and Local Connector (#1932)
* Add support for GitLab repo collection as well as Github Repo collection
* Refactor for repo collectors to be more compact

---------

Co-authored-by: Emil Rofors <emirof@gmail.com>
2024-07-23 12:23:51 -07:00
Timothy Carambat
f8e54b2328
Fine tuning order flow (cloud-based) (#1923)
* WIP fine-tuning order flow

* flow patches and typo

* Refine steps
add fine tuning CTA on chat page
add fine tuning banner and validation endpoints
add finetuning banner on relevant pages with perms check

* Add prod firebase url
2024-07-22 17:59:17 -07:00
timothycarambat
d0713e15f1 patch response body for swagger 2024-07-22 15:21:32 -07:00
Timothy Carambat
76aa2a4fd4
Implement support for selecting basic keep_alive times for Ollama (#1920) 2024-07-22 14:44:47 -07:00
timothycarambat
7b42a67acf docs: update requirement for /raw-text endpoint
resolves #1893
2024-07-22 13:19:57 -07:00
timothycarambat
2185753068 patch text.substring bug from compressor 2024-07-22 12:53:11 -07:00
Timothy Carambat
3198718975
Update references to new domain (#1916) 2024-07-22 11:05:34 -07:00
timothycarambat
f15529653f patch logger for full logs 2024-07-19 18:35:41 -07:00
timothycarambat
cec1a3d585 append stacktraces to winston 2024-07-19 18:13:54 -07:00
Timothy Carambat
b5a2437bb5
patch docker scount CVE in old express-ws pkg (#1907)
* patch CVE in old express-ws pkg

* patch workflow

* remove dev-image
2024-07-19 17:40:22 -07:00
Timothy Carambat
8180787c2e
Enable editing and setting of meta-tag information (#1892)
* Enable editing and setting of meta-tag information

* cleanup

* tmp build for testing

* finally always refresh

* unset workflow

* dev build

* rm tmp build
2024-07-19 15:58:43 -07:00
timothycarambat
f9929a28cb patch lint 2024-07-19 15:27:56 -07:00
Johnny Chen
c4158b97d6
bugfix: API new user return 400 if user creation failed (#1898)
* return 400 if user creation failed

The client might not check the error field unless they realize it's a 4xx error instead of 200 success.

* Update index.js

* use one-liner style

---------

Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
2024-07-19 15:27:21 -07:00
Timothy Carambat
5df6b5f7d9
Bump perplexity models (#1905)
* Added Supported Models Free Tier - chat_models.txt

Need to fill in correct Parameter Count.

* Bump perplexity model
closes #1901
closes #1900

---------

Co-authored-by: Tim-Hoekstra <135951177+Tim-Hoekstra@users.noreply.github.com>
2024-07-19 15:11:10 -07:00
Timothy Carambat
b275008b19
Docker image updates (#1896)
* bump base image

* Bump mysql

* attestations

* attestations perms

* attestations perms

* fix permissions for attestetions for master push

* test cleanup

* repin base image

* revert base

* patch frontend-clean

* force resolve braces to fixed version

* rebump
2024-07-19 08:09:32 -07:00
Timothy Carambat
75b6710052
Extend support for some chars in slugify of workspace (#1890)
resolves #1877
2024-07-18 10:35:02 -07:00
Sean Hatfield
e909b25b29
[FEAT] Prisma injection validation (#1874)
check all prisma models/model usage and patch any potential sql injection vulns
2024-07-16 16:40:05 -07:00
timothycarambat
86a66ba569 change alpaca format to include citations and system prompt 2024-07-15 16:05:53 -07:00
Sean Hatfield
8f0af88332
[FIX] Fix manual input model pref for agents (#1848)
* patch llm providers that have manual inputs for model pref

* refactor agent model fallback
update UI to show disabled providers to stop questions about provider limitations

* patch log on startup

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-07-11 14:03:24 -07:00
Sean Hatfield
ab56bae1bb
[FEAT] Ability to delete messages from conversation history (#1812)
* implement delete message feature

* add new delete chat endpoint that just hides messages from ui and fix perms

* refactor: add delete-event hooks for delete so we dont need to pass so much
refactor: implmentation of delete endpoint PUT

* update animation
linting
remove element on animiation to prevent popin

* linting and cleanup imports

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-07-03 18:28:08 -07:00
timothycarambat
8658b1e7c7 linting 2024-07-03 18:25:44 -07:00
Timothy Carambat
29c9eeaa5c
Add winston logging for production (#1811) 2024-07-03 16:39:33 -07:00
Sean Hatfield
8b5d9ccdb3
[FEAT] Fork chat to new thread (#1788)
* implement thread forking feature

* rename thread based on forked message

* refactor bulk message create for thread fork + bump prisma version

* revert prisma version bump

* add todo to bulkCreate function in workspace chats

* cast user input to expected type to prevent prisma injection

* refactor: update order of ops for thread fork

---------

Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
2024-07-03 14:44:35 -07:00
timothycarambat
89538c3614 fix typo in agent debug logs 2024-07-03 14:02:33 -07:00
timothycarambat
fbc9c868c4 chore: silence getGitVersion for docker env runtime 2024-07-03 11:50:31 -07:00
Timothy Carambat
cb6b3340f0
Update bgworker to use fork instead of worker_thread (#1808) 2024-07-03 11:44:34 -07:00
Timothy Carambat
0b845fbb1c
Deprecate .isSafe moderation (#1790)
Add type defs to helpers
2024-06-28 15:32:30 -07:00
Sean Hatfield
910eb36cfe
[FIX] OpenAI compatible endpoints query mode developer API bug fix (#1789)
fix query mode always responding with refusal message on develop api openai compatible endpoints
2024-06-28 14:23:16 -07:00
Timothy Carambat
7a78ad3960
Apply workspace topN to agent call (#1779)
resolves #1777
2024-06-27 17:58:28 -07:00
Timothy Carambat
9bd65f1567
[CHORE] Migration from vectordb to @lancedb/lancedb NodeJS SDK (#1766)
WIP on migration to @lancedb/lancedb NodeJS SDK
2024-06-26 21:57:16 -07:00
Sean Hatfield
77916d920b
[FEAT] Remove users_can_delete_workspaces feature (#1742)
remove all references to users_can_delete_workspaces option
2024-06-25 16:46:06 -07:00
Sean Hatfield
dde8bc238b
[FIX] Remove Azure URL validation (#1758)
remove azure url validation
2024-06-25 12:10:51 -07:00
Timothy Carambat
88a0335fd2
Add OpenAI compatible API support (#1757) 2024-06-24 18:28:21 -07:00
Sean Hatfield
e72fa8b370
[FEAT] Generic OpenAI embedding provider (#1664)
* implement generic openai embedding provider

* linting

* comment & description update for generic openai embedding provider

* fix privacy for generic

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-06-21 16:27:02 -07:00
Timothy Carambat
dc4ad6b5a9
[BETA] Live document sync (#1719)
* wip bg workers for live document sync

* Add ability to re-embed specific documents across many workspaces via background queue
bgworkser is gated behind expieremental system setting flag that needs to be explictly enabled
UI for watching/unwatching docments that are embedded.
TODO: UI to easily manage all bg tasks and see run results
TODO: UI to enable this feature and background endpoints to manage it

* create frontend views and paths
Move elements to correct experimental scope

* update migration to delete runs on removal of watched document

* Add watch support to YouTube transcripts (#1716)

* Add watch support to YouTube transcripts
refactor how sync is done for supported types

* Watch specific files in Confluence space (#1718)

Add failure-prune check for runs

* create tmp workflow modifications for beta image

* create tmp workflow modifications for beta image

* create tmp workflow modifications for beta image

* dual build
update copy of alert modals

* update job interval

* Add support for live-sync of Github files

* update copy for document sync feature

* hide Experimental features from UI

* update docs links

* [FEAT] Implement new settings menu for experimental features (#1735)

* implement new settings menu for experimental features

* remove unused context save bar

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>

* dont run job on boot

* unset workflow changes

* Add persistent encryption service
Relay key to collector so persistent encryption can be used
Encrypt any private data in chunkSources used for replay during resync jobs

* update jsDOC

* Linting and organization

* update modal copy for feature

---------

Co-authored-by: Sean Hatfield <seanhatfield5@gmail.com>
2024-06-21 13:38:50 -07:00
timothycarambat
c176fe38d2 remove async on dumpEnv 2024-06-20 20:50:14 -07:00
timothycarambat
ed53bbcf15 Merge branch 'master' of github.com:Mintplex-Labs/anything-llm 2024-06-20 18:03:33 -07:00
timothycarambat
9411791fe5 Simplify ENV management
auto write ENV on changes
Dont double-track envs that are already tracked
2024-06-20 18:03:21 -07:00
Sean Hatfield
12bf576e16
[FEAT] Workspace threads developer API endpoints (#1668)
* WIP workspace threads developer api endpoints

* fix swagger docs

* fix broken endpoints and test on single/multi user modes

* support for chatting with workspace thread by userId

* update swagger json
2024-06-20 16:27:35 -07:00
Sean Hatfield
c2523a9593
[FEAT] Persist query mode refusal responses as chat history (#1727)
* log query refusals to workspace chats but hide in ui

* linting

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-06-20 15:44:19 -07:00
Timothy Carambat
8b1ceb30c1
Add support for searXNG search for agents (#1733)
resolves #1367
2024-06-20 14:08:00 -07:00
Sean Hatfield
524edd6e69
[FEAT] Add support for Claude Sonnet 3.5 model (#1731)
add support for claude sonnet 3.5 model
2024-06-20 10:13:53 -07:00
timothycarambat
421c5c6b91 linting 2024-06-18 11:30:50 -07:00
lewismacnow
6c68bdbd51
1711 enhance workspace users api (#1712)
* Update workspace.js

adds userId to the workspaceUsers function

* Update openapi.json

Updates the workspace/users API docs to display correct 200 response example, including userId, username, role and last update datetime.
2024-06-18 09:35:40 -07:00
Sean Hatfield
1c473621aa
[FEAT] Add user management endpoints to list all users via developer API (#1708)
* add user management endpoints to list all users via developer api

* rename users to user management in swagger docs

* linting

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-06-17 16:24:41 -07:00
timothycarambat
0d5cc558c9 linting
patch UI exception with invalid return value from failed parse
resolves #1692
2024-06-17 13:55:10 -07:00
lewismacnow
38441f4b21
1693 get workspace users api (#1694)
* Enhance API - add GET for users with access to workspace

Adds GET request endpoint for retrieving a list of users with permissions to access the specified workspace

* Update Swagger for users with access to workspace

Adds swagger docs for the endpoint used to retrieve users with access to workspace. "v1/admin/workspaces/:workspaceId/users"
2024-06-17 13:52:01 -07:00
timothycarambat
05870ec8d0 chore: update no agent error message
style: update error message width
2024-06-12 13:19:06 -07:00
timothycarambat
4f15f95f53 hotfix: fix bad semantic HTML
feat: add /reset to bailout for agent session
fix: patch save-to-browser not enabling
2024-06-12 12:43:06 -07:00
Sean Hatfield
9c3014de70
[FEAT] Agent skills UI redesign (#1565)
* WIP agent settings redesign

* WIP rework new agent skill UI

* WIP save bar/agent styles

* WIP update settings fix

* desktop agent config UI implementation

* remove unneeded files

* fix sql and web browsing plugins not starting & add default badges

* fix serply merge conflict

* review: cleanup unused files/folders/components

* refactor components

* refactor components

* fix order of customized skills

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-06-12 10:52:32 -07:00
timothycarambat
29263dab52 patch: Fix API delete documents 2024-06-11 08:14:13 -07:00
timothycarambat
394b5aafb6 linting 2024-06-10 15:22:32 -07:00
Serply
7693240e21
1646-added serply (#1647)
* added serply search api

* undo remove of new line

---------

Co-authored-by: teampen <136991215+teampen@users.noreply.github.com>
2024-06-10 15:17:41 -07:00
Timothy Carambat
d470845931
feat: expand summarization to OSS LLM (#1648) 2024-06-10 14:31:39 -07:00
Timothy Carambat
b30dc1c956
patch: Auto thread renaming (#1636) 2024-06-07 17:12:54 -07:00
Sean Hatfield
3c98d15c6a
[FEAT] Rename new threads on thread creation (#1607)
* make thread name 'Thread' when new thread is created

* implement auto generated thread titles

* implement truncated prompt as thread name

* move rename of thread into workspaceThread function

---------

Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
2024-06-07 14:06:47 -07:00
timothycarambat
3434bf4686 linting 2024-06-07 10:09:35 -07:00
yonathan suarez
bb0a4b0531
Add API Endpoint for Deleting Documents in system (#1623)
* fix: "remove document system api"

* fix: "update text system system "

---------

Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
2024-06-07 09:22:03 -07:00
Sean Hatfield
1b8386b079
[FIX] ChromaDB namespace normalization (#1625)
* chromadb namespace normalization

* update normalization function with more clarity

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-06-06 15:38:05 -07:00
Anush
771889ad7f
[FIX] Incorrect vectors count with Qdrant (#1561)
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
2024-06-06 13:18:01 -07:00
Sean Hatfield
26c220503c
[FEAT] Edit message button (#1392)
* WIP edit message feature

* WIP edit message

* WIP editing messages feature

* Fix PFPs
TODO: Fix default user profile image
Add User and Assistant workspace response

* unset PFP changes for later PR

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-06-06 12:56:11 -07:00
Timothy Carambat
98cef508a6
Feature/devcontv2 (#1622)
* Updated apt-packages source for devcontainer

Switched the devcontainer's package source to a different repository to
align with updated dependencies and package availability. The previous
source from 'rocker-org' is replaced with 'devcontainers-contrib', which
may offer more recent or relevant development tools.

* Subject: Centralize prettier ignores and refine
config

Body:
Centralized all prettier ignore rules by removing individual
`.prettierignore` files in subprojects and updating the root
`.prettierignore` to include previously ignored patterns, ensuring
consistency across the workspace. Additionally, the prettier
configuration was refined by making the file pattern for `.config.js`
files consistent and adjusting quote styles for better readability. All
lint scripts across the project were updated to respect the centralized
ignore path, enhancing maintainability.

The consolidation simplifies the process of managing ignore rules as the
project scales, ensuring developers can focus on writing code without
worrying about divergent formatting standards. These changes also align
with introducing comprehensive linting across multiple environments to
keep the codebase clean and consistent.

This adjustment is a foundational step towards a more streamlined and
unified code base, making it easier for new contributors to adhere to
established coding standards and reducing the cognitive load associated
with managing multiple configuration files across the project.

* unset package json changes

---------

Co-authored-by: Francisco Bischoff <franzbischoff@gmail.com>
Co-authored-by: Francisco Bischoff <984592+franzbischoff@users.noreply.github.com>
2024-06-06 12:50:42 -07:00
Sean Hatfield
d29292ebd2
[FEAT] Add LiteLLM embedding provider support (#1579)
* add liteLLM embedding provider support

* update tooltip id

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-06-06 12:43:34 -07:00
Sean Hatfield
3f78ef413b
[FEAT] Support for gemini-1.0-pro model and fixes to prompt window limit (#1557)
support for gemini-1.0-pro model and fixes to prompt window limit
2024-05-29 08:17:35 +08:00