Commit Graph

802 Commits

Author SHA1 Message Date
Sean Hatfield
7fb76cfef0
add support for exporting to json and csv in workspace chats (#622)
* add support for exporting to json and csv in workspace chats

* safety encode URL options

* remove message about openai fine tuning on export success

* all defaults to jsonl
2024-01-18 17:59:51 -08:00
Sean Hatfield
08d33cfd8f
Merge pull request from GHSA-xmj6-g32r-fc5q
* remove support for import export anythingllm data

* remove unused imports
remove unused dep
update lockfile

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-01-18 13:57:23 -08:00
Sean Hatfield
56fa17caf2
create configurable topN per workspace (#616)
* create configurable topN per workspace

* Update TopN UI text
Fix fallbacks for all providers
Add SQLite CHECK to TOPN value

* merge with master
Update zilliz provider for variable TopN

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-01-18 12:34:20 -08:00
Timothy Carambat
683cd69a97
Display readable error message in chat failure (#621) 2024-01-18 12:01:53 -08:00
Timothy Carambat
658e7fa390
chore: Better VectorDb and Embedder error messages (#620)
* chore: propogate embedder and vectordb errors during document mutations

* add default value for errors on addDocuments
2024-01-18 11:40:48 -08:00
Timothy Carambat
0df86699e7
feat: Add support for Zilliz Cloud by Milvus (#615)
* feat: Add support for Zilliz Cloud by Milvus

* update placeholder text
update data handling stmt

* update zilliz descriptor
2024-01-17 18:00:54 -08:00
Sean Hatfield
3fe7a25759
add token context limit for native llm settings (#614)
Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-01-17 16:25:30 -08:00
Sean Hatfield
56dc49966d
add copy feature to assistant chat message (#611)
* add copy feature to assistant chat message

* fix tooltip not hiding on mobile

* fix: add tooltips
chore: breakout actions to extendable component + memoize
add CopyText to hook we can reuse
fix: Copy on code snippets broken, moved to event listener
fix: highlightjs patch for new API support
feat: add copy response support

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-01-17 16:22:06 -08:00
Sean Hatfield
c2c8fe9756
add support for mistral api (#610)
* add support for mistral api

* update docs to show support for Mistral

* add default temp to all providers, suggest different results per provider

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-01-17 14:42:05 -08:00
Sean Hatfield
90df37582b
Per workspace model selection (#582)
* WIP model selection per workspace (migrations and openai saves properly

* revert OpenAiOption

* add support for models per workspace for anthropic, localAi, ollama, openAi, and togetherAi

* remove unneeded comments

* update logic for when LLMProvider is reset, reset Ai provider files with master

* remove frontend/api reset of workspace chat and move logic to updateENV
add postUpdate callbacks to envs

* set preferred model for chat on class instantiation

* remove extra param

* linting

* remove unused var

* refactor chat model selection on workspace

* linting

* add fallback for base path to localai models

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-01-17 12:59:25 -08:00
Sean Hatfield
bf503ee0e9
add check to skip empty messages (#602)
* add check to skip empty messages

* add comment explaining prisma + sqlite not supporting createMany()
2024-01-16 18:23:51 -08:00
Timothy Carambat
b35feede87
570 document api return object (#608)
* Add support for fetching single document in documents folder

* Add document object to upload + support link scraping via API

* hotfixes for documentation

* update api docs
2024-01-16 16:04:22 -08:00
Timothy Carambat
c61cbd1502
Add support for fetching single document in documents folder (#607) 2024-01-16 14:58:49 -08:00
Timothy Carambat
e973c1edbf
Preload onboarding (#606)
* no lazy onboarding

* no lazy onboarding steps

* Do not lazy load onboarding to prevent lazy-load white flash
2024-01-16 14:15:46 -08:00
Sean Hatfield
8b11288764
truncate title to shorter length so x button is not hidden for longer… (#603)
* truncate title to shorter length so x button is not hidden for longer title names in the citation modal

* absolutely position x button on citation modal
2024-01-16 13:43:32 -08:00
Timothy Carambat
d0a3f1e3e1
Fix present diminsions on vectorDBs to be inferred for providers who require it (#605) 2024-01-16 13:41:01 -08:00
Timothy Carambat
f5bb064dee
Implement streaming for workspace chats via API (#604) 2024-01-16 10:37:46 -08:00
Timothy Carambat
bd158ce7b1
[Feat] Query mode to return no-result when no context found (#601)
* Query mode to return no-result when no context found

* update default error for sync chat

* remove unnecessary type conversion
2024-01-16 09:32:51 -08:00
timothycarambat
7aaa4b38e7 add flex role to export endpoint 2024-01-14 17:10:49 -08:00
timothycarambat
e1dcd5ded0 Normalize pfp path to prevent traversal 2024-01-14 16:53:44 -08:00
Timothy Carambat
026849df02
normalize paths for submit URLs of `remove-documents (#598)
normalize paths for submit URLs
2024-01-14 16:36:17 -08:00
timothycarambat
e150e99e45 bump onboard ai link 2024-01-14 11:26:08 -08:00
Timothy Carambat
4f6d93159f
improve native embedder handling of large files (#584)
* improve native embedder handling of large files

* perf changes

* ignore storage tmp
2024-01-13 00:32:43 -08:00
timothycarambat
b4ab0a5e8c nopub sha 2024-01-12 14:22:19 -08:00
timothycarambat
e5826d8c24 remove unneeded build step 2024-01-12 13:33:11 -08:00
Ahren Stevens-Taylor
315b92e164
572 add docker tags (#581)
* [FEAT]: Docker Tags specific to a build version #572

* fix: dockerhub repo name

* feat: add Docker build caches

* fix: docker username

Fix the DockerHub repository owner name
2024-01-12 13:31:59 -08:00
Shuyoou
6faa0efaa8
Issue #543 support milvus vector db (#579)
* issue #543 support milvus vector db

* migrate Milvus to use MilvusClient instead of ORM
normalize env setup for docs/implementation
feat: embedder model dimension added

* update comments

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-01-12 13:23:57 -08:00
Timothy Carambat
b2b2c2afe1
protect AWS CF deployments by automatically blocking metadata URL (#578) 2024-01-11 18:37:00 -08:00
Timothy Carambat
1563a1b20f
Strict link protocol validation (#577) 2024-01-11 12:29:00 -08:00
Timothy Carambat
7200a06ef0
prevent manager in multi-user from updatingENV via HTTP (#576)
* prevent manager in multi-user from updatingENV via HTTP

* remove unneeded args
2024-01-11 12:11:45 -08:00
Timothy Carambat
3c859ba303
Change pwd check to O(1) check to prevent timing attacks - single user mode (#575)
Change pwd check to O(1) check to prevent timing attacks
2024-01-11 10:54:55 -08:00
Timothy Carambat
a4ace56a40
Patch minor XSS opportunity where user can self-XSS themselves. (#574)
Patch minor XSS opportunity where user can self-XSS themselvess. There is not real vuln here as any instance is not public facing
2024-01-11 09:57:59 -08:00
timothycarambat
4af9b9d5cc fix: relative/absolute url patch for vector count
connect #516
2024-01-10 19:27:39 -08:00
timothycarambat
dfd03e332c patch stream response 2024-01-10 15:32:07 -08:00
Timothy Carambat
4e2c0f04b4
Dynamic vector count on workspace settings (#567)
* Dynamic vector count on workspace settings
Add count to be workspace specific, fallback to system count
Update layout of data in settings
Update OpenAI per-token embedding price

* linting
2024-01-10 13:18:48 -08:00
Sean Hatfield
1d39b8a2ce
add Together AI LLM support (#560)
* add Together AI LLM support

* update readme to support together ai

* Patch togetherAI implementation

* add model sorting/option labels by organization for model selection

* linting + add data handling for TogetherAI

* change truthy statement
patch validLLMSelection method

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-01-10 12:35:30 -08:00
Timothy Carambat
8cd3a92c66
[BUG] Fixed mass_assignment vuln (#566)
Fixed mass_assignment vuln

Co-authored-by: dastaj <78434825+dastaj@users.noreply.github.com>
2024-01-10 08:42:03 -08:00
Timothy Carambat
259079ac58
561 relative api docs url (#564)
* patch swagger host to be relative

* change tag on feature request template
2024-01-09 21:52:50 -08:00
Timothy Carambat
755c10b8ca
[API] patch swagger host to be relative (#563)
patch swagger host to be relative
2024-01-09 19:49:51 -08:00
timothycarambat
5b2c0ca782 add OnboardAI link 2024-01-09 18:07:41 -08:00
timothycarambat
b8192883c2 fix auto-tag on bug issue yaml 2024-01-09 18:06:01 -08:00
timothycarambat
4801df08c5 remove broken config link 2024-01-09 14:31:28 -08:00
timothycarambat
6a4c99affe unquote email in config 2024-01-09 14:31:02 -08:00
timothycarambat
964d9a7137 update build ignore 2024-01-09 14:28:56 -08:00
Timothy Carambat
fd4a230669
Setup issue and PR templates (#559)
* Setup issue templates
Allow ability to include blank issue
resolves #557
todo: PR template

* update templates + add PR template

* newlines
2024-01-09 14:25:53 -08:00
Sean Hatfield
5c3bb4b8cc
532 uiux add slash command modal (#555)
* WIP slash commands

* add slash command image

* WIP slash commands

* slash command menu feature complete

* move icons to slash command local

* update how slash command component works

* relint with new linter

* Finalize slash command input
Change empty workspace text layout
Patch dev unmount issues on Chatworkspace/index.jsx

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-01-09 13:07:09 -08:00
Timothy Carambat
58971e8b30
Build & Publish AnythingLLM for ARM64 and x86 (#549)
* Update build process to support multi-platform builds
Bump @lancedb/vectordb to 0.1.19 for ARM&AMD compatibility
Patch puppeteer on ARM builds because of broken chromium
resolves #539
resolves #548

---------

Co-authored-by: shatfield4 <seanhatfield5@gmail.com>
2024-01-08 16:15:01 -08:00
Timothy Carambat
cce48c2163
[Automated Builds] Patch lockfiles with new devdeps (#553)
patch lockfiles with new devdeps
2024-01-08 15:48:03 -08:00
Francisco Bischoff
990a2e85bf
devcontainer v1 (#297)
Implement support for GitHub codespaces and VSCode devcontainers
---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
Co-authored-by: Sean Hatfield <seanhatfield5@gmail.com>
2024-01-08 15:31:06 -08:00
Sagar Jaglan
5172bceec3
#520 fixed unauthorized token error (#535) 2024-01-08 13:16:00 -08:00