Commit Graph

442 Commits

Author SHA1 Message Date
Timothy Carambat
f3a6147ffd
Add support for Weaviate VectorDB (#181) 2023-08-08 18:02:30 -07:00
timothycarambat
970d38ccb5 remove console 2023-08-04 20:08:25 -07:00
Timothy Carambat
1f29cec918
Multiple LLM Support framework + AzureOpenAI Support (#180)
* Remove LangchainJS for chat support chaining
Implement runtime LLM selection
Implement AzureOpenAI Support for LLM + Emebedding
WIP on frontend
Update env to reflect the new fields

* Remove LangchainJS for chat support chaining
Implement runtime LLM selection
Implement AzureOpenAI Support for LLM + Emebedding
WIP on frontend
Update env to reflect the new fields

* Replace keys with LLM Selection in settings modal
Enforce checks for new ENVs depending on LLM selection
2023-08-04 14:56:27 -07:00
Timothy Carambat
6e8d81c01e
better vector db selection UI on settings (#175)
update logout button
2023-08-03 15:59:51 -07:00
timothycarambat
9bea7739ed move OpenAI to AiProvider folder in preparation for new AI provider support 2023-07-28 12:09:49 -07:00
Timothy Carambat
8929d96ed0
Move OpenAI api calls into its own interface/Class (#162)
* Move OpenAI api calls into its own interface/Class
move curate sources to be specific for each vectorDBs response for chat/query

* remove comment
2023-07-28 12:05:38 -07:00
Timothy Carambat
0a2f837fb2
improve citations to show all text chunks referred and expand the citation to view full referenced text (#161)
* improve citations to show all text chunks referred and expand the citation to view full referenced text
chunk text of same document together

* remove debug
2023-07-27 22:33:27 -07:00
timothycarambat
24ca64ce46 improve bulkd write speeds 2023-07-26 18:06:53 -07:00
Timothy Carambat
91f5f94200
[FEATURE] Enable the ability to have multi user instances (#158)
* multi user wip

* WIP MUM features

* invitation mgmt

* suspend or unsuspend users

* workspace mangement

* manage chats

* manage chats

* add Support for admin system settings for users to delete workspaces and limit chats per user

* fix issue ith system var
update app to lazy load invite page

* cleanup and bug fixes

* wrong method

* update readme

* update readme

* update readme

* bump version to 0.1.0
2023-07-25 10:37:04 -07:00
Timothy Carambat
6b6007f9ad
Enable the system owner to be able to update the system wide password and secret (#156)
* Enable the system owner to be able to update the system wide password and secret

* lint and cleanup
2023-07-20 15:25:47 -07:00
Timothy Carambat
5fa6145872
can now count and remove data in lancedb 0.1.12 so bumped version and added new functionality support (#155) 2023-07-20 13:09:56 -07:00
Timothy Carambat
c1deca4928
[Fork] Batch embed by jwaltz (#153)
* refactor: convert chunk embedding to one API call

* chore: lint

* fix chroma for batch and single vectorization of text

* Fix LanceDB multi and single vectorization

* Fix pinecone for single and multiple embeddings

---------

Co-authored-by: Jonathan Waltz <volcanicislander@gmail.com>
2023-07-20 12:05:23 -07:00
Timothy Carambat
5a7d8add6f
[Fork] Additions on franzbischoff resolution on #122 (#152)
* Related to Issue #122, Implemented custom prompt in workspace settings.

* run linter

* Remove code duplication for chat prompt injection

---------

Co-authored-by: Francisco Bischoff <franzbischoff@gmail.com>
2023-07-20 11:14:23 -07:00
Timothy Carambat
6d651db6d3
Enabled importing and exporting of entire AnythingLLM instance data (#146)
* WIP on imports

* undo nodismiss for testing

* remove old keys modal screen
add helper text on import complete

* return default setting modal to keys
2023-07-14 17:32:30 -07:00
timothycarambat
032c9d27b6 UI updates and linting 2023-06-28 12:54:18 -07:00
AntonioCiolino
98d7266291
History per workspace (#126)
* Enable saving history value per workspace

* added history

* added history

* Changes re: the PR for message limits from history
2023-06-28 12:53:26 -07:00
Timothy Carambat
82d132427c
dump error message to frontend instead of generic error (#114) 2023-06-26 17:54:55 -07:00
Timothy Carambat
60a00843df
add ability to purge document from custom documents as well as cleanup its associated cache file (#113)
* add ability to purge document from custom documents as well as cleanup its assoicated cache file

* update alert text
2023-06-26 17:20:09 -07:00
Timothy Carambat
9d0becb2ee
Add chat/converstaion mode as the default chat mode for all Vector Databases (#112)
* Add chat/converstaion mode as the default chat mode
Show menu for toggling options for chat/query/reset command
Show chat status below input
resolves #61

* remove console logs
2023-06-26 15:08:47 -07:00
Timothy Carambat
a3f5a936e2
Enable debug mode of ENV at runtime (#111)
* Enable debug mode of ENV at runtime
Update Storage README for those with SQLite issues

* add files
2023-06-26 11:38:38 -07:00
Timothy Carambat
c4eb46ca19
Upload and process documents via UI + document processor in docker image (#65)
* implement dnd uploader
show file upload progress
write files to hotdirector
build simple flaskAPI to process files one off

* move document processor calls to util
build out dockerfile to run both procs at the same time
update UI to check for document processor before upload
* disable pragma update on boot
* dockerfile changes

* add filetype restrictions based on python app support response and show rejected files in the UI

* cleanup

* stub migrations on boot to prevent exit condition

* update CF template for AWS deploy
2023-06-16 16:01:27 -07:00
Timothy Carambat
cf9bb99701
resolves #46 (#66)
ignore storage folder changes for nodemon
2023-06-16 01:29:26 -07:00
Timothy Carambat
2a556c275c
Implement Workspace-specific setting configs + other technical features (#58)
* 1. Define LLM Temperature as a workspace setting
2. Implement rudimentry table migration code for both new and existing repos to bring tables up to date
3. Trigger for workspace on update to update timestamp
4. Always fallback temp to 0.7
5. Extract WorkspaceModal into Tabbed content
6. Remove workspace name UNIQUE constraint (cannot be migrated :()
7. Add slug +seed when existing slug is already take
8. Seperate name from slug so display names can be changed

* remove blocking test return
2023-06-14 23:12:59 -07:00
Timothy Carambat
2403806949
add codeblock support for prompt replies and historical messages (#55)
* add codeblock support for prompt replies and historical messages
add markdown-it

* Fix spacing for HTML rendering
2023-06-14 13:35:55 -07:00
timothycarambat
ca8e8245fc remove lancedb logging 2023-06-14 00:27:19 -07:00
Jonathan Waltz
84a5707d6d
refactor: convert insert loop to 1 insert stmt (#19)
* fix: convert insert loop to 1 insert stmt

* chore: lint
2023-06-13 14:07:58 -07:00
timothycarambat
e114a90a17 update package scripts
update ignore for legacy repo copies
add docker instructions and use
2023-06-13 13:19:17 -07:00
frasergr
9f33b3dfcb
Docker support (#34)
* Updates for Linux for frontend/server

* frontend/server docker

* updated Dockerfile for deps related to node vectordb

* updates for collector in docker

* docker deps for ODT processing

* ignore another collector dir

* storage mount improvements; run as UID

* fix pypandoc version typo

* permissions fixes
2023-06-13 11:26:11 -07:00
timothycarambat
ef785af3f0 remove unused dep 2023-06-12 10:37:07 -07:00
Timothy Carambat
fa2b669940
resolves #14 (#15) 2023-06-09 12:59:22 -07:00
timothycarambat
e81dde4b9c add Pinecone Readme 2023-06-09 11:33:13 -07:00
Timothy Carambat
62e3f62e82
12 auth implementation (#13)
* Add Auth protection for cloud-based or private instances

* skip check on local dev
2023-06-09 11:27:27 -07:00
Timothy Carambat
914d4b1ff1
Update package.jsons to reflect new project name (#11) 2023-06-09 09:30:42 -07:00
timothycarambat
8947e63042 remove org id 2023-06-09 08:58:07 -07:00
timothycarambat
f0fd91db6f Reorg some files for clarity 2023-06-08 18:58:26 -07:00
Timothy Carambat
ad15e1f9b6
Lancedb support (#6)
* add start of lanceDB support

* lancedb initial support

* add null method for deletion of documents from namespace since LanceDB does not support
show warning modal on frontend for this

* update .env.example and lancedb methods for sourcing

* change export method

* update readme
2023-06-08 18:40:29 -07:00
timothycarambat
93ee4349ba manage default CTAs 2023-06-08 14:26:29 -07:00
timothycarambat
d5951fd5a2 change path for file storage for cloud-base persistance 2023-06-08 14:00:43 -07:00
timothycarambat
fc0431fca3 add endpoint saftey to prevent server dying in production 2023-06-08 13:13:48 -07:00
timothycarambat
4aa207cbe8 bump node version requirement
add conditional ENV setting
2023-06-08 10:29:17 -07:00
Timothy Carambat
6d01970df2
Implement Chroma Support (#1) 2023-06-07 21:31:35 -07:00
timothycarambat
27c58541bd inital commit 2023-06-03 19:28:07 -07:00