Commit Graph

331 Commits

Author SHA1 Message Date
Timothy Carambat
be9d8b0397
Infinite prompt input and compression implementation (#332)
* WIP on continuous prompt window summary

* wip

* Move chat out of VDB
simplify chat interface
normalize LLM model interface
have compression abstraction
Cleanup compressor
TODO: Anthropic stuff

* Implement compression for Anythropic
Fix lancedb sources

* cleanup vectorDBs and check that lance, chroma, and pinecone are returning valid metadata sources

* Resolve Weaviate citation sources not working with schema

* comment cleanup
2023-11-06 13:13:53 -08:00
Sean Hatfield
eebafd7327
Replace old modals to match UI (#328)
* workspace not found modal updated to match ui

* update text preview modal to match new ui

* Mobile styles

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2023-11-02 16:17:02 -07:00
timothycarambat
c3abbfbf27 Fix admin chat pagination 2023-11-02 16:12:29 -07:00
Sean Hatfield
ab1e9d66b5
Use toast in workspace settings tab (#322)
* fix white border around user avatar

* add toast into settings menu on change/error
2023-11-01 18:21:30 -07:00
Sean Hatfield
1c5d55c425
Fix white border around user avatar glitch (#320)
fix white border around user avatar
2023-11-01 14:17:27 -07:00
Timothy Carambat
67c85f1550
Implement retrieval and use of fine-tune models (#314)
* Implement retrieval and use of fine-tune models
Cleanup LLM selection code
resolves #311

* Cleanup from PR bot
2023-10-31 11:38:28 -07:00
Timothy Carambat
5d56ab623b
Anthropic claude 2 support (#305)
* WIP Anythropic support for chat, chat and query w/context

* Add onboarding support for Anthropic

* cleanup

* fix Anthropic answer parsing
move embedding selector to general util
2023-10-30 15:44:03 -07:00
Sean Hatfield
27809b34b5
Added telemetry for onboarding completion (#295)
* added telemetry for onboarding completion

* minor changes

* linting and remove empty object

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2023-10-26 13:49:01 -07:00
Sean Hatfield
c8c0328346
Citations UI bug fix (#294)
citations spacing fix
2023-10-26 13:27:20 -07:00
Sean Hatfield
5f4d42edb8
Sidebar bug fix (#291)
truncate workspace name to prevent gear icon from being shown
2023-10-26 10:58:55 -07:00
Timothy Carambat
a8ec0d9584
Compensate for upper OpenAI emedding limit chunk size (#292)
Limit is due to POST body max size. Sufficiently large requests will abort automatically
We should report that error back on the frontend during embedding
Update vectordb providers to return on failed
2023-10-26 10:57:37 -07:00
Timothy Carambat
9f1a5d9dbc
Add back in selector on prompt input (#289)
resolves #286
2023-10-25 16:01:45 -07:00
Timothy Carambat
9f7b0c837a
Fix prompt overlaying on chats (#288)
resolves #287
2023-10-25 15:23:55 -07:00
timothycarambat
f6ae992c32 onboarding username password bug preventing multi user signup 2023-10-24 17:23:12 -07:00
timothycarambat
9b9644e931 remove typo 2023-10-23 14:07:04 -07:00
timothycarambat
81bb7bfc69 w-fit on workspace modal 2023-10-23 13:58:51 -07:00
Timothy Carambat
708068a09e
AnythingLLM UI overhaul (#278)
* v2 Login screen (#254)

* adding gradients for modal and sidebar

* adding font setup

* redesigned login screen for MultiUserAuth

* completed multi user mode login screen

* linting

* login screen for single user auth redesign complete

* created reusable gradient for login screen

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>

* v2 sidebar (#262)

* adding gradients for modal and sidebar

* adding font setup

* redesigned login screen for MultiUserAuth

* completed multi user mode login screen

* linting

* login screen for single user auth redesign complete

* WIP sidebar redesign

* created reusable gradient for login screen

* remove dark mode items

* update new workspace button

* completed sidebar for desktop view

* add interactivity states

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>

* remove duplicated pkg

* v2 settings (#264)

* adding gradients for modal and sidebar

* adding font setup

* redesigned login screen for MultiUserAuth

* completed multi user mode login screen

* linting

* login screen for single user auth redesign complete

* WIP sidebar redesign

* created reusable gradient for login screen

* remove dark mode items

* update new workspace button

* completed sidebar for desktop view

* WIP added colors/gradients to admin settings

* WIP fix discord logo import

* WIP settings redesign - added routes for general settings and restyled components

* WIP settings for LLM Preference, VectorDB, ExportImport

* settings menu UI complete WIP functionality

* settings fully functional/removed dark mode logo

* linting

* removing unneeded dependency

* Fix admin sidebar visibility
Fix API Keys location and work with single/mum
Fix Appearance location - WIP on funcitonality

* update api key page

* fix permissions for appearance

* Single user mode fixes

* fix multi user mode enabled

* fix import export

* Rename AdminSidebar to SettingsSidebar

* Fix mobile sidebar links

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>

* V2 user logout (#265)

* Add user logout button

* hide other 3 dot button

* wrap admin routes

* V2 workspace modal (#267)

Update new workspace modal
remove duplicate tailwind colors

* v2 Settings modal styles (#266)

* EditUserModal styles complete

* workspaces modals styles complete

* create invite link modal styles complete

* create new api key modal styles complete

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>

* v2 Chats Redesign (#270)

* fix default message for new workspace

* prompt input box ui redesign complete

* ui tweak to prompt input

* WIP chat msg redesign

* chat container and historical chat messages redesign

* manage workspace modal appears when clicking upload a document on empty workspace

* fixed loading skeleton styles

* citations redesign complete

* restyle pending chat and prompt reply components

* default chat messages styles updated

* linting

* update how chats are returned

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>

* Onboarding modal flow for first time setup (#274)

* WIP onboarding modal flow

* onboarding flow complete and private route redirection for onboarding setep

* redirect to home on onboarding complete

* add onboarding redirect using paths.onboarding()

* Apply changes to auth flow, onboarding determination, and flows

* remove formref

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>

* v2 document picker (#275)

* remove unneeded comments

* WIP document picker UI

* WIP basic UI complete for document picker tab and settings tab

* linting

* settings menu complete, document row WIP

* WIP document picker loading from localFiles

* WIP file picker logic

* refactoring document picker to work with backend

* WIP refactoring document picker

* WIP refactor document picker to work with backend

* file uploading with dropzone working

* WIP deleting file when not embedded

* WIP embeddings

* WIP embedding with temp button and hardcoded paths

* WIP placeholder for WorkspaceDirectory component

* WIP WorkspaceDirectory

* WIP

* sort workspaceDocs and availibleDocs complete

* added directories util

* add and remove document from ws working

* v2 document picker complete

* reference modal ui bug fixes

* truncate function bug fix

* ManageWorkspace modal bug fixes

* blocking mobile users modal for workspace settings

* mobile ui fixes

* linting

* ui padding fixes

* citation bug fixes

* code review changes

* debounce handlers

* change tempFile object to array

* selection count fix

* Convert workspace modal to div
Memo workspace settings
update conditional rendering of workspace settings

* Show no documents

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>

* mobile sidebar styles

* padding on Mobile view
mobile sidebar items

* UI touchup

* suggestion implementations

* CSS fixes and animation perfomance change to GPU accelerated and 60fps

* change will-change

* remove transitions from onboarding modals, simplify on-change handlers

* Swap onboarding to memoized components and debounce onchange handlers

* remove console log

* remove Avenir font

---------

Co-authored-by: Sean Hatfield <seanhatfield5@gmail.com>
2023-10-23 13:10:34 -07:00
timothycarambat
ac2a0b0a06 reduce available types of models 2023-10-10 11:15:11 -07:00
timothycarambat
827a29d074 improve AUTH_TIMESTAMP clearing 2023-10-05 15:16:14 -07:00
timothycarambat
f9eec86672 Fix login modal popping up on protected routes 2023-10-05 14:34:30 -07:00
Timothy Carambat
62d39eb4fb
resolves #259 (#260)
Support API client for chroma
2023-09-29 13:20:06 -07:00
timothycarambat
a8933e56fd change member to default 2023-09-29 12:56:42 -07:00
Sean Hatfield
a126b5f5aa
Replace custom sqlite dbms with prisma (#239)
* WIP converted all sqlite models into prisma calls

* modify db setup and fix ApiKey model calls in admin.js

* renaming function params to be consistent

* converted adminEndpoints to utilize prisma orm

* converted chatEndpoints to utilize prisma orm

* converted inviteEndpoints to utilize prisma orm

* converted systemEndpoints to utilize prisma orm

* converted workspaceEndpoints to utilize prisma orm

* converting sql queries to prisma calls

* fixed default param bug for orderBy and limit

* fixed typo for workspace chats

* fixed order of deletion to account for sql relations

* fix invite CRUD and workspace management CRUD

* fixed CRUD for api keys

* created prisma setup scripts/docs for understanding how to use prisma

* prisma dependency change

* removing unneeded console.logs

* removing unneeded sql escape function

* linting and creating migration script

* migration from depreciated sqlite script update

* removing unneeded migrations in prisma folder

* create backup of old sqlite db and use transactions to ensure all operations complete successfully

* adding migrations to gitignore

* updated PRISMA.md docs for info on how to use sqlite migration script

* comment changes

* adding back migrations folder to repo

* Reviewing SQL and prisma integraiton on fresh repo

* update inline key replacement

* ensure migration script executes and maps foreign_keys regardless of db ordering

* run migration endpoint

* support new prisma backend

* bump version

* change migration call

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2023-09-28 14:00:03 -07:00
Sean Hatfield
ce6951b21f
Renamed all indicies to vectors to avoid confusion of vocab (#246)
* renamed all indicies to vectors to avoid confusion of vocab

* removing unneeded files

* changed indicies on frontend sidebar to vectors
2023-09-21 12:04:17 -07:00
Timothy Carambat
a505928934
Display better error messages from document processor (#243)
pass messages to frontend on success/failure
resolves #242
2023-09-18 16:50:20 -07:00
Sean Hatfield
bb822a8ab3
Improve UI for login modal (#235)
* removed loading skeleton from password authentication modal background

* added logo to login modals

* change BG colors for dark/light mode support

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2023-09-13 13:44:36 -07:00
Sean Hatfield
1ae68ffb59
Adding messages to embedding updates (#229)
* WIP added toast for embedding updates

* Update and retain new change state

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2023-09-11 16:59:19 -07:00
Timothy Carambat
3c88aec034
prevent exports path traversal (#233) 2023-09-11 13:07:48 -07:00
Sean Hatfield
6ea645f358
Hide delete workspace button on non-admin users if setting is disabled (#227)
* hide delete workspace button on non-admin users if setting is disabled

---------

Co-authored-by: Timothy Carambat <timothycarambat@Timothys-MacBook-Pro.local>
2023-08-31 00:28:30 +02:00
timothycarambat
a3d6d876c4 check role for settings when in mum 2023-08-24 08:20:44 -07:00
Sean Hatfield
3274d5ab6b
Implemented toast messages for all system settings (#219)
* implemented toast for appearance settings changes

* linting

* implemented toast on system settings changes
2023-08-23 20:35:34 -07:00
Timothy Carambat
761500c42f
fix settings overlay hiding underneath prompt (#222) 2023-08-23 19:25:29 -07:00
Timothy Carambat
defe6054b3
Full developer api (#221)
* Autodocument Swagger API with JSDocs on /v1/ endpoints for API access
implement single-player API keys
WIP Admin API Keys

* Create new api keys as both single and multi-user

* Add boot and telem

* Complete Admin API

* Complete endpoints
dark mode swagger

* update docs

* undo debug

* update docs and readme
2023-08-23 19:15:07 -07:00
Sean Hatfield
bdf9529e80
Implement toast for upload event (#214)
* WIP success fail messages for upload document

* added success/error msgs for uploading feedback and disabled fileUploadProgress in backend

* implemented toast instead of success/error msg on modal for upload

* left over merge

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2023-08-22 21:14:24 -07:00
Sean Hatfield
c0adcc129d
Success fail messages for upload document (#208)
* WIP success fail messages for upload document

* added success/error msgs for uploading feedback and disabled fileUploadProgress in backend

* remove unused middleware

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2023-08-22 19:18:47 -07:00
Sean Hatfield
b01e49bb3c
Autoscroll to end of chat fix (#201)
bug fix for autoscrolling on message send/recieve
2023-08-21 15:46:31 -07:00
Timothy Carambat
7ae0d28ef0
improve UX and useability for Settings and prefernces for mobile and desktop (#200)
* improve UX and useability for Settings and prefernces for mobile and desktop

* linting
2023-08-18 14:36:58 -07:00
Sean Hatfield
31fbb0784b
Custom default messages implementation for single and multi-user modes (#193)
* added ui for custom welcome messages and added label for custom logo in admin settings

* linting

* fixing img to use light/dark modes

* converted ChatBubble into component

* implemented backend for welcome messages and admin appearance page

* completed custom welcome messages for admin

* finished custom messages for single user mode

* merged with master and linted

* improved UI for appearance settings pages

* linted and merged with master

* small updates

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2023-08-16 17:30:46 -07:00
timothycarambat
a62e891fa0 force check api online via JSON response 2023-08-16 15:43:46 -07:00
timothycarambat
2036204f23 default frontend api to relative URL 2023-08-16 12:26:52 -07:00
Timothy Carambat
cf0b24af02
Add Qdrant support for embedding, chat, and conversation (#192)
* Add Qdrant support for embedding, chat, and conversation

* Change comments
2023-08-15 15:26:44 -07:00
timothycarambat
9640eb7650 Fallback logo on offline mode 2023-08-14 17:46:02 -07:00
Sean Hatfield
b557bb9ede
Logo customization for single-user & multi-user modes (#186)
* implemented logo customization for single-user mode

* removing unneeded comments

* added dark and light mode support for default logo

* implemented dark and light mode switching in frontend

* fixed dark and light mode switching for failed to load logo from backend

* removed unneeded comment

* custom logos for admin implemented

* refactor logo mgmt functions
abstract logo management utils into their own file for simplicity

* added settings tab for appearance on single-user mode

* unchecking files with unneeded changes

* fixed appearance settings tab to be hidden on multiuser mode

* allow readall for logo

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2023-08-14 15:22:55 -07:00
timothycarambat
79b311085b update form 2023-08-14 12:18:27 -07:00
timothycarambat
5d1215f293 dont show managed hosting while managed already 2023-08-11 17:33:17 -07:00
timothycarambat
6e56c51a53 add feedback form, hosting link, update readme, show promo image 2023-08-11 17:28:30 -07:00
Timothy Carambat
f3a6147ffd
Add support for Weaviate VectorDB (#181) 2023-08-08 18:02:30 -07:00
Erich Focht
4ac5e55413
Actually enable x-scrolling when citations overflow. (#178) 2023-08-07 13:36:23 -07:00
Timothy Carambat
1f29cec918
Multiple LLM Support framework + AzureOpenAI Support (#180)
* Remove LangchainJS for chat support chaining
Implement runtime LLM selection
Implement AzureOpenAI Support for LLM + Emebedding
WIP on frontend
Update env to reflect the new fields

* Remove LangchainJS for chat support chaining
Implement runtime LLM selection
Implement AzureOpenAI Support for LLM + Emebedding
WIP on frontend
Update env to reflect the new fields

* Replace keys with LLM Selection in settings modal
Enforce checks for new ENVs depending on LLM selection
2023-08-04 14:56:27 -07:00
timothycarambat
285bddb0fb default setting modal to keys
remove duplicate fields for vectordb config
2023-08-04 12:40:36 -07:00
Timothy Carambat
6e8d81c01e
better vector db selection UI on settings (#175)
update logout button
2023-08-03 15:59:51 -07:00
Timothy Carambat
0a2f837fb2
improve citations to show all text chunks referred and expand the citation to view full referenced text (#161)
* improve citations to show all text chunks referred and expand the citation to view full referenced text
chunk text of same document together

* remove debug
2023-07-27 22:33:27 -07:00
Timothy Carambat
91f5f94200
[FEATURE] Enable the ability to have multi user instances (#158)
* multi user wip

* WIP MUM features

* invitation mgmt

* suspend or unsuspend users

* workspace mangement

* manage chats

* manage chats

* add Support for admin system settings for users to delete workspaces and limit chats per user

* fix issue ith system var
update app to lazy load invite page

* cleanup and bug fixes

* wrong method

* update readme

* update readme

* update readme

* bump version to 0.1.0
2023-07-25 10:37:04 -07:00
timothycarambat
2d30b7e24f add discord invite link to UI 2023-07-21 12:05:39 -07:00
Timothy Carambat
6b6007f9ad
Enable the system owner to be able to update the system wide password and secret (#156)
* Enable the system owner to be able to update the system wide password and secret

* lint and cleanup
2023-07-20 15:25:47 -07:00
Timothy Carambat
5fa6145872
can now count and remove data in lancedb 0.1.12 so bumped version and added new functionality support (#155) 2023-07-20 13:09:56 -07:00
Timothy Carambat
5a7d8add6f
[Fork] Additions on franzbischoff resolution on #122 (#152)
* Related to Issue #122, Implemented custom prompt in workspace settings.

* run linter

* Remove code duplication for chat prompt injection

---------

Co-authored-by: Francisco Bischoff <franzbischoff@gmail.com>
2023-07-20 11:14:23 -07:00
Timothy Carambat
6d651db6d3
Enabled importing and exporting of entire AnythingLLM instance data (#146)
* WIP on imports

* undo nodismiss for testing

* remove old keys modal screen
add helper text on import complete

* return default setting modal to keys
2023-07-14 17:32:30 -07:00
timothycarambat
5b2c5cac08 make settings modal have tabs for future expansions 2023-07-14 15:04:25 -07:00
timothycarambat
032c9d27b6 UI updates and linting 2023-06-28 12:54:18 -07:00
AntonioCiolino
98d7266291
History per workspace (#126)
* Enable saving history value per workspace

* added history

* added history

* Changes re: the PR for message limits from history
2023-06-28 12:53:26 -07:00
Timothy Carambat
82d132427c
dump error message to frontend instead of generic error (#114) 2023-06-26 17:54:55 -07:00
Timothy Carambat
60a00843df
add ability to purge document from custom documents as well as cleanup its associated cache file (#113)
* add ability to purge document from custom documents as well as cleanup its assoicated cache file

* update alert text
2023-06-26 17:20:09 -07:00
Timothy Carambat
9d0becb2ee
Add chat/converstaion mode as the default chat mode for all Vector Databases (#112)
* Add chat/converstaion mode as the default chat mode
Show menu for toggling options for chat/query/reset command
Show chat status below input
resolves #61

* remove console logs
2023-06-26 15:08:47 -07:00
Timothy Carambat
a3f5a936e2
Enable debug mode of ENV at runtime (#111)
* Enable debug mode of ENV at runtime
Update Storage README for those with SQLite issues

* add files
2023-06-26 11:38:38 -07:00
Timothy Carambat
3945a77290
Add mobile stylings to UI (#75)
* initial chat pages mobile stylings

* fix sidebar

* manage workspace modal styles

* mobile styles v1
2023-06-16 23:50:56 -07:00
Timothy Carambat
c4eb46ca19
Upload and process documents via UI + document processor in docker image (#65)
* implement dnd uploader
show file upload progress
write files to hotdirector
build simple flaskAPI to process files one off

* move document processor calls to util
build out dockerfile to run both procs at the same time
update UI to check for document processor before upload
* disable pragma update on boot
* dockerfile changes

* add filetype restrictions based on python app support response and show rejected files in the UI

* cleanup

* stub migrations on boot to prevent exit condition

* update CF template for AWS deploy
2023-06-16 16:01:27 -07:00
Timothy Carambat
2a556c275c
Implement Workspace-specific setting configs + other technical features (#58)
* 1. Define LLM Temperature as a workspace setting
2. Implement rudimentry table migration code for both new and existing repos to bring tables up to date
3. Trigger for workspace on update to update timestamp
4. Always fallback temp to 0.7
5. Extract WorkspaceModal into Tabbed content
6. Remove workspace name UNIQUE constraint (cannot be migrated :()
7. Add slug +seed when existing slug is already take
8. Seperate name from slug so display names can be changed

* remove blocking test return
2023-06-14 23:12:59 -07:00
Timothy Carambat
2403806949
add codeblock support for prompt replies and historical messages (#55)
* add codeblock support for prompt replies and historical messages
add markdown-it

* Fix spacing for HTML rendering
2023-06-14 13:35:55 -07:00
frasergr
9f33b3dfcb
Docker support (#34)
* Updates for Linux for frontend/server

* frontend/server docker

* updated Dockerfile for deps related to node vectordb

* updates for collector in docker

* docker deps for ODT processing

* ignore another collector dir

* storage mount improvements; run as UID

* fix pypandoc version typo

* permissions fixes
2023-06-13 11:26:11 -07:00
Tim Asp
6eff067c98
Fix small react and JS errors (#40)
* Fix react html console errors

* Fix bug where clicking expand on empty directory throws error
2023-06-12 21:46:36 -07:00
Timothy Carambat
f9015df903
fix input color (#17) 2023-06-09 16:21:55 -07:00
Timothy Carambat
fa2b669940
resolves #14 (#15) 2023-06-09 12:59:22 -07:00
Timothy Carambat
62e3f62e82
12 auth implementation (#13)
* Add Auth protection for cloud-based or private instances

* skip check on local dev
2023-06-09 11:27:27 -07:00
timothycarambat
f0fd91db6f Reorg some files for clarity 2023-06-08 18:58:26 -07:00
Timothy Carambat
ad15e1f9b6
Lancedb support (#6)
* add start of lanceDB support

* lancedb initial support

* add null method for deletion of documents from namespace since LanceDB does not support
show warning modal on frontend for this

* update .env.example and lancedb methods for sourcing

* change export method

* update readme
2023-06-08 18:40:29 -07:00
timothycarambat
93ee4349ba manage default CTAs 2023-06-08 14:26:29 -07:00
timothycarambat
fc0431fca3 add endpoint saftey to prevent server dying in production 2023-06-08 13:13:48 -07:00
timothycarambat
abdaf4e63d add ENV var for api endpoint for frontend 2023-06-08 12:49:33 -07:00
Timothy Carambat
6d01970df2
Implement Chroma Support (#1) 2023-06-07 21:31:35 -07:00
timothycarambat
27c58541bd inital commit 2023-06-03 19:28:07 -07:00