Commit Graph

52 Commits

Author SHA1 Message Date
Sean Hatfield
348b36bf85
[FEAT] Confluence data connector (#1181)
* WIP Confluence data connector backend

* confluence data connector complete

* confluence citations

* fix citation for confluence

* Patch confulence integration

* fix Citation Icon for confluence

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-04-25 17:53:38 -07:00
Sean Hatfield
11f6419c3c
[FEAT] Implement new login screen UI & multi-user password reset (#1074)
* WIP new login screen UI

* update prisma schema/create new models for pw recovery

* WIP password recovery backend

* WIP reset password flow

* WIP pw reset flow

* password reset logic complete & functional UI

* WIP login screen redesign for single and multi user

* create placeholder modal to display recovery codes

* implement UI for recovery code modals/download recovery codes

* multiuser desktop password reset UI/functionality complete

* support single user mode for pw reset

* mobile styles for all password reset/login flows complete

* lint

* remove single user password recovery

* create PasswordRecovery util file to make more readable

* do not drop-replace users table in migration

* review pr

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-04-25 16:52:30 -07:00
Timothy Carambat
a8e731f548
Add ability for owner to reset just the workspace vector DB (#1094)
* Add ability for owner to reset just the workspace vector DB

* explicit border none
2024-04-12 14:50:39 -07:00
Sean Hatfield
41fe20f2e0
[FEAT] Implement new workspace members settings and admin users UI updates (#990)
* members workspace settings menu and admin users UI updates

* change copy/fix admin and managers in workspace when not showing in UI

* remove existing workspace user mgmt modal

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-04-02 14:53:35 -07:00
Timothy Carambat
1ecefe8bed
Add ability to add invitee to workspaces automatically (#975) 2024-03-26 16:38:32 -07:00
Sean Hatfield
4f268dfeb6
[FEAT] Document picker folders for organization (#902)
* implement alternating color rows for file picker

* implement alternating colors for workspace directory

* remove unneeded props/variables

* remove unused border classes

* WIP new folder UI

* remove unneeded expanded prop from filerow component

* folder creation UI and files object manipulation WIP

* folder creation & moving files complete

* add developer API support for creating folders and moving files

* update alternating row css for file picker

* remove unneeded props from merge

* normalize paths for folders
priority to custom docs folder
silently fail on duplicate folders

* update folder icon to custom svg

* linting and move FolderIcon to JSX

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-03-20 16:10:30 -07:00
Timothy Carambat
0e46a11cb6
Stop generation button during stream-response (#892)
* Stop generation button during stream-response

* add custom stop icon

* add stop to thread chats
2024-03-12 15:21:27 -07:00
Sean Hatfield
04399b1328
[FEAT] Ability to set workspace profile image (#847)
* WIP workspace pfp, CRUD functions complete

* implement fetching workspace pfp in UserIcon component

* update UI for workspace settings pfp

* minor css refactor

* WIP fixes to workspace pfp

* create responseCache for workspace pfp blob to improve performance

* fix cache not clearing when removing workspace pfp and remove unneeded util

* load workspace image once, dont reload

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-03-12 11:36:59 -07:00
Timothy Carambat
bbe3c0a5d6
Bulk remove files from file picker (#830)
Do bulk deletion on backend
2024-02-27 11:53:42 -08:00
Timothy Carambat
791c0ee9dc
Enable ability to do full-text query on documents (#758)
* Enable ability to do full-text query on documents
Show alert modal on first pin for client
Add ability to use pins in stream/chat/embed

* typo and copy update

* simplify spread of context and sources
2024-02-21 13:15:45 -08:00
Sean Hatfield
17c1913ccc
[FEAT]: Allow user to set support email (#726)
* implement custom support email for usermenu support button

* small refactor

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-02-19 10:30:41 -08:00
Timothy Carambat
51dbff0dcb
Breakout Chat/Query mode as a workspace setting (#734)
Remove useless icons in prompt bar
Add chatMode column to workspaces that defaults to chat
Add UI for toggle of chat mode with hint
Update UI for workspace settings to match designs
2024-02-16 14:50:40 -08:00
Sean Hatfield
f4b09a8c79
[FEAT] RLHF on response messages (#708)
* WIP RLHF works on historical messages

* refactor Actions component

* completed RLHF up and down votes for chats

* add defaults for HistoricalMessage params

* refactor RLHF implmenation
remove forwardRef on history items to prevent rerenders

* remove dup id

* Add rating to CSV output

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-02-13 11:33:05 -08:00
Sean Hatfield
1b29882c71
[FEAT] Improved CSV chat exports (#700)
* add more fields to csv export to make more useful

* refactor from review comments

* fix escapeCsv function

* catch export errors properly

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-02-13 10:12:59 -08:00
Timothy Carambat
406732830f
Implement workspace threading that is backwards compatible (#699)
* Implement workspace thread that is compatible with legacy versions

* last touches

* comment on chat qty enforcement
2024-02-08 18:37:22 -08:00
Sean Hatfield
b985524901
[FEAT] Customizable footer icon links in Appearance Settings (#694)
* WIP custom footer icons

* UI for updating footer icons complete and backend to save/modify

* add backend for unprotected footer fetch

* break out footer into separate component and render footer items using a cache for 1 hour

* wip review

* refactor & cleanup

* Optimize footer form component
Optimize caching for footer icons
Add validation on SystemSetting upserts
Normalize fallback items for footer_data

* Adjust max icons to 3

* fix success message on remove

* fix success message on remove

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-02-08 12:17:01 -08:00
Sean Hatfield
d789920a19
[FEAT] Automated audit logging (#667)
* WIP event logging - new table for events and new settings view for viewing

* WIP add logging

* UI for log rows

* rename files to Logging to prevent getting gitignore

* add metadata for all logging events and colored badges in logs page

* remove unneeded comment

* cleanup namespace for logging

* clean up backend calls

* update logging to show to => from settings changes

* add logging for invitations, created, deleted, and accepted

* add logging for user created, updated, suspended, or removed

* add logging for workspace deleted

* add logging for chat logs exported

* add logging for API keys, LLM, embedder, vector db, embed chat, and reset button

* modify event logs

* update to event log types

* simplify rendering of event badges

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-02-06 15:21:40 -08:00
Sean Hatfield
608f28d745
[FEAT] create custom prompt suggestions per workspace (#664)
* create custom suggested chat messages per workspace

* update how suggestedChats are passed to chat window

* update mobile styles

* update edit change handler

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-02-06 11:24:33 -08:00
Sean Hatfield
1846a99b93
[FEAT] Embedded AnythingLLM (#656)
* WIP embedded app

* WIP got response from backend in embedded app

* WIP streaming prints to embedded app

* implemented streaming and tailwind min for styling into embedded app

* WIP embedded app history functional

* load params from script tag into embedded app

* rough in modularization of embed chat
cleanup dev process for easier dev support
move all chat to components
todo: build process
todo: backend support

* remove eslint config

* Implement models and cleanup embed chat endpoints
Improve build process for embed
prod minification and bundle size awareness
WIP

* forgot files

* rename to embed folder

* introduce chat modal styles

* add middleware validations on embed chat

* auto open param and default greeting

* reset chat history

* Admin embed config page

* Admin Embed Chats mgmt page

* update embed

* nonpriv

* more style support
reopen if chat was last opened

* update comments

* remove unused imports

* allow change of workspace for embedconfig

* update failure to lookup message

* update reset script

* update instructions

* Add more styling options
Add sponsor text at bottom
Support dynamic container height
Loading animations

* publish new embed script

* Add back syntax highlighting and keep bundle small via dynamic script build

* add hint

* update readme

* update copy model for snippet with link to styles

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-02-05 14:21:34 -08:00
Timothy Carambat
9a237db3d1
Implement total permission overhaul (#629)
* Implement total permission overhaul
Add explicit permissions on each flex and strict route
Patch issues with role escalation and CRUD of users
Patch permissions on all routes for coverage
Improve middleware to accept role array for clarity

* update comments

* remove permissions to API-keys for manager. Manager could generate API-key and using high-privelege api-key give themselves admin

* update sidebar permissions for multi-user and single user

* update options for mobile sidebar
2024-01-22 14:14:01 -08:00
Sean Hatfield
7fb76cfef0
add support for exporting to json and csv in workspace chats (#622)
* add support for exporting to json and csv in workspace chats

* safety encode URL options

* remove message about openai fine tuning on export success

* all defaults to jsonl
2024-01-18 17:59:51 -08:00
Sean Hatfield
08d33cfd8f
Merge pull request from GHSA-xmj6-g32r-fc5q
* remove support for import export anythingllm data

* remove unused imports
remove unused dep
update lockfile

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-01-18 13:57:23 -08:00
Timothy Carambat
bd158ce7b1
[Feat] Query mode to return no-result when no context found (#601)
* Query mode to return no-result when no context found

* update default error for sync chat

* remove unnecessary type conversion
2024-01-16 09:32:51 -08:00
timothycarambat
4af9b9d5cc fix: relative/absolute url patch for vector count
connect #516
2024-01-10 19:27:39 -08:00
Timothy Carambat
4e2c0f04b4
Dynamic vector count on workspace settings (#567)
* Dynamic vector count on workspace settings
Add count to be workspace specific, fallback to system count
Update layout of data in settings
Update OpenAI per-token embedding price

* linting
2024-01-10 13:18:48 -08:00
Timothy Carambat
ecf4295537
Add ability to grab youtube transcripts via doc processor (#470)
* Add ability to grab youtube transcripts via doc processor

* dynamic imports
swap out Github for Youtube in placeholder text
2023-12-18 17:17:26 -08:00
Timothy Carambat
452582489e
GitHub loader extension + extension support v1 (#469)
* feat: implement github repo loading
fix: purge of folders
fix: rendering of sub-files

* noshow delete on custom-documents

* Add API key support because of rate limits

* WIP for frontend of data connectors

* wip

* Add frontend form for GitHub repo data connector

* remove console.logs
block custom-documents from being deleted

* remove _meta unused arg

* Add support for ignore pathing in request
Ignore path input via tagging

* Update hint
2023-12-18 15:48:02 -08:00
Sean Hatfield
fcb591d364
Add user PFP support and context to logo (#408)
* fix sizing of onboarding modals & lint

* fix extra scrolling on mobile onboarding flow

* added message to use desktop for onboarding

* linting

* add arrow to scroll to bottom (debounced) and fix chat scrolling to always scroll to very bottom on message history change

* fix for empty chat

* change mobile alert copy

* WIP adding PFP upload support

* WIP pfp for users

* edit account menu complete with change username/password and upload profile picture

* add pfp context to update all instances of usePfp hook on update

* linting

* add context for logo change to immediately update logo

* fix div with bullet points to use list-disc instead

* fix: small changes

* update multer file storage locations

* fix: use STORAGE_DIR for filepathing

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2023-12-07 14:11:51 -08:00
Timothy Carambat
f48e6b1a3e
chore: add @ as alias for frontend root (#414)
* chore: add @ as alias for frontend root

* fix: remove bad tag
2023-12-07 09:09:01 -08:00
timothycarambat
ef79649891 improve ms-sse-fetch to not re-fetch automatially 2023-12-06 12:04:10 -08:00
Sean Hatfield
7edfccaf9a
Adding url uploads to document picker (#375)
* WIP adding url uploads to document picker

* fix manual script for uploading url to custom-documents

* fix metadata for url scraping

* wip url parsing

* update how async link scraping works

* docker-compose defaults added
no autocomplete on URLs

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2023-11-16 17:15:01 -08:00
Timothy Carambat
4bb99ab4bf
Support LocalAi as LLM provider by @tlandenberger (#373)
* feature: add LocalAI as llm provider

* update Onboarding/mgmt settings
Grab models from models endpoint for localai
merge with master

* update streaming for complete chunk streaming
update localAI LLM to be able to stream

* force schema on URL

---------
Co-authored-by: timothycarambat <rambat1010@gmail.com>
Co-authored-by: tlandenberger <tobiaslandenberger@gmail.com>
2023-11-14 12:31:44 -08:00
Timothy Carambat
c22c50cca8
Enable chat streaming for LLMs (#354)
* [Draft] Enable chat streaming for LLMs

* stream only, move sendChat to deprecated

* Update TODO deprecation comments
update console output color for streaming disabled
2023-11-13 15:07:30 -08:00
Sean Hatfield
997482ef8f
added JSONL export to workspace chats (#345)
* added JSONL export to workspace chats

* change permissions for workspace chat settings

* change permissions for workspace chat settings

* Show error for correct limit on fine-tune
Change sidebar position and permission
Remove check for MUM

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2023-11-08 17:36:54 -08:00
Timothy Carambat
67c85f1550
Implement retrieval and use of fine-tune models (#314)
* Implement retrieval and use of fine-tune models
Cleanup LLM selection code
resolves #311

* Cleanup from PR bot
2023-10-31 11:38:28 -07:00
Timothy Carambat
708068a09e
AnythingLLM UI overhaul (#278)
* v2 Login screen (#254)

* adding gradients for modal and sidebar

* adding font setup

* redesigned login screen for MultiUserAuth

* completed multi user mode login screen

* linting

* login screen for single user auth redesign complete

* created reusable gradient for login screen

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>

* v2 sidebar (#262)

* adding gradients for modal and sidebar

* adding font setup

* redesigned login screen for MultiUserAuth

* completed multi user mode login screen

* linting

* login screen for single user auth redesign complete

* WIP sidebar redesign

* created reusable gradient for login screen

* remove dark mode items

* update new workspace button

* completed sidebar for desktop view

* add interactivity states

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>

* remove duplicated pkg

* v2 settings (#264)

* adding gradients for modal and sidebar

* adding font setup

* redesigned login screen for MultiUserAuth

* completed multi user mode login screen

* linting

* login screen for single user auth redesign complete

* WIP sidebar redesign

* created reusable gradient for login screen

* remove dark mode items

* update new workspace button

* completed sidebar for desktop view

* WIP added colors/gradients to admin settings

* WIP fix discord logo import

* WIP settings redesign - added routes for general settings and restyled components

* WIP settings for LLM Preference, VectorDB, ExportImport

* settings menu UI complete WIP functionality

* settings fully functional/removed dark mode logo

* linting

* removing unneeded dependency

* Fix admin sidebar visibility
Fix API Keys location and work with single/mum
Fix Appearance location - WIP on funcitonality

* update api key page

* fix permissions for appearance

* Single user mode fixes

* fix multi user mode enabled

* fix import export

* Rename AdminSidebar to SettingsSidebar

* Fix mobile sidebar links

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>

* V2 user logout (#265)

* Add user logout button

* hide other 3 dot button

* wrap admin routes

* V2 workspace modal (#267)

Update new workspace modal
remove duplicate tailwind colors

* v2 Settings modal styles (#266)

* EditUserModal styles complete

* workspaces modals styles complete

* create invite link modal styles complete

* create new api key modal styles complete

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>

* v2 Chats Redesign (#270)

* fix default message for new workspace

* prompt input box ui redesign complete

* ui tweak to prompt input

* WIP chat msg redesign

* chat container and historical chat messages redesign

* manage workspace modal appears when clicking upload a document on empty workspace

* fixed loading skeleton styles

* citations redesign complete

* restyle pending chat and prompt reply components

* default chat messages styles updated

* linting

* update how chats are returned

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>

* Onboarding modal flow for first time setup (#274)

* WIP onboarding modal flow

* onboarding flow complete and private route redirection for onboarding setep

* redirect to home on onboarding complete

* add onboarding redirect using paths.onboarding()

* Apply changes to auth flow, onboarding determination, and flows

* remove formref

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>

* v2 document picker (#275)

* remove unneeded comments

* WIP document picker UI

* WIP basic UI complete for document picker tab and settings tab

* linting

* settings menu complete, document row WIP

* WIP document picker loading from localFiles

* WIP file picker logic

* refactoring document picker to work with backend

* WIP refactoring document picker

* WIP refactor document picker to work with backend

* file uploading with dropzone working

* WIP deleting file when not embedded

* WIP embeddings

* WIP embedding with temp button and hardcoded paths

* WIP placeholder for WorkspaceDirectory component

* WIP WorkspaceDirectory

* WIP

* sort workspaceDocs and availibleDocs complete

* added directories util

* add and remove document from ws working

* v2 document picker complete

* reference modal ui bug fixes

* truncate function bug fix

* ManageWorkspace modal bug fixes

* blocking mobile users modal for workspace settings

* mobile ui fixes

* linting

* ui padding fixes

* citation bug fixes

* code review changes

* debounce handlers

* change tempFile object to array

* selection count fix

* Convert workspace modal to div
Memo workspace settings
update conditional rendering of workspace settings

* Show no documents

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>

* mobile sidebar styles

* padding on Mobile view
mobile sidebar items

* UI touchup

* suggestion implementations

* CSS fixes and animation perfomance change to GPU accelerated and 60fps

* change will-change

* remove transitions from onboarding modals, simplify on-change handlers

* Swap onboarding to memoized components and debounce onchange handlers

* remove console log

* remove Avenir font

---------

Co-authored-by: Sean Hatfield <seanhatfield5@gmail.com>
2023-10-23 13:10:34 -07:00
timothycarambat
f9eec86672 Fix login modal popping up on protected routes 2023-10-05 14:34:30 -07:00
Sean Hatfield
6ea645f358
Hide delete workspace button on non-admin users if setting is disabled (#227)
* hide delete workspace button on non-admin users if setting is disabled

---------

Co-authored-by: Timothy Carambat <timothycarambat@Timothys-MacBook-Pro.local>
2023-08-31 00:28:30 +02:00
Timothy Carambat
defe6054b3
Full developer api (#221)
* Autodocument Swagger API with JSDocs on /v1/ endpoints for API access
implement single-player API keys
WIP Admin API Keys

* Create new api keys as both single and multi-user

* Add boot and telem

* Complete Admin API

* Complete endpoints
dark mode swagger

* update docs

* undo debug

* update docs and readme
2023-08-23 19:15:07 -07:00
Sean Hatfield
c0adcc129d
Success fail messages for upload document (#208)
* WIP success fail messages for upload document

* added success/error msgs for uploading feedback and disabled fileUploadProgress in backend

* remove unused middleware

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2023-08-22 19:18:47 -07:00
Sean Hatfield
31fbb0784b
Custom default messages implementation for single and multi-user modes (#193)
* added ui for custom welcome messages and added label for custom logo in admin settings

* linting

* fixing img to use light/dark modes

* converted ChatBubble into component

* implemented backend for welcome messages and admin appearance page

* completed custom welcome messages for admin

* finished custom messages for single user mode

* merged with master and linted

* improved UI for appearance settings pages

* linted and merged with master

* small updates

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2023-08-16 17:30:46 -07:00
timothycarambat
a62e891fa0 force check api online via JSON response 2023-08-16 15:43:46 -07:00
Sean Hatfield
b557bb9ede
Logo customization for single-user & multi-user modes (#186)
* implemented logo customization for single-user mode

* removing unneeded comments

* added dark and light mode support for default logo

* implemented dark and light mode switching in frontend

* fixed dark and light mode switching for failed to load logo from backend

* removed unneeded comment

* custom logos for admin implemented

* refactor logo mgmt functions
abstract logo management utils into their own file for simplicity

* added settings tab for appearance on single-user mode

* unchecking files with unneeded changes

* fixed appearance settings tab to be hidden on multiuser mode

* allow readall for logo

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2023-08-14 15:22:55 -07:00
Timothy Carambat
91f5f94200
[FEATURE] Enable the ability to have multi user instances (#158)
* multi user wip

* WIP MUM features

* invitation mgmt

* suspend or unsuspend users

* workspace mangement

* manage chats

* manage chats

* add Support for admin system settings for users to delete workspaces and limit chats per user

* fix issue ith system var
update app to lazy load invite page

* cleanup and bug fixes

* wrong method

* update readme

* update readme

* update readme

* bump version to 0.1.0
2023-07-25 10:37:04 -07:00
Timothy Carambat
6b6007f9ad
Enable the system owner to be able to update the system wide password and secret (#156)
* Enable the system owner to be able to update the system wide password and secret

* lint and cleanup
2023-07-20 15:25:47 -07:00
Timothy Carambat
6d651db6d3
Enabled importing and exporting of entire AnythingLLM instance data (#146)
* WIP on imports

* undo nodismiss for testing

* remove old keys modal screen
add helper text on import complete

* return default setting modal to keys
2023-07-14 17:32:30 -07:00
Timothy Carambat
60a00843df
add ability to purge document from custom documents as well as cleanup its associated cache file (#113)
* add ability to purge document from custom documents as well as cleanup its assoicated cache file

* update alert text
2023-06-26 17:20:09 -07:00
Timothy Carambat
a3f5a936e2
Enable debug mode of ENV at runtime (#111)
* Enable debug mode of ENV at runtime
Update Storage README for those with SQLite issues

* add files
2023-06-26 11:38:38 -07:00
Timothy Carambat
c4eb46ca19
Upload and process documents via UI + document processor in docker image (#65)
* implement dnd uploader
show file upload progress
write files to hotdirector
build simple flaskAPI to process files one off

* move document processor calls to util
build out dockerfile to run both procs at the same time
update UI to check for document processor before upload
* disable pragma update on boot
* dockerfile changes

* add filetype restrictions based on python app support response and show rejected files in the UI

* cleanup

* stub migrations on boot to prevent exit condition

* update CF template for AWS deploy
2023-06-16 16:01:27 -07:00
Timothy Carambat
2a556c275c
Implement Workspace-specific setting configs + other technical features (#58)
* 1. Define LLM Temperature as a workspace setting
2. Implement rudimentry table migration code for both new and existing repos to bring tables up to date
3. Trigger for workspace on update to update timestamp
4. Always fallback temp to 0.7
5. Extract WorkspaceModal into Tabbed content
6. Remove workspace name UNIQUE constraint (cannot be migrated :()
7. Add slug +seed when existing slug is already take
8. Seperate name from slug so display names can be changed

* remove blocking test return
2023-06-14 23:12:59 -07:00