Commit Graph

115 Commits

Author SHA1 Message Date
timothycarambat
971c54e2c8 Merge branch 'master' of github.com:Mintplex-Labs/anything-llm into render 2024-03-26 14:12:09 -07:00
Sean Hatfield
35a155d3ac
Workspace prompt changes (#949)
* track workspace prompt change

* refactor

* modify arg order

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-03-21 15:25:42 -07:00
timothycarambat
373c833aad path API document mgmt with STORAGE_DIR 2024-03-21 15:10:44 -07:00
timothycarambat
22ef2c287d folder mgmt, use STORAGE_DIR 2024-03-21 15:03:52 -07:00
Sean Hatfield
4f268dfeb6
[FEAT] Document picker folders for organization (#902)
* implement alternating color rows for file picker

* implement alternating colors for workspace directory

* remove unneeded props/variables

* remove unused border classes

* WIP new folder UI

* remove unneeded expanded prop from filerow component

* folder creation UI and files object manipulation WIP

* folder creation & moving files complete

* add developer API support for creating folders and moving files

* update alternating row css for file picker

* remove unneeded props from merge

* normalize paths for folders
priority to custom docs folder
silently fail on duplicate folders

* update folder icon to custom svg

* linting and move FolderIcon to JSX

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-03-20 16:10:30 -07:00
timothycarambat
429ea0c805 Merge branch 'master' of github.com:Mintplex-Labs/anything-llm into render 2024-03-12 12:29:57 -07:00
Sean Hatfield
04399b1328
[FEAT] Ability to set workspace profile image (#847)
* WIP workspace pfp, CRUD functions complete

* implement fetching workspace pfp in UserIcon component

* update UI for workspace settings pfp

* minor css refactor

* WIP fixes to workspace pfp

* create responseCache for workspace pfp blob to improve performance

* fix cache not clearing when removing workspace pfp and remove unneeded util

* load workspace image once, dont reload

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-03-12 11:36:59 -07:00
timothycarambat
d1d29ff2e4 Merge branch 'master' of github.com:Mintplex-Labs/anything-llm into render 2024-03-08 12:56:28 -08:00
timothycarambat
273d824df4 patch: do not block response if user password check failed 2024-03-06 17:14:17 -08:00
timothycarambat
2b6e1db79b merge with master 2024-02-27 23:12:09 -08:00
Timothy Carambat
9e085baf5e
Add chat-export to API (#831) 2024-02-27 12:52:57 -08:00
Timothy Carambat
bbe3c0a5d6
Bulk remove files from file picker (#830)
Do bulk deletion on backend
2024-02-27 11:53:42 -08:00
timothycarambat
28069040f3 merge with master 2024-02-21 15:16:01 -08:00
timothycarambat
ae01785220 Merge branch 'master' of github.com:Mintplex-Labs/anything-llm into render 2024-02-21 15:11:45 -08:00
Timothy Carambat
791c0ee9dc
Enable ability to do full-text query on documents (#758)
* Enable ability to do full-text query on documents
Show alert modal on first pin for client
Add ability to use pins in stream/chat/embed

* typo and copy update

* simplify spread of context and sources
2024-02-21 13:15:45 -08:00
Sean Hatfield
17c1913ccc
[FEAT]: Allow user to set support email (#726)
* implement custom support email for usermenu support button

* small refactor

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-02-19 10:30:41 -08:00
Timothy Carambat
aad32db5e3
Migrate document processor to class (#735)
* Migrate document processor to class

* forgot "new"
2024-02-16 16:32:25 -08:00
Timothy Carambat
51dbff0dcb
Breakout Chat/Query mode as a workspace setting (#734)
Remove useless icons in prompt bar
Add chatMode column to workspaces that defaults to chat
Add UI for toggle of chat mode with hint
Update UI for workspace settings to match designs
2024-02-16 14:50:40 -08:00
Timothy Carambat
c59ab9da0a
Refactor LLM chat backend (#717)
* refactor stream/chat/embed-stram to be a single execution logic path so that it is easier to maintain and build upon

* no thread in sync chat since only api uses it
adjust import locations
2024-02-14 12:32:07 -08:00
Sean Hatfield
f4b09a8c79
[FEAT] RLHF on response messages (#708)
* WIP RLHF works on historical messages

* refactor Actions component

* completed RLHF up and down votes for chats

* add defaults for HistoricalMessage params

* refactor RLHF implmenation
remove forwardRef on history items to prevent rerenders

* remove dup id

* Add rating to CSV output

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-02-13 11:33:05 -08:00
Sean Hatfield
1b29882c71
[FEAT] Improved CSV chat exports (#700)
* add more fields to csv export to make more useful

* refactor from review comments

* fix escapeCsv function

* catch export errors properly

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-02-13 10:12:59 -08:00
Timothy Carambat
406732830f
Implement workspace threading that is backwards compatible (#699)
* Implement workspace thread that is compatible with legacy versions

* last touches

* comment on chat qty enforcement
2024-02-08 18:37:22 -08:00
Sean Hatfield
b985524901
[FEAT] Customizable footer icon links in Appearance Settings (#694)
* WIP custom footer icons

* UI for updating footer icons complete and backend to save/modify

* add backend for unprotected footer fetch

* break out footer into separate component and render footer items using a cache for 1 hour

* wip review

* refactor & cleanup

* Optimize footer form component
Optimize caching for footer icons
Add validation on SystemSetting upserts
Normalize fallback items for footer_data

* Adjust max icons to 3

* fix success message on remove

* fix success message on remove

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-02-08 12:17:01 -08:00
Timothy Carambat
48cb8f2897
Add support to upload rawText document via api (#692)
* Add support to upload rawText document via api

* update API doc endpoint with correct textContent key

* update response swagger doc
2024-02-07 15:17:32 -08:00
Sean Hatfield
d789920a19
[FEAT] Automated audit logging (#667)
* WIP event logging - new table for events and new settings view for viewing

* WIP add logging

* UI for log rows

* rename files to Logging to prevent getting gitignore

* add metadata for all logging events and colored badges in logs page

* remove unneeded comment

* cleanup namespace for logging

* clean up backend calls

* update logging to show to => from settings changes

* add logging for invitations, created, deleted, and accepted

* add logging for user created, updated, suspended, or removed

* add logging for workspace deleted

* add logging for chat logs exported

* add logging for API keys, LLM, embedder, vector db, embed chat, and reset button

* modify event logs

* update to event log types

* simplify rendering of event badges

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-02-06 15:21:40 -08:00
Sean Hatfield
36558350e7
Fix white screen bug when deleting chats under Workspace Chats menu (#681)
* use filter instead of ref to delete row from workspace chat component on delete/fix backend invalid json error

* remove ref from pagination on embed chats and fix white screen bug

* remove unneeded import

* normalize response object

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-02-06 13:11:44 -08:00
Sean Hatfield
608f28d745
[FEAT] create custom prompt suggestions per workspace (#664)
* create custom suggested chat messages per workspace

* update how suggestedChats are passed to chat window

* update mobile styles

* update edit change handler

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-02-06 11:24:33 -08:00
Sean Hatfield
1846a99b93
[FEAT] Embedded AnythingLLM (#656)
* WIP embedded app

* WIP got response from backend in embedded app

* WIP streaming prints to embedded app

* implemented streaming and tailwind min for styling into embedded app

* WIP embedded app history functional

* load params from script tag into embedded app

* rough in modularization of embed chat
cleanup dev process for easier dev support
move all chat to components
todo: build process
todo: backend support

* remove eslint config

* Implement models and cleanup embed chat endpoints
Improve build process for embed
prod minification and bundle size awareness
WIP

* forgot files

* rename to embed folder

* introduce chat modal styles

* add middleware validations on embed chat

* auto open param and default greeting

* reset chat history

* Admin embed config page

* Admin Embed Chats mgmt page

* update embed

* nonpriv

* more style support
reopen if chat was last opened

* update comments

* remove unused imports

* allow change of workspace for embedconfig

* update failure to lookup message

* update reset script

* update instructions

* Add more styling options
Add sponsor text at bottom
Support dynamic container height
Loading animations

* publish new embed script

* Add back syntax highlighting and keep bundle small via dynamic script build

* add hint

* update readme

* update copy model for snippet with link to styles

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-02-05 14:21:34 -08:00
Sean Hatfield
2f3db0e63a
[FEAT] support pinecone serverless (#639)
* migrate pinecone package to latest version and migrate pinecone vectordb provider class

* remove pinecone environment name env variable and update docs to reflect removal & serverless support complete

* migrate query for pinecone db

* typo in log

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-01-22 16:41:20 -08:00
Timothy Carambat
d5cde8b7c2
Apply permissioning on document modification endpoints (#637) 2024-01-22 14:31:19 -08:00
Timothy Carambat
9a237db3d1
Implement total permission overhaul (#629)
* Implement total permission overhaul
Add explicit permissions on each flex and strict route
Patch issues with role escalation and CRUD of users
Patch permissions on all routes for coverage
Improve middleware to accept role array for clarity

* update comments

* remove permissions to API-keys for manager. Manager could generate API-key and using high-privelege api-key give themselves admin

* update sidebar permissions for multi-user and single user

* update options for mobile sidebar
2024-01-22 14:14:01 -08:00
Timothy Carambat
0db6c3b2aa
Prevent private octets from link collection for self-hosted (#626) 2024-01-19 10:49:40 -08:00
timothycarambat
0efb3ab54c remove unneeded import multer 2024-01-19 10:22:37 -08:00
Sean Hatfield
7fb76cfef0
add support for exporting to json and csv in workspace chats (#622)
* add support for exporting to json and csv in workspace chats

* safety encode URL options

* remove message about openai fine tuning on export success

* all defaults to jsonl
2024-01-18 17:59:51 -08:00
Sean Hatfield
08d33cfd8f
Merge pull request from GHSA-xmj6-g32r-fc5q
* remove support for import export anythingllm data

* remove unused imports
remove unused dep
update lockfile

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-01-18 13:57:23 -08:00
Timothy Carambat
658e7fa390
chore: Better VectorDb and Embedder error messages (#620)
* chore: propogate embedder and vectordb errors during document mutations

* add default value for errors on addDocuments
2024-01-18 11:40:48 -08:00
timothycarambat
addb3d0c3e Update Render.com image for AnythignLLM to latest 2024-01-17 18:12:25 -08:00
Sean Hatfield
90df37582b
Per workspace model selection (#582)
* WIP model selection per workspace (migrations and openai saves properly

* revert OpenAiOption

* add support for models per workspace for anthropic, localAi, ollama, openAi, and togetherAi

* remove unneeded comments

* update logic for when LLMProvider is reset, reset Ai provider files with master

* remove frontend/api reset of workspace chat and move logic to updateENV
add postUpdate callbacks to envs

* set preferred model for chat on class instantiation

* remove extra param

* linting

* remove unused var

* refactor chat model selection on workspace

* linting

* add fallback for base path to localai models

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-01-17 12:59:25 -08:00
Timothy Carambat
b35feede87
570 document api return object (#608)
* Add support for fetching single document in documents folder

* Add document object to upload + support link scraping via API

* hotfixes for documentation

* update api docs
2024-01-16 16:04:22 -08:00
Timothy Carambat
c61cbd1502
Add support for fetching single document in documents folder (#607) 2024-01-16 14:58:49 -08:00
Timothy Carambat
f5bb064dee
Implement streaming for workspace chats via API (#604) 2024-01-16 10:37:46 -08:00
Timothy Carambat
bd158ce7b1
[Feat] Query mode to return no-result when no context found (#601)
* Query mode to return no-result when no context found

* update default error for sync chat

* remove unnecessary type conversion
2024-01-16 09:32:51 -08:00
timothycarambat
7aaa4b38e7 add flex role to export endpoint 2024-01-14 17:10:49 -08:00
timothycarambat
e1dcd5ded0 Normalize pfp path to prevent traversal 2024-01-14 16:53:44 -08:00
Timothy Carambat
026849df02
normalize paths for submit URLs of `remove-documents (#598)
normalize paths for submit URLs
2024-01-14 16:36:17 -08:00
Timothy Carambat
7200a06ef0
prevent manager in multi-user from updatingENV via HTTP (#576)
* prevent manager in multi-user from updatingENV via HTTP

* remove unneeded args
2024-01-11 12:11:45 -08:00
Timothy Carambat
3c859ba303
Change pwd check to O(1) check to prevent timing attacks - single user mode (#575)
Change pwd check to O(1) check to prevent timing attacks
2024-01-11 10:54:55 -08:00
Timothy Carambat
4e2c0f04b4
Dynamic vector count on workspace settings (#567)
* Dynamic vector count on workspace settings
Add count to be workspace specific, fallback to system count
Update layout of data in settings
Update OpenAI per-token embedding price

* linting
2024-01-10 13:18:48 -08:00
Timothy Carambat
8cd3a92c66
[BUG] Fixed mass_assignment vuln (#566)
Fixed mass_assignment vuln

Co-authored-by: dastaj <78434825+dastaj@users.noreply.github.com>
2024-01-10 08:42:03 -08:00
timothycarambat
a48a5ad6ad Merge branch 'master' of github.com:Mintplex-Labs/anything-llm into render 2024-01-08 17:01:23 -08:00