Commit Graph

257 Commits

Author SHA1 Message Date
timothycarambat
df2c01b176 patch OpenRouter model fetcher when key is not present 2024-04-26 15:58:30 -07:00
Timothy Carambat
2e813846dc
Agent skill: chart generation (#1103)
* WIP agent support

* move agent folder

* wip frontend socket

* checkpoint

* fix schema

* Checkpoint for plugins and AgentHandler

* refactor plugins and agent arch

* agent error reporting and handling

* add frontend elements for agents in prompt input

* WIP integrations for agents

* enable web-search agent config from frontend

* persist chat history

* update alert

* update migration
remove console logs
update close state for agent invocations

* add examples to dockerignore
Extract statusResponse to its own component

* update close method

* wrap scraping rejections

* add RAG search as funciton

* Add telem and link highlight

* chat support

* patch memory

* Add rechart as a plugin option

* Toggles for abilites of default agent (system wide)
Validate values for agent skills
Enable dynamic loading of skills
UI for toggle of skills

* add UI for toggle of configs for agent

* toggle WS or WSS protocol

* update NGNIX proxy pass

* move components around and capture failed websocket creation

* fix name

* tmp docker image

* reset workflow

* safety mark functions

* telem on tool calls

* remove hardcode short circuit

* separate web-browser from scrape

* extract summarizer to util
add abort handlers and controller for langchain stuff so socket close kills process

* langchain summarize verbose when in dev

* chart styling improvements + add title to chart

* fix legend from being cutoff in chart downloads

* remove cursor blink

---------

Co-authored-by: shatfield4 <seanhatfield5@gmail.com>
2024-04-26 11:18:55 -07:00
Sean Hatfield
11f6419c3c
[FEAT] Implement new login screen UI & multi-user password reset (#1074)
* WIP new login screen UI

* update prisma schema/create new models for pw recovery

* WIP password recovery backend

* WIP reset password flow

* WIP pw reset flow

* password reset logic complete & functional UI

* WIP login screen redesign for single and multi user

* create placeholder modal to display recovery codes

* implement UI for recovery code modals/download recovery codes

* multiuser desktop password reset UI/functionality complete

* support single user mode for pw reset

* mobile styles for all password reset/login flows complete

* lint

* remove single user password recovery

* create PasswordRecovery util file to make more readable

* do not drop-replace users table in migration

* review pr

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-04-25 16:52:30 -07:00
timothycarambat
dfaaf1680f update perplexity models
resolves #1188
2024-04-25 07:34:28 -07:00
Timothy Carambat
df17fbda36
Add generic OpenAI endpoint support (#1178)
* Add generic OpenAI endpoint support

* allow any input for model in case provider does not support models endpoint
2024-04-23 13:06:07 -07:00
Timothy Carambat
ac6ca13f60
1173 dynamic cache openrouter (#1176)
* patch agent invocation rule

* Add dynamic model cache from OpenRouter API for context length and available models
2024-04-23 11:10:54 -07:00
Sean Hatfield
897e168fd1
[FEAT] Add support for more groq models (Llama 3 and Gemma) (#1143)
add support for more groq models
2024-04-22 13:14:27 -07:00
Timothy Carambat
c65f890afc
Add LMStudio embedding endpoint support (#1141)
* Add LMStudio embedding endpoint support

* update alive path check for HEAD
remove commented JSX

* update comment
2024-04-19 15:36:07 -07:00
Timothy Carambat
b3c7c002db
Warn user when changing embedder with current progress already (#1135)
* Warn user when changing embedder with current progress already

* update comment
2024-04-19 09:51:58 -07:00
Timothy Carambat
58b744771f
Add support for Gemini-1.5 Pro (#1134)
* Add support for Gemini-1.5 Pro
bump @google/generative-ai pkg
Toggle apiVersion if beta model selected
resolves #1109

* update API messages due to package change
2024-04-19 08:59:46 -07:00
timothycarambat
e28c0469f4 bump togetherai models Apr 18, 2024
resolves #1126
2024-04-18 16:28:43 -07:00
Timothy Carambat
91ea37d17a
Patch model selection for openai (#1128)
prevent rerenders of web-browsing for agents
update openai latest pkg name
2024-04-18 13:32:50 -07:00
Timothy Carambat
9655880cf0
Update all vector dbs to filter duplicate source documents that may be pinned (#1122)
* Update all vector dbs to filter duplicate parents

* cleanup
2024-04-17 18:04:39 -07:00
Timothy Carambat
41978765bc
Improve document summarizer by being able to reference by direct name versus ID, which sometimes models fail to chain (#1121) 2024-04-17 14:34:58 -07:00
Timothy Carambat
81fd82e133
model specific summarization (#1119)
* model specific summarization

* update guard functions

* patch model picker and key inputs
2024-04-17 14:04:51 -07:00
Timothy Carambat
9449fcd737
Add Anthropic agent support with new API and tool_calling (#1116)
* Add Anthropic agent support with new API and tool_calling

* patch useProviderHook to unset default models on provider change
2024-04-17 11:54:58 -07:00
Timothy Carambat
8ebe1a515b
Gracefully handle bad agent auth (#1115)
use provider that is set
2024-04-16 16:42:06 -07:00
Timothy Carambat
f9ac27e9a4
Handle Anthropic streamable errors (#1113) 2024-04-16 16:25:32 -07:00
Timothy Carambat
661563408a
Enable dynamic GPT model dropdown (#1111)
* Enable dynamic GPT model dropdown
2024-04-16 14:54:39 -07:00
Timothy Carambat
a5bb77f97a
Agent support for @agent default agent inside workspace chat (#1093)
V1 of agent support via built-in `@agent` that can be invoked alongside normal workspace RAG chat.
2024-04-16 10:50:10 -07:00
Timothy Carambat
8306098b08
Bump all static model providers (#1101) 2024-04-14 12:55:21 -07:00
Timothy Carambat
6fde5570b3
remove unneeded answerKey for Anthropic (#1100)
resolves #1096
2024-04-14 12:04:38 -07:00
Timothy Carambat
d54e1c1f2d
expand support for non-US azure deployments (#1080)
* expand support for non-US azure deployments

* update conditional
2024-04-10 09:34:14 -07:00
Timothy Carambat
24b523d5eb
append missing import for some vectordb providers (#1066) 2024-04-07 14:40:23 -07:00
Timothy Carambat
ce98ff4653
Enable customization of chunk length and overlap (#1059)
* Enable customization of chunk length and overlap

* fix onboarding link
show max limit in UI and prevent overlap >= chunk size
2024-04-06 16:38:07 -07:00
Timothy Carambat
1c11a47f93
Check ollama alive for embedder (#1057) 2024-04-06 12:16:30 -07:00
Timothy Carambat
6f52a2b729
Embedder download - fallback URL (#1056)
* Embedder download - fallback URL

* improve logging for native embedder
2024-04-06 11:49:15 -07:00
Timothy Carambat
94b58249a3
Enable per-workspace provider/model combination (#1042)
* Enable per-workspace provider/model combination

* cleanup

* remove resetWorkspaceChatModels and wipeWorkspaceModelPreference to prevent workspace from resetting model

* add space

---------

Co-authored-by: shatfield4 <seanhatfield5@gmail.com>
2024-04-05 10:58:36 -07:00
timothycarambat
0b454016cf patch comkey path to fallback 2024-04-04 10:47:26 -07:00
timothycarambat
718062d033 patch milvus/zilliz auto-generated collection name
resolves #1027
2024-04-03 12:34:23 -07:00
Timothy Carambat
df2aac9f3c
useMLock for Ollama API chats (#1014) 2024-04-02 10:43:04 -07:00
Ivan Skodje
e9199bac12
[FEAT] Check port access in docker before showing a default error (#961)
* [FEAT] Added port checks in updateENV.validDockerizedUrl to prevent docker from assuming it cannot access localhost URLs

* [CHORE] Updated error message to include Linux URL

* Patch port checking for general loopbacks

* typo

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-04-02 10:34:50 -07:00
Timothy Carambat
f4088d9348
RSA-Signing on server<->collector communication via API (#1005)
* WIP integrity check between processes

* Implement integrity checking on document processor payloads
2024-04-01 13:56:35 -07:00
Timothy Carambat
b8d37d9f43
Handle no-mutli-part form data error (#1004) 2024-04-01 12:06:47 -07:00
timothycarambat
49f30e051c security: patch footer icon self-xss from privledged user 2024-03-29 13:39:11 -07:00
timothycarambat
bfedfebfab security: force sanitize env string set by user 2024-03-29 13:03:05 -07:00
Timothy Carambat
0dd6001fa6
Patch Gemini/Google AI errors (#977) 2024-03-26 17:20:12 -07:00
Timothy Carambat
efe9dfa5e3
handle expired token being null and prevent constant rerender (#956)
* handle expired token being null and prevent constant rerender

* reset defaults
2024-03-26 16:47:25 -07:00
Timothy Carambat
1ecefe8bed
Add ability to add invitee to workspaces automatically (#975) 2024-03-26 16:38:32 -07:00
timothycarambat
7de23dbb2d norm path logo 2024-03-26 15:25:46 -07:00
Timothy Carambat
1135853740
Patch LMStudio Inference server bug integration (#957) 2024-03-22 14:39:30 -07:00
timothycarambat
2ce77b943e filter null folders 2024-03-21 14:47:45 -07:00
Sean Hatfield
4f268dfeb6
[FEAT] Document picker folders for organization (#902)
* implement alternating color rows for file picker

* implement alternating colors for workspace directory

* remove unneeded props/variables

* remove unused border classes

* WIP new folder UI

* remove unneeded expanded prop from filerow component

* folder creation UI and files object manipulation WIP

* folder creation & moving files complete

* add developer API support for creating folders and moving files

* update alternating row css for file picker

* remove unneeded props from merge

* normalize paths for folders
priority to custom docs folder
silently fail on duplicate folders

* update folder icon to custom svg

* linting and move FolderIcon to JSX

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-03-20 16:10:30 -07:00
Timothy Carambat
7e7e957e32
Enable privacy and handling to be reviewed and modified (#910) 2024-03-14 16:56:15 -07:00
Timothy Carambat
0ada882991
Support external transcription providers (#909)
* Support External Transcription providers

* patch files

* update docs

* fix return data
2024-03-14 15:43:26 -07:00
Sean Hatfield
ac0e62d490
[FEAT] Anthropic Haiku model support (#901)
add Haiku model support
2024-03-13 17:32:02 -07:00
Timothy Carambat
0e46a11cb6
Stop generation button during stream-response (#892)
* Stop generation button during stream-response

* add custom stop icon

* add stop to thread chats
2024-03-12 15:21:27 -07:00
Sean Hatfield
04399b1328
[FEAT] Ability to set workspace profile image (#847)
* WIP workspace pfp, CRUD functions complete

* implement fetching workspace pfp in UserIcon component

* update UI for workspace settings pfp

* minor css refactor

* WIP fixes to workspace pfp

* create responseCache for workspace pfp blob to improve performance

* fix cache not clearing when removing workspace pfp and remove unneeded util

* load workspace image once, dont reload

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-03-12 11:36:59 -07:00
Sean Hatfield
e0d5d8039a
[FEAT] Claude 3 support and implement new version of Anthropic SDK (#863)
* implement new version of anthropic sdk and support new models

* remove handleAnthropicStream and move to handleStream inside anthropic provider

* update useGetProvidersModels for new anthropic models
2024-03-06 14:57:47 -08:00
Sean Hatfield
0634013788
[FEAT] Groq LLM support (#865)
* Groq LLM support complete

* update useGetProvidersModels for groq models

* Add definiations
update comments and error log reports
add example envs

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-03-06 14:48:38 -08:00
Gabriel Koo
4731ec8be8
[FIX] : missing import for parseAuthHeader in server/utils/vectorDbProviders/chroma/index.js (#869)
fix: import parseAuthHeader in chroma/index.js
2024-03-06 09:14:36 -08:00
Sean Hatfield
fde905aac1
[FEAT] JSON export append all metadata fields to workspace chats (#845)
have JSON export append all metadata fields
2024-02-29 17:04:59 -08:00
Timothy Carambat
b64cb199f9
788 ollama embedder (#814)
* Add Ollama embedder model support calls

* update docs
2024-02-26 16:12:20 -08:00
Timothy Carambat
a385ea3d82
CHORE: bump pplx model support (#791)
bump pplx model support
2024-02-23 17:33:16 -08:00
Sean Hatfield
633f425206
[FEAT] OpenRouter integration (#784)
* WIP openrouter integration

* add OpenRouter options to onboarding flow and data handling

* add todo to fix headers for rankings

* OpenRouter LLM support complete

* Fix hanging response stream with OpenRouter
update tagline
update comment

* update timeout comment

* wait for first chunk to start timer

* sort OpenRouter models by organization

* uppercase first letter of organization

* sort grouped models by org

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-02-23 17:18:58 -08:00
Sean Hatfield
80ced5eba4
[FEAT] PerplexityAI Support (#778)
* add LLM support for perplexity

* update README & example env

* fix ENV keys in example env files

* slight changes for QA of perplexity support

* Update Perplexity AI name

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-02-22 12:48:57 -08:00
Sean Hatfield
e99c74aec1
[DOCS] Update Docker documentation to show how to setup Ollama with Dockerized version of AnythingLLM (#774)
* update HOW_TO_USE_DOCKER to help with Ollama setup using docker

* update HOW_TO_USE_DOCKER

* styles update

* create separate README for ollama and link to it in HOW_TO_USE_DOCKER

* styling update
2024-02-21 18:42:32 -08:00
Timothy Carambat
791c0ee9dc
Enable ability to do full-text query on documents (#758)
* Enable ability to do full-text query on documents
Show alert modal on first pin for client
Add ability to use pins in stream/chat/embed

* typo and copy update

* simplify spread of context and sources
2024-02-21 13:15:45 -08:00
Timothy Carambat
aad32db5e3
Migrate document processor to class (#735)
* Migrate document processor to class

* forgot "new"
2024-02-16 16:32:25 -08:00
Timothy Carambat
32233974c2
Enable Alpaca JSON export format (#732)
* Enable Alpaca JSON export format

* Replace dom download link with filesave for browser compat
Fix layout of exported json types for readability
2024-02-16 12:35:53 -08:00
Timothy Carambat
44c71013c8
Enforce name requirements for Zilliz/Milvus (#723) 2024-02-14 13:01:05 -08:00
Timothy Carambat
c59ab9da0a
Refactor LLM chat backend (#717)
* refactor stream/chat/embed-stram to be a single execution logic path so that it is easier to maintain and build upon

* no thread in sync chat since only api uses it
adjust import locations
2024-02-14 12:32:07 -08:00
Sean Hatfield
f4b09a8c79
[FEAT] RLHF on response messages (#708)
* WIP RLHF works on historical messages

* refactor Actions component

* completed RLHF up and down votes for chats

* add defaults for HistoricalMessage params

* refactor RLHF implmenation
remove forwardRef on history items to prevent rerenders

* remove dup id

* Add rating to CSV output

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-02-13 11:33:05 -08:00
Sean Hatfield
1b29882c71
[FEAT] Improved CSV chat exports (#700)
* add more fields to csv export to make more useful

* refactor from review comments

* fix escapeCsv function

* catch export errors properly

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-02-13 10:12:59 -08:00
Timothy Carambat
406732830f
Implement workspace threading that is backwards compatible (#699)
* Implement workspace thread that is compatible with legacy versions

* last touches

* comment on chat qty enforcement
2024-02-08 18:37:22 -08:00
Timothy Carambat
f490c35456
Recover from fatal Ollama crash from LangChain library (#693)
Resolve fatal crash from Ollama failure
2024-02-07 16:23:17 -08:00
Timothy Carambat
48cb8f2897
Add support to upload rawText document via api (#692)
* Add support to upload rawText document via api

* update API doc endpoint with correct textContent key

* update response swagger doc
2024-02-07 15:17:32 -08:00
Timothy Carambat
aca5940650
Refactor handleStream to LLM Classes (#685) 2024-02-07 08:15:14 -08:00
Sean Hatfield
d789920a19
[FEAT] Automated audit logging (#667)
* WIP event logging - new table for events and new settings view for viewing

* WIP add logging

* UI for log rows

* rename files to Logging to prevent getting gitignore

* add metadata for all logging events and colored badges in logs page

* remove unneeded comment

* cleanup namespace for logging

* clean up backend calls

* update logging to show to => from settings changes

* add logging for invitations, created, deleted, and accepted

* add logging for user created, updated, suspended, or removed

* add logging for workspace deleted

* add logging for chat logs exported

* add logging for API keys, LLM, embedder, vector db, embed chat, and reset button

* modify event logs

* update to event log types

* simplify rendering of event badges

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-02-06 15:21:40 -08:00
timothycarambat
5d64f26066 patch admin pwd update 2024-02-06 14:39:56 -08:00
Timothy Carambat
2bc11d3f1a
Implement support for HuggingFace Inference Endpoints (#680) 2024-02-06 09:17:51 -08:00
Sean Hatfield
1846a99b93
[FEAT] Embedded AnythingLLM (#656)
* WIP embedded app

* WIP got response from backend in embedded app

* WIP streaming prints to embedded app

* implemented streaming and tailwind min for styling into embedded app

* WIP embedded app history functional

* load params from script tag into embedded app

* rough in modularization of embed chat
cleanup dev process for easier dev support
move all chat to components
todo: build process
todo: backend support

* remove eslint config

* Implement models and cleanup embed chat endpoints
Improve build process for embed
prod minification and bundle size awareness
WIP

* forgot files

* rename to embed folder

* introduce chat modal styles

* add middleware validations on embed chat

* auto open param and default greeting

* reset chat history

* Admin embed config page

* Admin Embed Chats mgmt page

* update embed

* nonpriv

* more style support
reopen if chat was last opened

* update comments

* remove unused imports

* allow change of workspace for embedconfig

* update failure to lookup message

* update reset script

* update instructions

* Add more styling options
Add sponsor text at bottom
Support dynamic container height
Loading animations

* publish new embed script

* Add back syntax highlighting and keep bundle small via dynamic script build

* add hint

* update readme

* update copy model for snippet with link to styles

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-02-05 14:21:34 -08:00
Timothy Carambat
dfab14a5d2
Patch lanceDB not deleting vectors from workspace (#655)
patch lanceDB not deleting vectors from workspace
documentVectors self-sanitize on delete of parent document
2024-01-29 09:49:22 -08:00
Sean Hatfield
9d41ff58e2
[FEAT] add support for new openai embedding models (#653)
* add support for new openai models

* QOL changes/improve logic for adding new openai embedding models

* add example file inputs for Openai embedding ENV selection;

* Fix if stmt conditional

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-01-29 08:48:27 -08:00
Hakeem Abbas
5614e2ed30
feature: Integrate Astra as vectorDBProvider (#648)
* feature: Integrate Astra as vectorDBProvider

feature: Integrate Astra as vectorDBProvider

* Update .env.example

* Add env.example to docker example file
Update spellcheck fo Astra
Update Astra key for vector selection
Update order of AstraDB options
Resize Astra logo image to 330x330
Update methods of Astra to take in latest vectorDB params like TopN and more
Update Astra interface to support default methods and avoid crash errors from 404 collections
Update Astra interface to comply to max chunk insertion limitations
Update Astra interface to dynamically set dimensionality from chunk 0 size on creation

* reset workspaces

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-01-26 13:07:53 -08:00
Sean Hatfield
21653b09fc
[FEAT] add gpt-4-turbo-preview (#651)
* add gpt-4-turbo-preview

* add gpt-4-turbo-preview to valid models
2024-01-26 13:03:50 -08:00
Timothy Carambat
8377600211
Patch Azure text completion persistence (#647) 2024-01-24 13:08:22 -08:00
Sean Hatfield
2f3db0e63a
[FEAT] support pinecone serverless (#639)
* migrate pinecone package to latest version and migrate pinecone vectordb provider class

* remove pinecone environment name env variable and update docs to reflect removal & serverless support complete

* migrate query for pinecone db

* typo in log

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-01-22 16:41:20 -08:00
Timothy Carambat
9a237db3d1
Implement total permission overhaul (#629)
* Implement total permission overhaul
Add explicit permissions on each flex and strict route
Patch issues with role escalation and CRUD of users
Patch permissions on all routes for coverage
Improve middleware to accept role array for clarity

* update comments

* remove permissions to API-keys for manager. Manager could generate API-key and using high-privelege api-key give themselves admin

* update sidebar permissions for multi-user and single user

* update options for mobile sidebar
2024-01-22 14:14:01 -08:00
Sean Hatfield
62cea07599
add gpt-3.5-turbo-1106 model for openai LLM (#636)
* add gpt-3.5-turbo-1106 model for openai LLM

* add gpt-3.5-turbo-1106 as valid model for backend and per workspace model selection
2024-01-22 13:19:47 -08:00
Sean Hatfield
288ff0d18c
fix vector cache not deleting cache after unembedding items with folders (#630) 2024-01-22 13:03:05 -08:00
Timothy Carambat
8a7324d0e7
Employ strict validations on document pathing (#627)
* Employ strict validations on document pathing

* add comment

* update validSubfolder var
2024-01-19 12:56:00 -08:00
timothycarambat
0efb3ab54c remove unneeded import multer 2024-01-19 10:22:37 -08:00
Timothy Carambat
44eb1e9ab0
617 persist special env keys (#624)
* add support for exporting to json and csv in workspace chats

* safety encode URL options

* remove message about openai fine tuning on export success

* all defaults to jsonl

* Persist special env keys on updates

---------

Co-authored-by: shatfield4 <seanhatfield5@gmail.com>
2024-01-18 18:13:24 -08:00
Sean Hatfield
08d33cfd8f
Merge pull request from GHSA-xmj6-g32r-fc5q
* remove support for import export anythingllm data

* remove unused imports
remove unused dep
update lockfile

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-01-18 13:57:23 -08:00
Sean Hatfield
56fa17caf2
create configurable topN per workspace (#616)
* create configurable topN per workspace

* Update TopN UI text
Fix fallbacks for all providers
Add SQLite CHECK to TOPN value

* merge with master
Update zilliz provider for variable TopN

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-01-18 12:34:20 -08:00
Timothy Carambat
658e7fa390
chore: Better VectorDb and Embedder error messages (#620)
* chore: propogate embedder and vectordb errors during document mutations

* add default value for errors on addDocuments
2024-01-18 11:40:48 -08:00
Timothy Carambat
0df86699e7
feat: Add support for Zilliz Cloud by Milvus (#615)
* feat: Add support for Zilliz Cloud by Milvus

* update placeholder text
update data handling stmt

* update zilliz descriptor
2024-01-17 18:00:54 -08:00
Sean Hatfield
3fe7a25759
add token context limit for native llm settings (#614)
Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-01-17 16:25:30 -08:00
Sean Hatfield
c2c8fe9756
add support for mistral api (#610)
* add support for mistral api

* update docs to show support for Mistral

* add default temp to all providers, suggest different results per provider

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-01-17 14:42:05 -08:00
Sean Hatfield
90df37582b
Per workspace model selection (#582)
* WIP model selection per workspace (migrations and openai saves properly

* revert OpenAiOption

* add support for models per workspace for anthropic, localAi, ollama, openAi, and togetherAi

* remove unneeded comments

* update logic for when LLMProvider is reset, reset Ai provider files with master

* remove frontend/api reset of workspace chat and move logic to updateENV
add postUpdate callbacks to envs

* set preferred model for chat on class instantiation

* remove extra param

* linting

* remove unused var

* refactor chat model selection on workspace

* linting

* add fallback for base path to localai models

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-01-17 12:59:25 -08:00
Timothy Carambat
b35feede87
570 document api return object (#608)
* Add support for fetching single document in documents folder

* Add document object to upload + support link scraping via API

* hotfixes for documentation

* update api docs
2024-01-16 16:04:22 -08:00
Timothy Carambat
c61cbd1502
Add support for fetching single document in documents folder (#607) 2024-01-16 14:58:49 -08:00
Timothy Carambat
d0a3f1e3e1
Fix present diminsions on vectorDBs to be inferred for providers who require it (#605) 2024-01-16 13:41:01 -08:00
Timothy Carambat
f5bb064dee
Implement streaming for workspace chats via API (#604) 2024-01-16 10:37:46 -08:00
Timothy Carambat
bd158ce7b1
[Feat] Query mode to return no-result when no context found (#601)
* Query mode to return no-result when no context found

* update default error for sync chat

* remove unnecessary type conversion
2024-01-16 09:32:51 -08:00
timothycarambat
e1dcd5ded0 Normalize pfp path to prevent traversal 2024-01-14 16:53:44 -08:00
Timothy Carambat
026849df02
normalize paths for submit URLs of `remove-documents (#598)
normalize paths for submit URLs
2024-01-14 16:36:17 -08:00
Timothy Carambat
4f6d93159f
improve native embedder handling of large files (#584)
* improve native embedder handling of large files

* perf changes

* ignore storage tmp
2024-01-13 00:32:43 -08:00
Shuyoou
6faa0efaa8
Issue #543 support milvus vector db (#579)
* issue #543 support milvus vector db

* migrate Milvus to use MilvusClient instead of ORM
normalize env setup for docs/implementation
feat: embedder model dimension added

* update comments

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-01-12 13:23:57 -08:00