Commit Graph

209 Commits

Author SHA1 Message Date
Sean Hatfield
f40309cfdb
Add id to all metadata to prevent errors in frontend document picker (#378)
add id to all metadata to prevent errors in frontend docuemnt picker

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2023-11-16 14:36:26 -08:00
Sean Hatfield
73f342eb19
Warning about switching embedder or vectordb (#385)
* added warning modal to LLM preference

* added warning modal for changing embedder

* remove warning from LLM preference & add warning to vector database selection

* linting

* remove comments and move warning modal to component

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2023-11-16 14:35:14 -08:00
timothycarambat
1e3d82e184 patch collector script 2023-11-16 10:25:23 -08:00
timothycarambat
c5dc68633b patch link scrape tool schema 2023-11-14 16:41:39 -08:00
Timothy Carambat
085745c5e4
Prevent lone-admin from locking themselves out the system (#376)
resolves #367
2023-11-14 14:43:40 -08:00
Sean Hatfield
dd33767059
365 update invite UI to match new UI (#370)
* update NewUserModal to match ui styles

* fix bg color of invite screen/auto login after accept invitation

* fix error text color

* cleanup

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2023-11-14 13:58:00 -08:00
Sean Hatfield
1aa58dcb7b
Disable prisma logs on prod (#371)
* disable prisma logs on prod

* linting

* keep const top level

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2023-11-14 13:53:11 -08:00
Tobias Landenberger
a96a9d41a3
LocalAI for embeddings (#361)
* feature: add localAi as embedding provider

* chore: add LocalAI image

* chore: add localai embedding examples to docker .env.example

* update setting env
pull models from localai API

* update comments on embedder
Dont show cost estimation on UI

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2023-11-14 13:49:31 -08:00
timothycarambat
f269124760 fix type error for filetypes 2023-11-14 13:49:18 -08:00
Timothy Carambat
4bb99ab4bf
Support LocalAi as LLM provider by @tlandenberger (#373)
* feature: add LocalAI as llm provider

* update Onboarding/mgmt settings
Grab models from models endpoint for localai
merge with master

* update streaming for complete chunk streaming
update localAI LLM to be able to stream

* force schema on URL

---------
Co-authored-by: timothycarambat <rambat1010@gmail.com>
Co-authored-by: tlandenberger <tobiaslandenberger@gmail.com>
2023-11-14 12:31:44 -08:00
Timothy Carambat
6957bc3ec0
Robots.txt (#369)
* assume default model where appropriate

* merge with master and fix other model refs

* disallow robots

* add public file
2023-11-13 15:22:24 -08:00
Timothy Carambat
8743be679b
assume default model where appropriate (#366)
* assume default model where appropriate

* merge with master and fix other model refs
2023-11-13 15:17:22 -08:00
Timothy Carambat
c22c50cca8
Enable chat streaming for LLMs (#354)
* [Draft] Enable chat streaming for LLMs

* stream only, move sendChat to deprecated

* Update TODO deprecation comments
update console output color for streaming disabled
2023-11-13 15:07:30 -08:00
Sean Hatfield
fa29003a46
Create manager role and limit default role (#351)
* added manager role to options

* block default role from editing workspace settings on workspace and text input box

* block default user from accessing settings at all

* create manager route

* let pass through if in single user mode

* fix permissions for manager and admin roles in settings

* fix settings button for single user and remove unneeded console.logs

* rename routes and paths for clarity

* admin, manager, default roles complete

* remove unneeded comments

* consistency changes

* manage permissions for mum modes

* update sidebar for single-user mode

* update comment on middleware
Modify permission setting for admins

* update render conditional

* Add role usage hint to each role

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2023-11-13 14:51:16 -08:00
Timothy Carambat
7fcf29d769
Add better code-block hightlighting and styles (#357)
* Add better code-block hightlighting and styles

* add fallback class
2023-11-10 16:31:21 -08:00
Timothy Carambat
2b17bf26a8
Posthog telemetry updates (#356)
track subuser anon
2023-11-10 16:02:46 -08:00
Tobias Landenberger
2914c09dd5
fix: adjust return type of addDocuments in case of no additions (#353) 2023-11-10 13:27:53 -08:00
timothycarambat
6e72519caf patch onboarding 2023-11-09 15:19:52 -08:00
timothycarambat
195d3cfcc3 patch filename 2023-11-09 13:42:47 -08:00
Francisco Bischoff
f499f1ba59
Using OpenAI API locally (#335)
* Using OpenAI API locally

* Infinite prompt input and compression implementation (#332)

* WIP on continuous prompt window summary

* wip

* Move chat out of VDB
simplify chat interface
normalize LLM model interface
have compression abstraction
Cleanup compressor
TODO: Anthropic stuff

* Implement compression for Anythropic
Fix lancedb sources

* cleanup vectorDBs and check that lance, chroma, and pinecone are returning valid metadata sources

* Resolve Weaviate citation sources not working with schema

* comment cleanup

* disable import on hosted instances (#339)

* disable import on hosted instances

* Update UI on disabled import/export

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>

* Add support for gpt-4-turbo 128K model (#340)

resolves #336
Add support for gpt-4-turbo 128K model

* 315 show citations based on relevancy score (#316)

* settings for similarity score threshold and prisma schema updated

* prisma schema migration for adding similarityScore setting

* WIP

* Min score default change

* added similarityThreshold checking for all vectordb providers

* linting

---------

Co-authored-by: shatfield4 <seanhatfield5@gmail.com>

* rename localai to lmstudio

* forgot files that were renamed

* normalize model interface

* add model and context window limits

* update LMStudio tagline

* Fully working LMStudio integration

---------
Co-authored-by: Francisco Bischoff <984592+franzbischoff@users.noreply.github.com>
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
Co-authored-by: Sean Hatfield <seanhatfield5@gmail.com>
2023-11-09 12:33:21 -08:00
Timothy Carambat
1ec774ab2e
[Chore] replace all React-feather icons with phosphor icons fully (#349)
replace all React-feather icons with phosphor icons fully
remove package-lock.json files - yarn only
2023-11-09 08:55:20 -08:00
Sean Hatfield
458ffed0c7
added onboarding data handling modal (#342)
* added onboarding data handling modal

* adding data handling modal component

* update element to list
Update copy

* remove useEffect dep

* refactor onboarding navigation using history

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2023-11-08 19:00:12 -08:00
Sean Hatfield
997482ef8f
added JSONL export to workspace chats (#345)
* added JSONL export to workspace chats

* change permissions for workspace chat settings

* change permissions for workspace chat settings

* Show error for correct limit on fine-tune
Change sidebar position and permission
Remove check for MUM

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2023-11-08 17:36:54 -08:00
Timothy Carambat
88d4808c52
315 show citations based on relevancy score (#316)
* settings for similarity score threshold and prisma schema updated

* prisma schema migration for adding similarityScore setting

* WIP

* Min score default change

* added similarityThreshold checking for all vectordb providers

* linting

---------

Co-authored-by: shatfield4 <seanhatfield5@gmail.com>
2023-11-06 16:49:29 -08:00
Timothy Carambat
d34ec68702
Add support for gpt-4-turbo 128K model (#340)
resolves #336
Add support for gpt-4-turbo 128K model
2023-11-06 14:22:19 -08:00
Sean Hatfield
5673dc522b
disable import on hosted instances (#339)
* disable import on hosted instances

* Update UI on disabled import/export

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2023-11-06 13:23:12 -08:00
Timothy Carambat
be9d8b0397
Infinite prompt input and compression implementation (#332)
* WIP on continuous prompt window summary

* wip

* Move chat out of VDB
simplify chat interface
normalize LLM model interface
have compression abstraction
Cleanup compressor
TODO: Anthropic stuff

* Implement compression for Anythropic
Fix lancedb sources

* cleanup vectorDBs and check that lance, chroma, and pinecone are returning valid metadata sources

* Resolve Weaviate citation sources not working with schema

* comment cleanup
2023-11-06 13:13:53 -08:00
Timothy Carambat
0751fb1fdd
Fix missing import on API (#333) 2023-11-03 12:50:56 -07:00
Sean Hatfield
eebafd7327
Replace old modals to match UI (#328)
* workspace not found modal updated to match ui

* update text preview modal to match new ui

* Mobile styles

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2023-11-02 16:17:02 -07:00
timothycarambat
c3abbfbf27 Fix admin chat pagination 2023-11-02 16:12:29 -07:00
timothycarambat
7d3d0e03c6 bind docker to docker/.env so dumps persist between restarts 2023-11-01 22:12:30 -07:00
timothycarambat
2afe3faed8 change how node is installed on docker 2023-11-01 20:10:49 -07:00
Sean Hatfield
ab1e9d66b5
Use toast in workspace settings tab (#322)
* fix white border around user avatar

* add toast into settings menu on change/error
2023-11-01 18:21:30 -07:00
Timothy Carambat
5441717294
normalize parser struct for all file types (#321) 2023-11-01 16:44:02 -07:00
Sean Hatfield
1c5d55c425
Fix white border around user avatar glitch (#320)
fix white border around user avatar
2023-11-01 14:17:27 -07:00
timothycarambat
24823cb5e2 patch workspace chat history windows to persist most recent chats, not the top n 2023-11-01 14:12:27 -07:00
Timothy Carambat
67c85f1550
Implement retrieval and use of fine-tune models (#314)
* Implement retrieval and use of fine-tune models
Cleanup LLM selection code
resolves #311

* Cleanup from PR bot
2023-10-31 11:38:28 -07:00
timothycarambat
745d2aeaff fix import path 2023-10-30 15:49:29 -07:00
Timothy Carambat
5d56ab623b
Anthropic claude 2 support (#305)
* WIP Anythropic support for chat, chat and query w/context

* Add onboarding support for Anthropic

* cleanup

* fix Anthropic answer parsing
move embedding selector to general util
2023-10-30 15:44:03 -07:00
Sean Hatfield
669d7a396d
282 return relevancy score with similarityresponse (#304)
* include score value in similarityResponse for weaviate

* include score value in si
milarityResponse for qdrant

* include score value in si
milarityResponse for pinecone

* include score value in similarityResponse for chroma

* include score value in similarityResponse for lancedb

* distance to similarity

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2023-10-30 12:46:38 -07:00
Francisco Bischoff
26dba59249
mbox parsing improvements v1 (#308)
* mbox parsing improvements v1

* autobots roll out!
2023-10-30 11:57:33 -07:00
timothycarambat
c3fa67bc86 Update AWS deployment 2023-10-29 13:06:43 -07:00
Timothy Carambat
9242578263
Aws docker fixes (#309)
* WIP aws fixes to latest changes;

* change default in docker env

* bump storage default for docker

* Updates to docker deployment
2023-10-29 11:03:41 -07:00
Sean Hatfield
27809b34b5
Added telemetry for onboarding completion (#295)
* added telemetry for onboarding completion

* minor changes

* linting and remove empty object

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2023-10-26 13:49:01 -07:00
Sean Hatfield
c8c0328346
Citations UI bug fix (#294)
citations spacing fix
2023-10-26 13:27:20 -07:00
Sean Hatfield
5f4d42edb8
Sidebar bug fix (#291)
truncate workspace name to prevent gear icon from being shown
2023-10-26 10:58:55 -07:00
Timothy Carambat
a8ec0d9584
Compensate for upper OpenAI emedding limit chunk size (#292)
Limit is due to POST body max size. Sufficiently large requests will abort automatically
We should report that error back on the frontend during embedding
Update vectordb providers to return on failed
2023-10-26 10:57:37 -07:00
Timothy Carambat
9f1a5d9dbc
Add back in selector on prompt input (#289)
resolves #286
2023-10-25 16:01:45 -07:00
Timothy Carambat
9f7b0c837a
Fix prompt overlaying on chats (#288)
resolves #287
2023-10-25 15:23:55 -07:00
timothycarambat
d509d7ed3d compress GIFs in readme 2023-10-25 11:38:47 -07:00