Commit Graph

18 Commits

Author SHA1 Message Date
Timothy Carambat
719521c307
Document Processor v2 (#442)
* wip: init refactor of document processor to JS

* add NodeJs PDF support

* wip: partity with python processor
feat: add pptx support

* fix: forgot files

* Remove python scripts totally

* wip:update docker to boot new collector

* add package.json support

* update dockerfile for new build

* update gitignore and linting

* add more protections on file lookup

* update package.json

* test build

* update docker commands to use cap-add=SYS_ADMIN so web scraper can run
update all scripts to reflect this
remove docker build for branch
2023-12-14 15:14:56 -08:00
timothycarambat
33de34f8dc add embedding engine to telem 2023-12-07 08:53:37 -08:00
Sean Hatfield
7edfccaf9a
Adding url uploads to document picker (#375)
* WIP adding url uploads to document picker

* fix manual script for uploading url to custom-documents

* fix metadata for url scraping

* wip url parsing

* update how async link scraping works

* docker-compose defaults added
no autocomplete on URLs

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2023-11-16 17:15:01 -08:00
Sean Hatfield
fa29003a46
Create manager role and limit default role (#351)
* added manager role to options

* block default role from editing workspace settings on workspace and text input box

* block default user from accessing settings at all

* create manager route

* let pass through if in single user mode

* fix permissions for manager and admin roles in settings

* fix settings button for single user and remove unneeded console.logs

* rename routes and paths for clarity

* admin, manager, default roles complete

* remove unneeded comments

* consistency changes

* manage permissions for mum modes

* update sidebar for single-user mode

* update comment on middleware
Modify permission setting for admins

* update render conditional

* Add role usage hint to each role

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2023-11-13 14:51:16 -08:00
Timothy Carambat
2b17bf26a8
Posthog telemetry updates (#356)
track subuser anon
2023-11-10 16:02:46 -08:00
Sean Hatfield
27809b34b5
Added telemetry for onboarding completion (#295)
* added telemetry for onboarding completion

* minor changes

* linting and remove empty object

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2023-10-26 13:49:01 -07:00
Timothy Carambat
a8ec0d9584
Compensate for upper OpenAI emedding limit chunk size (#292)
Limit is due to POST body max size. Sufficiently large requests will abort automatically
We should report that error back on the frontend during embedding
Update vectordb providers to return on failed
2023-10-26 10:57:37 -07:00
Sean Hatfield
a126b5f5aa
Replace custom sqlite dbms with prisma (#239)
* WIP converted all sqlite models into prisma calls

* modify db setup and fix ApiKey model calls in admin.js

* renaming function params to be consistent

* converted adminEndpoints to utilize prisma orm

* converted chatEndpoints to utilize prisma orm

* converted inviteEndpoints to utilize prisma orm

* converted systemEndpoints to utilize prisma orm

* converted workspaceEndpoints to utilize prisma orm

* converting sql queries to prisma calls

* fixed default param bug for orderBy and limit

* fixed typo for workspace chats

* fixed order of deletion to account for sql relations

* fix invite CRUD and workspace management CRUD

* fixed CRUD for api keys

* created prisma setup scripts/docs for understanding how to use prisma

* prisma dependency change

* removing unneeded console.logs

* removing unneeded sql escape function

* linting and creating migration script

* migration from depreciated sqlite script update

* removing unneeded migrations in prisma folder

* create backup of old sqlite db and use transactions to ensure all operations complete successfully

* adding migrations to gitignore

* updated PRISMA.md docs for info on how to use sqlite migration script

* comment changes

* adding back migrations folder to repo

* Reviewing SQL and prisma integraiton on fresh repo

* update inline key replacement

* ensure migration script executes and maps foreign_keys regardless of db ordering

* run migration endpoint

* support new prisma backend

* bump version

* change migration call

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2023-09-28 14:00:03 -07:00
Timothy Carambat
dc3dfbf314
patch SQL injection opportunities [LOW RISK] (#234) 2023-09-11 16:27:04 -07:00
timothycarambat
8bae47b007 return statement misssing 2023-08-24 17:57:35 -07:00
Sean Hatfield
c0adcc129d
Success fail messages for upload document (#208)
* WIP success fail messages for upload document

* added success/error msgs for uploading feedback and disabled fileUploadProgress in backend

* remove unused middleware

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>
2023-08-22 19:18:47 -07:00
Timothy Carambat
122f29069c
Enable optional Telemetry integration via posthog (#190)
* Enable optional Telemetry integration via posthog
2023-08-14 17:42:17 -07:00
Timothy Carambat
91f5f94200
[FEATURE] Enable the ability to have multi user instances (#158)
* multi user wip

* WIP MUM features

* invitation mgmt

* suspend or unsuspend users

* workspace mangement

* manage chats

* manage chats

* add Support for admin system settings for users to delete workspaces and limit chats per user

* fix issue ith system var
update app to lazy load invite page

* cleanup and bug fixes

* wrong method

* update readme

* update readme

* update readme

* bump version to 0.1.0
2023-07-25 10:37:04 -07:00
Timothy Carambat
c4eb46ca19
Upload and process documents via UI + document processor in docker image (#65)
* implement dnd uploader
show file upload progress
write files to hotdirector
build simple flaskAPI to process files one off

* move document processor calls to util
build out dockerfile to run both procs at the same time
update UI to check for document processor before upload
* disable pragma update on boot
* dockerfile changes

* add filetype restrictions based on python app support response and show rejected files in the UI

* cleanup

* stub migrations on boot to prevent exit condition

* update CF template for AWS deploy
2023-06-16 16:01:27 -07:00
Timothy Carambat
2a556c275c
Implement Workspace-specific setting configs + other technical features (#58)
* 1. Define LLM Temperature as a workspace setting
2. Implement rudimentry table migration code for both new and existing repos to bring tables up to date
3. Trigger for workspace on update to update timestamp
4. Always fallback temp to 0.7
5. Extract WorkspaceModal into Tabbed content
6. Remove workspace name UNIQUE constraint (cannot be migrated :()
7. Add slug +seed when existing slug is already take
8. Seperate name from slug so display names can be changed

* remove blocking test return
2023-06-14 23:12:59 -07:00
timothycarambat
fc0431fca3 add endpoint saftey to prevent server dying in production 2023-06-08 13:13:48 -07:00
Timothy Carambat
6d01970df2
Implement Chroma Support (#1) 2023-06-07 21:31:35 -07:00
timothycarambat
27c58541bd inital commit 2023-06-03 19:28:07 -07:00