Commit Graph

340 Commits

Author SHA1 Message Date
Timothy Carambat
75dd86967c
Implement AzureOpenAI model chat streaming (#518)
resolves #492
2024-01-03 16:25:39 -08:00
timothycarambat
8001f69454 Merge branch 'master' of github.com:Mintplex-Labs/anything-llm 2024-01-03 15:44:37 -08:00
timothycarambat
dc23961231 drop feeback path in paths 2024-01-03 15:44:26 -08:00
Timothy Carambat
ceadc8d467
patch gpt-4-turbo token allowance for Azure model (#514) 2024-01-02 12:49:48 -08:00
timothycarambat
237c544ebc Merge branch 'master' of github.com:Mintplex-Labs/anything-llm 2024-01-02 12:44:26 -08:00
timothycarambat
d99b87e7d8 patch workspace-chats API endpoint to be generally available instead of forced multi-user 2024-01-02 12:44:17 -08:00
Timothy Carambat
6d5968bf7e
Llm chore cleanup (#501)
* move internal functions to private in class
simplify lc message convertor

* Fix hanging Context text when none is present
2023-12-28 14:42:34 -08:00
Timothy Carambat
2a1202de54
Patch Ollama Streaming chunk issues (#500)
Replace stream/sync chats with Langchain interface for now
connect #499
ref: https://github.com/Mintplex-Labs/anything-llm/issues/495#issuecomment-1871476091
2023-12-28 13:59:47 -08:00
Timothy Carambat
d7481671ba
Prevent external service localhost question (#497)
* Prevent external service localhost question

* add 0.0.0.0 to docker-invalid URL

* clarify hint
2023-12-28 10:47:02 -08:00
Timothy Carambat
e0a0a8976d
Add Ollama as LLM provider option (#494)
* Add support for Ollama as LLM provider
resolves #493
2023-12-27 17:21:47 -08:00
Timothy Carambat
24227e48a7
Add LLM support for Google Gemini-Pro (#492)
resolves #489
2023-12-27 17:08:03 -08:00
timothycarambat
26549df6a9 touchup linting 2023-12-27 13:28:37 -08:00
timothycarambat
1a9a519afb Merge branch 'master' of github.com:Mintplex-Labs/anything-llm 2023-12-27 10:25:20 -08:00
timothycarambat
5931b60202 add FUNDING.yml 2023-12-27 10:25:12 -08:00
Sean Hatfield
1d9ba76b92
fix success is not defined error (#484) 2023-12-21 10:31:14 -08:00
timothycarambat
daadad3859 hoist var in extensions 2023-12-20 19:41:16 -08:00
timothycarambat
31ff4f0832 docs: chain windows docker commands for single enter run 2023-12-20 11:40:04 -08:00
Timothy Carambat
c613eff31c
[Docker] Windows Docker command in Powershell (#480)
* wip

* side by side test

* patch syntax highlighting

* remove spacing formatting

* swap powershell command
2023-12-20 11:33:00 -08:00
timothycarambat
049bfa14cb fix: fully separate chunkconcurrency from chunk length 2023-12-20 11:20:40 -08:00
timothycarambat
7bee849c65 chore: Force VectorCache to always be on;
update file picker spacing for attributes
2023-12-20 10:45:03 -08:00
timothycarambat
67725e807a fix: broken build file 2023-12-19 16:22:25 -08:00
timothycarambat
a7f6003277 fix: set lower maxChunk limit on native embedder to stay within resource constraints
chore: update comment for what embedding chunk means
2023-12-19 16:20:34 -08:00
timothycarambat
b40cfead88 Merge branch 'master' of github.com:Mintplex-Labs/anything-llm 2023-12-19 13:30:25 -08:00
timothycarambat
32b1f05062 add Top link 2023-12-19 13:30:17 -08:00
Timothy Carambat
f2fadd6d2e
Add placeholder collector ENV file (#476)
resolves #474
2023-12-19 13:27:09 -08:00
Timothy Carambat
f1fc2d90fa
Readme updates (#475)
* wip

* btn links

* button updates

* update dev instructions

* typo
2023-12-19 13:24:50 -08:00
timothycarambat
7c941c462c Add ignore-paths for github actions 2023-12-19 12:39:49 -08:00
Timothy Carambat
ecf4295537
Add ability to grab youtube transcripts via doc processor (#470)
* Add ability to grab youtube transcripts via doc processor

* dynamic imports
swap out Github for Youtube in placeholder text
2023-12-18 17:17:26 -08:00
Timothy Carambat
452582489e
GitHub loader extension + extension support v1 (#469)
* feat: implement github repo loading
fix: purge of folders
fix: rendering of sub-files

* noshow delete on custom-documents

* Add API key support because of rate limits

* WIP for frontend of data connectors

* wip

* Add frontend form for GitHub repo data connector

* remove console.logs
block custom-documents from being deleted

* remove _meta unused arg

* Add support for ignore pathing in request
Ignore path input via tagging

* Update hint
2023-12-18 15:48:02 -08:00
Timothy Carambat
2d700b13f6
fix: patch middle truncation func (#465)
resolves #456
2023-12-17 16:04:50 -08:00
Timothy Carambat
1c549e2844
fix: clarify custom truncate from npm truncate (#464)
chore: add @ where we can for less long imports
2023-12-17 15:58:49 -08:00
timothycarambat
2def9a67a1 docs: update default storage path to just write to user home dir for docker instructions 2023-12-17 15:48:56 -08:00
Ikko Eltociear Ashimine
4fe25cebe3
Update README.md (#454)
proprotional -> proportional
2023-12-17 10:44:27 -08:00
Timothy Carambat
65c7c0a518
fix: patch api key not persisting when setting LLM/Embedder (#458) 2023-12-16 10:21:36 -08:00
lunamidori5
ecfd146891
Update Dockerfile (added ffmpeg) (#451) 2023-12-15 19:35:38 -08:00
timothycarambat
d2e3506bb9 fix: transition on LLM and embedding screen
linting
2023-12-15 12:40:11 -08:00
Timothy Carambat
61db981017
feat: Embed on-instance Whisper model for audio/mp4 transcribing (#449)
* feat: Embed on-instance Whisper model for audio/mp4 transcribing
resolves #329

* additional logging

* add placeholder for tmp folder in collector storage
Add cleanup of hotdir and tmp on collector boot to prevent hanging files
split loading of model and file conversion into concurrency

* update README

* update model size

* update supported filetypes
2023-12-15 11:20:13 -08:00
Timothy Carambat
719521c307
Document Processor v2 (#442)
* wip: init refactor of document processor to JS

* add NodeJs PDF support

* wip: partity with python processor
feat: add pptx support

* fix: forgot files

* Remove python scripts totally

* wip:update docker to boot new collector

* add package.json support

* update dockerfile for new build

* update gitignore and linting

* add more protections on file lookup

* update package.json

* test build

* update docker commands to use cap-add=SYS_ADMIN so web scraper can run
update all scripts to reflect this
remove docker build for branch
2023-12-14 15:14:56 -08:00
timothycarambat
5f6a013139 Change server bootup log 2023-12-14 13:52:11 -08:00
Timothy Carambat
1e98da07bc
docs: placeholder for model downloads folder (#446) 2023-12-14 10:31:14 -08:00
Timothy Carambat
d094cc3c10
docs: update deployment templates to just pull the docker image (#445)
docs: update deployment templates to just pull the docker image and boot that instead of building
2023-12-13 19:55:23 -08:00
Timothy Carambat
cc3343ba79
add Docker internal URL to READMEs (#441)
add Docker internal URL
2023-12-13 11:59:14 -08:00
Timothy Carambat
da0cec7aa2
patch: remove unidecode as it was transliterating non-latin chars (#434)
resolves #298
2023-12-13 11:54:55 -08:00
timothycarambat
b444171ef3 fix: Update copy code button on markdown to work with new highlightjs lib implementation 2023-12-12 16:25:22 -08:00
Timothy Carambat
37cdb845a4
patch: implement @lunamidori hotfix for LocalAI streaming chunk overflows (#433)
* patch: implement @lunamidori hotfix for LocalAI streaming chunk overflows
resolves #416

* change log to error log

* log trace

* lint
2023-12-12 16:20:06 -08:00
Timothy Carambat
d4f4d85492
patch: fix non-latin filenames being encoded improperly during upload and chat (#432)
patch: fix non-latin filenames being messed up during upload and chat
connect #169
resolves #427
2023-12-12 16:07:23 -08:00
timothycarambat
7154f4f82b non-blocking send responses 2023-12-12 13:41:17 -08:00
Timothy Carambat
a84333901a
feat: implement questionnaire during onboarding (optional) (#429)
fix: PFP url check
2023-12-12 13:11:32 -08:00
Timothy Carambat
ce9233c258
feat: enable HTML uploads from UI (#422)
resolves #418
2023-12-11 14:40:33 -08:00
Timothy Carambat
cba66150d7
patch: API key to localai service calls (#421)
connect #417
2023-12-11 14:18:28 -08:00