kamp/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2024-11-15 19:00:33 +01:00
Code Issues Packages Projects Releases Wiki Activity
635 Commits 41 Branches 8 Tags 106 MiB
529e629eda
Commit Graph

635 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
timothycarambat
d99b87e7d8 patch workspace-chats API endpoint to be generally available instead of forced multi-user 2024-01-02 12:44:17 -08:00
Timothy Carambat
6d5968bf7e
Llm chore cleanup (#501) 
* move internal functions to private in class
simplify lc message convertor

* Fix hanging Context text when none is present
 2023-12-28 14:42:34 -08:00
Timothy Carambat
2a1202de54
Patch Ollama Streaming chunk issues (#500) 
Replace stream/sync chats with Langchain interface for now
connect #499
ref: https://github.com/Mintplex-Labs/anything-llm/issues/495#issuecomment-1871476091
 2023-12-28 13:59:47 -08:00
Timothy Carambat
d7481671ba
Prevent external service localhost question (#497) 
* Prevent external service localhost question

* add 0.0.0.0 to docker-invalid URL

* clarify hint
 2023-12-28 10:47:02 -08:00
Timothy Carambat
e0a0a8976d
Add Ollama as LLM provider option (#494) 
* Add support for Ollama as LLM provider
resolves #493
 2023-12-27 17:21:47 -08:00
Timothy Carambat
24227e48a7
Add LLM support for Google Gemini-Pro (#492) 
resolves #489
 2023-12-27 17:08:03 -08:00
timothycarambat
26549df6a9 touchup linting 2023-12-27 13:28:37 -08:00
timothycarambat
1a9a519afb Merge branch 'master' of github.com:Mintplex-Labs/anything-llm 2023-12-27 10:25:20 -08:00
timothycarambat
5931b60202 add FUNDING.yml 2023-12-27 10:25:12 -08:00
Sean Hatfield
1d9ba76b92
fix success is not defined error (#484) 2023-12-21 10:31:14 -08:00
timothycarambat
daadad3859 hoist var in extensions 2023-12-20 19:41:16 -08:00
timothycarambat
31ff4f0832 docs: chain windows docker commands for single enter run 2023-12-20 11:40:04 -08:00
Timothy Carambat
c613eff31c
[Docker] Windows Docker command in Powershell (#480) 
* wip

* side by side test

* patch syntax highlighting

* remove spacing formatting

* swap powershell command
 2023-12-20 11:33:00 -08:00
timothycarambat
049bfa14cb fix: fully separate chunkconcurrency from chunk length 2023-12-20 11:20:40 -08:00
timothycarambat
7bee849c65 chore: Force VectorCache to always be on; 
update file picker spacing for attributes
 2023-12-20 10:45:03 -08:00
timothycarambat
67725e807a fix: broken build file 2023-12-19 16:22:25 -08:00
timothycarambat
a7f6003277 fix: set lower maxChunk limit on native embedder to stay within resource constraints 
chore: update comment for what embedding chunk means
 2023-12-19 16:20:34 -08:00
timothycarambat
b40cfead88 Merge branch 'master' of github.com:Mintplex-Labs/anything-llm 2023-12-19 13:30:25 -08:00
timothycarambat
32b1f05062 add Top link 2023-12-19 13:30:17 -08:00
Timothy Carambat
f2fadd6d2e
Add placeholder collector ENV file (#476) 
resolves #474
 2023-12-19 13:27:09 -08:00
Timothy Carambat
f1fc2d90fa
Readme updates (#475) 
* wip

* btn links

* button updates

* update dev instructions

* typo
 2023-12-19 13:24:50 -08:00
timothycarambat
7c941c462c Add ignore-paths for github actions 2023-12-19 12:39:49 -08:00
Timothy Carambat
ecf4295537
Add ability to grab youtube transcripts via doc processor (#470) 
* Add ability to grab youtube transcripts via doc processor

* dynamic imports
swap out Github for Youtube in placeholder text
 2023-12-18 17:17:26 -08:00
Timothy Carambat
452582489e
GitHub loader extension + extension support v1 (#469) 
* feat: implement github repo loading
fix: purge of folders
fix: rendering of sub-files

* noshow delete on custom-documents

* Add API key support because of rate limits

* WIP for frontend of data connectors

* wip

* Add frontend form for GitHub repo data connector

* remove console.logs
block custom-documents from being deleted

* remove _meta unused arg

* Add support for ignore pathing in request
Ignore path input via tagging

* Update hint
 2023-12-18 15:48:02 -08:00
Timothy Carambat
2d700b13f6
fix: patch middle truncation func (#465) 
resolves #456
 2023-12-17 16:04:50 -08:00
Timothy Carambat
1c549e2844
fix: clarify custom truncate from npm truncate (#464) 
chore: add @ where we can for less long imports
 2023-12-17 15:58:49 -08:00
timothycarambat
2def9a67a1 docs: update default storage path to just write to user home dir for docker instructions 2023-12-17 15:48:56 -08:00
Ikko Eltociear Ashimine
4fe25cebe3
Update README.md (#454) 
proprotional -> proportional
 2023-12-17 10:44:27 -08:00
Timothy Carambat
65c7c0a518
fix: patch api key not persisting when setting LLM/Embedder (#458) 2023-12-16 10:21:36 -08:00
lunamidori5
ecfd146891
Update Dockerfile (added ffmpeg) (#451) 2023-12-15 19:35:38 -08:00
timothycarambat
d2e3506bb9 fix: transition on LLM and embedding screen 
linting
 2023-12-15 12:40:11 -08:00
Timothy Carambat
61db981017
feat: Embed on-instance Whisper model for audio/mp4 transcribing (#449) 
* feat: Embed on-instance Whisper model for audio/mp4 transcribing
resolves #329

* additional logging

* add placeholder for tmp folder in collector storage
Add cleanup of hotdir and tmp on collector boot to prevent hanging files
split loading of model and file conversion into concurrency

* update README

* update model size

* update supported filetypes
 2023-12-15 11:20:13 -08:00
Timothy Carambat
719521c307
Document Processor v2 (#442) 
* wip: init refactor of document processor to JS

* add NodeJs PDF support

* wip: partity with python processor
feat: add pptx support

* fix: forgot files

* Remove python scripts totally

* wip:update docker to boot new collector

* add package.json support

* update dockerfile for new build

* update gitignore and linting

* add more protections on file lookup

* update package.json

* test build

* update docker commands to use cap-add=SYS_ADMIN so web scraper can run
update all scripts to reflect this
remove docker build for branch
 2023-12-14 15:14:56 -08:00
timothycarambat
5f6a013139 Change server bootup log 2023-12-14 13:52:11 -08:00
Timothy Carambat
1e98da07bc
docs: placeholder for model downloads folder (#446) 2023-12-14 10:31:14 -08:00
Timothy Carambat
d094cc3c10
docs: update deployment templates to just pull the docker image (#445) 
docs: update deployment templates to just pull the docker image and boot that instead of building
 2023-12-13 19:55:23 -08:00
Timothy Carambat
cc3343ba79
add Docker internal URL to READMEs (#441) 
add Docker internal URL
 2023-12-13 11:59:14 -08:00
Timothy Carambat
da0cec7aa2
patch: remove unidecode as it was transliterating non-latin chars (#434) 
resolves #298
 2023-12-13 11:54:55 -08:00
timothycarambat
b444171ef3 fix: Update copy code button on markdown to work with new highlightjs lib implementation 2023-12-12 16:25:22 -08:00
Timothy Carambat
37cdb845a4
patch: implement @lunamidori hotfix for LocalAI streaming chunk overflows (#433) 
* patch: implement @lunamidori hotfix for LocalAI streaming chunk overflows
resolves #416

* change log to error log

* log trace

* lint
 2023-12-12 16:20:06 -08:00
Timothy Carambat
d4f4d85492
patch: fix non-latin filenames being encoded improperly during upload and chat (#432) 
patch: fix non-latin filenames being messed up during upload and chat
connect #169
resolves #427
 2023-12-12 16:07:23 -08:00
timothycarambat
7154f4f82b non-blocking send responses 2023-12-12 13:41:17 -08:00
Timothy Carambat
a84333901a
feat: implement questionnaire during onboarding (optional) (#429) 
fix: PFP url check
 2023-12-12 13:11:32 -08:00
Timothy Carambat
ce9233c258
feat: enable HTML uploads from UI (#422) 
resolves #418
 2023-12-11 14:40:33 -08:00
Timothy Carambat
cba66150d7
patch: API key to localai service calls (#421) 
connect #417
 2023-12-11 14:18:28 -08:00
Ikko Eltociear Ashimine
fef5169ca4
chore: update README.md (#420) 
standlone -> standalone
 2023-12-11 11:32:26 -08:00
Timothy Carambat
8cc1455b72
feat: add support for variable chunk length (#415) 
fix: cleanup code for embedding length clarify
resolves #388
 2023-12-07 16:27:36 -08:00
timothycarambat
48dd99b028 chore: update docker boot script for storage location export 2023-12-07 15:11:48 -08:00
Timothy Carambat
655ebd9479
[Feature] AnythingLLM use locally hosted Llama.cpp and GGUF files for inferencing (#413) 
* Implement use of native embedder (all-Mini-L6-v2)
stop showing prisma queries during dev

* Add native embedder as an available embedder selection

* wrap model loader in try/catch

* print progress on download

* add built-in LLM support (expiermental)

* Update to progress output for embedder

* move embedder selection options to component

* saftey checks for modelfile

* update ref

* Hide selection when on hosted subdomain

* update documentation
hide localLlama when on hosted

* saftey checks for storage of models

* update dockerfile to pre-build Llama.cpp bindings

* update lockfile

* add langchain doc comment

* remove extraneous --no-metal option

* Show data handling for private LLM

* persist model in memory for N+1 chats

* update import
update dev comment on token model size

* update primary README

* chore: more readme updates and remove screenshots - too much to maintain, just use the app!

* remove screeshot link
 2023-12-07 14:48:27 -08:00
timothycarambat
fecfb0fafc chore: remove unused NO_DEBUG env 2023-12-07 14:14:30 -08:00