Timothy Carambat
244ce2e307
Prevent concurrent downloads on first-doc upload ( #1267 )
2024-05-02 10:15:11 -07:00
Timothy Carambat
547d4859ef
Bump openai package to latest ( #1234 )
...
* Bump `openai` package to latest
Tested all except localai
* bump LocalAI support with latest image
* add deprecation notice
* linting
2024-04-30 12:33:42 -07:00
Timothy Carambat
bf435b2861
Adjust how text is split depending on input type ( #1238 )
...
resolves #1230
2024-04-30 10:11:56 -07:00
Timothy Carambat
c65f890afc
Add LMStudio embedding endpoint support ( #1141 )
...
* Add LMStudio embedding endpoint support
* update alive path check for HEAD
remove commented JSX
* update comment
2024-04-19 15:36:07 -07:00
Timothy Carambat
ce98ff4653
Enable customization of chunk length and overlap ( #1059 )
...
* Enable customization of chunk length and overlap
* fix onboarding link
show max limit in UI and prevent overlap >= chunk size
2024-04-06 16:38:07 -07:00
Timothy Carambat
1c11a47f93
Check ollama alive for embedder ( #1057 )
2024-04-06 12:16:30 -07:00
Timothy Carambat
6f52a2b729
Embedder download - fallback URL ( #1056 )
...
* Embedder download - fallback URL
* improve logging for native embedder
2024-04-06 11:49:15 -07:00
Timothy Carambat
b64cb199f9
788 ollama embedder ( #814 )
...
* Add Ollama embedder model support calls
* update docs
2024-02-26 16:12:20 -08:00
Sean Hatfield
9d41ff58e2
[FEAT] add support for new openai embedding models ( #653 )
...
* add support for new openai models
* QOL changes/improve logic for adding new openai embedding models
* add example file inputs for Openai embedding ENV selection;
* Fix if stmt conditional
---------
Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-01-29 08:48:27 -08:00
Timothy Carambat
658e7fa390
chore: Better VectorDb and Embedder error messages ( #620 )
...
* chore: propogate embedder and vectordb errors during document mutations
* add default value for errors on addDocuments
2024-01-18 11:40:48 -08:00
Timothy Carambat
d0a3f1e3e1
Fix present diminsions on vectorDBs to be inferred for providers who require it ( #605 )
2024-01-16 13:41:01 -08:00
Timothy Carambat
4f6d93159f
improve native embedder handling of large files ( #584 )
...
* improve native embedder handling of large files
* perf changes
* ignore storage tmp
2024-01-13 00:32:43 -08:00
Shuyoou
6faa0efaa8
Issue #543 support milvus vector db ( #579 )
...
* issue #543 support milvus vector db
* migrate Milvus to use MilvusClient instead of ORM
normalize env setup for docs/implementation
feat: embedder model dimension added
* update comments
---------
Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-01-12 13:23:57 -08:00
timothycarambat
049bfa14cb
fix: fully separate chunkconcurrency from chunk length
2023-12-20 11:20:40 -08:00
timothycarambat
a7f6003277
fix: set lower maxChunk limit on native embedder to stay within resource constraints
...
chore: update comment for what embedding chunk means
2023-12-19 16:20:34 -08:00
Timothy Carambat
cba66150d7
patch: API key to localai service calls ( #421 )
...
connect #417
2023-12-11 14:18:28 -08:00
Timothy Carambat
8cc1455b72
feat: add support for variable chunk length ( #415 )
...
fix: cleanup code for embedding length clarify
resolves #388
2023-12-07 16:27:36 -08:00
Timothy Carambat
88cdd8c872
Add built-in embedding engine into AnythingLLM ( #411 )
...
* Implement use of native embedder (all-Mini-L6-v2)
stop showing prisma queries during dev
* Add native embedder as an available embedder selection
* wrap model loader in try/catch
* print progress on download
* Update to progress output for embedder
* move embedder selection options to component
* forgot import
* add Data privacy alert updates for local embedder
2023-12-06 10:36:22 -08:00
Tobias Landenberger
a96a9d41a3
LocalAI for embeddings ( #361 )
...
* feature: add localAi as embedding provider
* chore: add LocalAI image
* chore: add localai embedding examples to docker .env.example
* update setting env
pull models from localai API
* update comments on embedder
Dont show cost estimation on UI
---------
Co-authored-by: timothycarambat <rambat1010@gmail.com>
2023-11-14 13:49:31 -08:00
Timothy Carambat
5d56ab623b
Anthropic claude 2 support ( #305 )
...
* WIP Anythropic support for chat, chat and query w/context
* Add onboarding support for Anthropic
* cleanup
* fix Anthropic answer parsing
move embedding selector to general util
2023-10-30 15:44:03 -07:00
