timothycarambat
3e088f22b1
fix: Patch tiktoken method missing
...
resolves #541
2024-01-05 09:39:19 -08:00
Sean Hatfield
d95d1a9dfd
529 UI update llm embedder and vectordb selection pages ( #533 )
...
* move llm, embedder, vectordb items to components folder
* add backdrop blur to search in llm, embedder, vectordb preferences
* implement searchable llm preference in settings
* implement searchable embedder in settings
* remove unused useState from embedder preferences
* implement searchable vector database in settings
* fix save changes button not appearing on change for llm, embedder, and vectordb settings pages
* sort selected items in all settings and put selected item at top of list
* no auto-top for selection
---------
Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-01-04 18:20:58 -08:00
Timothy Carambat
e9f7b9b79e
Handle undefined stream chunk for native LLM ( #534 )
2024-01-04 18:05:06 -08:00
pritchey
74d2711d80
523-Added support for HTTPS to Server. ( #524 )
...
* Added support for HTTPS to server.
* Move boot scripts to helper file
catch bad ssl boot config
fallback SSL boot to HTTP
---------
Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-01-04 17:22:15 -08:00
Sayan Gupta
b7d2756754
Issue #204 Added a check to ensure that 'chunk.payload' exists and contains the 'id' property ( #526 )
...
* Issue #204 Added a check to ensure that 'chunk.payload' exists and contains the 'id' property before attempting to destructure it
* run linter
* simplify condition and comment
---------
Co-authored-by: timothycarambat <rambat1010@gmail.com>
2024-01-04 16:39:43 -08:00
timothycarambat
a1b4ed43ba
allow native when on embedder
2024-01-04 16:12:20 -08:00
Timothy Carambat
d8ca92df88
Onboarding V2 ( #502 )
...
* WIP onboarding v2
* Welcome screen for onboarding complete
* fix home page and WIP create skeleton for llm preference search/options
* render llms as options
* add search functionality to llm preference & add survey step
* fix openai settings undefined & create custom logo onboarding page
* add user setup UI
* add data handling & privacy onboarding screen
* add create workspace onboarding screen
* fix survey width in onboarding
* create vector database connection onboarding page
* add workspace image & all skeleton ui complete
* fix navigation buttons and ui tweaks to fit on screen
* WIP LLMPreference
* LLM Preference screen fully functional
* create components for vector db options and fix styling of azure options
* remove unneeded comment
* vector db connection onboarding screen complete
* minor ui tweak to searchbar
* user setup page fully working
* create workspace onboarding page fully working
* useNavigate for navigation between pages
* mobile layout, cleanup old files, survey functionality implemented
* fix default logo appearing when should be blank & password setup bug fix
* Modify flow of onboarding
todo: embedding set up
* Add embedder setup screen & insert into flow
* update embedding back button
auto-dismiss toasts on each step
* move page defs under imports
fix bg color on mobile styling
---------
Co-authored-by: shatfield4 <seanhatfield5@gmail.com>
2024-01-04 15:54:31 -08:00
Timothy Carambat
92da23e963
Handle special token in TikToken ( #528 )
...
* Handle special token in TikToken
resolves #525
* remove duplicate method
add clarification comment on implementation
2024-01-04 15:47:00 -08:00
timothycarambat
a2a903741d
replace stored GIF with Github CDN hosted image
2024-01-04 10:59:24 -08:00
Timothy Carambat
75dd86967c
Implement AzureOpenAI model chat streaming ( #518 )
...
resolves #492
2024-01-03 16:25:39 -08:00
timothycarambat
8001f69454
Merge branch 'master' of github.com:Mintplex-Labs/anything-llm
2024-01-03 15:44:37 -08:00
timothycarambat
dc23961231
drop feeback path in paths
2024-01-03 15:44:26 -08:00
Timothy Carambat
ceadc8d467
patch gpt-4-turbo token allowance for Azure model ( #514 )
2024-01-02 12:49:48 -08:00
timothycarambat
237c544ebc
Merge branch 'master' of github.com:Mintplex-Labs/anything-llm
2024-01-02 12:44:26 -08:00
timothycarambat
d99b87e7d8
patch workspace-chats API endpoint to be generally available instead of forced multi-user
2024-01-02 12:44:17 -08:00
Timothy Carambat
6d5968bf7e
Llm chore cleanup ( #501 )
...
* move internal functions to private in class
simplify lc message convertor
* Fix hanging Context text when none is present
2023-12-28 14:42:34 -08:00
Timothy Carambat
2a1202de54
Patch Ollama Streaming chunk issues ( #500 )
...
Replace stream/sync chats with Langchain interface for now
connect #499
ref: https://github.com/Mintplex-Labs/anything-llm/issues/495#issuecomment-1871476091
2023-12-28 13:59:47 -08:00
Timothy Carambat
d7481671ba
Prevent external service localhost question ( #497 )
...
* Prevent external service localhost question
* add 0.0.0.0 to docker-invalid URL
* clarify hint
2023-12-28 10:47:02 -08:00
Timothy Carambat
e0a0a8976d
Add Ollama as LLM provider option ( #494 )
...
* Add support for Ollama as LLM provider
resolves #493
2023-12-27 17:21:47 -08:00
Timothy Carambat
24227e48a7
Add LLM support for Google Gemini-Pro ( #492 )
...
resolves #489
2023-12-27 17:08:03 -08:00
timothycarambat
26549df6a9
touchup linting
2023-12-27 13:28:37 -08:00
timothycarambat
1a9a519afb
Merge branch 'master' of github.com:Mintplex-Labs/anything-llm
2023-12-27 10:25:20 -08:00
timothycarambat
5931b60202
add FUNDING.yml
2023-12-27 10:25:12 -08:00
Sean Hatfield
1d9ba76b92
fix success is not defined error ( #484 )
2023-12-21 10:31:14 -08:00
timothycarambat
daadad3859
hoist var in extensions
2023-12-20 19:41:16 -08:00
timothycarambat
31ff4f0832
docs: chain windows docker commands for single enter run
2023-12-20 11:40:04 -08:00
Timothy Carambat
c613eff31c
[Docker] Windows Docker command in Powershell ( #480 )
...
* wip
* side by side test
* patch syntax highlighting
* remove spacing formatting
* swap powershell command
2023-12-20 11:33:00 -08:00
timothycarambat
049bfa14cb
fix: fully separate chunkconcurrency from chunk length
2023-12-20 11:20:40 -08:00
timothycarambat
7bee849c65
chore: Force VectorCache to always be on;
...
update file picker spacing for attributes
2023-12-20 10:45:03 -08:00
timothycarambat
67725e807a
fix: broken build file
2023-12-19 16:22:25 -08:00
timothycarambat
a7f6003277
fix: set lower maxChunk limit on native embedder to stay within resource constraints
...
chore: update comment for what embedding chunk means
2023-12-19 16:20:34 -08:00
timothycarambat
b40cfead88
Merge branch 'master' of github.com:Mintplex-Labs/anything-llm
2023-12-19 13:30:25 -08:00
timothycarambat
32b1f05062
add Top link
2023-12-19 13:30:17 -08:00
Timothy Carambat
f2fadd6d2e
Add placeholder collector ENV file ( #476 )
...
resolves #474
2023-12-19 13:27:09 -08:00
Timothy Carambat
f1fc2d90fa
Readme updates ( #475 )
...
* wip
* btn links
* button updates
* update dev instructions
* typo
2023-12-19 13:24:50 -08:00
timothycarambat
7c941c462c
Add ignore-paths for github actions
2023-12-19 12:39:49 -08:00
Timothy Carambat
ecf4295537
Add ability to grab youtube transcripts via doc processor ( #470 )
...
* Add ability to grab youtube transcripts via doc processor
* dynamic imports
swap out Github for Youtube in placeholder text
2023-12-18 17:17:26 -08:00
Timothy Carambat
452582489e
GitHub loader extension + extension support v1 ( #469 )
...
* feat: implement github repo loading
fix: purge of folders
fix: rendering of sub-files
* noshow delete on custom-documents
* Add API key support because of rate limits
* WIP for frontend of data connectors
* wip
* Add frontend form for GitHub repo data connector
* remove console.logs
block custom-documents from being deleted
* remove _meta unused arg
* Add support for ignore pathing in request
Ignore path input via tagging
* Update hint
2023-12-18 15:48:02 -08:00
Timothy Carambat
2d700b13f6
fix: patch middle truncation func ( #465 )
...
resolves #456
2023-12-17 16:04:50 -08:00
Timothy Carambat
1c549e2844
fix: clarify custom truncate from npm truncate ( #464 )
...
chore: add @ where we can for less long imports
2023-12-17 15:58:49 -08:00
timothycarambat
2def9a67a1
docs: update default storage path to just write to user home dir for docker instructions
2023-12-17 15:48:56 -08:00
Ikko Eltociear Ashimine
4fe25cebe3
Update README.md ( #454 )
...
proprotional -> proportional
2023-12-17 10:44:27 -08:00
Timothy Carambat
65c7c0a518
fix: patch api key not persisting when setting LLM/Embedder ( #458 )
2023-12-16 10:21:36 -08:00
lunamidori5
ecfd146891
Update Dockerfile (added ffmpeg) ( #451 )
2023-12-15 19:35:38 -08:00
timothycarambat
d2e3506bb9
fix: transition on LLM and embedding screen
...
linting
2023-12-15 12:40:11 -08:00
Timothy Carambat
61db981017
feat: Embed on-instance Whisper model for audio/mp4 transcribing ( #449 )
...
* feat: Embed on-instance Whisper model for audio/mp4 transcribing
resolves #329
* additional logging
* add placeholder for tmp folder in collector storage
Add cleanup of hotdir and tmp on collector boot to prevent hanging files
split loading of model and file conversion into concurrency
* update README
* update model size
* update supported filetypes
2023-12-15 11:20:13 -08:00
Timothy Carambat
719521c307
Document Processor v2 ( #442 )
...
* wip: init refactor of document processor to JS
* add NodeJs PDF support
* wip: partity with python processor
feat: add pptx support
* fix: forgot files
* Remove python scripts totally
* wip:update docker to boot new collector
* add package.json support
* update dockerfile for new build
* update gitignore and linting
* add more protections on file lookup
* update package.json
* test build
* update docker commands to use cap-add=SYS_ADMIN so web scraper can run
update all scripts to reflect this
remove docker build for branch
2023-12-14 15:14:56 -08:00
timothycarambat
5f6a013139
Change server bootup log
2023-12-14 13:52:11 -08:00
Timothy Carambat
1e98da07bc
docs: placeholder for model downloads folder ( #446 )
2023-12-14 10:31:14 -08:00
Timothy Carambat
d094cc3c10
docs: update deployment templates to just pull the docker image ( #445 )
...
docs: update deployment templates to just pull the docker image and boot that instead of building
2023-12-13 19:55:23 -08:00
