Commit Graph

25 Commits

Author SHA1 Message Date
timothycarambat
86a31d7551 Merge branch 'master' of github.com:Mintplex-Labs/anything-llm into render 2024-07-01 17:08:59 -07:00
Jason Zhang
fa4ab0f65f
fix: sanitize filename before writing (#1743)
* fix: sanitize filename before writing

Fixes: https://github.com/Mintplex-Labs/anything-llm/issues/1737

* fixup

* fixup
2024-06-25 15:45:09 -07:00
timothycarambat
393772c4a5 Merge branch 'master' of github.com:Mintplex-Labs/anything-llm into render 2024-06-12 09:05:57 -07:00
Chris Daniel
8a4dd2bdf5
[FEAT] add support for TSX files to be parsed as text (#1597)
add support for TSX files to be parsed as text
2024-06-03 17:01:41 +08:00
Sean Hatfield
9a38b32c74
[FEAT] Add support for R files to be parsed as text (#1577)
add support for R files to be parsed as text
2024-05-31 13:52:00 +08:00
timothycarambat
6e8a327d98 merge with master 2024-05-23 12:58:36 -07:00
Timothy Carambat
a89812703b
repatch path normalization (#1516) 2024-05-23 12:52:04 -07:00
timothycarambat
05488c81e0 undo path norm whitespace fix 2024-05-23 12:04:00 -07:00
timothycarambat
c6ad94d81a Merge branch 'master' of github.com:Mintplex-Labs/anything-llm into render 2024-05-22 13:43:09 -05:00
timothycarambat
e208074ef4 patch path normalization 2024-05-22 11:50:01 -05:00
timothycarambat
fde4e5400f Merge branch 'master' of github.com:Mintplex-Labs/anything-llm into render 2024-04-12 14:57:46 -07:00
Timothy Carambat
2c6135aa54
patch file types as plaintext (#1095)
resolves #1089
2024-04-12 14:54:33 -07:00
timothycarambat
75ced7e65a merge with master
Patch LLM selection for native to be disabled
2024-04-07 14:55:18 -07:00
timothycarambat
a4c1d42e41 merge with master 2024-04-02 14:33:32 -07:00
Timothy Carambat
752e3e22ed
Add more text file forced extensions (#1016) 2024-04-02 14:13:11 -07:00
Timothy Carambat
f4088d9348
RSA-Signing on server<->collector communication via API (#1005)
* WIP integrity check between processes

* Implement integrity checking on document processor payloads
2024-04-01 13:56:35 -07:00
timothycarambat
65f8a01505 merge with master 2024-03-06 16:43:36 -08:00
Timothy Carambat
ec90060d36
Re-map some file mimes to support text (#842)
re-map some file mimes to support text
2024-02-29 10:05:03 -08:00
timothycarambat
2b6e1db79b merge with master 2024-02-27 23:12:09 -08:00
Timothy Carambat
6d18d79bb7
Generic upload fallback as text file. (#808)
* Do not block any file upload
fallback unknown/unsupported types to text if possible

* reduce call for frontend

* patch
2024-02-26 13:43:54 -08:00
timothycarambat
addb3d0c3e Update Render.com image for AnythignLLM to latest 2024-01-17 18:12:25 -08:00
Timothy Carambat
b35feede87
570 document api return object (#608)
* Add support for fetching single document in documents folder

* Add document object to upload + support link scraping via API

* hotfixes for documentation

* update api docs
2024-01-16 16:04:22 -08:00
timothycarambat
0eb2fe7248 Map .env to storage .env file
map writeToServerDocuments to resolve to fixed storage mount for Render
2023-12-19 11:35:20 -08:00
Timothy Carambat
61db981017
feat: Embed on-instance Whisper model for audio/mp4 transcribing (#449)
* feat: Embed on-instance Whisper model for audio/mp4 transcribing
resolves #329

* additional logging

* add placeholder for tmp folder in collector storage
Add cleanup of hotdir and tmp on collector boot to prevent hanging files
split loading of model and file conversion into concurrency

* update README

* update model size

* update supported filetypes
2023-12-15 11:20:13 -08:00
Timothy Carambat
719521c307
Document Processor v2 (#442)
* wip: init refactor of document processor to JS

* add NodeJs PDF support

* wip: partity with python processor
feat: add pptx support

* fix: forgot files

* Remove python scripts totally

* wip:update docker to boot new collector

* add package.json support

* update dockerfile for new build

* update gitignore and linting

* add more protections on file lookup

* update package.json

* test build

* update docker commands to use cap-add=SYS_ADMIN so web scraper can run
update all scripts to reflect this
remove docker build for branch
2023-12-14 15:14:56 -08:00