11 Commits

Author	SHA1	Message	Date
timothycarambat	a4c1d42e41	merge with master	2024-04-02 14:33:32 -07:00
Timothy Carambat	f4088d9348	RSA-Signing on server<->collector communication via API (#1005) ... * WIP integrity check between processes * Implement integrity checking on document processor payloads	2024-04-01 13:56:35 -07:00
timothycarambat	65f8a01505	merge with master	2024-03-06 16:43:36 -08:00
Timothy Carambat	ec90060d36	Re-map some file mimes to support text (#842) ... re-map some file mimes to support text	2024-02-29 10:05:03 -08:00
timothycarambat	2b6e1db79b	merge with master	2024-02-27 23:12:09 -08:00
Timothy Carambat	6d18d79bb7	Generic upload fallback as text file. (#808) ... * Do not block any file upload fallback unknown/unsupported types to text if possible * reduce call for frontend * patch	2024-02-26 13:43:54 -08:00
timothycarambat	addb3d0c3e	Update Render.com image for AnythignLLM to latest	2024-01-17 18:12:25 -08:00
Timothy Carambat	b35feede87	570 document api return object (#608) ... * Add support for fetching single document in documents folder * Add document object to upload + support link scraping via API * hotfixes for documentation * update api docs	2024-01-16 16:04:22 -08:00
timothycarambat	0eb2fe7248	Map .env to storage .env file ... map writeToServerDocuments to resolve to fixed storage mount for Render	2023-12-19 11:35:20 -08:00
Timothy Carambat	61db981017	feat: Embed on-instance Whisper model for audio/mp4 transcribing (#449) ... * feat: Embed on-instance Whisper model for audio/mp4 transcribing resolves #329 * additional logging * add placeholder for tmp folder in collector storage Add cleanup of hotdir and tmp on collector boot to prevent hanging files split loading of model and file conversion into concurrency * update README * update model size * update supported filetypes	2023-12-15 11:20:13 -08:00
Timothy Carambat	719521c307	Document Processor v2 (#442) ... * wip: init refactor of document processor to JS * add NodeJs PDF support * wip: partity with python processor feat: add pptx support * fix: forgot files * Remove python scripts totally * wip:update docker to boot new collector * add package.json support * update dockerfile for new build * update gitignore and linting * add more protections on file lookup * update package.json * test build * update docker commands to use cap-add=SYS_ADMIN so web scraper can run update all scripts to reflect this remove docker build for branch	2023-12-14 15:14:56 -08:00