anything-llm/collector
Timothy Carambat dc4ad6b5a9
[BETA] Live document sync (#1719)
* wip bg workers for live document sync

* Add ability to re-embed specific documents across many workspaces via background queue
bgworkser is gated behind expieremental system setting flag that needs to be explictly enabled
UI for watching/unwatching docments that are embedded.
TODO: UI to easily manage all bg tasks and see run results
TODO: UI to enable this feature and background endpoints to manage it

* create frontend views and paths
Move elements to correct experimental scope

* update migration to delete runs on removal of watched document

* Add watch support to YouTube transcripts (#1716)

* Add watch support to YouTube transcripts
refactor how sync is done for supported types

* Watch specific files in Confluence space (#1718)

Add failure-prune check for runs

* create tmp workflow modifications for beta image

* create tmp workflow modifications for beta image

* create tmp workflow modifications for beta image

* dual build
update copy of alert modals

* update job interval

* Add support for live-sync of Github files

* update copy for document sync feature

* hide Experimental features from UI

* update docs links

* [FEAT] Implement new settings menu for experimental features (#1735)

* implement new settings menu for experimental features

* remove unused context save bar

---------

Co-authored-by: timothycarambat <rambat1010@gmail.com>

* dont run job on boot

* unset workflow changes

* Add persistent encryption service
Relay key to collector so persistent encryption can be used
Encrypt any private data in chunkSources used for replay during resync jobs

* update jsDOC

* Linting and organization

* update modal copy for feature

---------

Co-authored-by: Sean Hatfield <seanhatfield5@gmail.com>
2024-06-21 13:38:50 -07:00
..
extensions [BETA] Live document sync (#1719) 2024-06-21 13:38:50 -07:00
hotdir Document Processor v2 (#442) 2023-12-14 15:14:56 -08:00
middleware [BETA] Live document sync (#1719) 2024-06-21 13:38:50 -07:00
processLink Agent support for @agent default agent inside workspace chat (#1093) 2024-04-16 10:50:10 -07:00
processRawText Add support to upload rawText document via api (#692) 2024-02-07 15:17:32 -08:00
processSingleFile Fix Cannot read properties of undefined (reading 'length') (#1145) 2024-04-20 12:28:19 -07:00
storage feat: Embed on-instance Whisper model for audio/mp4 transcribing (#449) 2023-12-15 11:20:13 -08:00
utils [BETA] Live document sync (#1719) 2024-06-21 13:38:50 -07:00
.env.example devcontainer v1 (#297) 2024-01-08 15:31:06 -08:00
.gitignore Document Processor v2 (#442) 2023-12-14 15:14:56 -08:00
.nvmrc Document Processor v2 (#442) 2023-12-14 15:14:56 -08:00
index.js Agent support for @agent default agent inside workspace chat (#1093) 2024-04-16 10:50:10 -07:00
nodemon.json Document Processor v2 (#442) 2023-12-14 15:14:56 -08:00
package.json Feature/devcontv2 (#1622) 2024-06-06 12:50:42 -07:00
yarn.lock Bump openai package to latest (#1234) 2024-04-30 12:33:42 -07:00