Merge branch 'master' of github.com:Mintplex-Labs/anything-llm into render

2024-11-19 20:50:09 +01:00 · 2024-05-08 14:12:33 -07:00 · 2024-05-08 14:12:33 -07:00 · 8aba28b02e
commit 8aba28b02e
parent c60077a078 b2b41db110
51 changed files with 1294 additions and 47 deletions
--- a/.vscode/settings.json
+++ b/.vscode/settings.json
@ -17,6 +17,7 @@
    "hljs",
    "inferencing",
    "Langchain",
+    "lmstudio",
    "mbox",
    "Milvus",
    "Mintplex",
--- a/BARE_METAL.md
+++ b/BARE_METAL.md
@ -30,6 +30,13 @@ Here you can find the scripts and known working process to run AnythingLLM outsi
 STORAGE_DIR="/your/absolute/path/to/server/.env"
 ```

+5. Edit the `frontend/.env` file for the `VITE_BASE_API` to now be set to `/api`. This is documented in the .env for which one you should use.
+```
+# VITE_API_BASE='http://localhost:3001/api' # Use this URL when developing locally
+# VITE_API_BASE="https://$CODESPACE_NAME-3001.$GITHUB_CODESPACES_PORT_FORWARDING_DOMAIN/api" # for Github Codespaces
+VITE_API_BASE='/api' # Use this URL deploying on non-localhost address OR in docker.
+```
+
 ## To start the application

 AnythingLLM is comprised of three main sections. The `frontend`, `server`, and `collector`. When running in production you will be running `server` and `collector` on two different processes, with a build step for compilation of the frontend.
--- a/README.md
+++ b/README.md
@ -24,6 +24,10 @@
  </a>
 </p>

+<p align="center">
+  <b>English</b> · <a href='/README.zh-CN.md'>简体中文</a>
+</p>
+
 <p align="center">
 👉 AnythingLLM for desktop (Mac, Windows, & Linux)! <a href="https://useanything.com/download" target="_blank"> Download Now</a>
 </p>
@ -65,6 +69,7 @@ Some cool features of AnythingLLM

 - [Any open-source llama.cpp compatible model](/server/storage/models/README.md#text-generation-llm-selection)
 - [OpenAI](https://openai.com)
+- [OpenAI (Generic)](https://openai.com)
 - [Azure OpenAI](https://azure.microsoft.com/en-us/products/ai-services/openai-service)
 - [Anthropic](https://www.anthropic.com/)
 - [Google Gemini Pro](https://ai.google.dev/)
@ -77,6 +82,9 @@ Some cool features of AnythingLLM
 - [OpenRouter (chat models)](https://openrouter.ai/)
 - [Mistral](https://mistral.ai/)
 - [Groq](https://groq.com/)
+- [Cohere](https://cohere.com/)
+- [KoboldCPP](https://github.com/LostRuins/koboldcpp)
+- [Text Generation Web UI](https://github.com/oobabooga/text-generation-webui)

 **Supported Embedding models:**

@ -86,6 +94,7 @@ Some cool features of AnythingLLM
 - [LocalAi (all)](https://localai.io/)
 - [Ollama (all)](https://ollama.ai/)
 - [LM Studio (all)](https://lmstudio.ai)
+- [Cohere](https://cohere.com/)

 **Supported Transcription models:**

@ -143,7 +152,6 @@ Mintplex Labs & the community maintain a number of deployment methods, scripts,
 - create PR with branch name format of `<issue number>-<short name>`
 - yee haw let's merge

-
 ## Telemetry & Privacy

 AnythingLLM by Mintplex Labs Inc contains a telemetry feature that collects anonymous usage information.
--- a/README.zh-CN.md
+++ b/README.zh-CN.md
@ -0,0 +1,223 @@
+<a name="readme-top"></a>
+
+<p align="center">
+  <a href="https://useanything.com"><img src="https://github.com/Mintplex-Labs/anything-llm/blob/master/images/wordmark.png?raw=true" alt="AnythingLLM logo"></a>
+</p>
+
+<p align="center">
+    <b>AnythingLLM：</b> 您一直在寻找的全方位AI应用程序。<br />
+    与您的文档聊天，使用AI代理，高度可配置，多用户，无需繁琐的设置。
+</p>
+
+<p align="center">
+	<a href="https://discord.gg/6UyHPeGZAC" target="_blank">
+      <img src="https://img.shields.io/badge/chat-mintplex_labs-blue.svg?style=flat&logo=data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAACAAAAAgCAMAAABEpIrGAAAAIGNIUk0AAHomAACAhAAA+gAAAIDoAAB1MAAA6mAAADqYAAAXcJy6UTwAAAH1UExURQAAAP////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////r6+ubn5+7u7/3+/v39/enq6urq6/v7+97f39rb26eoqT1BQ0pOT4+Rkuzs7cnKykZKS0NHSHl8fdzd3ejo6UxPUUBDRdzc3RwgIh8jJSAkJm5xcvHx8aanqB4iJFBTVezt7V5hYlJVVuLj43p9fiImKCMnKZKUlaaoqSElJ21wcfT09O3u7uvr6zE0Nr6/wCUpK5qcnf7+/nh7fEdKTHx+f0tPUOTl5aipqiouMGtubz5CRDQ4OsTGxufn515hY7a3uH1/gXBydIOFhlVYWvX29qaoqCQoKs7Pz/Pz87/AwUtOUNfY2dHR0mhrbOvr7E5RUy8zNXR2d/f39+Xl5UZJSx0hIzQ3Odra2/z8/GlsbaGjpERHSezs7L/BwScrLTQ4Odna2zM3Obm7u3x/gKSmp9jZ2T1AQu/v71pdXkVISr2+vygsLiInKTg7PaOlpisvMcXGxzk8PldaXPLy8u7u7rm6u7S1tsDBwvj4+MPExbe4ueXm5s/Q0Kyf7ewAAAAodFJOUwAABClsrNjx/QM2l9/7lhmI6jTB/kA1GgKJN+nea6vy/MLZQYeVKK3rVA5tAAAAAWJLR0QB/wIt3gAAAAd0SU1FB+cKBAAmMZBHjXIAAAISSURBVDjLY2CAAkYmZhZWNnYODnY2VhZmJkYGVMDIycXNw6sBBbw8fFycyEoYGfkFBDVQgKAAPyMjQl5IWEQDDYgIC8FUMDKKsmlgAWyiEBWMjGJY5YEqxMAqGMWFNXAAYXGgAkYJSQ2cQFKCkYFRShq3AmkpRgYJbghbU0tbB0Tr6ukbgGhDI10gySfBwCwDUWBsYmpmDqQtLK2sbTQ0bO3sHYA8GWYGWWj4WTs6Obu4ami4OTm7exhqeHp5+4DCVJZBDmqdr7ufn3+ArkZgkJ+fU3CIRmgYWFiOARYGvo5OQUHhEUAFTkF+kVHRsLBgkIeyYmLjwoOc4hMSk5JTnINS06DC8gwcEEZ6RqZGlpOfc3ZObl5+gZ+TR2ERWFyBQQFMF5eklmqUpQb5+ReU61ZUOvkFVVXXQBSAraitq29o1GiKcfLzc29u0mjxBzq0tQ0kww5xZHtHUGeXhkZhdxBYgZ4d0LI6c4gjwd7siQQraOp1AivQ6CuAKZCDBBRQQQNQgUb/BGf3cqCCiZOcnCe3QQIKHNRTpk6bDgpZjRkzg3pBQTBrdtCcuZCgluAD0vPmL1gIdvSixUuWgqNs2YJ+DUhkEYxuggkGmOQUcckrioPTJCOXEnZ5JS5YslbGnuyVERlDDFvGEUPOWvwqaH6RVkHKeuDMK6SKnHlVhTgx8jeTmqy6Eij7K6nLqiGyPwChsa1MUrnq1wAAACV0RVh0ZGF0ZTpjcmVhdGUAMjAyMy0xMC0wNFQwMDozODo0OSswMDowMB9V0a8AAAAldEVYdGRhdGU6bW9kaWZ5ADIwMjMtMTAtMDRUMDA6Mzg6NDkrMDA6MDBuCGkTAAAAKHRFWHRkYXRlOnRpbWVzdGFtcAAyMDIzLTEwLTA0VDAwOjM4OjQ5KzAwOjAwOR1IzAAAAABJRU5ErkJggg==" alt="Discord">
+  </a> |
+  <a href="https://github.com/Mintplex-Labs/anything-llm/blob/master/LICENSE" target="_blank">
+      <img src="https://img.shields.io/static/v1?label=license&message=MIT&color=white" alt="许可证">
+  </a> |
+  <a href="https://docs.useanything.com" target="_blank">
+    文档
+  </a> |
+  <a href="https://my.mintplexlabs.com/aio-checkout?product=anythingllm" target="_blank">
+    托管实例
+  </a>
+</p>
+
+<p align="center">
+  <a href='/README.md'>English</a> · <b>简体中文</b>
+</p>
+
+<p align="center">
+👉 适用于桌面（Mac、Windows和Linux）的AnythingLLM！<a href="https://useanything.com/download" target="_blank">立即下载</a>
+</p>
+
+这是一个全栈应用程序，可以将任何文档、资源（如网址链接、音频、视频）或内容片段转换为上下文，以便任何大语言模型（LLM）在聊天期间作为参考使用。此应用程序允许您选择使用哪个LLM或向量数据库，同时支持多用户管理并设置不同权限。
+
+![聊天](https://github.com/Mintplex-Labs/anything-llm/assets/16845892/cfc5f47c-bd91-4067-986c-f3f49621a859)
+
+<details>
+<summary><kbd>观看演示视频！</kbd></summary>
+
+[![观看视频](/images/youtube.png)](https://youtu.be/f95rGD9trL0)
+
+</details>
+
+### 产品概览
+
+AnythingLLM是一个全栈应用程序，您可以使用现成的商业大语言模型或流行的开源大语言模型，再结合向量数据库解决方案构建一个私有ChatGPT，不再受制于人：您可以本地运行，也可以远程托管，并能够与您提供的任何文档智能聊天。
+
+AnythingLLM将您的文档划分为称为`workspaces` (工作区)的对象。工作区的功能类似于线程，同时增加了文档的容器化，。工作区可以共享文档，但工作区之间的内容不会互相干扰或污染，因此您可以保持每个工作区的上下文清晰。
+
+AnythingLLM的一些酷炫特性
+
+- **多用户实例支持和权限管理**
+- 工作区内的智能体Agent（浏览网页、运行代码等）
+- [为您的网站定制的可嵌入聊天窗口](./embed/README.md)
+- 支持多种文档类型（PDF、TXT、DOCX等）
+- 通过简单的用户界面管理向量数据库中的文档
+- 两种对话模式：`聊天`和`查询`。聊天模式保留先前的对话记录。查询模式则是是针对您的文档做简单问答
+- 聊天中会提供所引用的相应文档内容
+- 100%云部署就绪。
+- “部署你自己的LLM模型”。
+- 管理超大文档时高效、低耗。只需要一次就可以嵌入（Embedding)一个庞大的文档或文字记录。比其他文档聊天机器人解决方案节省90%的成本。
+- 全套的开发人员API，用于自定义集成！
+
+### 支持的LLM、嵌入模型、转录模型和向量数据库
+
+**支持的LLM：**
+
+- [任何与llama.cpp兼容的开源模型](/server/storage/models/README.md#text-generation-llm-selection)
+- [OpenAI](https://openai.com)
+- [OpenAI (通用)](https://openai.com)
+- [Azure OpenAI](https://azure.microsoft.com/en-us/products/ai-services/openai-service)
+- [Anthropic](https://www.anthropic.com/)
+- [Google Gemini Pro](https://ai.google.dev/)
+- [Hugging Face (聊天模型)](https://huggingface.co/)
+- [Ollama (聊天模型)](https://ollama.ai/)
+- [LM Studio (所有模型)](https://lmstudio.ai)
+- [LocalAi (所有模型)](https://localai.io/)
+- [Together AI (聊天模型)](https://www.together.ai/)
+- [Perplexity (聊天模型)](https://www.perplexity.ai/)
+- [OpenRouter (聊天模型)](https://openrouter.ai/)
+- [Mistral](https://mistral.ai/)
+- [Groq](https://groq.com/)
+- [Cohere](https://cohere.com/)
+- [KoboldCPP](https://github.com/LostRuins/koboldcpp)
+
+**支持的嵌入模型：**
+
+- [AnythingLLM原生嵌入器](/server/storage/models/README.md)（默认）
+- [OpenAI](https://openai.com)
+- [Azure OpenAI](https://azure.microsoft.com/en-us/products/ai-services/openai-service)
+- [LocalAi (全部)](https://localai.io/)
+- [Ollama (全部)](https://ollama.ai/)
+- [LM Studio (全部)](https://lmstudio.ai)
+- [Cohere](https://cohere.com/)
+
+**支持的转录模型：**
+
+- [AnythingLLM内置](https://github.com/Mintplex-Labs/anything-llm/tree/master/server/storage/models#audiovideo-transcription) （默认）
+- [OpenAI](https://openai.com/)
+
+**支持的向量数据库：**
+
+- [LanceDB](https://github.com/lancedb/lancedb) （默认）
+- [Astra DB](https://www.datastax.com/products/datastax-astra)
+- [Pinecone](https://pinecone.io)
+- [Chroma](https://trychroma.com)
+- [Weaviate](https://weaviate.io)
+- [QDrant](https://qdrant.tech)
+- [Milvus](https://milvus.io)
+- [Zilliz](https://zilliz.com)
+
+### 技术概览
+
+这个单库由三个主要部分组成：
+
+- `frontend`: 一个viteJS + React前端，您可以运行它来轻松创建和管理LLM可以使用的所有内容。
+- `server`: 一个NodeJS express服务器，用于处理所有交互并进行所有向量数据库管理和LLM交互。
+- `docker`: Docker指令和构建过程 + 从源代码构建的信息。
+- `collector`: NodeJS express服务器，用于从UI处理和解析文档。
+
+## 🛳 自托管
+
+Mintplex Labs和社区维护了许多部署方法、脚本和模板，您可以使用它们在本地运行AnythingLLM。请参阅下面的表格，了解如何在您喜欢的环境上部署，或自动部署。
+| Docker | AWS | GCP | Digital Ocean | Render.com |
+|----------------------------------------|----:|-----|---------------|------------|
+| [![在Docker上部署][docker-btn]][docker-deploy] | [![在AWS上部署][aws-btn]][aws-deploy] | [![在GCP上部署][gcp-btn]][gcp-deploy] | [![在DigitalOcean上部署][do-btn]][do-deploy] | [![在Render.com上部署][render-btn]][render-deploy] |
+
+| Railway                                             |
+| --------------------------------------------------- |
+| [![在Railway上部署][railway-btn]][railway-deploy] |
+
+[其他方案：不使用Docker配置AnythingLLM实例 →](./BARE_METAL.md)
+
+## 如何设置开发环境
+
+- `yarn setup` 填充每个应用程序部分所需的`.env`文件（从仓库的根目录）。
+  - 在开始下一步之前，先填写这些信息`server/.env.development`，不然代码无法正常执行。
+- `yarn dev:server` 在本地启动服务器（从仓库的根目录）。
+- `yarn dev:frontend` 在本地启动前端（从仓库的根目录）。
+- `yarn dev:collector` 然后运行文档收集器（从仓库的根目录）。
+
+[了解文档](./server/storage/documents/DOCUMENTS.md)
+
+[了解向量缓存](./server/storage/vector-cache/VECTOR_CACHE.md)
+
+## 如何贡献
+
+- 创建issue
+- 创建PR，分支名称格式为`<issue number>-<short name>`
+- 然后合并
+
+## 远程信息收集与隐私保护
+
+由Mintplex Labs Inc开发的AnythingLLM包含一个收集匿名使用信息的Telemetry功能。
+
+<details>
+<summary><kbd>有关AnythingLLM的远程信息收集与隐私保护更多信息</kbd></summary>
+
+
+
+
+### 为什么收集信息？
+
+我们使用这些信息来帮助我们理解AnythingLLM的使用情况，帮助我们确定新功能和错误修复的优先级，并帮助我们提高AnythingLLM的性能和稳定性。
+
+### 怎样关闭
+
+通过在服务器或docker的.env设置中将`DISABLE_TELEMETRY`设置为“true”来选择退出Telemetry远程信息收集功能。您也可以进入AnythingLLM应用>>>侧边栏最下方 >>> `隐私和数据` （Privacy&Data)>>>找到最下方的Anonymous Telemetry Enabled，点击绿色按钮让它变灰色，从而禁用信息收集功能。
+
+### 你们跟踪收集哪些信息？
+
+我们只会跟踪有助于我们做出产品和路线图决策的使用细节，具体包括：
+
+- 您的安装方式（Docker或桌面版）
+- 文档被添加或移除的时间。但不包括文档内的具体内容。我们只关注添加或移除文档这个行为。这些信息能让我们了解到文档功能的使用情况。
+- 使用中的向量数据库类型。让我们知道哪个向量数据库最受欢迎，并在后续更新中优先考虑相应的数据库。
+- 使用中的LLM类型。让我们知道谁才是最受欢迎的LLM模型，并在后续更新中优先考虑相应模型。
+- 信息被`发送`出去。这是最常规的“事件/行为/event”，并让我们了解到所有安装了这个项目的每日活动情况。同样，只收集`发送`这个行为的信息，我们不会收集关于聊天本身的性质或内容的任何信息。
+
+您可以通过查找所有调用`Telemetry.sendTelemetry`的位置来验证这些声明。此外，如果启用，这些事件也会被写入输出日志，因此您也可以看到发送了哪些具体数据。不收集IP或其他识别信息。Telemetry远程信息收集的方案来自[PostHog](https://posthog.com/) - 一个开源的远程信息收集服务。
+
+[在源代码中查看所有信息收集活动](https://github.com/search?q=repo%3AMintplex-Labs%2Fanything-llm%20.sendTelemetry\(&type=code)
+
+</details>
+
+## 🔗 更多产品
+
+- **[VectorAdmin][vector-admin]**：一个用于管理向量数据库的全方位GUI和工具套件。
+- **[OpenAI Assistant Swarm][assistant-swarm]**：一个智能体Agent就可以管理您所有的OpenAI助手。
+
+<div align="right">
+
+[![][back-to-top]](#readme-top)
+
+</div>
+
+---
+
+版权所有 © 2024 [Mintplex Labs][profile-link]。<br />
+本项目采用[MIT](./LICENSE)许可证。
+
+<!-- LINK GROUP -->
+
+[back-to-top]: https://img.shields.io/badge/-BACK_TO_TOP-222628?style=flat-square
+[profile-link]: https://github.com/mintplex-labs
+[vector-admin]: https://github.com/mintplex-labs/vector-admin
+[assistant-swarm]: https://github.com/Mintplex-Labs/openai-assistant-swarm
+[docker-btn]: ./images/deployBtns/docker.png
+[docker-deploy]: ./docker/HOW_TO_USE_DOCKER.md
+[aws-btn]: ./images/deployBtns/aws.png
+[aws-deploy]: ./cloud-deployments/aws/cloudformation/DEPLOY.md
+[gcp-btn]: https://deploy.cloud.run/button.svg
+[gcp-deploy]: ./cloud-deployments/gcp/deployment/DEPLOY.md
+[do-btn]: https://www.deploytodo.com/do-btn-blue.svg
+[do-deploy]: ./cloud-deployments/digitalocean/terraform/DEPLOY.md
+[render-btn]: https://render.com/images/deploy-to-render-button.svg
+[render-deploy]: https://render.com/deploy?repo=https://github.com/Mintplex-Labs/anything-llm&branch=render
+[render-btn]: https://render.com/images/deploy-to-render-button.svg
+[render-deploy]: https://render.com/deploy?repo=https://github.com/Mintplex-Labs/anything-llm&branch=render
+[railway-btn]: https://railway.app/button.svg
+[railway-deploy]: https://railway.app/template/HNSCS1?referralCode=WFgJkn
--- a/docker/.env.example
+++ b/docker/.env.example
@ -71,6 +71,10 @@ GID='1000'
 # KOBOLD_CPP_MODEL_PREF='koboldcpp/codellama-7b-instruct.Q4_K_S'
 # KOBOLD_CPP_MODEL_TOKEN_LIMIT=4096

+# LLM_PROVIDER='textgenwebui'
+# TEXT_GEN_WEB_UI_BASE_PATH='http://127.0.0.1:5000/v1'
+# TEXT_GEN_WEB_UI_TOKEN_LIMIT=4096
+
 # LLM_PROVIDER='generic-openai'
 # GENERIC_OPEN_AI_BASE_PATH='http://proxy.url.openai.com/v1'
 # GENERIC_OPEN_AI_MODEL_PREF='gpt-3.5-turbo'
--- a/embed/README.md
+++ b/embed/README.md
@ -87,6 +87,12 @@ REQUIRED data attributes:

 - `data-assistant-icon` - Set the icon of the chat assistant.

+- `data-window-height` - Set the chat window height. **must include CSS suffix:** `px`,`%`,`rem`
+
+- `data-window-width` - Set the chat window width. **must include CSS suffix:** `px`,`%`,`rem`
+
+- `data-text-size` - Set the text size of the chats in pixels.
+
 **Behavior Overrides**

 - `data-open-on-load` — Once loaded, open the chat as default. It can still be closed by the user.
--- a/embed/index.html
+++ b/embed/index.html
@ -1,5 +1,6 @@
 <!doctype html>
 <html lang="en">
+
  <body>
    <h1>This is an example testing page for embedded AnythingLLM.</h1>
    <!--
@ -8,4 +9,5 @@
      </script>
    -->
  </body>
+
 </html>
--- a/embed/src/App.jsx
+++ b/embed/src/App.jsx
@ -27,13 +27,19 @@ export default function App() {
  };

  const position = embedSettings.position || "bottom-right";
+  const windowWidth = embedSettings.windowWidth
+    ? `md:max-w-[${embedSettings.windowWidth}]`
+    : "md:max-w-[400px]";
+  const windowHeight = embedSettings.windowHeight
+    ? `md:max-h-[${embedSettings.windowHeight}]`
+    : "md:max-h-[700px]";

  return (
    <>
      <Head />
      <div className={`fixed inset-0 z-50 ${isChatOpen ? "block" : "hidden"}`}>
        <div
-          className={`w-full h-full bg-white md:max-w-[400px] md:max-h-[700px] md:fixed md:bottom-0 md:right-0 md:mb-4 md:mr-4 md:rounded-2xl md:border md:border-gray-300 md:shadow-[0_4px_14px_rgba(0,0,0,0.25)] ${positionClasses[position]}`}
+          className={`${windowHeight} ${windowWidth} h-full w-full bg-white md:fixed md:bottom-0 md:right-0 md:mb-4 md:mr-4 md:rounded-2xl md:border md:border-gray-300 md:shadow-[0_4px_14px_rgba(0,0,0,0.25)] ${positionClasses[position]}`}
          id="anything-llm-chat"
        >
          {isChatOpen && (
--- a/embed/src/components/ChatWindow/ChatContainer/ChatHistory/HistoricalMessage/index.jsx
+++ b/embed/src/components/ChatWindow/ChatContainer/ChatHistory/HistoricalMessage/index.jsx
@ -13,6 +13,10 @@ const HistoricalMessage = forwardRef(
    { uuid = v4(), message, role, sources = [], error = false, sentAt },
    ref
  ) => {
+    const textSize = !!embedderSettings.settings.textSize
+      ? `text-[${embedderSettings.settings.textSize}px]`
+      : "text-sm";
+
    return (
      <div className="py-[5px]">
        {role === "assistant" && (
@ -61,7 +65,7 @@ const HistoricalMessage = forwardRef(
                </div>
              ) : (
                <span
-                  className={`whitespace-pre-line font-medium flex flex-col gap-y-1 text-sm leading-[20px]`}
+                  className={`whitespace-pre-line font-medium flex flex-col gap-y-1 ${textSize} leading-[20px]`}
                  dangerouslySetInnerHTML={{
                    __html: DOMPurify.sanitize(renderMarkdown(message)),
                  }}
--- a/embed/src/components/ChatWindow/ChatContainer/ChatHistory/index.jsx
+++ b/embed/src/components/ChatWindow/ChatContainer/ChatHistory/index.jsx
@ -58,7 +58,7 @@ export default function ChatHistory({ settings = {}, history = [] }) {

  return (
    <div
-      className="pb-[30px] pt-[5px] rounded-lg px-2 h-full gap-y-2 overflow-y-scroll flex flex-col justify-start no-scroll md:max-h-[500px] max-h-[calc(100vh-200px)]"
+      className="pb-[30px] pt-[5px] rounded-lg px-2 h-full gap-y-2 overflow-y-scroll flex flex-col justify-start no-scroll md:max-h-[500px]"
      id="chat-history"
      ref={chatHistoryRef}
    >
--- a/embed/src/components/ChatWindow/ChatContainer/PromptInput/index.jsx
+++ b/embed/src/components/ChatWindow/ChatContainer/PromptInput/index.jsx
@ -46,7 +46,7 @@ export default function PromptInput({
  };

  return (
-    <div className="w-full absolute left-0 bottom-[25px] z-10 flex justify-center items-center px-5">
+    <div className="w-full sticky bottom-0 z-10 flex justify-center items-center px-5 bg-white">
      <form
        onSubmit={handleSubmit}
        className="flex flex-col gap-y-1 rounded-t-lg w-full items-center justify-center"
--- a/embed/src/components/ChatWindow/ChatContainer/index.jsx
+++ b/embed/src/components/ChatWindow/ChatContainer/index.jsx
@ -26,6 +26,7 @@ export default function ChatContainer({

  const handleSubmit = async (event) => {
    event.preventDefault();
+
    if (!message || message === "") return false;

    const prevChatHistory = [
@ -39,7 +40,6 @@ export default function ChatContainer({
        animate: true,
      },
    ];
-
    setChatHistory(prevChatHistory);
    setMessage("");
    setLoadingResponse(true);
@ -72,12 +72,15 @@ export default function ChatContainer({
      );
      return;
    }
+
    loadingResponse === true && fetchReply();
  }, [loadingResponse, chatHistory]);

  return (
-    <div className="h-full w-full relative">
+    <div className="h-full w-full flex flex-col">
+      <div className="flex-grow overflow-y-auto">
        <ChatHistory settings={settings} history={chatHistory} />
+      </div>
      <PromptInput
        message={message}
        submit={handleSubmit}
--- a/embed/src/components/ChatWindow/index.jsx
+++ b/embed/src/components/ChatWindow/index.jsx
@ -32,6 +32,7 @@ export default function ChatWindow({ closeChat, settings, sessionId }) {
  }

  setEventDelegatorForCodeSnippets();
+
  return (
    <div className="flex flex-col h-full">
      <ChatWindowHeader
@ -41,12 +42,14 @@ export default function ChatWindow({ closeChat, settings, sessionId }) {
        closeChat={closeChat}
        setChatHistory={setChatHistory}
      />
+      <div className="flex-grow overflow-y-auto">
        <ChatContainer
          sessionId={sessionId}
          settings={settings}
          knownHistory={chatHistory}
        />
-      <div className="-mt-2 pb-6 h-fit gap-y-2 z-10">
+      </div>
+      <div className="mt-4 pb-4 h-fit gap-y-2 z-10">
        <Sponsor settings={settings} />
        <ResetChat
          setChatHistory={setChatHistory}
@ -64,6 +67,7 @@ export default function ChatWindow({ closeChat, settings, sessionId }) {
 function copyCodeSnippet(uuid) {
  const target = document.querySelector(`[data-code="${uuid}"]`);
  if (!target) return false;
+
  const markdown =
    target.parentElement?.parentElement?.querySelector(
      "pre:first-of-type"
@ -71,6 +75,7 @@ function copyCodeSnippet(uuid) {
  if (!markdown) return false;

  window.navigator.clipboard.writeText(markdown);
+
  target.classList.add("text-green-500");
  const originalText = target.innerHTML;
  target.innerText = "Copied!";
--- a/embed/src/hooks/useScriptAttributes.js
+++ b/embed/src/hooks/useScriptAttributes.js
@ -23,6 +23,9 @@ const DEFAULT_SETTINGS = {
  position: "bottom-right", // position of chat button/window
  assistantName: "AnythingLLM Chat Assistant", // default assistant name
  assistantIcon: null, // default assistant icon
+  windowHeight: null, // height of chat window in number:css-prefix
+  windowWidth: null, // width of chat window in number:css-prefix
+  textSize: null, // text size in px (number only)

  // behaviors
  openOnLoad: "off", // or "on"
--- a/frontend/public/embed/anythingllm-chat-widget.min.js
+++ b/frontend/public/embed/anythingllm-chat-widget.min.js
--- a/frontend/src/components/LLMSelection/TextGenWebUIOptions/index.jsx
+++ b/frontend/src/components/LLMSelection/TextGenWebUIOptions/index.jsx
@ -0,0 +1,37 @@
+export default function TextGenWebUIOptions({ settings }) {
+  return (
+    <div className="flex gap-4 flex-wrap">
+      <div className="flex flex-col w-60">
+        <label className="text-white text-sm font-semibold block mb-4">
+          Base URL
+        </label>
+        <input
+          type="url"
+          name="TextGenWebUIBasePath"
+          className="bg-zinc-900 text-white placeholder:text-white/20 text-sm rounded-lg focus:border-white block w-full p-2.5"
+          placeholder="http://127.0.0.1:5000/v1"
+          defaultValue={settings?.TextGenWebUIBasePath}
+          required={true}
+          autoComplete="off"
+          spellCheck={false}
+        />
+      </div>
+      <div className="flex flex-col w-60">
+        <label className="text-white text-sm font-semibold block mb-4">
+          Token context window
+        </label>
+        <input
+          type="number"
+          name="TextGenWebUITokenLimit"
+          className="bg-zinc-900 text-white placeholder:text-white/20 text-sm rounded-lg focus:border-white block w-full p-2.5"
+          placeholder="Content window limit (eg: 4096)"
+          min={1}
+          onScroll={(e) => e.target.blur()}
+          defaultValue={settings?.TextGenWebUITokenLimit}
+          required={true}
+          autoComplete="off"
+        />
+      </div>
+    </div>
+  );
+}
--- a/frontend/src/components/Sidebar/ActiveWorkspaces/ThreadContainer/ThreadItem/index.jsx
+++ b/frontend/src/components/Sidebar/ActiveWorkspaces/ThreadContainer/ThreadItem/index.jsx
@ -1,7 +1,13 @@
 import Workspace from "@/models/workspace";
 import paths from "@/utils/paths";
 import showToast from "@/utils/toast";
-import { DotsThree, PencilSimple, Trash } from "@phosphor-icons/react";
+import {
+  ArrowCounterClockwise,
+  DotsThree,
+  PencilSimple,
+  Trash,
+  X,
+} from "@phosphor-icons/react";
 import { useEffect, useRef, useState } from "react";
 import { useParams } from "react-router-dom";
 import truncate from "truncate";
@ -14,7 +20,9 @@ export default function ThreadItem({
  workspace,
  thread,
  onRemove,
+  toggleMarkForDeletion,
  hasNext,
+  ctrlPressed = false,
 }) {
  const { slug } = useParams();
  const optionsContainer = useRef(null);
@ -57,14 +65,30 @@ export default function ThreadItem({
      />
      <div className="flex w-full items-center justify-between pr-2 group relative">
        {thread.deleted ? (
-          <a className="w-full">
+          <div className="w-full flex justify-between">
+            <div className="w-full ">
              <p className={`text-left text-sm text-slate-400/50 italic`}>
                deleted thread
              </p>
-          </a>
+            </div>
+            {ctrlPressed && (
+              <button
+                type="button"
+                className="border-none"
+                onClick={() => toggleMarkForDeletion(thread.id)}
+              >
+                <ArrowCounterClockwise
+                  className="text-zinc-300 hover:text-white"
+                  size={18}
+                />
+              </button>
+            )}
+          </div>
        ) : (
          <a
-            href={window.location.pathname === linkTo ? "#" : linkTo}
+            href={
+              window.location.pathname === linkTo || ctrlPressed ? "#" : linkTo
+            }
            className="w-full"
            aria-current={isActive ? "page" : ""}
          >
@ -79,15 +103,30 @@ export default function ThreadItem({
        )}
        {!!thread.slug && !thread.deleted && (
          <div ref={optionsContainer}>
+            {ctrlPressed ? (
+              <button
+                type="button"
+                className="border-none"
+                onClick={() => toggleMarkForDeletion(thread.id)}
+              >
+                <X
+                  className="text-zinc-300 hover:text-white"
+                  weight="bold"
+                  size={18}
+                />
+              </button>
+            ) : (
              <div className="flex items-center w-fit group-hover:visible md:invisible gap-x-1">
                <button
                  type="button"
+                  className="border-none"
                  onClick={() => setShowOptions(!showOptions)}
                  aria-label="Thread options"
                >
                  <DotsThree className="text-slate-300" size={25} />
                </button>
              </div>
+            )}
            {showOptions && (
              <OptionsMenu
                containerRef={optionsContainer}
--- a/frontend/src/components/Sidebar/ActiveWorkspaces/ThreadContainer/index.jsx
+++ b/frontend/src/components/Sidebar/ActiveWorkspaces/ThreadContainer/index.jsx
@ -1,7 +1,7 @@
 import Workspace from "@/models/workspace";
 import paths from "@/utils/paths";
 import showToast from "@/utils/toast";
-import { Plus, CircleNotch } from "@phosphor-icons/react";
+import { Plus, CircleNotch, Trash } from "@phosphor-icons/react";
 import { useEffect, useState } from "react";
 import ThreadItem from "./ThreadItem";
 import { useParams } from "react-router-dom";
@ -10,6 +10,7 @@ export default function ThreadContainer({ workspace }) {
  const { threadSlug = null } = useParams();
  const [threads, setThreads] = useState([]);
  const [loading, setLoading] = useState(true);
+  const [ctrlPressed, setCtrlPressed] = useState(false);

  useEffect(() => {
    async function fetchThreads() {
@ -21,6 +22,43 @@ export default function ThreadContainer({ workspace }) {
    fetchThreads();
  }, [workspace.slug]);

+  // Enable toggling of meta-key (ctrl on win and cmd/fn on others)
+  useEffect(() => {
+    const handleKeyDown = (event) => {
+      if (["Control", "Meta"].includes(event.key)) {
+        setCtrlPressed((prev) => !prev);
+        // when toggling, unset bulk progress so
+        // previously marked threads that were never deleted
+        // come back to life.
+        setThreads((prev) =>
+          prev.map((t) => {
+            return { ...t, deleted: false };
+          })
+        );
+      }
+    };
+    window.addEventListener("keydown", handleKeyDown);
+    return () => {
+      window.removeEventListener("keydown", handleKeyDown);
+    };
+  }, []);
+
+  const toggleForDeletion = (id) => {
+    setThreads((prev) =>
+      prev.map((t) => {
+        if (t.id !== id) return t;
+        return { ...t, deleted: !t.deleted };
+      })
+    );
+  };
+
+  const handleDeleteAll = async () => {
+    const slugs = threads.filter((t) => t.deleted === true).map((t) => t.slug);
+    await Workspace.threads.deleteBulk(workspace.slug, slugs);
+    setThreads((prev) => prev.filter((t) => !t.deleted));
+    setCtrlPressed(false);
+  };
+
  function removeThread(threadId) {
    setThreads((prev) =>
      prev.map((_t) => {
@ -28,6 +66,12 @@ export default function ThreadContainer({ workspace }) {
        return { ..._t, deleted: true };
      })
    );
+
+    // Show thread was deleted, but then remove from threads entirely so it will
+    // not appear in bulk-selection.
+    setTimeout(() => {
+      setThreads((prev) => prev.filter((t) => !t.deleted));
+    }, 500);
  }

  if (loading) {
@ -58,6 +102,8 @@ export default function ThreadContainer({ workspace }) {
        <ThreadItem
          key={thread.slug}
          idx={i + 1}
+          ctrlPressed={ctrlPressed}
+          toggleMarkForDeletion={toggleForDeletion}
          activeIdx={activeThreadIdx}
          isActive={activeThreadIdx === i + 1}
          workspace={workspace}
@ -66,6 +112,11 @@ export default function ThreadContainer({ workspace }) {
          hasNext={i !== threads.length - 1}
        />
      ))}
+      <DeleteAllThreadButton
+        ctrlPressed={ctrlPressed}
+        threads={threads}
+        onDelete={handleDeleteAll}
+      />
      <NewThreadButton workspace={workspace} />
    </div>
  );
@ -113,3 +164,28 @@ function NewThreadButton({ workspace }) {
    </button>
  );
 }
+
+function DeleteAllThreadButton({ ctrlPressed, threads, onDelete }) {
+  if (!ctrlPressed || threads.filter((t) => t.deleted).length === 0)
+    return null;
+  return (
+    <button
+      type="button"
+      onClick={onDelete}
+      className="w-full relative flex h-[40px] items-center border-none hover:bg-red-400/20 rounded-lg group"
+    >
+      <div className="flex w-full gap-x-2 items-center pl-4">
+        <div className="bg-zinc-600 p-2 rounded-lg h-[24px] w-[24px] flex items-center justify-center">
+          <Trash
+            weight="bold"
+            size={14}
+            className="shrink-0 text-slate-100 group-hover:text-red-400"
+          />
+        </div>
+        <p className="text-white text-left text-sm group-hover:text-red-400">
+          Delete Selected
+        </p>
+      </div>
+    </button>
+  );
+}
--- a/frontend/src/hooks/useGetProvidersModels.js
+++ b/frontend/src/hooks/useGetProvidersModels.js
@ -2,7 +2,12 @@ import System from "@/models/system";
 import { useEffect, useState } from "react";

 // Providers which cannot use this feature for workspace<>model selection
-export const DISABLED_PROVIDERS = ["azure", "lmstudio", "native"];
+export const DISABLED_PROVIDERS = [
+  "azure",
+  "lmstudio",
+  "native",
+  "textgenwebui",
+];
 const PROVIDER_DEFAULT_MODELS = {
  openai: [],
  gemini: ["gemini-pro", "gemini-1.5-pro-latest"],
@ -34,6 +39,7 @@ const PROVIDER_DEFAULT_MODELS = {
    "command-nightly",
    "command-light-nightly",
  ],
+  textgenwebui: [],
 };

 // For togetherAi, which has a large model list - we subgroup the options
--- a/frontend/src/media/llmprovider/text-generation-webui.png
+++ b/frontend/src/media/llmprovider/text-generation-webui.png
--- a/frontend/src/models/workspaceThread.js
+++ b/frontend/src/models/workspaceThread.js
@ -62,6 +62,18 @@ const WorkspaceThread = {
      .then((res) => res.ok)
      .catch(() => false);
  },
+  deleteBulk: async function (workspaceSlug, threadSlugs = []) {
+    return await fetch(
+      `${API_BASE}/workspace/${workspaceSlug}/thread-bulk-delete`,
+      {
+        method: "DELETE",
+        body: JSON.stringify({ slugs: threadSlugs }),
+        headers: baseHeaders(),
+      }
+    )
+      .then((res) => res.ok)
+      .catch(() => false);
+  },
  chatHistory: async function (workspaceSlug, threadSlug) {
    const history = await fetch(
      `${API_BASE}/workspace/${workspaceSlug}/thread/${threadSlug}/chats`,
--- a/frontend/src/pages/Admin/Invitations/NewInviteModal/index.jsx
+++ b/frontend/src/pages/Admin/Invitations/NewInviteModal/index.jsx
@ -58,7 +58,7 @@ export default function NewInviteModal({ closeModal }) {
  }, []);

  return (
-    <div className="relative w-[500px] max-w-2xl max-h-full">
+    <div className="relative w-[500px] max-w-2xl max-h-full overflow-auto">
      <div className="relative bg-main-gradient rounded-lg shadow">
        <div className="flex items-start justify-between p-4 border-b rounded-t border-gray-500/50">
          <h3 className="text-xl font-semibold text-white">
--- a/frontend/src/pages/GeneralSettings/LLMPreference/index.jsx
+++ b/frontend/src/pages/GeneralSettings/LLMPreference/index.jsx
@ -18,6 +18,7 @@ import PerplexityLogo from "@/media/llmprovider/perplexity.png";
 import OpenRouterLogo from "@/media/llmprovider/openrouter.jpeg";
 import GroqLogo from "@/media/llmprovider/groq.png";
 import KoboldCPPLogo from "@/media/llmprovider/koboldcpp.png";
+import TextGenWebUILogo from "@/media/llmprovider/text-generation-webui.png";
 import CohereLogo from "@/media/llmprovider/cohere.png";
 import PreLoader from "@/components/Preloader";
 import OpenAiOptions from "@/components/LLMSelection/OpenAiOptions";
@ -40,6 +41,7 @@ import LLMItem from "@/components/LLMSelection/LLMItem";
 import { CaretUpDown, MagnifyingGlass, X } from "@phosphor-icons/react";
 import CTAButton from "@/components/lib/CTAButton";
 import KoboldCPPOptions from "@/components/LLMSelection/KoboldCPPOptions";
+import TextGenWebUIOptions from "@/components/LLMSelection/TextGenWebUIOptions";

 export const AVAILABLE_LLM_PROVIDERS = [
  {
@ -166,6 +168,14 @@ export const AVAILABLE_LLM_PROVIDERS = [
      "KoboldCPPTokenLimit",
    ],
  },
+  {
+    name: "Oobabooga Web UI",
+    value: "textgenwebui",
+    logo: TextGenWebUILogo,
+    options: (settings) => <TextGenWebUIOptions settings={settings} />,
+    description: "Run local LLMs using Oobabooga's Text Generation Web UI.",
+    requiredConfig: ["TextGenWebUIBasePath", "TextGenWebUITokenLimit"],
+  },
  {
    name: "Cohere",
    value: "cohere",
--- a/frontend/src/pages/OnboardingFlow/Steps/DataHandling/index.jsx
+++ b/frontend/src/pages/OnboardingFlow/Steps/DataHandling/index.jsx
@ -16,6 +16,7 @@ import PerplexityLogo from "@/media/llmprovider/perplexity.png";
 import OpenRouterLogo from "@/media/llmprovider/openrouter.jpeg";
 import GroqLogo from "@/media/llmprovider/groq.png";
 import KoboldCPPLogo from "@/media/llmprovider/koboldcpp.png";
+import TextGenWebUILogo from "@/media/llmprovider/text-generation-webui.png";
 import CohereLogo from "@/media/llmprovider/cohere.png";
 import ZillizLogo from "@/media/vectordbs/zilliz.png";
 import AstraDBLogo from "@/media/vectordbs/astraDB.png";
@ -146,6 +147,13 @@ export const LLM_SELECTION_PRIVACY = {
    ],
    logo: KoboldCPPLogo,
  },
+  textgenwebui: {
+    name: "Oobabooga Web UI",
+    description: [
+      "Your model and chats are only accessible on the server running the Oobabooga Text Generation Web UI",
+    ],
+    logo: TextGenWebUILogo,
+  },
  "generic-openai": {
    name: "Generic OpenAI compatible service",
    description: [
--- a/frontend/src/pages/OnboardingFlow/Steps/LLMPreference/index.jsx
+++ b/frontend/src/pages/OnboardingFlow/Steps/LLMPreference/index.jsx
@ -15,6 +15,7 @@ import PerplexityLogo from "@/media/llmprovider/perplexity.png";
 import OpenRouterLogo from "@/media/llmprovider/openrouter.jpeg";
 import GroqLogo from "@/media/llmprovider/groq.png";
 import KoboldCPPLogo from "@/media/llmprovider/koboldcpp.png";
+import TextGenWebUILogo from "@/media/llmprovider/text-generation-webui.png";
 import CohereLogo from "@/media/llmprovider/cohere.png";
 import OpenAiOptions from "@/components/LLMSelection/OpenAiOptions";
 import GenericOpenAiOptions from "@/components/LLMSelection/GenericOpenAiOptions";
@ -38,6 +39,7 @@ import paths from "@/utils/paths";
 import showToast from "@/utils/toast";
 import { useNavigate } from "react-router-dom";
 import KoboldCPPOptions from "@/components/LLMSelection/KoboldCPPOptions";
+import TextGenWebUIOptions from "@/components/LLMSelection/TextGenWebUIOptions";

 const TITLE = "LLM Preference";
 const DESCRIPTION =
@ -109,6 +111,13 @@ const LLMS = [
    options: (settings) => <KoboldCPPOptions settings={settings} />,
    description: "Run local LLMs using koboldcpp.",
  },
+  {
+    name: "Oobabooga Web UI",
+    value: "textgenwebui",
+    logo: TextGenWebUILogo,
+    options: (settings) => <TextGenWebUIOptions settings={settings} />,
+    description: "Run local LLMs using Oobabooga's Text Generation Web UI.",
+  },
  {
    name: "Together AI",
    value: "togetherai",
--- a/frontend/src/pages/WorkspaceSettings/AgentConfig/AgentLLMSelection/index.jsx
+++ b/frontend/src/pages/WorkspaceSettings/AgentConfig/AgentLLMSelection/index.jsx
@ -2,10 +2,11 @@ import React, { useEffect, useRef, useState } from "react";
 import AnythingLLMIcon from "@/media/logo/anything-llm-icon.png";
 import AgentLLMItem from "./AgentLLMItem";
 import { AVAILABLE_LLM_PROVIDERS } from "@/pages/GeneralSettings/LLMPreference";
-import { CaretUpDown, MagnifyingGlass, X } from "@phosphor-icons/react";
+import { CaretUpDown, Gauge, MagnifyingGlass, X } from "@phosphor-icons/react";
 import AgentModelSelection from "../AgentModelSelection";

-const ENABLED_PROVIDERS = ["openai", "anthropic"];
+const ENABLED_PROVIDERS = ["openai", "anthropic", "lmstudio", "ollama"];
+const WARN_PERFORMANCE = ["lmstudio", "ollama"];

 const LLM_DEFAULT = {
  name: "Please make a selection",
@ -62,6 +63,19 @@ export default function AgentLLMSelection({
  const selectedLLMObject = LLMS.find((llm) => llm.value === selectedLLM);
  return (
    <div className="border-b border-white/40 pb-8">
+      {WARN_PERFORMANCE.includes(selectedLLM) && (
+        <div className="flex flex-col md:flex-row md:items-center gap-x-2 text-white mb-4 bg-blue-800/30 w-fit rounded-lg px-4 py-2">
+          <div className="gap-x-2 flex items-center">
+            <Gauge className="shrink-0" size={25} />
+            <p className="text-sm">
+              Performance of LLMs that do not explicitly support tool-calling is
+              highly dependent on the model's capabilities and accuracy. Some
+              abilities may be limited or non-functional.
+            </p>
+          </div>
+        </div>
+      )}
+
      <div className="flex flex-col">
        <label htmlFor="name" className="block input-label">
          Workspace Agent LLM Provider
--- a/server/.env.example
+++ b/server/.env.example
@ -68,6 +68,10 @@ JWT_SECRET="my-random-string-for-seeding" # Please generate random string at lea
 # KOBOLD_CPP_MODEL_PREF='koboldcpp/codellama-7b-instruct.Q4_K_S'
 # KOBOLD_CPP_MODEL_TOKEN_LIMIT=4096

+# LLM_PROVIDER='textgenwebui'
+# TEXT_GEN_WEB_UI_BASE_PATH='http://127.0.0.1:5000/v1'
+# TEXT_GEN_WEB_UI_TOKEN_LIMIT=4096
+
 # LLM_PROVIDER='generic-openai'
 # GENERIC_OPEN_AI_BASE_PATH='http://proxy.url.openai.com/v1'
 # GENERIC_OPEN_AI_MODEL_PREF='gpt-3.5-turbo'
--- a/server/endpoints/agentWebsocket.js
+++ b/server/endpoints/agentWebsocket.js
@ -51,7 +51,7 @@ function agentWebsocket(app) {
      await agentHandler.createAIbitat({ socket });
      await agentHandler.startAgentCluster();
    } catch (e) {
-      console.error(e.message);
+      console.error(e.message, e);
      socket?.send(JSON.stringify({ type: "wssFailure", content: e.message }));
      socket?.close();
    }
--- a/server/endpoints/workspaceThreads.js
+++ b/server/endpoints/workspaceThreads.js
@ -92,6 +92,29 @@ function workspaceThreadEndpoints(app) {
    }
  );

+  app.delete(
+    "/workspace/:slug/thread-bulk-delete",
+    [validatedRequest, flexUserRoleValid([ROLES.all]), validWorkspaceSlug],
+    async (request, response) => {
+      try {
+        const { slugs = [] } = reqBody(request);
+        if (slugs.length === 0) return response.sendStatus(200).end();
+
+        const user = await userFromSession(request, response);
+        const workspace = response.locals.workspace;
+        await WorkspaceThread.delete({
+          slug: { in: slugs },
+          user_id: user?.id ?? null,
+          workspace_id: workspace.id,
+        });
+        response.sendStatus(200).end();
+      } catch (e) {
+        console.log(e.message, e);
+        response.sendStatus(500).end();
+      }
+    }
+  );
+
  app.get(
    "/workspace/:slug/thread/:threadSlug/chats",
    [
--- a/server/models/systemSettings.js
+++ b/server/models/systemSettings.js
@ -368,6 +368,10 @@ const SystemSettings = {
      KoboldCPPBasePath: process.env.KOBOLD_CPP_BASE_PATH,
      KoboldCPPTokenLimit: process.env.KOBOLD_CPP_MODEL_TOKEN_LIMIT,

+      // Text Generation Web UI Keys
+      TextGenWebUIBasePath: process.env.TEXT_GEN_WEB_UI_BASE_PATH,
+      TextGenWebUITokenLimit: process.env.TEXT_GEN_WEB_UI_MODEL_TOKEN_LIMIT,
+
      // Generic OpenAI Keys
      GenericOpenAiBasePath: process.env.GENERIC_OPEN_AI_BASE_PATH,
      GenericOpenAiModelPref: process.env.GENERIC_OPEN_AI_MODEL_PREF,
--- a/server/models/workspaceThread.js
+++ b/server/models/workspaceThread.js
@ -61,7 +61,7 @@ const WorkspaceThread = {

  delete: async function (clause = {}) {
    try {
-      await prisma.workspace_threads.delete({
+      await prisma.workspace_threads.deleteMany({
        where: clause,
      });
      return true;
--- a/server/package.json
+++ b/server/package.json
@ -46,11 +46,13 @@
    "dotenv": "^16.0.3",
    "express": "^4.18.2",
    "express-ws": "^5.0.2",
+    "extract-json-from-string": "^1.0.1",
    "extract-zip": "^2.0.1",
    "graphql": "^16.7.1",
    "joi": "^17.11.0",
    "joi-password-complexity": "^5.2.0",
    "js-tiktoken": "^1.0.7",
+    "jsonrepair": "^3.7.0",
    "jsonwebtoken": "^8.5.1",
    "langchain": "0.1.36",
    "mime": "^3.0.0",
@ -58,6 +60,7 @@
    "multer": "^1.4.5-lts.1",
    "node-html-markdown": "^1.3.0",
    "node-llama-cpp": "^2.8.0",
+    "ollama": "^0.5.0",
    "openai": "4.38.5",
    "pinecone-client": "^1.1.0",
    "pluralize": "^8.0.0",
--- a/server/utils/AiProviders/textGenWebUI/index.js
+++ b/server/utils/AiProviders/textGenWebUI/index.js
@ -0,0 +1,131 @@
+const { NativeEmbedder } = require("../../EmbeddingEngines/native");
+const {
+  handleDefaultStreamResponseV2,
+} = require("../../helpers/chat/responses");
+
+class TextGenWebUILLM {
+  constructor(embedder = null) {
+    const { OpenAI: OpenAIApi } = require("openai");
+    if (!process.env.TEXT_GEN_WEB_UI_BASE_PATH)
+      throw new Error(
+        "TextGenWebUI must have a valid base path to use for the api."
+      );
+
+    this.basePath = process.env.TEXT_GEN_WEB_UI_BASE_PATH;
+    this.openai = new OpenAIApi({
+      baseURL: this.basePath,
+      apiKey: null,
+    });
+    this.model = null;
+    this.limits = {
+      history: this.promptWindowLimit() * 0.15,
+      system: this.promptWindowLimit() * 0.15,
+      user: this.promptWindowLimit() * 0.7,
+    };
+
+    this.embedder = !embedder ? new NativeEmbedder() : embedder;
+    this.defaultTemp = 0.7;
+    this.log(`Inference API: ${this.basePath} Model: ${this.model}`);
+  }
+
+  log(text, ...args) {
+    console.log(`\x1b[36m[${this.constructor.name}]\x1b[0m ${text}`, ...args);
+  }
+
+  #appendContext(contextTexts = []) {
+    if (!contextTexts || !contextTexts.length) return "";
+    return (
+      "\nContext:\n" +
+      contextTexts
+        .map((text, i) => {
+          return `[CONTEXT ${i}]:\n${text}\n[END CONTEXT ${i}]\n\n`;
+        })
+        .join("")
+    );
+  }
+
+  streamingEnabled() {
+    return "streamGetChatCompletion" in this;
+  }
+
+  // Ensure the user set a value for the token limit
+  // and if undefined - assume 4096 window.
+  promptWindowLimit() {
+    const limit = process.env.TEXT_GEN_WEB_UI_MODEL_TOKEN_LIMIT || 4096;
+    if (!limit || isNaN(Number(limit)))
+      throw new Error("No token context limit was set.");
+    return Number(limit);
+  }
+
+  // Short circuit since we have no idea if the model is valid or not
+  // in pre-flight for generic endpoints
+  isValidChatCompletionModel(_modelName = "") {
+    return true;
+  }
+
+  constructPrompt({
+    systemPrompt = "",
+    contextTexts = [],
+    chatHistory = [],
+    userPrompt = "",
+  }) {
+    const prompt = {
+      role: "system",
+      content: `${systemPrompt}${this.#appendContext(contextTexts)}`,
+    };
+    return [prompt, ...chatHistory, { role: "user", content: userPrompt }];
+  }
+
+  async isSafe(_input = "") {
+    // Not implemented so must be stubbed
+    return { safe: true, reasons: [] };
+  }
+
+  async getChatCompletion(messages = null, { temperature = 0.7 }) {
+    const result = await this.openai.chat.completions
+      .create({
+        model: this.model,
+        messages,
+        temperature,
+      })
+      .catch((e) => {
+        throw new Error(e.response.data.error.message);
+      });
+
+    if (!result.hasOwnProperty("choices") || result.choices.length === 0)
+      return null;
+    return result.choices[0].message.content;
+  }
+
+  async streamGetChatCompletion(messages = null, { temperature = 0.7 }) {
+    const streamRequest = await this.openai.chat.completions.create({
+      model: this.model,
+      stream: true,
+      messages,
+      temperature,
+    });
+    return streamRequest;
+  }
+
+  handleStream(response, stream, responseProps) {
+    return handleDefaultStreamResponseV2(response, stream, responseProps);
+  }
+
+  // Simple wrapper for dynamic embedder & normalize interface for all LLM implementations
+  async embedTextInput(textInput) {
+    return await this.embedder.embedTextInput(textInput);
+  }
+  async embedChunks(textChunks = []) {
+    return await this.embedder.embedChunks(textChunks);
+  }
+
+  async compressMessages(promptArgs = {}, rawHistory = []) {
+    const { messageArrayCompressor } = require("../../helpers/chat");
+    const messageArray = this.constructPrompt(promptArgs);
+    return await messageArrayCompressor(this, messageArray, rawHistory);
+  }
+}
+
+module.exports = {
+  TextGenWebUILLM,
+};
--- a/server/utils/agents/aibitat/index.js
+++ b/server/utils/agents/aibitat/index.js
@ -603,6 +603,18 @@ ${this.getHistory({ to: route.to })

      // Execute the function and return the result to the provider
      fn.caller = byAgent || "agent";
+
+      // For OSS LLMs we really need to keep tabs on what they are calling
+      // so we can log it here.
+      if (provider?.verbose) {
+        this?.introspect?.(
+          `[debug]: ${fn.caller} is attempting to call \`${name}\` tool`
+        );
+        this.handlerProps.log(
+          `[debug]: ${fn.caller} is attempting to call \`${name}\` tool`
+        );
+      }
+
      const result = await fn.handler(args);
      Telemetry.sendTelemetry("agent_tool_call", { tool: name }, null, true);
      return await this.handleExecution(
@ -727,6 +739,10 @@ ${this.getHistory({ to: route.to })
        return new Providers.OpenAIProvider({ model: config.model });
      case "anthropic":
        return new Providers.AnthropicProvider({ model: config.model });
+      case "lmstudio":
+        return new Providers.LMStudioProvider({});
+      case "ollama":
+        return new Providers.OllamaProvider({ model: config.model });

      default:
        throw new Error(
--- a/server/utils/agents/aibitat/plugins/memory.js
+++ b/server/utils/agents/aibitat/plugins/memory.js
@ -16,7 +16,37 @@ const memory = {
          tracker: new Deduplicator(),
          name: this.name,
          description:
-            "Search against local documents for context that is relevant to the query or store a snippet of text into memory for retrieval later. Storing information should only be done when the user specifically requests for information to be remembered or saved to long-term memory. You should use this tool before search the internet for information.",
+            "Search against local documents for context that is relevant to the query or store a snippet of text into memory for retrieval later. Storing information should only be done when the user specifically requests for information to be remembered or saved to long-term memory. You should use this tool before search the internet for information. Do not use this tool unless you are explicity told to 'remember' or 'store' information.",
+          examples: [
+            {
+              prompt: "What is AnythingLLM?",
+              call: JSON.stringify({
+                action: "search",
+                content: "What is AnythingLLM?",
+              }),
+            },
+            {
+              prompt: "What do you know about Plato's motives?",
+              call: JSON.stringify({
+                action: "search",
+                content: "What are the facts about Plato's motives?",
+              }),
+            },
+            {
+              prompt: "Remember that you are a robot",
+              call: JSON.stringify({
+                action: "store",
+                content: "I am a robot, the user told me that i am.",
+              }),
+            },
+            {
+              prompt: "Save that to memory please.",
+              call: JSON.stringify({
+                action: "store",
+                content: "<insert summary of conversation until now>",
+              }),
+            },
+          ],
          parameters: {
            $schema: "http://json-schema.org/draft-07/schema#",
            type: "object",
--- a/server/utils/agents/aibitat/plugins/save-file-browser.js
+++ b/server/utils/agents/aibitat/plugins/save-file-browser.js
@ -16,6 +16,32 @@ const saveFileInBrowser = {
          name: this.name,
          description:
            "Save content to a file when the user explicity asks for a download of the file.",
+          examples: [
+            {
+              prompt: "Save me that to a file named 'output'",
+              call: JSON.stringify({
+                file_content:
+                  "<content of the file we will write previous conversation>",
+                filename: "output.txt",
+              }),
+            },
+            {
+              prompt: "Save me that to my desktop",
+              call: JSON.stringify({
+                file_content:
+                  "<content of the file we will write previous conversation>",
+                filename: "<relevant filename>.txt",
+              }),
+            },
+            {
+              prompt: "Save me that to a file",
+              call: JSON.stringify({
+                file_content:
+                  "<content of the file we will write from previous conversation>",
+                filename: "<descriptive filename>.txt",
+              }),
+            },
+          ],
          parameters: {
            $schema: "http://json-schema.org/draft-07/schema#",
            type: "object",
--- a/server/utils/agents/aibitat/plugins/summarize.js
+++ b/server/utils/agents/aibitat/plugins/summarize.js
@ -19,6 +19,26 @@ const docSummarizer = {
          controller: new AbortController(),
          description:
            "Can get the list of files available to search with descriptions and can select a single file to open and summarize.",
+          examples: [
+            {
+              prompt: "Summarize example.txt",
+              call: JSON.stringify({
+                action: "summarize",
+                document_filename: "example.txt",
+              }),
+            },
+            {
+              prompt: "What files can you see?",
+              call: JSON.stringify({ action: "list", document_filename: null }),
+            },
+            {
+              prompt: "Tell me about readme.md",
+              call: JSON.stringify({
+                action: "summarize",
+                document_filename: "readme.md",
+              }),
+            },
+          ],
          parameters: {
            $schema: "http://json-schema.org/draft-07/schema#",
            type: "object",
--- a/server/utils/agents/aibitat/plugins/web-browsing.js
+++ b/server/utils/agents/aibitat/plugins/web-browsing.js
@ -13,7 +13,21 @@ const webBrowsing = {
          super: aibitat,
          name: this.name,
          description:
-            "Searches for a given query online using a search engine.",
+            "Searches for a given query using a search engine to get better results for the user query.",
+          examples: [
+            {
+              prompt: "Who won the world series today?",
+              call: JSON.stringify({ query: "Winner of today's world series" }),
+            },
+            {
+              prompt: "What is AnythingLLM?",
+              call: JSON.stringify({ query: "AnythingLLM" }),
+            },
+            {
+              prompt: "Current AAPL stock price",
+              call: JSON.stringify({ query: "AAPL stock price today" }),
+            },
+          ],
          parameters: {
            $schema: "http://json-schema.org/draft-07/schema#",
            type: "object",
--- a/server/utils/agents/aibitat/plugins/web-scraping.js
+++ b/server/utils/agents/aibitat/plugins/web-scraping.js
@ -16,7 +16,17 @@ const webScraping = {
          name: this.name,
          controller: new AbortController(),
          description:
-            "Scrapes the content of a webpage or online resource from a URL.",
+            "Scrapes the content of a webpage or online resource from a provided URL.",
+          examples: [
+            {
+              prompt: "What is useanything.com about?",
+              call: JSON.stringify({ uri: "https://useanything.com" }),
+            },
+            {
+              prompt: "Scrape https://example.com",
+              call: JSON.stringify({ uri: "https://example.com" }),
+            },
+          ],
          parameters: {
            $schema: "http://json-schema.org/draft-07/schema#",
            type: "object",
@ -24,7 +34,8 @@ const webScraping = {
              url: {
                type: "string",
                format: "uri",
-                description: "A web URL.",
+                description:
+                  "A complete web address URL including protocol. Assumes https if not provided.",
              },
            },
            additionalProperties: false,
--- a/server/utils/agents/aibitat/providers/ai-provider.js
+++ b/server/utils/agents/aibitat/providers/ai-provider.js
@ -4,16 +4,25 @@

 const { ChatOpenAI } = require("@langchain/openai");
 const { ChatAnthropic } = require("@langchain/anthropic");
+const DEFAULT_WORKSPACE_PROMPT =
+  "You are a helpful ai assistant who can assist the user and use tools available to help answer the users prompts and questions.";

 class Provider {
  _client;
  constructor(client) {
    if (this.constructor == Provider) {
-      throw new Error("Class is of abstract type and can't be instantiated");
+      return;
    }
    this._client = client;
  }

+  providerLog(text, ...args) {
+    console.log(
+      `\x1b[36m[AgentLLM${this?.model ? ` - ${this.model}` : ""}]\x1b[0m ${text}`,
+      ...args
+    );
+  }
+
  get client() {
    return this._client;
  }
@ -48,6 +57,15 @@ class Provider {
        return 8_000;
    }
  }
+
+  static systemPrompt(provider = null) {
+    switch (provider) {
+      case "lmstudio":
+        return "You are a helpful ai assistant who can assist the user and use tools available to help answer the users prompts and questions. Tools will be handled by another assistant and you will simply receive their responses to help answer the user prompt - always try to answer the user's prompt the best you can with the context available to you and your general knowledge.";
+      default:
+        return DEFAULT_WORKSPACE_PROMPT;
+    }
+  }
 }

 module.exports = Provider;
--- a/server/utils/agents/aibitat/providers/helpers/classes.js
+++ b/server/utils/agents/aibitat/providers/helpers/classes.js
@ -0,0 +1,16 @@
+function InheritMultiple(bases = []) {
+  class Bases {
+    constructor() {
+      bases.forEach((base) => Object.assign(this, new base()));
+    }
+  }
+
+  bases.forEach((base) => {
+    Object.getOwnPropertyNames(base.prototype)
+      .filter((prop) => prop != "constructor")
+      .forEach((prop) => (Bases.prototype[prop] = base.prototype[prop]));
+  });
+  return Bases;
+}
+
+module.exports = InheritMultiple;
--- a/server/utils/agents/aibitat/providers/helpers/untooled.js
+++ b/server/utils/agents/aibitat/providers/helpers/untooled.js
@ -0,0 +1,154 @@
+const { safeJsonParse } = require("../../../../http");
+const { Deduplicator } = require("../../utils/dedupe");
+
+// Useful inheritance class for a model which supports OpenAi schema for API requests
+// but does not have tool-calling or JSON output support.
+class UnTooled {
+  constructor() {
+    this.deduplicator = new Deduplicator();
+  }
+
+  cleanMsgs(messages) {
+    const modifiedMessages = [];
+    messages.forEach((msg) => {
+      if (msg.role === "function") {
+        const prevMsg = modifiedMessages[modifiedMessages.length - 1].content;
+        modifiedMessages[modifiedMessages.length - 1].content =
+          `${prevMsg}\n${msg.content}`;
+        return;
+      }
+      modifiedMessages.push(msg);
+    });
+    return modifiedMessages;
+  }
+
+  showcaseFunctions(functions = []) {
+    let output = "";
+    functions.forEach((def) => {
+      let shotExample = `-----------
+Function name: ${def.name}
+Function Description: ${def.description}
+Function parameters in JSON format:
+${JSON.stringify(def.parameters.properties, null, 4)}\n`;
+
+      if (Array.isArray(def.examples)) {
+        def.examples.forEach(({ prompt, call }) => {
+          shotExample += `Query: "${prompt}"\nJSON: ${call}\n`;
+        });
+      }
+      output += `${shotExample}-----------\n`;
+    });
+    return output;
+  }
+
+  /**
+   * Check if two arrays of strings or numbers have the same values
+   * @param {string[]|number[]} arr1
+   * @param {string[]|number[]} arr2
+   * @param {Object} [opts]
+   * @param {boolean} [opts.enforceOrder] - By default (false), the order of the values in the arrays doesn't matter.
+   * @return {boolean}
+   */
+  compareArrays(arr1, arr2, opts) {
+    function vKey(i, v) {
+      return (opts?.enforceOrder ? `${i}-` : "") + `${typeof v}-${v}`;
+    }
+
+    if (arr1.length !== arr2.length) return false;
+
+    const d1 = {};
+    const d2 = {};
+    for (let i = arr1.length - 1; i >= 0; i--) {
+      d1[vKey(i, arr1[i])] = true;
+      d2[vKey(i, arr2[i])] = true;
+    }
+
+    for (let i = arr1.length - 1; i >= 0; i--) {
+      const v = vKey(i, arr1[i]);
+      if (d1[v] !== d2[v]) return false;
+    }
+
+    for (let i = arr2.length - 1; i >= 0; i--) {
+      const v = vKey(i, arr2[i]);
+      if (d1[v] !== d2[v]) return false;
+    }
+
+    return true;
+  }
+
+  validFuncCall(functionCall = {}, functions = []) {
+    if (
+      !functionCall ||
+      !functionCall?.hasOwnProperty("name") ||
+      !functionCall?.hasOwnProperty("arguments")
+    ) {
+      return {
+        valid: false,
+        reason: "Missing name or arguments in function call.",
+      };
+    }
+
+    const foundFunc = functions.find((def) => def.name === functionCall.name);
+    if (!foundFunc) {
+      return { valid: false, reason: "Function name does not exist." };
+    }
+
+    const props = Object.keys(foundFunc.parameters.properties);
+    const fProps = Object.keys(functionCall.arguments);
+    if (!this.compareArrays(props, fProps)) {
+      return { valid: false, reason: "Invalid argument schema match." };
+    }
+
+    return { valid: true, reason: null };
+  }
+
+  async functionCall(messages, functions, chatCb = null) {
+    const history = [...messages].filter((msg) =>
+      ["user", "assistant"].includes(msg.role)
+    );
+    if (history[history.length - 1].role !== "user") return null;
+    const response = await chatCb({
+      messages: [
+        {
+          content: `You are a program which picks the most optimal function and parameters to call. 
+      DO NOT HAVE TO PICK A FUNCTION IF IT WILL NOT HELP ANSWER OR FULFILL THE USER'S QUERY.
+      When a function is selection, respond in JSON with no additional text.
+      When there is no relevant function to call - return with a regular chat text response.
+      Your task is to pick a **single** function that we will use to call, if any seem useful or relevant for the user query.
+
+      All JSON responses should have two keys.
+      'name': this is the name of the function name to call. eg: 'web-scraper', 'rag-memory', etc..
+      'arguments': this is an object with the function properties to invoke the function.
+      DO NOT INCLUDE ANY OTHER KEYS IN JSON RESPONSES.
+
+      Here are the available tools you can use an examples of a query and response so you can understand how each one works.
+      ${this.showcaseFunctions(functions)}
+
+      Now pick a function if there is an appropriate one to use given the last user message and the given conversation so far.`,
+          role: "system",
+        },
+        ...history,
+      ],
+    });
+
+    const call = safeJsonParse(response, null);
+    if (call === null) return { toolCall: null, text: response }; // failed to parse, so must be text.
+
+    const { valid, reason } = this.validFuncCall(call, functions);
+    if (!valid) {
+      this.providerLog(`Invalid function tool call: ${reason}.`);
+      return { toolCall: null, text: null };
+    }
+
+    if (this.deduplicator.isDuplicate(call.name, call.arguments)) {
+      this.providerLog(
+        `Function tool with exact arguments has already been called this stack.`
+      );
+      return { toolCall: null, text: null };
+    }
+
+    return { toolCall: call, text: null };
+  }
+}
+
+module.exports = UnTooled;
--- a/server/utils/agents/aibitat/providers/index.js
+++ b/server/utils/agents/aibitat/providers/index.js
@ -1,7 +1,11 @@
 const OpenAIProvider = require("./openai.js");
 const AnthropicProvider = require("./anthropic.js");
+const LMStudioProvider = require("./lmstudio.js");
+const OllamaProvider = require("./ollama.js");

 module.exports = {
  OpenAIProvider,
  AnthropicProvider,
+  LMStudioProvider,
+  OllamaProvider,
 };
--- a/server/utils/agents/aibitat/providers/lmstudio.js
+++ b/server/utils/agents/aibitat/providers/lmstudio.js
@ -0,0 +1,113 @@
+const OpenAI = require("openai");
+const Provider = require("./ai-provider.js");
+const InheritMultiple = require("./helpers/classes.js");
+const UnTooled = require("./helpers/untooled.js");
+
+/**
+ * The provider for the LMStudio provider.
+ */
+class LMStudioProvider extends InheritMultiple([Provider, UnTooled]) {
+  model;
+
+  constructor(_config = {}) {
+    super();
+    const model = process.env.LMSTUDIO_MODEL_PREF || "Loaded from Chat UI";
+    const client = new OpenAI({
+      baseURL: process.env.LMSTUDIO_BASE_PATH?.replace(/\/+$/, ""), // here is the URL to your LMStudio instance
+      apiKey: null,
+      maxRetries: 3,
+      model,
+    });
+    this._client = client;
+    this.model = model;
+    this.verbose = true;
+  }
+
+  get client() {
+    return this._client;
+  }
+
+  async #handleFunctionCallChat({ messages = [] }) {
+    return await this.client.chat.completions
+      .create({
+        model: this.model,
+        temperature: 0,
+        messages,
+      })
+      .then((result) => {
+        if (!result.hasOwnProperty("choices"))
+          throw new Error("LMStudio chat: No results!");
+        if (result.choices.length === 0)
+          throw new Error("LMStudio chat: No results length!");
+        return result.choices[0].message.content;
+      })
+      .catch((_) => {
+        return null;
+      });
+  }
+
+  /**
+   * Create a completion based on the received messages.
+   *
+   * @param messages A list of messages to send to the API.
+   * @param functions
+   * @returns The completion.
+   */
+  async complete(messages, functions = null) {
+    try {
+      let completion;
+      if (functions.length > 0) {
+        const { toolCall, text } = await this.functionCall(
+          messages,
+          functions,
+          this.#handleFunctionCallChat.bind(this)
+        );
+
+        if (toolCall !== null) {
+          this.providerLog(`Valid tool call found - running ${toolCall.name}.`);
+          this.deduplicator.trackRun(toolCall.name, toolCall.arguments);
+          return {
+            result: null,
+            functionCall: {
+              name: toolCall.name,
+              arguments: toolCall.arguments,
+            },
+            cost: 0,
+          };
+        }
+        completion = { content: text };
+      }
+
+      if (!completion?.content) {
+        this.providerLog(
+          "Will assume chat completion without tool call inputs."
+        );
+        const response = await this.client.chat.completions.create({
+          model: this.model,
+          messages: this.cleanMsgs(messages),
+        });
+        completion = response.choices[0].message;
+      }
+
+      return {
+        result: completion.content,
+        cost: 0,
+      };
+    } catch (error) {
+      throw error;
+    }
+  }
+
+  /**
+   * Get the cost of the completion.
+   *
+   * @param _usage The completion to get the cost for.
+   * @returns The cost of the completion.
+   * Stubbed since LMStudio has no cost basis.
+   */
+  getCost(_usage) {
+    return 0;
+  }
+}
+
+module.exports = LMStudioProvider;
--- a/server/utils/agents/aibitat/providers/ollama.js
+++ b/server/utils/agents/aibitat/providers/ollama.js
@ -0,0 +1,107 @@
+const Provider = require("./ai-provider.js");
+const InheritMultiple = require("./helpers/classes.js");
+const UnTooled = require("./helpers/untooled.js");
+const { Ollama } = require("ollama");
+
+/**
+ * The provider for the Ollama provider.
+ */
+class OllamaProvider extends InheritMultiple([Provider, UnTooled]) {
+  model;
+
+  constructor(config = {}) {
+    const {
+      // options = {},
+      model = null,
+    } = config;
+
+    super();
+    this._client = new Ollama({ host: process.env.OLLAMA_BASE_PATH });
+    this.model = model;
+    this.verbose = true;
+  }
+
+  get client() {
+    return this._client;
+  }
+
+  async #handleFunctionCallChat({ messages = [] }) {
+    const response = await this.client.chat({
+      model: this.model,
+      messages,
+      options: {
+        temperature: 0,
+      },
+    });
+    return response?.message?.content || null;
+  }
+
+  /**
+   * Create a completion based on the received messages.
+   *
+   * @param messages A list of messages to send to the API.
+   * @param functions
+   * @returns The completion.
+   */
+  async complete(messages, functions = null) {
+    try {
+      let completion;
+      if (functions.length > 0) {
+        const { toolCall, text } = await this.functionCall(
+          messages,
+          functions,
+          this.#handleFunctionCallChat.bind(this)
+        );
+
+        if (toolCall !== null) {
+          this.providerLog(`Valid tool call found - running ${toolCall.name}.`);
+          this.deduplicator.trackRun(toolCall.name, toolCall.arguments);
+          return {
+            result: null,
+            functionCall: {
+              name: toolCall.name,
+              arguments: toolCall.arguments,
+            },
+            cost: 0,
+          };
+        }
+        completion = { content: text };
+      }
+
+      if (!completion?.content) {
+        this.providerLog(
+          "Will assume chat completion without tool call inputs."
+        );
+        const response = await this.client.chat({
+          model: this.model,
+          messages: this.cleanMsgs(messages),
+          options: {
+            use_mlock: true,
+            temperature: 0.5,
+          },
+        });
+        completion = response.message;
+      }
+
+      return {
+        result: completion.content,
+        cost: 0,
+      };
+    } catch (error) {
+      throw error;
+    }
+  }
+
+  /**
+   * Get the cost of the completion.
+   *
+   * @param _usage The completion to get the cost for.
+   * @returns The cost of the completion.
+   * Stubbed since LMStudio has no cost basis.
+   */
+  getCost(_usage) {
+    return 0;
+  }
+}
+
+module.exports = OllamaProvider;
--- a/server/utils/agents/defaults.js
+++ b/server/utils/agents/defaults.js
@ -1,6 +1,7 @@
 const AgentPlugins = require("./aibitat/plugins");
 const { SystemSettings } = require("../../models/systemSettings");
 const { safeJsonParse } = require("../http");
+const Provider = require("./aibitat/providers/ai-provider");

 const USER_AGENT = {
  name: "USER",
@ -14,7 +15,7 @@ const USER_AGENT = {

 const WORKSPACE_AGENT = {
  name: "@agent",
-  getDefinition: async () => {
+  getDefinition: async (provider = null) => {
    const defaultFunctions = [
      AgentPlugins.memory.name, // RAG
      AgentPlugins.docSummarizer.name, // Doc Summary
@ -30,7 +31,7 @@ const WORKSPACE_AGENT = {
    });

    return {
-      role: "You are a helpful ai assistant who can assist the user and use tools available to help answer the users prompts and questions.",
+      role: Provider.systemPrompt(provider),
      functions: defaultFunctions,
    };
  },
--- a/server/utils/agents/index.js
+++ b/server/utils/agents/index.js
@ -77,14 +77,38 @@ class AgentHandler {
        if (!process.env.ANTHROPIC_API_KEY)
          throw new Error("Anthropic API key must be provided to use agents.");
        break;
+      case "lmstudio":
+        if (!process.env.LMSTUDIO_BASE_PATH)
+          throw new Error("LMStudio base path must be provided to use agents.");
+        break;
+      case "ollama":
+        if (!process.env.OLLAMA_BASE_PATH)
+          throw new Error("Ollama base path must be provided to use agents.");
+        break;
      default:
        throw new Error("No provider found to power agent cluster.");
    }
  }

+  #providerDefault() {
+    switch (this.provider) {
+      case "openai":
+        return "gpt-3.5-turbo";
+      case "anthropic":
+        return "claude-3-sonnet-20240229";
+      case "lmstudio":
+        return "server-default";
+      case "ollama":
+        return "llama3:latest";
+      default:
+        return "unknown";
+    }
+  }
+
  #providerSetupAndCheck() {
    this.provider = this.invocation.workspace.agentProvider || "openai";
-    this.model = this.invocation.workspace.agentModel || "gpt-3.5-turbo";
+    this.model =
+      this.invocation.workspace.agentModel || this.#providerDefault();
    this.log(`Start ${this.#invocationUUID}::${this.provider}:${this.model}`);
    this.#checkSetup();
  }
@ -137,7 +161,7 @@ class AgentHandler {
    this.aibitat.agent(USER_AGENT.name, await USER_AGENT.getDefinition());
    this.aibitat.agent(
      WORKSPACE_AGENT.name,
-      await WORKSPACE_AGENT.getDefinition()
+      await WORKSPACE_AGENT.getDefinition(this.provider)
    );

    this.#funcsToLoad = [
--- a/server/utils/helpers/index.js
+++ b/server/utils/helpers/index.js
@ -80,6 +80,9 @@ function getLLMProvider({ provider = null, model = null } = {}) {
    case "koboldcpp":
      const { KoboldCPPLLM } = require("../AiProviders/koboldCPP");
      return new KoboldCPPLLM(embedder, model);
+    case "textgenwebui":
+      const { TextGenWebUILLM } = require("../AiProviders/textGenWebUI");
+      return new TextGenWebUILLM(embedder, model);
    case "cohere":
      const { CohereLLM } = require("../AiProviders/cohere");
      return new CohereLLM(embedder, model);
--- a/server/utils/helpers/updateENV.js
+++ b/server/utils/helpers/updateENV.js
@ -146,6 +146,16 @@ const KEY_MAPPING = {
    checks: [nonZero],
  },

+  // Text Generation Web UI Settings
+  TextGenWebUIBasePath: {
+    envKey: "TEXT_GEN_WEB_UI_BASE_PATH",
+    checks: [isValidURL],
+  },
+  TextGenWebUITokenLimit: {
+    envKey: "TEXT_GEN_WEB_UI_MODEL_TOKEN_LIMIT",
+    checks: [nonZero],
+  },
+
  // Generic OpenAI InferenceSettings
  GenericOpenAiBasePath: {
    envKey: "GENERIC_OPEN_AI_BASE_PATH",
@ -418,6 +428,7 @@ function supportedLLM(input = "") {
    "openrouter",
    "groq",
    "koboldcpp",
+    "textgenwebui",
    "cohere",
    "generic-openai",
  ].includes(input);
--- a/server/utils/http/index.js
+++ b/server/utils/http/index.js
@ -8,6 +8,8 @@ process.env.NODE_ENV === "development"
    });
 const JWT = require("jsonwebtoken");
 const { User } = require("../../models/user");
+const { jsonrepair } = require("jsonrepair");
+const extract = require("extract-json-from-string");

 function reqBody(request) {
  return typeof request.body === "string"
@ -70,6 +72,18 @@ function safeJsonParse(jsonString, fallback = null) {
  try {
    return JSON.parse(jsonString);
  } catch {}
+
+  if (jsonString?.startsWith("[") || jsonString?.startsWith("{")) {
+    try {
+      const repairedJson = jsonrepair(jsonString);
+      return JSON.parse(repairedJson);
+    } catch {}
+  }
+
+  try {
+    return extract(jsonString)[0];
+  } catch {}
+
  return fallback;
 }

--- a/server/yarn.lock
+++ b/server/yarn.lock
@ -2678,6 +2678,11 @@ extract-files@^9.0.0:
  resolved "https://registry.yarnpkg.com/extract-files/-/extract-files-9.0.0.tgz#8a7744f2437f81f5ed3250ed9f1550de902fe54a"
  integrity sha512-CvdFfHkC95B4bBBk36hcEmvdR2awOdhhVUYH6S/zrVj3477zven/fJMYg7121h4T1xHZC+tetUpubpAhxwI7hQ==

+extract-json-from-string@^1.0.1:
+  version "1.0.1"
+  resolved "https://registry.yarnpkg.com/extract-json-from-string/-/extract-json-from-string-1.0.1.tgz#5001f17e6c905826dcd5989564e130959de60c96"
+  integrity sha512-xfQOSFYbELVs9QVkKsV9FZAjlAmXQ2SLR6FpfFX1kpn4QAvaGBJlrnVOblMLwrLPYc26H+q9qxo6JTd4E7AwgQ==
+
 extract-zip@^2.0.1:
  version "2.0.1"
  resolved "https://registry.yarnpkg.com/extract-zip/-/extract-zip-2.0.1.tgz#663dca56fe46df890d5f131ef4a06d22bb8ba13a"
@ -3743,6 +3748,11 @@ jsonpointer@^5.0.1:
  resolved "https://registry.yarnpkg.com/jsonpointer/-/jsonpointer-5.0.1.tgz#2110e0af0900fd37467b5907ecd13a7884a1b559"
  integrity sha512-p/nXbhSEcu3pZRdkW1OfJhpsVtW1gd4Wa1fnQc9YLiTfAjn0312eMKimbdIQzuZl9aa9xUGaRlP9T/CJE/ditQ==

+jsonrepair@^3.7.0:
+  version "3.7.0"
+  resolved "https://registry.yarnpkg.com/jsonrepair/-/jsonrepair-3.7.0.tgz#b4fddb9c8d29dd62263f4f037334099e28feac21"
+  integrity sha512-TwE50n4P4gdVfMQF2q+X+IGy4ntFfcuHHE8zjRyBcdtrRK0ORZsjOZD6zmdylk4p277nQBAlHgsEPWtMIQk4LQ==
+
 jsonwebtoken@^8.5.1:
  version "8.5.1"
  resolved "https://registry.yarnpkg.com/jsonwebtoken/-/jsonwebtoken-8.5.1.tgz#00e71e0b8df54c2121a1f26137df2280673bcc0d"
@ -4555,6 +4565,13 @@ octokit@^3.1.0:
    "@octokit/request-error" "^5.0.0"
    "@octokit/types" "^12.0.0"

+ollama@^0.5.0:
+  version "0.5.0"
+  resolved "https://registry.yarnpkg.com/ollama/-/ollama-0.5.0.tgz#cb9bc709d4d3278c9f484f751b0d9b98b06f4859"
+  integrity sha512-CRtRzsho210EGdK52GrUMohA2pU+7NbgEaBG3DcYeRmvQthDO7E2LHOkLlUUeaYUlNmEd8icbjC02ug9meSYnw==
+  dependencies:
+    whatwg-fetch "^3.6.20"
+
 on-finished@2.4.1:
  version "2.4.1"
  resolved "https://registry.yarnpkg.com/on-finished/-/on-finished-2.4.1.tgz#58c8c44116e54845ad57f14ab10b03533184ac3f"
@ -5975,7 +5992,7 @@ webidl-conversions@^3.0.0:
  resolved "https://registry.yarnpkg.com/webidl-conversions/-/webidl-conversions-3.0.1.tgz#24534275e2a7bc6be7bc86611cc16ae0a5654871"
  integrity sha512-2JAn3z8AR6rjK8Sm8orRC0h/bcl/DqL7tRPdGZ4I1CjdF+EaMLmYxBHyXuKL849eucPFhvBoxMsflfOb8kxaeQ==

-whatwg-fetch@^3.4.1:
+whatwg-fetch@^3.4.1, whatwg-fetch@^3.6.20:
  version "3.6.20"
  resolved "https://registry.yarnpkg.com/whatwg-fetch/-/whatwg-fetch-3.6.20.tgz#580ce6d791facec91d37c72890995a0b48d31c70"
  integrity sha512-EqhiFU6daOA8kpjOWTL0olhVOF3i7OrFzSYiGsEMB8GcXS+RrzauAERX65xMeNWVqxA6HXH2m69Z9LaKKdisfg==