Merge branch 'master' of github.com:Mintplex-Labs/anything-llm into render

.devcontainer/README.md

@@ -0,0 +1,72 @@
+# AnythingLLM Development Container Setup
+
+Welcome to the AnythingLLM development container configuration, designed to create a seamless and feature-rich development environment for this project.
+
+<center><h1><b>PLEASE READ THIS</b></h1></center>
+
+## Prerequisites
+
+- [Docker](https://www.docker.com/get-started)
+- [Visual Studio Code](https://code.visualstudio.com/)
+- [Remote - Containers](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-containers) VS Code extension
+
+## Features
+
+- **Base Image**: Built on `mcr.microsoft.com/devcontainers/javascript-node:1-18-bookworm`, thus Node.JS LTS v18.
+- **Additional Tools**: Includes `hadolint`, and essential apt-packages such as `curl`, `gnupg`, and more.
+- **Ports**: Configured to auto-forward ports `3000` (Frontend) and `3001` (Backend).
+- **Environment Variables**: Sets `NODE_ENV` to `development` and `ESLINT_USE_FLAT_CONFIG` to `true`.
+- **VS Code Extensions**: A suite of extensions such as `Prettier`, `Docker`, `ESLint`, and more are automatically installed. Please revise if you do not agree with any of these extensions. AI-powered extensions and time trackers are (for now) not included to avoid any privacy concerns, but you can install them later in your own environment.
+
+## Getting Started
+
+1. Using Github Codepaces. Just select to create a new workspace, and the devcontainer will be created for you.
+
+2. Using your Local VSCode (Release or Insiders). We suggest you first make a fork of the repo and then clone it to your local machine using VSCode tools. Then open the project folder in VSCode, which will prompt you to open the project in a devcontainer. Select yes, and the devcontainer will be created for you. If this does not happen, you can open the command palette and select "Remote-Containers: Reopen in Container".
+
+## On Creation:
+
+When the container is built for the first time, it will automatically run `yarn setup` to ensure everything is in place for the Collector, Server and Frontend. This command is expected to be automatically re-run if there is a content change on next reboot.
+
+## Work in the Container:
+
+Once the container is up, be patient. Some extensions may complain because dependencies are still being installed, and in the Extensions tab, some may ask you to "Reload" the project. Don't do that yet. First, wait until all settle down for the first time. We suggest you create a new VSCode profile for this devcontainer, so any configuration and extensions you change, won't affect your default profile.
+
+Checklist:
+
+- [ ] The usual message asking you to start the Server and Frontend in different windows are now "hidden" in the building process of the devcontainer. Don't forget to do as suggested.
+- [ ] Open a JavaScript file, for example "server/index.js" and check if `eslint` is working. It will complain that `'err' is defined but never used.`. This means it is working.
+- [ ] Open a React File, for example, "frontend/src/main.jsx," and check if `eslint` complains about `Fast refresh only works when a file has exports. Move your component(s) to a separate file.`. Again, it means `eslint` is working. Now check at the status bar if the `Prettier` has a double checkmark :heavy_check_mark: (double). It means Prettier is working. You will see a nice extension `Formatting:`:heavy_check_mark: that can be used to disable the `Format on Save` feature temporarily. 
+- [ ] Check if, on the left pane, you have the NPM Scripts (this may be disabled; look at the "Explorer" tree-dots up-right). There will be scripts inside the `package.json` files. You will basically need to run the `dev:collector`, `dev:server` and the `dev:frontend` in this order. When the frontend finishes starting, a window browser will open **inside** the VSCode. Still, you can open it outside.
+
+:warning: **Important for all developers** :warning:
+
+- [ ] Whe you are using the `NODE_ENV=development` the server will not store the configurations you set for security reasons. Please set the proper config on file `.env.development`. The side-effect if you don't, everytime you restart the server, you will be sent to the "Onboarding" page again.
+
+:warning: **Important for Github Codespaces** :warning:
+
+- [ ] When running the "Server" for the first time, its port will be automatically forward, but privately. Read the content of the `.env` file on the frontend folder about this, and make sure the port "Visibility" is set to "Public", so the frontend can reach the backend. Again, this is only needed for developing on Github Codespaces. We appreciate to know if you have a better solution.
+
+**For the Collector:**
+
+- [x] In the past, the Collector dwelled within the Python domain, but now it has journeyed to the splendid realm of Node.JS. Consequently, the configuration complexities of bygone versions are no longer a concern.
+
+### Now it is ready to start
+
+In the status bar you will see three shortcuts names `Collector`, `Server` and `Frontend`. Just click-and-wait on that order (don't forget to set the Server port 3001 to Public if you are using GH Codespaces **_before_** starting the Frontend).
+
+Now you can enjoy your time developing instead of reconfiguring everything.
+
+## Debugging with the devcontainers
+
+### For debugging the collector, server and frontend
+
+First, make sure the built-in extension (ms-vscode.js-debug) is active (I don't know why it would not be, but just in case). If you want, you can install the nightly version (ms-vscode.js-debug-nightly)
+
+Then, in the "Run and Debug" tab (Ctrl+shift+D), you can select on the menu:
+
+- Collector debug. This will start the collector in debug mode and attach the debugger. Works very well.
+- Server debug. This will start the server in debug mode and attach the debugger. Works very well.
+- Frontend debug. This will start the frontend in debug mode and attach the debugger. I am still struggling with this one. I don't know if VSCode can handle the .jsx files seamlessly as the pure .js on the server. Maybe there is a need for a particular configuration for Vite or React. Anyway, it starts. Another two configurations launch Chrome and Edge, and I think we could add breakpoints on .jsx files somehow. The best scenario would be always to use the embedded browser. WIP.
+
+Please leave comments on the Issues tab or the [![](https://img.shields.io/discord/1114740394715004990?logo=Discord&logoColor=white&label=Discord&labelColor=%235568ee&color=%2355A2DD&link=https%3A%2F%2Fdiscord.gg%2F6UyHPeGZAC)]("https://discord.gg/6UyHPeGZAC")

.devcontainer/devcontainer.json

@@ -0,0 +1,211 @@
+// For format details, see https://aka.ms/devcontainer.json. For config options, see the
+// README at: https://github.com/devcontainers/templates/tree/main/src/javascript-node
+{
+  "name": "Node.js",
+  // Or use a Dockerfile or Docker Compose file. More info: https://containers.dev/guide/dockerfile
+  // "build": {
+  //   "args": {
+  //     "ARG_UID": "1000",
+  //     "ARG_GID": "1000"
+  //   },
+  //   "dockerfile": "Dockerfile"
+  // },
+  // "containerUser": "anythingllm",
+  // Or use a Dockerfile or Docker Compose file. More info: https://containers.dev/guide/dockerfile
+  "image": "mcr.microsoft.com/devcontainers/javascript-node:1-18-bookworm",
+  // Features to add to the dev container. More info: https://containers.dev/features.
+  "features": {
+    // Docker very useful linter
+    "ghcr.io/dhoeric/features/hadolint:1": {
+      "version": "latest"
+    },
+    // Terraform support
+    "ghcr.io/devcontainers/features/terraform:1": {},
+    // Just a wrap to install needed packages
+    "ghcr.io/rocker-org/devcontainer-features/apt-packages:1": {
+      // Dependencies copied from ../docker/Dockerfile plus some dev stuff
+      "packages": [
+        "build-essential",
+        "ca-certificates",
+        "curl",
+        "ffmpeg",
+        "fonts-liberation",
+        "git",
+        "gnupg",
+        "htop",
+        "less",
+        "libappindicator1",
+        "libasound2",
+        "libatk-bridge2.0-0",
+        "libatk1.0-0",
+        "libc6",
+        "libcairo2",
+        "libcups2",
+        "libdbus-1-3",
+        "libexpat1",
+        "libfontconfig1",
+        "libgbm1",
+        "libgcc1",
+        "libgfortran5",
+        "libglib2.0-0",
+        "libgtk-3-0",
+        "libnspr4",
+        "libnss3",
+        "libpango-1.0-0",
+        "libpangocairo-1.0-0",
+        "libstdc++6",
+        "libx11-6",
+        "libx11-xcb1",
+        "libxcb1",
+        "libxcomposite1",
+        "libxcursor1",
+        "libxdamage1",
+        "libxext6",
+        "libxfixes3",
+        "libxi6",
+        "libxrandr2",
+        "libxrender1",
+        "libxss1",
+        "libxtst6",
+        "locales",
+        "lsb-release",
+        "procps",
+        "tzdata",
+        "wget",
+        "xdg-utils"
+      ]
+    }
+  },
+  "updateContentCommand": "yarn setup",
+  // Use 'postCreateCommand' to run commands after the container is created.
+  // This configures VITE for github codespaces
+  "postCreateCommand": "if [ \"${CODESPACES}\" = \"true\" ]; then echo 'VITE_API_BASE=\"https://$CODESPACE_NAME-3001.$GITHUB_CODESPACES_PORT_FORWARDING_DOMAIN/api\"' > ./frontend/.env; fi",
+  "portsAttributes": {
+    "3001": {
+      "label": "Backend",
+      "onAutoForward": "notify"
+    },
+    "3000": {
+      "label": "Frontend",
+      "onAutoForward": "openPreview"
+    }
+  },
+  "capAdd": [
+    "SYS_ADMIN" // needed for puppeteer using headless chrome in sandbox
+  ],
+  "remoteEnv": {
+    "NODE_ENV": "development",
+    "ESLINT_USE_FLAT_CONFIG": "true",
+    "ANYTHING_LLM_RUNTIME": "docker"
+  },
+  // "initializeCommand": "echo Initialize....",
+  "shutdownAction": "stopContainer",
+  // Configure tool-specific properties.
+  "customizations": {
+    "codespaces": {
+      "openFiles": [
+        "README.md",
+        ".devcontainer/README.md"
+      ]
+    },
+    "vscode": {
+      "openFiles": [
+        "README.md",
+        ".devcontainer/README.md"
+      ],
+      "extensions": [
+        "bierner.github-markdown-preview",
+        "bradlc.vscode-tailwindcss",
+        "dbaeumer.vscode-eslint",
+        "editorconfig.editorconfig",
+        "esbenp.prettier-vscode",
+        "exiasr.hadolint",
+        "flowtype.flow-for-vscode",
+        "gamunu.vscode-yarn",
+        "hashicorp.terraform",
+        "mariusschulz.yarn-lock-syntax",
+        "ms-azuretools.vscode-docker",
+        "streetsidesoftware.code-spell-checker",
+        "actboy168.tasks",
+        "tombonnike.vscode-status-bar-format-toggle",
+        "ms-vscode.js-debug"
+      ],
+      "settings": {
+        "[css]": {
+          "editor.defaultFormatter": "esbenp.prettier-vscode"
+        },
+        "[dockercompose]": {
+          "editor.defaultFormatter": "esbenp.prettier-vscode"
+        },
+        "[dockerfile]": {
+          "editor.defaultFormatter": "ms-azuretools.vscode-docker"
+        },
+        "[html]": {
+          "editor.defaultFormatter": "esbenp.prettier-vscode"
+        },
+        "[javascript]": {
+          "editor.defaultFormatter": "esbenp.prettier-vscode"
+        },
+        "[javascriptreact]": {
+          "editor.defaultFormatter": "esbenp.prettier-vscode"
+        },
+        "[json]": {
+          "editor.defaultFormatter": "esbenp.prettier-vscode"
+        },
+        "[jsonc]": {
+          "editor.defaultFormatter": "esbenp.prettier-vscode"
+        },
+        "[markdown]": {
+          "editor.defaultFormatter": "esbenp.prettier-vscode"
+        },
+        "[postcss]": {
+          "editor.defaultFormatter": "esbenp.prettier-vscode"
+        },
+        "[toml]": {
+          "editor.defaultFormatter": "tamasfe.even-better-toml"
+        },
+        "eslint.debug": true,
+        "eslint.enable": true,
+        "eslint.experimental.useFlatConfig": true,
+        "eslint.run": "onSave",
+        "files.associations": {
+          ".*ignore": "ignore",
+          ".editorconfig": "editorconfig",
+          ".env*": "properties",
+          ".flowconfig": "ini",
+          ".prettierrc": "json",
+          "*.css": "tailwindcss",
+          "*.md": "markdown",
+          "*.sh": "shellscript",
+          "docker-compose.*": "dockercompose",
+          "Dockerfile*": "dockerfile",
+          "yarn.lock": "yarnlock"
+        },
+        "javascript.format.enable": false,
+        "javascript.inlayHints.enumMemberValues.enabled": true,
+        "javascript.inlayHints.functionLikeReturnTypes.enabled": true,
+        "javascript.inlayHints.parameterTypes.enabled": true,
+        "javascript.inlayHints.variableTypes.enabled": true,
+        "js/ts.implicitProjectConfig.module": "CommonJS",
+        "json.format.enable": false,
+        "json.schemaDownload.enable": true,
+        "npm.autoDetect": "on",
+        "npm.packageManager": "yarn",
+        "prettier.useEditorConfig": false,
+        "tailwindCSS.files.exclude": [
+          "**/.git/**",
+          "**/node_modules/**",
+          "**/.hg/**",
+          "**/.svn/**",
+          "**/dist/**"
+        ],
+        "typescript.validate.enable": false,
+        "workbench.editorAssociations": {
+          "*.md": "vscode.markdown.preview.editor"
+        }
+      }
+    }
+  }
+  // Uncomment to connect as root instead. More info: https://aka.ms/dev-containers-non-root.
+  // "remoteUser": "root"
+}

.dockerignore

@@ -1,10 +1,10 @@
-server/storage/documents/**
+**/server/storage/documents/**
-server/storage/vector-cache/**
+**/server/storage/vector-cache/**
-server/storage/*.db
+**/server/storage/*.db
-server/storage/lancedb
+**/server/storage/lancedb
-collector/hotdir/**
+**/collector/hotdir/**
-collector/v-env/**
+**/collector/v-env/**
-collector/outputs/**
+**/collector/outputs/**
 **/node_modules/
 **/dist/
 **/v-env/
@@ -13,4 +13,5 @@ collector/outputs/**
 **/.env.*
 **/bundleinspector.html
 !docker/.env.example
-!frontend/.env.production
+!frontend/.env.production
+**/tmp/**

.editorconfig

@@ -4,9 +4,14 @@
 root = true
 
 [*]
+# Non-configurable Prettier behaviors
+charset = utf-8
+insert_final_newline = true
+trim_trailing_whitespace = true
+
+# Configurable Prettier behaviors
+# (change these if your Prettier config differs)
+end_of_line = lf
 indent_style = space
 indent_size = 2
-end_of_line = lf
+max_line_length = 80
-charset = utf-8
-trim_trailing_whitespace = true
-insert_final_newline = true

.github/FUNDING.yml

@@ -0,0 +1 @@
+github: Mintplex-Labs

.github/workflows/build-and-push-image.yaml

@@ -1,17 +1,28 @@
-name: Publish Docker image and Github Registry
+# This Github action is for publishing of the primary image for AnythingLLM
+# It will publish a linux/amd64 and linux/arm64 image at the same time
+# This file should ONLY BY USED FOR `master` BRANCH. 
+# TODO: Update `runs-on` for arm64 when GitHub finally supports
+# native arm environments as QEMU takes around 1 hour to build
+# ref: https://github.com/actions/runner-images/issues/5631 :(
+name: Publish AnythingLLM Primary Docker image (amd64/arm64)
+
+concurrency:
+  group: build-${{ github.ref }}
+  cancel-in-progress: true
 
 on:
   push:
-    branches: ['master']
+    branches: ['master'] # master branch only. Do not modify.
     paths-ignore:
       - '*.md'
       - 'cloud-deployments/*'
       - 'images/*'
       - '.vscode/*'
+      - '**/.env.example'
 
 jobs:
-  push_to_registries:
+  push_multi_platform_to_registries:
-    name: Push Docker image to multiple registries
+    name: Push Docker multi-platform image to multiple registries
     runs-on: ubuntu-latest
     permissions:
       packages: write
@@ -19,6 +30,17 @@ jobs:
     steps:
       - name: Check out the repo
         uses: actions/checkout@v4
+
+      - name: Parse repository name to lowercase
+        shell: bash
+        run: echo "repo=${GITHUB_REPOSITORY,,}" >> $GITHUB_OUTPUT
+        id: lowercase_repo
+
+      - name: Set up QEMU
+        uses: docker/setup-qemu-action@v3
+
+      - name: Set up Docker Buildx
+        uses: docker/setup-buildx-action@v3
       
       - name: Log in to Docker Hub
         uses: docker/login-action@f4ef78c080cd8ba55a85445d5b36e214a81df20a
@@ -41,11 +63,15 @@ jobs:
             mintplexlabs/anythingllm
             ghcr.io/${{ github.repository }}
       
-      - name: Build and push Docker images
+      - name: Build and push multi-platform Docker image
-        uses: docker/build-push-action@3b5e8027fcad23fda98b2e3ac259d8d67585f671
+        uses: docker/build-push-action@v5
         with:
           context: .
           file: ./docker/Dockerfile
           push: true
-          tags: ${{ steps.meta.outputs.tags }}
+          platforms: linux/amd64,linux/arm64
-          labels: ${{ steps.meta.outputs.labels }}
+          tags: |
+            ${{ steps.meta.outputs.tags }}
+            ${{ github.ref_name == 'master' && 'mintplexlabs/anythingllm:latest' || '' }}
+            ${{ github.ref_name == 'master' && format('ghcr.io/{0}:{1}', steps.lowercase_repo.outputs.repo, 'latest') || '' }}
+          labels: ${{ steps.meta.outputs.labels }}

.gitignore

@@ -7,4 +7,5 @@ __pycache__
 v-env
 .DS_Store
 aws_cf_deploy_anything_llm.json
-yarn.lock
+yarn.lock
+*.bak

.prettierignore

@@ -0,0 +1,12 @@
+# defaults
+**/.git
+**/.svn
+**/.hg
+**/node_modules
+
+#frontend
+frontend/bundleinspector.html
+**/dist
+
+#server
+server/swagger/openapi.json

.prettierrc

@@ -0,0 +1,38 @@
+{
+  "tabWidth": 2,
+  "useTabs": false,
+  "endOfLine": "lf",
+  "semi": true,
+  "singleQuote": false,
+  "printWidth": 80,
+  "trailingComma": "es5",
+  "bracketSpacing": true,
+  "bracketSameLine": false,
+  "overrides": [
+    {
+      "files": ["*.js", "*.mjs", "*.jsx"],
+      "options": {
+        "parser": "flow",
+        "arrowParens": "always"
+      }
+    },
+    {
+      "files": "*.config.js",
+      "options": {
+        "semi": false,
+        "parser": "flow",
+        "trailingComma": "none"
+      }
+    },
+    {
+      "files": "*.html",
+      "options": {
+        "bracketSameLine": true
+      }
+    },
+    {
+      "files": ".prettierrc",
+      "options": { "parser": "json" }
+    }
+  ]
+}

.vscode/launch.json

@@ -0,0 +1,74 @@
+{
+  // Use IntelliSense to learn about possible attributes.
+  // Hover to view descriptions of existing attributes.
+  // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
+  "version": "0.2.0",
+  "configurations": [
+    {
+      "name": "Collector debug",
+      "request": "launch",
+      "cwd": "${workspaceFolder}/collector",
+      "env": {
+        "NODE_ENV": "development"
+      },
+      "runtimeArgs": [
+        "index.js"
+      ],
+      // not using yarn/nodemon because it doesn't work with breakpoints
+      // "runtimeExecutable": "yarn",
+      "skipFiles": [
+        "<node_internals>/**"
+      ],
+      "type": "node"
+    },
+    {
+      "name": "Server debug",
+      "request": "launch",
+      "cwd": "${workspaceFolder}/server",
+      "env": {
+        "NODE_ENV": "development"
+      },
+      "runtimeArgs": [
+        "index.js"
+      ],
+      // not using yarn/nodemon because it doesn't work with breakpoints
+      // "runtimeExecutable": "yarn",
+      "skipFiles": [
+        "<node_internals>/**"
+      ],
+      "type": "node"
+    },
+    {
+      "name": "Frontend debug",
+      "request": "launch",
+      "cwd": "${workspaceFolder}/frontend",
+      "env": {
+        "NODE_ENV": "development",
+      },
+      "runtimeExecutable": "${workspaceFolder}/frontend/node_modules/.bin/vite",
+      "runtimeArgs": [
+        "--debug",
+        "--host=0.0.0.0"
+      ],
+      // "runtimeExecutable": "yarn",
+      "skipFiles": [
+        "<node_internals>/**"
+      ],
+      "type": "node"
+    },
+    {
+      "name": "Launch Edge",
+      "request": "launch",
+      "type": "msedge",
+      "url": "http://localhost:3000",
+      "webRoot": "${workspaceFolder}"
+    },
+    {
+      "type": "chrome",
+      "request": "launch",
+      "name": "Launch Chrome against localhost",
+      "url": "http://localhost:3000",
+      "webRoot": "${workspaceFolder}"
+    }
+  ]
+}

.vscode/settings.json

@@ -1,7 +1,11 @@
 {
   "cSpell.words": [
+    "Dockerized",
+    "Langchain",
+    "Ollama",
     "openai",
     "Qdrant",
     "Weaviate"
-  ]
+  ],
+  "eslint.experimental.useFlatConfig": true
 }

.vscode/tasks.json

@@ -0,0 +1,94 @@
+{
+  // See https://go.microsoft.com/fwlink/?LinkId=733558
+  // for the documentation about the tasks.json format
+  "version": "2.0.0",
+  "tasks": [
+    {
+      "type": "shell",
+      "options": {
+        "cwd": "${workspaceFolder}/collector",
+        "statusbar": {
+          "color": "#ffea00",
+          "detail": "Runs the collector",
+          "label": "Collector: $(play) run",
+          "running": {
+            "color": "#ffea00",
+            "label": "Collector: $(gear~spin) running"
+          }
+        }
+      },
+      "command": "cd ${workspaceFolder}/collector/ && yarn dev",
+      "runOptions": {
+        "instanceLimit": 1,
+        "reevaluateOnRerun": true
+      },
+      "presentation": {
+        "echo": true,
+        "reveal": "always",
+        "focus": false,
+        "panel": "shared",
+        "showReuseMessage": true,
+        "clear": false
+      },
+      "label": "Collector: run"
+    },
+    {
+      "type": "shell",
+      "options": {
+        "cwd": "${workspaceFolder}/server",
+        "statusbar": {
+          "color": "#ffea00",
+          "detail": "Runs the server",
+          "label": "Server: $(play) run",
+          "running": {
+            "color": "#ffea00",
+            "label": "Server: $(gear~spin) running"
+          }
+        }
+      },
+      "command": "cd ${workspaceFolder}/server/ && yarn dev",
+      "runOptions": {
+        "instanceLimit": 1,
+        "reevaluateOnRerun": true
+      },
+      "presentation": {
+        "echo": true,
+        "reveal": "always",
+        "focus": false,
+        "panel": "shared",
+        "showReuseMessage": true,
+        "clear": false
+      },
+      "label": "Server: run"
+    },
+    {
+      "type": "shell",
+      "options": {
+        "cwd": "${workspaceFolder}/frontend",
+        "statusbar": {
+          "color": "#ffea00",
+          "detail": "Runs the frontend",
+          "label": "Frontend: $(play) run",
+          "running": {
+            "color": "#ffea00",
+            "label": "Frontend: $(gear~spin) running"
+          }
+        }
+      },
+      "command": "cd ${workspaceFolder}/frontend/ && yarn dev",
+      "runOptions": {
+        "instanceLimit": 1,
+        "reevaluateOnRerun": true
+      },
+      "presentation": {
+        "echo": true,
+        "reveal": "always",
+        "focus": false,
+        "panel": "shared",
+        "showReuseMessage": true,
+        "clear": false
+      },
+      "label": "Frontend: run"
+    }
+  ]
+}

README.md

@@ -23,9 +23,13 @@
   </a>
 </p>
 
+<p align="center">
+👉 AnythingLLM for desktop! <a href="https://mintplexlabs.typeform.com/to/sFgD2TIb" target="_blank"> Sign up</a>
+</p>
+
 A full-stack application that enables you to turn any document, resource, or piece of content into context that any LLM can use as references during chatting. This application allows you to pick and choose which LLM or Vector Database you want to use as well as supporting multi-user management and permissions.
 
-![Chatting](/images/screenshots/chatting.gif)
+![Chatting](https://github.com/Mintplex-Labs/anything-llm/assets/16845892/cfc5f47c-bd91-4067-986c-f3f49621a859)
 
 <details>
 <summary><kbd>Watch the demo!</kbd></summary>
@@ -58,6 +62,8 @@ Some cool features of AnythingLLM
 - [OpenAI](https://openai.com)
 - [Azure OpenAI](https://azure.microsoft.com/en-us/products/ai-services/openai-service)
 - [Anthropic ClaudeV2](https://www.anthropic.com/)
+- [Google Gemini Pro](https://ai.google.dev/)
+- [Ollama (chat models)](https://ollama.ai/)
 - [LM Studio (all models)](https://lmstudio.ai)
 - [LocalAi (all models)](https://localai.io/)
 
@@ -91,19 +97,14 @@ Mintplex Labs & the community maintain a number of deployment methods, scripts,
 
 
 ## How to setup for development
-- `yarn setup` from the project root directory.
+- `yarn setup` To fill in the required `.env` files you'll need in each of the application sections (from root of repo).
-  - This will fill in the required `.env` files you'll need in each of the application sections. Go fill those out before proceeding or else things won't work right.
+  - Go fill those out before proceeding. Ensure `server/.env.development` is filled or else things won't work right.
-- `yarn prisma:setup` To build the Prisma client and migrate the database.
+- `yarn dev:server` To boot the server locally (from root of repo).
+- `yarn dev:frontend` To boot the frontend locally (from root of repo).
+- `yarn dev:collector` To then run the document collector (from root of repo).
+
 
-To boot the server locally (from root of repo):
-- ensure `server/.env.development` is set and filled out.
-`yarn dev:server`
- 
-To boot the frontend locally (from root of repo):
-`yarn dev:frontend`
 
-To then run the document collector (from root of repo)
-`yarn dev:collector`
 
 [Learn about documents](./server/storage/documents/DOCUMENTS.md)
 

clean.sh

@@ -1,3 +0,0 @@
-# Easily kill process on port because sometimes nodemon fails to reboot
-kill -9 $(lsof -t -i tcp:5000) &
-kill -9 $(lsof -t -i tcp:3001) # if running default for MacOS Monterey

collector/.env.example

@@ -1 +1 @@
-# Placeholder .env file for collector runtime
+# Placeholder .env file for collector runtime

collector/package.json

@@ -34,7 +34,7 @@
     "multer": "^1.4.5-lts.1",
     "officeparser": "^4.0.5",
     "pdf-parse": "^1.1.1",
-    "puppeteer": "^21.6.1",
+    "puppeteer": "~21.5.2",
     "slugify": "^1.6.6",
     "url-pattern": "^1.0.3",
     "uuid": "^9.0.0",

collector/processLink/convert/generic.js

@@ -18,7 +18,7 @@ async function scrapeGenericUrl(link) {
   const url = new URL(link);
   const filename = (url.host + "-" + url.pathname).replace(".", "_");
 
-  data = {
+  const data = {
     id: v4(),
     url: "file://" + slugify(filename) + ".html",
     title: slugify(filename) + ".html",

collector/processSingleFile/convert/asAudio.js

@@ -46,7 +46,7 @@ async function asAudio({ fullFilePath = "", filename = "" }) {
     return { success: false, reason: `No text content found in ${filename}.` };
   }
 
-  data = {
+  const data = {
     id: v4(),
     url: "file://" + fullFilePath,
     title: filename,

collector/processSingleFile/convert/asDocx.js

@@ -28,7 +28,7 @@ async function asDocX({ fullFilePath = "", filename = "" }) {
   }
 
   const content = pageContent.join("");
-  data = {
+  const data = {
     id: v4(),
     url: "file://" + fullFilePath,
     title: filename,

collector/processSingleFile/convert/asMbox.js

@@ -35,7 +35,7 @@ async function asMbox({ fullFilePath = "", filename = "" }) {
       `-- Working on message "${mail.subject || "Unknown subject"}" --`
     );
 
-    data = {
+    const data = {
       id: v4(),
       url: "file://" + fullFilePath,
       title: mail?.subject

collector/processSingleFile/convert/asOfficeMime.js

@@ -23,7 +23,7 @@ async function asOfficeMime({ fullFilePath = "", filename = "" }) {
     return { success: false, reason: `No text content found in ${filename}.` };
   }
 
-  data = {
+  const data = {
     id: v4(),
     url: "file://" + fullFilePath,
     title: filename,

collector/processSingleFile/convert/asPDF.js

@@ -33,7 +33,7 @@ async function asPDF({ fullFilePath = "", filename = "" }) {
   }
 
   const content = pageContent.join("");
-  data = {
+  const data = {
     id: v4(),
     url: "file://" + fullFilePath,
     title: docs[0]?.metadata?.pdf?.info?.Title || filename,

collector/processSingleFile/convert/asTxt.js

@@ -23,7 +23,7 @@ async function asTxt({ fullFilePath = "", filename = "" }) {
   }
 
   console.log(`-- Working ${filename} --`);
-  data = {
+  const data = {
     id: v4(),
     url: "file://" + fullFilePath,
     title: filename,

collector/processSingleFile/index.js

@@ -5,8 +5,7 @@ const {
   SUPPORTED_FILETYPE_CONVERTERS,
 } = require("../utils/constants");
 const { trashFile } = require("../utils/files");
-
+const RESERVED_FILES = ["__HOTDIR__.md"];
-RESERVED_FILES = ["__HOTDIR__.md"];
 
 async function processSingleFile(targetFilename) {
   const fullFilePath = path.resolve(WATCH_DIRECTORY, targetFilename);

collector/yarn.lock

@@ -134,10 +134,10 @@
   resolved "https://registry.yarnpkg.com/@protobufjs/utf8/-/utf8-1.1.0.tgz#a777360b5b39a1a2e5106f8e858f2fd2d060c570"
   integrity sha512-Vvn3zZrhQZkkBE8LSuW3em98c0FwgO4nxzv6OdSxPKJIEKY2bGbHn+mhGIPerzI4twdxaP8/0+06HBpwf345Lw==
 
-"@puppeteer/browsers@1.9.0":
+"@puppeteer/browsers@1.8.0":
-  version "1.9.0"
+  version "1.8.0"
-  resolved "https://registry.yarnpkg.com/@puppeteer/browsers/-/browsers-1.9.0.tgz#dfd0aad0bdc039572f1b57648f189525d627b7ff"
+  resolved "https://registry.yarnpkg.com/@puppeteer/browsers/-/browsers-1.8.0.tgz#fb6ee61de15e7f0e67737aea9f9bab1512dbd7d8"
-  integrity sha512-QwguOLy44YBGC8vuPP2nmpX4MUN2FzWbsnvZJtiCzecU3lHmVZkaC1tq6rToi9a200m8RzlVtDyxCS0UIDrxUg==
+  integrity sha512-TkRHIV6k2D8OlUe8RtG+5jgOF/H98Myx0M6AOafC8DdNVOFiBSFa5cpRDtpm8LXOa9sVwe0+e6Q3FC56X/DZfg==
   dependencies:
     debug "4.3.4"
     extract-zip "2.0.1"
@@ -608,10 +608,10 @@ chownr@^2.0.0:
   resolved "https://registry.yarnpkg.com/chownr/-/chownr-2.0.0.tgz#15bfbe53d2eab4cf70f18a8cd68ebe5b3cb1dece"
   integrity sha512-bIomtDF5KGpdogkLd9VspvFzk9KfpyyGlS8YFVZl7TGPBHL5snIOnxeshwVgPteQ9b4Eydl+pVbIyE1DcvCWgQ==
 
-chromium-bidi@0.5.1:
+chromium-bidi@0.4.33:
-  version "0.5.1"
+  version "0.4.33"
-  resolved "https://registry.yarnpkg.com/chromium-bidi/-/chromium-bidi-0.5.1.tgz#390c1af350c4887824a33d82190de1cc5c5680fc"
+  resolved "https://registry.yarnpkg.com/chromium-bidi/-/chromium-bidi-0.4.33.tgz#9a9aba5a5b07118c8e7d6405f8ee79f47418dd1d"
-  integrity sha512-dcCqOgq9fHKExc2R4JZs/oKbOghWpUNFAJODS8WKRtLhp3avtIH5UDCBrutdqZdh3pARogH8y1ObXm87emwb3g==
+  integrity sha512-IxoFM5WGQOIAd95qrSXzJUv4eXIrh+RvU3rwwqIiwYuvfE7U/Llj4fejbsJnjJMUYCuGtVQsY2gv7oGl4aTNSQ==
   dependencies:
     mitt "3.0.1"
     urlpattern-polyfill "9.0.0"
@@ -2584,26 +2584,26 @@ pump@^3.0.0:
     end-of-stream "^1.1.0"
     once "^1.3.1"
 
-puppeteer-core@21.6.1:
+puppeteer-core@21.5.2:
-  version "21.6.1"
+  version "21.5.2"
-  resolved "https://registry.yarnpkg.com/puppeteer-core/-/puppeteer-core-21.6.1.tgz#10eccb4dc3167c8c26bc21122fabb45a9fda9ca7"
+  resolved "https://registry.yarnpkg.com/puppeteer-core/-/puppeteer-core-21.5.2.tgz#6d3de4efb2ae65f1ee072043787b75594e88035f"
-  integrity sha512-0chaaK/RL9S1U3bsyR4fUeUfoj51vNnjWvXgG6DcsyMjwYNpLcAThv187i1rZCo7QhJP0wZN8plQkjNyrq2h+A==
+  integrity sha512-v4T0cWnujSKs+iEfmb8ccd7u4/x8oblEyKqplqKnJ582Kw8PewYAWvkH4qUWhitN3O2q9RF7dzkvjyK5HbzjLA==
   dependencies:
-    "@puppeteer/browsers" "1.9.0"
+    "@puppeteer/browsers" "1.8.0"
-    chromium-bidi "0.5.1"
+    chromium-bidi "0.4.33"
     cross-fetch "4.0.0"
     debug "4.3.4"
     devtools-protocol "0.0.1203626"
-    ws "8.15.1"
+    ws "8.14.2"
 
-puppeteer@^21.6.1:
+puppeteer@~21.5.2:
-  version "21.6.1"
+  version "21.5.2"
-  resolved "https://registry.yarnpkg.com/puppeteer/-/puppeteer-21.6.1.tgz#2ec0878906ff90b3a424f19e5eb006592abe25b6"
+  resolved "https://registry.yarnpkg.com/puppeteer/-/puppeteer-21.5.2.tgz#0a4a72175c0fd0944d6486f4734807e1671d527b"
-  integrity sha512-O+pbc61oj8ln6m8EJKncrsQFmytgRyFYERtk190PeLbJn5JKpmmynn2p1PiFrlhCitAQXLJ0MOy7F0TeyCRqBg==
+  integrity sha512-BaAGJOq8Fl6/cck6obmwaNLksuY0Bg/lIahCLhJPGXBFUD2mCffypa4A592MaWnDcye7eaHmSK9yot0pxctY8A==
   dependencies:
-    "@puppeteer/browsers" "1.9.0"
+    "@puppeteer/browsers" "1.8.0"
     cosmiconfig "8.3.6"
-    puppeteer-core "21.6.1"
+    puppeteer-core "21.5.2"
 
 qs@6.11.0:
   version "6.11.0"
@@ -3259,10 +3259,10 @@ wrappy@1:
   resolved "https://registry.yarnpkg.com/wrappy/-/wrappy-1.0.2.tgz#b5243d8f3ec1aa35f1364605bc0d1036e30ab69f"
   integrity sha512-l4Sp/DRseor9wL6EvV2+TuQn63dMkPjZ/sp9XkghTEbV9KlPS1xUsZ3u7/IQO4wxtcFB4bgpQPRcR3QCvezPcQ==
 
-ws@8.15.1:
+ws@8.14.2:
-  version "8.15.1"
+  version "8.14.2"
-  resolved "https://registry.yarnpkg.com/ws/-/ws-8.15.1.tgz#271ba33a45ca0cc477940f7f200cd7fba7ee1997"
+  resolved "https://registry.yarnpkg.com/ws/-/ws-8.14.2.tgz#6c249a806eb2db7a20d26d51e7709eab7b2e6c7f"
-  integrity sha512-W5OZiCjXEmk0yZ66ZN82beM5Sz7l7coYxpRkzS+p9PP+ToQry8szKh+61eNktr7EA9DOwvFGhfC605jDHbP6QQ==
+  integrity sha512-wEBG1ftX4jcglPxgFCMJmZ2PLtSbJ2Peg6TmpJFTbe9GZYOQCDPdMYu/Tm0/bGZkw8paZnJY45J4K2PZrLYq8g==
 
 xmlbuilder@^10.0.0:
   version "10.1.1"

docker/.env.example

@@ -1,5 +1,4 @@
 SERVER_PORT=3001
-CACHE_VECTORS="true"
 STORAGE_DIR="/app/server/storage"
 UID='1000'
 GID='1000'
@@ -12,6 +11,10 @@ GID='1000'
 # OPEN_AI_KEY=
 # OPEN_MODEL_PREF='gpt-3.5-turbo'
 
+# LLM_PROVIDER='gemini'
+# GEMINI_API_KEY=
+# GEMINI_LLM_MODEL_PREF='gemini-pro'
+
 # LLM_PROVIDER='azure'
 # AZURE_OPENAI_ENDPOINT=
 # AZURE_OPENAI_KEY=
@@ -32,6 +35,11 @@ GID='1000'
 # LOCAL_AI_MODEL_TOKEN_LIMIT=4096
 # LOCAL_AI_API_KEY="sk-123abc"
 
+# LLM_PROVIDER='ollama'
+# OLLAMA_BASE_PATH='http://host.docker.internal:11434'
+# OLLAMA_MODEL_PREF='llama2'
+# OLLAMA_MODEL_TOKEN_LIMIT=4096
+
 ###########################################
 ######## Embedding API SElECTION ##########
 ###########################################
@@ -45,7 +53,7 @@ GID='1000'
 # EMBEDDING_MODEL_PREF='my-embedder-model' # This is the "deployment" on Azure you want to use for embeddings. Not the base model. Valid base model is text-embedding-ada-002
 
 # EMBEDDING_ENGINE='localai'
-# EMBEDDING_BASE_PATH='https://localhost:8080/v1'
+# EMBEDDING_BASE_PATH='http://localhost:8080/v1'
 # EMBEDDING_MODEL_PREF='text-embedding-ada-002'
 # EMBEDDING_MODEL_MAX_CHUNK_LENGTH=1000 # The max chunk size in chars a string to embed can be
 
@@ -92,4 +100,13 @@ GID='1000'
 # PASSWORDUPPERCASE=1
 # PASSWORDNUMERIC=1
 # PASSWORDSYMBOL=1
-# PASSWORDREQUIREMENTS=4
+# PASSWORDREQUIREMENTS=4
+
+###########################################
+######## ENABLE HTTPS SERVER ##############
+###########################################
+# By enabling this and providing the path/filename for the key and cert,
+# the server will use HTTPS instead of HTTP.
+#ENABLE_HTTPS="true"
+#HTTPS_CERT_PATH="sslcert/cert.pem"
+#HTTPS_KEY_PATH="sslcert/key.pem"

docker/Dockerfile

@@ -1,11 +1,66 @@
 # Setup base image
 FROM ubuntu:jammy-20230522 AS base
 
-# Build arguments
 ARG ARG_UID=1000
 ARG ARG_GID=1000
 
-# Install system dependencies
+FROM base AS build-arm64
+RUN echo "Preparing build of AnythingLLM image for arm64 architecture"
+
+RUN DEBIAN_FRONTEND=noninteractive apt-get update && \
+    DEBIAN_FRONTEND=noninteractive apt-get install -yq --no-install-recommends \
+        unzip curl gnupg libgfortran5 libgbm1 tzdata netcat \
+        libasound2 libatk1.0-0 libc6 libcairo2 libcups2 libdbus-1-3 libexpat1 libfontconfig1 \
+        libgcc1 libglib2.0-0 libgtk-3-0 libnspr4 libpango-1.0-0 libx11-6 libx11-xcb1 libxcb1 \
+        libxcomposite1 libxcursor1 libxdamage1 libxext6 libxfixes3 libxi6 libxrandr2 libxrender1 \
+        libxss1 libxtst6 ca-certificates fonts-liberation libappindicator1 libnss3 lsb-release \
+        xdg-utils git build-essential ffmpeg && \
+    mkdir -p /etc/apt/keyrings && \
+    curl -fsSL https://deb.nodesource.com/gpgkey/nodesource-repo.gpg.key | gpg --dearmor -o /etc/apt/keyrings/nodesource.gpg && \
+    echo "deb [signed-by=/etc/apt/keyrings/nodesource.gpg] https://deb.nodesource.com/node_18.x nodistro main" | tee /etc/apt/sources.list.d/nodesource.list && \
+    apt-get update && \
+    apt-get install -yq --no-install-recommends nodejs && \
+    curl -LO https://github.com/yarnpkg/yarn/releases/download/v1.22.19/yarn_1.22.19_all.deb \
+        && dpkg -i yarn_1.22.19_all.deb \
+        && rm yarn_1.22.19_all.deb
+
+# Create a group and user with specific UID and GID
+RUN groupadd -g $ARG_GID anythingllm && \
+    useradd -u $ARG_UID -m -d /app -s /bin/bash -g anythingllm anythingllm && \
+    mkdir -p /app/frontend/ /app/server/ /app/collector/ && chown -R anythingllm:anythingllm /app
+
+# Copy docker helper scripts
+COPY ./docker/docker-entrypoint.sh /usr/local/bin/
+COPY ./docker/docker-healthcheck.sh /usr/local/bin/
+COPY --chown=anythingllm:anythingllm ./docker/.env.example /app/server/.env
+
+# Ensure the scripts are executable
+RUN chmod +x /usr/local/bin/docker-entrypoint.sh && \
+    chmod +x /usr/local/bin/docker-healthcheck.sh
+
+USER anythingllm
+WORKDIR /app
+
+# Puppeteer does not ship with an ARM86 compatible build for Chromium
+# so web-scraping would be broken in arm docker containers unless we patch it
+# by manually installing a compatible chromedriver.
+RUN echo "Need to patch Puppeteer x Chromium support for ARM86 - installing dep!" && \
+    curl https://playwright.azureedge.net/builds/chromium/1088/chromium-linux-arm64.zip -o chrome-linux.zip && \
+    unzip chrome-linux.zip && \
+    rm -rf chrome-linux.zip
+
+ENV PUPPETEER_SKIP_CHROMIUM_DOWNLOAD=true
+ENV CHROME_PATH=/app/chrome-linux/chrome
+ENV PUPPETEER_EXECUTABLE_PATH=/app/chrome-linux/chrome
+
+RUN echo "Done running arm64 specific installtion steps"
+
+#############################################
+
+# amd64-specific stage
+FROM base AS build-amd64
+RUN echo "Preparing build of AnythingLLM image for non-ARM architecture"
+
 RUN DEBIAN_FRONTEND=noninteractive apt-get update && \
     DEBIAN_FRONTEND=noninteractive apt-get install -yq --no-install-recommends \
         curl gnupg libgfortran5 libgbm1 tzdata netcat \
@@ -37,22 +92,25 @@ COPY --chown=anythingllm:anythingllm ./docker/.env.example /app/server/.env
 RUN chmod +x /usr/local/bin/docker-entrypoint.sh && \
     chmod +x /usr/local/bin/docker-healthcheck.sh
 
-USER anythingllm
+#############################################
+# COMMON BUILD FLOW FOR ALL ARCHS
+#############################################
+FROM build-${TARGETARCH} AS build
+RUN echo "Running common build flow of AnythingLLM image for all architectures"
 
+USER anythingllm
 WORKDIR /app
 
 # Install frontend dependencies
-FROM base as frontend-deps
+FROM build as frontend-deps
 
 COPY ./frontend/package.json ./frontend/yarn.lock ./frontend/
-RUN cd ./frontend/ && yarn install && yarn cache clean
+RUN cd ./frontend/ && yarn install --network-timeout 100000 && yarn cache clean
 
 # Install server dependencies
-FROM base as server-deps
+FROM build as server-deps
 COPY ./server/package.json ./server/yarn.lock ./server/
-RUN cd ./server/ && yarn install --production && yarn cache clean && \
+RUN cd ./server/ && yarn install --production --network-timeout 100000 && yarn cache clean
-    rm /app/server/node_modules/vectordb/x86_64-apple-darwin.node && \
-    rm /app/server/node_modules/vectordb/aarch64-apple-darwin.node
 
 # Compile Llama.cpp bindings for node-llama-cpp for this operating system.
 USER root
@@ -75,7 +133,7 @@ COPY --from=build-stage /app/frontend/dist ./server/public
 COPY --chown=anythingllm:anythingllm ./collector/ ./collector/
 
 # Install collector dependencies
-RUN cd /app/collector && yarn install --production && yarn cache clean
+RUN cd /app/collector && yarn install --production --network-timeout 100000 && yarn cache clean
 
 # Migrate and Run Prisma against known schema
 RUN cd ./server && npx prisma generate --schema=./prisma/schema.prisma
@@ -83,6 +141,7 @@ RUN cd ./server && npx prisma migrate deploy --schema=./prisma/schema.prisma
 
 # Setup the environment
 ENV NODE_ENV=production
+ENV ANYTHING_LLM_RUNTIME=docker
 
 # Expose the server port
 EXPOSE 3001
@@ -92,4 +151,4 @@ HEALTHCHECK --interval=1m --timeout=10s --start-period=1m \
   CMD /bin/bash /usr/local/bin/docker-healthcheck.sh || exit 1
 
 # Run the server
-ENTRYPOINT ["/bin/bash", "/usr/local/bin/docker-entrypoint.sh"]
+ENTRYPOINT ["/bin/bash", "/usr/local/bin/docker-entrypoint.sh"]

docker/HOW_TO_USE_DOCKER.md

@@ -29,7 +29,20 @@ Use the Dockerized version of AnythingLLM for a much faster and complete startup
 > It is best to mount the containers storage volume to a folder on your host machine
 > so that you can pull in future updates without deleting your existing data!
 
-`docker pull mintplexlabs/anythingllm:master`
+Pull in the latest image from docker. Supports both `amd64` and `arm64` CPU architectures.
+```shell
+docker pull mintplexlabs/anythingllm
+```
+
+<table>
+<tr>
+<th colspan="2">Mount the storage locally and run AnythingLLM in Docker</th>
+</tr>
+<tr>
+<td>
+  Linux/MacOs
+</td>
+<td>
 
 ```shell
 export STORAGE_LOCATION=$HOME/anythingllm && \
@@ -40,9 +53,34 @@ docker run -d -p 3001:3001 \
 -v ${STORAGE_LOCATION}:/app/server/storage \
 -v ${STORAGE_LOCATION}/.env:/app/server/.env \
 -e STORAGE_DIR="/app/server/storage" \
-mintplexlabs/anythingllm:master
+mintplexlabs/anythingllm
 ```
 
+</td>
+</tr>
+<tr>
+<td>
+  Windows
+</td>
+<td>
+
+```powershell
+# Run this in powershell terminal
+$env:STORAGE_LOCATION="$HOME\Documents\anythingllm"; `
+If(!(Test-Path $env:STORAGE_LOCATION)) {New-Item $env:STORAGE_LOCATION -ItemType Directory}; `
+If(!(Test-Path "$env:STORAGE_LOCATION\.env")) {New-Item "$env:STORAGE_LOCATION\.env"}; `
+docker run -d -p 3001:3001 `
+--cap-add SYS_ADMIN `
+-v "$env:STORAGE_LOCATION`:/app/server/storage" `
+-v "$env:STORAGE_LOCATION\.env:/app/server/.env" `
+-e STORAGE_DIR="/app/server/storage" `
+mintplexlabs/anythingllm;
+```
+
+</td>
+</tr>
+</table>
+
 Go to `http://localhost:3001` and you are now using AnythingLLM! All your data and progress will persist between
 container rebuilds or pulls from Docker Hub.
 

docker/docker-compose.yml

@@ -1,4 +1,4 @@
-version: '3.9'
+version: "3.9"
 
 name: anythingllm
 
@@ -10,7 +10,6 @@ services:
   anything-llm:
     container_name: anything-llm
     image: anything-llm:latest
-    platform: linux/amd64
     build:
       context: ../.
       dockerfile: ./docker/Dockerfile

eslint.config.js

@@ -0,0 +1,90 @@
+import globals from "./server/node_modules/globals/index.js"
+import eslintRecommended from "./server/node_modules/@eslint/js/src/index.js"
+import eslintConfigPrettier from "./server/node_modules/eslint-config-prettier/index.js"
+import prettier from "./server/node_modules/eslint-plugin-prettier/eslint-plugin-prettier.js"
+import react from "./server/node_modules/eslint-plugin-react/index.js"
+import reactRefresh from "./server/node_modules/eslint-plugin-react-refresh/index.js"
+import reactHooks from "./server/node_modules/eslint-plugin-react-hooks/index.js"
+import ftFlow from "./server/node_modules/eslint-plugin-ft-flow/dist/index.js"
+import hermesParser from "./server/node_modules/hermes-eslint/dist/index.js"
+
+const reactRecommended = react.configs.recommended
+const jsxRuntime = react.configs["jsx-runtime"]
+
+export default [
+  eslintRecommended.configs.recommended,
+  eslintConfigPrettier,
+  {
+    ignores: ["**/*.test.js"],
+    languageOptions: {
+      parser: hermesParser,
+      parserOptions: {
+        ecmaFeatures: { jsx: true }
+      },
+      ecmaVersion: 2020,
+      sourceType: "module",
+      globals: {
+        ...globals.browser,
+        ...globals.es2020,
+        ...globals.node
+      }
+    },
+    linterOptions: { reportUnusedDisableDirectives: true },
+    settings: { react: { version: "18.2" } },
+    plugins: {
+      ftFlow,
+      react,
+      "jsx-runtime": jsxRuntime,
+      "react-hooks": reactHooks,
+      prettier
+    },
+    rules: {
+      ...reactRecommended.rules,
+      ...reactHooks.configs.recommended.rules,
+      ...ftFlow.recommended,
+      "no-unused-vars": "warn",
+      "no-undef": "warn",
+      "no-empty": "warn",
+      "no-extra-boolean-cast": "warn",
+      "prettier/prettier": "warn"
+    }
+  },
+  {
+    files: ["frontend/src/**/*.js"],
+    plugins: {
+      ftFlow,
+      prettier
+    },
+    rules: {
+      "prettier/prettier": "warn"
+    }
+  },
+  {
+    files: [
+      "server/endpoints/**/*.js",
+      "server/models/**/*.js",
+      "server/swagger/**/*.js",
+      "server/utils/**/*.js",
+      "server/index.js"
+    ],
+    rules: {
+      "no-undef": "warn"
+    }
+  },
+  {
+    files: ["frontend/src/**/*.jsx"],
+    plugins: {
+      ftFlow,
+      react,
+      "jsx-runtime": jsxRuntime,
+      "react-hooks": reactHooks,
+      "react-refresh": reactRefresh,
+      prettier
+    },
+    rules: {
+      ...jsxRuntime.rules,
+      "react/prop-types": "off", // FIXME
+      "react-refresh/only-export-components": "warn"
+    }
+  }
+]

frontend/.env.example

@@ -1,2 +1,3 @@
 VITE_API_BASE='http://localhost:3001/api' # Use this URL when developing locally
+# VITE_API_BASE="https://$CODESPACE_NAME-3001.$GITHUB_CODESPACES_PORT_FORWARDING_DOMAIN/api" # for Github Codespaces
 # VITE_API_BASE='/api' # Use this URL deploying on non-localhost address OR in docker.

frontend/.eslintrc.cjs

@@ -1,20 +0,0 @@
-module.exports = {
-  "env": { "browser": true, "es2020": true },
-  "extends": [
-    "eslint:recommended",
-    "plugin:react/recommended",
-    "plugin:react/jsx-runtime",
-    "plugin:react-hooks/recommended"
-  ],
-  "files": ["**/*.js", "**/*.jsx"],
-  "linterOptions": { "reportUnusedDisableDirectives": true },
-  "parserOptions": { "ecmaVersion": "latest", "sourceType": "module", "ecmaFeatures": { "jsx": true } },
-  "settings": { "react": { "version": '18.2' } },
-  "plugins": [
-    "react-refresh",
-    "react-hooks"
-  ],
-  "rules": {
-    "react-refresh/only-export-components": "warn"
-  }
-}

frontend/.gitignore

@@ -9,6 +9,7 @@ lerna-debug.log*
 
 node_modules
 dist
+lib
 dist-ssr
 *.local
 
@@ -24,3 +25,4 @@ dist-ssr
 *.sw?
 bundleinspector.html
 .env.production
+flow-typed

frontend/package.json

@@ -1,10 +1,11 @@
 {
   "name": "anything-llm-frontend",
   "private": false,
-  "type": "module",
   "license": "MIT",
+  "type": "module",
   "scripts": {
     "start": "vite --open",
+    "dev": "NODE_ENV=development vite --debug --host=0.0.0.0",
     "build": "vite build",
     "lint": "yarn prettier --write ./src",
     "preview": "vite preview"
@@ -34,16 +35,24 @@
     "uuid": "^9.0.0"
   },
   "devDependencies": {
-    "@types/react": "^18.0.28",
+    "@types/react": "^18.2.23",
-    "@types/react-dom": "^18.0.11",
+    "@types/react-dom": "^18.2.8",
+    "@types/react-router-dom": "^5.3.3",
     "@vitejs/plugin-react": "^4.0.0-beta.0",
     "autoprefixer": "^10.4.14",
-    "eslint": "^8.38.0",
+    "eslint": "^8.50.0",
-    "eslint-plugin-react": "^7.32.2",
+    "eslint-config-prettier": "^9.0.0",
+    "eslint-plugin-ft-flow": "^3.0.0",
+    "eslint-plugin-prettier": "^5.0.0",
+    "eslint-plugin-react": "^7.33.2",
     "eslint-plugin-react-hooks": "^4.6.0",
-    "eslint-plugin-react-refresh": "^0.3.4",
+    "eslint-plugin-react-refresh": "^0.4.3",
+    "flow-bin": "^0.217.0",
+    "flow-remove-types": "^2.217.1",
+    "globals": "^13.21.0",
+    "hermes-eslint": "^0.15.0",
     "postcss": "^8.4.23",
-    "prettier": "^2.4.1",
+    "prettier": "^3.0.3",
     "rollup-plugin-visualizer": "^5.9.0",
     "tailwindcss": "^3.3.1",
     "vite": "^4.3.0"

frontend/src/App.jsx

@@ -120,6 +120,7 @@ export default function App() {
 
               {/* Onboarding Flow */}
               <Route path="/onboarding" element={<OnboardingFlow />} />
+              <Route path="/onboarding/:step" element={<OnboardingFlow />} />
             </Routes>
             <ToastContainer />
           </PfpProvider>

frontend/src/components/DefaultChat/index.jsx

@@ -16,9 +16,11 @@ import System from "@/models/system";
 import Jazzicon from "../UserIcon";
 import { userFromStorage } from "@/utils/request";
 import { AI_BACKGROUND_COLOR, USER_BACKGROUND_COLOR } from "@/utils/constants";
+import useUser from "@/hooks/useUser";
 
 export default function DefaultChatContainer() {
   const [mockMsgs, setMockMessages] = useState([]);
+  const { user } = useUser();
   const [fetchedMessages, setFetchedMessages] = useState([]);
   const {
     showing: showingNewWsModal,
@@ -163,13 +165,15 @@ export default function DefaultChatContainer() {
                 You can add and remove files at anytime.
               </span>
 
-              <button
+              {(!user || user?.role !== "default") && (
-                onClick={showNewWsModal}
+                <button
-                className="mt-5 w-fit transition-all duration-300 border border-slate-200 px-4 py-2 rounded-lg text-white text-sm items-center flex gap-x-2 hover:bg-slate-200 hover:text-slate-800 focus:ring-gray-800"
+                  onClick={showNewWsModal}
-              >
+                  className="mt-5 w-fit transition-all duration-300 border border-slate-200 px-4 py-2 rounded-lg text-white text-sm items-center flex gap-x-2 hover:bg-slate-200 hover:text-slate-800 focus:ring-gray-800"
-                <Plus className="h-4 w-4" />
+                >
-                <p>Create your first workspace</p>
+                  <Plus className="h-4 w-4" />
-              </button>
+                  <p>Create your first workspace</p>
+                </button>
+              )}
             </div>
           </div>
         </div>

frontend/src/components/EmbeddingSelection/AzureAiOptions/index.jsx

@@ -1,53 +1,55 @@
 export default function AzureAiOptions({ settings }) {
   return (
-    <>
+    <div className="w-full flex flex-col gap-y-4">
-      <div className="flex flex-col w-60">
+      <div className="w-full flex items-center gap-4">
-        <label className="text-white text-sm font-semibold block mb-4">
+        <div className="flex flex-col w-60">
-          Azure Service Endpoint
+          <label className="text-white text-sm font-semibold block mb-4">
-        </label>
+            Azure Service Endpoint
-        <input
+          </label>
-          type="url"
+          <input
-          name="AzureOpenAiEndpoint"
+            type="url"
-          className="bg-zinc-900 text-white placeholder-white placeholder-opacity-60 text-sm rounded-lg focus:border-white block w-full p-2.5"
+            name="AzureOpenAiEndpoint"
-          placeholder="https://my-azure.openai.azure.com"
+            className="bg-zinc-900 text-white placeholder-white placeholder-opacity-60 text-sm rounded-lg focus:border-white block w-full p-2.5"
-          defaultValue={settings?.AzureOpenAiEndpoint}
+            placeholder="https://my-azure.openai.azure.com"
-          required={true}
+            defaultValue={settings?.AzureOpenAiEndpoint}
-          autoComplete="off"
+            required={true}
-          spellCheck={false}
+            autoComplete="off"
-        />
+            spellCheck={false}
-      </div>
+          />
+        </div>
 
-      <div className="flex flex-col w-60">
+        <div className="flex flex-col w-60">
-        <label className="text-white text-sm font-semibold block mb-4">
+          <label className="text-white text-sm font-semibold block mb-4">
-          API Key
+            API Key
-        </label>
+          </label>
-        <input
+          <input
-          type="password"
+            type="password"
-          name="AzureOpenAiKey"
+            name="AzureOpenAiKey"
-          className="bg-zinc-900 text-white placeholder-white placeholder-opacity-60 text-sm rounded-lg focus:border-white block w-full p-2.5"
+            className="bg-zinc-900 text-white placeholder-white placeholder-opacity-60 text-sm rounded-lg focus:border-white block w-full p-2.5"
-          placeholder="Azure OpenAI API Key"
+            placeholder="Azure OpenAI API Key"
-          defaultValue={settings?.AzureOpenAiKey ? "*".repeat(20) : ""}
+            defaultValue={settings?.AzureOpenAiKey ? "*".repeat(20) : ""}
-          required={true}
+            required={true}
-          autoComplete="off"
+            autoComplete="off"
-          spellCheck={false}
+            spellCheck={false}
-        />
+          />
-      </div>
+        </div>
 
-      <div className="flex flex-col w-60">
+        <div className="flex flex-col w-60">
-        <label className="text-white text-sm font-semibold block mb-4">
+          <label className="text-white text-sm font-semibold block mb-4">
-          Embedding Deployment Name
+            Embedding Deployment Name
-        </label>
+          </label>
-        <input
+          <input
-          type="text"
+            type="text"
-          name="AzureOpenAiEmbeddingModelPref"
+            name="AzureOpenAiEmbeddingModelPref"
-          className="bg-zinc-900 text-white placeholder-white placeholder-opacity-60 text-sm rounded-lg focus:border-white block w-full p-2.5"
+            className="bg-zinc-900 text-white placeholder-white placeholder-opacity-60 text-sm rounded-lg focus:border-white block w-full p-2.5"
-          placeholder="Azure OpenAI embedding model deployment name"
+            placeholder="Azure OpenAI embedding model deployment name"
-          defaultValue={settings?.AzureOpenAiEmbeddingModelPref}
+            defaultValue={settings?.AzureOpenAiEmbeddingModelPref}
-          required={true}
+            required={true}
-          autoComplete="off"
+            autoComplete="off"
-          spellCheck={false}
+            spellCheck={false}
-        />
+          />
+        </div>
       </div>
-    </>
+    </div>
   );
 }

frontend/src/components/EmbeddingSelection/EmbedderItem/index.jsx

@@ -0,0 +1,39 @@
+export default function EmbedderItem({
+  name,
+  value,
+  image,
+  description,
+  checked,
+  onClick,
+}) {
+  return (
+    <div
+      onClick={() => onClick(value)}
+      className={`w-full hover:bg-white/10 p-2 rounded-md hover:cursor-pointer ${
+        checked && "bg-white/10"
+      }`}
+    >
+      <input
+        type="checkbox"
+        value={value}
+        className="peer hidden"
+        checked={checked}
+        readOnly={true}
+        formNoValidate={true}
+      />
+      <div className="flex gap-x-4 items-center">
+        <img
+          src={image}
+          alt={`${name} logo`}
+          className="w-10 h-10 rounded-md"
+        />
+        <div className="flex flex-col gap-y-1">
+          <div className="text-sm font-semibold">{name}</div>
+          <div className="mt-2 text-xs text-white tracking-wide">
+            {description}
+          </div>
+        </div>
+      </div>
+    </div>
+  );
+}

frontend/src/components/EmbeddingSelection/LocalAiOptions/index.jsx

@@ -1,4 +1,4 @@
-import { useEffect, useState } from "react";
+import React, { useEffect, useState } from "react";
 import System from "@/models/system";
 
 export default function LocalAiOptions({ settings }) {
@@ -10,7 +10,7 @@ export default function LocalAiOptions({ settings }) {
   const [apiKey, setApiKey] = useState(settings?.LocalAiApiKey);
 
   return (
-    <>
+    <div className="w-full flex flex-col gap-y-4">
       <div className="w-full flex items-center gap-4">
         <div className="flex flex-col w-60">
           <label className="text-white text-sm font-semibold block mb-4">
@@ -54,14 +54,11 @@ export default function LocalAiOptions({ settings }) {
       <div className="w-full flex items-center gap-4">
         <div className="flex flex-col w-60">
           <div className="flex flex-col gap-y-1 mb-4">
-            <label className="text-white text-sm font-semibold block">
+            <label className="text-white text-sm font-semibold flex items-center gap-x-2">
-              Local AI API Key
+              Local AI API Key{" "}
+              <p className="!text-xs !italic !font-thin">optional</p>
             </label>
-            <p className="text-xs italic text-white/60">
-              optional API key to use if running LocalAI with API keys.
-            </p>
           </div>
-
           <input
             type="password"
             name="LocalAiApiKey"
@@ -75,7 +72,7 @@ export default function LocalAiOptions({ settings }) {
           />
         </div>
       </div>
-    </>
+    </div>
   );
 }
 

frontend/src/components/EmbeddingSelection/OpenAiOptions/index.jsx

@@ -1,34 +1,36 @@
 export default function OpenAiOptions({ settings }) {
   return (
-    <>
+    <div className="w-full flex flex-col gap-y-4">
-      <div className="flex flex-col w-60">
+      <div className="w-full flex items-center gap-4">
-        <label className="text-white text-sm font-semibold block mb-4">
+        <div className="flex flex-col w-60">
-          API Key
+          <label className="text-white text-sm font-semibold block mb-4">
-        </label>
+            API Key
-        <input
+          </label>
-          type="password"
+          <input
-          name="OpenAiKey"
+            type="password"
-          className="bg-zinc-900 text-white placeholder-white placeholder-opacity-60 text-sm rounded-lg focus:border-white block w-full p-2.5"
+            name="OpenAiKey"
-          placeholder="OpenAI API Key"
+            className="bg-zinc-900 text-white placeholder-white placeholder-opacity-60 text-sm rounded-lg focus:border-white block w-full p-2.5"
-          defaultValue={settings?.OpenAiKey ? "*".repeat(20) : ""}
+            placeholder="OpenAI API Key"
-          required={true}
+            defaultValue={settings?.OpenAiKey ? "*".repeat(20) : ""}
-          autoComplete="off"
+            required={true}
-          spellCheck={false}
+            autoComplete="off"
-        />
+            spellCheck={false}
+          />
+        </div>
+        <div className="flex flex-col w-60">
+          <label className="text-white text-sm font-semibold block mb-4">
+            Model Preference
+          </label>
+          <select
+            disabled={true}
+            className="cursor-not-allowed bg-zinc-900 border border-gray-500 text-white text-sm rounded-lg block w-full p-2.5"
+          >
+            <option disabled={true} selected={true}>
+              text-embedding-ada-002
+            </option>
+          </select>
+        </div>
       </div>
-      <div className="flex flex-col w-60">
+    </div>
-        <label className="text-white text-sm font-semibold block mb-4">
-          Model Preference
-        </label>
-        <select
-          disabled={true}
-          className="cursor-not-allowed bg-zinc-900 border border-gray-500 text-white text-sm rounded-lg block w-full p-2.5"
-        >
-          <option disabled={true} selected={true}>
-            text-embedding-ada-002
-          </option>
-        </select>
-      </div>
-    </>
   );
 }

frontend/src/components/LLMSelection/AzureAiOptions/index.jsx

@@ -1,87 +1,92 @@
 export default function AzureAiOptions({ settings }) {
   return (
-    <>
+    <div className="w-full flex flex-col gap-y-4">
-      <div className="flex flex-col w-60">
+      <div className="w-full flex items-center gap-4">
-        <label className="text-white text-sm font-semibold block mb-4">
+        <div className="flex flex-col w-60">
-          Azure Service Endpoint
+          <label className="text-white text-sm font-semibold block mb-4">
-        </label>
+            Azure Service Endpoint
-        <input
+          </label>
-          type="url"
+          <input
-          name="AzureOpenAiEndpoint"
+            type="url"
-          className="bg-zinc-900 text-white placeholder-white placeholder-opacity-60 text-sm rounded-lg focus:border-white block w-full p-2.5"
+            name="AzureOpenAiEndpoint"
-          placeholder="https://my-azure.openai.azure.com"
+            className="bg-zinc-900 text-white placeholder-white placeholder-opacity-60 text-sm rounded-lg focus:border-white block w-full p-2.5"
-          defaultValue={settings?.AzureOpenAiEndpoint}
+            placeholder="https://my-azure.openai.azure.com"
-          required={true}
+            defaultValue={settings?.AzureOpenAiEndpoint}
-          autoComplete="off"
+            required={true}
-          spellCheck={false}
+            autoComplete="off"
-        />
+            spellCheck={false}
+          />
+        </div>
+
+        <div className="flex flex-col w-60">
+          <label className="text-white text-sm font-semibold block mb-4">
+            API Key
+          </label>
+          <input
+            type="password"
+            name="AzureOpenAiKey"
+            className="bg-zinc-900 text-white placeholder-white placeholder-opacity-60 text-sm rounded-lg focus:border-white block w-full p-2.5"
+            placeholder="Azure OpenAI API Key"
+            defaultValue={settings?.AzureOpenAiKey ? "*".repeat(20) : ""}
+            required={true}
+            autoComplete="off"
+            spellCheck={false}
+          />
+        </div>
+
+        <div className="flex flex-col w-60">
+          <label className="text-white text-sm font-semibold block mb-4">
+            Chat Deployment Name
+          </label>
+          <input
+            type="text"
+            name="AzureOpenAiModelPref"
+            className="bg-zinc-900 text-white placeholder-white placeholder-opacity-60 text-sm rounded-lg focus:border-white block w-full p-2.5"
+            placeholder="Azure OpenAI chat model deployment name"
+            defaultValue={settings?.AzureOpenAiModelPref}
+            required={true}
+            autoComplete="off"
+            spellCheck={false}
+          />
+        </div>
       </div>
 
-      <div className="flex flex-col w-60">
+      <div className="w-full flex items-center gap-4">
-        <label className="text-white text-sm font-semibold block mb-4">
+        <div className="flex flex-col w-60">
-          API Key
+          <label className="text-white text-sm font-semibold block mb-4">
-        </label>
+            Chat Model Token Limit
-        <input
+          </label>
-          type="password"
+          <select
-          name="AzureOpenAiKey"
+            name="AzureOpenAiTokenLimit"
-          className="bg-zinc-900 text-white placeholder-white placeholder-opacity-60 text-sm rounded-lg focus:border-white block w-full p-2.5"
+            defaultValue={settings?.AzureOpenAiTokenLimit || 4096}
-          placeholder="Azure OpenAI API Key"
+            className="bg-zinc-900 text-white placeholder-white placeholder-opacity-60 text-sm rounded-lg focus:border-white block w-full p-2.5"
-          defaultValue={settings?.AzureOpenAiKey ? "*".repeat(20) : ""}
+            required={true}
-          required={true}
+          >
-          autoComplete="off"
+            <option value={4096}>4,096 (gpt-3.5-turbo)</option>
-          spellCheck={false}
+            <option value={16384}>16,384 (gpt-3.5-16k)</option>
-        />
+            <option value={8192}>8,192 (gpt-4)</option>
-      </div>
+            <option value={32768}>32,768 (gpt-4-32k)</option>
+            <option value={128000}>128,000 (gpt-4-turbo)</option>
+          </select>
+        </div>
 
-      <div className="flex flex-col w-60">
+        <div className="flex flex-col w-60">
-        <label className="text-white text-sm font-semibold block mb-4">
+          <label className="text-white text-sm font-semibold block mb-4">
-          Chat Deployment Name
+            Embedding Deployment Name
-        </label>
+          </label>
-        <input
+          <input
-          type="text"
+            type="text"
-          name="AzureOpenAiModelPref"
+            name="AzureOpenAiEmbeddingModelPref"
-          className="bg-zinc-900 text-white placeholder-white placeholder-opacity-60 text-sm rounded-lg focus:border-white block w-full p-2.5"
+            className="bg-zinc-900 text-white placeholder-white placeholder-opacity-60 text-sm rounded-lg focus:border-white block w-full p-2.5"
-          placeholder="Azure OpenAI chat model deployment name"
+            placeholder="Azure OpenAI embedding model deployment name"
-          defaultValue={settings?.AzureOpenAiModelPref}
+            defaultValue={settings?.AzureOpenAiEmbeddingModelPref}
-          required={true}
+            required={true}
-          autoComplete="off"
+            autoComplete="off"
-          spellCheck={false}
+            spellCheck={false}
-        />
+          />
+        </div>
+        <div className="flex-flex-col w-60"></div>
       </div>
-
+    </div>
-      <div className="flex flex-col w-60">
-        <label className="text-white text-sm font-semibold block mb-4">
-          Chat Model Token Limit
-        </label>
-        <select
-          name="AzureOpenAiTokenLimit"
-          defaultValue={settings?.AzureOpenAiTokenLimit || 4096}
-          className="bg-zinc-900 text-white placeholder-white placeholder-opacity-60 text-sm rounded-lg focus:border-white block w-full p-2.5"
-          required={true}
-        >
-          <option value={4096}>4,096 (gpt-3.5-turbo)</option>
-          <option value={16384}>16,384 (gpt-3.5-16k)</option>
-          <option value={8192}>8,192 (gpt-4)</option>
-          <option value={32768}>32,768 (gpt-4-32k)</option>
-          <option value={128000}>128,000 (gpt-4-turbo)</option>
-        </select>
-      </div>
-
-      <div className="flex flex-col w-60">
-        <label className="text-white text-sm font-semibold block mb-4">
-          Embedding Deployment Name
-        </label>
-        <input
-          type="text"
-          name="AzureOpenAiEmbeddingModelPref"
-          className="bg-zinc-900 text-white placeholder-white placeholder-opacity-60 text-sm rounded-lg focus:border-white block w-full p-2.5"
-          placeholder="Azure OpenAI embedding model deployment name"
-          defaultValue={settings?.AzureOpenAiEmbeddingModelPref}
-          required={true}
-          autoComplete="off"
-          spellCheck={false}
-        />
-      </div>
-    </>
   );
 }

frontend/src/components/LLMSelection/GeminiLLMOptions/index.jsx

@@ -0,0 +1,43 @@
+export default function GeminiLLMOptions({ settings }) {
+  return (
+    <div className="w-full flex flex-col">
+      <div className="w-full flex items-center gap-4">
+        <div className="flex flex-col w-60">
+          <label className="text-white text-sm font-semibold block mb-4">
+            Google AI API Key
+          </label>
+          <input
+            type="password"
+            name="GeminiLLMApiKey"
+            className="bg-zinc-900 text-white placeholder-white placeholder-opacity-60 text-sm rounded-lg focus:border-white block w-full p-2.5"
+            placeholder="Google Gemini API Key"
+            defaultValue={settings?.GeminiLLMApiKey ? "*".repeat(20) : ""}
+            required={true}
+            autoComplete="off"
+            spellCheck={false}
+          />
+        </div>
+
+        <div className="flex flex-col w-60">
+          <label className="text-white text-sm font-semibold block mb-4">
+            Chat Model Selection
+          </label>
+          <select
+            name="GeminiLLMModelPref"
+            defaultValue={settings?.GeminiLLMModelPref || "gemini-pro"}
+            required={true}
+            className="bg-zinc-900 border border-gray-500 text-white text-sm rounded-lg block w-full p-2.5"
+          >
+            {["gemini-pro"].map((model) => {
+              return (
+                <option key={model} value={model}>
+                  {model}
+                </option>
+              );
+            })}
+          </select>
+        </div>
+      </div>
+    </div>
+  );
+}

frontend/src/components/LLMSelection/LLMItem/index.jsx

@@ -0,0 +1,39 @@
+export default function LLMItem({
+  name,
+  value,
+  image,
+  description,
+  checked,
+  onClick,
+}) {
+  return (
+    <div
+      onClick={() => onClick(value)}
+      className={`w-full hover:bg-white/10 p-2 rounded-md hover:cursor-pointer ${
+        checked && "bg-white/10"
+      }`}
+    >
+      <input
+        type="checkbox"
+        value={value}
+        className="peer hidden"
+        checked={checked}
+        readOnly={true}
+        formNoValidate={true}
+      />
+      <div className="flex gap-x-4 items-center">
+        <img
+          src={image}
+          alt={`${name} logo`}
+          className="w-10 h-10 rounded-md"
+        />
+        <div className="flex flex-col gap-y-1">
+          <div className="text-sm font-semibold">{name}</div>
+          <div className="mt-2 text-xs text-white tracking-wide">
+            {description}
+          </div>
+        </div>
+      </div>
+    </div>
+  );
+}

frontend/src/components/LLMSelection/LocalAiOptions/index.jsx

@@ -71,12 +71,10 @@ export default function LocalAiOptions({ settings, showAlert = false }) {
       <div className="w-full flex items-center gap-4">
         <div className="flex flex-col w-60">
           <div className="flex flex-col gap-y-1 mb-4">
-            <label className="text-white text-sm font-semibold block">
+            <label className="text-white text-sm font-semibold flex items-center gap-x-2">
-              Local AI API Key
+              Local AI API Key{" "}
+              <p className="!text-xs !italic !font-thin">optional</p>
             </label>
-            <p className="text-xs italic text-white/60">
-              optional API key to use if running LocalAI with API keys.
-            </p>
           </div>
 
           <input

frontend/src/components/LLMSelection/OllamaLLMOptions/index.jsx

@@ -0,0 +1,120 @@
+import { useEffect, useState } from "react";
+import System from "@/models/system";
+
+export default function OllamaLLMOptions({ settings }) {
+  const [basePathValue, setBasePathValue] = useState(
+    settings?.OllamaLLMBasePath
+  );
+  const [basePath, setBasePath] = useState(settings?.OllamaLLMBasePath);
+
+  return (
+    <div className="w-full flex flex-col gap-y-4">
+      <div className="w-full flex items-center gap-4">
+        <div className="flex flex-col w-60">
+          <label className="text-white text-sm font-semibold block mb-4">
+            Ollama Base URL
+          </label>
+          <input
+            type="url"
+            name="OllamaLLMBasePath"
+            className="bg-zinc-900 text-white placeholder-white placeholder-opacity-60 text-sm rounded-lg focus:border-white block w-full p-2.5"
+            placeholder="http://127.0.0.1:11434"
+            defaultValue={settings?.OllamaLLMBasePath}
+            required={true}
+            autoComplete="off"
+            spellCheck={false}
+            onChange={(e) => setBasePathValue(e.target.value)}
+            onBlur={() => setBasePath(basePathValue)}
+          />
+        </div>
+        <OllamaLLMModelSelection settings={settings} basePath={basePath} />
+        <div className="flex flex-col w-60">
+          <label className="text-white text-sm font-semibold block mb-4">
+            Token context window
+          </label>
+          <input
+            type="number"
+            name="OllamaLLMTokenLimit"
+            className="bg-zinc-900 text-white placeholder-white placeholder-opacity-60 text-sm rounded-lg focus:border-white block w-full p-2.5"
+            placeholder="4096"
+            min={1}
+            onScroll={(e) => e.target.blur()}
+            defaultValue={settings?.OllamaLLMTokenLimit}
+            required={true}
+            autoComplete="off"
+          />
+        </div>
+      </div>
+    </div>
+  );
+}
+
+function OllamaLLMModelSelection({ settings, basePath = null }) {
+  const [customModels, setCustomModels] = useState([]);
+  const [loading, setLoading] = useState(true);
+
+  useEffect(() => {
+    async function findCustomModels() {
+      if (!basePath) {
+        setCustomModels([]);
+        setLoading(false);
+        return;
+      }
+      setLoading(true);
+      const { models } = await System.customModels("ollama", null, basePath);
+      setCustomModels(models || []);
+      setLoading(false);
+    }
+    findCustomModels();
+  }, [basePath]);
+
+  if (loading || customModels.length == 0) {
+    return (
+      <div className="flex flex-col w-60">
+        <label className="text-white text-sm font-semibold block mb-4">
+          Chat Model Selection
+        </label>
+        <select
+          name="OllamaLLMModelPref"
+          disabled={true}
+          className="bg-zinc-900 border border-gray-500 text-white text-sm rounded-lg block w-full p-2.5"
+        >
+          <option disabled={true} selected={true}>
+            {!!basePath
+              ? "-- loading available models --"
+              : "-- waiting for URL --"}
+          </option>
+        </select>
+      </div>
+    );
+  }
+
+  return (
+    <div className="flex flex-col w-60">
+      <label className="text-white text-sm font-semibold block mb-4">
+        Chat Model Selection
+      </label>
+      <select
+        name="OllamaLLMModelPref"
+        required={true}
+        className="bg-zinc-900 border border-gray-500 text-white text-sm rounded-lg block w-full p-2.5"
+      >
+        {customModels.length > 0 && (
+          <optgroup label="Your loaded models">
+            {customModels.map((model) => {
+              return (
+                <option
+                  key={model.id}
+                  value={model.id}
+                  selected={settings.OllamaLLMModelPref === model.id}
+                >
+                  {model.id}
+                </option>
+              );
+            })}
+          </optgroup>
+        )}
+      </select>
+    </div>
+  );
+}

frontend/src/components/LLMSelection/OpenAiOptions/index.jsx

@@ -6,7 +6,7 @@ export default function OpenAiOptions({ settings }) {
   const [openAIKey, setOpenAIKey] = useState(settings?.OpenAiKey);
 
   return (
-    <>
+    <div className="flex gap-x-4">
       <div className="flex flex-col w-60">
         <label className="text-white text-sm font-semibold block mb-4">
           API Key
@@ -25,7 +25,7 @@ export default function OpenAiOptions({ settings }) {
         />
       </div>
       <OpenAIModelSelection settings={settings} apiKey={openAIKey} />
-    </>
+    </div>
   );
 }
 
@@ -87,7 +87,7 @@ function OpenAIModelSelection({ apiKey, settings }) {
                 <option
                   key={model}
                   value={model}
-                  selected={settings.OpenAiModelPref === model}
+                  selected={settings?.OpenAiModelPref === model}
                 >
                   {model}
                 </option>
@@ -102,7 +102,7 @@ function OpenAIModelSelection({ apiKey, settings }) {
                 <option
                   key={model.id}
                   value={model.id}
-                  selected={settings.OpenAiModelPref === model.id}
+                  selected={settings?.OpenAiModelPref === model.id}
                 >
                   {model.id}
                 </option>

frontend/src/components/Modals/MangeWorkspace/Documents/Directory/FileRow/index.jsx

@@ -60,7 +60,7 @@ export default function FileRow({
         selected ? "bg-sky-500/20" : ""
       } ${expanded ? "bg-sky-500/10" : ""}`}`}
     >
-      <div className="pl-2 col-span-6 flex gap-x-[4px] items-center">
+      <div className="pl-2 col-span-5 flex gap-x-[4px] items-center">
         <div
           className="shrink-0 w-3 h-3 rounded border-[1px] border-white flex justify-center items-center cursor-pointer"
           role="checkbox"
@@ -88,10 +88,12 @@ export default function FileRow({
           )}
         </div>
       </div>
-      <p className="col-span-2 pl-3.5 whitespace-nowrap">
+      <p className="col-span-3 pl-3.5 whitespace-nowrap">
         {formatDate(item?.published)}
       </p>
-      <p className="col-span-2 pl-2 uppercase">{getFileExtension(item.url)}</p>
+      <p className="col-span-2 pl-2 uppercase overflow-x-hidden">
+        {getFileExtension(item.url)}
+      </p>
       <div className="col-span-2 flex justify-end items-center">
         {item?.cached && (
           <div className="bg-white/10 rounded-3xl">

frontend/src/components/Modals/MangeWorkspace/Documents/Directory/index.jsx

@@ -71,8 +71,8 @@ export default function Directory({
 
         <div className="relative w-[560px] h-[310px] bg-zinc-900 rounded-2xl">
           <div className="rounded-t-2xl text-white/80 text-xs grid grid-cols-12 py-2 px-8 border-b border-white/20 shadow-lg bg-zinc-900 sticky top-0 z-10">
-            <p className="col-span-6">Name</p>
+            <p className="col-span-5">Name</p>
-            <p className="col-span-2">Date</p>
+            <p className="col-span-3">Date</p>
             <p className="col-span-2">Kind</p>
             <p className="col-span-2">Cached</p>
           </div>

frontend/src/components/Modals/MangeWorkspace/Documents/WorkspaceDirectory/WorkspaceFileRow/index.jsx

@@ -54,7 +54,7 @@ export default function WorkspaceFileRow({
       className={`items-center transition-all duration-200 text-white/80 text-xs grid grid-cols-12 py-2 pl-3.5 pr-8 border-b border-white/20 hover:bg-sky-500/20 cursor-pointer
           ${isMovedItem ? "bg-green-800/40" : ""}`}
     >
-      <div className="col-span-6 flex gap-x-[4px] items-center">
+      <div className="col-span-5 flex gap-x-[4px] items-center">
         <File
           className="text-base font-bold w-4 h-4 ml-3 mr-[3px]"
           weight="fill"
@@ -74,10 +74,12 @@ export default function WorkspaceFileRow({
           )}
         </div>
       </div>
-      <p className="col-span-2 pl-3.5 whitespace-nowrap">
+      <p className="col-span-3 pl-3.5 whitespace-nowrap">
         {formatDate(item?.published)}
       </p>
-      <p className="col-span-2 pl-2 uppercase">{getFileExtension(item.url)}</p>
+      <p className="col-span-2 pl-2 uppercase overflow-x-hidden">
+        {getFileExtension(item.url)}
+      </p>
       <div className="col-span-2 flex justify-end items-center">
         {item?.cached && (
           <div className="bg-white/10 rounded-3xl">

frontend/src/components/Modals/MangeWorkspace/Documents/WorkspaceDirectory/index.jsx

@@ -26,8 +26,8 @@ export default function WorkspaceDirectory({
         </div>
         <div className="relative w-[560px] h-[445px] bg-zinc-900 rounded-2xl mt-5">
           <div className="text-white/80 text-xs grid grid-cols-12 py-2 px-8 border-b border-white/20">
-            <p className="col-span-6">Name</p>
+            <p className="col-span-5">Name</p>
-            <p className="col-span-2">Date</p>
+            <p className="col-span-3">Date</p>
             <p className="col-span-2">Kind</p>
             <p className="col-span-2">Cached</p>
           </div>
@@ -55,8 +55,8 @@ export default function WorkspaceDirectory({
         }`}
       >
         <div className="text-white/80 text-xs grid grid-cols-12 py-2 px-8 border-b border-white/20 bg-zinc-900 sticky top-0 z-10">
-          <p className="col-span-6">Name</p>
+          <p className="col-span-5">Name</p>
-          <p className="col-span-2">Date</p>
+          <p className="col-span-3">Date</p>
           <p className="col-span-2">Kind</p>
           <p className="col-span-2">Cached</p>
         </div>

frontend/src/components/PrivateRoute/index.jsx

@@ -89,7 +89,7 @@ export function AdminRoute({ Component }) {
   if (isAuthd === null) return <FullScreenLoader />;
 
   if (shouldRedirectToOnboarding) {
-    return <Navigate to={paths.onboarding()} />;
+    return <Navigate to={paths.onboarding.home()} />;
   }
 
   const user = userFromStorage();
@@ -110,7 +110,7 @@ export function ManagerRoute({ Component }) {
   if (isAuthd === null) return <FullScreenLoader />;
 
   if (shouldRedirectToOnboarding) {
-    return <Navigate to={paths.onboarding()} />;
+    return <Navigate to={paths.onboarding.home()} />;
   }
 
   const user = userFromStorage();

frontend/src/components/VectorDBOption/index.jsx

@@ -1,39 +0,0 @@
-import React from "react";
-
-export default function VectorDBOption({
-  name,
-  link,
-  description,
-  value,
-  image,
-  checked = false,
-  onClick,
-}) {
-  return (
-    <div onClick={() => onClick(value)}>
-      <input
-        type="checkbox"
-        value={value}
-        className="peer hidden"
-        checked={checked}
-        readOnly={true}
-        formNoValidate={true}
-      />
-      <label className="transition-all duration-300 inline-flex flex-col h-full w-60 cursor-pointer items-start justify-between rounded-2xl bg-preference-gradient border-2 border-transparent shadow-md px-5 py-4 text-white hover:bg-selected-preference-gradient hover:text-underline hover:border-white/60 peer-checked:border-white peer-checked:border-opacity-90 peer-checked:bg-selected-preference-gradient">
-        <div className="flex items-center">
-          <img src={image} alt={name} className="h-10 w-10 rounded" />
-          <div className="ml-4 text-sm font-semibold">{name}</div>
-        </div>
-        <div className="mt-2 text-xs font-base text-white tracking-wide">
-          {description}
-        </div>
-        <a
-          href={`https://${link}`}
-          className="mt-2 text-xs text-white font-medium underline"
-        >
-          {link}
-        </a>
-      </label>
-    </div>
-  );
-}

frontend/src/components/VectorDBSelection/ChromaDBOptions/index.jsx

@@ -0,0 +1,51 @@
+export default function ChromaDBOptions({ settings }) {
+  return (
+    <div className="w-full flex flex-col gap-y-4">
+      <div className="w-full flex items-center gap-4">
+        <div className="flex flex-col w-60">
+          <label className="text-white text-sm font-semibold block mb-4">
+            Chroma Endpoint
+          </label>
+          <input
+            type="url"
+            name="ChromaEndpoint"
+            className="bg-zinc-900 text-white placeholder-white placeholder-opacity-60 text-sm rounded-lg focus:border-white block w-full p-2.5"
+            placeholder="http://localhost:8000"
+            defaultValue={settings?.ChromaEndpoint}
+            required={true}
+            autoComplete="off"
+            spellCheck={false}
+          />
+        </div>
+
+        <div className="flex flex-col w-60">
+          <label className="text-white text-sm font-semibold block mb-4">
+            API Header
+          </label>
+          <input
+            name="ChromaApiHeader"
+            autoComplete="off"
+            type="text"
+            defaultValue={settings?.ChromaApiHeader}
+            className="bg-zinc-900 text-white placeholder-white placeholder-opacity-60 text-sm rounded-lg focus:border-white block w-full p-2.5"
+            placeholder="X-Api-Key"
+          />
+        </div>
+
+        <div className="flex flex-col w-60">
+          <label className="text-white text-sm font-semibold block mb-4">
+            API Key
+          </label>
+          <input
+            name="ChromaApiKey"
+            autoComplete="off"
+            type="password"
+            defaultValue={settings?.ChromaApiKey ? "*".repeat(20) : ""}
+            className="bg-zinc-900 text-white placeholder-white placeholder-opacity-60 text-sm rounded-lg focus:border-white block w-full p-2.5"
+            placeholder="sk-myApiKeyToAccessMyChromaInstance"
+          />
+        </div>
+      </div>
+    </div>
+  );
+}

frontend/src/components/VectorDBSelection/LanceDBOptions/index.jsx

@@ -0,0 +1,9 @@
+export default function LanceDBOptions() {
+  return (
+    <div className="w-full h-10 items-center justify-center flex">
+      <p className="text-sm font-base text-white text-opacity-60">
+        There is no configuration needed for LanceDB.
+      </p>
+    </div>
+  );
+}

frontend/src/components/VectorDBSelection/PineconeDBOptions/index.jsx

@@ -0,0 +1,55 @@
+export default function PineconeDBOptions({ settings }) {
+  return (
+    <div className="w-full flex flex-col gap-y-4">
+      <div className="w-full flex items-center gap-4">
+        <div className="flex flex-col w-60">
+          <label className="text-white text-sm font-semibold block mb-4">
+            Pinecone DB API Key
+          </label>
+          <input
+            type="password"
+            name="PineConeKey"
+            className="bg-zinc-900 text-white placeholder-white placeholder-opacity-60 text-sm rounded-lg focus:border-white block w-full p-2.5"
+            placeholder="Pinecone API Key"
+            defaultValue={settings?.PineConeKey ? "*".repeat(20) : ""}
+            required={true}
+            autoComplete="off"
+            spellCheck={false}
+          />
+        </div>
+
+        <div className="flex flex-col w-60">
+          <label className="text-white text-sm font-semibold block mb-4">
+            Pinecone Index Environment
+          </label>
+          <input
+            type="text"
+            name="PineConeEnvironment"
+            className="bg-zinc-900 text-white placeholder-white placeholder-opacity-60 text-sm rounded-lg focus:border-white block w-full p-2.5"
+            placeholder="us-gcp-west-1"
+            defaultValue={settings?.PineConeEnvironment}
+            required={true}
+            autoComplete="off"
+            spellCheck={false}
+          />
+        </div>
+
+        <div className="flex flex-col w-60">
+          <label className="text-white text-sm font-semibold block mb-4">
+            Pinecone Index Name
+          </label>
+          <input
+            type="text"
+            name="PineConeIndex"
+            className="bg-zinc-900 text-white placeholder-white placeholder-opacity-60 text-sm rounded-lg focus:border-white block w-full p-2.5"
+            placeholder="my-index"
+            defaultValue={settings?.PineConeIndex}
+            required={true}
+            autoComplete="off"
+            spellCheck={false}
+          />
+        </div>
+      </div>
+    </div>
+  );
+}

frontend/src/components/VectorDBSelection/QDrantDBOptions/index.jsx

@@ -0,0 +1,38 @@
+export default function QDrantDBOptions({ settings }) {
+  return (
+    <div className="w-full flex flex-col gap-y-4">
+      <div className="w-full flex items-center gap-4">
+        <div className="flex flex-col w-60">
+          <label className="text-white text-sm font-semibold block mb-4">
+            QDrant API Endpoint
+          </label>
+          <input
+            type="url"
+            name="QdrantEndpoint"
+            className="bg-zinc-900 text-white placeholder-white placeholder-opacity-60 text-sm rounded-lg focus:border-white block w-full p-2.5"
+            placeholder="http://localhost:6633"
+            defaultValue={settings?.QdrantEndpoint}
+            required={true}
+            autoComplete="off"
+            spellCheck={false}
+          />
+        </div>
+
+        <div className="flex flex-col w-60">
+          <label className="text-white text-sm font-semibold block mb-4">
+            API Key
+          </label>
+          <input
+            type="password"
+            name="QdrantApiKey"
+            className="bg-zinc-900 text-white placeholder-white placeholder-opacity-60 text-sm rounded-lg focus:border-white block w-full p-2.5"
+            placeholder="wOeqxsYP4....1244sba"
+            defaultValue={settings?.QdrantApiKey}
+            autoComplete="off"
+            spellCheck={false}
+          />
+        </div>
+      </div>
+    </div>
+  );
+}

frontend/src/components/VectorDBSelection/VectorDBItem/index.jsx

@@ -0,0 +1,37 @@
+export default function VectorDBItem({
+  name,
+  value,
+  image,
+  description,
+  checked,
+  onClick,
+}) {
+  return (
+    <div
+      onClick={() => onClick(value)}
+      className={`w-full hover:bg-white/10 p-2 rounded-md hover:cursor-pointer ${
+        checked ? "bg-white/10" : ""
+      }`}
+    >
+      <input
+        type="checkbox"
+        value={value}
+        className="peer hidden"
+        checked={checked}
+        readOnly={true}
+        formNoValidate={true}
+      />
+      <div className="flex gap-x-4 items-center">
+        <img
+          src={image}
+          alt={`${name} logo`}
+          className="w-10 h-10 rounded-md"
+        />
+        <div className="flex flex-col gap-y-1">
+          <div className="text-sm font-semibold">{name}</div>
+          <div className="text-xs text-white tracking-wide">{description}</div>
+        </div>
+      </div>
+    </div>
+  );
+}

frontend/src/components/VectorDBSelection/WeaviateDBOptions/index.jsx

@@ -0,0 +1,38 @@
+export default function WeaviateDBOptions({ settings }) {
+  return (
+    <div className="w-full flex flex-col gap-y-4">
+      <div className="w-full flex items-center gap-4">
+        <div className="flex flex-col w-60">
+          <label className="text-white text-sm font-semibold block mb-4">
+            Weaviate Endpoint
+          </label>
+          <input
+            type="url"
+            name="WeaviateEndpoint"
+            className="bg-zinc-900 text-white placeholder-white placeholder-opacity-60 text-sm rounded-lg focus:border-white block w-full p-2.5"
+            placeholder="http://localhost:8080"
+            defaultValue={settings?.WeaviateEndpoint}
+            required={true}
+            autoComplete="off"
+            spellCheck={false}
+          />
+        </div>
+
+        <div className="flex flex-col w-60">
+          <label className="text-white text-sm font-semibold block mb-4">
+            API Key
+          </label>
+          <input
+            type="password"
+            name="WeaviateApiKey"
+            className="bg-zinc-900 text-white placeholder-white placeholder-opacity-60 text-sm rounded-lg focus:border-white block w-full p-2.5"
+            placeholder="sk-123Abcweaviate"
+            defaultValue={settings?.WeaviateApiKey}
+            autoComplete="off"
+            spellCheck={false}
+          />
+        </div>
+      </div>
+    </div>
+  );
+}

frontend/src/pages/GeneralSettings/EmbeddingPreference/index.jsx

@@ -8,25 +8,28 @@ import OpenAiLogo from "@/media/llmprovider/openai.png";
 import AzureOpenAiLogo from "@/media/llmprovider/azure.png";
 import LocalAiLogo from "@/media/llmprovider/localai.png";
 import PreLoader from "@/components/Preloader";
-import LLMProviderOption from "@/components/LLMSelection/LLMProviderOption";
 import ChangeWarningModal from "@/components/ChangeWarning";
 import OpenAiOptions from "@/components/EmbeddingSelection/OpenAiOptions";
 import AzureAiOptions from "@/components/EmbeddingSelection/AzureAiOptions";
 import LocalAiOptions from "@/components/EmbeddingSelection/LocalAiOptions";
 import NativeEmbeddingOptions from "@/components/EmbeddingSelection/NativeEmbeddingOptions";
+import EmbedderItem from "@/components/EmbeddingSelection/EmbedderItem";
+import { MagnifyingGlass } from "@phosphor-icons/react";
 
 export default function GeneralEmbeddingPreference() {
   const [saving, setSaving] = useState(false);
   const [hasChanges, setHasChanges] = useState(false);
   const [hasEmbeddings, setHasEmbeddings] = useState(false);
-  const [embeddingChoice, setEmbeddingChoice] = useState("openai");
   const [settings, setSettings] = useState(null);
   const [loading, setLoading] = useState(true);
+  const [searchQuery, setSearchQuery] = useState("");
+  const [filteredEmbedders, setFilteredEmbedders] = useState([]);
+  const [selectedEmbedder, setSelectedEmbedder] = useState(null);
 
   const handleSubmit = async (e) => {
     e.preventDefault();
     if (
-      embeddingChoice !== settings?.EmbeddingEngine &&
+      selectedEmbedder !== settings?.EmbeddingEngine &&
       hasChanges &&
       hasEmbeddings
     ) {
@@ -38,18 +41,18 @@ export default function GeneralEmbeddingPreference() {
 
   const handleSaveSettings = async () => {
     setSaving(true);
-    const data = new FormData(document.getElementById("embedding-form"));
+    const form = document.getElementById("embedding-form");
     const settingsData = {};
-    for (let [key, value] of data.entries()) {
+    const formData = new FormData(form);
-      settingsData[key] = value;
+    settingsData.EmbeddingEngine = selectedEmbedder;
-    }
+    for (var [key, value] of formData.entries()) settingsData[key] = value;
 
     const { error } = await System.updateSystem(settingsData);
     if (error) {
-      showToast(`Failed to save LLM settings: ${error}`, "error");
+      showToast(`Failed to save embedding settings: ${error}`, "error");
       setHasChanges(true);
     } else {
-      showToast("LLM preferences saved successfully.", "success");
+      showToast("Embedding preferences saved successfully.", "success");
       setHasChanges(false);
     }
     setSaving(false);
@@ -57,7 +60,7 @@ export default function GeneralEmbeddingPreference() {
   };
 
   const updateChoice = (selection) => {
-    setEmbeddingChoice(selection);
+    setSelectedEmbedder(selection);
     setHasChanges(true);
   };
 
@@ -65,13 +68,52 @@ export default function GeneralEmbeddingPreference() {
     async function fetchKeys() {
       const _settings = await System.keys();
       setSettings(_settings);
-      setEmbeddingChoice(_settings?.EmbeddingEngine || "openai");
+      setSelectedEmbedder(_settings?.EmbeddingEngine || "native");
       setHasEmbeddings(_settings?.HasExistingEmbeddings || false);
       setLoading(false);
     }
     fetchKeys();
   }, []);
 
+  const EMBEDDERS = [
+    {
+      name: "AnythingLLM Embedder",
+      value: "native",
+      logo: AnythingLLMIcon,
+      options: <NativeEmbeddingOptions settings={settings} />,
+      description:
+        "Use the built-in embedding engine for AnythingLLM. Zero setup!",
+    },
+    {
+      name: "OpenAI",
+      value: "openai",
+      logo: OpenAiLogo,
+      options: <OpenAiOptions settings={settings} />,
+      description: "The standard option for most non-commercial use.",
+    },
+    {
+      name: "Azure OpenAI",
+      value: "azure",
+      logo: AzureOpenAiLogo,
+      options: <AzureAiOptions settings={settings} />,
+      description: "The enterprise option of OpenAI hosted on Azure services.",
+    },
+    {
+      name: "Local AI",
+      value: "localai",
+      logo: LocalAiLogo,
+      options: <LocalAiOptions settings={settings} />,
+      description: "Run embedding models locally on your own machine.",
+    },
+  ];
+
+  useEffect(() => {
+    const filtered = EMBEDDERS.filter((embedder) =>
+      embedder.name.toLowerCase().includes(searchQuery.toLowerCase())
+    );
+    setFilteredEmbedders(filtered);
+  }, [searchQuery, selectedEmbedder]);
+
   return (
     <div className="w-screen h-screen overflow-hidden bg-sidebar flex">
       <ChangeWarningModal
@@ -98,7 +140,6 @@ export default function GeneralEmbeddingPreference() {
           <form
             id="embedding-form"
             onSubmit={handleSubmit}
-            onChange={() => setHasChanges(true)}
             className="flex w-full"
           >
             <div className="flex flex-col w-full px-1 md:px-20 md:py-12 py-16">
@@ -132,59 +173,52 @@ export default function GeneralEmbeddingPreference() {
                 <div className="text-white text-sm font-medium py-4">
                   Embedding Providers
                 </div>
-                <div className="w-full flex md:flex-wrap overflow-x-scroll gap-4 max-w-[900px]">
+                <div className="w-full">
-                  <input
+                  <div className="w-full relative border-slate-300/20 shadow border-4 rounded-xl text-white">
-                    hidden={true}
+                    <div className="w-full p-4 absolute top-0 rounded-t-lg backdrop-blur-sm">
-                    name="EmbeddingEngine"
+                      <div className="w-full flex items-center sticky top-0 z-20">
-                    value={embeddingChoice}
+                        <MagnifyingGlass
-                  />
+                          size={16}
-                  <LLMProviderOption
+                          weight="bold"
-                    name="AnythingLLM Embedder"
+                          className="absolute left-4 z-30 text-white"
-                    value="native"
+                        />
-                    description="Use the built-in embedding engine for AnythingLLM. Zero setup!"
+                        <input
-                    checked={embeddingChoice === "native"}
+                          type="text"
-                    image={AnythingLLMIcon}
+                          placeholder="Search Embedding providers"
-                    onClick={updateChoice}
+                          className="bg-zinc-600 z-20 pl-10 rounded-full w-full px-4 py-1 text-sm border-2 border-slate-300/40 outline-none focus:border-white text-white"
-                  />
+                          onChange={(e) => setSearchQuery(e.target.value)}
-                  <LLMProviderOption
+                          autoComplete="off"
-                    name="OpenAI"
+                          onKeyDown={(e) => {
-                    value="openai"
+                            if (e.key === "Enter") e.preventDefault();
-                    link="openai.com"
+                          }}
-                    description="Use OpenAI's text-embedding-ada-002 embedding model."
+                        />
-                    checked={embeddingChoice === "openai"}
+                      </div>
-                    image={OpenAiLogo}
+                    </div>
-                    onClick={updateChoice}
+                    <div className="px-4 pt-[70px] flex flex-col gap-y-1 max-h-[390px] overflow-y-auto no-scroll pb-4">
-                  />
+                      {filteredEmbedders.map((embedder) => {
-                  <LLMProviderOption
+                        return (
-                    name="Azure OpenAI"
+                          <EmbedderItem
-                    value="azure"
+                            key={embedder.name}
-                    link="azure.microsoft.com"
+                            name={embedder.name}
-                    description="The enterprise option of OpenAI hosted on Azure services."
+                            value={embedder.value}
-                    checked={embeddingChoice === "azure"}
+                            image={embedder.logo}
-                    image={AzureOpenAiLogo}
+                            description={embedder.description}
-                    onClick={updateChoice}
+                            checked={selectedEmbedder === embedder.value}
-                  />
+                            onClick={() => updateChoice(embedder.value)}
-                  <LLMProviderOption
+                          />
-                    name="LocalAI"
+                        );
-                    value="localai"
+                      })}
-                    link="localai.io"
+                    </div>
-                    description="Self hosted LocalAI embedding engine."
+                  </div>
-                    checked={embeddingChoice === "localai"}
+                  <div
-                    image={LocalAiLogo}
+                    onChange={() => setHasChanges(true)}
-                    onClick={updateChoice}
+                    className="mt-4 flex flex-col gap-y-1"
-                  />
+                  >
-                </div>
+                    {selectedEmbedder &&
-                <div className="mt-10 flex flex-wrap gap-4 max-w-[800px]">
+                      EMBEDDERS.find(
-                  {embeddingChoice === "native" && <NativeEmbeddingOptions />}
+                        (embedder) => embedder.value === selectedEmbedder
-                  {embeddingChoice === "openai" && (
+                      )?.options}
-                    <OpenAiOptions settings={settings} />
+                  </div>
-                  )}
-                  {embeddingChoice === "azure" && (
-                    <AzureAiOptions settings={settings} />
-                  )}
-                  {embeddingChoice === "localai" && (
-                    <LocalAiOptions settings={settings} />
-                  )}
                 </div>
               </>
             </div>

frontend/src/pages/GeneralSettings/LLMPreference/index.jsx

@@ -7,31 +7,44 @@ import AnythingLLMIcon from "@/media/logo/anything-llm-icon.png";
 import OpenAiLogo from "@/media/llmprovider/openai.png";
 import AzureOpenAiLogo from "@/media/llmprovider/azure.png";
 import AnthropicLogo from "@/media/llmprovider/anthropic.png";
+import GeminiLogo from "@/media/llmprovider/gemini.png";
+import OllamaLogo from "@/media/llmprovider/ollama.png";
 import LMStudioLogo from "@/media/llmprovider/lmstudio.png";
 import LocalAiLogo from "@/media/llmprovider/localai.png";
 import PreLoader from "@/components/Preloader";
-import LLMProviderOption from "@/components/LLMSelection/LLMProviderOption";
 import OpenAiOptions from "@/components/LLMSelection/OpenAiOptions";
 import AzureAiOptions from "@/components/LLMSelection/AzureAiOptions";
 import AnthropicAiOptions from "@/components/LLMSelection/AnthropicAiOptions";
 import LMStudioOptions from "@/components/LLMSelection/LMStudioOptions";
 import LocalAiOptions from "@/components/LLMSelection/LocalAiOptions";
 import NativeLLMOptions from "@/components/LLMSelection/NativeLLMOptions";
+import GeminiLLMOptions from "@/components/LLMSelection/GeminiLLMOptions";
+import OllamaLLMOptions from "@/components/LLMSelection/OllamaLLMOptions";
+import LLMItem from "@/components/LLMSelection/LLMItem";
+import { MagnifyingGlass } from "@phosphor-icons/react";
 
 export default function GeneralLLMPreference() {
   const [saving, setSaving] = useState(false);
   const [hasChanges, setHasChanges] = useState(false);
-  const [llmChoice, setLLMChoice] = useState("openai");
   const [settings, setSettings] = useState(null);
   const [loading, setLoading] = useState(true);
 
+  const [searchQuery, setSearchQuery] = useState("");
+  const [filteredLLMs, setFilteredLLMs] = useState([]);
+  const [selectedLLM, setSelectedLLM] = useState(null);
+
+  const isHosted = window.location.hostname.includes("useanything.com");
+
   const handleSubmit = async (e) => {
     e.preventDefault();
-    setSaving(true);
+    const form = e.target;
     const data = {};
-    const form = new FormData(e.target);
+    const formData = new FormData(form);
-    for (var [key, value] of form.entries()) data[key] = value;
+    data.LLMProvider = selectedLLM;
+    for (var [key, value] of formData.entries()) data[key] = value;
     const { error } = await System.updateSystem(data);
+    setSaving(true);
+
     if (error) {
       showToast(`Failed to save LLM settings: ${error}`, "error");
     } else {
@@ -42,7 +55,7 @@ export default function GeneralLLMPreference() {
   };
 
   const updateLLMChoice = (selection) => {
-    setLLMChoice(selection);
+    setSelectedLLM(selection);
     setHasChanges(true);
   };
 
@@ -50,12 +63,80 @@ export default function GeneralLLMPreference() {
     async function fetchKeys() {
       const _settings = await System.keys();
       setSettings(_settings);
-      setLLMChoice(_settings?.LLMProvider);
+      setSelectedLLM(_settings?.LLMProvider);
       setLoading(false);
     }
     fetchKeys();
   }, []);
 
+  useEffect(() => {
+    const filtered = LLMS.filter((llm) =>
+      llm.name.toLowerCase().includes(searchQuery.toLowerCase())
+    );
+    setFilteredLLMs(filtered);
+  }, [searchQuery, selectedLLM]);
+
+  const LLMS = [
+    {
+      name: "OpenAI",
+      value: "openai",
+      logo: OpenAiLogo,
+      options: <OpenAiOptions settings={settings} />,
+      description: "The standard option for most non-commercial use.",
+    },
+    {
+      name: "Azure OpenAI",
+      value: "azure",
+      logo: AzureOpenAiLogo,
+      options: <AzureAiOptions settings={settings} />,
+      description: "The enterprise option of OpenAI hosted on Azure services.",
+    },
+    {
+      name: "Anthropic",
+      value: "anthropic",
+      logo: AnthropicLogo,
+      options: <AnthropicAiOptions settings={settings} />,
+      description: "A friendly AI Assistant hosted by Anthropic.",
+    },
+    {
+      name: "Gemini",
+      value: "gemini",
+      logo: GeminiLogo,
+      options: <GeminiLLMOptions settings={settings} />,
+      description: "Google's largest and most capable AI model",
+    },
+    {
+      name: "Ollama",
+      value: "ollama",
+      logo: OllamaLogo,
+      options: <OllamaLLMOptions settings={settings} />,
+      description: "Run LLMs locally on your own machine.",
+    },
+    {
+      name: "LM Studio",
+      value: "lmstudio",
+      logo: LMStudioLogo,
+      options: <LMStudioOptions settings={settings} />,
+      description:
+        "Discover, download, and run thousands of cutting edge LLMs in a few clicks.",
+    },
+    {
+      name: "Local AI",
+      value: "localai",
+      logo: LocalAiLogo,
+      options: <LocalAiOptions settings={settings} />,
+      description: "Run LLMs locally on your own machine.",
+    },
+    {
+      name: "Native",
+      value: "native",
+      logo: AnythingLLMIcon,
+      options: <NativeLLMOptions settings={settings} />,
+      description:
+        "Use a downloaded custom Llama model for chatting on this AnythingLLM instance.",
+    },
+  ];
+
   return (
     <div className="w-screen h-screen overflow-hidden bg-sidebar flex">
       {!isMobile && <Sidebar />}
@@ -74,11 +155,7 @@ export default function GeneralLLMPreference() {
           className="relative md:ml-[2px] md:mr-[16px] md:my-[16px] md:rounded-[26px] bg-main-gradient w-full h-full overflow-y-scroll border-4 border-accent"
         >
           {isMobile && <SidebarMobileHeader />}
-          <form
+          <form onSubmit={handleSubmit} className="flex w-full">
-            onSubmit={handleSubmit}
-            onChange={() => setHasChanges(true)}
-            className="flex w-full"
-          >
             <div className="flex flex-col w-full px-1 md:px-20 md:py-12 py-16">
               <div className="w-full flex flex-col gap-y-1 pb-6 border-white border-b-2 border-opacity-10">
                 <div className="items-center flex gap-x-4">
@@ -105,83 +182,51 @@ export default function GeneralLLMPreference() {
               <div className="text-white text-sm font-medium py-4">
                 LLM Providers
               </div>
-              <div className="w-full flex md:flex-wrap overflow-x-scroll gap-4 max-w-[900px]">
+              <div className="w-full">
-                <input hidden={true} name="LLMProvider" value={llmChoice} />
+                <div className="w-full relative border-slate-300/20 shadow border-4 rounded-xl text-white">
-                <LLMProviderOption
+                  <div className="w-full p-4 absolute top-0 rounded-t-lg backdrop-blur-sm">
-                  name="OpenAI"
+                    <div className="w-full flex items-center sticky top-0">
-                  value="openai"
+                      <MagnifyingGlass
-                  link="openai.com"
+                        size={16}
-                  description="The standard option for most non-commercial use. Provides both chat and embedding."
+                        weight="bold"
-                  checked={llmChoice === "openai"}
+                        className="absolute left-4 z-30 text-white"
-                  image={OpenAiLogo}
+                      />
-                  onClick={updateLLMChoice}
+                      <input
-                />
+                        type="text"
-                <LLMProviderOption
+                        placeholder="Search LLM providers"
-                  name="Azure OpenAI"
+                        className="bg-zinc-600 z-20 pl-10 rounded-full w-full px-4 py-1 text-sm border-2 border-slate-300/40 outline-none focus:border-white text-white"
-                  value="azure"
+                        onChange={(e) => setSearchQuery(e.target.value)}
-                  link="azure.microsoft.com"
+                        autoComplete="off"
-                  description="The enterprise option of OpenAI hosted on Azure services. Provides both chat and embedding."
+                        onKeyDown={(e) => {
-                  checked={llmChoice === "azure"}
+                          if (e.key === "Enter") e.preventDefault();
-                  image={AzureOpenAiLogo}
+                        }}
-                  onClick={updateLLMChoice}
+                      />
-                />
+                    </div>
-                <LLMProviderOption
+                  </div>
-                  name="Anthropic Claude 2"
+                  <div className="px-4 pt-[70px] flex flex-col gap-y-1 max-h-[390px] overflow-y-auto no-scroll pb-4">
-                  value="anthropic"
+                    {filteredLLMs.map((llm) => {
-                  link="anthropic.com"
+                      if (llm.value === "native" && isHosted) return null;
-                  description="A friendly AI Assistant hosted by Anthropic. Provides chat services only!"
+                      return (
-                  checked={llmChoice === "anthropic"}
+                        <LLMItem
-                  image={AnthropicLogo}
+                          key={llm.name}
-                  onClick={updateLLMChoice}
+                          name={llm.name}
-                />
+                          value={llm.value}
-                <LLMProviderOption
+                          image={llm.logo}
-                  name="LM Studio"
+                          description={llm.description}
-                  value="lmstudio"
+                          checked={selectedLLM === llm.value}
-                  link="lmstudio.ai"
+                          onClick={() => updateLLMChoice(llm.value)}
-                  description="Discover, download, and run thousands of cutting edge LLMs in a few clicks."
+                        />
-                  checked={llmChoice === "lmstudio"}
+                      );
-                  image={LMStudioLogo}
+                    })}
-                  onClick={updateLLMChoice}
+                  </div>
-                />
+                </div>
-                <LLMProviderOption
+                <div
-                  name="Local AI"
+                  onChange={() => setHasChanges(true)}
-                  value="localai"
+                  className="mt-4 flex flex-col gap-y-1"
-                  link="localai.io"
+                >
-                  description="Run LLMs locally on your own machine."
+                  {selectedLLM &&
-                  checked={llmChoice === "localai"}
+                    LLMS.find((llm) => llm.value === selectedLLM)?.options}
-                  image={LocalAiLogo}
+                </div>
-                  onClick={updateLLMChoice}
-                />
-                {!window.location.hostname.includes("useanything.com") && (
-                  <LLMProviderOption
-                    name="Custom Llama Model"
-                    value="native"
-                    description="Use a downloaded custom Llama model for chatting on this AnythingLLM instance."
-                    checked={llmChoice === "native"}
-                    image={AnythingLLMIcon}
-                    onClick={updateLLMChoice}
-                  />
-                )}
-              </div>
-              <div className="mt-10 flex flex-wrap gap-4 max-w-[800px]">
-                {llmChoice === "openai" && (
-                  <OpenAiOptions settings={settings} />
-                )}
-                {llmChoice === "azure" && (
-                  <AzureAiOptions settings={settings} />
-                )}
-                {llmChoice === "anthropic" && (
-                  <AnthropicAiOptions settings={settings} showAlert={true} />
-                )}
-                {llmChoice === "lmstudio" && (
-                  <LMStudioOptions settings={settings} showAlert={true} />
-                )}
-                {llmChoice === "localai" && (
-                  <LocalAiOptions settings={settings} showAlert={true} />
-                )}
-                {llmChoice === "native" && (
-                  <NativeLLMOptions settings={settings} />
-                )}
               </div>
             </div>
           </form>

frontend/src/pages/GeneralSettings/VectorDatabase/index.jsx

@@ -9,36 +9,86 @@ import LanceDbLogo from "@/media/vectordbs/lancedb.png";
 import WeaviateLogo from "@/media/vectordbs/weaviate.png";
 import QDrantLogo from "@/media/vectordbs/qdrant.png";
 import PreLoader from "@/components/Preloader";
-import VectorDBOption from "@/components/VectorDBOption";
 import ChangeWarningModal from "@/components/ChangeWarning";
+import { MagnifyingGlass } from "@phosphor-icons/react";
+import LanceDBOptions from "@/components/VectorDBSelection/LanceDBOptions";
+import ChromaDBOptions from "@/components/VectorDBSelection/ChromaDBOptions";
+import PineconeDBOptions from "@/components/VectorDBSelection/PineconeDBOptions";
+import QDrantDBOptions from "@/components/VectorDBSelection/QDrantDBOptions";
+import WeaviateDBOptions from "@/components/VectorDBSelection/WeaviateDBOptions";
+import VectorDBItem from "@/components/VectorDBSelection/VectorDBItem";
 
 export default function GeneralVectorDatabase() {
   const [saving, setSaving] = useState(false);
   const [hasChanges, setHasChanges] = useState(false);
   const [hasEmbeddings, setHasEmbeddings] = useState(false);
-  const [vectorDB, setVectorDB] = useState("lancedb");
   const [settings, setSettings] = useState({});
   const [loading, setLoading] = useState(true);
+  const [searchQuery, setSearchQuery] = useState("");
+  const [filteredVDBs, setFilteredVDBs] = useState([]);
+  const [selectedVDB, setSelectedVDB] = useState(null);
 
   useEffect(() => {
     async function fetchKeys() {
       const _settings = await System.keys();
+      console.log(_settings);
       setSettings(_settings);
-      setVectorDB(_settings?.VectorDB || "lancedb");
+      setSelectedVDB(_settings?.VectorDB || "lancedb");
       setHasEmbeddings(_settings?.HasExistingEmbeddings || false);
       setLoading(false);
     }
     fetchKeys();
   }, []);
 
+  const VECTOR_DBS = [
+    {
+      name: "LanceDB",
+      value: "lancedb",
+      logo: LanceDbLogo,
+      options: <LanceDBOptions />,
+      description:
+        "100% local vector DB that runs on the same instance as AnythingLLM.",
+    },
+    {
+      name: "Chroma",
+      value: "chroma",
+      logo: ChromaLogo,
+      options: <ChromaDBOptions settings={settings} />,
+      description:
+        "Open source vector database you can host yourself or on the cloud.",
+    },
+    {
+      name: "Pinecone",
+      value: "pinecone",
+      logo: PineconeLogo,
+      options: <PineconeDBOptions settings={settings} />,
+      description: "100% cloud-based vector database for enterprise use cases.",
+    },
+    {
+      name: "QDrant",
+      value: "qdrant",
+      logo: QDrantLogo,
+      options: <QDrantDBOptions settings={settings} />,
+      description: "Open source local and distributed cloud vector database.",
+    },
+    {
+      name: "Weaviate",
+      value: "weaviate",
+      logo: WeaviateLogo,
+      options: <WeaviateDBOptions settings={settings} />,
+      description:
+        "Open source local and cloud hosted multi-modal vector database.",
+    },
+  ];
+
   const updateVectorChoice = (selection) => {
     setHasChanges(true);
-    setVectorDB(selection);
+    setSelectedVDB(selection);
   };
 
   const handleSubmit = async (e) => {
     e.preventDefault();
-    if (vectorDB !== settings?.VectorDB && hasChanges && hasEmbeddings) {
+    if (selectedVDB !== settings?.VectorDB && hasChanges && hasEmbeddings) {
       document.getElementById("confirmation-modal")?.showModal();
     } else {
       await handleSaveSettings();
@@ -47,24 +97,31 @@ export default function GeneralVectorDatabase() {
 
   const handleSaveSettings = async () => {
     setSaving(true);
-    const data = new FormData(document.getElementById("vectordb-form"));
+    const form = document.getElementById("vectordb-form");
     const settingsData = {};
-    for (let [key, value] of data.entries()) {
+    const formData = new FormData(form);
-      settingsData[key] = value;
+    settingsData.VectorDB = selectedVDB;
-    }
+    for (var [key, value] of formData.entries()) settingsData[key] = value;
 
     const { error } = await System.updateSystem(settingsData);
     if (error) {
-      showToast(`Failed to save LLM settings: ${error}`, "error");
+      showToast(`Failed to save vector database settings: ${error}`, "error");
       setHasChanges(true);
     } else {
-      showToast("LLM preferences saved successfully.", "success");
+      showToast("Vector database preferences saved successfully.", "success");
       setHasChanges(false);
     }
     setSaving(false);
     document.getElementById("confirmation-modal")?.close();
   };
 
+  useEffect(() => {
+    const filtered = VECTOR_DBS.filter((vdb) =>
+      vdb.name.toLowerCase().includes(searchQuery.toLowerCase())
+    );
+    setFilteredVDBs(filtered);
+  }, [searchQuery, selectedVDB]);
+
   return (
     <div className="w-screen h-screen overflow-hidden bg-sidebar flex">
       <ChangeWarningModal
@@ -91,7 +148,6 @@ export default function GeneralVectorDatabase() {
           <form
             id="vectordb-form"
             onSubmit={handleSubmit}
-            onChange={() => setHasChanges(true)}
             className="flex w-full"
           >
             <div className="flex flex-col w-full px-1 md:px-20 md:py-12 py-16">
@@ -119,236 +175,52 @@ export default function GeneralVectorDatabase() {
               <div className="text-white text-sm font-medium py-4">
                 Select your preferred vector database provider
               </div>
-              <div className="w-full flex md:flex-wrap overflow-x-scroll gap-4 max-w-[900px]">
+              <div className="w-full">
-                <input hidden={true} name="VectorDB" value={vectorDB} />
+                <div className="w-full relative border-slate-300/20 shadow border-4 rounded-xl text-white">
-                <VectorDBOption
+                  <div className="w-full p-4 absolute top-0 rounded-t-lg backdrop-blur-sm">
-                  name="Chroma"
+                    <div className="w-full flex items-center sticky top-0 z-20">
-                  value="chroma"
+                      <MagnifyingGlass
-                  link="trychroma.com"
+                        size={16}
-                  description="Open source vector database you can host yourself or on the cloud."
+                        weight="bold"
-                  checked={vectorDB === "chroma"}
+                        className="absolute left-4 z-30 text-white"
-                  image={ChromaLogo}
-                  onClick={updateVectorChoice}
-                />
-                <VectorDBOption
-                  name="Pinecone"
-                  value="pinecone"
-                  link="pinecone.io"
-                  description="100% cloud-based vector database for enterprise use cases."
-                  checked={vectorDB === "pinecone"}
-                  image={PineconeLogo}
-                  onClick={updateVectorChoice}
-                />
-                <VectorDBOption
-                  name="QDrant"
-                  value="qdrant"
-                  link="qdrant.tech"
-                  description="Open source local and distributed cloud vector database."
-                  checked={vectorDB === "qdrant"}
-                  image={QDrantLogo}
-                  onClick={updateVectorChoice}
-                />
-                <VectorDBOption
-                  name="Weaviate"
-                  value="weaviate"
-                  link="weaviate.io"
-                  description="Open source local and cloud hosted multi-modal vector database."
-                  checked={vectorDB === "weaviate"}
-                  image={WeaviateLogo}
-                  onClick={updateVectorChoice}
-                />
-                <VectorDBOption
-                  name="LanceDB"
-                  value="lancedb"
-                  link="lancedb.com"
-                  description="100% local vector DB that runs on the same instance as AnythingLLM."
-                  checked={vectorDB === "lancedb"}
-                  image={LanceDbLogo}
-                  onClick={updateVectorChoice}
-                />
-              </div>
-              <div className="mt-10 flex flex-wrap gap-4 max-w-[800px]">
-                {vectorDB === "pinecone" && (
-                  <>
-                    <div className="flex flex-col w-60">
-                      <label className="text-white text-sm font-semibold block mb-4">
-                        Pinecone DB API Key
-                      </label>
-                      <input
-                        type="password"
-                        name="PineConeKey"
-                        className="bg-zinc-900 text-white placeholder-white placeholder-opacity-60 text-sm rounded-lg focus:border-white block w-full p-2.5"
-                        placeholder="Pinecone API Key"
-                        defaultValue={
-                          settings?.PineConeKey ? "*".repeat(20) : ""
-                        }
-                        required={true}
-                        autoComplete="off"
-                        spellCheck={false}
                       />
-                    </div>
-
-                    <div className="flex flex-col w-60">
-                      <label className="text-white text-sm font-semibold block mb-4">
-                        Pinecone Index Environment
-                      </label>
                       <input
                         type="text"
-                        name="PineConeEnvironment"
+                        placeholder="Search vector databases"
-                        className="bg-zinc-900 text-white placeholder-white placeholder-opacity-60 text-sm rounded-lg focus:border-white block w-full p-2.5"
+                        className="bg-zinc-600 z-20 pl-10 rounded-full w-full px-4 py-1 text-sm border-2 border-slate-300/40 outline-none focus:border-white text-white"
-                        placeholder="us-gcp-west-1"
+                        onChange={(e) => {
-                        defaultValue={settings?.PineConeEnvironment}
+                          e.preventDefault();
-                        required={true}
+                          setSearchQuery(e.target.value);
+                        }}
                         autoComplete="off"
-                        spellCheck={false}
+                        onKeyDown={(e) => {
+                          if (e.key === "Enter") e.preventDefault();
+                        }}
                       />
                     </div>
-
-                    <div className="flex flex-col w-60">
-                      <label className="text-white text-sm font-semibold block mb-4">
-                        Pinecone Index Name
-                      </label>
-                      <input
-                        type="text"
-                        name="PineConeIndex"
-                        className="bg-zinc-900 text-white placeholder-white placeholder-opacity-60 text-sm rounded-lg focus:border-white block w-full p-2.5"
-                        placeholder="my-index"
-                        defaultValue={settings?.PineConeIndex}
-                        required={true}
-                        autoComplete="off"
-                        spellCheck={false}
-                      />
-                    </div>
-                  </>
-                )}
-
-                {vectorDB === "chroma" && (
-                  <>
-                    <div className="flex flex-col w-60">
-                      <label className="text-white text-sm font-semibold block mb-4">
-                        Chroma Endpoint
-                      </label>
-                      <input
-                        type="url"
-                        name="ChromaEndpoint"
-                        className="bg-zinc-900 text-white placeholder-white placeholder-opacity-60 text-sm rounded-lg focus:border-white block w-full p-2.5"
-                        placeholder="http://localhost:8000"
-                        defaultValue={settings?.ChromaEndpoint}
-                        required={true}
-                        autoComplete="off"
-                        spellCheck={false}
-                      />
-                    </div>
-
-                    <div className="flex flex-col w-60">
-                      <label className="text-white text-sm font-semibold block mb-4">
-                        API Header
-                      </label>
-                      <input
-                        name="ChromaApiHeader"
-                        autoComplete="off"
-                        type="text"
-                        defaultValue={settings?.ChromaApiHeader}
-                        className="bg-zinc-900 text-white placeholder-white placeholder-opacity-60 text-sm rounded-lg focus:border-white block w-full p-2.5"
-                        placeholder="X-Api-Key"
-                      />
-                    </div>
-
-                    <div className="flex flex-col w-60">
-                      <label className="text-white text-sm font-semibold block mb-4">
-                        API Key
-                      </label>
-                      <input
-                        name="ChromaApiKey"
-                        autoComplete="off"
-                        type="password"
-                        defaultValue={
-                          settings?.ChromaApiKey ? "*".repeat(20) : ""
-                        }
-                        className="bg-zinc-900 text-white placeholder-white placeholder-opacity-60 text-sm rounded-lg focus:border-white block w-full p-2.5"
-                        placeholder="sk-myApiKeyToAccessMyChromaInstance"
-                      />
-                    </div>
-                  </>
-                )}
-
-                {vectorDB === "lancedb" && (
-                  <div className="w-full h-40 items-center justify-center flex">
-                    <p className="text-sm font-base text-white text-opacity-60">
-                      There is no configuration needed for LanceDB.
-                    </p>
                   </div>
-                )}
+                  <div className="px-4 pt-[70px] flex flex-col gap-y-1 max-h-[390px] overflow-y-auto no-scroll pb-4">
-
+                    {filteredVDBs.map((vdb) => (
-                {vectorDB === "qdrant" && (
+                      <VectorDBItem
-                  <>
+                        key={vdb.name}
-                    <div className="flex flex-col w-60">
+                        name={vdb.name}
-                      <label className="text-white text-sm font-semibold block mb-4">
+                        value={vdb.value}
-                        QDrant API Endpoint
+                        image={vdb.logo}
-                      </label>
+                        description={vdb.description}
-                      <input
+                        checked={selectedVDB === vdb.value}
-                        type="url"
+                        onClick={() => updateVectorChoice(vdb.value)}
-                        name="QdrantEndpoint"
-                        className="bg-zinc-900 text-white placeholder-white placeholder-opacity-60 text-sm rounded-lg focus:border-white block w-full p-2.5"
-                        placeholder="http://localhost:6633"
-                        defaultValue={settings?.QdrantEndpoint}
-                        required={true}
-                        autoComplete="off"
-                        spellCheck={false}
                       />
-                    </div>
+                    ))}
-
+                  </div>
-                    <div className="flex flex-col w-60">
+                </div>
-                      <label className="text-white text-sm font-semibold block mb-4">
+                <div
-                        API Key
+                  onChange={() => setHasChanges(true)}
-                      </label>
+                  className="mt-4 flex flex-col gap-y-1"
-                      <input
+                >
-                        type="password"
+                  {selectedVDB &&
-                        name="QdrantApiKey"
+                    VECTOR_DBS.find((vdb) => vdb.value === selectedVDB)
-                        className="bg-zinc-900 text-white placeholder-white placeholder-opacity-60 text-sm rounded-lg focus:border-white block w-full p-2.5"
+                      ?.options}
-                        placeholder="wOeqxsYP4....1244sba"
+                </div>
-                        defaultValue={settings?.QdrantApiKey}
-                        autoComplete="off"
-                        spellCheck={false}
-                      />
-                    </div>
-                  </>
-                )}
-
-                {vectorDB === "weaviate" && (
-                  <>
-                    <div className="flex flex-col w-60">
-                      <label className="text-white text-sm font-semibold block mb-4">
-                        Weaviate Endpoint
-                      </label>
-                      <input
-                        type="url"
-                        name="WeaviateEndpoint"
-                        className="bg-zinc-900 text-white placeholder-white placeholder-opacity-60 text-sm rounded-lg focus:border-white block w-full p-2.5"
-                        placeholder="http://localhost:8080"
-                        defaultValue={settings?.WeaviateEndpoint}
-                        required={true}
-                        autoComplete="off"
-                        spellCheck={false}
-                      />
-                    </div>
-
-                    <div className="flex flex-col w-60">
-                      <label className="text-white text-sm font-semibold block mb-4">
-                        API Key
-                      </label>
-                      <input
-                        type="password"
-                        name="WeaviateApiKey"
-                        className="bg-zinc-900 text-white placeholder-white placeholder-opacity-60 text-sm rounded-lg focus:border-white block w-full p-2.5"
-                        placeholder="sk-123Abcweaviate"
-                        defaultValue={settings?.WeaviateApiKey}
-                        autoComplete="off"
-                        spellCheck={false}
-                      />
-                    </div>
-                  </>
-                )}
               </div>
             </div>
           </form>

frontend/src/pages/OnboardingFlow/OnboardingModal/Steps/AppearanceSetup/index.jsx

@@ -1,145 +0,0 @@
-import React, { memo, useEffect, useState } from "react";
-import System from "@/models/system";
-import AnythingLLM from "@/media/logo/anything-llm.png";
-import useLogo from "@/hooks/useLogo";
-import { Plus } from "@phosphor-icons/react";
-import showToast from "@/utils/toast";
-
-function AppearanceSetup({ prevStep, nextStep }) {
-  const { logo: _initLogo, setLogo: _setLogo } = useLogo();
-  const [logo, setLogo] = useState("");
-  const [isDefaultLogo, setIsDefaultLogo] = useState(true);
-
-  useEffect(() => {
-    async function logoInit() {
-      setLogo(_initLogo || "");
-      const _isDefaultLogo = await System.isDefaultLogo();
-      setIsDefaultLogo(_isDefaultLogo);
-    }
-    logoInit();
-  }, [_initLogo]);
-
-  const handleFileUpload = async (event) => {
-    const file = event.target.files[0];
-    if (!file) return false;
-
-    const objectURL = URL.createObjectURL(file);
-    setLogo(objectURL);
-
-    const formData = new FormData();
-    formData.append("logo", file);
-    const { success, error } = await System.uploadLogo(formData);
-    if (!success) {
-      showToast(`Failed to upload logo: ${error}`, "error");
-      setLogo(_initLogo);
-      return;
-    }
-
-    const logoURL = await System.fetchLogo();
-    _setLogo(logoURL);
-
-    showToast("Image uploaded successfully.", "success");
-    setIsDefaultLogo(false);
-  };
-
-  const handleRemoveLogo = async () => {
-    setLogo("");
-    setIsDefaultLogo(true);
-
-    const { success, error } = await System.removeCustomLogo();
-    if (!success) {
-      console.error("Failed to remove logo:", error);
-      showToast(`Failed to remove logo: ${error}`, "error");
-      const logoURL = await System.fetchLogo();
-      setLogo(logoURL);
-      setIsDefaultLogo(false);
-      return;
-    }
-
-    const logoURL = await System.fetchLogo();
-    _setLogo(logoURL);
-
-    showToast("Image successfully removed.", "success");
-  };
-
-  return (
-    <div className="w-full">
-      <div className="flex flex-col w-full px-8 py-4">
-        <div className="flex flex-col gap-y-2">
-          <h2 className="text-white text-sm font-medium">Custom Logo</h2>
-          <p className="text-sm font-base text-white/60">
-            Upload your custom logo to make your chatbot yours.
-          </p>
-        </div>
-        <div className="flex md:flex-row flex-col items-center">
-          <img
-            src={logo}
-            alt="Uploaded Logo"
-            className="w-48 h-48 object-contain mr-6"
-            hidden={isDefaultLogo}
-            onError={(e) => (e.target.src = AnythingLLM)}
-          />
-          <div className="flex flex-row gap-x-8">
-            <label className="mt-5 hover:opacity-60" hidden={!isDefaultLogo}>
-              <input
-                id="logo-upload"
-                type="file"
-                accept="image/*"
-                className="hidden"
-                onChange={handleFileUpload}
-              />
-              <div
-                className="w-80 py-4 bg-zinc-900/50 rounded-2xl border-2 border-dashed border-white border-opacity-60 justify-center items-center inline-flex cursor-pointer"
-                htmlFor="logo-upload"
-              >
-                <div className="flex flex-col items-center justify-center">
-                  <div className="rounded-full bg-white/40">
-                    <Plus className="w-6 h-6 text-black/80 m-2" />
-                  </div>
-                  <div className="text-white text-opacity-80 text-sm font-semibold py-1">
-                    Add a custom logo
-                  </div>
-                  <div className="text-white text-opacity-60 text-xs font-medium py-1">
-                    Recommended size: 800 x 200
-                  </div>
-                </div>
-              </div>
-            </label>
-            <button
-              onClick={handleRemoveLogo}
-              className="text-white text-base font-medium hover:text-opacity-60"
-            >
-              Delete
-            </button>
-          </div>
-        </div>
-      </div>
-      <div className="flex w-full justify-between items-center px-6 py-4 space-x-6 border-t rounded-b border-gray-500/50">
-        <button
-          onClick={prevStep}
-          type="button"
-          className="px-4 py-2 rounded-lg text-white hover:bg-sidebar"
-        >
-          Back
-        </button>
-        <div className="flex gap-2">
-          <button
-            onClick={() => nextStep("user_mode_setup")}
-            type="button"
-            className="px-4 py-2 rounded-lg text-white hover:bg-sidebar"
-          >
-            Skip
-          </button>
-          <button
-            onClick={() => nextStep("user_mode_setup")}
-            type="button"
-            className="border border-slate-200 px-4 py-2 rounded-lg text-slate-800 bg-slate-200 text-sm items-center flex gap-x-2 hover:text-white hover:bg-transparent focus:ring-gray-800 font-semibold shadow"
-          >
-            Continue
-          </button>
-        </div>
-      </div>
-    </div>
-  );
-}
-export default memo(AppearanceSetup);

frontend/src/pages/OnboardingFlow/OnboardingModal/Steps/CreateFirstWorkspace/index.jsx

@@ -1,68 +0,0 @@
-import React, { memo } from "react";
-import { useNavigate } from "react-router-dom";
-import paths from "@/utils/paths";
-import Workspace from "@/models/workspace";
-
-function CreateFirstWorkspace({ prevStep }) {
-  const navigate = useNavigate();
-
-  const handleCreate = async (e) => {
-    e.preventDefault();
-    const form = new FormData(e.target);
-    const { workspace, error } = await Workspace.new({
-      name: form.get("name"),
-      onboardingComplete: true,
-    });
-    if (!!workspace) {
-      navigate(paths.home());
-    } else {
-      alert(error);
-    }
-  };
-
-  return (
-    <div>
-      <form onSubmit={handleCreate} className="flex flex-col w-full">
-        <div className="flex flex-col w-full md:px-8 py-12">
-          <div className="space-y-6 flex h-full w-96">
-            <div className="w-full flex flex-col gap-y-4">
-              <div>
-                <label
-                  htmlFor="name"
-                  className="block mb-2 text-sm font-medium text-white"
-                >
-                  Workspace name
-                </label>
-                <input
-                  name="name"
-                  type="text"
-                  className="bg-zinc-900 border border-gray-500 text-white text-sm rounded-lg focus:ring-blue-500 focus:border-blue-500 block w-full p-2.5"
-                  placeholder="My workspace"
-                  minLength={4}
-                  required={true}
-                  autoComplete="off"
-                />
-              </div>
-            </div>
-          </div>
-        </div>
-        <div className="flex w-full justify-end items-center px-6 py-4 space-x-2 border-t rounded-b border-gray-500/50">
-          <button
-            onClick={prevStep}
-            type="button"
-            className="px-4 py-2 rounded-lg text-white hover:bg-sidebar"
-          >
-            Back
-          </button>
-          <button
-            type="submit"
-            className="border border-slate-200 px-4 py-2 rounded-lg text-slate-800 bg-slate-200 text-sm items-center flex gap-x-2 hover:text-white hover:bg-transparent focus:ring-gray-800 font-semibold shadow"
-          >
-            Finish
-          </button>
-        </div>
-      </form>
-    </div>
-  );
-}
-export default memo(CreateFirstWorkspace);

frontend/src/pages/OnboardingFlow/OnboardingModal/Steps/EmbeddingSelection/index.jsx

@@ -1,136 +0,0 @@
-import React, { memo, useEffect, useState } from "react";
-import AnythingLLMIcon from "@/media/logo/anything-llm-icon.png";
-import OpenAiLogo from "@/media/llmprovider/openai.png";
-import AzureOpenAiLogo from "@/media/llmprovider/azure.png";
-import LocalAiLogo from "@/media/llmprovider/localai.png";
-import System from "@/models/system";
-import PreLoader from "@/components/Preloader";
-import LLMProviderOption from "@/components/LLMSelection/LLMProviderOption";
-import OpenAiOptions from "@/components/EmbeddingSelection/OpenAiOptions";
-import AzureAiOptions from "@/components/EmbeddingSelection/AzureAiOptions";
-import LocalAiOptions from "@/components/EmbeddingSelection/LocalAiOptions";
-import NativeEmbeddingOptions from "@/components/EmbeddingSelection/NativeEmbeddingOptions";
-
-function EmbeddingSelection({ nextStep, prevStep, currentStep }) {
-  const [embeddingChoice, setEmbeddingChoice] = useState("native");
-  const [settings, setSettings] = useState(null);
-  const [loading, setLoading] = useState(true);
-  const updateChoice = (selection) => {
-    setEmbeddingChoice(selection);
-  };
-
-  useEffect(() => {
-    async function fetchKeys() {
-      const _settings = await System.keys();
-      setSettings(_settings);
-      setEmbeddingChoice(_settings?.EmbeddingEngine || "native");
-      setLoading(false);
-    }
-    fetchKeys();
-  }, [currentStep]);
-
-  const handleSubmit = async (e) => {
-    e.preventDefault();
-    const form = e.target;
-    const data = {};
-    const formData = new FormData(form);
-    for (var [key, value] of formData.entries()) data[key] = value;
-    const { error } = await System.updateSystem(data);
-    if (error) {
-      alert(`Failed to save LLM settings: ${error}`, "error");
-      return;
-    }
-    nextStep("vector_database");
-    return;
-  };
-
-  if (loading)
-    return (
-      <div className="w-full h-full flex justify-center items-center p-20">
-        <PreLoader />
-      </div>
-    );
-
-  return (
-    <div className="w-full">
-      <form onSubmit={handleSubmit} className="flex flex-col w-full">
-        <div className="flex flex-col w-full px-1 md:px-8 py-4">
-          <div className="text-white text-sm font-medium pb-4">
-            Embedding Provider
-          </div>
-          <div className="w-full flex md:flex-wrap overflow-x-scroll gap-4 max-w-[752px]">
-            <input
-              hidden={true}
-              name="EmbeddingEngine"
-              value={embeddingChoice}
-            />
-            <LLMProviderOption
-              name="AnythingLLM Embedder"
-              value="native"
-              description="Use the built-in embedding engine for AnythingLLM. Zero setup!"
-              checked={embeddingChoice === "native"}
-              image={AnythingLLMIcon}
-              onClick={updateChoice}
-            />
-            <LLMProviderOption
-              name="OpenAI"
-              value="openai"
-              link="openai.com"
-              description="The standard option for most non-commercial use. Provides both chat and embedding."
-              checked={embeddingChoice === "openai"}
-              image={OpenAiLogo}
-              onClick={updateChoice}
-            />
-            <LLMProviderOption
-              name="Azure OpenAI"
-              value="azure"
-              link="azure.microsoft.com"
-              description="The enterprise option of OpenAI hosted on Azure services. Provides both chat and embedding."
-              checked={embeddingChoice === "azure"}
-              image={AzureOpenAiLogo}
-              onClick={updateChoice}
-            />
-            <LLMProviderOption
-              name="LocalAI"
-              value="localai"
-              link="localai.io"
-              description="Self hosted LocalAI embedding engine."
-              checked={embeddingChoice === "localai"}
-              image={LocalAiLogo}
-              onClick={updateChoice}
-            />
-          </div>
-          <div className="mt-4 flex flex-wrap gap-4 max-w-[752px]">
-            {embeddingChoice === "native" && <NativeEmbeddingOptions />}
-            {embeddingChoice === "openai" && (
-              <OpenAiOptions settings={settings} />
-            )}
-            {embeddingChoice === "azure" && (
-              <AzureAiOptions settings={settings} />
-            )}
-            {embeddingChoice === "localai" && (
-              <LocalAiOptions settings={settings} />
-            )}
-          </div>
-        </div>
-        <div className="flex w-full justify-between items-center px-6 py-4 space-x-2 border-t rounded-b border-gray-500/50">
-          <button
-            onClick={prevStep}
-            type="button"
-            className="px-4 py-2 rounded-lg text-white hover:bg-sidebar"
-          >
-            Back
-          </button>
-          <button
-            type="submit"
-            className="border border-slate-200 px-4 py-2 rounded-lg text-slate-800 bg-slate-200 text-sm items-center flex gap-x-2 hover:text-white hover:bg-transparent focus:ring-gray-800 font-semibold shadow"
-          >
-            Continue
-          </button>
-        </div>
-      </form>
-    </div>
-  );
-}
-
-export default memo(EmbeddingSelection);

frontend/src/pages/OnboardingFlow/OnboardingModal/Steps/LLMSelection/index.jsx

@@ -1,157 +0,0 @@
-import React, { memo, useEffect, useState } from "react";
-import AnythingLLMIcon from "@/media/logo/anything-llm-icon.png";
-import OpenAiLogo from "@/media/llmprovider/openai.png";
-import AzureOpenAiLogo from "@/media/llmprovider/azure.png";
-import AnthropicLogo from "@/media/llmprovider/anthropic.png";
-import LMStudioLogo from "@/media/llmprovider/lmstudio.png";
-import LocalAiLogo from "@/media/llmprovider/localai.png";
-import System from "@/models/system";
-import PreLoader from "@/components/Preloader";
-import LLMProviderOption from "@/components/LLMSelection/LLMProviderOption";
-import OpenAiOptions from "@/components/LLMSelection/OpenAiOptions";
-import AzureAiOptions from "@/components/LLMSelection/AzureAiOptions";
-import AnthropicAiOptions from "@/components/LLMSelection/AnthropicAiOptions";
-import LMStudioOptions from "@/components/LLMSelection/LMStudioOptions";
-import LocalAiOptions from "@/components/LLMSelection/LocalAiOptions";
-import NativeLLMOptions from "@/components/LLMSelection/NativeLLMOptions";
-
-function LLMSelection({ nextStep, prevStep, currentStep }) {
-  const [llmChoice, setLLMChoice] = useState("openai");
-  const [settings, setSettings] = useState(null);
-  const [loading, setLoading] = useState(true);
-
-  const updateLLMChoice = (selection) => {
-    setLLMChoice(selection);
-  };
-
-  useEffect(() => {
-    async function fetchKeys() {
-      const _settings = await System.keys();
-      setSettings(_settings);
-      setLLMChoice(_settings?.LLMProvider || "openai");
-      setLoading(false);
-    }
-
-    if (currentStep === "llm_preference") {
-      fetchKeys();
-    }
-  }, []);
-
-  const handleSubmit = async (e) => {
-    e.preventDefault();
-    const form = e.target;
-    const data = {};
-    const formData = new FormData(form);
-    for (var [key, value] of formData.entries()) data[key] = value;
-    const { error } = await System.updateSystem(data);
-    if (error) {
-      alert(`Failed to save LLM settings: ${error}`, "error");
-      return;
-    }
-    nextStep("embedding_preferences");
-  };
-
-  if (loading)
-    return (
-      <div className="w-full h-full flex justify-center items-center p-20">
-        <PreLoader />
-      </div>
-    );
-
-  return (
-    <div>
-      <form onSubmit={handleSubmit} className="flex flex-col w-full">
-        <div className="flex flex-col w-full px-1 md:px-8 py-4">
-          <div className="text-white text-sm font-medium pb-4">
-            LLM Providers
-          </div>
-          <div className="w-full flex md:flex-wrap overflow-x-scroll gap-4 max-w-[752px]">
-            <input hidden={true} name="LLMProvider" value={llmChoice} />
-            <LLMProviderOption
-              name="OpenAI"
-              value="openai"
-              link="openai.com"
-              description="The standard option for most non-commercial use. Provides both chat and embedding."
-              checked={llmChoice === "openai"}
-              image={OpenAiLogo}
-              onClick={updateLLMChoice}
-            />
-            <LLMProviderOption
-              name="Azure OpenAI"
-              value="azure"
-              link="azure.microsoft.com"
-              description="The enterprise option of OpenAI hosted on Azure services. Provides both chat and embedding."
-              checked={llmChoice === "azure"}
-              image={AzureOpenAiLogo}
-              onClick={updateLLMChoice}
-            />
-            <LLMProviderOption
-              name="Anthropic Claude 2"
-              value="anthropic"
-              link="anthropic.com"
-              description="A friendly AI Assistant hosted by Anthropic. Provides chat services only!"
-              checked={llmChoice === "anthropic"}
-              image={AnthropicLogo}
-              onClick={updateLLMChoice}
-            />
-            <LLMProviderOption
-              name="LM Studio"
-              value="lmstudio"
-              link="lmstudio.ai"
-              description="Discover, download, and run thousands of cutting edge LLMs in a few clicks."
-              checked={llmChoice === "lmstudio"}
-              image={LMStudioLogo}
-              onClick={updateLLMChoice}
-            />
-            <LLMProviderOption
-              name="Local AI"
-              value="localai"
-              link="localai.io"
-              description="Run LLMs locally on your own machine."
-              checked={llmChoice === "localai"}
-              image={LocalAiLogo}
-              onClick={updateLLMChoice}
-            />
-            <LLMProviderOption
-              name="Custom Llama Model"
-              value="native"
-              description="Use a downloaded custom Llama model for chatting on this AnythingLLM instance."
-              checked={llmChoice === "native"}
-              image={AnythingLLMIcon}
-              onClick={updateLLMChoice}
-            />
-          </div>
-          <div className="mt-4 flex flex-wrap gap-4 max-w-[752px]">
-            {llmChoice === "openai" && <OpenAiOptions settings={settings} />}
-            {llmChoice === "azure" && <AzureAiOptions settings={settings} />}
-            {llmChoice === "anthropic" && (
-              <AnthropicAiOptions settings={settings} />
-            )}
-            {llmChoice === "lmstudio" && (
-              <LMStudioOptions settings={settings} />
-            )}
-            {llmChoice === "localai" && <LocalAiOptions settings={settings} />}
-            {llmChoice === "native" && <NativeLLMOptions settings={settings} />}
-          </div>
-        </div>
-        <div className="flex w-full justify-between items-center px-6 py-4 space-x-2 border-t rounded-b border-gray-500/50">
-          <button
-            onClick={prevStep}
-            type="button"
-            className="px-4 py-2 rounded-lg text-white hover:bg-sidebar"
-          >
-            Back
-          </button>
-          <button
-            type="submit"
-            className="border border-slate-200 px-4 py-2 rounded-lg text-slate-800 bg-slate-200 text-sm items-center flex gap-x-2 hover:text-white hover:bg-transparent focus:ring-gray-800 font-semibold shadow"
-          >
-            Continue
-          </button>
-        </div>
-      </form>
-    </div>
-  );
-}
-
-export default memo(LLMSelection);

frontend/src/pages/OnboardingFlow/OnboardingModal/Steps/MultiUserSetup/index.jsx

@@ -1,117 +0,0 @@
-import React, { useState, memo } from "react";
-import System from "@/models/system";
-import { AUTH_TIMESTAMP, AUTH_TOKEN, AUTH_USER } from "@/utils/constants";
-import debounce from "lodash.debounce";
-
-// Multi-user mode step
-function MultiUserSetup({ nextStep, prevStep }) {
-  const [username, setUsername] = useState("");
-  const [password, setPassword] = useState("");
-
-  const handleSubmit = async (e) => {
-    e.preventDefault();
-    const form = e.target;
-    const formData = new FormData(form);
-    const data = {
-      username: formData.get("username"),
-      password: formData.get("password"),
-    };
-    const { success, error } = await System.setupMultiUser(data);
-    if (!success) {
-      alert(error);
-      return;
-    }
-
-    // Auto-request token with credentials that was just set so they
-    // are not redirected to login after completion.
-    const { user, token } = await System.requestToken(data);
-    window.localStorage.setItem(AUTH_USER, JSON.stringify(user));
-    window.localStorage.setItem(AUTH_TOKEN, token);
-    window.localStorage.removeItem(AUTH_TIMESTAMP);
-
-    nextStep("data_handling");
-  };
-
-  const setNewUsername = (e) => setUsername(e.target.value);
-  const setNewPassword = (e) => setPassword(e.target.value);
-  const handleUsernameChange = debounce(setNewUsername, 500);
-  const handlePasswordChange = debounce(setNewPassword, 500);
-  return (
-    <div>
-      <form onSubmit={handleSubmit}>
-        <div className="flex flex-col w-full md:px-8 py-4">
-          <div className="space-y-6 flex h-full w-96">
-            <div className="w-full flex flex-col gap-y-4">
-              <div>
-                <label
-                  htmlFor="name"
-                  className="block mb-2 text-sm font-medium text-white"
-                >
-                  Admin account username
-                </label>
-                <input
-                  name="username"
-                  type="text"
-                  className="bg-zinc-900 border border-gray-500 text-white text-sm rounded-lg focus:ring-blue-500 focus:border-blue-500 block w-full p-2.5"
-                  placeholder="Your admin username"
-                  minLength={6}
-                  required={true}
-                  autoComplete="off"
-                  onChange={handleUsernameChange}
-                />
-              </div>
-              <div>
-                <label
-                  htmlFor="name"
-                  className="block mb-2 text-sm font-medium text-white"
-                >
-                  Admin account password
-                </label>
-                <input
-                  name="password"
-                  type="password"
-                  className="bg-zinc-900 border border-gray-500 text-white text-sm rounded-lg focus:ring-blue-500 focus:border-blue-500 block w-full p-2.5"
-                  placeholder="Your admin password"
-                  minLength={8}
-                  required={true}
-                  autoComplete="off"
-                  onChange={handlePasswordChange}
-                />
-              </div>
-              <p className="w-96 text-white text-opacity-80 text-xs font-base">
-                Username must be at least 6 characters long. Password must be at
-                least 8 characters long.
-              </p>
-            </div>
-          </div>
-        </div>
-        <div className="flex w-full justify-between items-center px-6 py-4 space-x-6 border-t rounded-b border-gray-500/50">
-          <div className="w-96 text-white text-opacity-80 text-xs font-base">
-            By default, you will be the only admin. As an admin you will need to
-            create accounts for all new users or admins. Do not lose your
-            password as only admins can reset passwords.
-          </div>
-          <div className="flex gap-2">
-            <button
-              onClick={prevStep}
-              type="button"
-              className="px-4 py-2 rounded-lg text-white hover:bg-sidebar"
-            >
-              Back
-            </button>
-            <button
-              type="submit"
-              className="border px-4 py-2 rounded-lg text-sm items-center flex gap-x-2
-              border-slate-200 text-slate-800 bg-slate-200 hover:text-white hover:bg-transparent focus:ring-gray-800 font-semibold shadow
-              disabled:border-gray-400 disabled:text-slate-800 disabled:bg-gray-400 disabled:cursor-not-allowed"
-              disabled={!(!!username && !!password)}
-            >
-              Continue
-            </button>
-          </div>
-        </div>
-      </form>
-    </div>
-  );
-}
-export default memo(MultiUserSetup);

frontend/src/pages/OnboardingFlow/OnboardingModal/Steps/PasswordProtection/index.jsx

@@ -1,103 +0,0 @@
-import React, { memo, useState } from "react";
-import System from "@/models/system";
-import { AUTH_TIMESTAMP, AUTH_TOKEN, AUTH_USER } from "@/utils/constants";
-import debounce from "lodash.debounce";
-
-function PasswordProtection({ nextStep, prevStep }) {
-  const [password, setPassword] = useState("");
-  const handleSubmit = async (e) => {
-    e.preventDefault();
-    const form = e.target;
-    const formData = new FormData(form);
-    const { error } = await System.updateSystemPassword({
-      usePassword: true,
-      newPassword: formData.get("password"),
-    });
-
-    if (error) {
-      alert(`Failed to set password: ${error}`, "error");
-      return;
-    }
-
-    // Auto-request token with password that was just set so they
-    // are not redirected to login after completion.
-    const { token } = await System.requestToken({
-      password: formData.get("password"),
-    });
-    window.localStorage.removeItem(AUTH_USER);
-    window.localStorage.removeItem(AUTH_TIMESTAMP);
-    window.localStorage.setItem(AUTH_TOKEN, token);
-
-    nextStep("data_handling");
-    return;
-  };
-
-  const handleSkip = () => {
-    nextStep("data_handling");
-  };
-
-  const setNewPassword = (e) => setPassword(e.target.value);
-  const handlePasswordChange = debounce(setNewPassword, 500);
-  return (
-    <div className="w-full">
-      <form className="flex flex-col w-full" onSubmit={handleSubmit}>
-        <div className="flex flex-col w-full px-1 md:px-8 py-4">
-          <div className="w-full flex flex-col gap-y-2 my-5">
-            <div className="w-80">
-              <div className="flex flex-col mb-3 ">
-                <label
-                  htmlFor="password"
-                  className="block font-medium text-white"
-                >
-                  New Password
-                </label>
-                <p className="text-slate-300 text-xs">
-                  must be at least 8 characters.
-                </p>
-              </div>
-              <input
-                onChange={handlePasswordChange}
-                name="password"
-                type="text"
-                className="bg-zinc-900 text-white text-sm rounded-lg focus:border-blue-500 block w-full p-2.5 placeholder-white placeholder-opacity-60 focus:ring-blue-500"
-                placeholder="Your Instance Password"
-                minLength={8}
-                required={true}
-                autoComplete="off"
-              />
-            </div>
-          </div>
-        </div>
-        <div className="flex w-full justify-between items-center px-6 py-4 space-x-2 border-t rounded-b border-gray-500/50">
-          <button
-            onClick={prevStep}
-            type="button"
-            className="px-4 py-2 rounded-lg text-white hover:bg-sidebar"
-          >
-            Back
-          </button>
-
-          <div className="flex gap-2">
-            <button
-              onClick={handleSkip}
-              type="button"
-              className="px-4 py-2 rounded-lg text-white hover:bg-sidebar"
-            >
-              Skip
-            </button>
-            <button
-              type="submit"
-              disabled={!password}
-              className="border px-4 py-2 rounded-lg text-sm items-center flex gap-x-2
-              border-slate-200 text-slate-800 bg-slate-200 hover:text-white hover:bg-transparent focus:ring-gray-800 font-semibold shadow
-              disabled:border-gray-400 disabled:text-slate-800 disabled:bg-gray-400 disabled:cursor-not-allowed"
-            >
-              Continue
-            </button>
-          </div>
-        </div>
-      </form>
-    </div>
-  );
-}
-export default memo(PasswordProtection);

frontend/src/pages/OnboardingFlow/OnboardingModal/Steps/UserModeSelection/index.jsx

@@ -1,47 +0,0 @@
-import React, { memo } from "react";
-
-// How many people will be using your instance step
-function UserModeSelection({ nextStep, prevStep }) {
-  const justMeClicked = () => {
-    nextStep("password_protection");
-  };
-
-  const myTeamClicked = () => {
-    nextStep("multi_user_mode");
-  };
-
-  return (
-    <div>
-      <div className="flex flex-col justify-center items-center px-20 py-14">
-        <div className="w-80 text-white text-center text-2xl font-base">
-          How many people will be using your instance?
-        </div>
-        <div className="flex gap-4 justify-center my-8">
-          <button
-            onClick={justMeClicked}
-            className="transition-all duration-200 border border-slate-200 px-4 py-2 rounded-lg text-slate-800 bg-slate-200 text-sm items-center flex gap-x-2 hover:text-white hover:bg-transparent focus:ring-gray-800 font-semibold shadow"
-          >
-            Just Me
-          </button>
-          <button
-            onClick={myTeamClicked}
-            className="transition-all duration-200 border border-slate-200 px-5 py-2.5 rounded-lg text-slate-800 bg-slate-200 text-sm items-center flex gap-x-2 hover:text-white hover:bg-transparent focus:ring-gray-800 font-semibold shadow"
-          >
-            My Team
-          </button>
-        </div>
-      </div>
-      <div className="flex w-full justify-between items-center p-6 space-x-2 border-t rounded-b border-gray-500/50">
-        <button
-          onClick={prevStep}
-          type="button"
-          className="px-4 py-2 rounded-lg text-white hover:bg-sidebar transition-all duration-300"
-        >
-          Back
-        </button>
-      </div>
-    </div>
-  );
-}
-
-export default memo(UserModeSelection);

frontend/src/pages/OnboardingFlow/OnboardingModal/Steps/UserQuestionnaire/index.jsx

@@ -1,240 +0,0 @@
-import { COMPLETE_QUESTIONNAIRE } from "@/utils/constants";
-import paths from "@/utils/paths";
-import { CheckCircle, Circle } from "@phosphor-icons/react";
-import React, { memo } from "react";
-
-async function sendQuestionnaire({ email, useCase, comment }) {
-  if (import.meta.env.DEV) return;
-  return fetch(`https://onboarding-wxich7363q-uc.a.run.app`, {
-    method: "POST",
-    body: JSON.stringify({
-      email,
-      useCase,
-      comment,
-      sourceId: "0VRjqHh6Vukqi0x0Vd0n/m8JuT7k8nOz",
-    }),
-  })
-    .then(() => {
-      window.localStorage.setItem(COMPLETE_QUESTIONNAIRE, true);
-      console.log(`✅ Questionnaire responses sent.`);
-    })
-    .catch((error) => {
-      console.error(`sendQuestionnaire`, error.message);
-    });
-}
-
-function UserQuestionnaire({ nextStep, prevStep }) {
-  const handleSubmit = async (e) => {
-    e.preventDefault();
-    const form = e.target;
-    const formData = new FormData(form);
-    nextStep("create_workspace");
-
-    await sendQuestionnaire({
-      email: formData.get("email"),
-      useCase: formData.get("use_case") || "other",
-      comment: formData.get("comment") || null,
-    });
-    return;
-  };
-
-  const handleSkip = () => {
-    nextStep("create_workspace");
-  };
-
-  if (!!window?.localStorage?.getItem(COMPLETE_QUESTIONNAIRE)) {
-    return (
-      <div className="w-full">
-        <div className="w-full flex items-center justify-center px-1 md:px-8 py-4">
-          <div className="w-auto flex flex-col gap-y-1 items-center">
-            <CheckCircle size={60} className="text-green-500" />
-            <p className="text-zinc-300">Thank you for your feedback!</p>
-            <a
-              href={paths.mailToMintplex()}
-              className="text-blue-400 underline text-xs"
-            >
-              team@mintplexlabs.com
-            </a>
-          </div>
-        </div>
-
-        <div className="flex w-full justify-between items-center px-6 py-4 space-x-2 border-t rounded-b border-gray-500/50">
-          <button
-            onClick={prevStep}
-            type="button"
-            className="px-4 py-2 rounded-lg text-white hover:bg-sidebar"
-          >
-            Back
-          </button>
-
-          <div className="flex gap-2">
-            <button
-              onClick={handleSkip}
-              type="button"
-              className="px-4 py-2 rounded-lg text-white hover:bg-sidebar"
-            >
-              Skip
-            </button>
-            <button
-              type="submit"
-              className="border px-4 py-2 rounded-lg text-sm items-center flex gap-x-2
-              border-slate-200 text-slate-800 bg-slate-200 hover:text-white hover:bg-transparent focus:ring-gray-800 font-semibold shadow
-              disabled:border-gray-400 disabled:text-slate-800 disabled:bg-gray-400 disabled:cursor-not-allowed"
-            >
-              Continue
-            </button>
-          </div>
-        </div>
-      </div>
-    );
-  }
-
-  return (
-    <div className="w-full">
-      <form className="flex flex-col w-full" onSubmit={handleSubmit}>
-        <div className="flex flex-col w-full px-1 md:px-8 py-4">
-          <div className="w-full flex flex-col gap-y-2 my-5">
-            <div className="w-80">
-              <div className="flex flex-col mb-3 ">
-                <label htmlFor="email" className="block font-medium text-white">
-                  What is your email?
-                </label>
-              </div>
-              <input
-                name="email"
-                type="email"
-                className="bg-zinc-900 text-white text-sm rounded-lg focus:border-blue-500 block w-full p-2.5 placeholder-white placeholder-opacity-60 focus:ring-blue-500"
-                placeholder="you@gmail.com"
-                required={true}
-                autoComplete="off"
-              />
-            </div>
-          </div>
-
-          <div className="w-full flex flex-col gap-y-2 my-5">
-            <div className="w-full">
-              <div className="flex flex-col mb-3 ">
-                <label
-                  htmlFor="use_case"
-                  className="block font-medium text-white"
-                >
-                  How are you planning to use AnythingLLM?
-                </label>
-              </div>
-
-              <div className="flex flex-col gap-y-2">
-                <div class="flex items-center ps-4 border border-zinc-400 rounded group radio-container hover:bg-blue-400/10">
-                  <input
-                    id="bordered-radio-1"
-                    type="radio"
-                    value="business"
-                    name="use_case"
-                    class="sr-only peer"
-                  />
-                  <Circle
-                    weight="fill"
-                    className="fill-transparent border border-gray-300 rounded-full peer-checked:fill-blue-500 peer-checked:border-none"
-                  />
-                  <label
-                    for="bordered-radio-1"
-                    class="w-full py-4 ms-2 text-sm font-medium text-gray-300"
-                  >
-                    For my business
-                  </label>
-                </div>
-                <div class="flex items-center ps-4 border border-zinc-400 rounded group radio-container hover:bg-blue-400/10">
-                  <input
-                    id="bordered-radio-2"
-                    type="radio"
-                    value="personal"
-                    name="use_case"
-                    class="sr-only peer"
-                  />
-                  <Circle
-                    weight="fill"
-                    className="fill-transparent border border-gray-300 rounded-full peer-checked:fill-blue-500 peer-checked:border-none"
-                  />
-                  <label
-                    for="bordered-radio-2"
-                    class="w-full py-4 ms-2 text-sm font-medium text-gray-300"
-                  >
-                    For personal use
-                  </label>
-                </div>
-                <div class="flex items-center ps-4 border border-zinc-400 rounded group radio-container hover:bg-blue-400/10">
-                  <input
-                    id="bordered-radio-3"
-                    type="radio"
-                    value="other"
-                    name="use_case"
-                    class="sr-only peer"
-                  />
-                  <Circle
-                    weight="fill"
-                    className="fill-transparent border border-gray-300 rounded-full peer-checked:fill-blue-500 peer-checked:border-none"
-                  />
-                  <label
-                    for="bordered-radio-3"
-                    class="w-full py-4 ms-2 text-sm font-medium text-gray-300"
-                  >
-                    I'm not sure yet
-                  </label>
-                </div>
-              </div>
-            </div>
-          </div>
-
-          <div className="w-full flex flex-col gap-y-2 my-5">
-            <div className="w-full">
-              <div className="flex flex-col mb-3 ">
-                <label
-                  htmlFor="comments"
-                  className="block font-medium text-white"
-                >
-                  Any comments for the team?
-                </label>
-              </div>
-              <textarea
-                name="comment"
-                rows={5}
-                className="bg-zinc-900 text-white text-sm rounded-lg focus:ring-blue-500 focus:border-blue-500 block w-full p-2.5"
-                placeholder="If you have any questions or comments right now, you can leave them here and we will get back to you. You can also email team@mintplexlabs.com"
-                wrap="soft"
-                autoComplete="off"
-              />
-            </div>
-          </div>
-        </div>
-
-        <div className="flex w-full justify-between items-center px-6 py-4 space-x-2 border-t rounded-b border-gray-500/50">
-          <button
-            onClick={prevStep}
-            type="button"
-            className="px-4 py-2 rounded-lg text-white hover:bg-sidebar"
-          >
-            Back
-          </button>
-
-          <div className="flex gap-2">
-            <button
-              onClick={handleSkip}
-              type="button"
-              className="px-4 py-2 rounded-lg text-white hover:bg-sidebar"
-            >
-              Skip
-            </button>
-            <button
-              type="submit"
-              className="border px-4 py-2 rounded-lg text-sm items-center flex gap-x-2
-              border-slate-200 text-slate-800 bg-slate-200 hover:text-white hover:bg-transparent focus:ring-gray-800 font-semibold shadow
-              disabled:border-gray-400 disabled:text-slate-800 disabled:bg-gray-400 disabled:cursor-not-allowed"
-            >
-              Continue
-            </button>
-          </div>
-        </div>
-      </form>
-    </div>
-  );
-}
-export default memo(UserQuestionnaire);

frontend/src/pages/OnboardingFlow/OnboardingModal/Steps/VectorDatabaseConnection/index.jsx

@@ -1,310 +0,0 @@
-import React, { memo, useEffect, useState } from "react";
-
-import VectorDBOption from "@/components/VectorDBOption";
-import ChromaLogo from "@/media/vectordbs/chroma.png";
-import PineconeLogo from "@/media/vectordbs/pinecone.png";
-import LanceDbLogo from "@/media/vectordbs/lancedb.png";
-import WeaviateLogo from "@/media/vectordbs/weaviate.png";
-import QDrantLogo from "@/media/vectordbs/qdrant.png";
-import System from "@/models/system";
-import PreLoader from "@/components/Preloader";
-
-function VectorDatabaseConnection({ nextStep, prevStep, currentStep }) {
-  const [vectorDB, setVectorDB] = useState("lancedb");
-  const [settings, setSettings] = useState({});
-  const [loading, setLoading] = useState(true);
-
-  useEffect(() => {
-    async function fetchKeys() {
-      const _settings = await System.keys();
-      setSettings(_settings);
-      setVectorDB(_settings?.VectorDB || "lancedb");
-      setLoading(false);
-    }
-    if (currentStep === "vector_database") {
-      fetchKeys();
-    }
-  }, [currentStep]);
-
-  const updateVectorChoice = (selection) => {
-    setVectorDB(selection);
-  };
-
-  const handleSubmit = async (e, formElement) => {
-    e.preventDefault();
-    const form = formElement || e.target;
-    const data = {};
-    const formData = new FormData(form);
-    for (var [key, value] of formData.entries()) data[key] = value;
-    const { error } = await System.updateSystem(data);
-    if (error) {
-      alert(`Failed to save settings: ${error}`, "error");
-      return;
-    }
-    nextStep("appearance");
-    return;
-  };
-
-  if (loading)
-    return (
-      <div className="w-full h-full flex justify-center items-center p-20">
-        <PreLoader />
-      </div>
-    );
-
-  return (
-    <div>
-      <form onSubmit={handleSubmit} className="flex flex-col w-full">
-        <div className="flex flex-col w-full px-1 md:px-8 py-4">
-          <div className="text-white text-sm font-medium pb-4">
-            Select your preferred vector database provider
-          </div>
-          <div className="w-full flex md:flex-wrap overflow-x-scroll gap-4 max-w-[752px]">
-            <input hidden={true} name="VectorDB" value={vectorDB} />
-            <VectorDBOption
-              name="Chroma"
-              value="chroma"
-              link="trychroma.com"
-              description="Open source vector database you can host yourself or on the cloud."
-              checked={vectorDB === "chroma"}
-              image={ChromaLogo}
-              onClick={updateVectorChoice}
-            />
-            <VectorDBOption
-              name="Pinecone"
-              value="pinecone"
-              link="pinecone.io"
-              description="100% cloud-based vector database for enterprise use cases."
-              checked={vectorDB === "pinecone"}
-              image={PineconeLogo}
-              onClick={updateVectorChoice}
-            />
-            <VectorDBOption
-              name="QDrant"
-              value="qdrant"
-              link="qdrant.tech"
-              description="Open source local and distributed cloud vector database."
-              checked={vectorDB === "qdrant"}
-              image={QDrantLogo}
-              onClick={updateVectorChoice}
-            />
-            <VectorDBOption
-              name="Weaviate"
-              value="weaviate"
-              link="weaviate.io"
-              description="Open source local and cloud hosted multi-modal vector database."
-              checked={vectorDB === "weaviate"}
-              image={WeaviateLogo}
-              onClick={updateVectorChoice}
-            />
-            <VectorDBOption
-              name="LanceDB"
-              value="lancedb"
-              link="lancedb.com"
-              description="100% local vector DB that runs on the same instance as AnythingLLM."
-              checked={vectorDB === "lancedb"}
-              image={LanceDbLogo}
-              onClick={updateVectorChoice}
-            />
-          </div>
-          <div className="mt-4 flex flex-wrap gap-4 max-w-[752px]">
-            {vectorDB === "pinecone" && (
-              <>
-                <div className="flex flex-col w-60">
-                  <label className="text-white text-sm font-semibold block mb-4">
-                    Pinecone DB API Key
-                  </label>
-                  <input
-                    type="password"
-                    name="PineConeKey"
-                    className="bg-zinc-900 text-white placeholder-white placeholder-opacity-60 text-sm rounded-lg focus:border-white block w-full p-2.5"
-                    placeholder="Pinecone API Key"
-                    defaultValue={settings?.PineConeKey ? "*".repeat(20) : ""}
-                    required={true}
-                    autoComplete="off"
-                    spellCheck={false}
-                  />
-                </div>
-
-                <div className="flex flex-col w-60">
-                  <label className="text-white text-sm font-semibold block mb-4">
-                    Pinecone Index Environment
-                  </label>
-                  <input
-                    type="text"
-                    name="PineConeEnvironment"
-                    className="bg-zinc-900 text-white placeholder-white placeholder-opacity-60 text-sm rounded-lg focus:border-white block w-full p-2.5"
-                    placeholder="us-gcp-west-1"
-                    defaultValue={settings?.PineConeEnvironment}
-                    required={true}
-                    autoComplete="off"
-                    spellCheck={false}
-                  />
-                </div>
-
-                <div className="flex flex-col w-60">
-                  <label className="text-white text-sm font-semibold block mb-4">
-                    Pinecone Index Name
-                  </label>
-                  <input
-                    type="text"
-                    name="PineConeIndex"
-                    className="bg-zinc-900 text-white placeholder-white placeholder-opacity-60 text-sm rounded-lg focus:border-white block w-full p-2.5"
-                    placeholder="my-index"
-                    defaultValue={settings?.PineConeIndex}
-                    required={true}
-                    autoComplete="off"
-                    spellCheck={false}
-                  />
-                </div>
-              </>
-            )}
-
-            {vectorDB === "chroma" && (
-              <>
-                <div className="flex flex-col w-60">
-                  <label className="text-white text-sm font-semibold block mb-4">
-                    Chroma Endpoint
-                  </label>
-                  <input
-                    type="url"
-                    name="ChromaEndpoint"
-                    className="bg-zinc-900 text-white placeholder-white placeholder-opacity-60 text-sm rounded-lg focus:border-white block w-full p-2.5"
-                    placeholder="http://localhost:8000"
-                    defaultValue={settings?.ChromaEndpoint}
-                    required={true}
-                    autoComplete="off"
-                    spellCheck={false}
-                  />
-                </div>
-
-                <div className="flex flex-col w-60">
-                  <label className="text-white text-sm font-semibold block mb-4">
-                    API Header
-                  </label>
-                  <input
-                    name="ChromaApiHeader"
-                    autoComplete="off"
-                    type="text"
-                    defaultValue={settings?.ChromaApiHeader}
-                    className="bg-zinc-900 text-white placeholder-white placeholder-opacity-60 text-sm rounded-lg focus:border-white block w-full p-2.5"
-                    placeholder="X-Api-Key"
-                  />
-                </div>
-
-                <div className="flex flex-col w-60">
-                  <label className="text-white text-sm font-semibold block mb-4">
-                    API Key
-                  </label>
-                  <input
-                    name="ChromaApiKey"
-                    autoComplete="off"
-                    type="password"
-                    defaultValue={settings?.ChromaApiKey ? "*".repeat(20) : ""}
-                    className="bg-zinc-900 text-white placeholder-white placeholder-opacity-60 text-sm rounded-lg focus:border-white block w-full p-2.5"
-                    placeholder="sk-myApiKeyToAccessMyChromaInstance"
-                  />
-                </div>
-              </>
-            )}
-
-            {vectorDB === "lancedb" && (
-              <div className="w-full h-10 items-center justify-center flex">
-                <p className="text-sm font-base text-white text-opacity-60">
-                  There is no configuration needed for LanceDB.
-                </p>
-              </div>
-            )}
-
-            {vectorDB === "qdrant" && (
-              <>
-                <div className="flex flex-col w-60">
-                  <label className="text-white text-sm font-semibold block mb-4">
-                    QDrant API Endpoint
-                  </label>
-                  <input
-                    type="url"
-                    name="QdrantEndpoint"
-                    className="bg-zinc-900 text-white placeholder-white placeholder-opacity-60 text-sm rounded-lg focus:border-white block w-full p-2.5"
-                    placeholder="http://localhost:6633"
-                    defaultValue={settings?.QdrantEndpoint}
-                    required={true}
-                    autoComplete="off"
-                    spellCheck={false}
-                  />
-                </div>
-
-                <div className="flex flex-col w-60">
-                  <label className="text-white text-sm font-semibold block mb-4">
-                    API Key
-                  </label>
-                  <input
-                    type="password"
-                    name="QdrantApiKey"
-                    className="bg-zinc-900 text-white placeholder-white placeholder-opacity-60 text-sm rounded-lg focus:border-white block w-full p-2.5"
-                    placeholder="wOeqxsYP4....1244sba"
-                    defaultValue={settings?.QdrantApiKey}
-                    autoComplete="off"
-                    spellCheck={false}
-                  />
-                </div>
-              </>
-            )}
-
-            {vectorDB === "weaviate" && (
-              <>
-                <div className="flex flex-col w-60">
-                  <label className="text-white text-sm font-semibold block mb-4">
-                    Weaviate Endpoint
-                  </label>
-                  <input
-                    type="url"
-                    name="WeaviateEndpoint"
-                    className="bg-zinc-900 text-white placeholder-white placeholder-opacity-60 text-sm rounded-lg focus:border-white block w-full p-2.5"
-                    placeholder="http://localhost:8080"
-                    defaultValue={settings?.WeaviateEndpoint}
-                    required={true}
-                    autoComplete="off"
-                    spellCheck={false}
-                  />
-                </div>
-
-                <div className="flex flex-col w-60">
-                  <label className="text-white text-sm font-semibold block mb-4">
-                    API Key
-                  </label>
-                  <input
-                    type="password"
-                    name="WeaviateApiKey"
-                    className="bg-zinc-900 text-white placeholder-white placeholder-opacity-60 text-sm rounded-lg focus:border-white block w-full p-2.5"
-                    placeholder="sk-123Abcweaviate"
-                    defaultValue={settings?.WeaviateApiKey}
-                    autoComplete="off"
-                    spellCheck={false}
-                  />
-                </div>
-              </>
-            )}
-          </div>
-        </div>
-        <div className="flex w-full justify-between items-center px-6 py-4 space-x-2 border-t rounded-b border-gray-500/50">
-          <button
-            onClick={prevStep}
-            type="button"
-            className="px-4 py-2 rounded-lg text-white hover:bg-sidebar"
-          >
-            Back
-          </button>
-          <button
-            type="submit"
-            className="border border-slate-200 px-4 py-2 rounded-lg text-slate-800 bg-slate-200 text-sm items-center flex gap-x-2 hover:text-white hover:bg-transparent focus:ring-gray-800 font-semibold shadow"
-          >
-            Continue
-          </button>
-        </div>
-      </form>
-    </div>
-  );
-}
-
-export default memo(VectorDatabaseConnection);

frontend/src/pages/OnboardingFlow/OnboardingModal/index.jsx

@@ -1,136 +0,0 @@
-import React, { useState } from "react";
-import { X } from "@phosphor-icons/react";
-import LLMSelection from "./Steps/LLMSelection";
-import VectorDatabaseConnection from "./Steps/VectorDatabaseConnection";
-import AppearanceSetup from "./Steps/AppearanceSetup";
-import UserModeSelection from "./Steps/UserModeSelection";
-import PasswordProtection from "./Steps/PasswordProtection";
-import MultiUserSetup from "./Steps/MultiUserSetup";
-import CreateFirstWorkspace from "./Steps/CreateFirstWorkspace";
-import EmbeddingSelection from "./Steps/EmbeddingSelection";
-import DataHandling from "./Steps/DataHandling";
-import UserQuestionnaire from "./Steps/UserQuestionnaire";
-
-const DIALOG_ID = "onboarding-modal";
-
-const STEPS = {
-  llm_preference: {
-    title: "LLM Preference",
-    description:
-      "These are the credentials and settings for your preferred LLM chat & embedding provider.",
-    component: LLMSelection,
-  },
-  embedding_preferences: {
-    title: "Embedding Preference",
-    description: "Choose a provider for embedding files and text.",
-    component: EmbeddingSelection,
-  },
-  vector_database: {
-    title: "Vector Database",
-    description:
-      "These are the credentials and settings for how your AnythingLLM instance will function.",
-    component: VectorDatabaseConnection,
-  },
-  appearance: {
-    title: "Appearance",
-    description:
-      "Customize the appearance of your AnythingLLM instance.\nFind more customization options on the appearance settings page.",
-    component: AppearanceSetup,
-  },
-  user_mode_setup: {
-    title: "User Mode Setup",
-    description: "Choose how many people will be using your instance.",
-    component: UserModeSelection,
-  },
-  password_protection: {
-    title: "Password Protect",
-    description:
-      "Protect your instance with a password. It is important to save this password as it cannot be recovered.",
-    component: PasswordProtection,
-  },
-  multi_user_mode: {
-    title: "Multi-User Mode",
-    description:
-      "Setup your instance to support your team by activating multi-user mode.",
-    component: MultiUserSetup,
-  },
-  data_handling: {
-    title: "Data Handling",
-    description:
-      "We are committed to transparency and control when it comes to your personal data.",
-    component: DataHandling,
-  },
-  user_questionnaire: {
-    title: "A little about yourself",
-    description:
-      "We use information about how you use AnythingLLM to make our product better.",
-    component: UserQuestionnaire,
-  },
-  create_workspace: {
-    title: "Create Workspace",
-    description: "To get started, create a new workspace.",
-    component: CreateFirstWorkspace,
-  },
-};
-
-export const OnboardingModalId = DIALOG_ID;
-export default function OnboardingModal({ setModalVisible }) {
-  const [currentStep, setCurrentStep] = useState("llm_preference");
-  const [history, setHistory] = useState(["llm_preference"]);
-
-  function hideModal() {
-    setModalVisible(false);
-  }
-
-  const nextStep = (stepKey) => {
-    setCurrentStep(stepKey);
-    setHistory([...history, stepKey]);
-  };
-
-  const prevStep = () => {
-    const currentStepIdx = history.indexOf(currentStep);
-    if (currentStepIdx === -1 || currentStepIdx === 0) {
-      setCurrentStep("llm_preference");
-      setHistory(["llm_preference"]);
-      return hideModal();
-    }
-
-    const prevStep = history[currentStepIdx - 1];
-    const _history = [...history].slice(0, currentStepIdx);
-    setCurrentStep(prevStep);
-    setHistory(_history);
-  };
-
-  const { component: StepComponent, ...step } = STEPS[currentStep];
-  return (
-    <dialog id={DIALOG_ID} className="bg-transparent outline-none">
-      <div className="relative max-h-full">
-        <div className="relative bg-main-gradient rounded-2xl shadow border-2 border-slate-300/10">
-          <div className="flex items-start justify-between px-6 py-4 border-b rounded-t border-gray-500/50">
-            <div className="flex flex-col gap-2">
-              <h3 className="text-xl font-semibold text-white">{step.title}</h3>
-              <p className="text-sm font-base text-white text-opacity-60 whitespace-pre">
-                {step.description || ""}
-              </p>
-            </div>
-
-            <button
-              onClick={hideModal}
-              type="button"
-              className="text-gray-400 bg-transparent rounded-lg text-sm p-1.5 ml-auto inline-flex items-center hover:border-white/60 bg-sidebar-button hover:bg-menu-item-selected-gradient hover:border-slate-100 hover:border-opacity-50 border-transparent border"
-            >
-              <X className="text-gray-300 text-lg" />
-            </button>
-          </div>
-          <div className="space-y-6 flex h-full w-full justify-center">
-            <StepComponent
-              currentStep={currentStep}
-              nextStep={nextStep}
-              prevStep={prevStep}
-            />
-          </div>
-        </div>
-      </div>
-    </dialog>
-  );
-}

frontend/src/pages/OnboardingFlow/Steps/CreateWorkspace/index.jsx

@@ -0,0 +1,99 @@
+import React, { useEffect, useRef, useState } from "react";
+import illustration from "@/media/illustrations/create-workspace.png";
+import paths from "@/utils/paths";
+import showToast from "@/utils/toast";
+import { useNavigate } from "react-router-dom";
+import Workspace from "@/models/workspace";
+
+const TITLE = "Create your first workspace";
+const DESCRIPTION =
+  "Create your first workspace and get started with AnythingLLM.";
+
+export default function CreateWorkspace({
+  setHeader,
+  setForwardBtn,
+  setBackBtn,
+}) {
+  const [workspaceName, setWorkspaceName] = useState("");
+  const navigate = useNavigate();
+  const createWorkspaceRef = useRef();
+
+  useEffect(() => {
+    setHeader({ title: TITLE, description: DESCRIPTION });
+    setBackBtn({ showing: false, disabled: false, onClick: handleBack });
+  }, []);
+
+  useEffect(() => {
+    if (workspaceName.length > 3) {
+      setForwardBtn({ showing: true, disabled: false, onClick: handleForward });
+    } else {
+      setForwardBtn({ showing: true, disabled: true, onClick: handleForward });
+    }
+  }, [workspaceName]);
+
+  const handleCreate = async (e) => {
+    e.preventDefault();
+    const form = new FormData(e.target);
+    const { workspace, error } = await Workspace.new({
+      name: form.get("name"),
+      onboardingComplete: true,
+    });
+    if (!!workspace) {
+      showToast(
+        "Workspace created successfully! Taking you to home...",
+        "success"
+      );
+      await new Promise((resolve) => setTimeout(resolve, 1000));
+      navigate(paths.home());
+    } else {
+      showToast(`Failed to create workspace: ${error}`, "error");
+    }
+  };
+
+  function handleForward() {
+    createWorkspaceRef.current.click();
+  }
+
+  function handleBack() {
+    navigate(paths.onboarding.survey());
+  }
+
+  return (
+    <form
+      onSubmit={handleCreate}
+      className="w-full flex items-center justify-center flex-col gap-y-2"
+    >
+      <img src={illustration} alt="Create workspace" />
+      <div className="flex flex-col gap-y-4 w-full max-w-[600px]">
+        {" "}
+        <div className="w-full mt-4">
+          <label
+            htmlFor="name"
+            className="block mb-3 text-sm font-medium text-white"
+          >
+            Workspace Name
+          </label>
+          <input
+            name="name"
+            type="text"
+            className="bg-zinc-900 text-white text-sm rounded-lg block w-full p-2.5"
+            placeholder="My Workspace"
+            minLength={4}
+            required={true}
+            autoComplete="off"
+            onChange={(e) => setWorkspaceName(e.target.value)}
+          />
+          <div className="mt-4 text-white text-opacity-80 text-xs font-base -mb-2">
+            Workspace name must be at least 4 characters.
+          </div>
+        </div>
+      </div>
+      <button
+        type="submit"
+        ref={createWorkspaceRef}
+        hidden
+        aria-hidden="true"
+      ></button>
+    </form>
+  );
+}

frontend/src/pages/OnboardingFlow/Steps/CustomLogo/index.jsx

@@ -0,0 +1,136 @@
+import useLogo from "@/hooks/useLogo";
+import System from "@/models/system";
+import showToast from "@/utils/toast";
+import { Plus } from "@phosphor-icons/react";
+import React, { useState, useEffect } from "react";
+import AnythingLLM from "@/media/logo/anything-llm.png";
+import paths from "@/utils/paths";
+import { useNavigate } from "react-router-dom";
+
+const TITLE = "Custom Logo";
+const DESCRIPTION =
+  "Upload your custom logo to make your chatbot yours. Optional.";
+
+export default function CustomLogo({ setHeader, setForwardBtn, setBackBtn }) {
+  const navigate = useNavigate();
+  function handleForward() {
+    navigate(paths.onboarding.userSetup());
+  }
+
+  function handleBack() {
+    navigate(paths.onboarding.vectorDatabase());
+  }
+
+  useEffect(() => {
+    setHeader({ title: TITLE, description: DESCRIPTION });
+    setForwardBtn({ showing: true, disabled: false, onClick: handleForward });
+    setBackBtn({ showing: true, disabled: false, onClick: handleBack });
+  }, []);
+
+  const { logo: _initLogo, setLogo: _setLogo } = useLogo();
+  const [logo, setLogo] = useState("");
+  const [isDefaultLogo, setIsDefaultLogo] = useState(true);
+
+  useEffect(() => {
+    async function logoInit() {
+      setLogo(_initLogo || "");
+      const _isDefaultLogo = await System.isDefaultLogo();
+      setIsDefaultLogo(_isDefaultLogo);
+    }
+    logoInit();
+  }, [_initLogo]);
+
+  const handleFileUpload = async (event) => {
+    const file = event.target.files[0];
+    if (!file) return false;
+
+    const objectURL = URL.createObjectURL(file);
+    setLogo(objectURL);
+
+    const formData = new FormData();
+    formData.append("logo", file);
+    const { success, error } = await System.uploadLogo(formData);
+    if (!success) {
+      showToast(`Failed to upload logo: ${error}`, "error");
+      setLogo(_initLogo);
+      return;
+    }
+
+    const logoURL = await System.fetchLogo();
+    _setLogo(logoURL);
+
+    showToast("Image uploaded successfully.", "success", { clear: true });
+    setIsDefaultLogo(false);
+  };
+
+  const handleRemoveLogo = async () => {
+    setLogo("");
+    setIsDefaultLogo(true);
+
+    const { success, error } = await System.removeCustomLogo();
+    if (!success) {
+      console.error("Failed to remove logo:", error);
+      showToast(`Failed to remove logo: ${error}`, "error");
+      const logoURL = await System.fetchLogo();
+      setLogo(logoURL);
+      setIsDefaultLogo(false);
+      return;
+    }
+
+    const logoURL = await System.fetchLogo();
+    _setLogo(logoURL);
+
+    showToast("Image successfully removed.", "success", { clear: true });
+  };
+
+  return (
+    <div className="flex items-center w-full">
+      <div className="flex gap-x-8 flex-col w-full">
+        {isDefaultLogo ? (
+          <label className="mt-5 hover:opacity-60 w-full flex justify-center transition-all duration-300">
+            <input
+              id="logo-upload"
+              type="file"
+              accept="image/*"
+              className="hidden"
+              onChange={handleFileUpload}
+            />
+            <div
+              className="max-w-[600px] w-full h-64 max-h-[600px] py-4 bg-zinc-900/50 rounded-2xl border-2 border-dashed border-white border-opacity-60 justify-center items-center inline-flex cursor-pointer"
+              htmlFor="logo-upload"
+            >
+              <div className="flex flex-col items-center justify-center">
+                <div className="rounded-full bg-white/40">
+                  <Plus className="w-6 h-6 text-black/80 m-2" />
+                </div>
+                <div className="text-white text-opacity-80 text-sm font-semibold py-1">
+                  Add a custom logo
+                </div>
+                <div className="text-white text-opacity-60 text-xs font-medium py-1">
+                  Recommended size: 800 x 200
+                </div>
+              </div>
+            </div>
+          </label>
+        ) : (
+          <div className="w-full flex justify-center">
+            <img
+              src={logo}
+              alt="Uploaded Logo"
+              className="w-48 h-48 object-contain mr-6"
+              hidden={isDefaultLogo}
+              onError={(e) => (e.target.src = AnythingLLM)}
+            />
+          </div>
+        )}
+
+        <button
+          onClick={handleRemoveLogo}
+          className="text-white text-base font-medium hover:text-opacity-60 mt-8"
+        >
+          Remove logo
+        </button>
+      </div>
+    </div>
+  );
+}

frontend/src/pages/OnboardingFlow/Steps/DataHandling/index.jsx

@@ -1,9 +1,11 @@
-import React, { memo, useEffect, useState } from "react";
+import PreLoader from "@/components/Preloader";
 import System from "@/models/system";
 import AnythingLLMIcon from "@/media/logo/anything-llm-icon.png";
 import OpenAiLogo from "@/media/llmprovider/openai.png";
 import AzureOpenAiLogo from "@/media/llmprovider/azure.png";
 import AnthropicLogo from "@/media/llmprovider/anthropic.png";
+import GeminiLogo from "@/media/llmprovider/gemini.png";
+import OllamaLogo from "@/media/llmprovider/ollama.png";
 import LMStudioLogo from "@/media/llmprovider/lmstudio.png";
 import LocalAiLogo from "@/media/llmprovider/localai.png";
 import ChromaLogo from "@/media/vectordbs/chroma.png";
@@ -11,8 +13,13 @@ import PineconeLogo from "@/media/vectordbs/pinecone.png";
 import LanceDbLogo from "@/media/vectordbs/lancedb.png";
 import WeaviateLogo from "@/media/vectordbs/weaviate.png";
 import QDrantLogo from "@/media/vectordbs/qdrant.png";
-import PreLoader from "@/components/Preloader";
+import React, { useState, useEffect } from "react";
+import paths from "@/utils/paths";
+import { useNavigate } from "react-router-dom";
 
+const TITLE = "Data Handling & Privacy";
+const DESCRIPTION =
+  "We are committed to transparency and control when it comes to your personal data.";
 const LLM_SELECTION_PRIVACY = {
   openai: {
     name: "OpenAI",
@@ -38,6 +45,14 @@ const LLM_SELECTION_PRIVACY = {
     ],
     logo: AnthropicLogo,
   },
+  gemini: {
+    name: "Google Gemini",
+    description: [
+      "Your chats are de-identified and used in training",
+      "Your prompts and document text are visible in responses to Google",
+    ],
+    logo: GeminiLogo,
+  },
   lmstudio: {
     name: "LMStudio",
     description: [
@@ -52,6 +67,13 @@ const LLM_SELECTION_PRIVACY = {
     ],
     logo: LocalAiLogo,
   },
+  ollama: {
+    name: "Ollama",
+    description: [
+      "Your model and chats are only accessible on the machine running Ollama models",
+    ],
+    logo: OllamaLogo,
+  },
   native: {
     name: "Custom Llama Model",
     description: [
@@ -134,26 +156,36 @@ const EMBEDDING_ENGINE_PRIVACY = {
   },
 };
 
-function DataHandling({ nextStep, prevStep, currentStep }) {
+export default function DataHandling({ setHeader, setForwardBtn, setBackBtn }) {
   const [llmChoice, setLLMChoice] = useState("openai");
   const [loading, setLoading] = useState(true);
   const [vectorDb, setVectorDb] = useState("pinecone");
   const [embeddingEngine, setEmbeddingEngine] = useState("openai");
+  const navigate = useNavigate();
 
   useEffect(() => {
+    setHeader({ title: TITLE, description: DESCRIPTION });
+    setForwardBtn({ showing: true, disabled: false, onClick: handleForward });
+    setBackBtn({ showing: false, disabled: false, onClick: handleBack });
     async function fetchKeys() {
       const _settings = await System.keys();
-      setLLMChoice(_settings?.LLMProvider);
+      setLLMChoice(_settings?.LLMProvider || "openai");
-      setVectorDb(_settings?.VectorDB);
+      setVectorDb(_settings?.VectorDB || "pinecone");
-      setEmbeddingEngine(_settings?.EmbeddingEngine);
+      setEmbeddingEngine(_settings?.EmbeddingEngine || "openai");
 
       setLoading(false);
     }
-    if (currentStep === "data_handling") {
+    fetchKeys();
-      fetchKeys();
-    }
   }, []);
 
+  function handleForward() {
+    navigate(paths.onboarding.survey());
+  }
+
+  function handleBack() {
+    navigate(paths.onboarding.userSetup());
+  }
+
   if (loading)
     return (
       <div className="w-full h-full flex justify-center items-center p-20">
@@ -162,7 +194,7 @@ function DataHandling({ nextStep, prevStep, currentStep }) {
     );
 
   return (
-    <div className="max-w-[750px]">
+    <div className="w-full flex items-center justify-center flex-col gap-y-6">
       <div className="p-8 flex flex-col gap-8">
         <div className="flex flex-col gap-y-2 border-b border-zinc-500/50 pb-4">
           <div className="text-white text-base font-bold">LLM Selection</div>
@@ -222,23 +254,6 @@ function DataHandling({ nextStep, prevStep, currentStep }) {
           </ul>
         </div>
       </div>
-      <div className="flex w-[650px] justify-between items-center px-6 py-4 space-x-2 border-t rounded-b border-gray-500/50">
-        <button
-          onClick={prevStep}
-          type="button"
-          className="px-4 py-2 rounded-lg text-white hover:bg-sidebar"
-        >
-          Back
-        </button>
-        <button
-          onClick={() => nextStep("user_questionnaire")}
-          className="border border-slate-200 px-4 py-2 rounded-lg text-slate-800 bg-slate-200 text-sm items-center flex gap-x-2 hover:text-white hover:bg-transparent focus:ring-gray-800 font-semibold shadow"
-        >
-          Continue
-        </button>
-      </div>
     </div>
   );
 }
-
-export default memo(DataHandling);

frontend/src/pages/OnboardingFlow/Steps/EmbeddingPreference/index.jsx

@@ -0,0 +1,170 @@
+import { MagnifyingGlass } from "@phosphor-icons/react";
+import { useEffect, useState, useRef } from "react";
+import AnythingLLMIcon from "@/media/logo/anything-llm-icon.png";
+import OpenAiLogo from "@/media/llmprovider/openai.png";
+import AzureOpenAiLogo from "@/media/llmprovider/azure.png";
+import LocalAiLogo from "@/media/llmprovider/localai.png";
+import NativeEmbeddingOptions from "@/components/EmbeddingSelection/NativeEmbeddingOptions";
+import OpenAiOptions from "@/components/EmbeddingSelection/OpenAiOptions";
+import AzureAiOptions from "@/components/EmbeddingSelection/AzureAiOptions";
+import LocalAiOptions from "@/components/EmbeddingSelection/LocalAiOptions";
+import EmbedderItem from "@/components/EmbeddingSelection/EmbedderItem";
+import System from "@/models/system";
+import paths from "@/utils/paths";
+import showToast from "@/utils/toast";
+import { useNavigate } from "react-router-dom";
+
+const TITLE = "Embedding Preference";
+const DESCRIPTION =
+  "AnythingLLM can work with many embedding models. This will be the model which turns documents into vectors.";
+
+export default function EmbeddingPreference({
+  setHeader,
+  setForwardBtn,
+  setBackBtn,
+}) {
+  const [searchQuery, setSearchQuery] = useState("");
+  const [filteredEmbedders, setFilteredEmbedders] = useState([]);
+  const [selectedEmbedder, setSelectedEmbedder] = useState(null);
+  const [settings, setSettings] = useState(null);
+  const formRef = useRef(null);
+  const hiddenSubmitButtonRef = useRef(null);
+  const navigate = useNavigate();
+
+  useEffect(() => {
+    async function fetchKeys() {
+      const _settings = await System.keys();
+      setSettings(_settings);
+      setSelectedEmbedder(_settings?.EmbeddingEngine || "native");
+    }
+    fetchKeys();
+  }, []);
+
+  const EMBEDDERS = [
+    {
+      name: "AnythingLLM Embedder",
+      value: "native",
+      logo: AnythingLLMIcon,
+      options: <NativeEmbeddingOptions settings={settings} />,
+      description:
+        "Use the built-in embedding engine for AnythingLLM. Zero setup!",
+    },
+    {
+      name: "OpenAI",
+      value: "openai",
+      logo: OpenAiLogo,
+      options: <OpenAiOptions settings={settings} />,
+      description: "The standard option for most non-commercial use.",
+    },
+    {
+      name: "Azure OpenAI",
+      value: "azure",
+      logo: AzureOpenAiLogo,
+      options: <AzureAiOptions settings={settings} />,
+      description: "The enterprise option of OpenAI hosted on Azure services.",
+    },
+    {
+      name: "Local AI",
+      value: "localai",
+      logo: LocalAiLogo,
+      options: <LocalAiOptions settings={settings} />,
+      description: "Run embedding models locally on your own machine.",
+    },
+  ];
+
+  function handleForward() {
+    if (hiddenSubmitButtonRef.current) {
+      hiddenSubmitButtonRef.current.click();
+    }
+  }
+
+  function handleBack() {
+    navigate(paths.onboarding.llmPreference());
+  }
+
+  const handleSubmit = async (e) => {
+    e.preventDefault();
+    const form = e.target;
+    const data = {};
+    const formData = new FormData(form);
+    data.EmbeddingEngine = selectedEmbedder;
+    for (var [key, value] of formData.entries()) data[key] = value;
+
+    const { error } = await System.updateSystem(data);
+    if (error) {
+      showToast(`Failed to save embedding settings: ${error}`, "error");
+      return;
+    }
+    showToast("Embedder settings saved successfully.", "success", {
+      clear: true,
+    });
+    navigate(paths.onboarding.vectorDatabase());
+  };
+
+  useEffect(() => {
+    setHeader({ title: TITLE, description: DESCRIPTION });
+    setForwardBtn({ showing: true, disabled: false, onClick: handleForward });
+    setBackBtn({ showing: true, disabled: false, onClick: handleBack });
+  }, []);
+
+  useEffect(() => {
+    const filtered = EMBEDDERS.filter((embedder) =>
+      embedder.name.toLowerCase().includes(searchQuery.toLowerCase())
+    );
+    setFilteredEmbedders(filtered);
+  }, [searchQuery, selectedEmbedder]);
+
+  return (
+    <div>
+      <form ref={formRef} onSubmit={handleSubmit} className="w-full">
+        <div className="w-full relative border-slate-300/40 shadow border-2 rounded-lg text-white">
+          <div className="w-full p-4 absolute top-0 rounded-t-lg backdrop-blur-sm">
+            <div className="w-full flex items-center sticky top-0 z-20">
+              <MagnifyingGlass
+                size={16}
+                weight="bold"
+                className="absolute left-4 z-30 text-white"
+              />
+              <input
+                type="text"
+                placeholder="Search Embedding providers"
+                className="bg-zinc-600 z-20 pl-10 rounded-full w-full px-4 py-1 text-sm border-2 border-slate-300/40 outline-none focus:border-white text-white"
+                onChange={(e) => setSearchQuery(e.target.value)}
+                autoComplete="off"
+                onKeyDown={(e) => {
+                  if (e.key === "Enter") e.preventDefault();
+                }}
+              />
+            </div>
+          </div>
+          <div className="px-4 pt-[70px] flex flex-col gap-y-1 max-h-[390px] overflow-y-auto no-scroll pb-4">
+            {filteredEmbedders.map((embedder) => {
+              return (
+                <EmbedderItem
+                  key={embedder.name}
+                  name={embedder.name}
+                  value={embedder.value}
+                  image={embedder.logo}
+                  description={embedder.description}
+                  checked={selectedEmbedder === embedder.value}
+                  onClick={() => setSelectedEmbedder(embedder.value)}
+                />
+              );
+            })}
+          </div>
+        </div>
+        <div className="mt-4 flex flex-col gap-y-1">
+          {selectedEmbedder &&
+            EMBEDDERS.find((embedder) => embedder.value === selectedEmbedder)
+              ?.options}
+        </div>
+        <button
+          type="submit"
+          ref={hiddenSubmitButtonRef}
+          hidden
+          aria-hidden="true"
+        ></button>
+      </form>
+    </div>
+  );
+}

frontend/src/pages/OnboardingFlow/Steps/Home/index.jsx

@@ -0,0 +1,41 @@
+import paths from "@/utils/paths";
+import LGroupImg from "./l_group.png";
+import RGroupImg from "./r_group.png";
+import AnythingLLMLogo from "@/media/logo/anything-llm.png";
+import { useNavigate } from "react-router-dom";
+
+export default function OnboardingHome() {
+  const navigate = useNavigate();
+  return (
+    <>
+      <div className="relative w-screen h-screen flex overflow-hidden bg-[#2C2F35] md:bg-main-gradient">
+        <div
+          className="hidden md:block fixed bottom-10 left-10 w-[320px] h-[320px] bg-no-repeat bg-contain"
+          style={{ backgroundImage: `url(${LGroupImg})` }}
+        ></div>
+
+        <div
+          className="hidden md:block fixed top-10 right-10 w-[320px] h-[320px] bg-no-repeat bg-contain"
+          style={{ backgroundImage: `url(${RGroupImg})` }}
+        ></div>
+
+        <div className="relative flex justify-center items-center m-auto">
+          <div className="flex flex-col justify-center items-center">
+            <p className="text-zinc-300 font-thin text-[24px]">Welcome to</p>
+            <img
+              src={AnythingLLMLogo}
+              alt="AnythingLLM"
+              className="md:h-[50px] flex-shrink-0 max-w-[300px]"
+            />
+            <button
+              onClick={() => navigate(paths.onboarding.llmPreference())}
+              className="animate-pulse w-full md:max-w-[350px] md:min-w-[300px] text-center py-3 bg-white text-black font-semibold text-sm my-10 rounded-md hover:bg-gray-200"
+            >
+              Get started
+            </button>
+          </div>
+        </div>
+      </div>
+    </>
+  );
+}

frontend/src/pages/OnboardingFlow/Steps/LLMPreference/index.jsx

@@ -0,0 +1,206 @@
+import { MagnifyingGlass } from "@phosphor-icons/react";
+import { useEffect, useState, useRef } from "react";
+import OpenAiLogo from "@/media/llmprovider/openai.png";
+import AzureOpenAiLogo from "@/media/llmprovider/azure.png";
+import AnthropicLogo from "@/media/llmprovider/anthropic.png";
+import GeminiLogo from "@/media/llmprovider/gemini.png";
+import OllamaLogo from "@/media/llmprovider/ollama.png";
+import LMStudioLogo from "@/media/llmprovider/lmstudio.png";
+import LocalAiLogo from "@/media/llmprovider/localai.png";
+import AnythingLLMIcon from "@/media/logo/anything-llm-icon.png";
+import OpenAiOptions from "@/components/LLMSelection/OpenAiOptions";
+import AzureAiOptions from "@/components/LLMSelection/AzureAiOptions";
+import AnthropicAiOptions from "@/components/LLMSelection/AnthropicAiOptions";
+import LMStudioOptions from "@/components/LLMSelection/LMStudioOptions";
+import LocalAiOptions from "@/components/LLMSelection/LocalAiOptions";
+import NativeLLMOptions from "@/components/LLMSelection/NativeLLMOptions";
+import GeminiLLMOptions from "@/components/LLMSelection/GeminiLLMOptions";
+import OllamaLLMOptions from "@/components/LLMSelection/OllamaLLMOptions";
+import LLMItem from "@/components/LLMSelection/LLMItem";
+import System from "@/models/system";
+import paths from "@/utils/paths";
+import showToast from "@/utils/toast";
+import { useNavigate } from "react-router-dom";
+
+const TITLE = "LLM Preference";
+const DESCRIPTION =
+  "AnythingLLM can work with many LLM providers. This will be the service which handles chatting.";
+
+export default function LLMPreference({
+  setHeader,
+  setForwardBtn,
+  setBackBtn,
+}) {
+  const [searchQuery, setSearchQuery] = useState("");
+  const [filteredLLMs, setFilteredLLMs] = useState([]);
+  const [selectedLLM, setSelectedLLM] = useState(null);
+  const [settings, setSettings] = useState(null);
+  const formRef = useRef(null);
+  const hiddenSubmitButtonRef = useRef(null);
+  const isHosted = window.location.hostname.includes("useanything.com");
+  const navigate = useNavigate();
+
+  useEffect(() => {
+    async function fetchKeys() {
+      const _settings = await System.keys();
+      setSettings(_settings);
+      setSelectedLLM(_settings?.LLMProvider || "openai");
+    }
+    fetchKeys();
+  }, []);
+
+  const LLMS = [
+    {
+      name: "OpenAI",
+      value: "openai",
+      logo: OpenAiLogo,
+      options: <OpenAiOptions settings={settings} />,
+      description: "The standard option for most non-commercial use.",
+    },
+    {
+      name: "Azure OpenAI",
+      value: "azure",
+      logo: AzureOpenAiLogo,
+      options: <AzureAiOptions settings={settings} />,
+      description: "The enterprise option of OpenAI hosted on Azure services.",
+    },
+    {
+      name: "Anthropic",
+      value: "anthropic",
+      logo: AnthropicLogo,
+      options: <AnthropicAiOptions settings={settings} />,
+      description: "A friendly AI Assistant hosted by Anthropic.",
+    },
+    {
+      name: "Gemini",
+      value: "gemini",
+      logo: GeminiLogo,
+      options: <GeminiLLMOptions settings={settings} />,
+      description: "Google's largest and most capable AI model",
+    },
+    {
+      name: "Ollama",
+      value: "ollama",
+      logo: OllamaLogo,
+      options: <OllamaLLMOptions settings={settings} />,
+      description: "Run LLMs locally on your own machine.",
+    },
+    {
+      name: "LM Studio",
+      value: "lmstudio",
+      logo: LMStudioLogo,
+      options: <LMStudioOptions settings={settings} />,
+      description:
+        "Discover, download, and run thousands of cutting edge LLMs in a few clicks.",
+    },
+    {
+      name: "Local AI",
+      value: "localai",
+      logo: LocalAiLogo,
+      options: <LocalAiOptions settings={settings} />,
+      description: "Run LLMs locally on your own machine.",
+    },
+    {
+      name: "Native",
+      value: "native",
+      logo: AnythingLLMIcon,
+      options: <NativeLLMOptions settings={settings} />,
+      description:
+        "Use a downloaded custom Llama model for chatting on this AnythingLLM instance.",
+    },
+  ];
+
+  function handleForward() {
+    if (hiddenSubmitButtonRef.current) {
+      hiddenSubmitButtonRef.current.click();
+    }
+  }
+
+  function handleBack() {
+    navigate(paths.onboarding.home());
+  }
+
+  const handleSubmit = async (e) => {
+    e.preventDefault();
+    const form = e.target;
+    const data = {};
+    const formData = new FormData(form);
+    data.LLMProvider = selectedLLM;
+    for (var [key, value] of formData.entries()) data[key] = value;
+
+    const { error } = await System.updateSystem(data);
+    if (error) {
+      showToast(`Failed to save LLM settings: ${error}`, "error");
+      return;
+    }
+    showToast("LLM settings saved successfully.", "success", { clear: true });
+    navigate(paths.onboarding.embeddingPreference());
+  };
+
+  useEffect(() => {
+    setHeader({ title: TITLE, description: DESCRIPTION });
+    setForwardBtn({ showing: true, disabled: false, onClick: handleForward });
+    setBackBtn({ showing: true, disabled: false, onClick: handleBack });
+  }, []);
+
+  useEffect(() => {
+    const filtered = LLMS.filter((llm) =>
+      llm.name.toLowerCase().includes(searchQuery.toLowerCase())
+    );
+    setFilteredLLMs(filtered);
+  }, [searchQuery, selectedLLM]);
+
+  return (
+    <div>
+      <form ref={formRef} onSubmit={handleSubmit} className="w-full">
+        <div className="w-full relative border-slate-300/40 shadow border-2 rounded-lg text-white">
+          <div className="w-full p-4 absolute top-0 rounded-t-lg backdrop-blur-sm">
+            <div className="w-full flex items-center sticky top-0">
+              <MagnifyingGlass
+                size={16}
+                weight="bold"
+                className="absolute left-4 z-30 text-white"
+              />
+              <input
+                type="text"
+                placeholder="Search LLM providers"
+                className="bg-zinc-600 z-20 pl-10 rounded-full w-full px-4 py-1 text-sm border-2 border-slate-300/40 outline-none focus:border-white text-white"
+                onChange={(e) => setSearchQuery(e.target.value)}
+                autoComplete="off"
+                onKeyDown={(e) => {
+                  if (e.key === "Enter") e.preventDefault();
+                }}
+              />
+            </div>
+          </div>
+          <div className="px-4 pt-[70px] flex flex-col gap-y-1 max-h-[390px] overflow-y-auto no-scroll pb-4">
+            {filteredLLMs.map((llm) => {
+              if (llm.value === "native" && isHosted) return null;
+              return (
+                <LLMItem
+                  key={llm.name}
+                  name={llm.name}
+                  value={llm.value}
+                  image={llm.logo}
+                  description={llm.description}
+                  checked={selectedLLM === llm.value}
+                  onClick={() => setSelectedLLM(llm.value)}
+                />
+              );
+            })}
+          </div>
+        </div>
+        <div className="mt-4 flex flex-col gap-y-1">
+          {selectedLLM &&
+            LLMS.find((llm) => llm.value === selectedLLM)?.options}
+        </div>
+        <button
+          type="submit"
+          ref={hiddenSubmitButtonRef}
+          hidden
+          aria-hidden="true"
+        ></button>
+      </form>
+    </div>
+  );
+}

frontend/src/pages/OnboardingFlow/Steps/Survey/index.jsx

@@ -0,0 +1,297 @@
+import { COMPLETE_QUESTIONNAIRE } from "@/utils/constants";
+import paths from "@/utils/paths";
+import { CheckCircle } from "@phosphor-icons/react";
+import React, { useState, useEffect, useRef } from "react";
+import { useNavigate } from "react-router-dom";
+
+const TITLE = "Welcome to AnythingLLM";
+const DESCRIPTION = "Help us make AnythingLLM built for your needs. Optional.";
+
+async function sendQuestionnaire({ email, useCase, comment }) {
+  if (import.meta.env.DEV) return;
+  return fetch(`https://onboarding-wxich7363q-uc.a.run.app`, {
+    method: "POST",
+    body: JSON.stringify({
+      email,
+      useCase,
+      comment,
+      sourceId: "0VRjqHh6Vukqi0x0Vd0n/m8JuT7k8nOz",
+    }),
+  })
+    .then(() => {
+      window.localStorage.setItem(COMPLETE_QUESTIONNAIRE, true);
+      console.log(`✅ Questionnaire responses sent.`);
+    })
+    .catch((error) => {
+      console.error(`sendQuestionnaire`, error.message);
+    });
+}
+
+export default function Survey({ setHeader, setForwardBtn, setBackBtn }) {
+  const [selectedOption, setSelectedOption] = useState("");
+  const formRef = useRef(null);
+  const navigate = useNavigate();
+  const submitRef = useRef(null);
+
+  function handleForward() {
+    if (!!window?.localStorage?.getItem(COMPLETE_QUESTIONNAIRE)) {
+      navigate(paths.onboarding.createWorkspace());
+      return;
+    }
+    if (submitRef.current) {
+      submitRef.current.click();
+    }
+  }
+
+  function skipSurvey() {
+    navigate(paths.onboarding.createWorkspace());
+  }
+
+  function handleBack() {
+    navigate(paths.onboarding.dataHandling());
+  }
+
+  useEffect(() => {
+    setHeader({ title: TITLE, description: DESCRIPTION });
+    setForwardBtn({ showing: true, disabled: false, onClick: handleForward });
+    setBackBtn({ showing: true, disabled: false, onClick: handleBack });
+  }, []);
+
+  const handleSubmit = async (e) => {
+    e.preventDefault();
+    const form = e.target;
+    const formData = new FormData(form);
+
+    await sendQuestionnaire({
+      email: formData.get("email"),
+      useCase: formData.get("use_case") || "other",
+      comment: formData.get("comment") || null,
+    });
+
+    navigate(paths.onboarding.createWorkspace());
+  };
+
+  if (!!window?.localStorage?.getItem(COMPLETE_QUESTIONNAIRE)) {
+    return (
+      <div className="w-full flex justify-center items-center py-40">
+        <div className="w-full flex items-center justify-center px-1 md:px-8 py-4">
+          <div className="w-auto flex flex-col gap-y-1 items-center">
+            <CheckCircle size={60} className="text-green-500" />
+            <p className="text-white text-lg">Thank you for your feedback!</p>
+            <a
+              href={paths.mailToMintplex()}
+              className="text-sky-400 underline text-xs"
+            >
+              team@mintplexlabs.com
+            </a>
+          </div>
+        </div>
+      </div>
+    );
+  }
+
+  return (
+    <div className="w-full flex justify-center">
+      <form onSubmit={handleSubmit} ref={formRef} className="">
+        <div className="md:min-w-[400px]">
+          <label htmlFor="email" className="text-white text-base font-medium">
+            What's your email?{" "}
+          </label>
+          <input
+            name="email"
+            type="email"
+            placeholder="you@gmail.com"
+            required={true}
+            className="mt-2 bg-zinc-900 text-white text-sm font-medium font-['Plus Jakarta Sans'] leading-tight w-full h-11 p-2.5 bg-zinc-900 rounded-lg"
+          />
+        </div>
+
+        <div className="mt-8">
+          <label
+            className="text-white text-base font-medium"
+            htmlFor="use_case"
+          >
+            What will you use AnythingLLM for?{" "}
+          </label>
+          <div className="mt-2 gap-y-3 flex flex-col">
+            <label
+              className={`transition-all duration-300 w-full h-11 p-2.5 bg-white/10 rounded-lg flex justify-start items-center gap-2.5 cursor-pointer border border-transparent ${
+                selectedOption === "business"
+                  ? "border-white border-opacity-40"
+                  : ""
+              } hover:border-white/60`}
+            >
+              <input
+                type="radio"
+                name="use_case"
+                value={"business"}
+                checked={selectedOption === "business"}
+                onChange={(e) => setSelectedOption(e.target.value)}
+                className="hidden"
+              />
+              <div
+                className={`w-4 h-4 rounded-full border-2 border-white mr-2 ${
+                  selectedOption === "business" ? "bg-white" : ""
+                }`}
+              ></div>
+              <div className="text-white text-sm font-medium font-['Plus Jakarta Sans'] leading-tight">
+                For my business
+              </div>
+            </label>
+            <label
+              className={`transition-all duration-300 w-full h-11 p-2.5 bg-white/10 rounded-lg flex justify-start items-center gap-2.5 cursor-pointer border border-transparent ${
+                selectedOption === "personal"
+                  ? "border-white border-opacity-40"
+                  : ""
+              } hover:border-white/60`}
+            >
+              <input
+                type="radio"
+                name="use_case"
+                value={"personal"}
+                checked={selectedOption === "personal"}
+                onChange={(e) => setSelectedOption(e.target.value)}
+                className="hidden"
+              />
+              <div
+                className={`w-4 h-4 rounded-full border-2 border-white mr-2 ${
+                  selectedOption === "personal" ? "bg-white" : ""
+                }`}
+              ></div>
+              <div className="text-white text-sm font-medium font-['Plus Jakarta Sans'] leading-tight">
+                For personal use
+              </div>
+            </label>
+            <label
+              className={`transition-all duration-300 w-full h-11 p-2.5 bg-white/10 rounded-lg flex justify-start items-center gap-2.5 cursor-pointer border border-transparent ${
+                selectedOption === "education"
+                  ? "border-white border-opacity-40"
+                  : ""
+              } hover:border-white/60`}
+            >
+              <input
+                type="radio"
+                name="use_case"
+                value={"education"}
+                checked={selectedOption === "education"}
+                onChange={(e) => setSelectedOption(e.target.value)}
+                className="hidden"
+              />
+              <div
+                className={`w-4 h-4 rounded-full border-2 border-white mr-2 ${
+                  selectedOption === "education" ? "bg-white" : ""
+                }`}
+              ></div>
+              <div className="text-white text-sm font-medium font-['Plus Jakarta Sans'] leading-tight">
+                For my education
+              </div>
+            </label>
+            <label
+              className={`transition-all duration-300 w-full h-11 p-2.5 bg-white/10 rounded-lg flex justify-start items-center gap-2.5 cursor-pointer border border-transparent ${
+                selectedOption === "side_hustle"
+                  ? "border-white border-opacity-40"
+                  : ""
+              } hover:border-white/60`}
+            >
+              <input
+                type="radio"
+                name="use_case"
+                value={"side_hustle"}
+                checked={selectedOption === "side_hustle"}
+                onChange={(e) => setSelectedOption(e.target.value)}
+                className="hidden"
+              />
+              <div
+                className={`w-4 h-4 rounded-full border-2 border-white mr-2 ${
+                  selectedOption === "side_hustle" ? "bg-white" : ""
+                }`}
+              ></div>
+              <div className="text-white text-sm font-medium font-['Plus Jakarta Sans'] leading-tight">
+                For my side-hustle
+              </div>
+            </label>
+            <label
+              className={`transition-all duration-300 w-full h-11 p-2.5 bg-white/10 rounded-lg flex justify-start items-center gap-2.5 cursor-pointer border border-transparent ${
+                selectedOption === "job" ? "border-white border-opacity-40" : ""
+              } hover:border-white/60`}
+            >
+              <input
+                type="radio"
+                name="use_case"
+                value={"job"}
+                checked={selectedOption === "job"}
+                onChange={(e) => setSelectedOption(e.target.value)}
+                className="hidden"
+              />
+              <div
+                className={`w-4 h-4 rounded-full border-2 border-white mr-2 ${
+                  selectedOption === "job" ? "bg-white" : ""
+                }`}
+              ></div>
+              <div className="text-white text-sm font-medium font-['Plus Jakarta Sans'] leading-tight">
+                For my job
+              </div>
+            </label>
+            <label
+              className={`transition-all duration-300 w-full h-11 p-2.5 bg-white/10 rounded-lg flex justify-start items-center gap-2.5 cursor-pointer border border-transparent ${
+                selectedOption === "other"
+                  ? "border-white border-opacity-40"
+                  : ""
+              } hover:border-white/60`}
+            >
+              <input
+                type="radio"
+                name="use_case"
+                value={"other"}
+                checked={selectedOption === "other"}
+                onChange={(e) => setSelectedOption(e.target.value)}
+                className="hidden"
+              />
+              <div
+                className={`w-4 h-4 rounded-full border-2 border-white mr-2 ${
+                  selectedOption === "other" ? "bg-white" : ""
+                }`}
+              ></div>
+              <div className="text-white text-sm font-medium font-['Plus Jakarta Sans'] leading-tight">
+                Other
+              </div>
+            </label>
+          </div>
+        </div>
+
+        <div className="mt-8">
+          <label htmlFor="comment" className="text-white text-base font-medium">
+            Any comments for the team?{" "}
+            <span className="text-neutral-400 text-base font-light">
+              (Optional)
+            </span>
+          </label>
+          <textarea
+            name="comment"
+            rows={5}
+            className="mt-2 bg-zinc-900 text-white text-sm rounded-lg focus:ring-blue-500 focus:border-blue-500 block w-full p-2.5"
+            placeholder="If you have any questions or comments right now, you can leave them here and we will get back to you. You can also email team@mintplexlabs.com"
+            wrap="soft"
+            autoComplete="off"
+          />
+        </div>
+        <button
+          type="submit"
+          ref={submitRef}
+          hidden
+          aria-hidden="true"
+        ></button>
+
+        <div className="w-full flex items-center justify-center">
+          <button
+            type="button"
+            onClick={skipSurvey}
+            className="text-white text-base font-medium text-opacity-30 hover:text-opacity-100 mt-8"
+          >
+            Skip Survey
+          </button>
+        </div>
+      </form>
+    </div>
+  );
+}

frontend/src/pages/OnboardingFlow/Steps/UserSetup/index.jsx

@@ -0,0 +1,336 @@
+import System from "@/models/system";
+import showToast from "@/utils/toast";
+import React, { useState, useEffect, useRef } from "react";
+import debounce from "lodash.debounce";
+import paths from "@/utils/paths";
+import { useNavigate } from "react-router-dom";
+import { AUTH_TIMESTAMP, AUTH_TOKEN, AUTH_USER } from "@/utils/constants";
+
+const TITLE = "User Setup";
+const DESCRIPTION = "Configure your user settings.";
+
+export default function UserSetup({ setHeader, setForwardBtn, setBackBtn }) {
+  const [selectedOption, setSelectedOption] = useState("");
+  const [singleUserPasswordValid, setSingleUserPasswordValid] = useState(false);
+  const [multiUserLoginValid, setMultiUserLoginValid] = useState(false);
+  const [enablePassword, setEnablePassword] = useState(false);
+  const myTeamSubmitRef = useRef(null);
+  const justMeSubmitRef = useRef(null);
+  const navigate = useNavigate();
+
+  function handleForward() {
+    if (selectedOption === "just_me" && enablePassword) {
+      justMeSubmitRef.current?.click();
+    } else if (selectedOption === "just_me" && !enablePassword) {
+      navigate(paths.onboarding.dataHandling());
+    } else if (selectedOption === "my_team") {
+      myTeamSubmitRef.current?.click();
+    }
+  }
+
+  function handleBack() {
+    navigate(paths.onboarding.customLogo());
+  }
+
+  useEffect(() => {
+    let isDisabled = true;
+    if (selectedOption === "just_me") {
+      isDisabled = !singleUserPasswordValid;
+    } else if (selectedOption === "my_team") {
+      isDisabled = !multiUserLoginValid;
+    }
+
+    setForwardBtn({
+      showing: true,
+      disabled: isDisabled,
+      onClick: handleForward,
+    });
+  }, [selectedOption, singleUserPasswordValid, multiUserLoginValid]);
+
+  useEffect(() => {
+    setHeader({ title: TITLE, description: DESCRIPTION });
+    setBackBtn({ showing: true, disabled: false, onClick: handleBack });
+  }, []);
+
+  return (
+    <div className="w-full flex items-center justify-center flex-col gap-y-6">
+      <div className="flex flex-col border rounded-lg border-white/20 p-8 items-center gap-y-4 w-full max-w-[600px]">
+        <div className=" text-white text-sm font-semibold md:-ml-44">
+          How many people will be using your instance?
+        </div>
+        <div className="flex flex-col md:flex-row gap-6 w-full justify-center">
+          <button
+            onClick={() => setSelectedOption("just_me")}
+            className={`${
+              selectedOption === "just_me"
+                ? "text-sky-400 border-sky-400/70"
+                : "text-white border-white/40"
+            } min-w-[230px] h-11 p-4 rounded-[10px] border-2  justify-center items-center gap-[100px] inline-flex hover:border-sky-400/70 hover:text-sky-400 transition-all duration-300`}
+          >
+            <div className="text-center text-sm font-bold">Just me</div>
+          </button>
+          <button
+            onClick={() => setSelectedOption("my_team")}
+            className={`${
+              selectedOption === "my_team"
+                ? "text-sky-400 border-sky-400/70"
+                : "text-white border-white/40"
+            } min-w-[230px] h-11 p-4 rounded-[10px] border-2  justify-center items-center gap-[100px] inline-flex hover:border-sky-400/70 hover:text-sky-400 transition-all duration-300`}
+          >
+            <div className="text-center text-sm font-bold">My team</div>
+          </button>
+        </div>
+      </div>
+      {selectedOption === "just_me" && (
+        <JustMe
+          setSingleUserPasswordValid={setSingleUserPasswordValid}
+          enablePassword={enablePassword}
+          setEnablePassword={setEnablePassword}
+          justMeSubmitRef={justMeSubmitRef}
+          navigate={navigate}
+        />
+      )}
+      {selectedOption === "my_team" && (
+        <MyTeam
+          setMultiUserLoginValid={setMultiUserLoginValid}
+          myTeamSubmitRef={myTeamSubmitRef}
+          navigate={navigate}
+        />
+      )}
+    </div>
+  );
+}
+
+const JustMe = ({
+  setSingleUserPasswordValid,
+  enablePassword,
+  setEnablePassword,
+  justMeSubmitRef,
+  navigate,
+}) => {
+  const [itemSelected, setItemSelected] = useState(false);
+  const [password, setPassword] = useState("");
+  const handleSubmit = async (e) => {
+    e.preventDefault();
+    const form = e.target;
+    const formData = new FormData(form);
+    const { error } = await System.updateSystemPassword({
+      usePassword: true,
+      newPassword: formData.get("password"),
+    });
+
+    if (error) {
+      showToast(`Failed to set password: ${error}`, "error");
+      return;
+    }
+
+    showToast("Password set successfully!", "success", { clear: true });
+
+    // Auto-request token with password that was just set so they
+    // are not redirected to login after completion.
+    const { token } = await System.requestToken({
+      password: formData.get("password"),
+    });
+    window.localStorage.removeItem(AUTH_USER);
+    window.localStorage.removeItem(AUTH_TIMESTAMP);
+    window.localStorage.setItem(AUTH_TOKEN, token);
+
+    navigate(paths.onboarding.dataHandling());
+  };
+
+  const setNewPassword = (e) => setPassword(e.target.value);
+  const handlePasswordChange = debounce(setNewPassword, 500);
+
+  function handleYes() {
+    setItemSelected(true);
+    setEnablePassword(true);
+  }
+
+  function handleNo() {
+    setItemSelected(true);
+    setEnablePassword(false);
+  }
+
+  useEffect(() => {
+    if (enablePassword && itemSelected && password.length >= 8) {
+      setSingleUserPasswordValid(true);
+    } else if (!enablePassword && itemSelected) {
+      setSingleUserPasswordValid(true);
+    } else {
+      setSingleUserPasswordValid(false);
+    }
+  });
+  return (
+    <div className="w-full flex items-center justify-center flex-col gap-y-6">
+      <div className="flex flex-col border rounded-lg border-white/20 p-8 items-center gap-y-4 w-full max-w-[600px]">
+        <div className=" text-white text-sm font-semibold md:-ml-56">
+          Would you like to set up a password?
+        </div>
+        <div className="flex flex-col md:flex-row gap-6 w-full justify-center">
+          <button
+            onClick={handleYes}
+            className={`${
+              enablePassword && itemSelected
+                ? "text-sky-400 border-sky-400/70"
+                : "text-white border-white/40"
+            } min-w-[230px] h-11 p-4 rounded-[10px] border-2  justify-center items-center gap-[100px] inline-flex hover:border-sky-400/70 hover:text-sky-400 transition-all duration-300`}
+          >
+            <div className="text-center text-sm font-bold">Yes</div>
+          </button>
+          <button
+            onClick={handleNo}
+            className={`${
+              !enablePassword && itemSelected
+                ? "text-sky-400 border-sky-400/70"
+                : "text-white border-white/40"
+            } min-w-[230px] h-11 p-4 rounded-[10px] border-2  justify-center items-center gap-[100px] inline-flex hover:border-sky-400/70 hover:text-sky-400 transition-all duration-300`}
+          >
+            <div className="text-center text-sm font-bold">No</div>
+          </button>
+        </div>
+        {enablePassword && (
+          <form className="w-full mt-4" onSubmit={handleSubmit}>
+            <label
+              htmlFor="name"
+              className="block mb-3 text-sm font-medium text-white"
+            >
+              Instance Password
+            </label>
+            <input
+              name="password"
+              type="password"
+              className="bg-zinc-900 text-white text-sm rounded-lg block w-full p-2.5"
+              placeholder="Your admin password"
+              minLength={6}
+              required={true}
+              autoComplete="off"
+              onChange={handlePasswordChange}
+            />
+            <div className="mt-4 text-white text-opacity-80 text-xs font-base -mb-2">
+              Passwords must be at least 8 characters.
+              <br />
+              <i>
+                It's important to save this password because there is no
+                recovery method.
+              </i>{" "}
+            </div>
+            <button
+              type="submit"
+              ref={justMeSubmitRef}
+              hidden
+              aria-hidden="true"
+            ></button>
+          </form>
+        )}
+      </div>
+    </div>
+  );
+};
+
+const MyTeam = ({ setMultiUserLoginValid, myTeamSubmitRef, navigate }) => {
+  const [username, setUsername] = useState("");
+  const [password, setPassword] = useState("");
+
+  const handleSubmit = async (e) => {
+    e.preventDefault();
+    const form = e.target;
+    const formData = new FormData(form);
+    const data = {
+      username: formData.get("username"),
+      password: formData.get("password"),
+    };
+    const { success, error } = await System.setupMultiUser(data);
+    if (!success) {
+      showToast(`Error: ${error}`, "error");
+      return;
+    }
+
+    showToast("Multi-user login enabled.", "success", { clear: true });
+    navigate(paths.onboarding.dataHandling());
+
+    // Auto-request token with credentials that was just set so they
+    // are not redirected to login after completion.
+    const { user, token } = await System.requestToken(data);
+    window.localStorage.setItem(AUTH_USER, JSON.stringify(user));
+    window.localStorage.setItem(AUTH_TOKEN, token);
+    window.localStorage.removeItem(AUTH_TIMESTAMP);
+  };
+
+  const setNewUsername = (e) => setUsername(e.target.value);
+  const setNewPassword = (e) => setPassword(e.target.value);
+  const handleUsernameChange = debounce(setNewUsername, 500);
+  const handlePasswordChange = debounce(setNewPassword, 500);
+
+  useEffect(() => {
+    if (username.length >= 6 && password.length >= 8) {
+      setMultiUserLoginValid(true);
+    } else {
+      setMultiUserLoginValid(false);
+    }
+  }, [username, password]);
+  return (
+    <div className="w-full flex items-center justify-center border max-w-[600px] rounded-lg border-white/20">
+      <form onSubmit={handleSubmit}>
+        <div className="flex flex-col w-full md:px-8 px-2 py-4">
+          <div className="space-y-6 flex h-full w-full">
+            <div className="w-full flex flex-col gap-y-4">
+              <div>
+                <label
+                  htmlFor="name"
+                  className="block mb-3 text-sm font-medium text-white"
+                >
+                  Admin account username
+                </label>
+                <input
+                  name="username"
+                  type="text"
+                  className="bg-zinc-900 text-white text-sm rounded-lg block w-full p-2.5"
+                  placeholder="Your admin username"
+                  minLength={6}
+                  required={true}
+                  autoComplete="off"
+                  onChange={handleUsernameChange}
+                />
+              </div>
+              <div className="mt-4">
+                <label
+                  htmlFor="name"
+                  className="block mb-3 text-sm font-medium text-white"
+                >
+                  Admin account password
+                </label>
+                <input
+                  name="password"
+                  type="password"
+                  className="bg-zinc-900 text-white text-sm rounded-lg block w-full p-2.5"
+                  placeholder="Your admin password"
+                  minLength={8}
+                  required={true}
+                  autoComplete="off"
+                  onChange={handlePasswordChange}
+                />
+              </div>
+              <p className="w-96 text-white text-opacity-80 text-xs font-base">
+                Username must be at least 6 characters long. Password must be at
+                least 8 characters long.
+              </p>
+            </div>
+          </div>
+        </div>
+        <div className="flex w-full justify-between items-center px-6 py-4 space-x-6 border-t rounded-b border-gray-500/50">
+          <div className=" text-white text-opacity-80 text-xs font-base">
+            By default, you will be the only admin. Once onboarding is completed
+            you can create and invite others to be users or admins. Do not lose
+            your password as only admins can reset passwords.
+          </div>
+        </div>
+        <button
+          type="submit"
+          ref={myTeamSubmitRef}
+          hidden
+          aria-hidden="true"
+        ></button>
+      </form>
+    </div>
+  );
+};

frontend/src/pages/OnboardingFlow/Steps/VectorDatabaseConnection/index.jsx

@@ -0,0 +1,177 @@
+import React, { useEffect, useState, useRef } from "react";
+import { MagnifyingGlass } from "@phosphor-icons/react";
+import ChromaLogo from "@/media/vectordbs/chroma.png";
+import PineconeLogo from "@/media/vectordbs/pinecone.png";
+import LanceDbLogo from "@/media/vectordbs/lancedb.png";
+import WeaviateLogo from "@/media/vectordbs/weaviate.png";
+import QDrantLogo from "@/media/vectordbs/qdrant.png";
+import System from "@/models/system";
+import paths from "@/utils/paths";
+import PineconeDBOptions from "@/components/VectorDBSelection/PineconeDBOptions";
+import ChromaDBOptions from "@/components/VectorDBSelection/ChromaDBOptions";
+import QDrantDBOptions from "@/components/VectorDBSelection/QDrantDBOptions";
+import WeaviateDBOptions from "@/components/VectorDBSelection/WeaviateDBOptions";
+import LanceDBOptions from "@/components/VectorDBSelection/LanceDBOptions";
+import showToast from "@/utils/toast";
+import { useNavigate } from "react-router-dom";
+import VectorDBItem from "@/components/VectorDBSelection/VectorDBItem";
+
+const TITLE = "Vector Database Connection";
+const DESCRIPTION =
+  "These are the credentials and settings for your vector database of choice.";
+
+export default function VectorDatabaseConnection({
+  setHeader,
+  setForwardBtn,
+  setBackBtn,
+}) {
+  const [searchQuery, setSearchQuery] = useState("");
+  const [filteredVDBs, setFilteredVDBs] = useState([]);
+  const [selectedVDB, setSelectedVDB] = useState(null);
+  const [settings, setSettings] = useState(null);
+  const formRef = useRef(null);
+  const hiddenSubmitButtonRef = useRef(null);
+  const navigate = useNavigate();
+
+  useEffect(() => {
+    async function fetchKeys() {
+      const _settings = await System.keys();
+      setSettings(_settings);
+      setSelectedVDB(_settings?.VectorDB || "lancedb");
+    }
+    fetchKeys();
+  }, []);
+
+  const VECTOR_DBS = [
+    {
+      name: "LanceDB",
+      value: "lancedb",
+      logo: LanceDbLogo,
+      options: <LanceDBOptions />,
+      description:
+        "100% local vector DB that runs on the same instance as AnythingLLM.",
+    },
+    {
+      name: "Chroma",
+      value: "chroma",
+      logo: ChromaLogo,
+      options: <ChromaDBOptions settings={settings} />,
+      description:
+        "Open source vector database you can host yourself or on the cloud.",
+    },
+    {
+      name: "Pinecone",
+      value: "pinecone",
+      logo: PineconeLogo,
+      options: <PineconeDBOptions settings={settings} />,
+      description: "100% cloud-based vector database for enterprise use cases.",
+    },
+    {
+      name: "QDrant",
+      value: "qdrant",
+      logo: QDrantLogo,
+      options: <QDrantDBOptions settings={settings} />,
+      description: "Open source local and distributed cloud vector database.",
+    },
+    {
+      name: "Weaviate",
+      value: "weaviate",
+      logo: WeaviateLogo,
+      options: <WeaviateDBOptions settings={settings} />,
+      description:
+        "Open source local and cloud hosted multi-modal vector database.",
+    },
+  ];
+
+  function handleForward() {
+    if (hiddenSubmitButtonRef.current) {
+      hiddenSubmitButtonRef.current.click();
+    }
+  }
+
+  function handleBack() {
+    navigate(paths.onboarding.embeddingPreference());
+  }
+
+  const handleSubmit = async (e) => {
+    e.preventDefault();
+    const form = e.target;
+    const data = {};
+    const formData = new FormData(form);
+    data.VectorDB = selectedVDB;
+    for (var [key, value] of formData.entries()) data[key] = value;
+    const { error } = await System.updateSystem(data);
+    if (error) {
+      showToast(`Failed to save Vector Database settings: ${error}`, "error");
+      return;
+    }
+    showToast("Vector Database settings saved successfully.", "success", {
+      clear: true,
+    });
+    navigate(paths.onboarding.customLogo());
+  };
+
+  useEffect(() => {
+    setHeader({ title: TITLE, description: DESCRIPTION });
+    setForwardBtn({ showing: true, disabled: false, onClick: handleForward });
+    setBackBtn({ showing: true, disabled: false, onClick: handleBack });
+  }, []);
+
+  useEffect(() => {
+    const filtered = VECTOR_DBS.filter((vdb) =>
+      vdb.name.toLowerCase().includes(searchQuery.toLowerCase())
+    );
+    setFilteredVDBs(filtered);
+  }, [searchQuery, selectedVDB]);
+
+  return (
+    <>
+      <form ref={formRef} onSubmit={handleSubmit} className="w-full">
+        <div className="w-full relative border-slate-300/40 shadow border-2 rounded-lg text-white pb-4">
+          <div className="w-full p-4 absolute top-0 rounded-t-lg backdrop-blur-sm">
+            <div className="w-full flex items-center sticky top-0 z-20">
+              <MagnifyingGlass
+                size={16}
+                weight="bold"
+                className="absolute left-4 z-30 text-white"
+              />
+              <input
+                type="text"
+                placeholder="Search vector databases"
+                className="bg-zinc-600 z-20 pl-10 rounded-full w-full px-4 py-1 text-sm border-2 border-slate-300/40 outline-none focus:border-white text-white"
+                onChange={(e) => setSearchQuery(e.target.value)}
+                autoComplete="off"
+                onKeyDown={(e) => {
+                  if (e.key === "Enter") e.preventDefault();
+                }}
+              />
+            </div>
+          </div>
+          <div className="px-4 pt-[70px] flex flex-col gap-y-1 max-h-[390px] overflow-y-auto no-scroll">
+            {filteredVDBs.map((vdb) => (
+              <VectorDBItem
+                key={vdb.name}
+                name={vdb.name}
+                value={vdb.value}
+                image={vdb.logo}
+                description={vdb.description}
+                checked={selectedVDB === vdb.value}
+                onClick={setSelectedVDB}
+              />
+            ))}
+          </div>
+        </div>
+        <div className="mt-4 flex flex-col gap-y-1">
+          {selectedVDB &&
+            VECTOR_DBS.find((vdb) => vdb.value === selectedVDB)?.options}
+        </div>
+        <button
+          type="submit"
+          ref={hiddenSubmitButtonRef}
+          hidden
+          aria-hidden="true"
+        ></button>
+      </form>
+    </>
+  );
+}

frontend/src/pages/OnboardingFlow/Steps/index.jsx

@@ -0,0 +1,130 @@
+import { ArrowLeft, ArrowRight } from "@phosphor-icons/react";
+import { lazy, useState } from "react";
+import { isMobile } from "react-device-detect";
+const OnboardingSteps = {
+  home: lazy(() => import("./Home")),
+  "llm-preference": lazy(() => import("./LLMPreference")),
+  "embedding-preference": lazy(() => import("./EmbeddingPreference")),
+  "vector-database": lazy(() => import("./VectorDatabaseConnection")),
+  "custom-logo": lazy(() => import("./CustomLogo")),
+  "user-setup": lazy(() => import("./UserSetup")),
+  "data-handling": lazy(() => import("./DataHandling")),
+  survey: lazy(() => import("./Survey")),
+  "create-workspace": lazy(() => import("./CreateWorkspace")),
+};
+
+export default OnboardingSteps;
+
+export function OnboardingLayout({ children }) {
+  const [header, setHeader] = useState({
+    title: "",
+    description: "",
+  });
+  const [backBtn, setBackBtn] = useState({
+    showing: false,
+    disabled: true,
+    onClick: () => null,
+  });
+  const [forwardBtn, setForwardBtn] = useState({
+    showing: false,
+    disabled: true,
+    onClick: () => null,
+  });
+
+  if (isMobile) {
+    return (
+      <div className="w-screen h-screen overflow-y-auto bg-[#2C2F35] overflow-hidden">
+        <div className="flex flex-col">
+          <div className="w-full relative py-10 px-2">
+            <div className="flex flex-col w-fit mx-auto gap-y-1 mb-[55px]">
+              <h1 className="text-white font-semibold text-center text-2xl">
+                {header.title}
+              </h1>
+              <p className="text-zinc-400 text-base text-center">
+                {header.description}
+              </p>
+            </div>
+            {children(setHeader, setBackBtn, setForwardBtn)}
+          </div>
+          <div className="flex w-full justify-center gap-x-4 pb-20">
+            <div className="flex justify-center items-center">
+              {backBtn.showing && (
+                <button
+                  disabled={backBtn.disabled}
+                  onClick={backBtn.onClick}
+                  className="group p-2 rounded-lg border-2 border-zinc-300 disabled:border-zinc-600 h-fit w-fit disabled:not-allowed hover:bg-zinc-100 disabled:hover:bg-transparent"
+                >
+                  <ArrowLeft
+                    className="text-white group-hover:text-black group-disabled:text-gray-500"
+                    size={30}
+                  />
+                </button>
+              )}
+            </div>
+
+            <div className="flex justify-center items-center">
+              {forwardBtn.showing && (
+                <button
+                  disabled={forwardBtn.disabled}
+                  onClick={forwardBtn.onClick}
+                  className="group p-2 rounded-lg border-2 border-zinc-300 disabled:border-zinc-600 h-fit w-fit disabled:not-allowed hover:bg-zinc-100 disabled:hover:bg-transparent"
+                >
+                  <ArrowRight
+                    className="text-white group-hover:text-black group-disabled:text-gray-500"
+                    size={30}
+                  />
+                </button>
+              )}
+            </div>
+          </div>
+        </div>
+      </div>
+    );
+  }
+
+  return (
+    <div className="w-screen overflow-y-auto bg-[#2C2F35] md:bg-main-gradient flex justify-center overflow-hidden">
+      <div className="flex w-1/5 h-screen justify-center items-center">
+        {backBtn.showing && (
+          <button
+            disabled={backBtn.disabled}
+            onClick={backBtn.onClick}
+            className="group p-2 rounded-lg border-2 border-zinc-300 disabled:border-zinc-600 h-fit w-fit disabled:not-allowed hover:bg-zinc-100 disabled:hover:bg-transparent"
+          >
+            <ArrowLeft
+              className="text-white group-hover:text-black group-disabled:text-gray-500"
+              size={30}
+            />
+          </button>
+        )}
+      </div>
+
+      <div className="w-full md:w-3/5 relative h-full py-10">
+        <div className="flex flex-col w-fit mx-auto gap-y-1 mb-[55px]">
+          <h1 className="text-white font-semibold text-center text-2xl">
+            {header.title}
+          </h1>
+          <p className="text-zinc-400 text-base text-center">
+            {header.description}
+          </p>
+        </div>
+        {children(setHeader, setBackBtn, setForwardBtn)}
+      </div>
+
+      <div className="flex w-1/5 h-screen justify-center items-center">
+        {forwardBtn.showing && (
+          <button
+            disabled={forwardBtn.disabled}
+            onClick={forwardBtn.onClick}
+            className="group p-2 rounded-lg border-2 border-zinc-300 disabled:border-zinc-600 h-fit w-fit disabled:not-allowed hover:bg-zinc-100 disabled:hover:bg-transparent"
+          >
+            <ArrowRight
+              className="text-white group-hover:text-black group-disabled:text-gray-500"
+              size={30}
+            />
+          </button>
+        )}
+      </div>
+    </div>
+  );
+}

frontend/src/pages/OnboardingFlow/index.jsx

@@ -1,57 +1,21 @@
-import React, { useEffect, useState } from "react";
+import React from "react";
-import OnboardingModal, { OnboardingModalId } from "./OnboardingModal";
+import OnboardingSteps, { OnboardingLayout } from "./Steps";
-import useLogo from "@/hooks/useLogo";
+import { useParams } from "react-router-dom";
-import { isMobile } from "react-device-detect";
 
 export default function OnboardingFlow() {
-  const { logo } = useLogo();
+  const { step } = useParams();
-  const [modalVisible, setModalVisible] = useState(false);
+  const StepPage = OnboardingSteps[step || "home"];
-
+  if (step === "home" || !step) return <StepPage />;
-  useEffect(() => {
-    if (modalVisible) {
-      document.getElementById(OnboardingModalId)?.showModal();
-    }
-  }, [modalVisible]);
-
-  function showModal() {
-    setModalVisible(true);
-  }
-
-  if (isMobile) {
-    return (
-      <div className="w-screen h-full bg-sidebar flex items-center justify-center">
-        <div className="w-fit p-20 py-24 border-2 border-slate-300/10 rounded-2xl bg-main-gradient shadow-lg">
-          <div className="text-white text-2xl font-base text-center">
-            Welcome to
-          </div>
-          <img src={logo} alt="logo" className="w-80 mx-auto m-3 mb-11" />
-          <div className="flex justify-center items-center">
-            <p className="text-white text-sm italic text-center">
-              Please use a desktop browser to continue onboarding.
-            </p>
-          </div>
-        </div>
-      </div>
-    );
-  }
 
   return (
-    <div className="w-screen h-full bg-sidebar flex items-center justify-center">
+    <OnboardingLayout>
-      <div className="w-fit p-20 py-24 border-2 border-slate-300/10 rounded-2xl bg-main-gradient shadow-lg">
+      {(setHeader, setBackBtn, setForwardBtn) => (
-        <div className="text-white text-2xl font-base text-center">
+        <StepPage
-          Welcome to
+          setHeader={setHeader}
-        </div>
+          setBackBtn={setBackBtn}
-        <img src={logo} alt="logo" className="w-80 mx-auto m-3 mb-11" />
+          setForwardBtn={setForwardBtn}
-        <div className="flex justify-center items-center">
+        />
-          <button
+      )}
-            className="border border-slate-200 px-5 py-2.5 rounded-lg text-slate-800 bg-slate-200 text-sm items-center flex gap-x-2 hover:text-white hover:bg-transparent focus:ring-gray-800 font-semibold shadow animate-pulse"
+    </OnboardingLayout>
-            onClick={showModal}
-          >
-            Get Started
-          </button>
-        </div>
-      </div>
-      {modalVisible && <OnboardingModal setModalVisible={setModalVisible} />}
-    </div>
   );
 }

frontend/src/utils/paths.js

@@ -7,8 +7,34 @@ export default {
   login: () => {
     return "/login";
   },
-  onboarding: () => {
+  onboarding: {
-    return "/onboarding";
+    home: () => {
+      return "/onboarding";
+    },
+    survey: () => {
+      return "/onboarding/survey";
+    },
+    llmPreference: () => {
+      return "/onboarding/llm-preference";
+    },
+    embeddingPreference: () => {
+      return "/onboarding/embedding-preference";
+    },
+    vectorDatabase: () => {
+      return "/onboarding/vector-database";
+    },
+    customLogo: () => {
+      return "/onboarding/custom-logo";
+    },
+    userSetup: () => {
+      return "/onboarding/user-setup";
+    },
+    dataHandling: () => {
+      return "/onboarding/data-handling";
+    },
+    createWorkspace: () => {
+      return "/onboarding/create-workspace";
+    },
   },
   github: () => {
     return "https://github.com/Mintplex-Labs/anything-llm";
@@ -25,9 +51,6 @@ export default {
   hosting: () => {
     return "https://my.mintplexlabs.com/aio-checkout?product=anythingllm";
   },
-  feedback: () => {
-    return "https://mintplexlabs.typeform.com/to/i0KE3aEW";
-  },
   workspace: {
     chat: (slug) => {
       return `/workspace/${slug}`;

frontend/tailwind.config.js

@@ -1,54 +1,88 @@
 /** @type {import('tailwindcss').Config} */
 export default {
-  content: ["./src/**/*.{js,jsx}"],
+  content: {
+    relative: true,
+    files: [
+      "./src/components/**/*.{js,jsx}",
+      "./src/hooks/**/*.js",
+      "./src/models/**/*.js",
+      "./src/pages/**/*.{js,jsx}",
+      "./src/utils/**/*.js",
+      "./src/*.jsx",
+      "./index.html"
+    ]
+  },
   theme: {
     extend: {
       rotate: {
-        '270': '270deg',
+        "270": "270deg",
-        '360': '360deg',
+        "360": "360deg"
       },
       colors: {
-        'black-900': '#141414',
+        "black-900": "#141414",
-        'accent': '#3D4147',
+        accent: "#3D4147",
-        'sidebar-button': '#31353A',
+        "sidebar-button": "#31353A",
-        'sidebar': '#25272C',
+        sidebar: "#25272C",
-        'historical-msg-system': 'rgba(255, 255, 255, 0.05);',
+        "historical-msg-system": "rgba(255, 255, 255, 0.05);",
-        'historical-msg-user': '#2C2F35',
+        "historical-msg-user": "#2C2F35"
       },
       backgroundImage: {
-        'preference-gradient': 'linear-gradient(180deg, #5A5C63 0%, rgba(90, 92, 99, 0.28) 100%);',
+        "preference-gradient":
-        'chat-msg-user-gradient': 'linear-gradient(180deg, #3D4147 0%, #2C2F35 100%);',
+          "linear-gradient(180deg, #5A5C63 0%, rgba(90, 92, 99, 0.28) 100%);",
-        'selected-preference-gradient': 'linear-gradient(180deg, #313236 0%, rgba(63.40, 64.90, 70.13, 0) 100%);',
+        "chat-msg-user-gradient":
-        'main-gradient': 'linear-gradient(180deg, #3D4147 0%, #2C2F35 100%)',
+          "linear-gradient(180deg, #3D4147 0%, #2C2F35 100%);",
-        'modal-gradient': 'linear-gradient(180deg, #3D4147 0%, #2C2F35 100%)',
+        "selected-preference-gradient":
-        'sidebar-gradient': 'linear-gradient(90deg, #5B616A 0%, #3F434B 100%)',
+          "linear-gradient(180deg, #313236 0%, rgba(63.40, 64.90, 70.13, 0) 100%);",
-        'menu-item-gradient': 'linear-gradient(90deg, #3D4147 0%, #2C2F35 100%)',
+        "main-gradient": "linear-gradient(180deg, #3D4147 0%, #2C2F35 100%)",
-        'menu-item-selected-gradient': 'linear-gradient(90deg, #5B616A 0%, #3F434B 100%)',
+        "modal-gradient": "linear-gradient(180deg, #3D4147 0%, #2C2F35 100%)",
-        'workspace-item-gradient': 'linear-gradient(90deg, #3D4147 0%, #2C2F35 100%)',
+        "sidebar-gradient": "linear-gradient(90deg, #5B616A 0%, #3F434B 100%)",
-        'workspace-item-selected-gradient': 'linear-gradient(90deg, #5B616A 0%, #3F434B 100%)',
+        "menu-item-gradient":
-        'switch-selected': 'linear-gradient(146deg, #5B616A 0%, #3F434B 100%)',
+          "linear-gradient(90deg, #3D4147 0%, #2C2F35 100%)",
+        "menu-item-selected-gradient":
+          "linear-gradient(90deg, #5B616A 0%, #3F434B 100%)",
+        "workspace-item-gradient":
+          "linear-gradient(90deg, #3D4147 0%, #2C2F35 100%)",
+        "workspace-item-selected-gradient":
+          "linear-gradient(90deg, #5B616A 0%, #3F434B 100%)",
+        "switch-selected": "linear-gradient(146deg, #5B616A 0%, #3F434B 100%)"
       },
       fontFamily: {
-        'sans': ['plus-jakarta-sans', 'ui-sans-serif', 'system-ui', '-apple-system', 'BlinkMacSystemFont', '"Segoe UI"', 'Roboto', '"Helvetica Neue"', 'Arial', '"Noto Sans"', 'sans-serif', '"Apple Color Emoji"', '"Segoe UI Emoji"', '"Segoe UI Symbol"', '"Noto Color Emoji"'],
+        sans: [
+          "plus-jakarta-sans",
+          "ui-sans-serif",
+          "system-ui",
+          "-apple-system",
+          "BlinkMacSystemFont",
+          '"Segoe UI"',
+          "Roboto",
+          '"Helvetica Neue"',
+          "Arial",
+          '"Noto Sans"',
+          "sans-serif",
+          '"Apple Color Emoji"',
+          '"Segoe UI Emoji"',
+          '"Segoe UI Symbol"',
+          '"Noto Color Emoji"'
+        ]
       },
       animation: {
-        sweep: 'sweep 0.5s ease-in-out',
+        sweep: "sweep 0.5s ease-in-out"
       },
       keyframes: {
         sweep: {
-          '0%': { transform: 'scaleX(0)', transformOrigin: 'bottom left' },
+          "0%": { transform: "scaleX(0)", transformOrigin: "bottom left" },
-          '100%': { transform: 'scaleX(1)', transformOrigin: 'bottom left' },
+          "100%": { transform: "scaleX(1)", transformOrigin: "bottom left" }
         },
         fadeIn: {
-          '0%': { opacity: 0 },
+          "0%": { opacity: 0 },
-          '100%': { opacity: 1 },
+          "100%": { opacity: 1 }
         },
         fadeOut: {
-          '0%': { opacity: 1 },
+          "0%": { opacity: 1 },
-          '100%': { opacity: 0 },
+          "100%": { opacity: 0 }
-        },
+        }
       }
-    },
+    }
   },
-  plugins: [],
+  plugins: []
 }

frontend/vite.config.js

@@ -1,23 +1,23 @@
-import { defineConfig } from 'vite'
+import { defineConfig } from "vite"
-import { fileURLToPath, URL } from "url";
+import { fileURLToPath, URL } from "url"
-import postcss from './postcss.config.js'
+import postcss from "./postcss.config.js"
-import react from '@vitejs/plugin-react'
+import react from "@vitejs/plugin-react"
-import dns from 'dns'
+import dns from "dns"
-import { visualizer } from "rollup-plugin-visualizer";
+import { visualizer } from "rollup-plugin-visualizer"
 
-dns.setDefaultResultOrder('verbatim')
+dns.setDefaultResultOrder("verbatim")
 
 // https://vitejs.dev/config/
 export default defineConfig({
   server: {
     port: 3000,
-    host: 'localhost'
+    host: "localhost"
   },
   define: {
-    'process.env': process.env
+    "process.env": process.env
   },
   css: {
-    postcss,
+    postcss
   },
   plugins: [
     react(),
@@ -26,12 +26,15 @@ export default defineConfig({
       open: false,
       gzipSize: true,
       brotliSize: true,
-      filename: "bundleinspector.html", // will be saved in project's root
+      filename: "bundleinspector.html" // will be saved in project's root
-    }),
+    })
   ],
   resolve: {
     alias: [
-      { find: '@', replacement: fileURLToPath(new URL('./src', import.meta.url)) },
+      {
+        find: "@",
+        replacement: fileURLToPath(new URL("./src", import.meta.url))
+      },
       {
         process: "process/browser",
         stream: "stream-browserify",
@@ -39,23 +42,22 @@ export default defineConfig({
         util: "util",
         find: /^~.+/,
         replacement: (val) => {
-          return val.replace(/^~/, "");
+          return val.replace(/^~/, "")
-        },
+        }
-      },
+      }
-    ],
+    ]
   },
   build: {
     commonjsOptions: {
-      transformMixedEsModules: true,
+      transformMixedEsModules: true
     }
   },
   optimizeDeps: {
     esbuildOptions: {
       define: {
-        global: 'globalThis'
+        global: "globalThis"
       },
-      plugins: [
+      plugins: []
-      ]
     }
   }
 })

package.json

@@ -3,6 +3,7 @@
   "version": "0.2.0",
   "description": "The best solution for turning private documents into a chat bot using off-the-shelf tools and commercially viable AI technologies.",
   "main": "index.js",
+  "type": "module",
   "author": "Timothy Carambat (Mintplex Labs)",
   "license": "MIT",
   "engines": {
@@ -14,7 +15,7 @@
     "setup:envs": "cp -n ./frontend/.env.example ./frontend/.env && cp -n ./server/.env.example ./server/.env.development && cp -n ./collector/.env.example ./collector/.env && cp -n ./docker/.env.example ./docker/.env && echo \"All ENV files copied!\n\"",
     "dev:server": "cd server && yarn dev",
     "dev:collector": "cd collector && yarn dev",
-    "dev:frontend": "cd frontend && yarn start",
+    "dev:frontend": "cd frontend && yarn dev",
     "prisma:generate": "cd server && npx prisma generate",
     "prisma:migrate": "cd server && npx prisma migrate dev --name init",
     "prisma:seed": "cd server && npx prisma db seed",
@@ -25,4 +26,4 @@
     "generate::gcp_deployment": "node cloud-deployments/gcp/deployment/generate.mjs"
   },
   "private": false
-}
+}

server/.env.example

@@ -1,5 +1,4 @@
 SERVER_PORT=3001
-CACHE_VECTORS="true"
 JWT_SECRET="my-random-string-for-seeding" # Please generate random string at least 12 chars long.
 
 ###########################################
@@ -9,6 +8,10 @@ JWT_SECRET="my-random-string-for-seeding" # Please generate random string at lea
 # OPEN_AI_KEY=
 # OPEN_MODEL_PREF='gpt-3.5-turbo'
 
+# LLM_PROVIDER='gemini'
+# GEMINI_API_KEY=
+# GEMINI_LLM_MODEL_PREF='gemini-pro'
+
 # LLM_PROVIDER='azure'
 # AZURE_OPENAI_ENDPOINT=
 # AZURE_OPENAI_KEY=
@@ -29,6 +32,11 @@ JWT_SECRET="my-random-string-for-seeding" # Please generate random string at lea
 # LOCAL_AI_MODEL_TOKEN_LIMIT=4096
 # LOCAL_AI_API_KEY="sk-123abc"
 
+# LLM_PROVIDER='ollama'
+# OLLAMA_BASE_PATH='http://host.docker.internal:11434'
+# OLLAMA_MODEL_PREF='llama2'
+# OLLAMA_MODEL_TOKEN_LIMIT=4096
+
 ###########################################
 ######## Embedding API SElECTION ##########
 ###########################################
@@ -42,7 +50,7 @@ JWT_SECRET="my-random-string-for-seeding" # Please generate random string at lea
 # EMBEDDING_MODEL_PREF='my-embedder-model' # This is the "deployment" on Azure you want to use for embeddings. Not the base model. Valid base model is text-embedding-ada-002
 
 # EMBEDDING_ENGINE='localai'
-# EMBEDDING_BASE_PATH='https://localhost:8080/v1'
+# EMBEDDING_BASE_PATH='http://localhost:8080/v1'
 # EMBEDDING_MODEL_PREF='text-embedding-ada-002'
 # EMBEDDING_MODEL_MAX_CHUNK_LENGTH=1000 # The max chunk size in chars a string to embed can be
 
@@ -91,3 +99,12 @@ VECTOR_DB="lancedb"
 #PASSWORDNUMERIC=1
 #PASSWORDSYMBOL=1
 #PASSWORDREQUIREMENTS=4
+
+###########################################
+######## ENABLE HTTPS SERVER ##############
+###########################################
+# By enabling this and providing the path/filename for the key and cert,
+# the server will use HTTPS instead of HTTP.
+#ENABLE_HTTPS="true"
+#HTTPS_CERT_PATH="sslcert/cert.pem"
+#HTTPS_KEY_PATH="sslcert/key.pem"

server/.flowconfig

@@ -0,0 +1,30 @@
+# How to config: https://flow.org/en/docs/config/
+[version]
+
+[options]
+all=false
+emoji=false
+include_warnings=false
+lazy_mode=false
+
+[include]
+
+[ignore]
+.*/node_modules/resolve/test/.*
+
+[untyped]
+# <PROJECT_ROOT>/src/models/.*
+
+[declarations]
+
+[libs]
+
+[lints]
+all=warn
+
+[strict]
+nonstrict-import
+unclear-type
+unsafe-getters-setters
+untyped-import
+untyped-type-import

server/endpoints/api/admin/index.js

@@ -523,16 +523,9 @@ function apiAdminEndpoints(app) {
       schema: {
         "$ref": "#/definitions/InvalidAPIKey"
       }
-    }
-     #swagger.responses[401] = {
-      description: "Instance is not in Multi-User mode. Method denied",
     }
     */
       try {
-        if (!multiUserMode(response)) {
-          response.sendStatus(401).end();
-          return;
-        }
         const pgSize = 20;
         const { offset = 0 } = reqBody(request);
         const chats = await WorkspaceChats.whereWithData(

server/endpoints/system.js

@@ -797,7 +797,7 @@ function systemEndpoints(app) {
       try {
         const { id } = request.params;
         await WorkspaceChats.delete({ id: Number(id) });
-        response.status(200).json({ success, error });
+        response.sendStatus(200).end();
       } catch (e) {
         console.error(e);
         response.sendStatus(500).end();

server/index.js

@@ -19,10 +19,9 @@ const { getVectorDbClass } = require("./utils/helpers");
 const { adminEndpoints } = require("./endpoints/admin");
 const { inviteEndpoints } = require("./endpoints/invite");
 const { utilEndpoints } = require("./endpoints/utils");
-const { Telemetry } = require("./models/telemetry");
 const { developerEndpoints } = require("./endpoints/api");
-const setupTelemetry = require("./utils/telemetry");
 const { extensionEndpoints } = require("./endpoints/extensions");
+const { bootHTTP, bootSSL } = require("./utils/boot");
 const app = express();
 const apiRouter = express.Router();
 const FILE_LIMIT = "3GB";
@@ -99,20 +98,8 @@ app.all("*", function (_, response) {
   response.sendStatus(404);
 });
 
-app
+if (!!process.env.ENABLE_HTTPS) {
-  .listen(process.env.SERVER_PORT || 3001, async () => {
+  bootSSL(app, process.env.SERVER_PORT || 3001);
-    await setupTelemetry();
+} else {
-    console.log(
+  bootHTTP(app, process.env.SERVER_PORT || 3001);
-      `Primary server listening on port ${process.env.SERVER_PORT || 3001}`
+}
-    );
-  })
-  .on("error", function (err) {
-    process.once("SIGUSR2", function () {
-      Telemetry.flush();
-      process.kill(process.pid, "SIGUSR2");
-    });
-    process.on("SIGINT", function () {
-      Telemetry.flush();
-      process.kill(process.pid, "SIGINT");
-    });
-  });

server/jsconfig.json

@@ -0,0 +1,14 @@
+{
+  "compilerOptions": {
+    "module": "commonjs",
+    "target": "ES2020"
+  },
+  "include": [
+    "./endpoints/**/*",
+    "./models/**/*",
+    "./utils/**/*",
+    "./swagger/**/*",
+    "index.js"
+  ],
+  "exclude": ["node_modules", "storage"]
+}

server/models/systemSettings.js

@@ -91,6 +91,20 @@ const SystemSettings = {
           }
         : {}),
 
+      ...(llmProvider === "gemini"
+        ? {
+            GeminiLLMApiKey: !!process.env.GEMINI_API_KEY,
+            GeminiLLMModelPref:
+              process.env.GEMINI_LLM_MODEL_PREF || "gemini-pro",
+
+            // For embedding credentials when Gemini is selected.
+            OpenAiKey: !!process.env.OPEN_AI_KEY,
+            AzureOpenAiEndpoint: process.env.AZURE_OPENAI_ENDPOINT,
+            AzureOpenAiKey: !!process.env.AZURE_OPENAI_KEY,
+            AzureOpenAiEmbeddingModelPref: process.env.EMBEDDING_MODEL_PREF,
+          }
+        : {}),
+
       ...(llmProvider === "lmstudio"
         ? {
             LMStudioBasePath: process.env.LMSTUDIO_BASE_PATH,
@@ -116,6 +130,20 @@ const SystemSettings = {
             AzureOpenAiEmbeddingModelPref: process.env.EMBEDDING_MODEL_PREF,
           }
         : {}),
+
+      ...(llmProvider === "ollama"
+        ? {
+            OllamaLLMBasePath: process.env.OLLAMA_BASE_PATH,
+            OllamaLLMModelPref: process.env.OLLAMA_MODEL_PREF,
+            OllamaLLMTokenLimit: process.env.OLLAMA_MODEL_TOKEN_LIMIT,
+
+            // For embedding credentials when ollama is selected.
+            OpenAiKey: !!process.env.OPEN_AI_KEY,
+            AzureOpenAiEndpoint: process.env.AZURE_OPENAI_ENDPOINT,
+            AzureOpenAiKey: !!process.env.AZURE_OPENAI_KEY,
+            AzureOpenAiEmbeddingModelPref: process.env.EMBEDDING_MODEL_PREF,
+          }
+        : {}),
       ...(llmProvider === "native"
         ? {
             NativeLLMModelPref: process.env.NATIVE_LLM_MODEL_PREF,