diff --git a/.devcontainer/README.md b/.devcontainer/README.md new file mode 100644 index 000000000..c81b72f6e --- /dev/null +++ b/.devcontainer/README.md @@ -0,0 +1,72 @@ +# AnythingLLM Development Container Setup + +Welcome to the AnythingLLM development container configuration, designed to create a seamless and feature-rich development environment for this project. + +

PLEASE READ THIS

+ +## Prerequisites + +- [Docker](https://www.docker.com/get-started) +- [Visual Studio Code](https://code.visualstudio.com/) +- [Remote - Containers](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-containers) VS Code extension + +## Features + +- **Base Image**: Built on `mcr.microsoft.com/devcontainers/javascript-node:1-18-bookworm`, thus Node.JS LTS v18. +- **Additional Tools**: Includes `hadolint`, and essential apt-packages such as `curl`, `gnupg`, and more. +- **Ports**: Configured to auto-forward ports `3000` (Frontend) and `3001` (Backend). +- **Environment Variables**: Sets `NODE_ENV` to `development` and `ESLINT_USE_FLAT_CONFIG` to `true`. +- **VS Code Extensions**: A suite of extensions such as `Prettier`, `Docker`, `ESLint`, and more are automatically installed. Please revise if you do not agree with any of these extensions. AI-powered extensions and time trackers are (for now) not included to avoid any privacy concerns, but you can install them later in your own environment. + +## Getting Started + +1. Using Github Codepaces. Just select to create a new workspace, and the devcontainer will be created for you. + +2. Using your Local VSCode (Release or Insiders). We suggest you first make a fork of the repo and then clone it to your local machine using VSCode tools. Then open the project folder in VSCode, which will prompt you to open the project in a devcontainer. Select yes, and the devcontainer will be created for you. If this does not happen, you can open the command palette and select "Remote-Containers: Reopen in Container". + +## On Creation: + +When the container is built for the first time, it will automatically run `yarn setup` to ensure everything is in place for the Collector, Server and Frontend. This command is expected to be automatically re-run if there is a content change on next reboot. + +## Work in the Container: + +Once the container is up, be patient. Some extensions may complain because dependencies are still being installed, and in the Extensions tab, some may ask you to "Reload" the project. Don't do that yet. First, wait until all settle down for the first time. We suggest you create a new VSCode profile for this devcontainer, so any configuration and extensions you change, won't affect your default profile. + +Checklist: + +- [ ] The usual message asking you to start the Server and Frontend in different windows are now "hidden" in the building process of the devcontainer. Don't forget to do as suggested. +- [ ] Open a JavaScript file, for example "server/index.js" and check if `eslint` is working. It will complain that `'err' is defined but never used.`. This means it is working. +- [ ] Open a React File, for example, "frontend/src/main.jsx," and check if `eslint` complains about `Fast refresh only works when a file has exports. Move your component(s) to a separate file.`. Again, it means `eslint` is working. Now check at the status bar if the `Prettier` has a double checkmark :heavy_check_mark: (double). It means Prettier is working. You will see a nice extension `Formatting:`:heavy_check_mark: that can be used to disable the `Format on Save` feature temporarily. +- [ ] Check if, on the left pane, you have the NPM Scripts (this may be disabled; look at the "Explorer" tree-dots up-right). There will be scripts inside the `package.json` files. You will basically need to run the `dev:collector`, `dev:server` and the `dev:frontend` in this order. When the frontend finishes starting, a window browser will open **inside** the VSCode. Still, you can open it outside. + +:warning: **Important for all developers** :warning: + +- [ ] Whe you are using the `NODE_ENV=development` the server will not store the configurations you set for security reasons. Please set the proper config on file `.env.development`. The side-effect if you don't, everytime you restart the server, you will be sent to the "Onboarding" page again. + +:warning: **Important for Github Codespaces** :warning: + +- [ ] When running the "Server" for the first time, its port will be automatically forward, but privately. Read the content of the `.env` file on the frontend folder about this, and make sure the port "Visibility" is set to "Public", so the frontend can reach the backend. Again, this is only needed for developing on Github Codespaces. We appreciate to know if you have a better solution. + +**For the Collector:** + +- [x] In the past, the Collector dwelled within the Python domain, but now it has journeyed to the splendid realm of Node.JS. Consequently, the configuration complexities of bygone versions are no longer a concern. + +### Now it is ready to start + +In the status bar you will see three shortcuts names `Collector`, `Server` and `Frontend`. Just click-and-wait on that order (don't forget to set the Server port 3001 to Public if you are using GH Codespaces **_before_** starting the Frontend). + +Now you can enjoy your time developing instead of reconfiguring everything. + +## Debugging with the devcontainers + +### For debugging the collector, server and frontend + +First, make sure the built-in extension (ms-vscode.js-debug) is active (I don't know why it would not be, but just in case). If you want, you can install the nightly version (ms-vscode.js-debug-nightly) + +Then, in the "Run and Debug" tab (Ctrl+shift+D), you can select on the menu: + +- Collector debug. This will start the collector in debug mode and attach the debugger. Works very well. +- Server debug. This will start the server in debug mode and attach the debugger. Works very well. +- Frontend debug. This will start the frontend in debug mode and attach the debugger. I am still struggling with this one. I don't know if VSCode can handle the .jsx files seamlessly as the pure .js on the server. Maybe there is a need for a particular configuration for Vite or React. Anyway, it starts. Another two configurations launch Chrome and Edge, and I think we could add breakpoints on .jsx files somehow. The best scenario would be always to use the embedded browser. WIP. + +Please leave comments on the Issues tab or the [![](https://img.shields.io/discord/1114740394715004990?logo=Discord&logoColor=white&label=Discord&labelColor=%235568ee&color=%2355A2DD&link=https%3A%2F%2Fdiscord.gg%2F6UyHPeGZAC)]("https://discord.gg/6UyHPeGZAC") diff --git a/.devcontainer/devcontainer.json b/.devcontainer/devcontainer.json new file mode 100644 index 000000000..83792da78 --- /dev/null +++ b/.devcontainer/devcontainer.json @@ -0,0 +1,211 @@ +// For format details, see https://aka.ms/devcontainer.json. For config options, see the +// README at: https://github.com/devcontainers/templates/tree/main/src/javascript-node +{ + "name": "Node.js", + // Or use a Dockerfile or Docker Compose file. More info: https://containers.dev/guide/dockerfile + // "build": { + // "args": { + // "ARG_UID": "1000", + // "ARG_GID": "1000" + // }, + // "dockerfile": "Dockerfile" + // }, + // "containerUser": "anythingllm", + // Or use a Dockerfile or Docker Compose file. More info: https://containers.dev/guide/dockerfile + "image": "mcr.microsoft.com/devcontainers/javascript-node:1-18-bookworm", + // Features to add to the dev container. More info: https://containers.dev/features. + "features": { + // Docker very useful linter + "ghcr.io/dhoeric/features/hadolint:1": { + "version": "latest" + }, + // Terraform support + "ghcr.io/devcontainers/features/terraform:1": {}, + // Just a wrap to install needed packages + "ghcr.io/rocker-org/devcontainer-features/apt-packages:1": { + // Dependencies copied from ../docker/Dockerfile plus some dev stuff + "packages": [ + "build-essential", + "ca-certificates", + "curl", + "ffmpeg", + "fonts-liberation", + "git", + "gnupg", + "htop", + "less", + "libappindicator1", + "libasound2", + "libatk-bridge2.0-0", + "libatk1.0-0", + "libc6", + "libcairo2", + "libcups2", + "libdbus-1-3", + "libexpat1", + "libfontconfig1", + "libgbm1", + "libgcc1", + "libgfortran5", + "libglib2.0-0", + "libgtk-3-0", + "libnspr4", + "libnss3", + "libpango-1.0-0", + "libpangocairo-1.0-0", + "libstdc++6", + "libx11-6", + "libx11-xcb1", + "libxcb1", + "libxcomposite1", + "libxcursor1", + "libxdamage1", + "libxext6", + "libxfixes3", + "libxi6", + "libxrandr2", + "libxrender1", + "libxss1", + "libxtst6", + "locales", + "lsb-release", + "procps", + "tzdata", + "wget", + "xdg-utils" + ] + } + }, + "updateContentCommand": "yarn setup", + // Use 'postCreateCommand' to run commands after the container is created. + // This configures VITE for github codespaces + "postCreateCommand": "if [ \"${CODESPACES}\" = \"true\" ]; then echo 'VITE_API_BASE=\"https://$CODESPACE_NAME-3001.$GITHUB_CODESPACES_PORT_FORWARDING_DOMAIN/api\"' > ./frontend/.env; fi", + "portsAttributes": { + "3001": { + "label": "Backend", + "onAutoForward": "notify" + }, + "3000": { + "label": "Frontend", + "onAutoForward": "openPreview" + } + }, + "capAdd": [ + "SYS_ADMIN" // needed for puppeteer using headless chrome in sandbox + ], + "remoteEnv": { + "NODE_ENV": "development", + "ESLINT_USE_FLAT_CONFIG": "true", + "ANYTHING_LLM_RUNTIME": "docker" + }, + // "initializeCommand": "echo Initialize....", + "shutdownAction": "stopContainer", + // Configure tool-specific properties. + "customizations": { + "codespaces": { + "openFiles": [ + "README.md", + ".devcontainer/README.md" + ] + }, + "vscode": { + "openFiles": [ + "README.md", + ".devcontainer/README.md" + ], + "extensions": [ + "bierner.github-markdown-preview", + "bradlc.vscode-tailwindcss", + "dbaeumer.vscode-eslint", + "editorconfig.editorconfig", + "esbenp.prettier-vscode", + "exiasr.hadolint", + "flowtype.flow-for-vscode", + "gamunu.vscode-yarn", + "hashicorp.terraform", + "mariusschulz.yarn-lock-syntax", + "ms-azuretools.vscode-docker", + "streetsidesoftware.code-spell-checker", + "actboy168.tasks", + "tombonnike.vscode-status-bar-format-toggle", + "ms-vscode.js-debug" + ], + "settings": { + "[css]": { + "editor.defaultFormatter": "esbenp.prettier-vscode" + }, + "[dockercompose]": { + "editor.defaultFormatter": "esbenp.prettier-vscode" + }, + "[dockerfile]": { + "editor.defaultFormatter": "ms-azuretools.vscode-docker" + }, + "[html]": { + "editor.defaultFormatter": "esbenp.prettier-vscode" + }, + "[javascript]": { + "editor.defaultFormatter": "esbenp.prettier-vscode" + }, + "[javascriptreact]": { + "editor.defaultFormatter": "esbenp.prettier-vscode" + }, + "[json]": { + "editor.defaultFormatter": "esbenp.prettier-vscode" + }, + "[jsonc]": { + "editor.defaultFormatter": "esbenp.prettier-vscode" + }, + "[markdown]": { + "editor.defaultFormatter": "esbenp.prettier-vscode" + }, + "[postcss]": { + "editor.defaultFormatter": "esbenp.prettier-vscode" + }, + "[toml]": { + "editor.defaultFormatter": "tamasfe.even-better-toml" + }, + "eslint.debug": true, + "eslint.enable": true, + "eslint.experimental.useFlatConfig": true, + "eslint.run": "onSave", + "files.associations": { + ".*ignore": "ignore", + ".editorconfig": "editorconfig", + ".env*": "properties", + ".flowconfig": "ini", + ".prettierrc": "json", + "*.css": "tailwindcss", + "*.md": "markdown", + "*.sh": "shellscript", + "docker-compose.*": "dockercompose", + "Dockerfile*": "dockerfile", + "yarn.lock": "yarnlock" + }, + "javascript.format.enable": false, + "javascript.inlayHints.enumMemberValues.enabled": true, + "javascript.inlayHints.functionLikeReturnTypes.enabled": true, + "javascript.inlayHints.parameterTypes.enabled": true, + "javascript.inlayHints.variableTypes.enabled": true, + "js/ts.implicitProjectConfig.module": "CommonJS", + "json.format.enable": false, + "json.schemaDownload.enable": true, + "npm.autoDetect": "on", + "npm.packageManager": "yarn", + "prettier.useEditorConfig": false, + "tailwindCSS.files.exclude": [ + "**/.git/**", + "**/node_modules/**", + "**/.hg/**", + "**/.svn/**", + "**/dist/**" + ], + "typescript.validate.enable": false, + "workbench.editorAssociations": { + "*.md": "vscode.markdown.preview.editor" + } + } + } + } + // Uncomment to connect as root instead. More info: https://aka.ms/dev-containers-non-root. + // "remoteUser": "root" +} diff --git a/.dockerignore b/.dockerignore index 1c919b282..32582ced3 100644 --- a/.dockerignore +++ b/.dockerignore @@ -1,10 +1,10 @@ -server/storage/documents/** -server/storage/vector-cache/** -server/storage/*.db -server/storage/lancedb -collector/hotdir/** -collector/v-env/** -collector/outputs/** +**/server/storage/documents/** +**/server/storage/vector-cache/** +**/server/storage/*.db +**/server/storage/lancedb +**/collector/hotdir/** +**/collector/v-env/** +**/collector/outputs/** **/node_modules/ **/dist/ **/v-env/ @@ -13,4 +13,5 @@ collector/outputs/** **/.env.* **/bundleinspector.html !docker/.env.example -!frontend/.env.production \ No newline at end of file +!frontend/.env.production +**/tmp/** diff --git a/.editorconfig b/.editorconfig index 5d47c21c4..e175db54f 100644 --- a/.editorconfig +++ b/.editorconfig @@ -4,9 +4,14 @@ root = true [*] +# Non-configurable Prettier behaviors +charset = utf-8 +insert_final_newline = true +trim_trailing_whitespace = true + +# Configurable Prettier behaviors +# (change these if your Prettier config differs) +end_of_line = lf indent_style = space indent_size = 2 -end_of_line = lf -charset = utf-8 -trim_trailing_whitespace = true -insert_final_newline = true +max_line_length = 80 diff --git a/.github/FUNDING.yml b/.github/FUNDING.yml new file mode 100644 index 000000000..411f850fd --- /dev/null +++ b/.github/FUNDING.yml @@ -0,0 +1 @@ +github: Mintplex-Labs \ No newline at end of file diff --git a/.github/workflows/build-and-push-image.yaml b/.github/workflows/build-and-push-image.yaml index 943af5526..b8ac6348f 100644 --- a/.github/workflows/build-and-push-image.yaml +++ b/.github/workflows/build-and-push-image.yaml @@ -1,17 +1,28 @@ -name: Publish Docker image and Github Registry +# This Github action is for publishing of the primary image for AnythingLLM +# It will publish a linux/amd64 and linux/arm64 image at the same time +# This file should ONLY BY USED FOR `master` BRANCH. +# TODO: Update `runs-on` for arm64 when GitHub finally supports +# native arm environments as QEMU takes around 1 hour to build +# ref: https://github.com/actions/runner-images/issues/5631 :( +name: Publish AnythingLLM Primary Docker image (amd64/arm64) + +concurrency: + group: build-${{ github.ref }} + cancel-in-progress: true on: push: - branches: ['master'] + branches: ['master'] # master branch only. Do not modify. paths-ignore: - '*.md' - 'cloud-deployments/*' - 'images/*' - '.vscode/*' + - '**/.env.example' jobs: - push_to_registries: - name: Push Docker image to multiple registries + push_multi_platform_to_registries: + name: Push Docker multi-platform image to multiple registries runs-on: ubuntu-latest permissions: packages: write @@ -19,6 +30,17 @@ jobs: steps: - name: Check out the repo uses: actions/checkout@v4 + + - name: Parse repository name to lowercase + shell: bash + run: echo "repo=${GITHUB_REPOSITORY,,}" >> $GITHUB_OUTPUT + id: lowercase_repo + + - name: Set up QEMU + uses: docker/setup-qemu-action@v3 + + - name: Set up Docker Buildx + uses: docker/setup-buildx-action@v3 - name: Log in to Docker Hub uses: docker/login-action@f4ef78c080cd8ba55a85445d5b36e214a81df20a @@ -41,11 +63,15 @@ jobs: mintplexlabs/anythingllm ghcr.io/${{ github.repository }} - - name: Build and push Docker images - uses: docker/build-push-action@3b5e8027fcad23fda98b2e3ac259d8d67585f671 + - name: Build and push multi-platform Docker image + uses: docker/build-push-action@v5 with: context: . file: ./docker/Dockerfile push: true - tags: ${{ steps.meta.outputs.tags }} - labels: ${{ steps.meta.outputs.labels }} + platforms: linux/amd64,linux/arm64 + tags: | + ${{ steps.meta.outputs.tags }} + ${{ github.ref_name == 'master' && 'mintplexlabs/anythingllm:latest' || '' }} + ${{ github.ref_name == 'master' && format('ghcr.io/{0}:{1}', steps.lowercase_repo.outputs.repo, 'latest') || '' }} + labels: ${{ steps.meta.outputs.labels }} \ No newline at end of file diff --git a/.gitignore b/.gitignore index b21eadd77..f6a7e551f 100644 --- a/.gitignore +++ b/.gitignore @@ -7,4 +7,5 @@ __pycache__ v-env .DS_Store aws_cf_deploy_anything_llm.json -yarn.lock \ No newline at end of file +yarn.lock +*.bak diff --git a/.prettierignore b/.prettierignore new file mode 100644 index 000000000..faedf3258 --- /dev/null +++ b/.prettierignore @@ -0,0 +1,12 @@ +# defaults +**/.git +**/.svn +**/.hg +**/node_modules + +#frontend +frontend/bundleinspector.html +**/dist + +#server +server/swagger/openapi.json diff --git a/.prettierrc b/.prettierrc new file mode 100644 index 000000000..3574c1dfd --- /dev/null +++ b/.prettierrc @@ -0,0 +1,38 @@ +{ + "tabWidth": 2, + "useTabs": false, + "endOfLine": "lf", + "semi": true, + "singleQuote": false, + "printWidth": 80, + "trailingComma": "es5", + "bracketSpacing": true, + "bracketSameLine": false, + "overrides": [ + { + "files": ["*.js", "*.mjs", "*.jsx"], + "options": { + "parser": "flow", + "arrowParens": "always" + } + }, + { + "files": "*.config.js", + "options": { + "semi": false, + "parser": "flow", + "trailingComma": "none" + } + }, + { + "files": "*.html", + "options": { + "bracketSameLine": true + } + }, + { + "files": ".prettierrc", + "options": { "parser": "json" } + } + ] +} diff --git a/.vscode/launch.json b/.vscode/launch.json new file mode 100644 index 000000000..911b0a765 --- /dev/null +++ b/.vscode/launch.json @@ -0,0 +1,74 @@ +{ + // Use IntelliSense to learn about possible attributes. + // Hover to view descriptions of existing attributes. + // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387 + "version": "0.2.0", + "configurations": [ + { + "name": "Collector debug", + "request": "launch", + "cwd": "${workspaceFolder}/collector", + "env": { + "NODE_ENV": "development" + }, + "runtimeArgs": [ + "index.js" + ], + // not using yarn/nodemon because it doesn't work with breakpoints + // "runtimeExecutable": "yarn", + "skipFiles": [ + "/**" + ], + "type": "node" + }, + { + "name": "Server debug", + "request": "launch", + "cwd": "${workspaceFolder}/server", + "env": { + "NODE_ENV": "development" + }, + "runtimeArgs": [ + "index.js" + ], + // not using yarn/nodemon because it doesn't work with breakpoints + // "runtimeExecutable": "yarn", + "skipFiles": [ + "/**" + ], + "type": "node" + }, + { + "name": "Frontend debug", + "request": "launch", + "cwd": "${workspaceFolder}/frontend", + "env": { + "NODE_ENV": "development", + }, + "runtimeExecutable": "${workspaceFolder}/frontend/node_modules/.bin/vite", + "runtimeArgs": [ + "--debug", + "--host=0.0.0.0" + ], + // "runtimeExecutable": "yarn", + "skipFiles": [ + "/**" + ], + "type": "node" + }, + { + "name": "Launch Edge", + "request": "launch", + "type": "msedge", + "url": "http://localhost:3000", + "webRoot": "${workspaceFolder}" + }, + { + "type": "chrome", + "request": "launch", + "name": "Launch Chrome against localhost", + "url": "http://localhost:3000", + "webRoot": "${workspaceFolder}" + } + ] +} diff --git a/.vscode/settings.json b/.vscode/settings.json index dde2d134b..82165a178 100644 --- a/.vscode/settings.json +++ b/.vscode/settings.json @@ -1,7 +1,11 @@ { "cSpell.words": [ + "Dockerized", + "Langchain", + "Ollama", "openai", "Qdrant", "Weaviate" - ] + ], + "eslint.experimental.useFlatConfig": true } \ No newline at end of file diff --git a/.vscode/tasks.json b/.vscode/tasks.json new file mode 100644 index 000000000..fd0eb3bd2 --- /dev/null +++ b/.vscode/tasks.json @@ -0,0 +1,94 @@ +{ + // See https://go.microsoft.com/fwlink/?LinkId=733558 + // for the documentation about the tasks.json format + "version": "2.0.0", + "tasks": [ + { + "type": "shell", + "options": { + "cwd": "${workspaceFolder}/collector", + "statusbar": { + "color": "#ffea00", + "detail": "Runs the collector", + "label": "Collector: $(play) run", + "running": { + "color": "#ffea00", + "label": "Collector: $(gear~spin) running" + } + } + }, + "command": "cd ${workspaceFolder}/collector/ && yarn dev", + "runOptions": { + "instanceLimit": 1, + "reevaluateOnRerun": true + }, + "presentation": { + "echo": true, + "reveal": "always", + "focus": false, + "panel": "shared", + "showReuseMessage": true, + "clear": false + }, + "label": "Collector: run" + }, + { + "type": "shell", + "options": { + "cwd": "${workspaceFolder}/server", + "statusbar": { + "color": "#ffea00", + "detail": "Runs the server", + "label": "Server: $(play) run", + "running": { + "color": "#ffea00", + "label": "Server: $(gear~spin) running" + } + } + }, + "command": "cd ${workspaceFolder}/server/ && yarn dev", + "runOptions": { + "instanceLimit": 1, + "reevaluateOnRerun": true + }, + "presentation": { + "echo": true, + "reveal": "always", + "focus": false, + "panel": "shared", + "showReuseMessage": true, + "clear": false + }, + "label": "Server: run" + }, + { + "type": "shell", + "options": { + "cwd": "${workspaceFolder}/frontend", + "statusbar": { + "color": "#ffea00", + "detail": "Runs the frontend", + "label": "Frontend: $(play) run", + "running": { + "color": "#ffea00", + "label": "Frontend: $(gear~spin) running" + } + } + }, + "command": "cd ${workspaceFolder}/frontend/ && yarn dev", + "runOptions": { + "instanceLimit": 1, + "reevaluateOnRerun": true + }, + "presentation": { + "echo": true, + "reveal": "always", + "focus": false, + "panel": "shared", + "showReuseMessage": true, + "clear": false + }, + "label": "Frontend: run" + } + ] +} diff --git a/README.md b/README.md index 9ed7cc609..62d58d870 100644 --- a/README.md +++ b/README.md @@ -23,9 +23,13 @@

+

+👉 AnythingLLM for desktop! Sign up +

+ A full-stack application that enables you to turn any document, resource, or piece of content into context that any LLM can use as references during chatting. This application allows you to pick and choose which LLM or Vector Database you want to use as well as supporting multi-user management and permissions. -![Chatting](/images/screenshots/chatting.gif) +![Chatting](https://github.com/Mintplex-Labs/anything-llm/assets/16845892/cfc5f47c-bd91-4067-986c-f3f49621a859)
Watch the demo! @@ -58,6 +62,8 @@ Some cool features of AnythingLLM - [OpenAI](https://openai.com) - [Azure OpenAI](https://azure.microsoft.com/en-us/products/ai-services/openai-service) - [Anthropic ClaudeV2](https://www.anthropic.com/) +- [Google Gemini Pro](https://ai.google.dev/) +- [Ollama (chat models)](https://ollama.ai/) - [LM Studio (all models)](https://lmstudio.ai) - [LocalAi (all models)](https://localai.io/) @@ -91,19 +97,14 @@ Mintplex Labs & the community maintain a number of deployment methods, scripts, ## How to setup for development -- `yarn setup` from the project root directory. - - This will fill in the required `.env` files you'll need in each of the application sections. Go fill those out before proceeding or else things won't work right. -- `yarn prisma:setup` To build the Prisma client and migrate the database. +- `yarn setup` To fill in the required `.env` files you'll need in each of the application sections (from root of repo). + - Go fill those out before proceeding. Ensure `server/.env.development` is filled or else things won't work right. +- `yarn dev:server` To boot the server locally (from root of repo). +- `yarn dev:frontend` To boot the frontend locally (from root of repo). +- `yarn dev:collector` To then run the document collector (from root of repo). + -To boot the server locally (from root of repo): -- ensure `server/.env.development` is set and filled out. -`yarn dev:server` - -To boot the frontend locally (from root of repo): -`yarn dev:frontend` -To then run the document collector (from root of repo) -`yarn dev:collector` [Learn about documents](./server/storage/documents/DOCUMENTS.md) diff --git a/clean.sh b/clean.sh deleted file mode 100644 index 966cc1773..000000000 --- a/clean.sh +++ /dev/null @@ -1,3 +0,0 @@ -# Easily kill process on port because sometimes nodemon fails to reboot -kill -9 $(lsof -t -i tcp:5000) & -kill -9 $(lsof -t -i tcp:3001) # if running default for MacOS Monterey diff --git a/collector/.env.example b/collector/.env.example index 91d0e10a5..ea4aba395 100644 --- a/collector/.env.example +++ b/collector/.env.example @@ -1 +1 @@ -# Placeholder .env file for collector runtime \ No newline at end of file +# Placeholder .env file for collector runtime diff --git a/collector/package.json b/collector/package.json index 0e81b72a5..01cf26b6b 100644 --- a/collector/package.json +++ b/collector/package.json @@ -34,7 +34,7 @@ "multer": "^1.4.5-lts.1", "officeparser": "^4.0.5", "pdf-parse": "^1.1.1", - "puppeteer": "^21.6.1", + "puppeteer": "~21.5.2", "slugify": "^1.6.6", "url-pattern": "^1.0.3", "uuid": "^9.0.0", diff --git a/collector/processLink/convert/generic.js b/collector/processLink/convert/generic.js index 6af41f6cf..f42dcd171 100644 --- a/collector/processLink/convert/generic.js +++ b/collector/processLink/convert/generic.js @@ -18,7 +18,7 @@ async function scrapeGenericUrl(link) { const url = new URL(link); const filename = (url.host + "-" + url.pathname).replace(".", "_"); - data = { + const data = { id: v4(), url: "file://" + slugify(filename) + ".html", title: slugify(filename) + ".html", diff --git a/collector/processSingleFile/convert/asAudio.js b/collector/processSingleFile/convert/asAudio.js index 97e958df7..a15207fba 100644 --- a/collector/processSingleFile/convert/asAudio.js +++ b/collector/processSingleFile/convert/asAudio.js @@ -46,7 +46,7 @@ async function asAudio({ fullFilePath = "", filename = "" }) { return { success: false, reason: `No text content found in ${filename}.` }; } - data = { + const data = { id: v4(), url: "file://" + fullFilePath, title: filename, diff --git a/collector/processSingleFile/convert/asDocx.js b/collector/processSingleFile/convert/asDocx.js index 7c3368657..7a64a042d 100644 --- a/collector/processSingleFile/convert/asDocx.js +++ b/collector/processSingleFile/convert/asDocx.js @@ -28,7 +28,7 @@ async function asDocX({ fullFilePath = "", filename = "" }) { } const content = pageContent.join(""); - data = { + const data = { id: v4(), url: "file://" + fullFilePath, title: filename, diff --git a/collector/processSingleFile/convert/asMbox.js b/collector/processSingleFile/convert/asMbox.js index 58df98cd6..30883f21b 100644 --- a/collector/processSingleFile/convert/asMbox.js +++ b/collector/processSingleFile/convert/asMbox.js @@ -35,7 +35,7 @@ async function asMbox({ fullFilePath = "", filename = "" }) { `-- Working on message "${mail.subject || "Unknown subject"}" --` ); - data = { + const data = { id: v4(), url: "file://" + fullFilePath, title: mail?.subject diff --git a/collector/processSingleFile/convert/asOfficeMime.js b/collector/processSingleFile/convert/asOfficeMime.js index 49a96b8b1..a6eb0351a 100644 --- a/collector/processSingleFile/convert/asOfficeMime.js +++ b/collector/processSingleFile/convert/asOfficeMime.js @@ -23,7 +23,7 @@ async function asOfficeMime({ fullFilePath = "", filename = "" }) { return { success: false, reason: `No text content found in ${filename}.` }; } - data = { + const data = { id: v4(), url: "file://" + fullFilePath, title: filename, diff --git a/collector/processSingleFile/convert/asPDF.js b/collector/processSingleFile/convert/asPDF.js index 2509887e8..f6d869d5c 100644 --- a/collector/processSingleFile/convert/asPDF.js +++ b/collector/processSingleFile/convert/asPDF.js @@ -33,7 +33,7 @@ async function asPDF({ fullFilePath = "", filename = "" }) { } const content = pageContent.join(""); - data = { + const data = { id: v4(), url: "file://" + fullFilePath, title: docs[0]?.metadata?.pdf?.info?.Title || filename, diff --git a/collector/processSingleFile/convert/asTxt.js b/collector/processSingleFile/convert/asTxt.js index 229c175e8..ad35e5476 100644 --- a/collector/processSingleFile/convert/asTxt.js +++ b/collector/processSingleFile/convert/asTxt.js @@ -23,7 +23,7 @@ async function asTxt({ fullFilePath = "", filename = "" }) { } console.log(`-- Working ${filename} --`); - data = { + const data = { id: v4(), url: "file://" + fullFilePath, title: filename, diff --git a/collector/processSingleFile/index.js b/collector/processSingleFile/index.js index 3884ca9bd..37c9fd5c5 100644 --- a/collector/processSingleFile/index.js +++ b/collector/processSingleFile/index.js @@ -5,8 +5,7 @@ const { SUPPORTED_FILETYPE_CONVERTERS, } = require("../utils/constants"); const { trashFile } = require("../utils/files"); - -RESERVED_FILES = ["__HOTDIR__.md"]; +const RESERVED_FILES = ["__HOTDIR__.md"]; async function processSingleFile(targetFilename) { const fullFilePath = path.resolve(WATCH_DIRECTORY, targetFilename); diff --git a/collector/yarn.lock b/collector/yarn.lock index 6501aac95..e181b245b 100644 --- a/collector/yarn.lock +++ b/collector/yarn.lock @@ -134,10 +134,10 @@ resolved "https://registry.yarnpkg.com/@protobufjs/utf8/-/utf8-1.1.0.tgz#a777360b5b39a1a2e5106f8e858f2fd2d060c570" integrity sha512-Vvn3zZrhQZkkBE8LSuW3em98c0FwgO4nxzv6OdSxPKJIEKY2bGbHn+mhGIPerzI4twdxaP8/0+06HBpwf345Lw== -"@puppeteer/browsers@1.9.0": - version "1.9.0" - resolved "https://registry.yarnpkg.com/@puppeteer/browsers/-/browsers-1.9.0.tgz#dfd0aad0bdc039572f1b57648f189525d627b7ff" - integrity sha512-QwguOLy44YBGC8vuPP2nmpX4MUN2FzWbsnvZJtiCzecU3lHmVZkaC1tq6rToi9a200m8RzlVtDyxCS0UIDrxUg== +"@puppeteer/browsers@1.8.0": + version "1.8.0" + resolved "https://registry.yarnpkg.com/@puppeteer/browsers/-/browsers-1.8.0.tgz#fb6ee61de15e7f0e67737aea9f9bab1512dbd7d8" + integrity sha512-TkRHIV6k2D8OlUe8RtG+5jgOF/H98Myx0M6AOafC8DdNVOFiBSFa5cpRDtpm8LXOa9sVwe0+e6Q3FC56X/DZfg== dependencies: debug "4.3.4" extract-zip "2.0.1" @@ -608,10 +608,10 @@ chownr@^2.0.0: resolved "https://registry.yarnpkg.com/chownr/-/chownr-2.0.0.tgz#15bfbe53d2eab4cf70f18a8cd68ebe5b3cb1dece" integrity sha512-bIomtDF5KGpdogkLd9VspvFzk9KfpyyGlS8YFVZl7TGPBHL5snIOnxeshwVgPteQ9b4Eydl+pVbIyE1DcvCWgQ== -chromium-bidi@0.5.1: - version "0.5.1" - resolved "https://registry.yarnpkg.com/chromium-bidi/-/chromium-bidi-0.5.1.tgz#390c1af350c4887824a33d82190de1cc5c5680fc" - integrity sha512-dcCqOgq9fHKExc2R4JZs/oKbOghWpUNFAJODS8WKRtLhp3avtIH5UDCBrutdqZdh3pARogH8y1ObXm87emwb3g== +chromium-bidi@0.4.33: + version "0.4.33" + resolved "https://registry.yarnpkg.com/chromium-bidi/-/chromium-bidi-0.4.33.tgz#9a9aba5a5b07118c8e7d6405f8ee79f47418dd1d" + integrity sha512-IxoFM5WGQOIAd95qrSXzJUv4eXIrh+RvU3rwwqIiwYuvfE7U/Llj4fejbsJnjJMUYCuGtVQsY2gv7oGl4aTNSQ== dependencies: mitt "3.0.1" urlpattern-polyfill "9.0.0" @@ -2584,26 +2584,26 @@ pump@^3.0.0: end-of-stream "^1.1.0" once "^1.3.1" -puppeteer-core@21.6.1: - version "21.6.1" - resolved "https://registry.yarnpkg.com/puppeteer-core/-/puppeteer-core-21.6.1.tgz#10eccb4dc3167c8c26bc21122fabb45a9fda9ca7" - integrity sha512-0chaaK/RL9S1U3bsyR4fUeUfoj51vNnjWvXgG6DcsyMjwYNpLcAThv187i1rZCo7QhJP0wZN8plQkjNyrq2h+A== +puppeteer-core@21.5.2: + version "21.5.2" + resolved "https://registry.yarnpkg.com/puppeteer-core/-/puppeteer-core-21.5.2.tgz#6d3de4efb2ae65f1ee072043787b75594e88035f" + integrity sha512-v4T0cWnujSKs+iEfmb8ccd7u4/x8oblEyKqplqKnJ582Kw8PewYAWvkH4qUWhitN3O2q9RF7dzkvjyK5HbzjLA== dependencies: - "@puppeteer/browsers" "1.9.0" - chromium-bidi "0.5.1" + "@puppeteer/browsers" "1.8.0" + chromium-bidi "0.4.33" cross-fetch "4.0.0" debug "4.3.4" devtools-protocol "0.0.1203626" - ws "8.15.1" + ws "8.14.2" -puppeteer@^21.6.1: - version "21.6.1" - resolved "https://registry.yarnpkg.com/puppeteer/-/puppeteer-21.6.1.tgz#2ec0878906ff90b3a424f19e5eb006592abe25b6" - integrity sha512-O+pbc61oj8ln6m8EJKncrsQFmytgRyFYERtk190PeLbJn5JKpmmynn2p1PiFrlhCitAQXLJ0MOy7F0TeyCRqBg== +puppeteer@~21.5.2: + version "21.5.2" + resolved "https://registry.yarnpkg.com/puppeteer/-/puppeteer-21.5.2.tgz#0a4a72175c0fd0944d6486f4734807e1671d527b" + integrity sha512-BaAGJOq8Fl6/cck6obmwaNLksuY0Bg/lIahCLhJPGXBFUD2mCffypa4A592MaWnDcye7eaHmSK9yot0pxctY8A== dependencies: - "@puppeteer/browsers" "1.9.0" + "@puppeteer/browsers" "1.8.0" cosmiconfig "8.3.6" - puppeteer-core "21.6.1" + puppeteer-core "21.5.2" qs@6.11.0: version "6.11.0" @@ -3259,10 +3259,10 @@ wrappy@1: resolved "https://registry.yarnpkg.com/wrappy/-/wrappy-1.0.2.tgz#b5243d8f3ec1aa35f1364605bc0d1036e30ab69f" integrity sha512-l4Sp/DRseor9wL6EvV2+TuQn63dMkPjZ/sp9XkghTEbV9KlPS1xUsZ3u7/IQO4wxtcFB4bgpQPRcR3QCvezPcQ== -ws@8.15.1: - version "8.15.1" - resolved "https://registry.yarnpkg.com/ws/-/ws-8.15.1.tgz#271ba33a45ca0cc477940f7f200cd7fba7ee1997" - integrity sha512-W5OZiCjXEmk0yZ66ZN82beM5Sz7l7coYxpRkzS+p9PP+ToQry8szKh+61eNktr7EA9DOwvFGhfC605jDHbP6QQ== +ws@8.14.2: + version "8.14.2" + resolved "https://registry.yarnpkg.com/ws/-/ws-8.14.2.tgz#6c249a806eb2db7a20d26d51e7709eab7b2e6c7f" + integrity sha512-wEBG1ftX4jcglPxgFCMJmZ2PLtSbJ2Peg6TmpJFTbe9GZYOQCDPdMYu/Tm0/bGZkw8paZnJY45J4K2PZrLYq8g== xmlbuilder@^10.0.0: version "10.1.1" diff --git a/docker/.env.example b/docker/.env.example index 650fad8d6..9b2b24c3f 100644 --- a/docker/.env.example +++ b/docker/.env.example @@ -1,5 +1,4 @@ SERVER_PORT=3001 -CACHE_VECTORS="true" STORAGE_DIR="/app/server/storage" UID='1000' GID='1000' @@ -12,6 +11,10 @@ GID='1000' # OPEN_AI_KEY= # OPEN_MODEL_PREF='gpt-3.5-turbo' +# LLM_PROVIDER='gemini' +# GEMINI_API_KEY= +# GEMINI_LLM_MODEL_PREF='gemini-pro' + # LLM_PROVIDER='azure' # AZURE_OPENAI_ENDPOINT= # AZURE_OPENAI_KEY= @@ -32,6 +35,11 @@ GID='1000' # LOCAL_AI_MODEL_TOKEN_LIMIT=4096 # LOCAL_AI_API_KEY="sk-123abc" +# LLM_PROVIDER='ollama' +# OLLAMA_BASE_PATH='http://host.docker.internal:11434' +# OLLAMA_MODEL_PREF='llama2' +# OLLAMA_MODEL_TOKEN_LIMIT=4096 + ########################################### ######## Embedding API SElECTION ########## ########################################### @@ -45,7 +53,7 @@ GID='1000' # EMBEDDING_MODEL_PREF='my-embedder-model' # This is the "deployment" on Azure you want to use for embeddings. Not the base model. Valid base model is text-embedding-ada-002 # EMBEDDING_ENGINE='localai' -# EMBEDDING_BASE_PATH='https://localhost:8080/v1' +# EMBEDDING_BASE_PATH='http://localhost:8080/v1' # EMBEDDING_MODEL_PREF='text-embedding-ada-002' # EMBEDDING_MODEL_MAX_CHUNK_LENGTH=1000 # The max chunk size in chars a string to embed can be @@ -92,4 +100,13 @@ GID='1000' # PASSWORDUPPERCASE=1 # PASSWORDNUMERIC=1 # PASSWORDSYMBOL=1 -# PASSWORDREQUIREMENTS=4 \ No newline at end of file +# PASSWORDREQUIREMENTS=4 + +########################################### +######## ENABLE HTTPS SERVER ############## +########################################### +# By enabling this and providing the path/filename for the key and cert, +# the server will use HTTPS instead of HTTP. +#ENABLE_HTTPS="true" +#HTTPS_CERT_PATH="sslcert/cert.pem" +#HTTPS_KEY_PATH="sslcert/key.pem" diff --git a/docker/Dockerfile b/docker/Dockerfile index 01dde61c9..595099fb7 100644 --- a/docker/Dockerfile +++ b/docker/Dockerfile @@ -1,11 +1,66 @@ # Setup base image FROM ubuntu:jammy-20230522 AS base -# Build arguments ARG ARG_UID=1000 ARG ARG_GID=1000 -# Install system dependencies +FROM base AS build-arm64 +RUN echo "Preparing build of AnythingLLM image for arm64 architecture" + +RUN DEBIAN_FRONTEND=noninteractive apt-get update && \ + DEBIAN_FRONTEND=noninteractive apt-get install -yq --no-install-recommends \ + unzip curl gnupg libgfortran5 libgbm1 tzdata netcat \ + libasound2 libatk1.0-0 libc6 libcairo2 libcups2 libdbus-1-3 libexpat1 libfontconfig1 \ + libgcc1 libglib2.0-0 libgtk-3-0 libnspr4 libpango-1.0-0 libx11-6 libx11-xcb1 libxcb1 \ + libxcomposite1 libxcursor1 libxdamage1 libxext6 libxfixes3 libxi6 libxrandr2 libxrender1 \ + libxss1 libxtst6 ca-certificates fonts-liberation libappindicator1 libnss3 lsb-release \ + xdg-utils git build-essential ffmpeg && \ + mkdir -p /etc/apt/keyrings && \ + curl -fsSL https://deb.nodesource.com/gpgkey/nodesource-repo.gpg.key | gpg --dearmor -o /etc/apt/keyrings/nodesource.gpg && \ + echo "deb [signed-by=/etc/apt/keyrings/nodesource.gpg] https://deb.nodesource.com/node_18.x nodistro main" | tee /etc/apt/sources.list.d/nodesource.list && \ + apt-get update && \ + apt-get install -yq --no-install-recommends nodejs && \ + curl -LO https://github.com/yarnpkg/yarn/releases/download/v1.22.19/yarn_1.22.19_all.deb \ + && dpkg -i yarn_1.22.19_all.deb \ + && rm yarn_1.22.19_all.deb + +# Create a group and user with specific UID and GID +RUN groupadd -g $ARG_GID anythingllm && \ + useradd -u $ARG_UID -m -d /app -s /bin/bash -g anythingllm anythingllm && \ + mkdir -p /app/frontend/ /app/server/ /app/collector/ && chown -R anythingllm:anythingllm /app + +# Copy docker helper scripts +COPY ./docker/docker-entrypoint.sh /usr/local/bin/ +COPY ./docker/docker-healthcheck.sh /usr/local/bin/ +COPY --chown=anythingllm:anythingllm ./docker/.env.example /app/server/.env + +# Ensure the scripts are executable +RUN chmod +x /usr/local/bin/docker-entrypoint.sh && \ + chmod +x /usr/local/bin/docker-healthcheck.sh + +USER anythingllm +WORKDIR /app + +# Puppeteer does not ship with an ARM86 compatible build for Chromium +# so web-scraping would be broken in arm docker containers unless we patch it +# by manually installing a compatible chromedriver. +RUN echo "Need to patch Puppeteer x Chromium support for ARM86 - installing dep!" && \ + curl https://playwright.azureedge.net/builds/chromium/1088/chromium-linux-arm64.zip -o chrome-linux.zip && \ + unzip chrome-linux.zip && \ + rm -rf chrome-linux.zip + +ENV PUPPETEER_SKIP_CHROMIUM_DOWNLOAD=true +ENV CHROME_PATH=/app/chrome-linux/chrome +ENV PUPPETEER_EXECUTABLE_PATH=/app/chrome-linux/chrome + +RUN echo "Done running arm64 specific installtion steps" + +############################################# + +# amd64-specific stage +FROM base AS build-amd64 +RUN echo "Preparing build of AnythingLLM image for non-ARM architecture" + RUN DEBIAN_FRONTEND=noninteractive apt-get update && \ DEBIAN_FRONTEND=noninteractive apt-get install -yq --no-install-recommends \ curl gnupg libgfortran5 libgbm1 tzdata netcat \ @@ -37,22 +92,25 @@ COPY --chown=anythingllm:anythingllm ./docker/.env.example /app/server/.env RUN chmod +x /usr/local/bin/docker-entrypoint.sh && \ chmod +x /usr/local/bin/docker-healthcheck.sh -USER anythingllm +############################################# +# COMMON BUILD FLOW FOR ALL ARCHS +############################################# +FROM build-${TARGETARCH} AS build +RUN echo "Running common build flow of AnythingLLM image for all architectures" +USER anythingllm WORKDIR /app # Install frontend dependencies -FROM base as frontend-deps +FROM build as frontend-deps COPY ./frontend/package.json ./frontend/yarn.lock ./frontend/ -RUN cd ./frontend/ && yarn install && yarn cache clean +RUN cd ./frontend/ && yarn install --network-timeout 100000 && yarn cache clean # Install server dependencies -FROM base as server-deps +FROM build as server-deps COPY ./server/package.json ./server/yarn.lock ./server/ -RUN cd ./server/ && yarn install --production && yarn cache clean && \ - rm /app/server/node_modules/vectordb/x86_64-apple-darwin.node && \ - rm /app/server/node_modules/vectordb/aarch64-apple-darwin.node +RUN cd ./server/ && yarn install --production --network-timeout 100000 && yarn cache clean # Compile Llama.cpp bindings for node-llama-cpp for this operating system. USER root @@ -75,7 +133,7 @@ COPY --from=build-stage /app/frontend/dist ./server/public COPY --chown=anythingllm:anythingllm ./collector/ ./collector/ # Install collector dependencies -RUN cd /app/collector && yarn install --production && yarn cache clean +RUN cd /app/collector && yarn install --production --network-timeout 100000 && yarn cache clean # Migrate and Run Prisma against known schema RUN cd ./server && npx prisma generate --schema=./prisma/schema.prisma @@ -83,6 +141,7 @@ RUN cd ./server && npx prisma migrate deploy --schema=./prisma/schema.prisma # Setup the environment ENV NODE_ENV=production +ENV ANYTHING_LLM_RUNTIME=docker # Expose the server port EXPOSE 3001 @@ -92,4 +151,4 @@ HEALTHCHECK --interval=1m --timeout=10s --start-period=1m \ CMD /bin/bash /usr/local/bin/docker-healthcheck.sh || exit 1 # Run the server -ENTRYPOINT ["/bin/bash", "/usr/local/bin/docker-entrypoint.sh"] +ENTRYPOINT ["/bin/bash", "/usr/local/bin/docker-entrypoint.sh"] \ No newline at end of file diff --git a/docker/HOW_TO_USE_DOCKER.md b/docker/HOW_TO_USE_DOCKER.md index 5b5a58067..81a3cd835 100644 --- a/docker/HOW_TO_USE_DOCKER.md +++ b/docker/HOW_TO_USE_DOCKER.md @@ -29,7 +29,20 @@ Use the Dockerized version of AnythingLLM for a much faster and complete startup > It is best to mount the containers storage volume to a folder on your host machine > so that you can pull in future updates without deleting your existing data! -`docker pull mintplexlabs/anythingllm:master` +Pull in the latest image from docker. Supports both `amd64` and `arm64` CPU architectures. +```shell +docker pull mintplexlabs/anythingllm +``` + + + + + + + + + + + + + +
Mount the storage locally and run AnythingLLM in Docker
+ Linux/MacOs + ```shell export STORAGE_LOCATION=$HOME/anythingllm && \ @@ -40,9 +53,34 @@ docker run -d -p 3001:3001 \ -v ${STORAGE_LOCATION}:/app/server/storage \ -v ${STORAGE_LOCATION}/.env:/app/server/.env \ -e STORAGE_DIR="/app/server/storage" \ -mintplexlabs/anythingllm:master +mintplexlabs/anythingllm ``` +
+ Windows + + +```powershell +# Run this in powershell terminal +$env:STORAGE_LOCATION="$HOME\Documents\anythingllm"; ` +If(!(Test-Path $env:STORAGE_LOCATION)) {New-Item $env:STORAGE_LOCATION -ItemType Directory}; ` +If(!(Test-Path "$env:STORAGE_LOCATION\.env")) {New-Item "$env:STORAGE_LOCATION\.env"}; ` +docker run -d -p 3001:3001 ` +--cap-add SYS_ADMIN ` +-v "$env:STORAGE_LOCATION`:/app/server/storage" ` +-v "$env:STORAGE_LOCATION\.env:/app/server/.env" ` +-e STORAGE_DIR="/app/server/storage" ` +mintplexlabs/anythingllm; +``` + +
+ Go to `http://localhost:3001` and you are now using AnythingLLM! All your data and progress will persist between container rebuilds or pulls from Docker Hub. diff --git a/docker/docker-compose.yml b/docker/docker-compose.yml index 5ec67cab4..313e4175b 100644 --- a/docker/docker-compose.yml +++ b/docker/docker-compose.yml @@ -1,4 +1,4 @@ -version: '3.9' +version: "3.9" name: anythingllm @@ -10,7 +10,6 @@ services: anything-llm: container_name: anything-llm image: anything-llm:latest - platform: linux/amd64 build: context: ../. dockerfile: ./docker/Dockerfile diff --git a/eslint.config.js b/eslint.config.js new file mode 100644 index 000000000..b6ef861c9 --- /dev/null +++ b/eslint.config.js @@ -0,0 +1,90 @@ +import globals from "./server/node_modules/globals/index.js" +import eslintRecommended from "./server/node_modules/@eslint/js/src/index.js" +import eslintConfigPrettier from "./server/node_modules/eslint-config-prettier/index.js" +import prettier from "./server/node_modules/eslint-plugin-prettier/eslint-plugin-prettier.js" +import react from "./server/node_modules/eslint-plugin-react/index.js" +import reactRefresh from "./server/node_modules/eslint-plugin-react-refresh/index.js" +import reactHooks from "./server/node_modules/eslint-plugin-react-hooks/index.js" +import ftFlow from "./server/node_modules/eslint-plugin-ft-flow/dist/index.js" +import hermesParser from "./server/node_modules/hermes-eslint/dist/index.js" + +const reactRecommended = react.configs.recommended +const jsxRuntime = react.configs["jsx-runtime"] + +export default [ + eslintRecommended.configs.recommended, + eslintConfigPrettier, + { + ignores: ["**/*.test.js"], + languageOptions: { + parser: hermesParser, + parserOptions: { + ecmaFeatures: { jsx: true } + }, + ecmaVersion: 2020, + sourceType: "module", + globals: { + ...globals.browser, + ...globals.es2020, + ...globals.node + } + }, + linterOptions: { reportUnusedDisableDirectives: true }, + settings: { react: { version: "18.2" } }, + plugins: { + ftFlow, + react, + "jsx-runtime": jsxRuntime, + "react-hooks": reactHooks, + prettier + }, + rules: { + ...reactRecommended.rules, + ...reactHooks.configs.recommended.rules, + ...ftFlow.recommended, + "no-unused-vars": "warn", + "no-undef": "warn", + "no-empty": "warn", + "no-extra-boolean-cast": "warn", + "prettier/prettier": "warn" + } + }, + { + files: ["frontend/src/**/*.js"], + plugins: { + ftFlow, + prettier + }, + rules: { + "prettier/prettier": "warn" + } + }, + { + files: [ + "server/endpoints/**/*.js", + "server/models/**/*.js", + "server/swagger/**/*.js", + "server/utils/**/*.js", + "server/index.js" + ], + rules: { + "no-undef": "warn" + } + }, + { + files: ["frontend/src/**/*.jsx"], + plugins: { + ftFlow, + react, + "jsx-runtime": jsxRuntime, + "react-hooks": reactHooks, + "react-refresh": reactRefresh, + prettier + }, + rules: { + ...jsxRuntime.rules, + "react/prop-types": "off", // FIXME + "react-refresh/only-export-components": "warn" + } + } +] diff --git a/frontend/.env.example b/frontend/.env.example index 44057dac4..73cd07c62 100644 --- a/frontend/.env.example +++ b/frontend/.env.example @@ -1,2 +1,3 @@ VITE_API_BASE='http://localhost:3001/api' # Use this URL when developing locally +# VITE_API_BASE="https://$CODESPACE_NAME-3001.$GITHUB_CODESPACES_PORT_FORWARDING_DOMAIN/api" # for Github Codespaces # VITE_API_BASE='/api' # Use this URL deploying on non-localhost address OR in docker. diff --git a/frontend/.eslintrc.cjs b/frontend/.eslintrc.cjs deleted file mode 100644 index 1bf2b3226..000000000 --- a/frontend/.eslintrc.cjs +++ /dev/null @@ -1,20 +0,0 @@ -module.exports = { - "env": { "browser": true, "es2020": true }, - "extends": [ - "eslint:recommended", - "plugin:react/recommended", - "plugin:react/jsx-runtime", - "plugin:react-hooks/recommended" - ], - "files": ["**/*.js", "**/*.jsx"], - "linterOptions": { "reportUnusedDisableDirectives": true }, - "parserOptions": { "ecmaVersion": "latest", "sourceType": "module", "ecmaFeatures": { "jsx": true } }, - "settings": { "react": { "version": '18.2' } }, - "plugins": [ - "react-refresh", - "react-hooks" - ], - "rules": { - "react-refresh/only-export-components": "warn" - } -} diff --git a/frontend/.gitignore b/frontend/.gitignore index d5d0099dc..196c8f691 100644 --- a/frontend/.gitignore +++ b/frontend/.gitignore @@ -9,6 +9,7 @@ lerna-debug.log* node_modules dist +lib dist-ssr *.local @@ -24,3 +25,4 @@ dist-ssr *.sw? bundleinspector.html .env.production +flow-typed diff --git a/frontend/package.json b/frontend/package.json index 19fe1d0ab..ff2698953 100644 --- a/frontend/package.json +++ b/frontend/package.json @@ -1,10 +1,11 @@ { "name": "anything-llm-frontend", "private": false, - "type": "module", "license": "MIT", + "type": "module", "scripts": { "start": "vite --open", + "dev": "NODE_ENV=development vite --debug --host=0.0.0.0", "build": "vite build", "lint": "yarn prettier --write ./src", "preview": "vite preview" @@ -34,16 +35,24 @@ "uuid": "^9.0.0" }, "devDependencies": { - "@types/react": "^18.0.28", - "@types/react-dom": "^18.0.11", + "@types/react": "^18.2.23", + "@types/react-dom": "^18.2.8", + "@types/react-router-dom": "^5.3.3", "@vitejs/plugin-react": "^4.0.0-beta.0", "autoprefixer": "^10.4.14", - "eslint": "^8.38.0", - "eslint-plugin-react": "^7.32.2", + "eslint": "^8.50.0", + "eslint-config-prettier": "^9.0.0", + "eslint-plugin-ft-flow": "^3.0.0", + "eslint-plugin-prettier": "^5.0.0", + "eslint-plugin-react": "^7.33.2", "eslint-plugin-react-hooks": "^4.6.0", - "eslint-plugin-react-refresh": "^0.3.4", + "eslint-plugin-react-refresh": "^0.4.3", + "flow-bin": "^0.217.0", + "flow-remove-types": "^2.217.1", + "globals": "^13.21.0", + "hermes-eslint": "^0.15.0", "postcss": "^8.4.23", - "prettier": "^2.4.1", + "prettier": "^3.0.3", "rollup-plugin-visualizer": "^5.9.0", "tailwindcss": "^3.3.1", "vite": "^4.3.0" diff --git a/frontend/src/App.jsx b/frontend/src/App.jsx index 7224f2e9c..8007b5ad1 100644 --- a/frontend/src/App.jsx +++ b/frontend/src/App.jsx @@ -120,6 +120,7 @@ export default function App() { {/* Onboarding Flow */} } /> + } /> diff --git a/frontend/src/components/DefaultChat/index.jsx b/frontend/src/components/DefaultChat/index.jsx index ca29b4700..8bc6c7cb7 100644 --- a/frontend/src/components/DefaultChat/index.jsx +++ b/frontend/src/components/DefaultChat/index.jsx @@ -16,9 +16,11 @@ import System from "@/models/system"; import Jazzicon from "../UserIcon"; import { userFromStorage } from "@/utils/request"; import { AI_BACKGROUND_COLOR, USER_BACKGROUND_COLOR } from "@/utils/constants"; +import useUser from "@/hooks/useUser"; export default function DefaultChatContainer() { const [mockMsgs, setMockMessages] = useState([]); + const { user } = useUser(); const [fetchedMessages, setFetchedMessages] = useState([]); const { showing: showingNewWsModal, @@ -163,13 +165,15 @@ export default function DefaultChatContainer() { You can add and remove files at anytime. - + {(!user || user?.role !== "default") && ( + + )} diff --git a/frontend/src/components/EmbeddingSelection/AzureAiOptions/index.jsx b/frontend/src/components/EmbeddingSelection/AzureAiOptions/index.jsx index e7767900a..c782c51f3 100644 --- a/frontend/src/components/EmbeddingSelection/AzureAiOptions/index.jsx +++ b/frontend/src/components/EmbeddingSelection/AzureAiOptions/index.jsx @@ -1,53 +1,55 @@ export default function AzureAiOptions({ settings }) { return ( - <> -
- - -
+
+
+
+ + +
-
- - -
+
+ + +
-
- - +
+ + +
- +
); } diff --git a/frontend/src/components/EmbeddingSelection/EmbedderItem/index.jsx b/frontend/src/components/EmbeddingSelection/EmbedderItem/index.jsx new file mode 100644 index 000000000..b37b645f9 --- /dev/null +++ b/frontend/src/components/EmbeddingSelection/EmbedderItem/index.jsx @@ -0,0 +1,39 @@ +export default function EmbedderItem({ + name, + value, + image, + description, + checked, + onClick, +}) { + return ( +
onClick(value)} + className={`w-full hover:bg-white/10 p-2 rounded-md hover:cursor-pointer ${ + checked && "bg-white/10" + }`} + > + +
+ {`${name} +
+
{name}
+
+ {description} +
+
+
+
+ ); +} diff --git a/frontend/src/components/EmbeddingSelection/LocalAiOptions/index.jsx b/frontend/src/components/EmbeddingSelection/LocalAiOptions/index.jsx index 2b976e142..651d3e950 100644 --- a/frontend/src/components/EmbeddingSelection/LocalAiOptions/index.jsx +++ b/frontend/src/components/EmbeddingSelection/LocalAiOptions/index.jsx @@ -1,4 +1,4 @@ -import { useEffect, useState } from "react"; +import React, { useEffect, useState } from "react"; import System from "@/models/system"; export default function LocalAiOptions({ settings }) { @@ -10,7 +10,7 @@ export default function LocalAiOptions({ settings }) { const [apiKey, setApiKey] = useState(settings?.LocalAiApiKey); return ( - <> +
); } diff --git a/frontend/src/components/EmbeddingSelection/OpenAiOptions/index.jsx b/frontend/src/components/EmbeddingSelection/OpenAiOptions/index.jsx index f38f7c445..dd00d67ab 100644 --- a/frontend/src/components/EmbeddingSelection/OpenAiOptions/index.jsx +++ b/frontend/src/components/EmbeddingSelection/OpenAiOptions/index.jsx @@ -1,34 +1,36 @@ export default function OpenAiOptions({ settings }) { return ( - <> -
- - +
+
+
+ + +
+
+ + +
-
- - -
- +
); } diff --git a/frontend/src/components/LLMSelection/AzureAiOptions/index.jsx b/frontend/src/components/LLMSelection/AzureAiOptions/index.jsx index 297865101..ce54d3d60 100644 --- a/frontend/src/components/LLMSelection/AzureAiOptions/index.jsx +++ b/frontend/src/components/LLMSelection/AzureAiOptions/index.jsx @@ -1,87 +1,92 @@ export default function AzureAiOptions({ settings }) { return ( - <> -
- - +
+
+
+ + +
+ +
+ + +
+ +
+ + +
-
- - -
+
+
+ + +
-
- - +
+ + +
+
- -
- - -
- -
- - -
- +
); } diff --git a/frontend/src/components/LLMSelection/GeminiLLMOptions/index.jsx b/frontend/src/components/LLMSelection/GeminiLLMOptions/index.jsx new file mode 100644 index 000000000..4d09e0432 --- /dev/null +++ b/frontend/src/components/LLMSelection/GeminiLLMOptions/index.jsx @@ -0,0 +1,43 @@ +export default function GeminiLLMOptions({ settings }) { + return ( +
+
+
+ + +
+ +
+ + +
+
+
+ ); +} diff --git a/frontend/src/components/LLMSelection/LLMItem/index.jsx b/frontend/src/components/LLMSelection/LLMItem/index.jsx new file mode 100644 index 000000000..b6db5d130 --- /dev/null +++ b/frontend/src/components/LLMSelection/LLMItem/index.jsx @@ -0,0 +1,39 @@ +export default function LLMItem({ + name, + value, + image, + description, + checked, + onClick, +}) { + return ( +
onClick(value)} + className={`w-full hover:bg-white/10 p-2 rounded-md hover:cursor-pointer ${ + checked && "bg-white/10" + }`} + > + +
+ {`${name} +
+
{name}
+
+ {description} +
+
+
+
+ ); +} diff --git a/frontend/src/components/LLMSelection/LocalAiOptions/index.jsx b/frontend/src/components/LLMSelection/LocalAiOptions/index.jsx index 7a3852786..c7abd09af 100644 --- a/frontend/src/components/LLMSelection/LocalAiOptions/index.jsx +++ b/frontend/src/components/LLMSelection/LocalAiOptions/index.jsx @@ -71,12 +71,10 @@ export default function LocalAiOptions({ settings, showAlert = false }) {
-
+
+
+ + setBasePathValue(e.target.value)} + onBlur={() => setBasePath(basePathValue)} + /> +
+ +
+ + e.target.blur()} + defaultValue={settings?.OllamaLLMTokenLimit} + required={true} + autoComplete="off" + /> +
+
+
+ ); +} + +function OllamaLLMModelSelection({ settings, basePath = null }) { + const [customModels, setCustomModels] = useState([]); + const [loading, setLoading] = useState(true); + + useEffect(() => { + async function findCustomModels() { + if (!basePath) { + setCustomModels([]); + setLoading(false); + return; + } + setLoading(true); + const { models } = await System.customModels("ollama", null, basePath); + setCustomModels(models || []); + setLoading(false); + } + findCustomModels(); + }, [basePath]); + + if (loading || customModels.length == 0) { + return ( +
+ + +
+ ); + } + + return ( +
+ + +
+ ); +} diff --git a/frontend/src/components/LLMSelection/OpenAiOptions/index.jsx b/frontend/src/components/LLMSelection/OpenAiOptions/index.jsx index fd2f87665..cbd83edb9 100644 --- a/frontend/src/components/LLMSelection/OpenAiOptions/index.jsx +++ b/frontend/src/components/LLMSelection/OpenAiOptions/index.jsx @@ -6,7 +6,7 @@ export default function OpenAiOptions({ settings }) { const [openAIKey, setOpenAIKey] = useState(settings?.OpenAiKey); return ( - <> +
- +
); } @@ -87,7 +87,7 @@ function OpenAIModelSelection({ apiKey, settings }) { @@ -102,7 +102,7 @@ function OpenAIModelSelection({ apiKey, settings }) { diff --git a/frontend/src/components/Modals/MangeWorkspace/Documents/Directory/FileRow/index.jsx b/frontend/src/components/Modals/MangeWorkspace/Documents/Directory/FileRow/index.jsx index cd695dfcf..61187f55b 100644 --- a/frontend/src/components/Modals/MangeWorkspace/Documents/Directory/FileRow/index.jsx +++ b/frontend/src/components/Modals/MangeWorkspace/Documents/Directory/FileRow/index.jsx @@ -60,7 +60,7 @@ export default function FileRow({ selected ? "bg-sky-500/20" : "" } ${expanded ? "bg-sky-500/10" : ""}`}`} > -
+
-

+

{formatDate(item?.published)}

-

{getFileExtension(item.url)}

+

+ {getFileExtension(item.url)} +

{item?.cached && (
diff --git a/frontend/src/components/Modals/MangeWorkspace/Documents/Directory/index.jsx b/frontend/src/components/Modals/MangeWorkspace/Documents/Directory/index.jsx index 1dd83de9a..8a140a410 100644 --- a/frontend/src/components/Modals/MangeWorkspace/Documents/Directory/index.jsx +++ b/frontend/src/components/Modals/MangeWorkspace/Documents/Directory/index.jsx @@ -71,8 +71,8 @@ export default function Directory({
-

Name

-

Date

+

Name

+

Date

Kind

Cached

diff --git a/frontend/src/components/Modals/MangeWorkspace/Documents/WorkspaceDirectory/WorkspaceFileRow/index.jsx b/frontend/src/components/Modals/MangeWorkspace/Documents/WorkspaceDirectory/WorkspaceFileRow/index.jsx index ceb751558..99493dbaf 100644 --- a/frontend/src/components/Modals/MangeWorkspace/Documents/WorkspaceDirectory/WorkspaceFileRow/index.jsx +++ b/frontend/src/components/Modals/MangeWorkspace/Documents/WorkspaceDirectory/WorkspaceFileRow/index.jsx @@ -54,7 +54,7 @@ export default function WorkspaceFileRow({ className={`items-center transition-all duration-200 text-white/80 text-xs grid grid-cols-12 py-2 pl-3.5 pr-8 border-b border-white/20 hover:bg-sky-500/20 cursor-pointer ${isMovedItem ? "bg-green-800/40" : ""}`} > -
+
-

+

{formatDate(item?.published)}

-

{getFileExtension(item.url)}

+

+ {getFileExtension(item.url)} +

{item?.cached && (
diff --git a/frontend/src/components/Modals/MangeWorkspace/Documents/WorkspaceDirectory/index.jsx b/frontend/src/components/Modals/MangeWorkspace/Documents/WorkspaceDirectory/index.jsx index e1ec21dd4..12080b9b2 100644 --- a/frontend/src/components/Modals/MangeWorkspace/Documents/WorkspaceDirectory/index.jsx +++ b/frontend/src/components/Modals/MangeWorkspace/Documents/WorkspaceDirectory/index.jsx @@ -26,8 +26,8 @@ export default function WorkspaceDirectory({
-

Name

-

Date

+

Name

+

Date

Kind

Cached

@@ -55,8 +55,8 @@ export default function WorkspaceDirectory({ }`} >
-

Name

-

Date

+

Name

+

Date

Kind

Cached

diff --git a/frontend/src/components/PrivateRoute/index.jsx b/frontend/src/components/PrivateRoute/index.jsx index 464064632..165141bbb 100644 --- a/frontend/src/components/PrivateRoute/index.jsx +++ b/frontend/src/components/PrivateRoute/index.jsx @@ -89,7 +89,7 @@ export function AdminRoute({ Component }) { if (isAuthd === null) return ; if (shouldRedirectToOnboarding) { - return ; + return ; } const user = userFromStorage(); @@ -110,7 +110,7 @@ export function ManagerRoute({ Component }) { if (isAuthd === null) return ; if (shouldRedirectToOnboarding) { - return ; + return ; } const user = userFromStorage(); diff --git a/frontend/src/components/VectorDBOption/index.jsx b/frontend/src/components/VectorDBOption/index.jsx deleted file mode 100644 index 0dbe97c19..000000000 --- a/frontend/src/components/VectorDBOption/index.jsx +++ /dev/null @@ -1,39 +0,0 @@ -import React from "react"; - -export default function VectorDBOption({ - name, - link, - description, - value, - image, - checked = false, - onClick, -}) { - return ( -
onClick(value)}> - - -
- ); -} diff --git a/frontend/src/components/VectorDBSelection/ChromaDBOptions/index.jsx b/frontend/src/components/VectorDBSelection/ChromaDBOptions/index.jsx new file mode 100644 index 000000000..ae7af68fb --- /dev/null +++ b/frontend/src/components/VectorDBSelection/ChromaDBOptions/index.jsx @@ -0,0 +1,51 @@ +export default function ChromaDBOptions({ settings }) { + return ( +
+
+
+ + +
+ +
+ + +
+ +
+ + +
+
+
+ ); +} diff --git a/frontend/src/components/VectorDBSelection/LanceDBOptions/index.jsx b/frontend/src/components/VectorDBSelection/LanceDBOptions/index.jsx new file mode 100644 index 000000000..942a3666d --- /dev/null +++ b/frontend/src/components/VectorDBSelection/LanceDBOptions/index.jsx @@ -0,0 +1,9 @@ +export default function LanceDBOptions() { + return ( +
+

+ There is no configuration needed for LanceDB. +

+
+ ); +} diff --git a/frontend/src/components/VectorDBSelection/PineconeDBOptions/index.jsx b/frontend/src/components/VectorDBSelection/PineconeDBOptions/index.jsx new file mode 100644 index 000000000..5491f758c --- /dev/null +++ b/frontend/src/components/VectorDBSelection/PineconeDBOptions/index.jsx @@ -0,0 +1,55 @@ +export default function PineconeDBOptions({ settings }) { + return ( +
+
+
+ + +
+ +
+ + +
+ +
+ + +
+
+
+ ); +} diff --git a/frontend/src/components/VectorDBSelection/QDrantDBOptions/index.jsx b/frontend/src/components/VectorDBSelection/QDrantDBOptions/index.jsx new file mode 100644 index 000000000..e1e9d90f6 --- /dev/null +++ b/frontend/src/components/VectorDBSelection/QDrantDBOptions/index.jsx @@ -0,0 +1,38 @@ +export default function QDrantDBOptions({ settings }) { + return ( +
+
+
+ + +
+ +
+ + +
+
+
+ ); +} diff --git a/frontend/src/components/VectorDBSelection/VectorDBItem/index.jsx b/frontend/src/components/VectorDBSelection/VectorDBItem/index.jsx new file mode 100644 index 000000000..47f067f5c --- /dev/null +++ b/frontend/src/components/VectorDBSelection/VectorDBItem/index.jsx @@ -0,0 +1,37 @@ +export default function VectorDBItem({ + name, + value, + image, + description, + checked, + onClick, +}) { + return ( +
onClick(value)} + className={`w-full hover:bg-white/10 p-2 rounded-md hover:cursor-pointer ${ + checked ? "bg-white/10" : "" + }`} + > + +
+ {`${name} +
+
{name}
+
{description}
+
+
+
+ ); +} diff --git a/frontend/src/components/VectorDBSelection/WeaviateDBOptions/index.jsx b/frontend/src/components/VectorDBSelection/WeaviateDBOptions/index.jsx new file mode 100644 index 000000000..5d7494ed1 --- /dev/null +++ b/frontend/src/components/VectorDBSelection/WeaviateDBOptions/index.jsx @@ -0,0 +1,38 @@ +export default function WeaviateDBOptions({ settings }) { + return ( +
+
+
+ + +
+ +
+ + +
+
+
+ ); +} diff --git a/frontend/src/media/illustrations/create-workspace.png b/frontend/src/media/illustrations/create-workspace.png new file mode 100644 index 000000000..8e31174e5 Binary files /dev/null and b/frontend/src/media/illustrations/create-workspace.png differ diff --git a/frontend/src/media/llmprovider/gemini.png b/frontend/src/media/llmprovider/gemini.png new file mode 100644 index 000000000..aa81cfd86 Binary files /dev/null and b/frontend/src/media/llmprovider/gemini.png differ diff --git a/frontend/src/media/llmprovider/ollama.png b/frontend/src/media/llmprovider/ollama.png new file mode 100644 index 000000000..2a898a6eb Binary files /dev/null and b/frontend/src/media/llmprovider/ollama.png differ diff --git a/frontend/src/pages/GeneralSettings/EmbeddingPreference/index.jsx b/frontend/src/pages/GeneralSettings/EmbeddingPreference/index.jsx index 1abf3a4b4..ddcb81319 100644 --- a/frontend/src/pages/GeneralSettings/EmbeddingPreference/index.jsx +++ b/frontend/src/pages/GeneralSettings/EmbeddingPreference/index.jsx @@ -8,25 +8,28 @@ import OpenAiLogo from "@/media/llmprovider/openai.png"; import AzureOpenAiLogo from "@/media/llmprovider/azure.png"; import LocalAiLogo from "@/media/llmprovider/localai.png"; import PreLoader from "@/components/Preloader"; -import LLMProviderOption from "@/components/LLMSelection/LLMProviderOption"; import ChangeWarningModal from "@/components/ChangeWarning"; import OpenAiOptions from "@/components/EmbeddingSelection/OpenAiOptions"; import AzureAiOptions from "@/components/EmbeddingSelection/AzureAiOptions"; import LocalAiOptions from "@/components/EmbeddingSelection/LocalAiOptions"; import NativeEmbeddingOptions from "@/components/EmbeddingSelection/NativeEmbeddingOptions"; +import EmbedderItem from "@/components/EmbeddingSelection/EmbedderItem"; +import { MagnifyingGlass } from "@phosphor-icons/react"; export default function GeneralEmbeddingPreference() { const [saving, setSaving] = useState(false); const [hasChanges, setHasChanges] = useState(false); const [hasEmbeddings, setHasEmbeddings] = useState(false); - const [embeddingChoice, setEmbeddingChoice] = useState("openai"); const [settings, setSettings] = useState(null); const [loading, setLoading] = useState(true); + const [searchQuery, setSearchQuery] = useState(""); + const [filteredEmbedders, setFilteredEmbedders] = useState([]); + const [selectedEmbedder, setSelectedEmbedder] = useState(null); const handleSubmit = async (e) => { e.preventDefault(); if ( - embeddingChoice !== settings?.EmbeddingEngine && + selectedEmbedder !== settings?.EmbeddingEngine && hasChanges && hasEmbeddings ) { @@ -38,18 +41,18 @@ export default function GeneralEmbeddingPreference() { const handleSaveSettings = async () => { setSaving(true); - const data = new FormData(document.getElementById("embedding-form")); + const form = document.getElementById("embedding-form"); const settingsData = {}; - for (let [key, value] of data.entries()) { - settingsData[key] = value; - } + const formData = new FormData(form); + settingsData.EmbeddingEngine = selectedEmbedder; + for (var [key, value] of formData.entries()) settingsData[key] = value; const { error } = await System.updateSystem(settingsData); if (error) { - showToast(`Failed to save LLM settings: ${error}`, "error"); + showToast(`Failed to save embedding settings: ${error}`, "error"); setHasChanges(true); } else { - showToast("LLM preferences saved successfully.", "success"); + showToast("Embedding preferences saved successfully.", "success"); setHasChanges(false); } setSaving(false); @@ -57,7 +60,7 @@ export default function GeneralEmbeddingPreference() { }; const updateChoice = (selection) => { - setEmbeddingChoice(selection); + setSelectedEmbedder(selection); setHasChanges(true); }; @@ -65,13 +68,52 @@ export default function GeneralEmbeddingPreference() { async function fetchKeys() { const _settings = await System.keys(); setSettings(_settings); - setEmbeddingChoice(_settings?.EmbeddingEngine || "openai"); + setSelectedEmbedder(_settings?.EmbeddingEngine || "native"); setHasEmbeddings(_settings?.HasExistingEmbeddings || false); setLoading(false); } fetchKeys(); }, []); + const EMBEDDERS = [ + { + name: "AnythingLLM Embedder", + value: "native", + logo: AnythingLLMIcon, + options: , + description: + "Use the built-in embedding engine for AnythingLLM. Zero setup!", + }, + { + name: "OpenAI", + value: "openai", + logo: OpenAiLogo, + options: , + description: "The standard option for most non-commercial use.", + }, + { + name: "Azure OpenAI", + value: "azure", + logo: AzureOpenAiLogo, + options: , + description: "The enterprise option of OpenAI hosted on Azure services.", + }, + { + name: "Local AI", + value: "localai", + logo: LocalAiLogo, + options: , + description: "Run embedding models locally on your own machine.", + }, + ]; + + useEffect(() => { + const filtered = EMBEDDERS.filter((embedder) => + embedder.name.toLowerCase().includes(searchQuery.toLowerCase()) + ); + setFilteredEmbedders(filtered); + }, [searchQuery, selectedEmbedder]); + return (
setHasChanges(true)} className="flex w-full" >
@@ -132,59 +173,52 @@ export default function GeneralEmbeddingPreference() {
Embedding Providers
-
- - - - - -
-
- {embeddingChoice === "native" && } - {embeddingChoice === "openai" && ( - - )} - {embeddingChoice === "azure" && ( - - )} - {embeddingChoice === "localai" && ( - - )} +
+
+
+
+ + setSearchQuery(e.target.value)} + autoComplete="off" + onKeyDown={(e) => { + if (e.key === "Enter") e.preventDefault(); + }} + /> +
+
+
+ {filteredEmbedders.map((embedder) => { + return ( + updateChoice(embedder.value)} + /> + ); + })} +
+
+
setHasChanges(true)} + className="mt-4 flex flex-col gap-y-1" + > + {selectedEmbedder && + EMBEDDERS.find( + (embedder) => embedder.value === selectedEmbedder + )?.options} +
diff --git a/frontend/src/pages/GeneralSettings/LLMPreference/index.jsx b/frontend/src/pages/GeneralSettings/LLMPreference/index.jsx index 1c18d1ff1..d72cf3c2b 100644 --- a/frontend/src/pages/GeneralSettings/LLMPreference/index.jsx +++ b/frontend/src/pages/GeneralSettings/LLMPreference/index.jsx @@ -7,31 +7,44 @@ import AnythingLLMIcon from "@/media/logo/anything-llm-icon.png"; import OpenAiLogo from "@/media/llmprovider/openai.png"; import AzureOpenAiLogo from "@/media/llmprovider/azure.png"; import AnthropicLogo from "@/media/llmprovider/anthropic.png"; +import GeminiLogo from "@/media/llmprovider/gemini.png"; +import OllamaLogo from "@/media/llmprovider/ollama.png"; import LMStudioLogo from "@/media/llmprovider/lmstudio.png"; import LocalAiLogo from "@/media/llmprovider/localai.png"; import PreLoader from "@/components/Preloader"; -import LLMProviderOption from "@/components/LLMSelection/LLMProviderOption"; import OpenAiOptions from "@/components/LLMSelection/OpenAiOptions"; import AzureAiOptions from "@/components/LLMSelection/AzureAiOptions"; import AnthropicAiOptions from "@/components/LLMSelection/AnthropicAiOptions"; import LMStudioOptions from "@/components/LLMSelection/LMStudioOptions"; import LocalAiOptions from "@/components/LLMSelection/LocalAiOptions"; import NativeLLMOptions from "@/components/LLMSelection/NativeLLMOptions"; +import GeminiLLMOptions from "@/components/LLMSelection/GeminiLLMOptions"; +import OllamaLLMOptions from "@/components/LLMSelection/OllamaLLMOptions"; +import LLMItem from "@/components/LLMSelection/LLMItem"; +import { MagnifyingGlass } from "@phosphor-icons/react"; export default function GeneralLLMPreference() { const [saving, setSaving] = useState(false); const [hasChanges, setHasChanges] = useState(false); - const [llmChoice, setLLMChoice] = useState("openai"); const [settings, setSettings] = useState(null); const [loading, setLoading] = useState(true); + const [searchQuery, setSearchQuery] = useState(""); + const [filteredLLMs, setFilteredLLMs] = useState([]); + const [selectedLLM, setSelectedLLM] = useState(null); + + const isHosted = window.location.hostname.includes("useanything.com"); + const handleSubmit = async (e) => { e.preventDefault(); - setSaving(true); + const form = e.target; const data = {}; - const form = new FormData(e.target); - for (var [key, value] of form.entries()) data[key] = value; + const formData = new FormData(form); + data.LLMProvider = selectedLLM; + for (var [key, value] of formData.entries()) data[key] = value; const { error } = await System.updateSystem(data); + setSaving(true); + if (error) { showToast(`Failed to save LLM settings: ${error}`, "error"); } else { @@ -42,7 +55,7 @@ export default function GeneralLLMPreference() { }; const updateLLMChoice = (selection) => { - setLLMChoice(selection); + setSelectedLLM(selection); setHasChanges(true); }; @@ -50,12 +63,80 @@ export default function GeneralLLMPreference() { async function fetchKeys() { const _settings = await System.keys(); setSettings(_settings); - setLLMChoice(_settings?.LLMProvider); + setSelectedLLM(_settings?.LLMProvider); setLoading(false); } fetchKeys(); }, []); + useEffect(() => { + const filtered = LLMS.filter((llm) => + llm.name.toLowerCase().includes(searchQuery.toLowerCase()) + ); + setFilteredLLMs(filtered); + }, [searchQuery, selectedLLM]); + + const LLMS = [ + { + name: "OpenAI", + value: "openai", + logo: OpenAiLogo, + options: , + description: "The standard option for most non-commercial use.", + }, + { + name: "Azure OpenAI", + value: "azure", + logo: AzureOpenAiLogo, + options: , + description: "The enterprise option of OpenAI hosted on Azure services.", + }, + { + name: "Anthropic", + value: "anthropic", + logo: AnthropicLogo, + options: , + description: "A friendly AI Assistant hosted by Anthropic.", + }, + { + name: "Gemini", + value: "gemini", + logo: GeminiLogo, + options: , + description: "Google's largest and most capable AI model", + }, + { + name: "Ollama", + value: "ollama", + logo: OllamaLogo, + options: , + description: "Run LLMs locally on your own machine.", + }, + { + name: "LM Studio", + value: "lmstudio", + logo: LMStudioLogo, + options: , + description: + "Discover, download, and run thousands of cutting edge LLMs in a few clicks.", + }, + { + name: "Local AI", + value: "localai", + logo: LocalAiLogo, + options: , + description: "Run LLMs locally on your own machine.", + }, + { + name: "Native", + value: "native", + logo: AnythingLLMIcon, + options: , + description: + "Use a downloaded custom Llama model for chatting on this AnythingLLM instance.", + }, + ]; + return (
{!isMobile && } @@ -74,11 +155,7 @@ export default function GeneralLLMPreference() { className="relative md:ml-[2px] md:mr-[16px] md:my-[16px] md:rounded-[26px] bg-main-gradient w-full h-full overflow-y-scroll border-4 border-accent" > {isMobile && } -
setHasChanges(true)} - className="flex w-full" - > +
@@ -105,83 +182,51 @@ export default function GeneralLLMPreference() {
LLM Providers
-
- - - - - - - {!window.location.hostname.includes("useanything.com") && ( - - )} -
-
- {llmChoice === "openai" && ( - - )} - {llmChoice === "azure" && ( - - )} - {llmChoice === "anthropic" && ( - - )} - {llmChoice === "lmstudio" && ( - - )} - {llmChoice === "localai" && ( - - )} - {llmChoice === "native" && ( - - )} +
+
+
+
+ + setSearchQuery(e.target.value)} + autoComplete="off" + onKeyDown={(e) => { + if (e.key === "Enter") e.preventDefault(); + }} + /> +
+
+
+ {filteredLLMs.map((llm) => { + if (llm.value === "native" && isHosted) return null; + return ( + updateLLMChoice(llm.value)} + /> + ); + })} +
+
+
setHasChanges(true)} + className="mt-4 flex flex-col gap-y-1" + > + {selectedLLM && + LLMS.find((llm) => llm.value === selectedLLM)?.options} +
diff --git a/frontend/src/pages/GeneralSettings/VectorDatabase/index.jsx b/frontend/src/pages/GeneralSettings/VectorDatabase/index.jsx index 1635fef8b..9ef9cff2d 100644 --- a/frontend/src/pages/GeneralSettings/VectorDatabase/index.jsx +++ b/frontend/src/pages/GeneralSettings/VectorDatabase/index.jsx @@ -9,36 +9,86 @@ import LanceDbLogo from "@/media/vectordbs/lancedb.png"; import WeaviateLogo from "@/media/vectordbs/weaviate.png"; import QDrantLogo from "@/media/vectordbs/qdrant.png"; import PreLoader from "@/components/Preloader"; -import VectorDBOption from "@/components/VectorDBOption"; import ChangeWarningModal from "@/components/ChangeWarning"; +import { MagnifyingGlass } from "@phosphor-icons/react"; +import LanceDBOptions from "@/components/VectorDBSelection/LanceDBOptions"; +import ChromaDBOptions from "@/components/VectorDBSelection/ChromaDBOptions"; +import PineconeDBOptions from "@/components/VectorDBSelection/PineconeDBOptions"; +import QDrantDBOptions from "@/components/VectorDBSelection/QDrantDBOptions"; +import WeaviateDBOptions from "@/components/VectorDBSelection/WeaviateDBOptions"; +import VectorDBItem from "@/components/VectorDBSelection/VectorDBItem"; export default function GeneralVectorDatabase() { const [saving, setSaving] = useState(false); const [hasChanges, setHasChanges] = useState(false); const [hasEmbeddings, setHasEmbeddings] = useState(false); - const [vectorDB, setVectorDB] = useState("lancedb"); const [settings, setSettings] = useState({}); const [loading, setLoading] = useState(true); + const [searchQuery, setSearchQuery] = useState(""); + const [filteredVDBs, setFilteredVDBs] = useState([]); + const [selectedVDB, setSelectedVDB] = useState(null); useEffect(() => { async function fetchKeys() { const _settings = await System.keys(); + console.log(_settings); setSettings(_settings); - setVectorDB(_settings?.VectorDB || "lancedb"); + setSelectedVDB(_settings?.VectorDB || "lancedb"); setHasEmbeddings(_settings?.HasExistingEmbeddings || false); setLoading(false); } fetchKeys(); }, []); + const VECTOR_DBS = [ + { + name: "LanceDB", + value: "lancedb", + logo: LanceDbLogo, + options: , + description: + "100% local vector DB that runs on the same instance as AnythingLLM.", + }, + { + name: "Chroma", + value: "chroma", + logo: ChromaLogo, + options: , + description: + "Open source vector database you can host yourself or on the cloud.", + }, + { + name: "Pinecone", + value: "pinecone", + logo: PineconeLogo, + options: , + description: "100% cloud-based vector database for enterprise use cases.", + }, + { + name: "QDrant", + value: "qdrant", + logo: QDrantLogo, + options: , + description: "Open source local and distributed cloud vector database.", + }, + { + name: "Weaviate", + value: "weaviate", + logo: WeaviateLogo, + options: , + description: + "Open source local and cloud hosted multi-modal vector database.", + }, + ]; + const updateVectorChoice = (selection) => { setHasChanges(true); - setVectorDB(selection); + setSelectedVDB(selection); }; const handleSubmit = async (e) => { e.preventDefault(); - if (vectorDB !== settings?.VectorDB && hasChanges && hasEmbeddings) { + if (selectedVDB !== settings?.VectorDB && hasChanges && hasEmbeddings) { document.getElementById("confirmation-modal")?.showModal(); } else { await handleSaveSettings(); @@ -47,24 +97,31 @@ export default function GeneralVectorDatabase() { const handleSaveSettings = async () => { setSaving(true); - const data = new FormData(document.getElementById("vectordb-form")); + const form = document.getElementById("vectordb-form"); const settingsData = {}; - for (let [key, value] of data.entries()) { - settingsData[key] = value; - } + const formData = new FormData(form); + settingsData.VectorDB = selectedVDB; + for (var [key, value] of formData.entries()) settingsData[key] = value; const { error } = await System.updateSystem(settingsData); if (error) { - showToast(`Failed to save LLM settings: ${error}`, "error"); + showToast(`Failed to save vector database settings: ${error}`, "error"); setHasChanges(true); } else { - showToast("LLM preferences saved successfully.", "success"); + showToast("Vector database preferences saved successfully.", "success"); setHasChanges(false); } setSaving(false); document.getElementById("confirmation-modal")?.close(); }; + useEffect(() => { + const filtered = VECTOR_DBS.filter((vdb) => + vdb.name.toLowerCase().includes(searchQuery.toLowerCase()) + ); + setFilteredVDBs(filtered); + }, [searchQuery, selectedVDB]); + return (
setHasChanges(true)} className="flex w-full" >
@@ -119,236 +175,52 @@ export default function GeneralVectorDatabase() {
Select your preferred vector database provider
-
- - - - - - -
-
- {vectorDB === "pinecone" && ( - <> -
- - +
+
+
+ -
- -
- { + e.preventDefault(); + setSearchQuery(e.target.value); + }} autoComplete="off" - spellCheck={false} + onKeyDown={(e) => { + if (e.key === "Enter") e.preventDefault(); + }} />
- -
- - -
- - )} - - {vectorDB === "chroma" && ( - <> -
- - -
- -
- - -
- -
- - -
- - )} - - {vectorDB === "lancedb" && ( -
-

- There is no configuration needed for LanceDB. -

- )} - - {vectorDB === "qdrant" && ( - <> -
- - + {filteredVDBs.map((vdb) => ( + updateVectorChoice(vdb.value)} /> -
- -
- - -
- - )} - - {vectorDB === "weaviate" && ( - <> -
- - -
- -
- - -
- - )} + ))} +
+
+
setHasChanges(true)} + className="mt-4 flex flex-col gap-y-1" + > + {selectedVDB && + VECTOR_DBS.find((vdb) => vdb.value === selectedVDB) + ?.options} +
diff --git a/frontend/src/pages/OnboardingFlow/OnboardingModal/Steps/AppearanceSetup/index.jsx b/frontend/src/pages/OnboardingFlow/OnboardingModal/Steps/AppearanceSetup/index.jsx deleted file mode 100644 index 30e87b0ac..000000000 --- a/frontend/src/pages/OnboardingFlow/OnboardingModal/Steps/AppearanceSetup/index.jsx +++ /dev/null @@ -1,145 +0,0 @@ -import React, { memo, useEffect, useState } from "react"; -import System from "@/models/system"; -import AnythingLLM from "@/media/logo/anything-llm.png"; -import useLogo from "@/hooks/useLogo"; -import { Plus } from "@phosphor-icons/react"; -import showToast from "@/utils/toast"; - -function AppearanceSetup({ prevStep, nextStep }) { - const { logo: _initLogo, setLogo: _setLogo } = useLogo(); - const [logo, setLogo] = useState(""); - const [isDefaultLogo, setIsDefaultLogo] = useState(true); - - useEffect(() => { - async function logoInit() { - setLogo(_initLogo || ""); - const _isDefaultLogo = await System.isDefaultLogo(); - setIsDefaultLogo(_isDefaultLogo); - } - logoInit(); - }, [_initLogo]); - - const handleFileUpload = async (event) => { - const file = event.target.files[0]; - if (!file) return false; - - const objectURL = URL.createObjectURL(file); - setLogo(objectURL); - - const formData = new FormData(); - formData.append("logo", file); - const { success, error } = await System.uploadLogo(formData); - if (!success) { - showToast(`Failed to upload logo: ${error}`, "error"); - setLogo(_initLogo); - return; - } - - const logoURL = await System.fetchLogo(); - _setLogo(logoURL); - - showToast("Image uploaded successfully.", "success"); - setIsDefaultLogo(false); - }; - - const handleRemoveLogo = async () => { - setLogo(""); - setIsDefaultLogo(true); - - const { success, error } = await System.removeCustomLogo(); - if (!success) { - console.error("Failed to remove logo:", error); - showToast(`Failed to remove logo: ${error}`, "error"); - const logoURL = await System.fetchLogo(); - setLogo(logoURL); - setIsDefaultLogo(false); - return; - } - - const logoURL = await System.fetchLogo(); - _setLogo(logoURL); - - showToast("Image successfully removed.", "success"); - }; - - return ( -
-
-
-

Custom Logo

-

- Upload your custom logo to make your chatbot yours. -

-
-
- (e.target.src = AnythingLLM)} - /> -
- - -
-
-
-
- -
- - -
-
-
- ); -} -export default memo(AppearanceSetup); diff --git a/frontend/src/pages/OnboardingFlow/OnboardingModal/Steps/CreateFirstWorkspace/index.jsx b/frontend/src/pages/OnboardingFlow/OnboardingModal/Steps/CreateFirstWorkspace/index.jsx deleted file mode 100644 index d2624ef6a..000000000 --- a/frontend/src/pages/OnboardingFlow/OnboardingModal/Steps/CreateFirstWorkspace/index.jsx +++ /dev/null @@ -1,68 +0,0 @@ -import React, { memo } from "react"; -import { useNavigate } from "react-router-dom"; -import paths from "@/utils/paths"; -import Workspace from "@/models/workspace"; - -function CreateFirstWorkspace({ prevStep }) { - const navigate = useNavigate(); - - const handleCreate = async (e) => { - e.preventDefault(); - const form = new FormData(e.target); - const { workspace, error } = await Workspace.new({ - name: form.get("name"), - onboardingComplete: true, - }); - if (!!workspace) { - navigate(paths.home()); - } else { - alert(error); - } - }; - - return ( -
-
-
-
-
-
- - -
-
-
-
-
- - -
-
-
- ); -} -export default memo(CreateFirstWorkspace); diff --git a/frontend/src/pages/OnboardingFlow/OnboardingModal/Steps/EmbeddingSelection/index.jsx b/frontend/src/pages/OnboardingFlow/OnboardingModal/Steps/EmbeddingSelection/index.jsx deleted file mode 100644 index 1f44c463b..000000000 --- a/frontend/src/pages/OnboardingFlow/OnboardingModal/Steps/EmbeddingSelection/index.jsx +++ /dev/null @@ -1,136 +0,0 @@ -import React, { memo, useEffect, useState } from "react"; -import AnythingLLMIcon from "@/media/logo/anything-llm-icon.png"; -import OpenAiLogo from "@/media/llmprovider/openai.png"; -import AzureOpenAiLogo from "@/media/llmprovider/azure.png"; -import LocalAiLogo from "@/media/llmprovider/localai.png"; -import System from "@/models/system"; -import PreLoader from "@/components/Preloader"; -import LLMProviderOption from "@/components/LLMSelection/LLMProviderOption"; -import OpenAiOptions from "@/components/EmbeddingSelection/OpenAiOptions"; -import AzureAiOptions from "@/components/EmbeddingSelection/AzureAiOptions"; -import LocalAiOptions from "@/components/EmbeddingSelection/LocalAiOptions"; -import NativeEmbeddingOptions from "@/components/EmbeddingSelection/NativeEmbeddingOptions"; - -function EmbeddingSelection({ nextStep, prevStep, currentStep }) { - const [embeddingChoice, setEmbeddingChoice] = useState("native"); - const [settings, setSettings] = useState(null); - const [loading, setLoading] = useState(true); - const updateChoice = (selection) => { - setEmbeddingChoice(selection); - }; - - useEffect(() => { - async function fetchKeys() { - const _settings = await System.keys(); - setSettings(_settings); - setEmbeddingChoice(_settings?.EmbeddingEngine || "native"); - setLoading(false); - } - fetchKeys(); - }, [currentStep]); - - const handleSubmit = async (e) => { - e.preventDefault(); - const form = e.target; - const data = {}; - const formData = new FormData(form); - for (var [key, value] of formData.entries()) data[key] = value; - const { error } = await System.updateSystem(data); - if (error) { - alert(`Failed to save LLM settings: ${error}`, "error"); - return; - } - nextStep("vector_database"); - return; - }; - - if (loading) - return ( -
- -
- ); - - return ( -
-
-
-
- Embedding Provider -
-
- - - - - -
-
- {embeddingChoice === "native" && } - {embeddingChoice === "openai" && ( - - )} - {embeddingChoice === "azure" && ( - - )} - {embeddingChoice === "localai" && ( - - )} -
-
-
- - -
-
-
- ); -} - -export default memo(EmbeddingSelection); diff --git a/frontend/src/pages/OnboardingFlow/OnboardingModal/Steps/LLMSelection/index.jsx b/frontend/src/pages/OnboardingFlow/OnboardingModal/Steps/LLMSelection/index.jsx deleted file mode 100644 index bb87486ba..000000000 --- a/frontend/src/pages/OnboardingFlow/OnboardingModal/Steps/LLMSelection/index.jsx +++ /dev/null @@ -1,157 +0,0 @@ -import React, { memo, useEffect, useState } from "react"; -import AnythingLLMIcon from "@/media/logo/anything-llm-icon.png"; -import OpenAiLogo from "@/media/llmprovider/openai.png"; -import AzureOpenAiLogo from "@/media/llmprovider/azure.png"; -import AnthropicLogo from "@/media/llmprovider/anthropic.png"; -import LMStudioLogo from "@/media/llmprovider/lmstudio.png"; -import LocalAiLogo from "@/media/llmprovider/localai.png"; -import System from "@/models/system"; -import PreLoader from "@/components/Preloader"; -import LLMProviderOption from "@/components/LLMSelection/LLMProviderOption"; -import OpenAiOptions from "@/components/LLMSelection/OpenAiOptions"; -import AzureAiOptions from "@/components/LLMSelection/AzureAiOptions"; -import AnthropicAiOptions from "@/components/LLMSelection/AnthropicAiOptions"; -import LMStudioOptions from "@/components/LLMSelection/LMStudioOptions"; -import LocalAiOptions from "@/components/LLMSelection/LocalAiOptions"; -import NativeLLMOptions from "@/components/LLMSelection/NativeLLMOptions"; - -function LLMSelection({ nextStep, prevStep, currentStep }) { - const [llmChoice, setLLMChoice] = useState("openai"); - const [settings, setSettings] = useState(null); - const [loading, setLoading] = useState(true); - - const updateLLMChoice = (selection) => { - setLLMChoice(selection); - }; - - useEffect(() => { - async function fetchKeys() { - const _settings = await System.keys(); - setSettings(_settings); - setLLMChoice(_settings?.LLMProvider || "openai"); - setLoading(false); - } - - if (currentStep === "llm_preference") { - fetchKeys(); - } - }, []); - - const handleSubmit = async (e) => { - e.preventDefault(); - const form = e.target; - const data = {}; - const formData = new FormData(form); - for (var [key, value] of formData.entries()) data[key] = value; - const { error } = await System.updateSystem(data); - if (error) { - alert(`Failed to save LLM settings: ${error}`, "error"); - return; - } - nextStep("embedding_preferences"); - }; - - if (loading) - return ( -
- -
- ); - - return ( -
-
-
-
- LLM Providers -
-
- - - - - - - -
-
- {llmChoice === "openai" && } - {llmChoice === "azure" && } - {llmChoice === "anthropic" && ( - - )} - {llmChoice === "lmstudio" && ( - - )} - {llmChoice === "localai" && } - {llmChoice === "native" && } -
-
-
- - -
-
-
- ); -} - -export default memo(LLMSelection); diff --git a/frontend/src/pages/OnboardingFlow/OnboardingModal/Steps/MultiUserSetup/index.jsx b/frontend/src/pages/OnboardingFlow/OnboardingModal/Steps/MultiUserSetup/index.jsx deleted file mode 100644 index 71310abfc..000000000 --- a/frontend/src/pages/OnboardingFlow/OnboardingModal/Steps/MultiUserSetup/index.jsx +++ /dev/null @@ -1,117 +0,0 @@ -import React, { useState, memo } from "react"; -import System from "@/models/system"; -import { AUTH_TIMESTAMP, AUTH_TOKEN, AUTH_USER } from "@/utils/constants"; -import debounce from "lodash.debounce"; - -// Multi-user mode step -function MultiUserSetup({ nextStep, prevStep }) { - const [username, setUsername] = useState(""); - const [password, setPassword] = useState(""); - - const handleSubmit = async (e) => { - e.preventDefault(); - const form = e.target; - const formData = new FormData(form); - const data = { - username: formData.get("username"), - password: formData.get("password"), - }; - const { success, error } = await System.setupMultiUser(data); - if (!success) { - alert(error); - return; - } - - // Auto-request token with credentials that was just set so they - // are not redirected to login after completion. - const { user, token } = await System.requestToken(data); - window.localStorage.setItem(AUTH_USER, JSON.stringify(user)); - window.localStorage.setItem(AUTH_TOKEN, token); - window.localStorage.removeItem(AUTH_TIMESTAMP); - - nextStep("data_handling"); - }; - - const setNewUsername = (e) => setUsername(e.target.value); - const setNewPassword = (e) => setPassword(e.target.value); - const handleUsernameChange = debounce(setNewUsername, 500); - const handlePasswordChange = debounce(setNewPassword, 500); - return ( -
-
-
-
-
-
- - -
-
- - -
-

- Username must be at least 6 characters long. Password must be at - least 8 characters long. -

-
-
-
-
-
- By default, you will be the only admin. As an admin you will need to - create accounts for all new users or admins. Do not lose your - password as only admins can reset passwords. -
-
- - -
-
-
-
- ); -} -export default memo(MultiUserSetup); diff --git a/frontend/src/pages/OnboardingFlow/OnboardingModal/Steps/PasswordProtection/index.jsx b/frontend/src/pages/OnboardingFlow/OnboardingModal/Steps/PasswordProtection/index.jsx deleted file mode 100644 index 4504288e6..000000000 --- a/frontend/src/pages/OnboardingFlow/OnboardingModal/Steps/PasswordProtection/index.jsx +++ /dev/null @@ -1,103 +0,0 @@ -import React, { memo, useState } from "react"; -import System from "@/models/system"; -import { AUTH_TIMESTAMP, AUTH_TOKEN, AUTH_USER } from "@/utils/constants"; -import debounce from "lodash.debounce"; - -function PasswordProtection({ nextStep, prevStep }) { - const [password, setPassword] = useState(""); - const handleSubmit = async (e) => { - e.preventDefault(); - const form = e.target; - const formData = new FormData(form); - const { error } = await System.updateSystemPassword({ - usePassword: true, - newPassword: formData.get("password"), - }); - - if (error) { - alert(`Failed to set password: ${error}`, "error"); - return; - } - - // Auto-request token with password that was just set so they - // are not redirected to login after completion. - const { token } = await System.requestToken({ - password: formData.get("password"), - }); - window.localStorage.removeItem(AUTH_USER); - window.localStorage.removeItem(AUTH_TIMESTAMP); - window.localStorage.setItem(AUTH_TOKEN, token); - - nextStep("data_handling"); - return; - }; - - const handleSkip = () => { - nextStep("data_handling"); - }; - - const setNewPassword = (e) => setPassword(e.target.value); - const handlePasswordChange = debounce(setNewPassword, 500); - return ( -
-
-
-
-
-
- -

- must be at least 8 characters. -

-
- -
-
-
-
- - -
- - -
-
-
-
- ); -} -export default memo(PasswordProtection); diff --git a/frontend/src/pages/OnboardingFlow/OnboardingModal/Steps/UserModeSelection/index.jsx b/frontend/src/pages/OnboardingFlow/OnboardingModal/Steps/UserModeSelection/index.jsx deleted file mode 100644 index b78c32533..000000000 --- a/frontend/src/pages/OnboardingFlow/OnboardingModal/Steps/UserModeSelection/index.jsx +++ /dev/null @@ -1,47 +0,0 @@ -import React, { memo } from "react"; - -// How many people will be using your instance step -function UserModeSelection({ nextStep, prevStep }) { - const justMeClicked = () => { - nextStep("password_protection"); - }; - - const myTeamClicked = () => { - nextStep("multi_user_mode"); - }; - - return ( -
-
-
- How many people will be using your instance? -
-
- - -
-
-
- -
-
- ); -} - -export default memo(UserModeSelection); diff --git a/frontend/src/pages/OnboardingFlow/OnboardingModal/Steps/UserQuestionnaire/index.jsx b/frontend/src/pages/OnboardingFlow/OnboardingModal/Steps/UserQuestionnaire/index.jsx deleted file mode 100644 index a0cb97fd7..000000000 --- a/frontend/src/pages/OnboardingFlow/OnboardingModal/Steps/UserQuestionnaire/index.jsx +++ /dev/null @@ -1,240 +0,0 @@ -import { COMPLETE_QUESTIONNAIRE } from "@/utils/constants"; -import paths from "@/utils/paths"; -import { CheckCircle, Circle } from "@phosphor-icons/react"; -import React, { memo } from "react"; - -async function sendQuestionnaire({ email, useCase, comment }) { - if (import.meta.env.DEV) return; - return fetch(`https://onboarding-wxich7363q-uc.a.run.app`, { - method: "POST", - body: JSON.stringify({ - email, - useCase, - comment, - sourceId: "0VRjqHh6Vukqi0x0Vd0n/m8JuT7k8nOz", - }), - }) - .then(() => { - window.localStorage.setItem(COMPLETE_QUESTIONNAIRE, true); - console.log(`✅ Questionnaire responses sent.`); - }) - .catch((error) => { - console.error(`sendQuestionnaire`, error.message); - }); -} - -function UserQuestionnaire({ nextStep, prevStep }) { - const handleSubmit = async (e) => { - e.preventDefault(); - const form = e.target; - const formData = new FormData(form); - nextStep("create_workspace"); - - await sendQuestionnaire({ - email: formData.get("email"), - useCase: formData.get("use_case") || "other", - comment: formData.get("comment") || null, - }); - return; - }; - - const handleSkip = () => { - nextStep("create_workspace"); - }; - - if (!!window?.localStorage?.getItem(COMPLETE_QUESTIONNAIRE)) { - return ( -
-
-
- -

Thank you for your feedback!

- - team@mintplexlabs.com - -
-
- -
- - -
- - -
-
-
- ); - } - - return ( -
-
-
-
-
-
- -
- -
-
- -
-
-
- -
- -
-
- - - -
-
- - - -
-
- - - -
-
-
-
- -
-
-
- -
-