Image inpainting tool powered by SOTA AI Model. Remove any unwanted object, defect, people from your pictures or erase and replace(powered by stable diffusion) any thing on your pictures.
eea85b834e
This patch brings in a massive number of changes to the frontend of the application. Please feel free to discuss the proposed changes with me at any time. Implemented Recoil as a state management system. Why Recoil? It is a robust library built by developers at Facebook for state management. It has an extremely simple API for implementation that is in sync with React syntax compared to any other state management system out there and works amazingly well. While the official release status is beta as it becomes fully featured, the library is already used in various systems at Facebook and is very stable for the use cases of this application. Why global state management? One of the major issues I saw with the current file structure is that there is minimal code splitting and it makes further development of the frontend a cumbersome task. I have broken down the frontend into various easy to access components isolating the GUI from the logic. To avoid prop drilling, we need global state management to handle the necessary tasks. This will also facilitate the addition of any new features greatly. Code Splitting. Majority of the components that can be isolated in the application have now been done so. All New Custom CSS & Removal of Tailwind While Tailwind is a great way to deploy beautiful interfaces quickly, anyone trying to stylize the application further needs to be familiar with Tailwind which makes it harder for more people to work on it. Not to mention, I am not a particular fan of flooding JSX elements with inline CSS classes. That makes reading the code extremely hard and bloats up component code drastically. As a replacement to Tailwind, I implemented a custom styling system using SCSS as a developer dependency. In the new system, all the general and shared styles are in the styles folder and all the component styles are in the same folder as the component for easy access.The _index.scss file now acts as a central import for every other stylesheet that needs to be loaded. What Changed? The entire application looks and feels like the current implementation with minimal changes. The green (#bdff01) highlight used in the application has now been changed to a bright yellow (rgb(255, 190, 0)) because I felt it better suited the new Dark Mode (see below). The swipe bar for comparing before and after images has now been removed and instead the comparison is a smooth fade effect. I felt this was better to analyze image changes rather than a swiper. // Can add the swipe back if needed. Dark Mode A brand new Dark Mode has been added for the application. Users can enable and disable by tapping the button in the header or by using the Shift + D hotkey. Other Misc New Features When the editor image is now zoomed out to its default size, the image now also gets centered back. TODO The currently used react-zoom-pinch-pan module is not mobile friendly. It does not allow brush strokes. Need to figure out a way to fix this. Further optimization of the frontend code with better code splitting and performance. When using the LaMa model, the first stroke has a delayed response from the backend but the ones that follow are almost immediate. I believe this is happening because of the initialization of the model on the first stroke. I wonder if either of us can look at it and see if this can somehow be preloaded so the user experience is smooth from the first stroke. Enable threading for the desktop application mode so flaskwebgui does not block the main applications Python console. |
||
|---|---|---|
| lama_cleaner | ||
| .gitignore | ||
| Dockerfile | ||
| LICENSE | ||
| main.py | ||
| README.md | ||
| requirements.txt | ||
| setup.py | ||
Lama-cleaner: Image inpainting tool powered by SOTA AI model
https://user-images.githubusercontent.com/3998421/153323093-b664bb68-2928-480b-b59b-7c1ee24a4507.mp4
- Support multiple model architectures
- High resolution support
- Run as a desktop APP
- Multi stroke support. Press and hold the
cmd/ctrlkey to enable multi stroke mode. - Zoom & Pan
- Keep image EXIF data
Quick Start
- Install requirements:
pip3 install -r requirements.txt - Start server:
python3 main.py, open http://localhost:8080
Available commands for main.py
| Name | Description | Default |
|---|---|---|
| --model | lama or ldm. See details in Model Comparison | lama |
| --device | cuda or cpu | cuda |
| --ldm-steps | The larger the value, the better the result, but it will be more time-consuming | 50 |
| --crop-trigger-size | If image size large then crop-trigger-size, crop each area from original image to do inference. Mainly for performance and memory reasons on very large image. | 2042,2042 |
| --crop-margin | Margin around bounding box of painted stroke when crop mode triggered. | 256 |
| --gui | Launch lama-cleaner as a desktop application | |
| --gui_size | Set the window size for the application | 1200 900 |
| --input | Path to image you want to load by default | None |
| --port | Port for flask web server | 8080 |
| --debug | Enable debug mode for flask web server |
Model Comparison
Diffusion model(ldm) is MUCH MORE slower than GANs(lama)(1080x720 image takes 8s on 3090), but it's possible to get better result, see below example:
| Original Image | LaMa | LDM |
|---|---|---|
 |
 |
 |
Blogs about diffusion models:
- https://lilianweng.github.io/posts/2021-07-11-diffusion-models/
- https://yang-song.github.io/blog/2021/score/
Development
Only needed if you plan to modify the frontend and recompile yourself.
Fronted
Frontend code are modified from cleanup.pictures, You can experience their great online services here.
- Install dependencies:
cd lama_cleaner/app/ && yarn - Start development server:
yarn dev - Build:
yarn build
Docker
Run within a Docker container. Set the CACHE_DIR to models location path. Optionally add a -d option to
the docker run command below to run as a daemon.
Build Docker image
docker build -f Dockerfile -t lamacleaner .
Run Docker (cpu)
docker run -p 8080:8080 -e CACHE_DIR=/app/models -v $(pwd)/models:/app/models -v $(pwd):/app --rm lamacleaner python3 main.py --device=cpu --port=8080
Run Docker (gpu)
docker run --gpus all -p 8080:8080 -e CACHE_DIR=/app/models -v $(pwd)/models:/app/models -v $(pwd):/app --rm lamacleaner python3 main.py --device=cuda --port=8080
Then open http://localhost:8080
Like My Work?
