2022-07-22 17:20:12 +02:00
< h1 align = "center" > Lama Cleaner< / h1 >
< p align = "center" > A free and open-source inpainting tool powered by SOTA AI model.< / p >
< p align = "center" >
< a href = "https://github.com/Sanster/lama-cleaner" >
< img alt = "total download" src = "https://pepy.tech/badge/lama-cleaner" / >
< / a >
< a href = "https://pypi.org/project/lama-cleaner/" >
< img alt = "version" src = "https://img.shields.io/pypi/v/lama-cleaner" / >
< / a >
< a href = "https://colab.research.google.com/drive/1e3ZkAJxvkK3uzaTGu91N9TvI_Mahs0Wb?usp=sharing" >
< img alt = "Open in Colab" src = "https://colab.research.google.com/assets/colab-badge.svg" / >
< / a >
< / p >
2022-07-25 15:43:00 +02:00
2022-07-22 17:20:12 +02:00
## Features
- Completely free and open-source
- Fully self-hosted
2022-09-26 03:16:11 +02:00
- Classical image inpainting algorithm powered by [cv2 ](https://docs.opencv.org/3.4/df/d3d/tutorial_py_inpainting.html )
2022-07-22 17:20:12 +02:00
- Multiple SOTA AI models
2022-03-24 14:36:47 +01:00
1. [LaMa ](https://github.com/saic-mdal/lama )
1. [LDM ](https://github.com/CompVis/latent-diffusion )
2022-07-22 17:20:12 +02:00
1. [ZITS ](https://github.com/DQiaole/ZITS_inpainting )
2022-08-22 17:24:02 +02:00
1. [MAT ](https://github.com/fenglinglwb/MAT )
2022-09-04 15:23:58 +02:00
1. [FcF ](https://github.com/SHI-Labs/FcF-Inpainting )
2022-09-22 16:53:20 +02:00
1. [SD1.4 ](https://github.com/CompVis/stable-diffusion )
2022-07-22 17:20:12 +02:00
- Support CPU & GPU
2022-09-04 15:23:58 +02:00
- Various inpainting [strategy ](#inpainting-strategy )
2022-07-22 17:20:12 +02:00
- Run as a desktop APP
## Usage
2022-09-26 03:16:11 +02:00
| Usage | Before | After |
| ---------------------- | --------------------------------------------- | -------------------------------------------------------------- |
| Remove unwanted things |  |  |
| Remove unwanted person |  |  |
| Remove Text |  |  |
| Remove watermark |  |  |
| Fix old photo |  |  |
| Text Driven Inpainting |  | Prompt: a fox sitting on a bench< br />  |
2021-12-12 07:54:37 +01:00
2022-07-22 17:20:12 +02:00
## Quick Start
2021-11-15 08:22:34 +01:00
2022-04-18 16:40:23 +02:00
```bash
pip install lama-cleaner
2022-03-04 06:44:53 +01:00
2022-07-22 17:20:12 +02:00
# Model will be downloaded automatically
lama-cleaner --model=lama --device=cpu --port=8080
# Lama Cleaner is now running at http://localhost:8080
2022-04-18 16:40:23 +02:00
```
2022-07-22 17:20:12 +02:00
Available arguments:
2022-03-04 06:44:53 +01:00
2022-09-29 07:13:09 +02:00
| Name | Description | Default |
|-------------------|-------------------------------------------------------------------------------------------------------------------------------| -------- |
| --model | lama/ldm/zits/mat/fcf/sd1.4 See details in [Inpaint Model ](#inpainting-model ) | lama |
| --hf_access_token | stable-diffusion(sd) model need [huggingface access token ](https://huggingface.co/docs/hub/security-tokens ) to download model | |
| --sd-run-local | Once the model as downloaded, you can pass this arg and remove `--hf_access_token` | |
| --sd-disable-nsfw | Disable stable-diffusion NSFW checker. | |
| --sd-cpu-textencoder | Always run stable-diffusion TextEncoder model on CPU. | |
| --device | cuda or cpu | cuda |
| --port | Port for backend flask web server | 8080 |
| --gui | Launch lama-cleaner as a desktop application | |
| --gui_size | Set the window size for the application | 1200 900 |
| --input | Path to image you want to load by default | None |
| --debug | Enable debug mode for flask web server | |
2022-03-04 06:44:53 +01:00
2022-07-22 17:20:12 +02:00
## Inpainting Model
2022-06-12 16:06:29 +02:00
2022-09-26 03:16:11 +02:00
| Model | Description | Config |
| ----- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| cv2 | :+1: No GPU is required, and for simple backgrounds, the results may even be better than AI models. | |
| LaMa | :+1: Generalizes well on high resolutions(~2k)< br / > | |
2022-09-19 10:06:37 +02:00
| LDM | :+1: Possible to get better and more detail result < br /> :+1: The balance of time and quality can be achieved by adjusting `steps` < br /> :neutral_face: Slower than GAN model< br /> :neutral_face: Need more GPU memory | `Steps` : You can get better result with large steps, but it will be more time-consuming < br /> `Sampler` : ddim or [plms ](https://arxiv.org/abs/2202.09778 ). In general plms can get [better results ](https://github.com/Sanster/lama-cleaner/releases/tag/0.13.0 ) with fewer steps |
2022-09-26 03:16:11 +02:00
| ZITS | :+1: Better holistic structures compared with previous methods < br /> :neutral_face: Wireframe module is **very** slow on CPU | `Wireframe` : Enable edge and line detect |
| MAT | TODO | |
| FcF | :+1: Better structure and texture generation < br / > :neutral_face: Only support fixed size (512x512) input | |
| SD1.4 | :+1: SOTA text-to-image diffusion model | |
2022-06-12 16:06:29 +02:00
2022-07-22 17:20:12 +02:00
### LaMa vs LDM
2022-06-12 16:06:29 +02:00
2022-07-22 17:20:12 +02:00
| Original Image | LaMa | LDM |
| ----------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------- |
|  |  |  |
2022-06-12 16:06:29 +02:00
2022-07-22 17:20:12 +02:00
### LaMa vs ZITS
2022-06-12 16:06:29 +02:00
2022-07-22 17:20:12 +02:00
| Original Image | ZITS | LaMa |
| ---------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------- |
|  |  |  |
2022-06-12 16:06:29 +02:00
2022-07-22 17:20:12 +02:00
Image is from [ZITS ](https://github.com/DQiaole/ZITS_inpainting ) paper. I didn't find a good example to show the advantages of ZITS and let me know if you have a good example. There can also be possible problems with my code, if you find them, please let me know too!
2022-06-12 16:06:29 +02:00
2022-09-04 15:23:58 +02:00
### LaMa vs FcF
2022-06-12 16:06:29 +02:00
2022-09-04 15:23:58 +02:00
| Original Image | Lama | FcF |
| ----------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------- |
|  |  |  |
2022-06-12 16:06:29 +02:00
2022-09-04 15:23:58 +02:00
## Inpainting Strategy
Lama Cleaner provides three ways to run inpainting model on images, you can change it in the settings dialog.
| Strategy | Description | VRAM |
| ------------ | ---------------------------------------------------------------------------------------------------------------------------------------------- | -------------------- |
| **Original** | Use the resolution of the original image | :tada: |
| **Resize** | Resize the image to a smaller size before inpainting. Lama Cleaner will make sure that the area of the image outside the mask is not degraded. | :tada: :tada: |
| **Crop** | Crop masking area from the original image to do inpainting | :tada: :tada: :tada: |
2021-11-15 08:22:34 +01:00
2022-07-25 04:16:01 +02:00
## Download Model Mannually
If you have problems downloading the model automatically when lama-cleaner start,
you can download it manually. By default lama-cleaner will load model from `TORCH_HOME=~/.cache/torch/hub/checkpoints/` ,
you can set `TORCH_HOME` to other folder and put the models there.
- Github:
- [LaMa ](https://github.com/Sanster/models/releases/tag/add_big_lama )
- [LDM ](https://github.com/Sanster/models/releases/tag/add_ldm )
- [ZITS ](https://github.com/Sanster/models/releases/tag/add_zits )
2022-09-04 15:23:58 +02:00
- [MAT ](https://github.com/Sanster/models/releases/tag/add_mat )
- [FcF ](https://github.com/Sanster/models/releases/tag/add_fcf )
2022-07-25 04:16:01 +02:00
- Baidu:
- https://pan.baidu.com/s/1vUd3BVqIpK6e8N_EA_ZJfw
- passward: flsu
2021-11-15 08:22:34 +01:00
## Development
2022-02-05 12:58:25 +01:00
Only needed if you plan to modify the frontend and recompile yourself.
2022-04-25 15:08:19 +02:00
### Frontend
2021-11-15 08:22:34 +01:00
2022-03-04 06:44:53 +01:00
Frontend code are modified from [cleanup.pictures ](https://github.com/initml/cleanup.pictures ), You can experience their
great online services [here ](https://cleanup.pictures/ ).
2021-11-15 08:22:34 +01:00
- Install dependencies:`cd lama_cleaner/app/ && yarn`
2022-04-04 15:51:33 +02:00
- Start development server: `yarn start`
2021-11-15 08:22:34 +01:00
- Build: `yarn build`
2021-11-15 20:11:46 +01:00
2021-11-15 23:41:59 +01:00
## Docker
2021-11-16 14:21:41 +01:00
2022-03-04 06:44:53 +01:00
Run within a Docker container. Set the `CACHE_DIR` to models location path. Optionally add a `-d` option to
the `docker run` command below to run as a daemon.
2021-11-15 23:41:59 +01:00
2021-11-15 23:51:27 +01:00
### Build Docker image
2021-11-16 14:21:41 +01:00
2021-11-15 20:11:46 +01:00
```
docker build -f Dockerfile -t lamacleaner .
2021-11-15 23:51:27 +01:00
```
### Run Docker (cpu)
2021-11-16 14:21:41 +01:00
2021-11-15 23:51:27 +01:00
```
2022-09-29 03:15:56 +02:00
docker run -p 8080:8080 -e CACHE_DIR=/app/models -v $(pwd)/models:/app/models -v $(pwd):/app --rm lamacleaner \
python3 main.py --device=cpu --port=8080 --host=0.0.0.0
2021-11-15 20:11:46 +01:00
```
2021-11-15 23:51:27 +01:00
### Run Docker (gpu)
2021-11-16 14:21:41 +01:00
2021-11-15 20:11:46 +01:00
```
2022-09-29 03:15:56 +02:00
docker run --gpus all -p 8080:8080 -e CACHE_DIR=/app/models -v $(pwd)/models:/app/models -v $(pwd):/app --rm lamacleaner \
python3 main.py --device=cuda --port=8080 --host=0.0.0.0
2021-11-15 20:11:46 +01:00
```
2021-11-16 14:21:41 +01:00
Then open [http://localhost:8080 ](http://localhost:8080 )
