update README.md
This commit is contained in:
Qing
parent
4f6fbc57b7
commit
933cc44b5c
134
README.md
134
README.md
@ -1,85 +1,95 @@
|
||||
# Lama-cleaner: Image inpainting tool powered by SOTA AI model
|
||||
<h1 align="center">Lama Cleaner</h1>
|
||||
<p align="center">A free and open-source inpainting tool powered by SOTA AI model.</p>
|
||||
|
||||
[](https://pepy.tech/project/lama-cleaner)
|
||||
[](https://pepy.tech/project/lama-cleaner)
|
||||
|
||||
<a href="https://colab.research.google.com/drive/1e3ZkAJxvkK3uzaTGu91N9TvI_Mahs0Wb?usp=sharing" target="_parent"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open in Colab"/></a>
|
||||
<p align="center">
|
||||
<a href="https://github.com/Sanster/lama-cleaner">
|
||||
<img alt="total download" src="https://pepy.tech/badge/lama-cleaner" />
|
||||
</a>
|
||||
<a href="https://pypi.org/project/lama-cleaner/">
|
||||
<img alt="version" src="https://img.shields.io/pypi/v/lama-cleaner" />
|
||||
</a>
|
||||
<a href="https://colab.research.google.com/drive/1e3ZkAJxvkK3uzaTGu91N9TvI_Mahs0Wb?usp=sharing">
|
||||
<img alt="Open in Colab" src="https://colab.research.google.com/assets/colab-badge.svg" />
|
||||
</a>
|
||||
<a href="https://www.buymeacoffee.com/Sanster">
|
||||
<img height="20px" src="https://cdn.buymeacoffee.com/buttons/v2/default-blue.png" alt="Sanster" />
|
||||
</a>
|
||||
</p>
|
||||
|
||||
|
||||
|
||||
https://user-images.githubusercontent.com/3998421/153323093-b664bb68-2928-480b-b59b-7c1ee24a4507.mp4
|
||||
## Features
|
||||
|
||||
- [x] Support multiple model architectures
|
||||
- Completely free and open-source
|
||||
- Fully self-hosted
|
||||
- Multiple SOTA AI models
|
||||
1. [LaMa](https://github.com/saic-mdal/lama)
|
||||
1. [LDM](https://github.com/CompVis/latent-diffusion)
|
||||
- [x] Support CPU & GPU
|
||||
- [x] High resolution support
|
||||
- [x] Run as a desktop APP
|
||||
- [x] Multi stroke support. Press and hold the `cmd/ctrl` key to enable multi stroke mode.
|
||||
- [x] Zoom & Pan
|
||||
1. [ZITS](https://github.com/DQiaole/ZITS_inpainting)
|
||||
- Support CPU & GPU
|
||||
- Various high-resolution image processing [strategy](#high-resolution-strategy)
|
||||
- Run as a desktop APP
|
||||
|
||||
## Install
|
||||
## Usage
|
||||
|
||||
| Usage | Before | After |
|
||||
| ---------------------- | --------------------------------------------- | --------------------------------------------------- |
|
||||
| Remove unwanted things |  |  |
|
||||
| Remove unwanted person |  |  |
|
||||
| Remove watermark |  |  |
|
||||
| Fix old photo |  |  |
|
||||
|
||||
## Quick Start
|
||||
|
||||
```bash
|
||||
pip install lama-cleaner
|
||||
|
||||
lama-cleaner --device=cpu --port=8080
|
||||
# Model will be downloaded automatically
|
||||
lama-cleaner --model=lama --device=cpu --port=8080
|
||||
# Lama Cleaner is now running at http://localhost:8080
|
||||
```
|
||||
|
||||
Available commands:
|
||||
Available arguments:
|
||||
|
||||
| Name | Description | Default |
|
||||
| ---------- | ------------------------------------------------ | -------- |
|
||||
| --model | lama or ldm. See details in **Model Comparison** | lama |
|
||||
| --device | cuda or cpu | cuda |
|
||||
| --gui | Launch lama-cleaner as a desktop application | |
|
||||
| --gui_size | Set the window size for the application | 1200 900 |
|
||||
| --input | Path to image you want to load by default | None |
|
||||
| --port | Port for flask web server | 8080 |
|
||||
| --debug | Enable debug mode for flask web server | |
|
||||
| Name | Description | Default |
|
||||
| ---------- | ---------------------------------------------------------------- | -------- |
|
||||
| --model | lama/ldm/zits. See details in [Inpaint Model](#inpainting-model) | lama |
|
||||
| --device | cuda or cpu | cuda |
|
||||
| --port | Port for backend flask web server | 8080 |
|
||||
| --gui | Launch lama-cleaner as a desktop application | |
|
||||
| --gui_size | Set the window size for the application | 1200 900 |
|
||||
| --input | Path to image you want to load by default | None |
|
||||
| --debug | Enable debug mode for flask web server | |
|
||||
|
||||
## Settings
|
||||
## Inpainting Model
|
||||
|
||||
You can change the configs of inpainting process in the settings interface of the web page.
|
||||
| Model | Description | Config |
|
||||
| ----- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| LaMa | :+1: Generalizes well on high resolutions(~2k)<br/> | |
|
||||
| LDM | :+1: Possiblablity to get better and more detail result <br/> :+1: The balance of time and quality can be achieved by adjusting `steps` <br/> :neutral_face: Slower than GAN model<br/> :neutral_face: Need more GPU memory | `Steps`: You can get better result with large steps, but it will be more time-consuming <br/> `Sampler`: ddim or [plms](https://arxiv.org/abs/2202.09778). In general plms can get [better results](https://github.com/Sanster/lama-cleaner/releases/tag/0.13.0) with fewer steps |
|
||||
| ZITS | :+1: Better holistic structures compared with previous methods <br/> :neutral_face: Wireframe module is **very** slow on CPU | `Wireframe`: Enable edge and line detect |
|
||||
|
||||
<img src="./assets/settings.png" width="400px">
|
||||
|
||||
### Inpainting Model
|
||||
|
||||
Select the inpainting model to use, and set the configs corresponding to the model.
|
||||
|
||||
LaMa model has no configs that can be specified at runtime.
|
||||
|
||||
LDM model has two configs to control the quality of final result:
|
||||
1. Steps: You can get better result with large steps, but it will be more time-consuming
|
||||
2. Sampler: ddim or [plms](https://arxiv.org/abs/2202.09778). In general plms can get better results with fewer steps
|
||||
|
||||
|
||||
### High Resolution Strategy
|
||||
|
||||
There are three strategies for handling high-resolution images.
|
||||
|
||||
- **Original**: Use the original resolution of the picture, suitable for picture size below 2K.
|
||||
- **Resize**: Resize the longer side of the image to a specific size(keep ratio), then do inpainting on the resized image.
|
||||
The inpainting result will be pasted back on the original image to make sure other part of image not loss quality.
|
||||
- **Crop**: Crop masking area from the original image to do inpainting, and paste the result back.
|
||||
Mainly for performance and memory reasons on high resolution image. This strategy may give better results for ldm model.
|
||||
|
||||
|
||||
## Model Comparison
|
||||
|
||||
| Model | Pron | Corn |
|
||||
|-------|-----------------------------------------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------|
|
||||
| LaMa | - Perform will on high resolution image(~2k)<br/> - Faster than diffusion model | |
|
||||
| LDM | - It's possible to get better and more detail result, see example below<br/> - The balance of time and quality can be achieved by steps | - Slower than GAN model<br/> - Need more GPU memory<br/> - Not good for high resolution images |
|
||||
### LaMa vs LDM
|
||||
|
||||
| Original Image | LaMa | LDM |
|
||||
| ----------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
|  |  |  |
|
||||
|
||||
Blogs about diffusion models:
|
||||
### LaMa vs ZITS
|
||||
|
||||
- https://lilianweng.github.io/posts/2021-07-11-diffusion-models/
|
||||
- https://yang-song.github.io/blog/2021/score/
|
||||
| Original Image | ZITS | LaMa |
|
||||
| ---------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------- |
|
||||
|  |  |  |
|
||||
|
||||
Image is from [ZITS](https://github.com/DQiaole/ZITS_inpainting) paper. I didn't find a good example to show the advantages of ZITS and let me know if you have a good example. There can also be possible problems with my code, if you find them, please let me know too!
|
||||
|
||||
## High Resolution Strategy
|
||||
|
||||
There are three strategies for handling high-resolution images.
|
||||
|
||||
- **Original**: Use the original resolution of the picture, suitable for picture size below 2K.
|
||||
- **Resize**: Resize the image to a smaller size, then do inpainting on the resized image. The inpainting area will be croped, upsampled and pasted back on the original image to make sure other part of image not loss quality.
|
||||
- **Crop**: Crop masking area from the original image to do inpainting, and paste the result back. Mainly for performance and memory reasons on high resolution image.
|
||||
|
||||
## Development
|
||||
|
||||
@ -118,9 +128,3 @@ docker run --gpus all -p 8080:8080 -e CACHE_DIR=/app/models -v $(pwd)/models:/ap
|
||||
```
|
||||
|
||||
Then open [http://localhost:8080](http://localhost:8080)
|
||||
|
||||
## Like My Work?
|
||||
|
||||
<a href="https://www.buymeacoffee.com/Sanster">
|
||||
<img height="50em" src="https://cdn.buymeacoffee.com/buttons/v2/default-blue.png" alt="Sanster" />
|
||||
</a>
|
||||
|
||||
BIN
assets/dark.png
Normal file
BIN
assets/dark.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 1.8 MiB |
BIN
assets/old_photo.jpg
Normal file
BIN
assets/old_photo.jpg
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 29 KiB |
BIN
assets/old_photo_clean.jpg
Normal file
BIN
assets/old_photo_clean.jpg
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 53 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 80 KiB |
BIN
assets/unwant_object.jpg
Normal file
BIN
assets/unwant_object.jpg
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 165 KiB |
BIN
assets/unwant_object_clean.jpg
Normal file
BIN
assets/unwant_object_clean.jpg
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 162 KiB |
BIN
assets/unwant_person.jpg
Normal file
BIN
assets/unwant_person.jpg
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 113 KiB |
BIN
assets/unwant_person_clean.jpg
Normal file
BIN
assets/unwant_person_clean.jpg
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 109 KiB |
BIN
assets/watermark.jpg
Normal file
BIN
assets/watermark.jpg
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 292 KiB |
BIN
assets/watermark_cleanup.jpg
Normal file
BIN
assets/watermark_cleanup.jpg
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 693 KiB |
Loading…
Reference in New Issue
Block a user