90s Cartoon v1

Ani4K is a 2x model for creating natural-looking 2K and 4K upscales of modern anime. The model places emphasis on depth of field and blur effect retention, which are extremely common in modern anime and something that many anime models struggle with. In addition, Ani4K is trained for strong detail retention, WEB/BD compression clean-up, and producing pleasing lineart. 

More info on the Github page: https://github.com/Sirosky/Sirosky-Upscaling-Models.

Blur Retention Comparisons (including vs. xinntao's 2x paper model): https://imgsli.com/MTg2NTg5/4/5

General Comparisons: https://imgsli.com/MTg2ODI2

Ani4K

Purpose: Anime/Pretrained

A 2x Ultra Lite model coming in under 8MB. Trained with over 15 sets of LRs ranging in a wide amount of issues. Handles Rainbows, Dot Crawl, MPEG/H.264 Compression, and may even assist in removing halos, and fixing blurriness in certain cases. This is my first public model for everyone. Best when used on old anime that is grainy. I can't say what anime it's best suited for as I have tried multiple series, and have found it does a good job on most all the tests. I wouldn't say use this for Western Animation, but it may work. I have done a few tests that I have shown in the upscale results, but that was chained with other models to achieve such a result. This model is meant to retain the more natural look of a series. There is a color shift on the end result, not drastic, but still noticable. I figure you should fix any color issues in post that way to give a more polished upscale. Big thanks to @SaurusX for the model name, and just helping out in general with anything.

AnimeClassics UltraLite

Real-time 2x Real-ESRGAN Compact/UltraCompact/SuperUltraCompact models designed for upscaling 1080p anime to 4K. 

The aim of these models is to address scaling, blur, oversharpening, and compression artifacts while upscaling to deliver a result that appears as if the anime was originally mastered in 4K resolution. Can be set up to run in real-time with mpv on Windows using https://github.com/the-database/mpv-upscale-2x_animejanai

The development of the V3 models spanned over seven months, during which over 100 release candidate models were trained and meticulously refined. The V3 models introduce several improvements compared to V2, including:

More faithful appearance to original source
Improved handling of oversharpening artifacts, ringing, aliasing
Better at preserving intentional blur in scenes using depth of field
More accurate line colors, darkness, and thickness
Better preservation of soft shadow edges

Overall, the V3 models yield significantly more natural and faithful results compared to the V2 models.test

AnimeJaNai_HD_V3_Compact

Real-time 2x Real-ESRGAN Compact/UltraCompact/SuperUltraCompact models designed for upscaling 1080p anime to 4K.

The aim of these models is to address scaling, blur, oversharpening, and compression artifacts while upscaling to deliver a result that appears as if the anime was originally mastered in 4K resolution. Can be set up to run in real-time with mpv on Windows using https://github.com/the-database/mpv-upscale-2x_animejanai

The development of the V3 models spanned over seven months, during which over 100 release candidate models were trained and meticulously refined. The V3 models introduce several improvements compared to V2, including:

More faithful appearance to original source Improved handling of oversharpening artifacts, ringing, aliasing Better at preserving intentional blur in scenes using depth of field More accurate line colors, darkness, and thickness Better preservation of soft shadow edges

Overall, the V3 models yield significantly more natural and faithful results compared to the V2 models.

AnimeJaNai_HD_V3_SuperUltraCompact

AnimeJaNai_HD_V3_UltraCompact

Realtime 2x model intended for high or medium quality 1080p anime with an emphasis on correcting the inherit blurriness of anime while preserving details and colors. Not suitable for artifact-heavy or highly compressed content as it will just sharpen artifacts. Also works with SD anime by running the model twice. Can be set up to run with mpv on Windows using https://github.com/the-database/mpv-upscale-2x_animejanai

Minimum of RTX 3080 is recommended for running UltraCompact model on 1080p in realtime; RTX 4090 is required to run Compact on 1080p in realtime. SuperUltraCompact should run in realtime on 1080p on some lower cards. The compact model is recommended on SD content. 

Samples: https://imgsli.com/MTUxMDYx \
Comparisons to Anime4K + other compact models and upscalers: https://imgsli.com/MTUxMjY4

AnimeJaNai Standard v1 Compact

AnimeJaNai Standard v1 SuperUltraCompact

This is a collection of pretrained models for Real-ESRGAN's Compact architecture. There are 1x, 2x, and 4x models, as well as 1x and 2x "UltraCompact" and "SuperUltraCompact" models (think of these as the equivalent to ESRGAN "lite" models). By using these are pretrains for your models, you can ensure that your models are able to be interpolated with other Compact models that were trained from these. These pretrains are compatible with most existing compact models.

Compact Pretrain

DAT_2_X2

2xESRGAN

Pretrained: RRDB_PSNR_x4.pth

The original RRDB_PSNR_x4.pth model converted to 1x, 2x, 8x and 16x scales, intended to be used as pretrained models for new models at those scales. These are compatible with victor's 4xESRGAN.pth conversions

2x PSNR Pretrained Model

Official paper pretrain model

SRFormer: Permuted Self-Attention for Single Image Super-Resolution

Yupeng Zhou 1, Zhen Li 1, Chun-Le Guo 1, Song Bai 2, Ming-Ming Cheng 1, Qibin Hou 1

1TMCC, School of Computer Science, Nankai University

2ByteDance, Singapore

arXiv

The official PyTorch implementation of SRFormer: Permuted Self-Attention for Single Image Super-Resolution (arxiv). SRFormer achieves state-of-the-art performance in

    classical image SR
    lightweight image SR
    real-world image SR

The results can be found here.

    Abstract: In this paper, we introduce SRFormer, a simple yet effective Transformer-based model for single image super-resolution. We rethink the design of the popular shifted window self-attention, expose and analyze several characteristic issues of it, and present permuted self-attention (PSA). PSA strikes an appropriate balance between the channel and spatial information for self-attention, allowing each Transformer block to build pairwise correlations within large windows with even less computational burden. Our permuted self-attention is simple and can be easily applied to existing super-resolution networks based on Transformers. Without any bells and whistles, we show that our SRFormer achieves a 33.86dB PSNR score on the Urban100 dataset, which is 0.46dB higher than that of SwinIR but uses fewer parameters and computations. We hope our simple and effective approach can serve as a useful tool for future research in super-resolution model design. Our code is publicly available at https://github.com/HVision-NKU/SRFormer.

Contents

    Installation & Dataset
    Training
    Testing
    Results
    Pretrain Models
    Citations
    License and Acknowledgement

Installation & Dataset

    python 3.8
    pyTorch >= 1.7.0

cd SRFormer
pip install -r requirements.txt
python setup.py develop

Dataset

We use the same training and testing sets as SwinIR, the following datasets need to be downloaded for training.
Task 	Training Set 	Testing Set
classical image SR 	DIV2K (800 training images) or DIV2K +Flickr2K (2650 images) 	Set5 + Set14 + BSD100 + Urban100 + Manga109
lightweight image SR 	DIV2K (800 training images) 	Set5 + Set14 + BSD100 + Urban100 + Manga109
real-world image SR 	DIV2K (800 training images) +Flickr2K (2650 images) + OST (10324 images for sky,water,grass,mountain,building,plant,animal) 	RealSRSet+5images
Training

    Please download the dataset corresponding to the task and place them in the folder specified by the training option in folder /options/train/SRFormer
    Follow the instructions below to train our SRFormer.

# train SRFormer for classical SR task
./scripts/dist_train.sh 4 options/train/SRFormer/train_SRFormer_SRx2_scratch.yml
./scripts/dist_train.sh 4 options/train/SRFormer/train_SRFormer_SRx3_scratch.yml
./scripts/dist_train.sh 4 options/train/SRFormer/train_SRFormer_SRx4_scratch.yml
# train SRFormer for lightweight SR task
./scripts/dist_train.sh 4 options/train/SRFormer/train_SRFormer_light_SRx2_scratch.yml
./scripts/dist_train.sh 4 options/train/SRFormer/train_SRFormer_light_SRx3_scratch.yml
./scripts/dist_train.sh 4 options/train/SRFormer/train_SRFormer_light_SRx4_scratch.yml

Testing

# test SRFormer for classical SR task
python basicsr/test.py -opt options/test/SRFormer/test_SRFormer_DF2Ksrx2.yml
python basicsr/test.py -opt options/test/SRFormer/test_SRFormer_DF2Ksrx3.yml
python basicsr/test.py -opt options/test/SRFormer/test_SRFormer_DF2Ksrx4.yml
# test SRFormer for lightweight SR task
python basicsr/test.py -opt options/test/SRFormer/test_SRFormer_light_DIV2Ksrx2.yml
python basicsr/test.py -opt options/test/SRFormer/test_SRFormer_light_DIV2Ksrx3.yml
python basicsr/test.py -opt options/test/SRFormer/test_SRFormer_light_DIV2Ksrx4.yml

Results

We provide the results on classical image SR, lightweight image SR, realworld image SR. More results can be found in the paper. The visual results of SRFormer will upload to google drive soon.

Classical image SR

    Results of Table 4 in the paper

    Results of Figure 4 in the paper

Lightweight image SR

    Results of Table 5 in the paper

    Results of Figure 5 in the paper

Model size comparison

    Results of Table 1 and Table 2 in the Supplementary Material

Realworld image SR

    Results of Figure 8 in the paper

Pretrain Models

Pretrain Models can be download from google drive. To reproduce the results in the article, you can download them and put them in the /PretrainModel folder.
Citations

You may want to cite:

@article{zhou2023srformer,
  title={SRFormer: Permuted Self-Attention for Single Image Super-Resolution},
  author={Zhou, Yupeng and Li, Zhen and Guo, Chun-Le and Bai, Song and Cheng, Ming-Ming and Hou, Qibin},
  journal={arXiv preprint arXiv:2303.09735},
  year={2023}
}

License and Acknowledgement

This project is released under the Apache 2.0 license. The codes are based on BasicSR, Swin Transformer, and SwinIR. Please also follow their licenses. Thanks for their awesome works.

SRFormerLight_SRx2_DIV2K

SuperUltraCompact Pretrain

[Swift-SRGAN - Rethinking Super-Resolution for real-time inference](https://github.com/Koushik0901/Swift-SRGAN)

In recent years, there have been several advancements in the task of image super-resolution using the state of the art Deep Learning-based architectures. Many super-resolution-based techniques previously published, require high-end and top-of-the-line Graphics Processing Unit (GPUs) to perform image super-resolution. With the increasing advancements in Deep Learning approaches, neural networks have become more and more compute hungry. We took a step back and, focused on creating a real-time efficient solution. We present an architecture that is faster and smaller in terms of its memory footprint. The proposed architecture uses Depth-wise Separable Convolutions to extract features and, it performs on-par with other super-resolution GANs (Generative Adversarial Networks) while maintaining real-time inference and a low memory footprint. A real-time super-resolution enables streaming high resolution media content even under poor bandwidth conditions. While maintaining an efficient trade-off between the accuracy and latency, we are able to produce a comparable performance model which is one-eighth (1/8) the size of super-resolution GANs and computes 74 times faster than super-resolution GANs.

NOTE: The author used the wrong file extensions for the models *on GitHub*. You will download a `.pth.tar` file. This is not actually a TAR file. Change the file extension to just `.pth` and the model will work.

Swift-SRGAN 2x

UltraCompact Pretrain

This model is based on a private model by @eula 5600x 3070 named 4x_eula_anifilm_v1_225k. He sent me a copy of the model, and I decided to train a compact model based on it with his permission. This model seems to fix the majority of the issues the original model had while being far faster, it's just a tiny bit softer in some images.

The dataset consists of Dragon Ball movies converted to YUV24 with @sgdisk --zap-all /dev/sda's help to reduce artifacts, then upscaled with ArtClarity and eula_anifilm. LRs are the original frames right from DVD. As a result, this model corrects some color space issues. The 2x model's HRs were downscaled by 50% with Lanczos.

The 2x and 4x models are pretty close in output despite being trained separately. The 2x model is a bit softer overall.

The models in the Real-ESRGAN Compatible folder are the original output from Real-ESRGANs training code for compatibility reasons.

Anifilm Compact (2x)

This is just a very basic 2x model to start your anime models with on SPAN, nothing special. I tried to keep it as basic as possible

2x-span_anime_pretrain

Official 2x pretrain for [SPAN](https://github.com/hongyuanyu/span).

spanx2_ch48

Purpose: pre-workout to speed up your workouts

While experimenting with color models for cugan, I just ran into problems that the official models are not imported correctly and have additional processing in addition to resizing. As a result, for my project I trained 2 x cugans for neosr; only resizing losses were applied to the dataset `(“lanczos”, “bicubic”, “bilenear”, “HAMMING”, “NEAREST”)`

wtp manga cover pretrained

[Link to Github Release](https://github.com/Phhofm/models/releases/tag/4xDRCT-mssim-pretrains)

Since no [DRCT](https://github.com/ming053l/DRCT/tree/main) model releases exist yet, I am releasing my drct-s, drct and drct-l pretrains. 
These can be used to speed up and stabilize early training stages when training new drct models.
Trained with mssim (and augs, and color and lumaloss) on downscaled [nomosuni](https://github.com/muslll/neosr/tree/0ecbeb98dcefd97801da0ea39ca2cd03a87fce21?tab=readme-ov-file#datasets).
Training files are also provided additionally.

## 4xmssim_drct-l_pretrain

**Scale:** 4  
**Architecture:** [DRCT](https://github.com/ming053l/DRCT)  
**Architecture Option:** [DRCT-L](https://github.com/ming053l/DRCT/blob/8c13e63135f494913cd504c073e41ef52250d1d4/options/train/train_DRCT-L_SRx4_ImageNet_from_scratch.yml#L55)  

**Author:** Philip Hofmann  
**License:** CC-BY-0.4  
**Purpose:** Pretrained  
**Subject:** Anime, Photography  
**Input Type:** Images  
**Release Date:** 28.04.2024  

**Dataset:** [nomos_uni](https://github.com/muslll/neosr?tab=readme-ov-file#datasets)  
**Dataset Size:** 2989  
**OTF (on the fly augmentations):** No  
**Pretrained Model:** None (=From Scratch)  
**Iterations:** 108'000  
**Batch Size:** 2-6  
**GT Size:** 128-320  

**Description:** A pretrain to start drct-l model training, since there are no official released drct pretrains yet I trained these myself and release them here.

4xmssim_drct-l_pretrain

[Link to Github Release](https://github.com/Phhofm/models/releases/tag/4xDRCT-mssim-pretrains)

Since no [DRCT](https://github.com/ming053l/DRCT/tree/main) model releases exist yet, I am releasing my drct-s, drct and drct-l pretrains. 
These can be used to speed up and stabilize early training stages when training new drct models.
Trained with mssim (and augs, and color and lumaloss) on downscaled [nomosuni](https://github.com/muslll/neosr/tree/0ecbeb98dcefd97801da0ea39ca2cd03a87fce21?tab=readme-ov-file#datasets).
Training files are also provided additionally.

## 4xmssim_drct_pretrain

**Scale:** 4  
**Architecture:** [DRCT](https://github.com/ming053l/DRCT)  
**Architecture Option:** [DRCT](https://github.com/ming053l/DRCT/blob/8c13e63135f494913cd504c073e41ef52250d1d4/options/train/train_DRCT_SRx4_ImageNet_from_scratch.yml#L55)  

**Author:** Philip Hofmann  
**License:** CC-BY-0.4  
**Purpose:** Pretrained  
**Subject:** Anime, Photography  
**Input Type:** Images  
**Release Date:** 28.04.2024  

**Dataset:** [nomos_uni](https://github.com/muslll/neosr?tab=readme-ov-file#datasets)  
**Dataset Size:** 2989  
**OTF (on the fly augmentations):** No  
**Pretrained Model:** None (=From Scratch)  
**Iterations:** 95'000  
**Batch Size:** 2-6  
**GT Size:** 128-320   

**Description:** A pretrain to start drct model training, since there are no official released drct pretrains yet I trained these myself and release them here.

4xmssim_drct_pretrain

[Link to Github Release](https://github.com/Phhofm/models/releases/tag/4xDRCT-mssim-pretrains)

Since no [DRCT](https://github.com/ming053l/DRCT/tree/main) model releases exist yet, I am releasing my drct-s, drct and drct-l pretrains. 
These can be used to speed up and stabilize early training stages when training new drct models.
Trained with mssim (and augs, and color and lumaloss) on downscaled [nomosuni](https://github.com/muslll/neosr/tree/0ecbeb98dcefd97801da0ea39ca2cd03a87fce21?tab=readme-ov-file#datasets).
Training files are also provided additionally.


## 4xmssim_drct-s_pretrain.pth  

**Scale:** 4  
**Architecture:** [DRCT](https://github.com/ming053l/DRCT)  
**Architecture Option:** [DRCT-S](https://github.com/muslll/neosr/blob/0ecbeb98dcefd97801da0ea39ca2cd03a87fce21/neosr/archs/drct_arch.py#L923)  

**Author:** Philip Hofmann  
**License:** CC-BY-0.4  
**Purpose:** Pretrained  
**Subject:** Anime, Photography  
**Input Type:** Images  
**Release Date:** 28.04.2024  

**Dataset:** [nomos_uni](https://github.com/muslll/neosr?tab=readme-ov-file#datasets)  
**Dataset Size:** 2989  
**OTF (on the fly augmentations):** No  
**Pretrained Model:** None (=From Scratch)  
**Iterations:** 75'000  
**Batch Size:** 6  
**GT Size:** 128-320  

**Description:** A pretrain to start drct-s model training, since there are no official released drct pretrains yet I trained these myself and release them here.

4xmssim_drct-s_pretrain

[Github Release Link](https://github.com/Phhofm/models/releases/4xSPAN_pretrains)

[Neosr](https://github.com/muslll/neosr)'s latest update from yesterday included a [new adaptation of the multi-scale ssim loss](https://github.com/muslll/neosr/wiki/Losses#mssim_opt).     
This was an experiment to test out the difference between making a SPAN pretrain with pixel loss with L1 criteria (as often used in research) vs mssim loss as its only loss.     
Models are provided so they can be used for tests or also used as a pretrain for another SPAN model.  

---

## 4xmssim_span_pretrain

Scale: 4  
Architecture: SPAN  

Author: Philip Hofmann    
License: CC-BY-4.0  
Purpose: Pretrain  
Subject: Realistic, Anime  
Date: 10.04.2024  

Dataset: [nomos_uni](https://github.com/muslll/neosr)  
Dataset Size: 2989   
OTF (on the fly augmentations): No  
Pretrained Model: None  
Iterations: 80'000  
Batch Size: 12  
GT Size: 128  

Description: 4x SPAN pretrain trained on [neosr](https://github.com/muslll/neosr)'s [new adaptation of the multi-scale ssim loss](https://github.com/muslll/neosr/wiki/Losses#mssim_opt) from yesterdays update on downsampled nomos_uni dataset using kim's [dataset destroyer](https://github.com/Kim2091/helpful-scripts/tree/main/Dataset%20Destroyer) with down_up,linear,cubic_mitchell,lanczos,gauss,box (while down_up used the same and with range = 0.15,1.5). 
The new augmentations except CutBlur have also been used (since CutBlur is meant to be applied to real-world SR and may cause undesired effects if applied to bicubic-only). 
Config and training log provided for more details.
 
Showcase: 
[7 Slowpics Examples](https://slow.pics/c/zyilXhKU)

4xmssim_span_pretrain

[Github Release Link](https://github.com/Phhofm/models/releases/4xSPAN_pretrains)

[Neosr](https://github.com/muslll/neosr)'s latest update from yesterday included a [new adaptation of the multi-scale ssim loss](https://github.com/muslll/neosr/wiki/Losses#mssim_opt). 
This was an experiment to test out the difference between making a SPAN pretrain with pixel loss with L1 criteria (as often used in research) vs mssim loss as its only loss. 
Models are provided so they can be used for tests or also used as a pretrain for another SPAN model.

---

## 4xpix_span_pretrain

Scale: 4  
Architecture: SPAN  

Author: Philip Hofmann    
License: CC-BY-4.0  
Purpose: Pretrain  
Subject: Realistic, Anime  
Date: 10.04.2024  

Dataset: [nomos_uni](https://github.com/muslll/neosr)  
Dataset Size: 2989   
OTF (on the fly augmentations): No  
Pretrained Model: None  
Iterations: 80'000  
Batch Size: 12  
GT Size: 128  

Description: 4x SPAN pretrain trained on pixel loss with L1 criteria (as often used in research) on downsampled nomos_uni dataset using kim's [dataset destroyer](https://github.com/Kim2091/helpful-scripts/tree/main/Dataset%20Destroyer) with down_up,linear,cubic_mitchell,lanczos,gauss,box (while down_up used the same and with range = 0.15,1.5). 
The new augmentations except CutBlur have also been used (since CutBlur is meant to be applied to real-world SR and may cause undesired effects if applied to bicubic-only). 
Config and training log provided for more details.
 
Showcase: 
[7 Slowpics Examples](https://slow.pics/c/zyilXhKU)

Architecture	SPAN
Scale	2x
Color Mode	RGB
License	None
Date	2023-12-09
Dataset	HFA2k_LUDVAE modified
Training iterations	67000
Training OTF	No

2x-span_anime_pretrain

Similar Models