16xESRGAN

This is a collection of pretrained models for Real-ESRGAN's Compact architecture. There are 1x, 2x, and 4x models, as well as 1x and 2x "UltraCompact" and "SuperUltraCompact" models (think of these as the equivalent to ESRGAN "lite" models). By using these are pretrains for your models, you can ensure that your models are able to be interpolated with other Compact models that were trained from these. These pretrains are compatible with most existing compact models.

Compact Pretrain

1xESRGAN

Pretrained: RRDB_PSNR_x4.pth

The original RRDB_PSNR_x4.pth model converted to 1x, 2x, 8x and 16x scales, intended to be used as pretrained models for new models at those scales. These are compatible with victor's 4xESRGAN.pth conversions

1x PSNR Pretrained Model

SuperUltraCompact Pretrain

UltraCompact Pretrain

Basic 1x model to use for SPAN anime models. Enjoy

1x-span_anime_pretrain

2xESRGAN

Official paper pretrain model

SRFormer: Permuted Self-Attention for Single Image Super-Resolution

Yupeng Zhou 1, Zhen Li 1, Chun-Le Guo 1, Song Bai 2, Ming-Ming Cheng 1, Qibin Hou 1

1TMCC, School of Computer Science, Nankai University

2ByteDance, Singapore

arXiv

The official PyTorch implementation of SRFormer: Permuted Self-Attention for Single Image Super-Resolution (arxiv). SRFormer achieves state-of-the-art performance in

    classical image SR
    lightweight image SR
    real-world image SR

The results can be found here.

    Abstract: In this paper, we introduce SRFormer, a simple yet effective Transformer-based model for single image super-resolution. We rethink the design of the popular shifted window self-attention, expose and analyze several characteristic issues of it, and present permuted self-attention (PSA). PSA strikes an appropriate balance between the channel and spatial information for self-attention, allowing each Transformer block to build pairwise correlations within large windows with even less computational burden. Our permuted self-attention is simple and can be easily applied to existing super-resolution networks based on Transformers. Without any bells and whistles, we show that our SRFormer achieves a 33.86dB PSNR score on the Urban100 dataset, which is 0.46dB higher than that of SwinIR but uses fewer parameters and computations. We hope our simple and effective approach can serve as a useful tool for future research in super-resolution model design. Our code is publicly available at https://github.com/HVision-NKU/SRFormer.

Contents

    Installation & Dataset
    Training
    Testing
    Results
    Pretrain Models
    Citations
    License and Acknowledgement

Installation & Dataset

    python 3.8
    pyTorch >= 1.7.0

cd SRFormer
pip install -r requirements.txt
python setup.py develop

Dataset

We use the same training and testing sets as SwinIR, the following datasets need to be downloaded for training.
Task 	Training Set 	Testing Set
classical image SR 	DIV2K (800 training images) or DIV2K +Flickr2K (2650 images) 	Set5 + Set14 + BSD100 + Urban100 + Manga109
lightweight image SR 	DIV2K (800 training images) 	Set5 + Set14 + BSD100 + Urban100 + Manga109
real-world image SR 	DIV2K (800 training images) +Flickr2K (2650 images) + OST (10324 images for sky,water,grass,mountain,building,plant,animal) 	RealSRSet+5images
Training

    Please download the dataset corresponding to the task and place them in the folder specified by the training option in folder /options/train/SRFormer
    Follow the instructions below to train our SRFormer.

# train SRFormer for classical SR task
./scripts/dist_train.sh 4 options/train/SRFormer/train_SRFormer_SRx2_scratch.yml
./scripts/dist_train.sh 4 options/train/SRFormer/train_SRFormer_SRx3_scratch.yml
./scripts/dist_train.sh 4 options/train/SRFormer/train_SRFormer_SRx4_scratch.yml
# train SRFormer for lightweight SR task
./scripts/dist_train.sh 4 options/train/SRFormer/train_SRFormer_light_SRx2_scratch.yml
./scripts/dist_train.sh 4 options/train/SRFormer/train_SRFormer_light_SRx3_scratch.yml
./scripts/dist_train.sh 4 options/train/SRFormer/train_SRFormer_light_SRx4_scratch.yml

Testing

# test SRFormer for classical SR task
python basicsr/test.py -opt options/test/SRFormer/test_SRFormer_DF2Ksrx2.yml
python basicsr/test.py -opt options/test/SRFormer/test_SRFormer_DF2Ksrx3.yml
python basicsr/test.py -opt options/test/SRFormer/test_SRFormer_DF2Ksrx4.yml
# test SRFormer for lightweight SR task
python basicsr/test.py -opt options/test/SRFormer/test_SRFormer_light_DIV2Ksrx2.yml
python basicsr/test.py -opt options/test/SRFormer/test_SRFormer_light_DIV2Ksrx3.yml
python basicsr/test.py -opt options/test/SRFormer/test_SRFormer_light_DIV2Ksrx4.yml

Results

We provide the results on classical image SR, lightweight image SR, realworld image SR. More results can be found in the paper. The visual results of SRFormer will upload to google drive soon.

Classical image SR

    Results of Table 4 in the paper

    Results of Figure 4 in the paper

Lightweight image SR

    Results of Table 5 in the paper

    Results of Figure 5 in the paper

Model size comparison

    Results of Table 1 and Table 2 in the Supplementary Material

Realworld image SR

    Results of Figure 8 in the paper

Pretrain Models

Pretrain Models can be download from google drive. To reproduce the results in the article, you can download them and put them in the /PretrainModel folder.
Citations

You may want to cite:

@article{zhou2023srformer,
  title={SRFormer: Permuted Self-Attention for Single Image Super-Resolution},
  author={Zhou, Yupeng and Li, Zhen and Guo, Chun-Le and Bai, Song and Cheng, Ming-Ming and Hou, Qibin},
  journal={arXiv preprint arXiv:2303.09735},
  year={2023}
}

License and Acknowledgement

This project is released under the Apache 2.0 license. The codes are based on BasicSR, Swin Transformer, and SwinIR. Please also follow their licenses. Thanks for their awesome works.

SRFormerLight_SRx2_DIV2K

Official 4x finetune pretrain from [ATD](https://github.com/LabShuHangGU/Adaptive-Token-Dictionary)

# Adaptive Token Dictionary

This repository is an official implementation of the paper "Transcending the Limit of Local Window: Advanced Super-Resolution Transformer with Adaptive Token Dictionary", CVPR, 2024.

[[Arxiv](https://arxiv.org/abs/2401.08209)] [[visual results](https://drive.google.com/drive/folders/1HwEbAGU6WEw9ZGbFdt__BOJo_5DKflEb?usp=sharing)] [[pretrained models](https://drive.google.com/drive/folders/1D3BvTS1xBcaU1mp50k3pBzUWb7qjRvmB?usp=sharing)]

By Leheng Zhang, Yawei Li, Xingyu Zhou, Xiaorui Zhao, and Shuhang Gu.

> **Abstract:** Single Image Super-Resolution is a classic computer vision problem that involves estimating high-resolution (HR) images from low-resolution (LR) ones. Although deep neural networks (DNNs), especially Transformers for super-resolution, have seen significant advancements in recent years, challenges still remain, particularly in limited receptive field caused by window-based self-attention. To address these issues, we introduce a group of auxiliary **A**daptive **T**oken **D**ictionary to SR Transformer and establish an **ATD**-SR method. The introduced token dictionary could learn prior information from training data and adapt the learned prior to specific testing image through an adaptive refinement step. The refinement strategy could not only provide global information to all input tokens but also group image tokens into categories. Based on category partitions, we further propose a category-based self-attention mechanism designed to leverage distant but similar tokens for enhancing input features. The experimental results show that our method achieves the best performance on various single image super-resolution benchmarks.
> 
> <img width="800" src="figures/tdca-acmsa.png"> 
> <img width="800" src="figures/arch.png"> 
> <br/><br/>
> <img width="800" src="figures/viscomp_076.png">

003_ATD_SRx4_finetune

Official DAT2 4x Pretrain of Dual Aggregation Transformer for Image Super-Resolution DAT

Transformer has recently gained considerable popularity in low-level vision tasks, including image super-resolution (SR). These networks utilize self-attention along different dimensions, spatial or channel, and achieve impressive performance. This inspires us to combine the two dimensions in Transformer for a more powerful representation capability. Based on the above idea, we propose a novel Transformer model, Dual Aggregation Transformer (DAT), for image SR. Our DAT aggregates features across spatial and channel dimensions, in the inter-block and intra-block dual manner. Specifically, we alternately apply spatial and channel self-attention in consecutive Transformer blocks. The alternate strategy enables DAT to capture the global context and realize inter-block feature aggregation. Furthermore, we propose the adaptive interaction module (AIM) and the spatial-gate feed-forward network (SGFN) to achieve intra-block feature aggregation. AIM complements two self-attention mechanisms from corresponding dimensions. Meanwhile, SGFN introduces additional non-linear spatial information in the feed-forward network. Extensive experiments show that our DAT surpasses current methods.

DAT_2_x4

[ESRGAN](https://github.com/xinntao/ESRGAN)

Pretrained: RRDB_ESRGAN_x4.pth

4xESRGAN

4x realistic upscaler that may also work for general purpose usage. Trained on Bicubic downscaled tiles from very high quality images.

FrankendataPretainer_SRFormer

Purpose: Pretrain to quick start new models.

craft pretrain

Model trained only on downscale degradation (bicubic, bilinear, nearest, lanczos and mitchell). Can be used to start new Real-CUGAN models. ONNX included on the dir.

cugan_pretrain

Model trained only on downscale degradation (bicubic, bilinear, nearest, lanczos and mitchell). Can be used to start new DCTLSA models.

dctlsa_pretrained

Meant to quick start new DITN models trained on `neosr`. Trained only on downscale degradation (bicubic, bilinear, nearest, lanczos and mitchell).
Note: [Flash Attention](https://github.com/muslll/neosr/blob/master/neosr/archs/ditn_arch.py#L108) breaks inference compatibility with official code.

ditn_w16_pretrain

[Link to Github Release](https://github.com/Phhofm/models/releases/tag/4xDRCT-mssim-pretrains)

Since no [DRCT](https://github.com/ming053l/DRCT/tree/main) model releases exist yet, I am releasing my drct-s, drct and drct-l pretrains. 
These can be used to speed up and stabilize early training stages when training new drct models.
Trained with mssim (and augs, and color and lumaloss) on downscaled [nomosuni](https://github.com/muslll/neosr/tree/0ecbeb98dcefd97801da0ea39ca2cd03a87fce21?tab=readme-ov-file#datasets).
Training files are also provided additionally.

## 4xmssim_drct-l_pretrain

**Scale:** 4  
**Architecture:** [DRCT](https://github.com/ming053l/DRCT)  
**Architecture Option:** [DRCT-L](https://github.com/ming053l/DRCT/blob/8c13e63135f494913cd504c073e41ef52250d1d4/options/train/train_DRCT-L_SRx4_ImageNet_from_scratch.yml#L55)  

**Author:** Philip Hofmann  
**License:** CC-BY-0.4  
**Purpose:** Pretrained  
**Subject:** Anime, Photography  
**Input Type:** Images  
**Release Date:** 28.04.2024  

**Dataset:** [nomos_uni](https://github.com/muslll/neosr?tab=readme-ov-file#datasets)  
**Dataset Size:** 2989  
**OTF (on the fly augmentations):** No  
**Pretrained Model:** None (=From Scratch)  
**Iterations:** 108'000  
**Batch Size:** 2-6  
**GT Size:** 128-320  

**Description:** A pretrain to start drct-l model training, since there are no official released drct pretrains yet I trained these myself and release them here.

4xmssim_drct-l_pretrain

[Link to Github Release](https://github.com/Phhofm/models/releases/tag/4xDRCT-mssim-pretrains)

Since no [DRCT](https://github.com/ming053l/DRCT/tree/main) model releases exist yet, I am releasing my drct-s, drct and drct-l pretrains. 
These can be used to speed up and stabilize early training stages when training new drct models.
Trained with mssim (and augs, and color and lumaloss) on downscaled [nomosuni](https://github.com/muslll/neosr/tree/0ecbeb98dcefd97801da0ea39ca2cd03a87fce21?tab=readme-ov-file#datasets).
Training files are also provided additionally.

## 4xmssim_drct_pretrain

**Scale:** 4  
**Architecture:** [DRCT](https://github.com/ming053l/DRCT)  
**Architecture Option:** [DRCT](https://github.com/ming053l/DRCT/blob/8c13e63135f494913cd504c073e41ef52250d1d4/options/train/train_DRCT_SRx4_ImageNet_from_scratch.yml#L55)  

**Author:** Philip Hofmann  
**License:** CC-BY-0.4  
**Purpose:** Pretrained  
**Subject:** Anime, Photography  
**Input Type:** Images  
**Release Date:** 28.04.2024  

**Dataset:** [nomos_uni](https://github.com/muslll/neosr?tab=readme-ov-file#datasets)  
**Dataset Size:** 2989  
**OTF (on the fly augmentations):** No  
**Pretrained Model:** None (=From Scratch)  
**Iterations:** 95'000  
**Batch Size:** 2-6  
**GT Size:** 128-320   

**Description:** A pretrain to start drct model training, since there are no official released drct pretrains yet I trained these myself and release them here.

4xmssim_drct_pretrain

[Link to Github Release](https://github.com/Phhofm/models/releases/tag/4xDRCT-mssim-pretrains)

Since no [DRCT](https://github.com/ming053l/DRCT/tree/main) model releases exist yet, I am releasing my drct-s, drct and drct-l pretrains. 
These can be used to speed up and stabilize early training stages when training new drct models.
Trained with mssim (and augs, and color and lumaloss) on downscaled [nomosuni](https://github.com/muslll/neosr/tree/0ecbeb98dcefd97801da0ea39ca2cd03a87fce21?tab=readme-ov-file#datasets).
Training files are also provided additionally.


## 4xmssim_drct-s_pretrain.pth  

**Scale:** 4  
**Architecture:** [DRCT](https://github.com/ming053l/DRCT)  
**Architecture Option:** [DRCT-S](https://github.com/muslll/neosr/blob/0ecbeb98dcefd97801da0ea39ca2cd03a87fce21/neosr/archs/drct_arch.py#L923)  

**Author:** Philip Hofmann  
**License:** CC-BY-0.4  
**Purpose:** Pretrained  
**Subject:** Anime, Photography  
**Input Type:** Images  
**Release Date:** 28.04.2024  

**Dataset:** [nomos_uni](https://github.com/muslll/neosr?tab=readme-ov-file#datasets)  
**Dataset Size:** 2989  
**OTF (on the fly augmentations):** No  
**Pretrained Model:** None (=From Scratch)  
**Iterations:** 75'000  
**Batch Size:** 6  
**GT Size:** 128-320  

**Description:** A pretrain to start drct-s model training, since there are no official released drct pretrains yet I trained these myself and release them here.

4xmssim_drct-s_pretrain

[Link to Github Release](https://github.com/Phhofm/models/releases/tag/4xmssim_hma_pretrains)

Since no official [HMA](https://github.com/korouuuuu/HMA) model releases exist yet, I am releasing my [hma](https://github.com/muslll/neosr/blob/a50b56de418c938425fd3606697f62d61dbddfc3/neosr/archs/hma_arch.py#L1148) and [hma_medium](https://github.com/muslll/neosr/blob/a50b56de418c938425fd3606697f62d61dbddfc3/neosr/archs/hma_arch.py#L1386) mssim pretrains.   
These can be used to speed up and stabilize early training stages when training new hma models.  
Trained with mssim on [nomosv2](https://github.com/muslll/neosr/tree/a50b56de418c938425fd3606697f62d61dbddfc3?tab=readme-ov-file#-datasets).  


## 4xmssim_hma_pretrain  

**Scale:** 4  
**Architecture:** [HMA](https://github.com/korouuuuu/HMA)  
**Architecture Option:** [hma](https://github.com/muslll/neosr/blob/a50b56de418c938425fd3606697f62d61dbddfc3/neosr/archs/hma_arch.py#L1148)  

**Author:** Philip Hofmann  
**License:** CC-BY-0.4  
**Purpose:** Pretrained  
**Subject:** Photography  
**Input Type:** Images  
**Release Date:** 19.07.2024  

**Dataset:** [nomosv2](https://github.com/muslll/neosr/tree/a50b56de418c938425fd3606697f62d61dbddfc3?tab=readme-ov-file#-datasets)  
**Dataset Size:** 6000  
**OTF (on the fly augmentations):** No  
**Pretrained Model:** None (=From Scratch)  
**Iterations:** 205'000  
**Batch Size:** 4  
**Patch Size:** 96  

**Description:** A pretrain to start hma model training.  

---

## 4xmssim_hma_medium_pretrain  

**Scale:** 4  
**Architecture:** [HMA](https://github.com/korouuuuu/HMA)  
**Architecture Option:** [hma_medium](https://github.com/muslll/neosr/blob/a50b56de418c938425fd3606697f62d61dbddfc3/neosr/archs/hma_arch.py#L1386)  

**Author:** Philip Hofmann  
**License:** CC-BY-0.4  
**Purpose:** Pretrained  
**Subject:** Photography  
**Input Type:** Images  
**Release Date:** 19.07.2024  

**Dataset:** [nomosv2](https://github.com/muslll/neosr/tree/a50b56de418c938425fd3606697f62d61dbddfc3?tab=readme-ov-file#-datasets)  
**Dataset Size:** 6000  
**OTF (on the fly augmentations):** No  
**Pretrained Model:** None (=From Scratch)  
**Iterations:** 150'000  
**Batch Size:** 4  
**Patch Size:** 48  

**Description:** A pretrain to start hma_medium model training.  

---

**Showcase:**  
[slow.pics](https://slow.pics/c/Eoq18VDx)

 4xmssim_hma_pretrains

[Link to Github Release](https://github.com/Phhofm/models/releases/tag/4xmssim_mosr_pretrain)

# 4xmssim_mosr_pretrain  
Scale: 4  
Architecture: [MoSR](https://github.com/umzi2/MoSR)  
Architecture Option: [mosr](https://github.com/umzi2/MoSR/blob/95c5bf73cca014493fe952c2fbc0bdbe593da08f/neosr/archs/mosr_arch.py#L117)  

Author: Philip Hofmann  
License: CC-BY-0.4  
Purpose: Pretrained  
Subject: Photography  
Input Type: Images  
Release Date: 25.08.2024  

Dataset: [nomosv2](https://github.com/muslll/neosr/?tab=readme-ov-file#-datasets)  
Dataset Size: 6000  
OTF (on the fly augmentations): No  
Pretrained Model: None  
Iterations: 420'000  
Batch Size: 6  
Patch Size: 64  

Description:   
A 4x [MoSR](https://github.com/umzi2/MoSR) pretrain model with mssim only on the Nomos2 dataset, usable as a pretrain when starting training new mosr models.  

Training metrics graphs:  
![image](https://github.com/user-attachments/assets/b66b75d8-f83c-4b6e-8094-ef5e5cba9c81)  

Showcase:  
[Slowpics](https://slow.pics/c/T59QQaQ0)

 4xmssim_mosr_pretrain

4xmssim_realplksr_dysample_pretrain  
Scale: 4  
Architecture: [RealPLKSR with Dysample](https://github.com/muslll/neosr/?tab=readme-ov-file#supported-archs)  
Architecture Option: [realplksr](https://github.com/muslll/neosr/blob/master/neosr/archs/realplksr_arch.py)  

Author: Philip Hofmann  
License: CC-BY-0.4  
Purpose: Pretrained  
Subject: Photography  
Input Type: Images  
Release Date: 27.06.2024  

Dataset: [nomosv2](https://github.com/muslll/neosr/?tab=readme-ov-file#-datasets)  
Dataset Size: 6000  
OTF (on the fly augmentations): No  
Pretrained Model: None (=From Scratch)  
Iterations: 200'000  
Batch Size: 8  
GT Size: 192, 512  

Description: [Dysample](https://arxiv.org/pdf/2308.15085) had been recently added to RealPLKSR, which from what I had seen can resolve or help avoid the checkerboard / grid pattern on inference outputs. So with the [commits from three days ago, the 24.06.24, on neosr](https://github.com/muslll/neosr/commits/master/?since=2024-06-24&until=2024-06-24), I wanted to create a 4x photo pretrain I can then use to train more realplksr models with dysample specifically to stabilize training at the beginning.

Showcase:  
[Imgsli](https://imgsli.com/Mjc0OTA1)  
[Slowpics](https://slow.pics/c/I9grkcqM)

4xmssim_realplksr_dysample_pretrain

[Github Release Link](https://github.com/Phhofm/models/releases/4xSPAN_pretrains)

[Neosr](https://github.com/muslll/neosr)'s latest update from yesterday included a [new adaptation of the multi-scale ssim loss](https://github.com/muslll/neosr/wiki/Losses#mssim_opt).     
This was an experiment to test out the difference between making a SPAN pretrain with pixel loss with L1 criteria (as often used in research) vs mssim loss as its only loss.     
Models are provided so they can be used for tests or also used as a pretrain for another SPAN model.  

---

## 4xmssim_span_pretrain

Scale: 4  
Architecture: SPAN  

Author: Philip Hofmann    
License: CC-BY-4.0  
Purpose: Pretrain  
Subject: Realistic, Anime  
Date: 10.04.2024  

Dataset: [nomos_uni](https://github.com/muslll/neosr)  
Dataset Size: 2989   
OTF (on the fly augmentations): No  
Pretrained Model: None  
Iterations: 80'000  
Batch Size: 12  
GT Size: 128  

Description: 4x SPAN pretrain trained on [neosr](https://github.com/muslll/neosr)'s [new adaptation of the multi-scale ssim loss](https://github.com/muslll/neosr/wiki/Losses#mssim_opt) from yesterdays update on downsampled nomos_uni dataset using kim's [dataset destroyer](https://github.com/Kim2091/helpful-scripts/tree/main/Dataset%20Destroyer) with down_up,linear,cubic_mitchell,lanczos,gauss,box (while down_up used the same and with range = 0.15,1.5). 
The new augmentations except CutBlur have also been used (since CutBlur is meant to be applied to real-world SR and may cause undesired effects if applied to bicubic-only). 
Config and training log provided for more details.
 
Showcase: 
[7 Slowpics Examples](https://slow.pics/c/zyilXhKU)

No example images yet

1xESRGAN

Good for

Similar models

Model

Rights