Description: 4x photo upscaler for faces, trained on the FaceUp dataset. These models are an improvement over the previously released 4xFFHQDAT and are its successors. These models are released together with the FaceUp dataset, plus the accompanying [youtube video](https://www.youtube.com/watch?v=TBiVIzQkptI)

This model comes in 4 different versions:  
4xFaceUpDAT (for good quality input)  
4xFaceUpLDAT (for lower quality input, can additionally denoise)  
4xFaceUpSharpDAT (for good quality input, produces sharper output, trained without USM but sharpened input images, good quality input)  
4xFaceUpSharpLDAT (for lower quality input, produces sharper output, trained without USM but sharpened input images, can additionally denoise)  

I recommend trying out 4xFaceUpDAT

4xFaceUpDAT

4xFaceUpSharpDAT

A normal ESRGAN model without degradations and without any pretrain, simply an RRDBNet model trained on paired dataset (4x downsampled) on the full LSDIR dataset (84,991 images / 165 GB)

4xLSDIR

Upscale small good quality photos to 4x their size

My first ever model 😄 Well, it’s not the best, but, it’s something 😉 
I provide some 15 examples from the validation set here for you to visually see the generated output (with chaiNNer), photo dimensions are in the name

LSDIR Compact

Upscale photos to x4 their size.

4xLSDIRCompactv2 supersedes the previously released models, it combines all my progress on my compact model. Both CompactC and CompactR had received around 8 hours more training since release with batch size 10 (CompactR had only been up to 5 on release), and these two were then interpolated together. This allows v2 to handle some degradations, while preserving the details of the CompactC model. Examples: https://imgsli.com/MTY0Njgz/0/2

LSDIR Compact v2

Upscale small photos with compression to 4x their size.

Trying to extend my previous model to be able to handle compression (JPG 100-30) by manually altering the training dataset, since 4xLSDIRCompact can't handle compression. Use this instead of 4xLSDIRCompact if your photo has compression (like an image from the web).

LSDIR Compact C

Upscale compressed photos to x4 their size.

Able to handle JPG compression (30-100).

LSDIR Compact C3

Upscale good quality input photos to x4 their size

The original 4xLSDIRCompact a bit more trained, cannot handle degradation

LSDIRCompactN

Upscale small photos with compression, noise and slight blur to 4x their size.

Extending my last 4xLSDIRCompact model to Real-ESRGAN, meaning trained on synthetic data instead to handle more kinds of degradations, it should be able to handle compression, noise, and slight blur.

Here is a comparison to show that 4xLSDIRCompact cannot handle compression artifacts, and that these two models will produce better output for that specific scenario. These models are not ‘better’ than the previous one, they are just meant to handle a different use case.

LSDIR Compact R

Upscale (degraded) photos to x4 their size. 

Trained on synthetic data, meant to handle more degradations

LSDIR Compact R3

The RealESRGAN_x4plus finetuned with the big LSDIR dataset (84,991 images / 165 GB), with manually added jpg compression.

4xLSDIRplusC

The RealESRGAN_x4plus finetuned with the big LSDIR dataset (84,991 images / 165 GB), no degradation.

4xLSDIRplusN

[Link to Github Release](https://github.com/Phhofm/models/releases/tag/4xNomos2_hq_dat2)

# 4xNomos2_hq_dat2   
Scale: 4   
Architecture: [DAT](https://github.com/zhengchen1999/dat)   
Architecture Option: [dat2](https://github.com/muslll/neosr/blob/5fba7f162d36052010169e6517dec3b406c569ab/neosr/archs/dat_arch.py#L1111)   

Author: Philip Hofmann   
License: CC-BY-0.4   
Purpose: Upscaler   
Subject: Photography   
Input Type: Images   
Release Date: 29.08.2024   

Dataset: [nomosv2](https://github.com/muslll/neosr/?tab=readme-ov-file#-datasets)   
Dataset Size: 6000   
OTF (on the fly augmentations): No   
Pretrained Model: DAT_2_x4   
Iterations: 140'000   
Batch Size: 4   
Patch Size: 48   

Description:    
A dat2 4x upscaling model, similiar to the [4xNomos2_hq_mosr](https://github.com/Phhofm/models/releases/tag/4xNomos2_hq_mosr) model, trained and for usage on non-degraded input to give good quality output.    

I scored 7 validation outputs of each of the 21 checkpoints (10k-210k) of this model training with 68 metrics.    
[The metric scores can be found in this google sheet](https://docs.google.com/spreadsheets/d/1NL-by7WvZyDMHj5XN8UeDALVSSwH70IKvwV65ATWqrA/edit?usp=sharing).   
The corresponding image files for this scoring can be [found here](https://drive.google.com/file/d/1ZTp9fBMeawftNqzg4RN9_zIvHtul5jVc/view?usp=sharing)     
Screenshot of the google sheet:     
![|100](https://i.slow.pics/VZJTsrUv.webp)

Release checkpoint has been selected by looking at the scores, manually inspecting, and then getting responses on discord to this quick visual test, A B or C, which denote different checkpoints: https://slow.pics/c/8Akzj6rR   

Checkpoint B is the one released here, but you can also try out [Checkpoint A](https://github.com/Phhofm/models/releases/download/4xNomos2_hq_dat2/4xNomos2_hq_dat2_150000.pth) or [Checkpoint C](https://github.com/Phhofm/models/releases/download/4xNomos2_hq_dat2/4xNomos2_hq_dat2_10000.pth) if you like them better.

## Model Showcase:
[Slowpics](https://slow.pics/c/yuue9WpF)

4xNomos2_hq_dat2

[Link to Github Release](https://github.com/Phhofm/models/releases/tag/4xNomos2_hq_mosr)

# 4xNomos2_hq_mosr  
Scale: 4  
Architecture: [MoSR](https://github.com/umzi2/MoSR)  
Architecture Option: [mosr](https://github.com/umzi2/MoSR/blob/95c5bf73cca014493fe952c2fbc0bdbe593da08f/neosr/archs/mosr_arch.py#L117)  

Author: Philip Hofmann  
License: CC-BY-0.4  
Purpose: Upscaler  
Subject: Photography  
Input Type: Images  
Release Date: 25.08.2024  

Dataset: [nomosv2](https://github.com/muslll/neosr/?tab=readme-ov-file#-datasets)  
Dataset Size: 6000  
OTF (on the fly augmentations): No  
Pretrained Model: [4xmssim_mosr_pretrain](https://github.com/Phhofm/models/releases/tag/4xmssim_mosr_pretrain)  
Iterations: 190'000  
Batch Size: 6  
Patch Size: 64  

Description:   
A 4x [MoSR](https://github.com/umzi2/MoSR) upscaling model, meant for non-degraded input, since this model was trained on non-degraded input to give good quality output.   

If your input is degraded, use a 1x degrade model first. So for example if your input is a .jpg file, you could use a 1x dejpg model first.  

Model Showcase:  [Slowpics](https://slow.pics/c/cqGJb0gT)

4xNomos2_hq_mosr

4xNomos2_otf_esrgan

Scale: 4  
Architecture: ESRGAN  
Architecture Option: esrgan  
[Github Release Link](https://github.com/Phhofm/models/releases/tag/4xNomos2_otf_esrgan)  

Author: Philip Hofmann  
License: CC-BY-0.4  
Purpose: Restoration  
Subject: Photography  
Input Type: Images  
Release Date: 22.06.2024  

Dataset: Nomos-v2  
Dataset Size: 6000  
OTF (on the fly augmentations): Yes  
Pretrained Model: RealESRGAN_x4plus  
Iterations: 246'000  
Batch Size: 8  
GT Size: 256  

Description:  

4x ESRGAN model for photography, trained using the Real-ESRGAN otf degradation pipeline.  

Showcase:
[Slow Pics 8 Examples](https://slow.pics/s/Y5wdyTPh?canvas-mode=fit-width&image-fit=contain)

4xNomos2_otf_esrgan

4xNomos2_realplksr_dysample  
Scale: 4  
Architecture: [RealPLKSR with Dysample](https://github.com/muslll/neosr/?tab=readme-ov-file#supported-archs)  
Architecture Option: [realplksr](https://github.com/muslll/neosr/blob/master/neosr/archs/realplksr_arch.py)  
[Github Release](https://github.com/Phhofm/models/releases/tag/4xNomos2_realplksr_dysample)  

Author: Philip Hofmann  
License: CC-BY-0.4  
Purpose: Pretrained  
Subject: Photography  
Input Type: Images  
Release Date: 30.06.2024  

Dataset: [nomosv2](https://github.com/muslll/neosr/?tab=readme-ov-file#-datasets)  
Dataset Size: 6000  
OTF (on the fly augmentations): No  
Pretrained Model: [4xmssim_realplksr_dysample_pretrain](https://github.com/Phhofm/models/releases/tag/4xmssim_realplksr_dysample_pretrain)  
Iterations: 185'000  
Batch Size: 8  
GT Size: 256, 512  

Description: 
A Dysample RealPLKSR 4x upscaling model that was trained with / handles jpg compression down to 70 on the Nomosv2 dataset, preserves DoF.   
Based on the [4xmssim_realplksr_dysample_pretrain](https://github.com/Phhofm/models/releases/tag/4xmssim_realplksr_dysample_pretrain) I released 3 days ago.  
This model affects / saturate colors, which can be counteracted a bit by using wavelet color fix, as used in these examples.  

Showcase:
[Slowpics](https://slow.pics/s/p3sMnd5l)

4xNomos2_realplksr_dysample

[Link to Github Release](https://github.com/Phhofm/models/releases/4xNomos8k_atd_jpg)

Name: 4xNomos8k_atd_jpg   
License: CC BY 4.0   
Author: Philip Hofmann   
Network: [ATD](https://github.com/LabShuHangGU/Adaptive-Token-Dictionary)   
Scale: 4   
Release Date: 22.03.2024   
Purpose: 4x photo upscaler, handles jpg compression   
Iterations: 240'000   
epoch: 152   
batch_size: 6, 3   
HR_size: 128, 192   
Dataset: nomos8k   
Number of train images: 8492   
OTF Training: Yes   
Pretrained_Model_G: 003_ATD_SRx4_finetune   

Description:
4x photo upscaler which handles jpg compression. This model will preserve noise. Trained on the very recently released (~2 weeks ago) Adaptive-Token-Dictionary network.   

Training details: 
AdamW optimizer with U-Net SN discriminator and BFloat16.
Degraded with otf jpg compression down to 40, re-compression down to 40, together with resizes and the blur kernels.  
Losses: PixelLoss using CHC (Clipped Huber with Cosine Similarity Loss), PerceptualLoss using Huber, GANLoss, [LDL](https://github.com/csjliang/LDL) using Huber, YCbCr Color Loss (bt601) and Luma Loss (CIE XYZ) on [neosr](https://github.com/muslll/neosr).

7 Examples:
[Slowpics](https://slow.pics/s/uwnoI435)

 4xNomos8k_atd_jpg

I release my span otf series:
They come in three variations: weak, middle, and strong.
Mainly meant for photos (can be tried on other things of course).

(Also there is a non-otf span model I had been working on simultaneously that I will release shortly, should give better results on less degraded input in comparison to this span otf series)

Basically I trained the otf_strong for 90k iter and then medium and weak based off that, with some more training to de-learn (tone down) the (too?) strong degradations. Used discrim resets to correct occuring colorloss in all of them. gt size was for the most part 512 with batch 9 (since i hoped it would give better results) with 0.55 it/s training speed (first 40k at the beginning were gt size 256 with batch 20 with 0.58 it/s).

Nomos8k_span_otf_medium

[Link to Github Release](https://github.com/Phhofm/models/releases/tag/4xDRCT-mssim-pretrains)

Since no [DRCT](https://github.com/ming053l/DRCT/tree/main) model releases exist yet, I am releasing my drct-s, drct and drct-l pretrains. 
These can be used to speed up and stabilize early training stages when training new drct models.
Trained with mssim (and augs, and color and lumaloss) on downscaled [nomosuni](https://github.com/muslll/neosr/tree/0ecbeb98dcefd97801da0ea39ca2cd03a87fce21?tab=readme-ov-file#datasets).
Training files are also provided additionally.

## 4xmssim_drct-l_pretrain

**Scale:** 4  
**Architecture:** [DRCT](https://github.com/ming053l/DRCT)  
**Architecture Option:** [DRCT-L](https://github.com/ming053l/DRCT/blob/8c13e63135f494913cd504c073e41ef52250d1d4/options/train/train_DRCT-L_SRx4_ImageNet_from_scratch.yml#L55)  

**Author:** Philip Hofmann  
**License:** CC-BY-0.4  
**Purpose:** Pretrained  
**Subject:** Anime, Photography  
**Input Type:** Images  
**Release Date:** 28.04.2024  

**Dataset:** [nomos_uni](https://github.com/muslll/neosr?tab=readme-ov-file#datasets)  
**Dataset Size:** 2989  
**OTF (on the fly augmentations):** No  
**Pretrained Model:** None (=From Scratch)  
**Iterations:** 108'000  
**Batch Size:** 2-6  
**GT Size:** 128-320  

**Description:** A pretrain to start drct-l model training, since there are no official released drct pretrains yet I trained these myself and release them here.

4xmssim_drct-l_pretrain

[Link to Github Release](https://github.com/Phhofm/models/releases/tag/4xDRCT-mssim-pretrains)

Since no [DRCT](https://github.com/ming053l/DRCT/tree/main) model releases exist yet, I am releasing my drct-s, drct and drct-l pretrains. 
These can be used to speed up and stabilize early training stages when training new drct models.
Trained with mssim (and augs, and color and lumaloss) on downscaled [nomosuni](https://github.com/muslll/neosr/tree/0ecbeb98dcefd97801da0ea39ca2cd03a87fce21?tab=readme-ov-file#datasets).
Training files are also provided additionally.

## 4xmssim_drct_pretrain

**Scale:** 4  
**Architecture:** [DRCT](https://github.com/ming053l/DRCT)  
**Architecture Option:** [DRCT](https://github.com/ming053l/DRCT/blob/8c13e63135f494913cd504c073e41ef52250d1d4/options/train/train_DRCT_SRx4_ImageNet_from_scratch.yml#L55)  

**Author:** Philip Hofmann  
**License:** CC-BY-0.4  
**Purpose:** Pretrained  
**Subject:** Anime, Photography  
**Input Type:** Images  
**Release Date:** 28.04.2024  

**Dataset:** [nomos_uni](https://github.com/muslll/neosr?tab=readme-ov-file#datasets)  
**Dataset Size:** 2989  
**OTF (on the fly augmentations):** No  
**Pretrained Model:** None (=From Scratch)  
**Iterations:** 95'000  
**Batch Size:** 2-6  
**GT Size:** 128-320   

**Description:** A pretrain to start drct model training, since there are no official released drct pretrains yet I trained these myself and release them here.

4xmssim_drct_pretrain

[Link to Github Release](https://github.com/Phhofm/models/releases/tag/4xDRCT-mssim-pretrains)

Since no [DRCT](https://github.com/ming053l/DRCT/tree/main) model releases exist yet, I am releasing my drct-s, drct and drct-l pretrains. 
These can be used to speed up and stabilize early training stages when training new drct models.
Trained with mssim (and augs, and color and lumaloss) on downscaled [nomosuni](https://github.com/muslll/neosr/tree/0ecbeb98dcefd97801da0ea39ca2cd03a87fce21?tab=readme-ov-file#datasets).
Training files are also provided additionally.


## 4xmssim_drct-s_pretrain.pth  

**Scale:** 4  
**Architecture:** [DRCT](https://github.com/ming053l/DRCT)  
**Architecture Option:** [DRCT-S](https://github.com/muslll/neosr/blob/0ecbeb98dcefd97801da0ea39ca2cd03a87fce21/neosr/archs/drct_arch.py#L923)  

**Author:** Philip Hofmann  
**License:** CC-BY-0.4  
**Purpose:** Pretrained  
**Subject:** Anime, Photography  
**Input Type:** Images  
**Release Date:** 28.04.2024  

**Dataset:** [nomos_uni](https://github.com/muslll/neosr?tab=readme-ov-file#datasets)  
**Dataset Size:** 2989  
**OTF (on the fly augmentations):** No  
**Pretrained Model:** None (=From Scratch)  
**Iterations:** 75'000  
**Batch Size:** 6  
**GT Size:** 128-320  

**Description:** A pretrain to start drct-s model training, since there are no official released drct pretrains yet I trained these myself and release them here.

4xmssim_drct-s_pretrain

[Link to Github Release](https://github.com/Phhofm/models/releases/tag/4xmssim_hma_pretrains)

Since no official [HMA](https://github.com/korouuuuu/HMA) model releases exist yet, I am releasing my [hma](https://github.com/muslll/neosr/blob/a50b56de418c938425fd3606697f62d61dbddfc3/neosr/archs/hma_arch.py#L1148) and [hma_medium](https://github.com/muslll/neosr/blob/a50b56de418c938425fd3606697f62d61dbddfc3/neosr/archs/hma_arch.py#L1386) mssim pretrains.   
These can be used to speed up and stabilize early training stages when training new hma models.  
Trained with mssim on [nomosv2](https://github.com/muslll/neosr/tree/a50b56de418c938425fd3606697f62d61dbddfc3?tab=readme-ov-file#-datasets).  


## 4xmssim_hma_pretrain  

**Scale:** 4  
**Architecture:** [HMA](https://github.com/korouuuuu/HMA)  
**Architecture Option:** [hma](https://github.com/muslll/neosr/blob/a50b56de418c938425fd3606697f62d61dbddfc3/neosr/archs/hma_arch.py#L1148)  

**Author:** Philip Hofmann  
**License:** CC-BY-0.4  
**Purpose:** Pretrained  
**Subject:** Photography  
**Input Type:** Images  
**Release Date:** 19.07.2024  

**Dataset:** [nomosv2](https://github.com/muslll/neosr/tree/a50b56de418c938425fd3606697f62d61dbddfc3?tab=readme-ov-file#-datasets)  
**Dataset Size:** 6000  
**OTF (on the fly augmentations):** No  
**Pretrained Model:** None (=From Scratch)  
**Iterations:** 205'000  
**Batch Size:** 4  
**Patch Size:** 96  

**Description:** A pretrain to start hma model training.  

---

## 4xmssim_hma_medium_pretrain  

**Scale:** 4  
**Architecture:** [HMA](https://github.com/korouuuuu/HMA)  
**Architecture Option:** [hma_medium](https://github.com/muslll/neosr/blob/a50b56de418c938425fd3606697f62d61dbddfc3/neosr/archs/hma_arch.py#L1386)  

**Author:** Philip Hofmann  
**License:** CC-BY-0.4  
**Purpose:** Pretrained  
**Subject:** Photography  
**Input Type:** Images  
**Release Date:** 19.07.2024  

**Dataset:** [nomosv2](https://github.com/muslll/neosr/tree/a50b56de418c938425fd3606697f62d61dbddfc3?tab=readme-ov-file#-datasets)  
**Dataset Size:** 6000  
**OTF (on the fly augmentations):** No  
**Pretrained Model:** None (=From Scratch)  
**Iterations:** 150'000  
**Batch Size:** 4  
**Patch Size:** 48  

**Description:** A pretrain to start hma_medium model training.  

---

**Showcase:**  
[slow.pics](https://slow.pics/c/Eoq18VDx)

 4xmssim_hma_pretrains

[Link to Github Release](https://github.com/Phhofm/models/releases/tag/4xmssim_mosr_pretrain)

# 4xmssim_mosr_pretrain  
Scale: 4  
Architecture: [MoSR](https://github.com/umzi2/MoSR)  
Architecture Option: [mosr](https://github.com/umzi2/MoSR/blob/95c5bf73cca014493fe952c2fbc0bdbe593da08f/neosr/archs/mosr_arch.py#L117)  

Author: Philip Hofmann  
License: CC-BY-0.4  
Purpose: Pretrained  
Subject: Photography  
Input Type: Images  
Release Date: 25.08.2024  

Dataset: [nomosv2](https://github.com/muslll/neosr/?tab=readme-ov-file#-datasets)  
Dataset Size: 6000  
OTF (on the fly augmentations): No  
Pretrained Model: None  
Iterations: 420'000  
Batch Size: 6  
Patch Size: 64  

Description:   
A 4x [MoSR](https://github.com/umzi2/MoSR) pretrain model with mssim only on the Nomos2 dataset, usable as a pretrain when starting training new mosr models.  

Training metrics graphs:  
![image](https://github.com/user-attachments/assets/b66b75d8-f83c-4b6e-8094-ef5e5cba9c81)  

Showcase:  
[Slowpics](https://slow.pics/c/T59QQaQ0)

 4xmssim_mosr_pretrain

4xmssim_realplksr_dysample_pretrain  
Scale: 4  
Architecture: [RealPLKSR with Dysample](https://github.com/muslll/neosr/?tab=readme-ov-file#supported-archs)  
Architecture Option: [realplksr](https://github.com/muslll/neosr/blob/master/neosr/archs/realplksr_arch.py)  

Author: Philip Hofmann  
License: CC-BY-0.4  
Purpose: Pretrained  
Subject: Photography  
Input Type: Images  
Release Date: 27.06.2024  

Dataset: [nomosv2](https://github.com/muslll/neosr/?tab=readme-ov-file#-datasets)  
Dataset Size: 6000  
OTF (on the fly augmentations): No  
Pretrained Model: None (=From Scratch)  
Iterations: 200'000  
Batch Size: 8  
GT Size: 192, 512  

Description: [Dysample](https://arxiv.org/pdf/2308.15085) had been recently added to RealPLKSR, which from what I had seen can resolve or help avoid the checkerboard / grid pattern on inference outputs. So with the [commits from three days ago, the 24.06.24, on neosr](https://github.com/muslll/neosr/commits/master/?since=2024-06-24&until=2024-06-24), I wanted to create a 4x photo pretrain I can then use to train more realplksr models with dysample specifically to stabilize training at the beginning.

Showcase:  
[Imgsli](https://imgsli.com/Mjc0OTA1)  
[Slowpics](https://slow.pics/c/I9grkcqM)

4xmssim_realplksr_dysample_pretrain

[Github Release Link](https://github.com/Phhofm/models/releases/4xSPAN_pretrains)

[Neosr](https://github.com/muslll/neosr)'s latest update from yesterday included a [new adaptation of the multi-scale ssim loss](https://github.com/muslll/neosr/wiki/Losses#mssim_opt).     
This was an experiment to test out the difference between making a SPAN pretrain with pixel loss with L1 criteria (as often used in research) vs mssim loss as its only loss.     
Models are provided so they can be used for tests or also used as a pretrain for another SPAN model.  

---

## 4xmssim_span_pretrain

Scale: 4  
Architecture: SPAN  

Author: Philip Hofmann    
License: CC-BY-4.0  
Purpose: Pretrain  
Subject: Realistic, Anime  
Date: 10.04.2024  

Dataset: [nomos_uni](https://github.com/muslll/neosr)  
Dataset Size: 2989   
OTF (on the fly augmentations): No  
Pretrained Model: None  
Iterations: 80'000  
Batch Size: 12  
GT Size: 128  

Description: 4x SPAN pretrain trained on [neosr](https://github.com/muslll/neosr)'s [new adaptation of the multi-scale ssim loss](https://github.com/muslll/neosr/wiki/Losses#mssim_opt) from yesterdays update on downsampled nomos_uni dataset using kim's [dataset destroyer](https://github.com/Kim2091/helpful-scripts/tree/main/Dataset%20Destroyer) with down_up,linear,cubic_mitchell,lanczos,gauss,box (while down_up used the same and with range = 0.15,1.5). 
The new augmentations except CutBlur have also been used (since CutBlur is meant to be applied to real-world SR and may cause undesired effects if applied to bicubic-only). 
Config and training log provided for more details.
 
Showcase: 
[7 Slowpics Examples](https://slow.pics/c/zyilXhKU)

Architecture	ESRGAN
Scale	4x
Color Mode	RGB
License	CC-BY-4.0 Private use Commercial use Distribution Modifications Credit required State Changes No Liability & Warranty Disclaimer
Date	2023-07-06
Dataset	LSDIR
Dataset size	84991
Training iterations	225000
Training batch size	5
Training HR size	192

4xLSDIR

Similar Models