1xDeJPG OmniSR model

Meant to remove jpg artifacts. First compression round on the training dataset was 40-95, second round was 80-100.

Example: https://slow.pics/c/70nGEvPR

PS these are non-otf models. otf experiments have not quite done it for me (probably would need more adjusting), but they can be found in the experiments folder and their outputs in the outputs folder: https://drive.google.com/drive/folders/1RRXZCRVqlqU_iaeF-RGUM9UBWtI0HseY?usp=drive_link

DeJPG_OmniSR

1xDeJPG SRFormer light model

Meant to remove jpg artifacts. First compression round on the training dataset was 40-95, second round was 80-100.

onnx conversions in this folder which contains all files: https://drive.google.com/drive/folders/1RRXZCRVqlqU_iaeF-RGUM9UBWtI0HseY

Example: https://slow.pics/c/70nGEvPR

PS these are non-otf models. otf experiments have not quite done it for me (probably would need more adjusting), but they can be found in the experiments folder and their outputs in the outputs folder: https://drive.google.com/drive/folders/1RRXZCRVqlqU_iaeF-RGUM9UBWtI0HseY?usp=drive_link

DeJPG_SRFormer_light

Purpose: 1x exposure correction of photos

This model is meant as an experiment to see if compact can be used to train on photos to exposure correct those using the pixel, perceptual, color, color and ldl losses. There is no brightness loss. Still it seems to kinda work.

4 Examples: https://slow.pics/s/nuLodV0z

ExposureCorrection compact

[Github Release Link](https://github.com/Phhofm/models/releases/Ludvae200)

Name: Ludvae200   
License: CC BY 4.0   
Author: Philip Hofmann   
Network: [LUD-VAE](https://github.com/zhengdharia/LUD-VAE)   
Scale: 1   
Release Date: 25.03.2024   
Purpose: 1x realistic noise degradation model    
Iterations: 190'000   
H_size: 64   
n_channels: 3   
dataloader_batch_size: 16    
H_noise_level: 8   
L_noise_level: 3   
Dataset: [RealLR200](https://drive.google.com/drive/folders/1L2VsQYQRKhWJxe6yWZU9FgBWSgBCk6mz)   
Number of train images: 200   
OTF Training: No   
Pretrained_Model_G: None   

Description:
1x realistic noise degradation model, trained on the [RealLR200](https://drive.google.com/drive/folders/1L2VsQYQRKhWJxe6yWZU9FgBWSgBCk6mz) dataset as found released on the [SeeSR](https://github.com/cswry/SeeSR) github repo.
Next to the ludvae200.pth model file, I provide a ludvae200.zip file which not only contains the code but also an inference script to run this model on the dataset of your choice.
Adapt the ludvae200_inference.py script accordingly by adjusting the file paths at the beginning section, to your input folder, output folder, the folder path holding the ludvae200.pth model, and a folder path where you want the text file to be generated. I made the textfile generation the same way as I did in [Kim's Dataset Destroyer](https://github.com/Kim2091/helpful-scripts/tree/main/Dataset%20Destroyer), which means you will have each image file logged with each of the values used to degrade that specific image file in the resulting text file, which will append only and never overwrite.

You can also adjust the strength settings inside the inference script file to fit to your needs. If you in general want less strong noise for example, you should adjust the temperature upper limit from 0.4 to 0.2 or go even lower.
So in line 96 change "temperature_strength = uniform(0.1,0.4)" to "temperature_strength = uniform(0.1,0.2)" just to give an example.

These values are defaulted to my needs of my last dataset degradation workflow I used, but feel free to adjust these values. You can also do the same as I did, temporarily using deterministic values with multiple runs to determine the min and max values of noise generation you deem suitable for your dataset needs.

See the examples of what this looked like for my last dataset workflow I used my model in.

Ludvae200

Purpose: 1x exposure correction of overexposed photos

This model is meant as an experiment to see if compact can be used to train on overexposed images to exposure correct those using the pixel, perceptual, color, color and ldl losses. There is no brightness loss. Still it seems to kinda work.

2 Examples: https://slow.pics/s/KG4ELfGD

OverExposureCorrection compact

Purpose: 1x exposure correction of underexposed photos

This model is meant as an experiment to see if compact can be used to train on underexposed images to exposure correct those using the pixel, perceptual, color, color and ldl losses. There is no brightness loss. Still it seems to kinda work.

3 Examples: https://slow.pics/s/Lnh2zcuK

UnderExposureCorrection compact

Purpose: 2x upscaler for evangelion episodes

For the evangelion upscale project still, this time a dat2 model. A 2x upscaler for evangelion episodes on the evangelion dataset provided by pwnsweet, which called for model trainers to train models on it.

Slowpoke Pics 4 Examples

Evangelion dat2

Purpose: 2x anime upscaler

2x omnisr version of the 2xHFA2kCompact model.

Original Link: https://drive.google.com/drive/folders/1QfiPT1uHi7Yxb4Gjhl25M2YI6hWU5hVt?usp=drive_link

HFA2OmniSR

Purpose: 2x anime upscaler

2x Real-CUGAN version of the 2xHFA2kCompact model.

Original Link: https://drive.google.com/drive/folders/1-ijn4RE5vSWJuIwoBTZdA3cKlraw8PcY

Slow Pics examples:
[Example 1](https://slow.pics/c/auAsNwNZ)
[Example 2](https://slow.pics/c/HX2wuZ3E)
[Ludvae1](https://slow.pics/c/oDQT5d3u)
[Ludvae2](https://slow.pics/c/5tJrT8FF)

HFA2Real CUGAN

Purpose: 2x anime upscaler

2x SwinIR-Small version of the 2xHFA2kCompact model.

Original Link: https://drive.google.com/drive/folders/1yM2_BkktUXS2-i1sTCYZuHRyfx-ak2o8

HFA2SwinIR S

Purpose: A lightweight anime 2x upscaling model with realistic degradations, based on musl's HFA2k_LUDVAE dataset

2x lightweight anime upscaler with realistic degradations (compression, noise, blur) based on musl's HFA2k_LUDVAE dataset.

Original Link: https://drive.google.com/drive/folders/1Fe1Uu3-L5YWaVSwPGBTIPG4xU7kK9rZK?usp=drive_link

HFA2k LUDVAE SPAN

Purpose: A lightweight anime 2x upscaling model with realistic degradations, based on musl's HFA2k_LUDVAE dataset

2x lightweight anime upscaler with realistic degradations (compression, noise, blur) based on musl's HFA2k_LUDVAE dataset.

HFA2k LUDVAE compact

Purpose: 2x anime upscaler which tries to preserve dof

2x anime upscaler, a non-otf version of the HFA2kCompact model which tries to preserve dof by not deblurring.

Original Link: https://drive.google.com/drive/folders/1RZbALUTCf4DYDTVMBR74b8Nt-7m6TEyk?usp=drive_link

HFA2k compact multijpg

A 2x Compact anime upscale model that handles AVC (h264) degradation. Applied h264 crf 20-28 degradation together with bicubic, bilinear, box and lanczos downsampling on musl's HFA2k dataset with Kim's dataset destroyer.

2xHFA2kAVCCompact

A 2x EDSR M anime upscale model that handles AVC (h264) degradation. Applied h264 crf 20-28 degradation and bicubic, bilinear, box and lanczos downsampling on musl's HFA2k dataset with Kim's dataset destroyer.

2xHFA2kAVCEDSR_M

The second released community trained model on the OmniSR network. Trained with multiscale discriminator to fix decreased output brightness occuring with OmniSR. 2x anime upscale that handles AVC (h264) compression since h264 crf 20-28 degradation together with bicubic, bilinear, box and lanczos downsampling has been applied on musl's HFA2k dataset with Kim's dataset destroyer.

2xHFA2kAVCOmniSR

2x SRFormer_light anime upscale model that handles AVC (h264) compression since h264 crf 20-28 degradation together with bicubic, bilinear, box and lanczos downsampling has been applied on musl's HFA2k dataset with Kim's dataset destroyer for training.


If you want to run this model with chaiNNer (or another application) you need to use the onnx files with an onnx upscale node. All onnx conversions can be found in the [onnx folder on my repo](https://github.com/Phhofm/models/tree/main/2xHFA2kAVCSRFormer_light/onnx).

Example 1: https://imgsli.com/MTkxMTQz

Example 2: https://imgsli.com/MTkxMTQ0

2xHFA2kAVCSRFormer_light

A compact anime 2x upscaling model based on musl's HFA2k dataset.

Compact 2x anime upscaler with otf compression and blur. The '2xHFA2kCompact.pth' (4.6 MB) is the original trained model file, the other model files are conversions using chaiNNer. Trained on musl's latest dataset release for Anime SISR, which has been extracted from modern anime films, where the selection criteria was high SNR, no DOF and high frequency information.

Examples: https://imgsli.com/MTcxNjA4
Example input files: https://drive.google.com/drive/folders/1VSPw8m7VbZO6roM9syE7Nf2QYwwn7GUz 
Example output files: https://drive.google.com/drive/folders/1NFfomnv6d5RtWy_GwOsKO3uZ_HNOo-2i

Future work: Started training a 4xRRDBNet version of this, bigger model so maybe better results but slower, for still/ single anime images. Will release in the future or drop, based on achieved results.

HFA2kCompact

Purpose: 2x anime upscaler

2x span version of the hfa2k model.

Original Link: https://drive.google.com/drive/folders/1Fe1Uu3-L5YWaVSwPGBTIPG4xU7kK9rZK?usp=drive_link

HFA2kSPAN

Purpose: 2x anime upscaler

2x shallow esrgan version of the HFA2kCompact model.  
This model should be usable with [FAST_Anime_VSR ](https://github.com/Kiteretsu77/FAST_Anime_VSR) using TensorRT for fast inference, as should my [2xHFA2kReal-CUGAN](https://drive.google.com/file/d/1wqlK-rQjPGKJ5pNoVgnK9gcNF1tA8EjV/view?usp=drive_link) model.  

Original Link: https://drive.google.com/drive/folders/1_gtYqZGQgrpq55gsGf17MG_QUP4XeFJD

Slow Pics examples:  
[Example 1](https://slow.pics/c/RZj6GMwS)  
[Example 2](https://slow.pics/c/Q3DHaU45)  
[Ludvae1](https://slow.pics/c/fJi4IphY)  
[Ludvae2](https://slow.pics/c/iIhgHokD)

HFA2kShallowESRGAN

2x upscaler for the AI generated image output. Trained on 43856 images from lexica.art, so its trained specifically on that model but should work in general on ai generated images.

2xLexicaRRDBNet

Its like 2xLexicaRRDBNet model, but trained for some more with l1_gt_usm and percep_gt_usm set to true, resulting in sharper outputs. I provide both so they can be chosen based on preferrence of the user.

2xLexicaRRDBNet_Sharp

2x upscaler for AI generated images. Trained on 43856 images from lexica.art, so its trained specifically on that model but should work in general on ai generated images.

2xLexicaSwinIR

2x compact fast universal upscaler pair trained with jpg degradation (down to 40) and multiscale (down_up, bicubic, bilinear, box, nearest, lanczos).

2x-NomosUni_compact_multijpg

Purpose: 2x fast universal upscaler with medium degradation handling (jpg compression, noise, blur)

2x compact fast universal upscaler with medium degradation handling using the Real-ESRGAN training pipeline, based off 2xNomosUni_compact_otf_strong. Handles jpg compression, some noise, and some blur (so dejpgs, denoises and deblurs).

Original Link: https://drive.google.com/drive/folders/1YdOPyb0Ht9YjoRZILf87hSocyAx43CLc

Examples:
[RealPhoto](https://imgsli.com/MjMyNTUy)
[Noisy](https://imgsli.com/MjMyNTUz)

NomosUni compact otf medium

Purpose: 2x universal upscaler

2x esrgan universal upscaler, pair trained with jpg degradation (down to 40) and multiscale (down_up, bicubic, bilinear, box, nearest, lanczos).

Original Link: https://drive.google.com/drive/folders/12zKVS74mz0NtBKGlkx_r0ytUVoeDiTX3?usp=drive_link

Try it out in [this huggingface space](https://huggingface.co/spaces/Phips/upscale) I made (since the spaces uses cpu, testing locally on your gpu would be faster)

Slow Pics crops examples:
[ani](https://slow.pics/c/LLP9p89w)
[face](https://slow.pics/c/4IsxVMAB)
[dof_wheat](https://slow.pics/c/zj4q6Mvi)

NomosUni esrgan multijpg

2x span fast universal upscaler pair trained with jpg degradation (down to 40) and multiscale (down_up, bicubic, bilinear, box, nearest, lanczos).

2x-NomosUni_span_multijpg

Purpose: 2x fast universal DoF preserving upscaler

2x span fast universal DoF preserving upscaler, pair trained with jpg degradation (down to 40) and multiscale (down_up, bicubic, bilinear, box, nearest, lanczos).

Original Link: https://drive.google.com/drive/folders/1_P56B3k1XrBsa-f698zPHAuLaqQ8sSWF

Examples:
[dofwheat](https://slow.pics/c/k0U59uJC)
[digitalart](https://slow.pics/c/yyY0xMAn)  
[face](https://slow.pics/c/MdV3Lr7N)

NomosUni span multijpg ldl

A 2x photo upscaling compact model based on microsofts image pairs. This was one of the earliest models I started training and finished it now for release.

Parimg Compact

4x photo upscaler for faces with otf jpg compression, blur and resize, trained on FFHQ dataset. This has been trained on and for faces, but i guess can also be used for other photos, might be able to retain skin detail. This is not face restoration, but simply a 4x upscaler trained on faces, therefore input images need to be of good quality if good output quality is desired.

4xFFHQDAT

Since the above 4xFFHQDAT model is not able to handle the noise present in low quality input images, i made a small variant/finetune of this, the 4xFFHQLDAT model. This model might come in handy if your input image is of bad quality/not suited for the previous model. I basically made this model in a response to an input image posted in upscaling-results channel as a request to this upscale model (since 4xFFHQDAT would not be able to handle noise), see Imgsli1 example below for result.

4xFFHQLDAT

Description: 4x photo upscaler for faces, trained on the FaceUp dataset. These models are an improvement over the previously released 4xFFHQDAT and are its successors. These models are released together with the FaceUp dataset, plus the accompanying [youtube video](https://www.youtube.com/watch?v=TBiVIzQkptI)

This model comes in 4 different versions:  
4xFaceUpDAT (for good quality input)  
4xFaceUpLDAT (for lower quality input, can additionally denoise)  
4xFaceUpSharpDAT (for good quality input, produces sharper output, trained without USM but sharpened input images, good quality input)  
4xFaceUpSharpLDAT (for lower quality input, produces sharper output, trained without USM but sharpened input images, can additionally denoise)  

I recommend trying out 4xFaceUpDAT

4xFaceUpDAT

4xFaceUpLDAT

4xFaceUpSharpDAT

4xFaceUpSharpLDAT

4x anime image upscaler with a bit of otf jpg compression and blur. Trained on musl's hfa2k dataset release for Anime SISR, which has been extracted from modern anime films, where the selection criteria was high SNR, no DOF and high frequency information.

Examples: https://imgsli.com/MTc2NDgx (PS Example input images are small, each around 500x280 px) \
Example input files: https://drive.google.com/drive/folders/1RI6gGqRy-KxDujbaIrpEMdvFkIWHvfPt \
Example output files: https://drive.google.com/drive/folders/1GqHwPlFp6bIQl4R1AxmrmJ6vUUD8FUxH \
All my model files can be found on my github repo https://github.com/Phhofm/models

HFA2k

Purpose: 4x anime upscaler handling video compression artifacts, trained for 200 epochs

4x anime upscaler handling video compression artifacts since trained with otf degradations for "mpeg2video", "libxvid", "libx264", "libx265" with crf 20-32, mpeg bitrate 3800-5800 (together with the standard Real-ESRGAN otf pipeline). A faster arch using this otf degradation pipeline would be great for handling video compression artifacts. Since this one is a GRL model and therefore slow, [as noted by the dev](https://github.com/Kiteretsu77/VCISR-official/issues/3) maybe more for research purposes (or more for single images/screenshots). Trained using [VCISR](https://github.com/Kiteretsu77/VCISR-official) for 200 epochs.  

"This is epoch 200 and the start iteration is 85959 with learning rate 2.5e-05"  

Slow Pics examples:  
[h264_crf28](https://slow.pics/c/fqOnxXep)  
[ludvae1](https://slow.pics/c/OcukRy7d)  
[ludvae2](https://slow.pics/c/FI33mX4F)

HFA2k VCISR GRLGAN ep200

4x anime super-resolution with real degradation.

4xHFA2kLUDVAEGRL_small

4x lightweight anime upscaler with realistic degradations (compression, noise, blur).

4xHFA2kLUDVAESRFormer_light

4xHFA2kLUDVAESwinIR_light

A normal ESRGAN model without degradations and without any pretrain, simply an RRDBNet model trained on paired dataset (4x downsampled) on the full LSDIR dataset (84,991 images / 165 GB)

4xLSDIR

Upscale small good quality photos to 4x their size

My first ever model 😄 Well, it’s not the best, but, it’s something 😉 
I provide some 15 examples from the validation set here for you to visually see the generated output (with chaiNNer), photo dimensions are in the name

LSDIR Compact

Upscale photos to x4 their size.

4xLSDIRCompactv2 supersedes the previously released models, it combines all my progress on my compact model. Both CompactC and CompactR had received around 8 hours more training since release with batch size 10 (CompactR had only been up to 5 on release), and these two were then interpolated together. This allows v2 to handle some degradations, while preserving the details of the CompactC model. Examples: https://imgsli.com/MTY0Njgz/0/2

LSDIR Compact v2

Upscale small photos with compression to 4x their size.

Trying to extend my previous model to be able to handle compression (JPG 100-30) by manually altering the training dataset, since 4xLSDIRCompact can't handle compression. Use this instead of 4xLSDIRCompact if your photo has compression (like an image from the web).

LSDIR Compact C

Upscale compressed photos to x4 their size.

Able to handle JPG compression (30-100).

LSDIR Compact C3

Upscale good quality input photos to x4 their size

The original 4xLSDIRCompact a bit more trained, cannot handle degradation

LSDIRCompactN

Upscale small photos with compression, noise and slight blur to 4x their size.

Extending my last 4xLSDIRCompact model to Real-ESRGAN, meaning trained on synthetic data instead to handle more kinds of degradations, it should be able to handle compression, noise, and slight blur.

Here is a comparison to show that 4xLSDIRCompact cannot handle compression artifacts, and that these two models will produce better output for that specific scenario. These models are not ‘better’ than the previous one, they are just meant to handle a different use case.

LSDIR Compact R

Upscale (degraded) photos to x4 their size. 

Trained on synthetic data, meant to handle more degradations

LSDIR Compact R3

A 4x photo upscale DAT model trained with otf (resize, jpg, small blur) on the LSDIR dataset.

4xLSDIRDAT

Interpolation of 4xLSDIRplusC and 4xLSDIRplusR to handle jpg compression and a little bit of noise/blur

4xLSDIRplus

The RealESRGAN_x4plus finetuned with the big LSDIR dataset (84,991 images / 165 GB), with manually added jpg compression.

4xLSDIRplusC

The RealESRGAN_x4plus finetuned with the big LSDIR dataset (84,991 images / 165 GB), no degradation.

4xLSDIRplusN

The RealESRGAN_x4plus finetuned with the big LSDIR dataset (84,991 images / 165 GB), with jpg compression and noise and blur

4xLSDIRplusR

4x ai generated image upscaler trained with otf

The 4xLexicaDAT2_hb generated some weird lines on some edges. 4xNomosUniDAT is a different checkpoint of 4xNomosUniDAT_otf (145000), I liked the result a bit more in that example.

LexicaDAT2_otf

4x upscaler for AI generated images. Trained on 43856 images from lexica.art, so its trained specifically on that model but should work in general on ai generated images.

4xLexicaHAT

[Link to Github Release](https://github.com/Phhofm/models/releases/4xNomos8k_atd_jpg)

Name: 4xNomos8k_atd_jpg   
License: CC BY 4.0   
Author: Philip Hofmann   
Network: [ATD](https://github.com/LabShuHangGU/Adaptive-Token-Dictionary)   
Scale: 4   
Release Date: 22.03.2024   
Purpose: 4x photo upscaler, handles jpg compression   
Iterations: 240'000   
epoch: 152   
batch_size: 6, 3   
HR_size: 128, 192   
Dataset: nomos8k   
Number of train images: 8492   
OTF Training: Yes   
Pretrained_Model_G: 003_ATD_SRx4_finetune   

Description:
4x photo upscaler which handles jpg compression. This model will preserve noise. Trained on the very recently released (~2 weeks ago) Adaptive-Token-Dictionary network.   

Training details: 
AdamW optimizer with U-Net SN discriminator and BFloat16.
Degraded with otf jpg compression down to 40, re-compression down to 40, together with resizes and the blur kernels.  
Losses: PixelLoss using CHC (Clipped Huber with Cosine Similarity Loss), PerceptualLoss using Huber, GANLoss, [LDL](https://github.com/csjliang/LDL) using Huber, YCbCr Color Loss (bt601) and Luma Loss (CIE XYZ) on [neosr](https://github.com/muslll/neosr).

7 Examples:
[Slowpics](https://slow.pics/s/uwnoI435)

 4xNomos8k_atd_jpg

I release my span otf series:
They come in three variations: weak, middle, and strong.
Mainly meant for photos (can be tried on other things of course).

(Also there is a non-otf span model I had been working on simultaneously that I will release shortly, should give better results on less degraded input in comparison to this span otf series)

Basically I trained the otf_strong for 90k iter and then medium and weak based off that, with some more training to de-learn (tone down) the (too?) strong degradations. Used discrim resets to correct occuring colorloss in all of them. gt size was for the most part 512 with batch 9 (since i hoped it would give better results) with 0.55 it/s training speed (first 40k at the beginning were gt size 256 with batch 20 with 0.58 it/s).

Nomos8k_span_otf_medium

Nomos8k_span_otf_strong

Nomos8k_span_otf_weak

A 4x photo upscaler with otf jpg compression, blur and resize, trained on musl's Nomos8k_sfw dataset for realisic sr, this time based on the [DAT arch](https://github.com/zhengchen1999/DAT), as a finetune on the official 4x DAT model.


The 295 MB file is the pth file which can be run with the [dat reo github code](https://github.com/zhengchen1999/DAT). The 85.8 MB file is an onnx conversion.


All Files can be found in [this google drive folder](https://drive.google.com/drive/folders/1b2vQHxlFQrVW22osIhQbDk98sdXzzFkx). If above onnx file is not working, you can try the other conversions in the onnx subfolder.


Examples:

[Imgsli1](https://imgsli.com/MTk4Mjg1) (generated with onnx file)

[Imgsli2](https://imgsli.com/MTk4Mjg2) (generated with onnx file)

[Imgsli](https://imgsli.com/MTk4Mjk5) (generated with testscript of dat repo on the three test images in dataset/single with pth file)

4xNomos8kDAT

Nomos8kHAT-L_otf

4x photo upscaler with otf jpg compression and blur, trained on musl's Nomos8k_sfw dataset for realisic sr

Nomos8kSC

4x photo upscaler with otf jpg compression and blur, trained on musl's Nomos8k_sfw dataset for realisic sr. 
Provided is a 16fp onnx (154.1MB) download, and a pth (316.2MB) download.
Since this is a big model, upscaling might take a while.

4xNomos8kSCHAT-L

4x photo upscaler with otf jpg compression and blur, trained on musl's Nomos8k_sfw dataset for realisic sr. 
HAT-S version/model. 
Fp16 Onnx (56.5MB) and pth (77.3MB) file download provided.

4xNomos8kSCHAT-S

4x photo upscaler with otf jpg compression and blur, trained on musl's Nomos8k_sfw dataset for realisic sr. SRFormer base model/version. Provided are onnx fp16 (45.8MB) and pth (185.4MB) downloads.

4xNomos8kSCSRFormer

Purpose: 4x universal DoF preserving upscaler

4x universal DoF preserving upscaler, pair trained with jpg degradation (down to 40) and multiscale (down_up, bicubic, bilinear, box, nearest, lanczos) in neosr with adamw, unet and pixel, perceptual, gan and color losses.
Similiar to the last model I released, with same dataset, this is a full RGT model in comparison.

FP32 ONNX conversion is provided in the google drive folder for you to run it.

6 Examples (To check JPG compression handling see Example Nr.4, to check Depth of Field handlin see Example Nr.1 & Nr.6):
[Slowpics](https://slow.pics/s/iMuE3vE5)

NomosUni rgt multijpg

Purpose: 4x universal DoF preserving upscaler

4x universal DoF preserving upscaler, pair trained with jpg degradation (down to 40) and multiscale (down_up, bicubic, bilinear, box, nearest, lanczos) in neosr with adamw, unet and pixel, perceptual, gan, color and ldl losses.

Examples:
[Imgsli](https://imgsli.com/MjQwNDk5)
[Slowpics](https://slow.pics/s/sIRl3w7J)

NomosUni rgt s multijpg

4x fast universal upscaler pair trained with jpg degradation (down to 40) and multiscale (down_up, bicubic, bilinear, box, nearest, lanczos).

4x-NomosUni_span_multijpg

4x Multipurpose DAT upscaler

Trained on DAT with Adan, U-Net SN, huber pixel loss, huber perceptial loss, vanilla gan loss, huber ldl loss and huber focal-frequency loss, on paired nomos_uni (universal dataset containing photographs, anime, text, maps, music sheets, paintings ..) with added jpg compression 40-100 and down_up, bicubic, bilinear, box, nearest and lanczos scales. No blur degradation had been introduced in the training dataset to keep the model from trying to sharpen blurry backgrounds.

The three strengths of this model (design purpose):
1. Multipurpose
2. Handles bokeh effect
3. Handles jpg compression

This model will not:
- Denoise
- Deblur

4xNomosUniDAT_bokeh_jpg

NomosUniDAT_otf

4x general purpose upscaler for non-degraded content.

Has been trained as paired dataset on box downsamples as a DAT2 model, AdamW with pixel, perceptual, gan, color, ldl and ff loss.

4xNomosUniDAT2_box

Purpose: 4x universal DoF preserving upscaler

4x universal DoF preserving upscaler, pair trained with jpg degradation (down to 40) and multiscale (down_up, bicubic, bilinear, box, nearest, lanczos).

NomosUniDAT2 multijpg ldl

NomosUniDAT2 multijpg ldl sharp

4x photo upscaler on the SSDIR_Sharp dataset, trained with otf, using the same settings as Real_HAT_GAN.  
These otf values are very high in my opinion. This was an experiment if I used the same settings as Real_HAT_GAN (therefore the name).   
This model will denoise very strongly and smooth out a lot (so a lot of details will be lost. But maybe this effect might be beneficial to someone)

4xReal_SSDIR_DAT_GAN

[Link to Github Release](https://github.com/Phhofm/models/releases/4xRealWebPhoto_v2_rgt_s)

Name: 4xRealWebPhoto_v2_rgt_s   
License: CC BY 4.0   
Author: Philip Hofmann   
Network: RGT   
Network Option: RGT-S   
Scale: 4   
Release Date: 10.03.2024   
Purpose: 4x real web photo upscaler, meant for upscaling photos downloaded from the web   
Iterations: 220'000   
epoch: 5   
batch_size: 16   
HR_size: 128   
Dataset: 4xRealWebPhoto_v2 (see details in attached pdf file in github release)   
Number of train images: 1'086'976 (or 543'488 pairs)   
OTF Training: No   
Pretrained_Model_G: RGT_S_x4   

Description:
4x real web photo upscaler, meant for upscaling photos downloaded from the web. Trained on my v2 of my 4xRealWebPhoto dataset, it should be able to handle realistic noise, jpg and webp compression and re-compression, scaling and rescaling with multiple downscampling algos, and handle a little bit of lens blur.

Thought featuring degraded images in the examples, this model should also be able to handle good quality input.

Details about the approach/dataset I made to train this model (and therefore also what this model would be capable of handling) is in the attached pdf in the github release.

My previous tries of this dataset, meaning v0 and v1, will get a separate entry, though this version would be recommended over them.

12 Examples on [Slowpics](https://slow.pics/s/HxXtE1eZ)

 4xRealWebPhoto_v2_rgt_s

[Link to Github Release](https://github.com/Phhofm/models/releases/4xRealWebPhoto_v3_atd)

Name: 4xRealWebPhoto_v3_atd   
License: CC BY 4.0   
Author: Philip Hofmann   
Network: [ATD](https://github.com/LabShuHangGU/Adaptive-Token-Dictionary)   
Scale: 4   
Release Date: 22.03.2024   
Purpose: 4x upscaler for photos downloaded from the web   
Iterations: 250'000   
epoch: 10   
batch_size: 6, 3   
HR_size: 128, 192   
Dataset: 4xRealWebPhoto_v3   
Number of train images: 101'904   
OTF Training: No   
Pretrained_Model_G: 003_ATD_SRx4_finetune   

Description:   
4x real web photo upscaler, meant for upscaling photos downloaded from the web. Trained on my v3 of my 4xRealWebPhoto dataset, it should be able to handle noise, jpg and webp (re)compression, (re)scaling, and just a little bit of lens blur, while also be able to handle good quality input. Trained on the very recently released (~2 weeks ago) Adaptive-Token-Dictionary network. 

My 4xRealWebPhoto dataset tried to simulate the use-case of a photo being uploaded to the web and being processed by the service provides (like on a social media platform) so compression/downscaling, then maybe being downloaded and re-uploaded by another used where it, again, were processed by the service provider. I included different variants in the dataset. The pdf with info to the v2 dataset can be found [here](https://github.com/Phhofm/models/releases/download/4xRealWebPhoto_v2_rgt_s/4xRealWebPhoto_v2.pdf), while i simply included whats different in the v3 png:

![4xRealWebPhoto_v3](https://github.com/Phhofm/models/assets/14755670/2ec67e48-bf21-4b57-9f27-69bc49b84315)


Training details: 
AdamW optimizer with U-Net SN discriminator and BFloat16.
Degraded with otf jpg compression down to 40, re-compression down to 40, together with resizes and the blur kernels.  
Losses: PixelLoss using CHC (Clipped Huber with Cosine Similarity Loss), PerceptualLoss using Huber, GANLoss, [LDL](https://github.com/csjliang/LDL) using Huber, [Focal Frequency](https://github.com/EndlessSora/focal-frequency-loss), [Gradient Variance](https://github.com/lusinlu/gradient-variance-loss) with Huber, YCbCr Color Loss (bt601) and Luma Loss (CIE XYZ) on [neosr](https://github.com/muslll/neosr) with norm: true.

11 Examples:
[Slowpics](https://slow.pics/s/plgWVh4j)

 4xRealWebPhoto_v3_atd

[Link to Github Release](https://github.com/Phhofm/models/releases/4xRealWebPhoto_v4_dat2)

## 4xRealWebPhoto_v4_dat2

**Scale:** 4   
**Architecture:** DAT   

**Author:** Philip Hofmann   
**License:** CC-BY-4.0   
**Purpose:** Compression Removal, Deblur, Denoise, JPEG, WEBP, Restoration   
**Subject:** Photography   
**Input Type:** Images   
**Date:** 04.04.2024   

**Architecture Option:** DAT-2   
**I/O Channels:** 3(RGB)->3(RGB)   

**Dataset:** Nomos8k   
**Dataset Size:** 8492   
**OTF (on the fly augmentations):** No   
**Pretrained Model:** DAT_2_x4   
**Iterations:** 243'000   
**Batch Size:** 4-6   
**GT Size:** 128-256   

**Description:** 4x Upscaling Model for Photos from the Web. The dataset consists of only downscaled photos (to handle good quality), downscaled and compressed photos (uploaded to the web and compressed by service provider), and downscale, compressed, rescaled, recompressed photos (downloaded from the web and re-uploaded to the web).

Applied lens blur, realistic noise with my ludvae200 model, JPG and WEBP compression (40-95), and down_up, linear, cubic_mitchell, lanczos, gaussian and box downsampling algorithms. For details on the degradation process, check out the pdf with its explanations and visualizations.

This is basically a dat2 version of my previous 4xRealWebPhoto_v3_atd model, but trained with a bit stronger noise values, and also a single image per variant so drastically reduced training dataset size.
 

**Showcase:** 
[12 Slowpics Examples](https://slow.pics/s/TvJ21pJG)

4xRealWebPhoto_v4_dat2

[Link to Github Release](https://github.com/Phhofm/models/releases/tag/4xRealWebPhoto_v4_drct-l)

## 4xRealWebPhoto_v4_drct-l

**Scale:** 4  
**Architecture:** [DRCT](https://github.com/ming053l/DRCT)  
**Architecture Option:** [DRCT-L](https://github.com/ming053l/DRCT/blob/8c13e63135f494913cd504c073e41ef52250d1d4/options/train/train_DRCT-L_SRx4_finetune_from_ImageNet_pretrain.yml#L56)

**Author:** Philip Hofmann  
**License:** CC-BY-0.4  
**Purpose:** Restoration  
**Subject:** Realistic, Photography  
**Input Type:** Images  
**Release Date:** 02.05.2024  

**Dataset:** 4xRealWebPhoto_v4  
**Dataset Size:** 8492  
**OTF (on the fly augmentations):** No  
**Pretrained Model:** 4xmssim_drct-l_pretrain  
**Iterations:** 260'000  
**Batch Size:** 6,4  
**GT Size:** 128,192  

**Description:** The first real-world drct model, so I am releasing it, or at least my try at it, maybe others will be able to get better results than me, I think I'd recommend my [4xRealWebPhoto_v3_atd](https://github.com/Phhofm/models/releases/tag/4xRealWebPhoto_v3_atd) model over this one if a real-world model for upscaling photos downloaded from the web is desired.   
This model is based on my previously released drct pretrain. Used mixup, cutmix, resizemix augmentations, and mssim, perceptual, gan, dists, ldl, focalfrequency, gradvar, color and luma losses.

**Showcase:**
[Slow.pics](https://slow.pics/s/VOKVChT9)

4xRealWebPhoto_v4_drct-l

Description: 4x photo upscaler on the SSDIR_Sharp dataset, trained with otf for jpg, resize and small blur. This model sharpens.

4xSSDIRDAT

[Link to Github Release](https://github.com/Phhofm/models/releases/4xTextureDAT2_otf)

Name: 4xTextureDAT2_otf   
Author: Philip Hofmann   
Release: 13.12.2023   
License: CC BY 4.0   
Network: DAT   
Arch Option: DAT2   
Scale: 4   
Purpose: 4x texture image upscaler, handling jpg compression, some noise and slight blur   
Iterations: 125000   
batch_size: 6   
HR_size: 128   
Dataset: GTAV_512_Dataset   
Number of train images: 30122   
OTF Training: Yes   
Pretrained_Model_G: DAT_2_x4   

Description: 4x upscale texture images, trained with the Real-ESRGAN otf pipeline so handles jpg compression, some noise and slight blur

4xTextureDAT2_otf

[Github Release Link](https://github.com/Phhofm/models/releases/tag/4xTextures_GTAV_rgt-s)

## 4xTextures_GTAV_rgt-s

**Scale:** 4   
**Architecture:** [RGT](https://github.com/zhengchen1999/RGT)   
**Architecture Option:** RGT-S  

**License:** CC-BY-0.4  
**Purpose:** Restoration   
**Subject:** Game Textures   
**Input Type:** Images   
**Release Date:** 04.05.2024   

**Dataset:** [GTAV_512_Textures](https://discord.com/channels/547949405949657098/905446120333930566/1132367991808466975)   
**Dataset Size:** 8492   
**OTF (on the fly augmentations):** No   
**Pretrained Model:** RGT_S_x4   
**Iterations:** 165'000   
**Batch Size:** 6,4   
**GT Size:** 128,256   

**Description:** A model to upscale game textures, trained on GTAV Textures, handles jpg compression down to 80.

**Showcase:**
[Slow Pics](https://slow.pics/s/GqXdJ6Pd)

4xTextures_GTAV_rgt-s

[Link to Github Release](https://github.com/Phhofm/models/releases/tag/4xDRCT-mssim-pretrains)

Since no [DRCT](https://github.com/ming053l/DRCT/tree/main) model releases exist yet, I am releasing my drct-s, drct and drct-l pretrains. 
These can be used to speed up and stabilize early training stages when training new drct models.
Trained with mssim (and augs, and color and lumaloss) on downscaled [nomosuni](https://github.com/muslll/neosr/tree/0ecbeb98dcefd97801da0ea39ca2cd03a87fce21?tab=readme-ov-file#datasets).
Training files are also provided additionally.

## 4xmssim_drct-l_pretrain

**Scale:** 4  
**Architecture:** [DRCT](https://github.com/ming053l/DRCT)  
**Architecture Option:** [DRCT-L](https://github.com/ming053l/DRCT/blob/8c13e63135f494913cd504c073e41ef52250d1d4/options/train/train_DRCT-L_SRx4_ImageNet_from_scratch.yml#L55)  

**Author:** Philip Hofmann  
**License:** CC-BY-0.4  
**Purpose:** Pretrained  
**Subject:** Anime, Photography  
**Input Type:** Images  
**Release Date:** 28.04.2024  

**Dataset:** [nomos_uni](https://github.com/muslll/neosr?tab=readme-ov-file#datasets)  
**Dataset Size:** 2989  
**OTF (on the fly augmentations):** No  
**Pretrained Model:** None (=From Scratch)  
**Iterations:** 108'000  
**Batch Size:** 2-6  
**GT Size:** 128-320  

**Description:** A pretrain to start drct-l model training, since there are no official released drct pretrains yet I trained these myself and release them here.

4xmssim_drct-l_pretrain

[Link to Github Release](https://github.com/Phhofm/models/releases/tag/4xDRCT-mssim-pretrains)

Since no [DRCT](https://github.com/ming053l/DRCT/tree/main) model releases exist yet, I am releasing my drct-s, drct and drct-l pretrains. 
These can be used to speed up and stabilize early training stages when training new drct models.
Trained with mssim (and augs, and color and lumaloss) on downscaled [nomosuni](https://github.com/muslll/neosr/tree/0ecbeb98dcefd97801da0ea39ca2cd03a87fce21?tab=readme-ov-file#datasets).
Training files are also provided additionally.

## 4xmssim_drct_pretrain

**Scale:** 4  
**Architecture:** [DRCT](https://github.com/ming053l/DRCT)  
**Architecture Option:** [DRCT](https://github.com/ming053l/DRCT/blob/8c13e63135f494913cd504c073e41ef52250d1d4/options/train/train_DRCT_SRx4_ImageNet_from_scratch.yml#L55)  

**Author:** Philip Hofmann  
**License:** CC-BY-0.4  
**Purpose:** Pretrained  
**Subject:** Anime, Photography  
**Input Type:** Images  
**Release Date:** 28.04.2024  

**Dataset:** [nomos_uni](https://github.com/muslll/neosr?tab=readme-ov-file#datasets)  
**Dataset Size:** 2989  
**OTF (on the fly augmentations):** No  
**Pretrained Model:** None (=From Scratch)  
**Iterations:** 95'000  
**Batch Size:** 2-6  
**GT Size:** 128-320   

**Description:** A pretrain to start drct model training, since there are no official released drct pretrains yet I trained these myself and release them here.

4xmssim_drct_pretrain

[Link to Github Release](https://github.com/Phhofm/models/releases/tag/4xDRCT-mssim-pretrains)

Since no [DRCT](https://github.com/ming053l/DRCT/tree/main) model releases exist yet, I am releasing my drct-s, drct and drct-l pretrains. 
These can be used to speed up and stabilize early training stages when training new drct models.
Trained with mssim (and augs, and color and lumaloss) on downscaled [nomosuni](https://github.com/muslll/neosr/tree/0ecbeb98dcefd97801da0ea39ca2cd03a87fce21?tab=readme-ov-file#datasets).
Training files are also provided additionally.


## 4xmssim_drct-s_pretrain.pth  

**Scale:** 4  
**Architecture:** [DRCT](https://github.com/ming053l/DRCT)  
**Architecture Option:** [DRCT-S](https://github.com/muslll/neosr/blob/0ecbeb98dcefd97801da0ea39ca2cd03a87fce21/neosr/archs/drct_arch.py#L923)  

**Author:** Philip Hofmann  
**License:** CC-BY-0.4  
**Purpose:** Pretrained  
**Subject:** Anime, Photography  
**Input Type:** Images  
**Release Date:** 28.04.2024  

**Dataset:** [nomos_uni](https://github.com/muslll/neosr?tab=readme-ov-file#datasets)  
**Dataset Size:** 2989  
**OTF (on the fly augmentations):** No  
**Pretrained Model:** None (=From Scratch)  
**Iterations:** 75'000  
**Batch Size:** 6  
**GT Size:** 128-320  

**Description:** A pretrain to start drct-s model training, since there are no official released drct pretrains yet I trained these myself and release them here.

4xmssim_drct-s_pretrain

[Github Release Link](https://github.com/Phhofm/models/releases/4xSPAN_pretrains)

[Neosr](https://github.com/muslll/neosr)'s latest update from yesterday included a [new adaptation of the multi-scale ssim loss](https://github.com/muslll/neosr/wiki/Losses#mssim_opt).     
This was an experiment to test out the difference between making a SPAN pretrain with pixel loss with L1 criteria (as often used in research) vs mssim loss as its only loss.     
Models are provided so they can be used for tests or also used as a pretrain for another SPAN model.  

---

## 4xmssim_span_pretrain

Scale: 4  
Architecture: SPAN  

Author: Philip Hofmann    
License: CC-BY-4.0  
Purpose: Pretrain  
Subject: Realistic, Anime  
Date: 10.04.2024  

Dataset: [nomos_uni](https://github.com/muslll/neosr)  
Dataset Size: 2989   
OTF (on the fly augmentations): No  
Pretrained Model: None  
Iterations: 80'000  
Batch Size: 12  
GT Size: 128  

Description: 4x SPAN pretrain trained on [neosr](https://github.com/muslll/neosr)'s [new adaptation of the multi-scale ssim loss](https://github.com/muslll/neosr/wiki/Losses#mssim_opt) from yesterdays update on downsampled nomos_uni dataset using kim's [dataset destroyer](https://github.com/Kim2091/helpful-scripts/tree/main/Dataset%20Destroyer) with down_up,linear,cubic_mitchell,lanczos,gauss,box (while down_up used the same and with range = 0.15,1.5). 
The new augmentations except CutBlur have also been used (since CutBlur is meant to be applied to real-world SR and may cause undesired effects if applied to bicubic-only). 
Config and training log provided for more details.
 
Showcase: 
[7 Slowpics Examples](https://slow.pics/c/zyilXhKU)

4xmssim_span_pretrain

[Github Release Link](https://github.com/Phhofm/models/releases/4xSPAN_pretrains)

[Neosr](https://github.com/muslll/neosr)'s latest update from yesterday included a [new adaptation of the multi-scale ssim loss](https://github.com/muslll/neosr/wiki/Losses#mssim_opt). 
This was an experiment to test out the difference between making a SPAN pretrain with pixel loss with L1 criteria (as often used in research) vs mssim loss as its only loss. 
Models are provided so they can be used for tests or also used as a pretrain for another SPAN model.

---

## 4xpix_span_pretrain

Scale: 4  
Architecture: SPAN  

Author: Philip Hofmann    
License: CC-BY-4.0  
Purpose: Pretrain  
Subject: Realistic, Anime  
Date: 10.04.2024  

Dataset: [nomos_uni](https://github.com/muslll/neosr)  
Dataset Size: 2989   
OTF (on the fly augmentations): No  
Pretrained Model: None  
Iterations: 80'000  
Batch Size: 12  
GT Size: 128  

Description: 4x SPAN pretrain trained on pixel loss with L1 criteria (as often used in research) on downsampled nomos_uni dataset using kim's [dataset destroyer](https://github.com/Kim2091/helpful-scripts/tree/main/Dataset%20Destroyer) with down_up,linear,cubic_mitchell,lanczos,gauss,box (while down_up used the same and with range = 0.15,1.5). 
The new augmentations except CutBlur have also been used (since CutBlur is meant to be applied to real-world SR and may cause undesired effects if applied to bicubic-only). 
Config and training log provided for more details.
 
Showcase: 
[7 Slowpics Examples](https://slow.pics/c/zyilXhKU)

Helaman's Models