Nice to release a model again! This one is intended for realistic imagery, and works especially well on faces, hair, and nature shots. It should only be used on somewhat clear shots, without a lot of grain. I trained this model on SPAN, which as of the time of release, you'll need chaiNNer-nightly for. I aimed for a softer, more natural look for this model with as few artifacts as possible.

In addition to the Normal model, I've included a "soft" model. The Soft model is... softer. Basically it was an earlier version of the model with a more limited dataset. It produces more natural output on games or rendered content, but suffers a bit more with realistic stuff.

Note: In shots with DOF (depth of field) or bokeh, unfortunately there will be artifacts. 

Compatibility: You'll have to use the latest chaiNNer-nightly to use this model

ClearRealityV1

Description: 4x realistic upscaler that may also work for general purpose usage. It was trained with OTF random degradation with a very low to very high range of degradations, including blur, noise, and compression. Trained with the same Frankendata dataset that I used for the pretrain model.

Frankendata_FullDegradation_SRFormer

Upscale photos to x4 their size.

4xLSDIRCompactv2 supersedes the previously released models, it combines all my progress on my compact model. Both CompactC and CompactR had received around 8 hours more training since release with batch size 10 (CompactR had only been up to 5 on release), and these two were then interpolated together. This allows v2 to handle some degradations, while preserving the details of the CompactC model. Examples: https://imgsli.com/MTY0Njgz/0/2

LSDIR Compact v2

Upscale small photos with compression to 4x their size.

Trying to extend my previous model to be able to handle compression (JPG 100-30) by manually altering the training dataset, since 4xLSDIRCompact can't handle compression. Use this instead of 4xLSDIRCompact if your photo has compression (like an image from the web).

LSDIR Compact C

Upscale compressed photos to x4 their size.

Able to handle JPG compression (30-100).

LSDIR Compact C3

Upscale small photos with compression, noise and slight blur to 4x their size.

Extending my last 4xLSDIRCompact model to Real-ESRGAN, meaning trained on synthetic data instead to handle more kinds of degradations, it should be able to handle compression, noise, and slight blur.

Here is a comparison to show that 4xLSDIRCompact cannot handle compression artifacts, and that these two models will produce better output for that specific scenario. These models are not ‘better’ than the previous one, they are just meant to handle a different use case.

LSDIR Compact R

Upscale (degraded) photos to x4 their size. 

Trained on synthetic data, meant to handle more degradations

LSDIR Compact R3

The RealESRGAN_x4plus finetuned with the big LSDIR dataset (84,991 images / 165 GB), with manually added jpg compression.

4xLSDIRplusC

4xNomos2_otf_esrgan

Scale: 4  
Architecture: ESRGAN  
Architecture Option: esrgan  
[Github Release Link](https://github.com/Phhofm/models/releases/tag/4xNomos2_otf_esrgan)  

Author: Philip Hofmann  
License: CC-BY-0.4  
Purpose: Restoration  
Subject: Photography  
Input Type: Images  
Release Date: 22.06.2024  

Dataset: Nomos-v2  
Dataset Size: 6000  
OTF (on the fly augmentations): Yes  
Pretrained Model: RealESRGAN_x4plus  
Iterations: 246'000  
Batch Size: 8  
GT Size: 256  

Description:  

4x ESRGAN model for photography, trained using the Real-ESRGAN otf degradation pipeline.  

Showcase:
[Slow Pics 8 Examples](https://slow.pics/s/Y5wdyTPh?canvas-mode=fit-width&image-fit=contain)

4xNomos2_otf_esrgan

4xNomos2_realplksr_dysample  
Scale: 4  
Architecture: [RealPLKSR with Dysample](https://github.com/muslll/neosr/?tab=readme-ov-file#supported-archs)  
Architecture Option: [realplksr](https://github.com/muslll/neosr/blob/master/neosr/archs/realplksr_arch.py)  
[Github Release](https://github.com/Phhofm/models/releases/tag/4xNomos2_realplksr_dysample)  

Author: Philip Hofmann  
License: CC-BY-0.4  
Purpose: Pretrained  
Subject: Photography  
Input Type: Images  
Release Date: 30.06.2024  

Dataset: [nomosv2](https://github.com/muslll/neosr/?tab=readme-ov-file#-datasets)  
Dataset Size: 6000  
OTF (on the fly augmentations): No  
Pretrained Model: [4xmssim_realplksr_dysample_pretrain](https://github.com/Phhofm/models/releases/tag/4xmssim_realplksr_dysample_pretrain)  
Iterations: 185'000  
Batch Size: 8  
GT Size: 256, 512  

Description: 
A Dysample RealPLKSR 4x upscaling model that was trained with / handles jpg compression down to 70 on the Nomosv2 dataset, preserves DoF.   
Based on the [4xmssim_realplksr_dysample_pretrain](https://github.com/Phhofm/models/releases/tag/4xmssim_realplksr_dysample_pretrain) I released 3 days ago.  
This model affects / saturate colors, which can be counteracted a bit by using wavelet color fix, as used in these examples.  

Showcase:
[Slowpics](https://slow.pics/s/p3sMnd5l)

4xNomos2_realplksr_dysample

[Link to Github Release](https://github.com/Phhofm/models/releases/4xNomos8k_atd_jpg)

Name: 4xNomos8k_atd_jpg   
License: CC BY 4.0   
Author: Philip Hofmann   
Network: [ATD](https://github.com/LabShuHangGU/Adaptive-Token-Dictionary)   
Scale: 4   
Release Date: 22.03.2024   
Purpose: 4x photo upscaler, handles jpg compression   
Iterations: 240'000   
epoch: 152   
batch_size: 6, 3   
HR_size: 128, 192   
Dataset: nomos8k   
Number of train images: 8492   
OTF Training: Yes   
Pretrained_Model_G: 003_ATD_SRx4_finetune   

Description:
4x photo upscaler which handles jpg compression. This model will preserve noise. Trained on the very recently released (~2 weeks ago) Adaptive-Token-Dictionary network.   

Training details: 
AdamW optimizer with U-Net SN discriminator and BFloat16.
Degraded with otf jpg compression down to 40, re-compression down to 40, together with resizes and the blur kernels.  
Losses: PixelLoss using CHC (Clipped Huber with Cosine Similarity Loss), PerceptualLoss using Huber, GANLoss, [LDL](https://github.com/csjliang/LDL) using Huber, YCbCr Color Loss (bt601) and Luma Loss (CIE XYZ) on [neosr](https://github.com/muslll/neosr).

7 Examples:
[Slowpics](https://slow.pics/s/uwnoI435)

 4xNomos8k_atd_jpg

I release my span otf series:
They come in three variations: weak, middle, and strong.
Mainly meant for photos (can be tried on other things of course).

(Also there is a non-otf span model I had been working on simultaneously that I will release shortly, should give better results on less degraded input in comparison to this span otf series)

Basically I trained the otf_strong for 90k iter and then medium and weak based off that, with some more training to de-learn (tone down) the (too?) strong degradations. Used discrim resets to correct occuring colorloss in all of them. gt size was for the most part 512 with batch 9 (since i hoped it would give better results) with 0.55 it/s training speed (first 40k at the beginning were gt size 256 with batch 20 with 0.58 it/s).

Nomos8k_span_otf_medium

Nomos8k_span_otf_strong

Nomos8k_span_otf_weak

Nomos8kHAT-L_otf

4x photo upscaler with otf jpg compression and blur, trained on musl's Nomos8k_sfw dataset for realisic sr

Nomos8kSC

4xNomosWebPhoto_RealPLKSR  

Scale: 4  
Architecture: RealPLKSR  
Architecture Option: realplksr  
[Link to Github Release](https://github.com/Phhofm/models/releases/tag/4xNomosWebPhoto_RealPLKSR)  

Author: Philip Hofmann  
License: CC-BY-0.4  
Purpose: Restoration  
Subject: Photography  
Input Type: Images  
Release Date: 28.05.2024  

Dataset: Nomos-v2  
Dataset Size: 6000  
OTF (on the fly augmentations): No  
Pretrained Model: 4x_realplksr_gan_pretrain  
Iterations: 404'000, 445'000  
Batch Size: 12, 4  
GT Size: 128, 256, 512  

Description:

short: 4x RealPLKSR model for photography, trained with realistic noise, lens blur, jpg and webp re-compression.

full: My newest version of my RealWebPhoto series, this time I used the newly released Nomos-v2 dataset by musl.   
I then made 12 different low resolution degraded folders, using kim's datasetdestroyer for scaling and compression, my ludvae200 model for realistic noise, and umzi's wtp_dataset_destroyer with its floating point lens blur implementation for better control.   
I then mixed them together in a single lr folder and trained for 460'000 iters, checked the results, and upon kims suggestion of using interpolation, I tested and am releasing this interpolation between the checkpoints 404'000 and 445'000.  

This model has been trained on neosr using mixup, cutmix, resizemix, cutblur, nadam, unet, multisteplr, mssim, perceptual, gan, dists, ldl, ff, color and lumaloss, and interpolated using the current chaiNNer nightly version.  

This model took multiple retrainings and reworks of the dataset, until I am now satisfied enough with the quality to release this version.   

For more details on the whole process see the pdf file in the attachement.  

I am also attaching the 404'000, 445'000 and 460'000 checkpoints for completeness.    

PS in general degradation strengths have been reduced/adjusted in comparison to my previous RealWebPhoto models  

Showcase:
[Slow Pics 10 Examples](https://slow.pics/s/euvEv4hL)

4xNomosWebPhoto_RealPLKSR

4xNomosWebPhoto_atd

Scale: 4  
Architecture: [ATD](https://github.com/LabShuHangGU/Adaptive-Token-Dictionary)  
Architecture Option: atd  
[Github Release Link](https://github.com/Phhofm/models/releases/tag/4xNomosWebPhoto_atd)  


Author: Philip Hofmann   
License: CC-BY-0.4  
Purpose: Restoration  
Subject: Photography  
Input Type: Images  
Release Date: 07.06.2024  

Dataset: Nomos-v2  
Dataset Size: 6000  
OTF (on the fly augmentations): No  
Pretrained Model: 003_ATD_SRx4_finetune.pth  
Iterations: 460'000  
Batch Size: 6, 2  
GT Size: 128, 192  

Description:  

4x ATD model for photography, trained with realistic noise, lens blur, jpg and webp re-compression.   
ATD version of [4xNomosWebPhoto_RealPLKSR](https://github.com/Phhofm/models/releases/tag/4xNomosWebPhoto_RealPLKSR), trained on the same dataset and in the same way. For more information look into the [4xNomosWebPhoto_RealPLKSR](https://github.com/Phhofm/models/releases/tag/4xNomosWebPhoto_RealPLKSR) release, and the pdf file in its attachments.

Showcase:
[Slow Pics 18 Examples](https://slow.pics/s/GkS39XYi)

4xNomosWebPhoto_atd

I simply wanted to release an ESRGAN model just because I had not trained one for quite a while and just wanted to revisit this older arch for the current series.

4xNomosWebPhoto_esrgan

Scale: 4  
Architecture: ESRGAN  
Architecture Option: esrgan  
[Github Release Link](https://github.com/Phhofm/models/releases/tag/4xNomosWebPhoto_esrgan)  

Author: Philip Hofmann  
License: CC-BY-0.4  
Purpose: Restoration  
Subject: Photography  
Input Type: Images  
Release Date: 16.06.2024  

Dataset: Nomos-v2  
Dataset Size: 6000  
OTF (on the fly augmentations): No  
Pretrained Model: RealESRGAN_x4plus  
Iterations: 210'000  
Batch Size: 12  
GT Size: 256  

Description:

4x ESRGAN model for photography, trained with realistic noise, lens blur, jpg and webp re-compression.  
ESRGAN version of 4xNomosWebPhoto_RealPLKSR, trained on the same dataset and in a similiar way.  
For more information look into the 4xNomosWebPhoto_RealPLKSR release, and the pdf file in its attachments.

4xNomosWebPhoto_esrgan

[Link to Github Release](https://github.com/Phhofm/models/releases/4xRealWebPhoto_v2_rgt_s)

Name: 4xRealWebPhoto_v2_rgt_s   
License: CC BY 4.0   
Author: Philip Hofmann   
Network: RGT   
Network Option: RGT-S   
Scale: 4   
Release Date: 10.03.2024   
Purpose: 4x real web photo upscaler, meant for upscaling photos downloaded from the web   
Iterations: 220'000   
epoch: 5   
batch_size: 16   
HR_size: 128   
Dataset: 4xRealWebPhoto_v2 (see details in attached pdf file in github release)   
Number of train images: 1'086'976 (or 543'488 pairs)   
OTF Training: No   
Pretrained_Model_G: RGT_S_x4   

Description:
4x real web photo upscaler, meant for upscaling photos downloaded from the web. Trained on my v2 of my 4xRealWebPhoto dataset, it should be able to handle realistic noise, jpg and webp compression and re-compression, scaling and rescaling with multiple downscampling algos, and handle a little bit of lens blur.

Thought featuring degraded images in the examples, this model should also be able to handle good quality input.

Details about the approach/dataset I made to train this model (and therefore also what this model would be capable of handling) is in the attached pdf in the github release.

My previous tries of this dataset, meaning v0 and v1, will get a separate entry, though this version would be recommended over them.

12 Examples on [Slowpics](https://slow.pics/s/HxXtE1eZ)

 4xRealWebPhoto_v2_rgt_s

[Link to Github Release](https://github.com/Phhofm/models/releases/4xRealWebPhoto_v3_atd)

Name: 4xRealWebPhoto_v3_atd   
License: CC BY 4.0   
Author: Philip Hofmann   
Network: [ATD](https://github.com/LabShuHangGU/Adaptive-Token-Dictionary)   
Scale: 4   
Release Date: 22.03.2024   
Purpose: 4x upscaler for photos downloaded from the web   
Iterations: 250'000   
epoch: 10   
batch_size: 6, 3   
HR_size: 128, 192   
Dataset: 4xRealWebPhoto_v3   
Number of train images: 101'904   
OTF Training: No   
Pretrained_Model_G: 003_ATD_SRx4_finetune   

Description:   
4x real web photo upscaler, meant for upscaling photos downloaded from the web. Trained on my v3 of my 4xRealWebPhoto dataset, it should be able to handle noise, jpg and webp (re)compression, (re)scaling, and just a little bit of lens blur, while also be able to handle good quality input. Trained on the very recently released (~2 weeks ago) Adaptive-Token-Dictionary network. 

My 4xRealWebPhoto dataset tried to simulate the use-case of a photo being uploaded to the web and being processed by the service provides (like on a social media platform) so compression/downscaling, then maybe being downloaded and re-uploaded by another used where it, again, were processed by the service provider. I included different variants in the dataset. The pdf with info to the v2 dataset can be found [here](https://github.com/Phhofm/models/releases/download/4xRealWebPhoto_v2_rgt_s/4xRealWebPhoto_v2.pdf), while i simply included whats different in the v3 png:

![4xRealWebPhoto_v3](https://github.com/Phhofm/models/assets/14755670/2ec67e48-bf21-4b57-9f27-69bc49b84315)


Training details: 
AdamW optimizer with U-Net SN discriminator and BFloat16.
Degraded with otf jpg compression down to 40, re-compression down to 40, together with resizes and the blur kernels.  
Losses: PixelLoss using CHC (Clipped Huber with Cosine Similarity Loss), PerceptualLoss using Huber, GANLoss, [LDL](https://github.com/csjliang/LDL) using Huber, [Focal Frequency](https://github.com/EndlessSora/focal-frequency-loss), [Gradient Variance](https://github.com/lusinlu/gradient-variance-loss) with Huber, YCbCr Color Loss (bt601) and Luma Loss (CIE XYZ) on [neosr](https://github.com/muslll/neosr) with norm: true.

11 Examples:
[Slowpics](https://slow.pics/s/plgWVh4j)

 4xRealWebPhoto_v3_atd

[Link to Github Release](https://github.com/Phhofm/models/releases/4xRealWebPhoto_v4_dat2)

## 4xRealWebPhoto_v4_dat2

**Scale:** 4   
**Architecture:** DAT   

**Author:** Philip Hofmann   
**License:** CC-BY-4.0   
**Purpose:** Compression Removal, Deblur, Denoise, JPEG, WEBP, Restoration   
**Subject:** Photography   
**Input Type:** Images   
**Date:** 04.04.2024   

**Architecture Option:** DAT-2   
**I/O Channels:** 3(RGB)->3(RGB)   

**Dataset:** Nomos8k   
**Dataset Size:** 8492   
**OTF (on the fly augmentations):** No   
**Pretrained Model:** DAT_2_x4   
**Iterations:** 243'000   
**Batch Size:** 4-6   
**GT Size:** 128-256   

**Description:** 4x Upscaling Model for Photos from the Web. The dataset consists of only downscaled photos (to handle good quality), downscaled and compressed photos (uploaded to the web and compressed by service provider), and downscale, compressed, rescaled, recompressed photos (downloaded from the web and re-uploaded to the web).

Applied lens blur, realistic noise with my ludvae200 model, JPG and WEBP compression (40-95), and down_up, linear, cubic_mitchell, lanczos, gaussian and box downsampling algorithms. For details on the degradation process, check out the pdf with its explanations and visualizations.

This is basically a dat2 version of my previous 4xRealWebPhoto_v3_atd model, but trained with a bit stronger noise values, and also a single image per variant so drastically reduced training dataset size.
 

**Showcase:** 
[12 Slowpics Examples](https://slow.pics/s/TvJ21pJG)

4xRealWebPhoto_v4_dat2

[Link to Github Release](https://github.com/Phhofm/models/releases/tag/4xRealWebPhoto_v4_drct-l)

## 4xRealWebPhoto_v4_drct-l

**Scale:** 4  
**Architecture:** [DRCT](https://github.com/ming053l/DRCT)  
**Architecture Option:** [DRCT-L](https://github.com/ming053l/DRCT/blob/8c13e63135f494913cd504c073e41ef52250d1d4/options/train/train_DRCT-L_SRx4_finetune_from_ImageNet_pretrain.yml#L56)

**Author:** Philip Hofmann  
**License:** CC-BY-0.4  
**Purpose:** Restoration  
**Subject:** Realistic, Photography  
**Input Type:** Images  
**Release Date:** 02.05.2024  

**Dataset:** 4xRealWebPhoto_v4  
**Dataset Size:** 8492  
**OTF (on the fly augmentations):** No  
**Pretrained Model:** 4xmssim_drct-l_pretrain  
**Iterations:** 260'000  
**Batch Size:** 6,4  
**GT Size:** 128,192  

**Description:** The first real-world drct model, so I am releasing it, or at least my try at it, maybe others will be able to get better results than me, I think I'd recommend my [4xRealWebPhoto_v3_atd](https://github.com/Phhofm/models/releases/tag/4xRealWebPhoto_v3_atd) model over this one if a real-world model for upscaling photos downloaded from the web is desired.   
This model is based on my previously released drct pretrain. Used mixup, cutmix, resizemix augmentations, and mssim, perceptual, gan, dists, ldl, focalfrequency, gradvar, color and luma losses.

**Showcase:**
[Slow.pics](https://slow.pics/s/VOKVChT9)

4xRealWebPhoto_v4_drct-l

Rybu

Description: 4x photo upscaler on the SSDIR_Sharp dataset, trained with otf for jpg, resize and small blur. This model sharpens.

Architecture	ATD
Scale	4x
Size	210dim16w256cat
Color Mode	RGB
License	CC-BY-4.0 Private use Commercial use Distribution Modifications Credit required State Changes No Liability & Warranty Disclaimer
Date	2024-06-07
Dataset	Nomos-v2
Dataset size	6000
Training iterations	460000
Training batch size	2
Training HR size	192
Training OTF	No

4xNomosWebPhoto_atd

Similar Models