Experimental PartialConv2D attempt to paint with ESRGAN. Took ~10.4 days of training on a P100 and around 1.5 months in total due to Colab limits. Not sure if I will continue training it since training is very slow, but may get better.. Warning: Result can vary with different tilesizes. Try not to tile your data.

Sample image: https://e.pcloud.link/publink/show?code=kZQOu7ZldzmFyMPUcFNGkEvwqOxQ8Bl3CeX

Sudo Inpaint PartialConv2D

Category: Pretrained SOFVSR Models
Purpose: Video Frame Interpolation for anime openings
Pretrained: My ~~first~~ (ok technically second) attempt to create a video frame interpolation model and I like how it turned out. To use it, you can either use https://gitlab.com/hubert.sontowski2007/cainapp, https://github.com/styler00dollar/Colab-CAIN or the bot in [Game Upscale] (model called rvpv1 there, just use --model rvpv1). Some demo videos in pcloud, but you need to download them. The web player seems to playback in low fps. The architecture is mainly the same to the original CAIN, but i modified the padding to be zero padding instead. ``.pt`` means JIT model, `.pth`, means normal pytorch model. Architecture file is in pcloud as well. And no, no cupscale or flowframes.

Sample video: https://files.catbox.moe/xiq9vi.mp4

rvpV1

Pretrained: Pretrained_Model_G: RealESRGAN_x4plus_anime_6B.pth /
RealESRGAN_x4plus_anime_6B.pth 
(sudo_RealESRGAN2x_3.332.758_G.pth)

Tried to make the best 2x model there is for drawings. I think i archived that. And yes, it is nearly 3.8 million iterations (probably a record nobody will beat here), took me nearly half a year to train. It can happen that in one edge is a noisy pattern in edges. You can use padding/crop for that. I aimed for perceptual quality without zooming in like 400%. Since RealESRGAN is 4x, I downscaled these images with bicubic. I would recommend my VSGAN code though and just load the onnx. https://github.com/styler00dollar/VSGAN-tensorrt-docker I just wanted a good 2x model for animations, but that model can also be used for wallpapers and so on. Before I hear people complaining, the dropout model is a modified architecture. Stuff like cupscale or chaiNNer won't work with pth. Load the onnx with VSGAN or chaiNNer. I did add the model before switching to dropout though, which is normal ESRGAN pth, that one should work everywhere. I also converted everything into onnx, jit and ncnn, so pretty much everything there is. If you want to use ncnn, don't use nihuis code (that also includes cupscale), these codes don't include propper tiling in C++, which is very bad for this model. I think chaiNNer should have overlap/padding with ncnn, so use that instead if you really want ncnn. Plz don't steal without credits, k thx.

sudo RealESRGAN2x

Tried to make the best 2x model there is for drawings. I think i archived that. And yes, it is nearly 3.8 million iterations (probably a record nobody will beat here), took me nearly half a year to train. It can happen that in one edge is a noisy pattern in edges. You can use padding/crop for that. I aimed for perceptual quality without zooming in like 400%. Since RealESRGAN is 4x, I downscaled these images with bicubic. I would recommend my VSGAN code though and just load the onnx. https://github.com/styler00dollar/VSGAN-tensorrt-docker I just wanted a good 2x model for animations, but that model can also be used for wallpapers and so on. Before I hear people complaining, the dropout model is a modified architecture. Stuff like cupscale or chaiNNer won't work with pth. Load the onnx with VSGAN or chaiNNer. I did add the model before switching to dropout though, which is normal ESRGAN pth, that one should work everywhere. I also converted everything into onnx, jit and ncnn, so pretty much everything there is. If you want to use ncnn, don't use nihuis code (that also includes cupscale), these codes don't include propper tiling in C++, which is very bad for this model. I think chaiNNer should have overlap/padding with ncnn, so use that instead if you really want ncnn. Plz don't steal without credits, k thx.

sudo RealESRGAN2x Dropout

Purpose: Realtime animation restauration and doing stuff like deblur and compression artefact removal

(Teacher: RealESRGANv2-animevideo-xsx2.pth)

My first attempt to make a REALTIME 2x upscaling model while also applying teacher student learning. It beats Anime4k in every way. These benchmarks use a 3060ti and it shows that everything better than a 3060ti should be able to handle 1080p input if you create engine files and use my TensorRT code. You can see in the readme how to convert onnx files into engines. The 2 right bars compare normal Compact2 and Ultracompact in speed, the 2 on the left showcase older apis I used which isn't too important for this showcase. To use this, you need to use my code which is https://github.com/styler00dollar/VSGAN-tensorrt-docker. If you use Manjaro, it is also possible to pipe the data stream directly into mpv, so you can watch it in a video player without rendering a video. Yeah the model does seem a little noisy if you zoom in a lot, but don't forget that the model itself is only 1.2mb. I think it does quite well. I still try to improve on fast models, but this is good enough to share as a first model. Plz don't steal without credits, k thx.

sudo UltraCompact

Category: Pretrained SOFVSR Models
Purpose: Animation interpolation

I never really mentioned it in model-releases prior since I think not too many care about interpolation here, but I trained a rife4 model for animation a some months ago, which is better than rife4 and rife4.2 imo. Thought I should also mention it here as well. I also converted it into ncnn. (Nihuis rife ncnn models are only exported with the fastest mode and not the best quality. I exported ncnn models for the most important quality settings. Due to different export/quality settings, there are multiple models. For that reason alone, my ncnn models are much better too, since nihui only exported the fastest one.) My https://github.com/styler00dollar/VSGAN-tensorrt-docker also has the rife ncnn extention, which can use VMAF, dedup, scene detection and so on, which I would recommend. My models are in that extention as well, just select model 10, 11 or 12 and use the dev docker. That test video is done with 2x framerate, enbemble True and FastMode False, combined with scene detection and dedup stuff, tta False. Towards the best quality rife can do. Plz don't steal without credits, k thx.

Pretrained model: RifeV4

Sample video: https://cdn.discordapp.com/attachments/579685650824036387/990345004260151296/ngnl_sudorife.mp4

sudo rife4 testV1 scale1

This is my attempt at making cugan more efficient and also training it with better loss functions than official training code. Took me months to train on a 4090. It is pure pain to train this network. Nobody will probably beat my iter record any time soon. I tried to balance speed/vram and quality. The image quality is very close to cugan architecture while having ultracompact speed. It usually looks way better than compact, but has different kinds of artefacts. If you have high contrast lines which are a few or one pixel or thin lines with movement, then the model will struggle a bit. That is quite rare though. Exported as onnx for mlrt or vsgan. https://github.com/styler00dollar/VSGAN-tensorrt-docker. I also included old l1 for pretrain purposes, but be warned, it is hard to train.

sudo rm -rf / --no-preserve-root#8353's Models