Artifacts

[Zarxrax]

Hi, I have been trying out PLKSR within neosr training framework.
Sometimes the output will degrade into weird noise on parts of the image.
Is this something that you encountered while working on PLKSR?
I wasn't sure if this was an issue with PLKSR itself, or perhaps a problem with its implementation in neosr. Or do you have any tips to prevent this kind of issue?

[dslisleedh]

I appreciate your interest in PLKSR! I haven't encountered this problem because I only generated the resulting images after completing the training. I also checked the implementation by neosr, and it seems fine. One possible guess is that PLKSR's training can be quite hard because it doesn't use intermediate normalization to reduce latency. I wonder if adding a normalization layer or using a pre-trained model could address this issue.

[umzi2]

using pre-training instantly leads to artifacts, from the way I solved it, it’s a warm-up and a pre-trained discriminator

[umzi2]

just as it seemed to me that 2x models are less stable, and the more losses and the more unstable the discriminator and optimizer are, the more often these artifacts appear. I think plksr is just very sensitive and this is something to take into account when training. I really liked your architecture, but it’s really difficult to train and artifacts can appear in the most unexpected places and usually this happens on the same images

[dslisleedh]

I don't know about Real-image SR as I haven't trained it, but I agree that PLKSR is harder to train compared to other models. For example, compared to other models (e.g. DITN, SRFormer-light), PLKSR had to reduce the learning rate in the earlier iterations to avoid gradient explosion. I think this is probably a big weakness of PLKSR.