Oddball wrote: Wed Mar 23, 2022 9:31 am
What exactly does this enhancement due to the regular backgrounds which were purposely already blurry and distorted?
The easiest way to find out is to just take such a background and upscale it. If you can't do it yourself due to some technical reasons put the picture in here and I'll upscale it for you.
EDIT:
I might have misunderstood your question and maybe you want to hear an accurate technical explanation of HOW the tool operates instead of seeing an example of what it does to a specific picture. If this is the case then I am afraid that an answer to your question might not even exist.
While it is possible to look at the individual neurons of a neural network from the perspective of a computer scientist to understand how they operate on fundamental level and how the programmer fit a neural network into his software and which ideas he used in order to train it, FULLY understanding HOW the finished solution turns a small picture into a big one might be impossible.
A trained neural network doesn't look like something that is easy to understand, it looks like an unfathomable mixed serving of spaghetti. (Of course this analogy is simplified and the actual paths in neural networks look more like trees with A LOT of individually weighted branches and not straight ropes with just two ends on each side. This makes a neural network even more complicated than spaghetti.) The computer that processes such a network doesn't NEED to understand it. It just dumbly runs through all the "innumerable" warped paths noodle by noodle and when it's done it gives a result.
The thing about trained neural networks is that the user usually doesn't even try to understand WHAT EXACTLY it does to the data in order to process it. The usual approach is that you first train a neural network with a data set as large as possible (In this case the data set was made of many anime pictures. It is this automated training process that actually adjusts the weights of the individual branches and thus fills the network with all its overwhelming complexity.) and after that you JUST USE the trained network to process some new arbitrary data in hope that you will get a result that is good enough for you. That's it. There just IS no step that you could call "the user understands the network he created" involved in this approach.
Asking HOW EXACTLY a trained neural network turns some new arbitrary data into a result is like asking the question "How does a human brain operate?". Some parts of your brain were created by evolution and other parts of your brain are trained during your lifetime. A biologist might explain to you how individual nerve cells operate or even give you a very simplified overview of the individual areas of the brain, but the EXACT FULL explanation of a full operating brain is a mystery and might remain a mystery forever.
So we can experiment with individual pictures if you want and hope that the results will answer your questions or otherwise I can not help you. Even the dev of this tool who knows exactly what he has programmed would probably give a similar answer. He would be able to explain how he put the individual pieces of the software together in order to create a working solution of course, but he would never be able to FULLY explain the internal machinery of his neural network AFTER he trained it.
