KS could be EASILY turned into a 4K remake [CONTAINS SPOILERS!!!]

[DerailedOmegaMale]

Just to show that "lanczos" works just fine when the picture is already blurry and that no additional processing is required in that case:

ORIGINAL:

rin_wisp_blurred.jpg (130 KiB) Viewed 15685 times

LANCZOS UPSCALED BY 4:

rin_wisp_blurred - 4x lanczos.png (5.79 MiB) Viewed 15685 times

AI (note the eyes that were unintentionally sharpened):

rin_wisp_blurred.png (9.52 MiB) Viewed 15673 times

[DerailedOmegaMale]

In pictures with strong horizontal blur the problem with unintentional sharpening was actually not as bad as I was afraid it will be. And the few small problematic areas (like for instance the barely visible eyebrows completely disappearing) can still be carefully masked out and replaced with lanczos up-scaling as I just described.

ORIGINAL:

emitrack_finish.jpg (166.81 KiB) Viewed 15675 times

AI UPSCALED:

emitrack_finish.png (5.42 MiB) Viewed 15678 times

comparison6.2.png (2.74 MiB) Viewed 15649 times

[DerailedOmegaMale]

Another very good example where lanczos just happens to work better than AI due to STRONG intended blur. (The first one was the blurry picture with Rin.)

ORIGINAL:

busride.jpg (72.82 KiB) Viewed 15672 times

LANCZOS UPSCALED:

busride - 4x lanczos.png (2.71 MiB) Viewed 15671 times

AI UPSCALED:

busride.png (4.19 MiB) Viewed 15672 times

[DerailedOmegaMale]

"DCCI" might also be an interesting method for the cases where AI fails, but I'm to lazy to test it out, so here is a picture by someone else.
According to the results so far AI is the best method when sharpness is desired and "DCCI" is the best method when sharpness is undesired. (Just imagine that the "lanczos" based examples above are replaced with the better "DCCI". "DCCI" looks almost the same, but is actually a bit better than "lanczos", since the undesired staircase effect is missing.)
And as I already mentioned you are not forced to choose ONLY ONE method for each picture. You can apply different methods in different places of the same picture by using masks, so the sharp areas can use AI while the few areas where AI is unintentionally messing up important details can be smoothly "overwritten" with DCCI.


Also here is the DCCI version of Emi for comparison with lanczos. The staircase effect is no longer visible.

imgonline-com-ua-BigPic-ay7xfa9GUABI.png (470.39 KiB) Viewed 15634 times

[Oddball]

Greatly appreciate the work you're doing. Especially since my computer doesn't want to run the thing.

With that said, Miki does look a little bit off when upscaled. Emi looks fine, but there's something off with Miki's face and hair.

[DerailedOmegaMale]

my computer doesn't want to run the thing.

I'm not surprised that many people can't use it. The current global chip shortage made the prices of the GPUs skyrocket. If I didn't "swallow a bitter pill" and upgraded mine I also wouldn't be able to use this.
If this tool doesn't work for you than it is most probably due to the lack of RAM usable by GPU. In Windows you can check yours with: Press <ESC+Shift+Control>, select "Performance Tab", select "GPU". If it has 2 GB or less than this tool is most probably not going to work. (And if you have a laptop you might be lucky and be able to assign more of it to the GPU without upgrading any of your hardware.)

P.S.: You HAVE to run the tool from the command line or it will not work.

With that said, Miki does look a little bit off when upscaled. Emi looks fine, but there's something off with Miki's face and hair.

Right (except that Emi's eyebrow is missing). In this case I selected a picture where I EXPECTED the AI to fail on purpose in order to see what happens under unfavorable conditions. The reason why Emi looks good and Miki is distorted is because the author of the picture added a bit of horizontal blur to Miki, but not to Emi. You can see that if you look at the original picture.
As already mentioned in the rare cases where AI fails the according areas of the picture can be replaced with DCCI by means of masking. DCCI is better suited for areas that are intentionally blurry.
During my last few messages blurry pictures were deliberately "cherry picked" to see what happens, but fortunately the OVERWHELMING MAJORITY of the pictures in KS are not blurry and thus do not have this kind of problems.

[DerailedOmegaMale]

In case that there are people with strong enough hardware who are not super experienced with computers and find the usage of the command-line to complicated I can provide a batch-file that helps with that and allows starting the conversion process by means of simple double-clicking.

[DerailedOmegaMale]

Anyway it seems like the devs already started to get angry because I am sharing to many pictures under their copyright here, so I am stopping doing that for now.

My intention was to show what is possible and I think that I have shown enough.

P.S.:
I feel the need to resolve a misunderstanding.
The batch file I was talking about in my previous message is supposed to help using the tool to upscale some pictures that you already have. It will not help with copying the pictures out of the game and it will not help with modifying the game or remaking the game. Thus it is not suitable to ease any kinds of KS related copyright violations.
A remake of KS can not be legally published without the consent of 4LS.

[DerailedOmegaMale]

One more thing I wanted to show off is how the masking I mentioned earlier works and how it helps to restore the few details that are messed up by the AI.

As I mentioned earlier I am not very happy with how the AI removed Emi's eyebrow, which makes her facial expression less dramatic. To be honest I am also not very happy with how her mouth ended up looking. So here is how this can be fixed with masking. First I make two up-scaled versions with AI and DCCI. Then I use them as two layers in some proper image editing software (like Gimp, which is free and GPLed) with DCCI being the top layer. Then I add a mask to the top layer and mark details which I want to restore. The masks determines where the DCCI-layer is allowed to overwrite the AI-layer. It is also possible to make parts of the DCCI-layer semi transparent by using grey colors between 100% black and 100% white inside the mask. Note how I used grey colors by blurring the mask a little bit in order to make the transition more smooth. The result can be seen below. (Let a proper artist do it instead of me and he/she will achieve even better results.)

Of course bigger areas (like for instance blurry clouds or trees) can be fixed the same way as well by drawing bigger masks.

To summarize the general idea: DCCI is used to overwrite the results from the AI as rarely as possible and only as often as necessary.

Note that this picture is a cherry-picked exception. In case of the most KS pictures the AI works pretty well and this kind of problems (like missing eyebrows) do not happen in the first place.

maskingexample.png (788.68 KiB) Viewed 15544 times

comparison7.png (260.36 KiB) Viewed 15534 times

[Grayest]

Very nice, thanks for showing this off to us.

Someone actually did an HD patch of KS using waifu2x, maybe you'd be interested in checking it out.
Who knows, if you brought this up to them, they might do an update for the patch using these tools instead of whatever they used. (Given they have a good enough computer, that is.)

[DerailedOmegaMale]

...

I think we should avoid discussing already existing unofficial modifications here or the whole thread might be deleted by the moderators. I started this thread to show of what's possible and what 4LS could do if only they wanted to, not to violate their copyrights or help anyone violate their copyrights. They are against actual modifications done by someone else so let's not provoke them into deleting the whole thread.

Actually if my effort means anything to you then I'd ask you to be so nice and remove your last message. Thanks in advance. I already received a warning from the mods, so the probability that the whole thread might be deleted is high.


---


Otherwise thanks for mentioning Waifu2x. A 2022 video that gives a side by side comparison of Waifu2x to RealESRGAN and some other stuff is available here:

IMHO RealESRGAN provides better results, but Waifu2x might still be an interesting alternative to DCCI for those cases where RealESRGAN provides incorrect results. In the video a slightly different model of RealESRGAN is used (not the same one I used in my examples, the one I used should be slightly better since unlike the video version it doesn't sacrifice quality in order to work faster).

[DerailedOmegaMale]

Here is the entire image properly up-scaled by combining AI and DCCI (view it at full size!!1) and the mask that was used to do so. Both Gimp and Photoshop are capable of applying blur to the mask automatically. Note how damn easy it was to make such a mask. Except for the eyebrow and the mouth (that I mentioned earlier) I literally just had to draw a simple shape with my mouse.

original:

emitrack_finish.jpg (166.81 KiB) Viewed 15486 times

up-scaled by 4 times:

emitrack_finish-fixed.png (4.43 MiB) Viewed 15486 times

mask:

mask.png (479.11 KiB) Viewed 15486 times

[DerailedOmegaMale]

A cut out piece for those who do not know how to view the picture in full size.
You might see that the sharpness of a long straight line that goes right through the boundary of the mask is transitioning a bit to quickly, but of course this can be fixed by working on the mask just a little longer. (I'm not going to do that since I only wanted to show a quick and dirty example.)

cropped.png (1 MiB) Viewed 15483 times

[DerailedOmegaMale]

So far the general approach to up-scale a picture would look like this.

The path I marked green is what I expect for the overwhelming majority of the pictures. (I actually tested MANY pictures from KS. I just didn't share all of them here. Actually I was rather surprised how damn well the AI worked in case of KS and other VNs in comparison to some other arbitrary anime material I tested.)

chart.png (50.1 KiB) Viewed 15410 times

[Oddball]

So far the general approach to up-scale a picture would look like this.

The path I marked green is what I expect for the overwhelming majority of the pictures. (I actually tested MANY pictures from KS. I just didn't share all of them here. Actually I was rather surprised how damn well the AI worked in case of KS and other VNs in comparison to some other arbitrary anime material I tested.)

chart.png

Ah, come on, man. You can't tell us you did a bunch more pictures and then not share the pictures. That's just a tease.