KS could be EASILY turned into a 4K remake [CONTAINS SPOILERS!!!]

A forum for general discussion of the game: Open to all punters


User avatar
DerailedOmegaMale
Posts: 69
Joined: Tue Nov 05, 2013 3:09 pm

Re: KS could be EASILY turned into a 4K remake [CONTAINS SPOILERS!!!]

Post by DerailedOmegaMale »

Just to show that "lanczos" works just fine when the picture is already blurry and that no additional processing is required in that case:

ORIGINAL:
rin_wisp_blurred.jpg
rin_wisp_blurred.jpg (130 KiB) Viewed 15334 times
LANCZOS UPSCALED BY 4:
rin_wisp_blurred - 4x lanczos.png
rin_wisp_blurred - 4x lanczos.png (5.79 MiB) Viewed 15334 times
AI (note the eyes that were unintentionally sharpened):
rin_wisp_blurred.png
rin_wisp_blurred.png (9.52 MiB) Viewed 15322 times
Last edited by DerailedOmegaMale on Mon Mar 28, 2022 6:58 pm, edited 2 times in total.
User avatar
DerailedOmegaMale
Posts: 69
Joined: Tue Nov 05, 2013 3:09 pm

Re: KS could be EASILY turned into a 4K remake [CONTAINS SPOILERS!!!]

Post by DerailedOmegaMale »

In pictures with strong horizontal blur the problem with unintentional sharpening was actually not as bad as I was afraid it will be. And the few small problematic areas (like for instance the barely visible eyebrows completely disappearing) can still be carefully masked out and replaced with lanczos up-scaling as I just described.

ORIGINAL:
emitrack_finish.jpg
emitrack_finish.jpg (166.81 KiB) Viewed 15324 times
AI UPSCALED:
emitrack_finish.png
emitrack_finish.png (5.42 MiB) Viewed 15327 times
comparison6.2.png
comparison6.2.png (2.74 MiB) Viewed 15298 times
Last edited by DerailedOmegaMale on Tue Mar 29, 2022 1:20 am, edited 3 times in total.
User avatar
DerailedOmegaMale
Posts: 69
Joined: Tue Nov 05, 2013 3:09 pm

Re: KS could be EASILY turned into a 4K remake [CONTAINS SPOILERS!!!]

Post by DerailedOmegaMale »

Another very good example where lanczos just happens to work better than AI due to STRONG intended blur. (The first one was the blurry picture with Rin.)

ORIGINAL:
busride.jpg
busride.jpg (72.82 KiB) Viewed 15321 times
LANCZOS UPSCALED:
busride - 4x lanczos.png
busride - 4x lanczos.png (2.71 MiB) Viewed 15320 times
AI UPSCALED:
busride.png
busride.png (4.19 MiB) Viewed 15321 times
Last edited by DerailedOmegaMale on Mon Mar 28, 2022 8:12 pm, edited 3 times in total.
User avatar
DerailedOmegaMale
Posts: 69
Joined: Tue Nov 05, 2013 3:09 pm

Re: KS could be EASILY turned into a 4K remake [CONTAINS SPOILERS!!!]

Post by DerailedOmegaMale »

"DCCI" might also be an interesting method for the cases where AI fails, but I'm to lazy to test it out, so here is a picture by someone else.
According to the results so far AI is the best method when sharpness is desired and "DCCI" is the best method when sharpness is undesired. (Just imagine that the "lanczos" based examples above are replaced with the better "DCCI". "DCCI" looks almost the same, but is actually a bit better than "lanczos", since the undesired staircase effect is missing.)
And as I already mentioned you are not forced to choose ONLY ONE method for each picture. You can apply different methods in different places of the same picture by using masks, so the sharp areas can use AI while the few areas where AI is unintentionally messing up important details can be smoothly "overwritten" with DCCI.
Image

Also here is the DCCI version of Emi for comparison with lanczos. The staircase effect is no longer visible.
imgonline-com-ua-BigPic-ay7xfa9GUABI.png
imgonline-com-ua-BigPic-ay7xfa9GUABI.png (470.39 KiB) Viewed 15283 times
User avatar
Oddball
Posts: 3023
Joined: Wed Feb 29, 2012 2:05 pm

Re: KS could be EASILY turned into a 4K remake [CONTAINS SPOILERS!!!]

Post by Oddball »

Greatly appreciate the work you're doing. Especially since my computer doesn't want to run the thing.

With that said, Miki does look a little bit off when upscaled. Emi looks fine, but there's something off with Miki's face and hair.
Not Dead Yet
User avatar
DerailedOmegaMale
Posts: 69
Joined: Tue Nov 05, 2013 3:09 pm

Re: KS could be EASILY turned into a 4K remake [CONTAINS SPOILERS!!!]

Post by DerailedOmegaMale »

Oddball wrote: Tue Mar 29, 2022 10:51 pm my computer doesn't want to run the thing.
I'm not surprised that many people can't use it. The current global chip shortage made the prices of the GPUs skyrocket. If I didn't "swallow a bitter pill" and upgraded mine I also wouldn't be able to use this.
If this tool doesn't work for you than it is most probably due to the lack of RAM usable by GPU. In Windows you can check yours with: Press <ESC+Shift+Control>, select "Performance Tab", select "GPU". If it has 2 GB or less than this tool is most probably not going to work. (And if you have a laptop you might be lucky and be able to assign more of it to the GPU without upgrading any of your hardware.)

P.S.: You HAVE to run the tool from the command line or it will not work.
Oddball wrote: Tue Mar 29, 2022 10:51 pm With that said, Miki does look a little bit off when upscaled. Emi looks fine, but there's something off with Miki's face and hair.
Right (except that Emi's eyebrow is missing). In this case I selected a picture where I EXPECTED the AI to fail on purpose in order to see what happens under unfavorable conditions. The reason why Emi looks good and Miki is distorted is because the author of the picture added a bit of horizontal blur to Miki, but not to Emi. You can see that if you look at the original picture.
As already mentioned in the rare cases where AI fails the according areas of the picture can be replaced with DCCI by means of masking. DCCI is better suited for areas that are intentionally blurry.
During my last few messages blurry pictures were deliberately "cherry picked" to see what happens, but fortunately the OVERWHELMING MAJORITY of the pictures in KS are not blurry and thus do not have this kind of problems.
User avatar
DerailedOmegaMale
Posts: 69
Joined: Tue Nov 05, 2013 3:09 pm

Re: KS could be EASILY turned into a 4K remake [CONTAINS SPOILERS!!!]

Post by DerailedOmegaMale »

In case that there are people with strong enough hardware who are not super experienced with computers and find the usage of the command-line to complicated I can provide a batch-file that helps with that and allows starting the conversion process by means of simple double-clicking.
User avatar
DerailedOmegaMale
Posts: 69
Joined: Tue Nov 05, 2013 3:09 pm

Re: KS could be EASILY turned into a 4K remake [CONTAINS SPOILERS!!!]

Post by DerailedOmegaMale »

Anyway it seems like the devs already started to get angry because I am sharing to many pictures under their copyright here, so I am stopping doing that for now.

My intention was to show what is possible and I think that I have shown enough.

P.S.:
I feel the need to resolve a misunderstanding.
The batch file I was talking about in my previous message is supposed to help using the tool to upscale some pictures that you already have. It will not help with copying the pictures out of the game and it will not help with modifying the game or remaking the game. Thus it is not suitable to ease any kinds of KS related copyright violations.
A remake of KS can not be legally published without the consent of 4LS.
User avatar
DerailedOmegaMale
Posts: 69
Joined: Tue Nov 05, 2013 3:09 pm

Re: KS could be EASILY turned into a 4K remake [CONTAINS SPOILERS!!!]

Post by DerailedOmegaMale »

One more thing I wanted to show off is how the masking I mentioned earlier works and how it helps to restore the few details that are messed up by the AI.

As I mentioned earlier I am not very happy with how the AI removed Emi's eyebrow, which makes her facial expression less dramatic. To be honest I am also not very happy with how her mouth ended up looking. So here is how this can be fixed with masking. First I make two up-scaled versions with AI and DCCI. Then I use them as two layers in some proper image editing software (like Gimp, which is free and GPLed) with DCCI being the top layer. Then I add a mask to the top layer and mark details which I want to restore. The masks determines where the DCCI-layer is allowed to overwrite the AI-layer. It is also possible to make parts of the DCCI-layer semi transparent by using grey colors between 100% black and 100% white inside the mask. Note how I used grey colors by blurring the mask a little bit in order to make the transition more smooth. The result can be seen below. (Let a proper artist do it instead of me and he/she will achieve even better results.)

Of course bigger areas (like for instance blurry clouds or trees) can be fixed the same way as well by drawing bigger masks.

To summarize the general idea: DCCI is used to overwrite the results from the AI as rarely as possible and only as often as necessary.

Note that this picture is a cherry-picked exception. In case of the most KS pictures the AI works pretty well and this kind of problems (like missing eyebrows) do not happen in the first place.

maskingexample.png
maskingexample.png (788.68 KiB) Viewed 15193 times
comparison7.png
comparison7.png (260.36 KiB) Viewed 15183 times
User avatar
Grayest
Posts: 34
Joined: Tue Sep 29, 2020 1:13 pm
Location: USA
Contact:

Re: KS could be EASILY turned into a 4K remake [CONTAINS SPOILERS!!!]

Post by Grayest »

Very nice, thanks for showing this off to us.

Someone actually did an HD patch of KS using waifu2x, maybe you'd be interested in checking it out.
Who knows, if you brought this up to them, they might do an update for the patch using these tools instead of whatever they used. (Given they have a good enough computer, that is.)
What if the real 'Best girl' was the friends we made along the way?
User avatar
DerailedOmegaMale
Posts: 69
Joined: Tue Nov 05, 2013 3:09 pm

Re: KS could be EASILY turned into a 4K remake [CONTAINS SPOILERS!!!]

Post by DerailedOmegaMale »

Grayest wrote: Sat Apr 02, 2022 9:30 am ...
I think we should avoid discussing already existing unofficial modifications here or the whole thread might be deleted by the moderators. I started this thread to show of what's possible and what 4LS could do if only they wanted to, not to violate their copyrights or help anyone violate their copyrights. They are against actual modifications done by someone else so let's not provoke them into deleting the whole thread.

Actually if my effort means anything to you then I'd ask you to be so nice and remove your last message. Thanks in advance. I already received a warning from the mods, so the probability that the whole thread might be deleted is high.


---


Otherwise thanks for mentioning Waifu2x. A 2022 video that gives a side by side comparison of Waifu2x to RealESRGAN and some other stuff is available here:

IMHO RealESRGAN provides better results, but Waifu2x might still be an interesting alternative to DCCI for those cases where RealESRGAN provides incorrect results. In the video a slightly different model of RealESRGAN is used (not the same one I used in my examples, the one I used should be slightly better since unlike the video version it doesn't sacrifice quality in order to work faster).
User avatar
DerailedOmegaMale
Posts: 69
Joined: Tue Nov 05, 2013 3:09 pm

Re: KS could be EASILY turned into a 4K remake [CONTAINS SPOILERS!!!]

Post by DerailedOmegaMale »

Here is the entire image properly up-scaled by combining AI and DCCI (view it at full size!!1) and the mask that was used to do so. Both Gimp and Photoshop are capable of applying blur to the mask automatically. Note how damn easy it was to make such a mask. Except for the eyebrow and the mouth (that I mentioned earlier) I literally just had to draw a simple shape with my mouse.

original:
emitrack_finish.jpg
emitrack_finish.jpg (166.81 KiB) Viewed 15135 times
up-scaled by 4 times:
emitrack_finish-fixed.png
emitrack_finish-fixed.png (4.43 MiB) Viewed 15135 times
mask:
mask.png
mask.png (479.11 KiB) Viewed 15135 times
User avatar
DerailedOmegaMale
Posts: 69
Joined: Tue Nov 05, 2013 3:09 pm

Re: KS could be EASILY turned into a 4K remake [CONTAINS SPOILERS!!!]

Post by DerailedOmegaMale »

A cut out piece for those who do not know how to view the picture in full size.
You might see that the sharpness of a long straight line that goes right through the boundary of the mask is transitioning a bit to quickly, but of course this can be fixed by working on the mask just a little longer. (I'm not going to do that since I only wanted to show a quick and dirty example.)
cropped.png
cropped.png (1 MiB) Viewed 15132 times
User avatar
DerailedOmegaMale
Posts: 69
Joined: Tue Nov 05, 2013 3:09 pm

Re: KS could be EASILY turned into a 4K remake [CONTAINS SPOILERS!!!]

Post by DerailedOmegaMale »

So far the general approach to up-scale a picture would look like this.

The path I marked green is what I expect for the overwhelming majority of the pictures. (I actually tested MANY pictures from KS. I just didn't share all of them here. Actually I was rather surprised how damn well the AI worked in case of KS and other VNs in comparison to some other arbitrary anime material I tested.)
chart.png
chart.png (50.1 KiB) Viewed 15059 times
User avatar
Oddball
Posts: 3023
Joined: Wed Feb 29, 2012 2:05 pm

Re: KS could be EASILY turned into a 4K remake [CONTAINS SPOILERS!!!]

Post by Oddball »

DerailedOmegaMale wrote: Tue Apr 05, 2022 2:36 am So far the general approach to up-scale a picture would look like this.

The path I marked green is what I expect for the overwhelming majority of the pictures. (I actually tested MANY pictures from KS. I just didn't share all of them here. Actually I was rather surprised how damn well the AI worked in case of KS and other VNs in comparison to some other arbitrary anime material I tested.)

chart.png
Ah, come on, man. You can't tell us you did a bunch more pictures and then not share the pictures. That's just a tease.
Not Dead Yet
Post Reply