Neural network image upscaling

Discussion forum for Celestia developers; topics may only be started by members of the developers group, but anyone can post replies.
Topic author
john71
Posts: 767
Joined: 10.08.2016
With us: 5 years 3 months

Post #21by john71 » 10.11.2021, 21:38

fyr02 wrote:This result is not much better than the 4K one.
all that is being done is edge sharpening and noise reduction on the clouds, which is both unrealistic and not worth the extra file space that comes with the 16K.

I'm sorry, but it seems you don't understand upscaling. 4x upscaling should produce a much worse quality when zooming in (try it in Gimp). Cupscale produces a better and 4x upscaled image.

I was able to create a good quality 64k texture from a 14k Jupiter texture.

https://celestia.space/forum/download/file.php?id=11595&filename=Jupiter+64k-1.png&mode=view

Added after 26 minutes:

Normal image enhancing is like this:
New vs old-detail-comparison.JPG


But Cupscale's AI recreates the image (zoom in to see the VAST difference):

New vs old-detail-comparison.JPG-4x_BSRGAN.jpg

Avatar
fyr02
Posts: 240
Joined: 19.04.2019
Age: 16
With us: 2 years 7 months
Location: San Diego, California (United States)

Post #22by fyr02 » 10.11.2021, 22:07

No, this is not what I am saying.
I'm saying that in the case of this texture, it is not justifiable to upscale the Neptune texture to 16K.
If you keep continually upscaling it to whatever resolution, eventually the data will be overfit and in the end you are not increasing the quality by much (in fact, it is the opposite, you are slightly decreasing the accuracy) when you upscale past a certain point.
In this case, it's not much worth it to upscale Neptune past 6K. Such miniscule improvements (which Cupscale does such as keeping edges, reducing noise, which is the point of the ai upscale) are not representative of the real texture, and as such upscaling the texture past 4-6K is useless.

In the case of Jupiter, when the original data was 14k, this limit of data improvement lies somewhere between 32K and 64K. Upscaling your textures (such as in this case, where 2K was upscaled to 16K) will not improve the quality past a certain point. all that is being done is increasing the weight of the file.
Image

Topic author
john71
Posts: 767
Joined: 10.08.2016
With us: 5 years 3 months

Post #23by john71 » 10.11.2021, 22:11

fyr02 wrote:I'm saying that in the case of this texture, it is not justifiable to upscale the Neptune texture to 16K.

It is not true either.

When you are getting closer to Neptune in Celestia, the 16k texture gives you a much more realistic feeling. 4k is not enough, it seems blurry.

Added after 4 minutes 4 seconds:
fyr02 wrote:Such miniscule improvements (which Cupscale does such as keeping edges, reducing noise, which is the point of the ai upscale) are not representative of the real texture, and as such upscaling the texture past 4-6K is useless.

Take a look at the upscaled picture above. Cupscale is doing MUCH MORE. It recreates the image at a higher resolution.

Added after 9 hours 24 minutes:
fyr02 wrote:Upscaling your textures (such as in this case, where 2K was upscaled to 16K) will not improve the quality past a certain point. all that is being done is increasing the weight of the file.

I beg to differ.

Cupscale creates artificial mathematical details. Of course they are not "real" details, but they are NEW details. The new texture has a lot more information, so it adds a new aesthetic layer to the planet.

If you don't like this new, artificial aesthetic layer, use a 4k texture.

In my opinion the new artificial details make the planet more realistic, even if these details are just mathematical constructs.

Avatar
fyr02
Posts: 240
Joined: 19.04.2019
Age: 16
With us: 2 years 7 months
Location: San Diego, California (United States)

Post #24by fyr02 » 13.11.2021, 04:15

john71 wrote:Take a look at the upscaled picture above. Cupscale is doing MUCH MORE. It recreates the image at a higher resolution

This is exactly what I am saying. The edge-detection algorithims that help to keep the texture clean at higher resolutions suffers from overfitting at this scale. Nearly all of the clouds in the upscaled version has dark outlines and artifacts around the clouds - this is a result of the edge-detection algorithm, which has overfit the data and included the anti-aliasing in the final result.

This is very clearly not good - take this from someone who has worked with denoising and upscaling algorithms. It is essential to not overfit the data.
You can upscale it to 4K, but no more. otherwise the data will be overfit. Cupscale has its flaws, like any other algorithms, and as such it's important not to overdo something like this.

overfitting.png
Image

Topic author
john71
Posts: 767
Joined: 10.08.2016
With us: 5 years 3 months

Post #25by john71 » 13.11.2021, 07:01

fyr02 wrote:Cupscale has its flaws, like any other algorithms, and as such it's important not to overdo something like this.

You seem to forget the following facts:

1.) 90+% of the upscaled image has no problems at all, like no artifacts

2.) this is the RAW output of Cupscale, the program can be fine tuned

3.) those minimal problems can be fixed by manual editing in Gimp (for example using noise reduction algorithms before the conversion)

4.) the overall aesthetic result is much better than the very blurry 4k base texture.

Avatar
trappistplanets
Posts: 487
Joined: 11.06.2020
With us: 1 year 5 months

Post #26by trappistplanets » 13.11.2021, 12:54

i am sorry, but i haft to agree with fyr

Topic author
john71
Posts: 767
Joined: 10.08.2016
With us: 5 years 3 months

Post #27by john71 » 13.11.2021, 13:05

trappistplanets wrote:i am sorry, but i haft to agree with fyr

No problem, it is a matter of taste anyway.

Although I think the FACT that people are developing AI upscaling is in itself proves it is DIFFERENT than previous efforts and there is a demand for it. So I don't think it is worthless, so what are we talking about anyway?

Topic author
john71
Posts: 767
Joined: 10.08.2016
With us: 5 years 3 months

Post #28by john71 » 25.11.2021, 12:28

"Cupscaled" 16k Saturn texture, from the 4k base image (Runar Thorvaldsen's Motherload file):

th_saturn.png-16x_BSRGAN.jpg


Return to “Ideas & News”

Who is online