Jump to content
Sign in to follow this  
MrFlibble

Neural upscale of 8-bit graphics: C&C

Recommended Posts

Posted (edited)

I've been increasingly interested in the possibilities of scaling up video game graphics using neural networks recently, and after a relatively brief but overall productive stint with waifu2x I realised its limitations and tried out ESRGAN. While this network, like many others of this kind, is intended to process photographic images and not video gamer graphics, a model was trained by one of the users that is quite suitable for art.

Without going into much detail, I scaled up three official screenshots from Command & ConquerThe Covert Operations and Red Alert, then downsized them to 640x480 and converted to the respective original palettes:

ICjYWtq.png 

YX1YI6c.png 

5HVY26w.png

Note that this was done purely for fun/out of curiosity, but the results are pretty interesting!

Edited by MrFlibble

Share this post


Link to post

Holy smokes, this looks absolutely amazing! :o

Share this post


Link to post

It'd still need human corrections because it looks watercolour-ey, but definitely interesting results.

Share this post


Link to post

Looks alright, needs touch up though. The Nod soldier in my signature is an example of waifu2x.

Share this post


Link to post

Looks pretty interesting. I dig the oil painting effect. I'm not sure whether I preferred this over crisp pixels, but I'd like to see this in action on a large display.

Share this post


Link to post

Actually I just tried a different network from the same developers as ESRGAN, SFTGAN. Unlike ESRGAN, this one does not scale up the image itself, rather, it processes input scaled by other methods to recover texture detail. So I fed it some waifu2x images, and the results seem quite more suited to C&C screenshots:

0Tm7Aab.png

cjC0xAu.png

The images got sharper but also feel more jaggy though.

I think this could produce possibly better results with renders as well.

Share this post


Link to post

The upper image somehow looks worse. The Humvees look borked and apparently the algorithms are confused by the dirt/sand.

The lower one looks good.

Share this post


Link to post

Actually I like how the sand and rock textures came out grainy, they're too smoothed in the original Manga image (which I also had to apply surface blur to to get rid of the faux JPEG compression artifacts it created because the model was trained on JPEG images).

Also if you look closely at the Hummvees, at least you can see their mounted guns in the second image,  which are almost indiscernible blobs in the Manga one.

Share this post


Link to post
Posted (edited)

In the meantime, someone made a whole bunch of new models that are significant improvement upon the original Manga109Attempt. I tested them a bit and that guy's own Manga109 version can produce vastly superior results (haven't bothered converting them to original palettes this time):

VQlN3ul.png

4SN5fBf.png

P4rGp84.png

This is possibly the best results I got so far with C&C images.

I used only a little pre-processing on each image:

  • use Scaler Test to resize the original image to 4x with xBRZ scaler (don't forget to uncheck the display scaler name option)
  • load the scaled image in GIMP, apply Gaussian blur at 1 pixel radius, scale down to original size using Sinc interpolation (or, apply Gaussian at 1.5 pixels and use Bicubic)

For the model, I used an interpolation of mymanga109_250000 from the site I linked to above with RRDB_ESRGAN_x4 from the default ESRGAN package. In net_interp.py, find and change code to:

net_PSNR_path = './models/RRDB_ESRGAN_x4.pth'
net_ESRGAN_path = './models/mymanga109_250000.pth'

 

Edited by MrFlibble

Share this post


Link to post

That last batch looks pretty neat.

Share this post


Link to post
Posted (edited)

Thanks! Here's a few more made with the same method:

weLeKeO.png

auflFGG.png

ersVQHr.png

Edited by MrFlibble
fixed typo

Share this post


Link to post

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

  • Recently Browsing   0 members

    No registered users viewing this page.

×