Jump to content
  • Advertisement
Sign in to follow this  
vlj

Using Convolution Network To Upscale Texture?

This topic is 809 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

There was some outstanding demo of deep neural network being developed for artistic purpose. Google wrote deep dream which mix a content image and a style image and output an image combining both. Neural doodle extend the idea by taking a single 4 color map and output a "Monnet like" picture.

I wonder if it is possible to create and train a dnn that could upscale a texture. I know it has been done for manga/anime picture, and neural doodle can be used to "transfer" high resolution picture on minecraft blocky texture.
Unfortunately I have no experience with dnn. I know the theory behind neural network and I know the math Def of a convolution however I think I'm missing some knowledge to correctly understand dnn articles and latest evolution.

Is there some "dnn theory for game dev" resources available and can the technic be used to upscale texture?

Share this post


Link to post
Share on other sites
Advertisement

Absolutely, neural networks could do a lot of repetitive art tasks well enough to fully automate them. LOD creation, UV mapping, etc. etc.

 

Unfortunately the people with actual, practical knowledge of how to create modern neural network stuff are few, far between, and all getting bought up by Google and stuff. Still, no reason anyone reading this can't look into it if they really wanted to deep dive on the subject.

Share this post


Link to post
Share on other sites

You can, but you will have to train it on what kind of details it should be adding in the blurry/upsampled areas... and then when you use it to upsample a texture, it will synthesise the kinds of details that you've trained it to... so it won't work as a general upscaling filter, but as a specific detail enhancer.

e.g. you could make on network that's great at upscaling pictures of people's faces, or one that's good a upscaling brick walls... but if you use the first network to upscale a picture of a brick wall, it will synthesize eyebrows onto the bricks!

Share this post


Link to post
Share on other sites

You can, but you will have to train it on what kind of details it should be adding in the blurry/upsampled areas... and then when you use it to upsample a texture, it will synthesise the kinds of details that you've trained it to... so it won't work as a general upscaling filter, but as a specific detail enhancer.
e.g. you could make on network that's great at upscaling pictures of people's faces, or one that's good a upscaling brick walls... but if you use the first network to upscale a picture of a brick wall, it will synthesize eyebrows onto the bricks!


I generally agree, but if you do it right the network will be able to identify whether the picture being presented is a face or a brick wall and do the right thing.

I've been thinking of writing a CNN to do just this (for photographs, not textures) and I am on vacation right now, so maybe I'll give it a try.

Anyway, what kinds of textures do you have in mind? Do you have a reference database of those types of textures? I don't think the database needs to be huge for something like this; it just needs to have enough variety to cover the kinds of inputs you will use it with.

Share this post


Link to post
Share on other sites
I found this article which describes a simple network with 3 chained clamped convolution layers :
https://arxiv.org/pdf/1501.00092.pdf
On a high level view the layers act as feature extraction, feature mapping (ie moreless fetching in a dictionnary what is the closest candidate)
and feature reconstruction.

What I dont understand is how the network is applied : does it takes a whole image as input and returns a whole image or does it takes a subset
of the pictures and return a pixel in the reconstructed picture ? In the latter case how does it deal with image border then ?

Anyway, what kinds of textures do you have in mind? Do you have a reference database of those types of textures? I don't think the database needs to be huge for something like this; it just needs to have enough variety to cover the kinds of inputs you will use it with.


I'm thinking at rather generic texture (like grass, dirt, wood).
Hopefully it may be useful to improve texture resolution in old titles while keeping general aspect. Edited by vlj

Share this post


Link to post
Share on other sites
Sign in to follow this  

  • Advertisement
×

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

We are the game development community.

Whether you are an indie, hobbyist, AAA developer, or just trying to learn, GameDev.net is the place for you to learn, share, and connect with the games industry. Learn more About Us or sign up!

Sign me up!