Jump to content
  • Advertisement
Sign in to follow this  
Prozak

Low level Resources regarding Convolutional Neural Networks

This topic is 706 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

Hi guys,

 

could someone point me to a book resource, or similar, regarding the implementation of CNNs for identification of features in images, such as letters, and then more complex features, like animals?

 

I would like to understand the underlying theory, and then do a CUDA implementation. Yes, I know there are libraries out there that already do this quite efficiently, but I would like to do this from scratch for academic, and then internal company reasons.

 

Thanks everyone and anyone for any help on this.

Share this post


Link to post
Share on other sites
Advertisement

http://colah.github.io/

 

I'm using CNNs for my master thesis on generated game art, so I am far from an expert in deep learning yet, and I would love to hear some of the internal company reasons for doing it from scratch, rather than using an existing library? I'm using Lasagne for my thesis, mainly because I found it most approachable.

Share this post


Link to post
Share on other sites

There is no specific reasons for doing it from scratch for company reasons, except that if I have the knowledge to do it from scratch, then I might adapt it into lower tier hardware we have lying around. None the less we already got a couple of 1080s on order.

 

Thank you for your site, do you know of any books that might help here?

 

Again, thank you for your time.

Share this post


Link to post
Share on other sites
There is a general book on deep learning by MIT that looks interesting. It hasn't been published yet, but you can read it online: http://www.deeplearningbook.org/ . Disclaimer: I haven't read the whole thing.

You can find an example of training a CNN for digit recognition in the TensorFlow tutorials. Implementing that and playing with it should be quite illuminating.

Share this post


Link to post
Share on other sites

You can get the 'theory'  by simply searching for it online.

 

Basically it is a way to preprocess certain types of data (like largish pixel images) by repeatedly running 'small'  feature filter NN in parallel locally (with overlaps) across the whole image .  

 

Each local area of the regular grid (image) is processed to extract/integrate generic patterns/trends (like line/boundry detection or spotting a solid blob) from the basic data.   Further layer processing  then (in parallel) integrate that first order detecting larger patterns/trends (like spotting a 'corner').   Later layers then look for the super patterns which classify the picture.

 

The advantage is the lower 'detail' filter NNs are fairly small (some like 5x5 local groupings) and can be well formed to do their task.  They can be run in a massively parallel manner (you apply that layers same filter in an array scanning fashion ) and integrate/collapse each next layers input data til the final classification  (several layers itself) which detects combinations of the macro patterns.  

 

A 'divide and conquer' solution eliminating/minimizing ALOT of the  NxN  input weights (in the lower layers) such large data input arrays would require if done monolithicly.  

 

 

40+ years ago anatomical research was done that showed that the retina of eyes do operations like this (the low level feature detection). 

Edited by wodinoneeye

Share this post


Link to post
Share on other sites

What exactly are you looking for when you say "theory"?

 

Compared to much of the rest of AI, the theory of deep learning is fairly nonexistent.

 

If you just want to know some of the math that it uses (information theory, multivariate calculus, etc), there is no shortcut to actually studying the field. But, if you just want to get learn the bare minimum to do neural nets, Yoshua Bengio's book is good, and, if that's too slow, Michael Nielsen's book is for you.

Share this post


Link to post
Share on other sites
Sign in to follow this  

  • Advertisement
×

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

We are the game development community.

Whether you are an indie, hobbyist, AAA developer, or just trying to learn, GameDev.net is the place for you to learn, share, and connect with the games industry. Learn more About Us or sign up!

Sign me up!