Hi guys,

could someone point me to a book resource, or similar, regarding the implementation of CNNs for identification of features in images, such as letters, and then more complex features, like animals?

I would like to understand the underlying theory, and then do a CUDA implementation. Yes, I know there are libraries out there that already do this quite efficiently, but I would like to do this from scratch for academic, and then internal company reasons.

Thanks everyone and anyone for any help on this.

http://colah.github.io/

I'm using CNNs for my master thesis on generated game art, so I am far from an expert in deep learning yet, and I would love to hear some of the internal company reasons for doing it from scratch, rather than using an existing library? I'm using Lasagne for my thesis, mainly because I found it most approachable.

There is no specific reasons for doing it from scratch for company reasons, except that if I have the knowledge to do it from scratch, then I might adapt it into lower tier hardware we have lying around. None the less we already got a couple of 1080s on order.

Thank you for your site, do you know of any books that might help here?

Again, thank you for your time.

You can get the 'theory' by simply searching for it online.

Basically it is a way to preprocess certain types of data (like largish pixel images) by repeatedly running 'small' feature filter NN in parallel locally (with overlaps) across the whole image .

Each local area of the regular grid (image) is processed to extract/integrate generic patterns/trends (like line/boundry detection or spotting a solid blob) from the basic data. Further layer processing then (in parallel) integrate that first order detecting larger patterns/trends (like spotting a 'corner'). Later layers then look for the super patterns which classify the picture.

The advantage is the lower 'detail' filter NNs are fairly small (some like 5x5 local groupings) and can be well formed to do their task. They can be run in a massively parallel manner (you apply that layers same filter in an array scanning fashion ) and integrate/collapse each next layers input data til the final classification (several layers itself) which detects combinations of the macro patterns.

A 'divide and conquer' solution eliminating/minimizing ALOT of the NxN input weights (in the lower layers) such large data input arrays would require if done monolithicly.

40+ years ago anatomical research was done that showed that the retina of eyes do operations like this (the low level feature detection).

http://neuralnetworksanddeeplearning.com/

This walk you through all the theory very succinctly so you get up to date.