I'm currently working on a piece of coursework for my neural computing module. It's a simple assignment where I have to code a single-layer perceptron to identify colours.
There are three categories - red, green and amber, and the perceptron has to learn how to identify one of them - and make a binary choice on it. Does the incoming data match the current set or not?
I write up the algorithm to take elements from the training set, and on each iteration it verifies against the testing set - and outputs the number of errors occured. The idea is you'll get a 1/x type graph where error is near 100% before any learning and close to (or at) 0% when it's learnt the correct weights.
Depending on the specified learning rate, it seems to take around 45 iterations before it converges on a solution.
Part of the assignment requires us to modify the various parameters and inputs in order to see their effect. For example, having inputs between [0..1] and [-1..+1]. On this note, I came up with a whole new plan [grin]
Because I know stuff about graphics, I know that you can transform colours into different colour spaces. This can be very useful when it comes to extracting different bits of information (e.g. luminance), so I thought the same would work here.
For those not familiar with colour spaces:
I chose to transform the incoming RGB values to HSV format, initially just because I could - later I realised there was an interesting property of HSV that does wonders for my initial task.
At first i was pretty sure that there was something wrong with my NNet and that I'd broken it - instead of the 45 iterations it was taking 2-3 at most to find a solution. I did the usual debugging to make sure it was working as intended. It was.
A little while later I cracked the reason - HUE (the 'H' in HSV). As far as this NNet was concerned it could now represent the classes of colour with a single field - the contents of the other two inputs was pretty much irrelevant now.
For the RED samples H = ~0o
For the GREEN samples H = ~120o - ~180o
For the AMBER samples H = ~30o
And, as you can see, the different values of H are fairly well seperated, making it quite simple for it to pick a weight that suits the necessary input.
I have to admit I had a nice smug grin on my face when I worked out what was happening [grin]
To quote Muhammad on this:
A classic case of optimizing algorithms rather than optimizing code
Whilst I stumbled across this optimization by random chance rather than an asserted effort to find an optimization, it is still a good example.
So, the moral of the story... think at the algorithmic level from time-to-time. Question what inputs and processes you're using - a simple change might make an amazing difference [smile]