Sign in to follow this  
Max_Payne

*Implementing* A Neural Network

Recommended Posts

Max_Payne    757
Well, I understand the theory allright. The implementation seems simple enough as far as the network itself. I can create a class represensing a neuron, with an std::map of pointers to inputs in other layers, where each entry is a pair containing a pointer to the neuron and a float for the weight. In fact, I can probably even get rid of the pointer if I just assume a fixed indexing order (although it may be practical if I delete links with very reduced weight after training). What I find difficult to understand is how to program the algorithm to train the network. I found no tutorial that explained this clearly enough. I'm not really looking for source code either, but for a *clear* description of how the algorithm works. Clear enough to be able to implement it.

Share this post


Link to post
Share on other sites
Max_Payne    757
Quote:
Original post by NicoDeLuciferi
Which algorithm do you want to use, backpropagation?


Yes Indeed. But I could not find a clear explanation of how the algorithm works. The tutorials I found basically just said:

"Well, it's the process you use to train your network, I hope you can learn it from somewhere else folks, since I've never implemented it and I just wanted to write an incomplete tutorial about neural networks because I think they're cool".

Share this post


Link to post
Share on other sites
chadjohnson    116
There's an excellent book called, "Neural Networks In Computer Intelligence," which explains it very clearly. One additional thing I found that helps though is to have a bias value for each and every neuron, not just for the network as a whole.

I'll send a scan from the book to the e-mail address on your profile.

Share this post


Link to post
Share on other sites
Max_Payne    757
Quote:
Original post by chadjohnson
There's an excellent book called, "Neural Networks In Computer Intelligence," which explains it very clearly. One additional thing I found that helps though is to have a bias value for each and every neuron, not just for the network as a whole.

I'll send a scan from the book to the e-mail address on your profile.


Thank you :)

Share this post


Link to post
Share on other sites
FlowingOoze    236
Here's a link http://www.willamette.edu/~gorr/classes/cs449/backprop.html.
Basically the backpropagation algorithm is just a form of the gradient descent method. Here's MathWorld on gradient descent: http://mathworld.wolfram.com/MethodofSteepestDescent.html

Share this post


Link to post
Share on other sites
Horizon    145
Backpropagation works as follows:
- Determine the output of the network for a certain input. For training purposes you need to also have the correct output for this sample;
- Compare the output to the known correct output and determine the error from this. The error is usually simply a number between 0 and 1 for every output neuron;
- 'Backpropagate' this error to every neuron as if the direction of the network was reversed, all connections reversed and treat this error as input.
- This will give you an error for every neuron in your network, this is a measure of the neuron's part in the overall error. Now adjust all connections proportionally to the height of their contribution to the error.

I hope this is clear enough, otherwise I think you should look up some of those links above, for the exact formulas etc.

Quote:
Original post by chadjohnson
One additional thing I found that helps though is to have a bias value for each and every neuron, not just for the network as a whole.


What we did when implementing a neural network was provide every neuron with a connection to the bias. This way the bias can be constant over the entire network functioning as a kind of neuron with constant output, but every neuron's connection to the bias can change.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this