Sign in to follow this  
intrest86

Neural Net Backpropogation with a Sigmoid Activation Function and Binary Inputs?

Recommended Posts

Hello... the title just about said it all >_> I've programmed a neural net, and I'm feeding it a set of binary inputs. The ANN is a multilevel feedforward NN, and I'm not sure of the correct backprop function to use, and am using something kind of slapped together. My activation function: f(x)=1/(1+e^-x) According to (http://www.ee.umd.edu/medlab/papers/nnsumm/node7.html) the correct error function for the output is: e(x)=(Desired-x)*f'(x) f'(x)=(e^-x)/(1+e^-x)^2 Now, the first problem: Assume that my desired output is not binary {0,1}. Now assume that I DO get one of these outputs. 1 is obviously a problem, because f(1) is undefined. I jumped the gun, e(0) is defined for the final output. But then, when the error is backpropogated, the result is multiplied by the incoming signal. If you look at the error function for the input nodes, nothing can ever happen. Either 1 is input, and the error is undefined, or the input is 0 and the error is always 0! That means that the weights from input to the first hidden layer will always stay at their original randomized values. What am I messing up?

Share this post


Link to post
Share on other sites
Yeah, I was being stupid and tired ^_^. In fact, so tired that in my head e^-x became ln(x)^-1 >_<

So is my formula correct then? And if so, why do I see so many places that say the correct error function for a sigmoid activation function includes x(1-x), which WOULD completely screw up binary inputs... I think...

Share this post


Link to post
Share on other sites
Quote:
Original post by alvaro
Actually, I think the formula is something like f(x)/(1-f(x)), but I am about to go to bed and I don't feel like doing any numbers myself.


You are almost correct. If f(x) is a sigmoid, then f`(x) = f(x) * ( 1 - f(x))

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this