Jump to content
  • Advertisement
Sign in to follow this  
intrest86

Neural Net Backpropogation with a Sigmoid Activation Function and Binary Inputs?

This topic is 4967 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

Hello... the title just about said it all >_> I've programmed a neural net, and I'm feeding it a set of binary inputs. The ANN is a multilevel feedforward NN, and I'm not sure of the correct backprop function to use, and am using something kind of slapped together. My activation function: f(x)=1/(1+e^-x) According to (http://www.ee.umd.edu/medlab/papers/nnsumm/node7.html) the correct error function for the output is: e(x)=(Desired-x)*f'(x) f'(x)=(e^-x)/(1+e^-x)^2 Now, the first problem: Assume that my desired output is not binary {0,1}. Now assume that I DO get one of these outputs. 1 is obviously a problem, because f(1) is undefined. I jumped the gun, e(0) is defined for the final output. But then, when the error is backpropogated, the result is multiplied by the incoming signal. If you look at the error function for the input nodes, nothing can ever happen. Either 1 is input, and the error is undefined, or the input is 0 and the error is always 0! That means that the weights from input to the first hidden layer will always stay at their original randomized values. What am I messing up?

Share this post


Link to post
Share on other sites
Advertisement
What do you mean f(1) is not defined?

f(1) = 1/(1+exp(-1)) = 1+(1+1/e), which is a perfectly good value.

Share this post


Link to post
Share on other sites
Yeah, I was being stupid and tired ^_^. In fact, so tired that in my head e^-x became ln(x)^-1 >_<

So is my formula correct then? And if so, why do I see so many places that say the correct error function for a sigmoid activation function includes x(1-x), which WOULD completely screw up binary inputs... I think...

Share this post


Link to post
Share on other sites
Actually, I think the formula is something like f(x)/(1-f(x)), but I am about to go to bed and I don't feel like doing any numbers myself.

Share this post


Link to post
Share on other sites
Quote:
Original post by alvaro
Actually, I think the formula is something like f(x)/(1-f(x)), but I am about to go to bed and I don't feel like doing any numbers myself.


You are almost correct. If f(x) is a sigmoid, then f`(x) = f(x) * ( 1 - f(x))

Share this post


Link to post
Share on other sites
Sign in to follow this  

  • Advertisement
×

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

We are the game development community.

Whether you are an indie, hobbyist, AAA developer, or just trying to learn, GameDev.net is the place for you to learn, share, and connect with the games industry. Learn more About Us or sign up!

Sign me up!