Jump to content

  • Log In with Google      Sign In   
  • Create Account

Recurrent neural network with bias node?


Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.

  • You cannot reply to this topic
3 replies to this topic

#1 domokato   Members   -  Reputation: 104

Like
1Likes
Like

Posted 20 April 2012 - 01:17 AM

Hi guys,

I'm trying to use a genetic algorithm to train a recurrent neural network. I kind of understand what bias nodes are for in feed forward networks. Do I need one for a recurrent neural network?

Thanks

Sponsor:

#2 Tournicoti   Prime Members   -  Reputation: 684

Like
1Likes
Like

Posted 20 April 2012 - 05:05 AM

Hello domokato Posted Image

Hi guys,

I'm trying to use a genetic algorithm to train a recurrent neural network. I kind of understand what bias nodes are for in feed forward networks. Do I need one for a recurrent neural network?

Thanks


Yes, for the same reason as for feedforward networks. (it permits to shift the activation function along x axis) .

For example, if I have a 2-inputs unit with 1 recurrent loop (on its output), the input vector is for time n : [ input1n , input2n , outputn-1 , 1 ]. So 4 weights too.
The output is then computed like in a feedfordward network.

Nico

EDIT :

About bias :

I take this activation function
bool f(float v)
{
return v>0.0f;
}
and I take a unit with 2 boolean inputs with weights equal to 1.

With a bias of 0, the unit performs a OR (input1+input2>0.0)
With a bias of -1, the unit performs a AND (input1+input2-1>0.0)

My point is that a bias must (should) be added to any unit that performs a linear combination of its inputs.

#3 domokato   Members   -  Reputation: 104

Like
0Likes
Like

Posted 22 April 2012 - 02:40 AM

Oh, I get it. Thanks for the explanation

#4 jefferytitan   Crossbones+   -  Reputation: 2242

Like
0Likes
Like

Posted 23 April 2012 - 12:58 AM

I'm curious, I haven't encountered using a GA to train a neural net before. I guess it's one way to get a large amount of classified training data. How's it working out for you?




Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.



PARTNERS