Recurrent neural network with bias node?
Prime Members - Reputation: 684
Posted 20 April 2012 - 05:05 AM
I'm trying to use a genetic algorithm to train a recurrent neural network. I kind of understand what bias nodes are for in feed forward networks. Do I need one for a recurrent neural network?
Yes, for the same reason as for feedforward networks. (it permits to shift the activation function along x axis) .
For example, if I have a 2-inputs unit with 1 recurrent loop (on its output), the input vector is for time n : [ input1n , input2n , outputn-1 , 1 ]. So 4 weights too.
The output is then computed like in a feedfordward network.
About bias :
I take this activation function
bool f(float v)
and I take a unit with 2 boolean inputs with weights equal to 1.
With a bias of 0, the unit performs a OR (input1+input2>0.0)
With a bias of -1, the unit performs a AND (input1+input2-1>0.0)
My point is that a bias must (should) be added to any unit that performs a linear combination of its inputs.