Sign in to follow this  
firefly2442

Definition of Hidden Unit

Recommended Posts

Hello,

How would you define a hidden unit and their purpose in an artificial neural network? I know that they're connected and separate from input or output units but I'm not sure I could describe in plain English what they are used for.

Thanks.

Share this post


Link to post
Share on other sites
[size="2"][font="verdana,sans-serif"][font="arial"]Visual definition:[/font][/font][font="arial"][font="verdana,sans-serif"][img]http://upload.wikimedia.org/wikipedia/commons/thumb/e/e4/Artificial_neural_network.svg/560px-Artificial_neural_network.svg.png[/img]
[/font][/font]
[font="verdana, sans-serif"]
[/font][font="arial"][font="verdana,sans-serif"]
[/font][/font][font="arial"][font="verdana,sans-serif"]Purpose: [/font][/font][font="arial"][font="verdana,sans-serif"]An artificial neural network is a computational model. [/font][font="verdana, sans-serif"]Adding hidden units allows you to complexify it so as it can solve more difficult problems. As an analogy, it's like increasing the degree of a polynomial.[/font][/font][/size]

Share this post


Link to post
Share on other sites
Hidden LAYER is the term Ive always heard.

Anyway it allows an additional recombination of input factors for more indirect reasoning patterns).

Some NN have more than one hidden layer though I recall it complcates the learning strategy (back propagation) used and
makes the teaching prpcess more difficult (but yet might be nessessary for the problem space).

Its notr uncommon for a Neural Net to fail its learning process and have to be redone with a different random start patter or to require more
training data to properly learn.

Share this post


Link to post
Share on other sites

Without the hidden unit all you are ever doing is constructing a linear equation. (i1 * w1 + b).
If the problem at hand has a non linear response (like a sine wave, or the exclusive-or function) then you can't accurately model it using a linear equation.
The hidden node allows the networks to produce a non linear response to the inputs.



Share this post


Link to post
Share on other sites
[quote name='wodinoneeye' timestamp='1320475156' post='4880719']
Hidden LAYER is the term Ive always heard.
[/quote]
In many NN topologies hidden [i]units[/i] can be grouped in hidden [i]layers[/i], but it isn't always the case; layers are only a convenient abstraction of the actual computation that takes place in units.
For example, there could be a small number of acyclical, arbitrary connections between a small number of hidden units that are processed one or a few at a time rather than collectively a whole layer at a time; it's the usual style for networks obtained by genetic programming, which grow and mutate by adding or altering individual units and connections.

Share this post


Link to post
Share on other sites
Hello
The aim of the hidden layers is to introduce non-linearity in input combination
Therefore, the activation function of a hidden unit must be non-linear (sigmoid for instance)

Otherwise, (linear activation in hidden layers) the ANN can be replaced with an equivalent single layer ANN
(since a composition of linear combinations can be replaced with an equivalent single linear combination)

btw :

unit = 'artificial neuron'
layer = set of units at the same depth

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this