Can the threshold function of an artifical neuron be something like the following? f(t)= { if t=(some number) then f(t)=1 else f(t)=0 } All the threshold functions i''ve seen seem to follow a greater than or less than pattern. I can''t say I''ve read a great deal on NN''s so i''m curious if its just my limited exposure or if there is any convention or is the threshold function limited to <,> for a particular reason? Also if there is no particular reason why you can''t use other threshold funtions is there any known consequences with mixing ''neurons'' with different threshold funtions in the same NN? Thanks for any help or comments in advance.

You can use whatever threshold function you like although your network would have a lot of difficulty learning anything using the one you describe! Check out my neural net tutorial. It will help your understanding.

Additionally, in answer to your last question. It is common practice in many networks to have tanh sigmoid activations in the hidden layer -1 < a < 1 and the logistic sigmoid in the output layer 0 < a < 1.



ai-junkie.com

[edited by - fup on September 30, 2002 7:20:01 PM]

Thanks for the reply Fup, I had just read your tutorial before I posted . Just wanted to make sure of it, I had already read about sigmoid and linear threshold functions but I never saw one with the function I described.

If you are interested in why I''m asking about the threshold function in question is that the book I have introduces linear separability by talking about how Minsky and Papert proved an XOR function could not be done in a simple single layer perceptron network. But when I thought about it wouldn''t the following work:

W1 = -1
W2 = -1

Threshold funtion

f(t) = {if t = -1 then f(t) = 1 else f(t) = 0}

That would give you the truth table for an XOR using a single layer perceptron.

I haven''t read any original material on the subject just the summary in the book, so again am I missing something?

Thanks in advance for any help.

Yah, you are missing something : )

Your threashold function makes the net non-linear

This isn''t actually a good thing. Try to find weights for your net for other logical operators (like ''or''). Your net isn''t able to learn all linearly seperatable input/outputs sets.

For the most part your threshold or acivation function must be either Heaviside or create a continuous error surface.

- mongrelprogrammer

Mongrel programmer, what exactly do you mean it makes <>the network non-linear? And why is that a bad thing? Step functions (heaviside - had to look that up) aren''t linear either, so whats the problem?

Well, the threshold function doensn''t have to be linear - it is how it affects the net''s linearness (if that is even a word...). Your threshold function divides the input into 3 parts, allowing you to hardcode the weights for a specific problem (xor). Think about how you would train such a network, the single layer methods woudn''t work (percpetron rule, hebb, ect) since they expect a Heaviside function (which, as you are probably now aware exibhit specific traints). And the gradient methods woudn''t work since - well - there is no gradient : )

So basically you are hardcoding your net into specific results while taking away its ability to learn other patterns that it should (ie. or).

- mongrelprogrammer