Quote:Original post by NickGeorgia Yep, I see. Still the linearly separable problem. Good catch. Hidden layer is our friend again.
The AP's reply (which I believe to be correct) indicates that the original implementation is faulty since it supposedly did not rely on a hidden layer. My original reply to this would have been to repeatedly try different random initial weights since the stable pattern stored may be occasionally the inverse of what is desired (depending on the initial conditions and the training algorithm used).
Thanks for all the responses, I gave up XOR and made it learn AND instead. I start trying to do something else with the net, and of course I ran into problems. :P Im gonna try some more then Im going to put up a thread here about it. Feel free to help some more there. [smile]
Why do people use logical operations in NN tests? Is it just because they are SO simple - because to me the whole idea of using a NN to do such operations is a bad example of why you need them in the first place!
Probably I've missed something and doing entirely analytical stuff with NNs (ie long division, cube roots) is a whole field of its own, but I thought I'd check...
Other than fundamental testing...well, for one reason, a neural network would have less parameters than a truth table (7 compared to 12 for the AND example).
Now expand the example, say we had a 512 x 512 bitmap image (0 or 1) and we want this image to map to a yes or no that the image is my signature. The truth table would be very large. NN would reduce the number of parameters greatly.
So in conclusion, one reason to use a NN is that the number of parameters are reduced. For small stuff like XOR, maybe not so great, but at least we know something more about NN--i.e. required hidden layer. Anyway, anyone else have any thoughts?
Well, the reason I choosed to do a logical operation in my neural net was that it was the simpliest operation I could come up with. First, everything is binary, input and output, second, there is only need for a single perceptron with only two inputs.
And why "most people" tend to use them is the same reason I think. They are simple. And often when you read articles or similiar about NN's they are often laid up as tutorials, and in tutorials you show a very basic example so the readers can get a grasp. Maybe you finish of with a "this could be used to manage operations as complex as...".
But I agree, its not in the logical operations that the power of NN's lie.
Out of interest, can inputs/outputs of a NN be vector in nature or must they be scalar? So if we're dealing in 3D coordinates must we triple the number of perceptrons etc?
And on a tangent to my other question - CAN you do long division/cube roots etc in a NN; it's the closest model to our brains, but maybe we're so inefficient that you need billions of neurons to do arithmetic in a net?!
Yep, sqrt, sine, etc. you can do that with neural network within an interval. See my journal for example. How many nodes is a question though.
As for the other question, I've only seen scalar, though you can view the input as a vector and dot product with the input weights is what is normally done first. You could also view the input as a vector to a series of different neural networks also I suppose if you were to do it that way.
You could feed a complex value to a neural network as two inputs. A vector of length n could be given as n inputs. As far as what really happens in our brains, real neurons are a lot more complicated that the artificial neurons in an artificial neural network. It takes a supercomputer just to simulate one.
And complex numbers in our brain? Brains work on chemistry, not math. It may be possible to model things using math, but ultimately it's all chemical reactions. Usually the only places complex numbers show up in modeling real world systems is in places where the imaginary parts ultimately cancel out. If you have a model of a system that has imaginary numbers as part of a physical quantity, and there's no sensible alternative way to interpret the imaginary part, something is wrong.
