Understanding Neural Net Programs
I'm trying to learn about basic Neural Networks and programming for them (C++). I've downloaded several source codes from tutorials etc. and libraries. They all seem to work the same by creating the network with layers and neurons and apply the backpropagtion in some manner. I've compiled several programs and run them easily enough.
As a metric to compare their functionality and ease of use I've tried to apply the typical XOR problem to each of them. It seems half of the programs handle this problem well by learning and solving for 1 and 0. The other half seem to solve for 0.5 which is the case for a Multilayer Perceptron code I have.
What's the difference between different algorithms in NN's approach to this problem? I read that the XOR problem cannot be solved by MLP but why does another progam solve it easily? I've tried reading several tutorials but I get lost in the nomenclature.
I hope someone can shed some light on Backpropagation, FeedForwad, etc.
It's actually possible for a Multilayer Perceptron to solve the XOR problem. The XOR problem comes from the fact that a Single-Layer Perceptron (SLP) is only a linear separator, which means that it can only separate a space in 2 half-spaces. With that in mind, it's easy to see that it's impossible to find a line (in a 2D case) that would correctly classify outputs from the XOR function:
http://neuralpad.org/images/xor.jpg (the 2 axes are the inputs from the XOR function, white and black dots are the possible outputs, respectively 0 and 1)
By adding a second layer to the SLP, it becomes possible to separate the outputs correctly.
I'm not familiar with any great resources for practical neural networks implementation, but getting your hands on the book Artificial Intelligence: A Modern Approach (a library would be preferable considering the fact that you will only be interested in a couple of chapters) might be a good idea if you wan't to learn about the nomenclature (seeing that the book is more theoretical and academical than practical).
http://neuralpad.org/images/xor.jpg (the 2 axes are the inputs from the XOR function, white and black dots are the possible outputs, respectively 0 and 1)
By adding a second layer to the SLP, it becomes possible to separate the outputs correctly.
I'm not familiar with any great resources for practical neural networks implementation, but getting your hands on the book Artificial Intelligence: A Modern Approach (a library would be preferable considering the fact that you will only be interested in a couple of chapters) might be a good idea if you wan't to learn about the nomenclature (seeing that the book is more theoretical and academical than practical).
You might check out this book:
http://www.amazon.com/Techniques-Programming-Premier-Press-Development/dp/193184108X/ref=pd_sim_b_7
it goes into a lot of practical neural net stuff for video games.
Instead of back propogation, it relies on genetic algorithms to train networks which is simpler IMO.
(:
If you are looking more for an academic reference, this book is a few years old but is an AMAZING resource. It talks a lot about various networks and how they work and different functions being used for adjusting learning weights. It also covers BAM (bidirectional associative memory) which doesn't need to be trained. And i guess as a bonus it has a lot of info on fuzzy logic as well.
http://www.amazon.com/Neural-Networks-Fuzzy-Logic-Book/dp/155828298X/ref=sr_1_1?ie=UTF8&s=books&qid=1265676968&sr=1-1
http://www.amazon.com/Techniques-Programming-Premier-Press-Development/dp/193184108X/ref=pd_sim_b_7
it goes into a lot of practical neural net stuff for video games.
Instead of back propogation, it relies on genetic algorithms to train networks which is simpler IMO.
(:
If you are looking more for an academic reference, this book is a few years old but is an AMAZING resource. It talks a lot about various networks and how they work and different functions being used for adjusting learning weights. It also covers BAM (bidirectional associative memory) which doesn't need to be trained. And i guess as a bonus it has a lot of info on fuzzy logic as well.
http://www.amazon.com/Neural-Networks-Fuzzy-Logic-Book/dp/155828298X/ref=sr_1_1?ie=UTF8&s=books&qid=1265676968&sr=1-1
Let me rephrase my question.
What is the difference between a Multilayer Perceptron Network and a Feedforward Backpropagation network?
What is the difference between a Multilayer Perceptron Network and a Feedforward Backpropagation network?
Nothing, really. "Multilayer Perceptron Network" describes a NN with at least one hidden layer, regardless of training algorithm, though backpropagation is by far the most common training algorithm. "Feedforward Backpropagation" describes a gradient-descent-based NN training algorithm, which is most often used to train multilayer NNs.
Oh, and as long as I'm posting in an NN thread, I should really point out that the primary use of NNs is by people who don't know the range of machine learning techniques available to them, don't understand the situations in which NNs are inferior to other machine learning techniques, or incorrectly assume that because NNs have "neural" in the name, they are intrinsically more capable of emulating or rivaling human decision-making processes than other machine learning techniques.
Oh, and as long as I'm posting in an NN thread, I should really point out that the primary use of NNs is by people who don't know the range of machine learning techniques available to them, don't understand the situations in which NNs are inferior to other machine learning techniques, or incorrectly assume that because NNs have "neural" in the name, they are intrinsically more capable of emulating or rivaling human decision-making processes than other machine learning techniques.
Quote:Original post by Sneftel
Oh, and as long as I'm posting in an NN thread, I should really point out that the primary use of NNs is by people who don't know the range of machine learning techniques available to them, don't understand the situations in which NNs are inferior to other machine learning techniques, or incorrectly assume that because NNs have "neural" in the name, they are intrinsically more capable of emulating or rivaling human decision-making processes than other machine learning techniques.
But that's what The Terminator uses, so it's cool.
Quote:Original post by Sneftel
Oh, and as long as I'm posting in an NN thread, I should really point out that the primary use of NNs is by people who don't know the range of machine learning techniques available to them, don't understand the situations in which NNs are inferior to other machine learning techniques, or incorrectly assume that because NNs have "neural" in the name, they are intrinsically more capable of emulating or rivaling human decision-making processes than other machine learning techniques.
Yeah, that seems to be the standard reply these days.
I'm no huge fan of NNs and I've been told it's but one of many classifiers. For many courses though NNs seem to be the only classifier that is taught, which would be a nicer explanation for their misguided popularity than willful ignorance. I'd like to think I somewhat understand the limitations of NNs and to me they still seem like a useful tool to model an implicit function suspected to govern a set of data. That would be a sane use case, no? [smile]
NN's can work and can work quite well. You can use them for other things like time series prediction as well(I have). The problems with NN's is there is no agreed upon good way to set up a problem. I referenced and excellent paper in one of my papers, but forget its title??? A basic abstract of the paper is that many researchers claim they know the perfect number of input nodes/hidden nodes/output nodes or ratio of nodes but a lot of the findings contradict each other. It is really hard to know if you are using the right number of nodes and if you can do much better with a different combination. Change one number and because of their nature and they are a data driven technique the results can be quite different. I like NNs but they are far from perfect.
Quote:Original post by Sneftel
Oh, and as long as I'm posting in an NN thread, I should really point out that the primary use of NNs is by people who don't know the range of machine learning techniques available to them, don't understand the situations in which NNs are inferior to other machine learning techniques, or incorrectly assume that because NNs have "neural" in the name, they are intrinsically more capable of emulating or rivaling human decision-making processes than other machine learning techniques.
Very true, and well said.
Quote:Original post by remigius
I'd like to think I somewhat understand the limitations of NNs and to me they still seem like a useful tool to model an implicit function suspected to govern a set of data. That would be a sane use case, no? [smile]
It's sane, but not necessarily optimal. There are many other ways of modelling and approximating functions, many of which will be more suitable than neural networks, especially if you know a little about the characteristics of the function.
This topic is closed to new replies.
Advertisement
Popular Topics
Advertisement