Jump to content
  • Advertisement
Sign in to follow this  
tobinare

Understanding Neural Net Programs

This topic is 3109 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

I'm trying to learn about basic Neural Networks and programming for them (C++). I've downloaded several source codes from tutorials etc. and libraries. They all seem to work the same by creating the network with layers and neurons and apply the backpropagtion in some manner. I've compiled several programs and run them easily enough. As a metric to compare their functionality and ease of use I've tried to apply the typical XOR problem to each of them. It seems half of the programs handle this problem well by learning and solving for 1 and 0. The other half seem to solve for 0.5 which is the case for a Multilayer Perceptron code I have. What's the difference between different algorithms in NN's approach to this problem? I read that the XOR problem cannot be solved by MLP but why does another progam solve it easily? I've tried reading several tutorials but I get lost in the nomenclature. I hope someone can shed some light on Backpropagation, FeedForwad, etc.

Share this post


Link to post
Share on other sites
Advertisement
It's actually possible for a Multilayer Perceptron to solve the XOR problem. The XOR problem comes from the fact that a Single-Layer Perceptron (SLP) is only a linear separator, which means that it can only separate a space in 2 half-spaces. With that in mind, it's easy to see that it's impossible to find a line (in a 2D case) that would correctly classify outputs from the XOR function:

http://neuralpad.org/images/xor.jpg (the 2 axes are the inputs from the XOR function, white and black dots are the possible outputs, respectively 0 and 1)

By adding a second layer to the SLP, it becomes possible to separate the outputs correctly.

I'm not familiar with any great resources for practical neural networks implementation, but getting your hands on the book Artificial Intelligence: A Modern Approach (a library would be preferable considering the fact that you will only be interested in a couple of chapters) might be a good idea if you wan't to learn about the nomenclature (seeing that the book is more theoretical and academical than practical).

Share this post


Link to post
Share on other sites
You might check out this book:

http://www.amazon.com/Techniques-Programming-Premier-Press-Development/dp/193184108X/ref=pd_sim_b_7

it goes into a lot of practical neural net stuff for video games.

Instead of back propogation, it relies on genetic algorithms to train networks which is simpler IMO.

(:

If you are looking more for an academic reference, this book is a few years old but is an AMAZING resource. It talks a lot about various networks and how they work and different functions being used for adjusting learning weights. It also covers BAM (bidirectional associative memory) which doesn't need to be trained. And i guess as a bonus it has a lot of info on fuzzy logic as well.

http://www.amazon.com/Neural-Networks-Fuzzy-Logic-Book/dp/155828298X/ref=sr_1_1?ie=UTF8&s=books&qid=1265676968&sr=1-1

Share this post


Link to post
Share on other sites
Let me rephrase my question.

What is the difference between a Multilayer Perceptron Network and a Feedforward Backpropagation network?

Share this post


Link to post
Share on other sites
Nothing, really. "Multilayer Perceptron Network" describes a NN with at least one hidden layer, regardless of training algorithm, though backpropagation is by far the most common training algorithm. "Feedforward Backpropagation" describes a gradient-descent-based NN training algorithm, which is most often used to train multilayer NNs.

Oh, and as long as I'm posting in an NN thread, I should really point out that the primary use of NNs is by people who don't know the range of machine learning techniques available to them, don't understand the situations in which NNs are inferior to other machine learning techniques, or incorrectly assume that because NNs have "neural" in the name, they are intrinsically more capable of emulating or rivaling human decision-making processes than other machine learning techniques.

Share this post


Link to post
Share on other sites
Quote:
Original post by Sneftel
Oh, and as long as I'm posting in an NN thread, I should really point out that the primary use of NNs is by people who don't know the range of machine learning techniques available to them, don't understand the situations in which NNs are inferior to other machine learning techniques, or incorrectly assume that because NNs have "neural" in the name, they are intrinsically more capable of emulating or rivaling human decision-making processes than other machine learning techniques.

But that's what The Terminator uses, so it's cool.

Share this post


Link to post
Share on other sites
Quote:
Original post by Sneftel
Oh, and as long as I'm posting in an NN thread, I should really point out that the primary use of NNs is by people who don't know the range of machine learning techniques available to them, don't understand the situations in which NNs are inferior to other machine learning techniques, or incorrectly assume that because NNs have "neural" in the name, they are intrinsically more capable of emulating or rivaling human decision-making processes than other machine learning techniques.


Yeah, that seems to be the standard reply these days.

I'm no huge fan of NNs and I've been told it's but one of many classifiers. For many courses though NNs seem to be the only classifier that is taught, which would be a nicer explanation for their misguided popularity than willful ignorance. I'd like to think I somewhat understand the limitations of NNs and to me they still seem like a useful tool to model an implicit function suspected to govern a set of data. That would be a sane use case, no? [smile]

Share this post


Link to post
Share on other sites
NN's can work and can work quite well. You can use them for other things like time series prediction as well(I have). The problems with NN's is there is no agreed upon good way to set up a problem. I referenced and excellent paper in one of my papers, but forget its title??? A basic abstract of the paper is that many researchers claim they know the perfect number of input nodes/hidden nodes/output nodes or ratio of nodes but a lot of the findings contradict each other. It is really hard to know if you are using the right number of nodes and if you can do much better with a different combination. Change one number and because of their nature and they are a data driven technique the results can be quite different. I like NNs but they are far from perfect.

Share this post


Link to post
Share on other sites
Quote:
Original post by Sneftel
Oh, and as long as I'm posting in an NN thread, I should really point out that the primary use of NNs is by people who don't know the range of machine learning techniques available to them, don't understand the situations in which NNs are inferior to other machine learning techniques, or incorrectly assume that because NNs have "neural" in the name, they are intrinsically more capable of emulating or rivaling human decision-making processes than other machine learning techniques.


Very true, and well said.

Share this post


Link to post
Share on other sites
Quote:
Original post by remigius
I'd like to think I somewhat understand the limitations of NNs and to me they still seem like a useful tool to model an implicit function suspected to govern a set of data. That would be a sane use case, no? [smile]

It's sane, but not necessarily optimal. There are many other ways of modelling and approximating functions, many of which will be more suitable than neural networks, especially if you know a little about the characteristics of the function.

Share this post


Link to post
Share on other sites
Sign in to follow this  

  • Advertisement
×

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

We are the game development community.

Whether you are an indie, hobbyist, AAA developer, or just trying to learn, GameDev.net is the place for you to learn, share, and connect with the games industry. Learn more About Us or sign up!

Sign me up!