Genetic Algorithm vs Neural network

Brief?

Neural networks work well when you want to try and recognise certain patterns in a group of inputs, even when some of those inputs vary somewhat.

Genetic algorithms work well when you want to find the best combination of several parameters to achieve a certain result, but can't easily predict which combinations are best without trial and error.

I don't think I've ever seen a problem that could be equally well solved by both.

Both are (roughly) regression methods.

The only reason to use neural networks is if you MUST feed each item in your training set one by one.

The only reason to use GA are... I cant think of one, sorry.

For more serious regression methods, if you are interested:

If your model is a mixture of several distributions, use EM-algorithms

If you can compute the derivative of your fitness model, use Levenberg-Marquad

If you cannot compute the derivative, but have a lot of processing time, use Nelder-Mead.

If you cannot compute the derivative or cannot describe the model, use SVM for regression. Its almost always a good option too.

You don't have to choose one of them since you can do both at the same time(applying a genetic algorithm to the weights of a neural net). The primary difference that I note when choosing one is that a genetic algorithm solves for one solution and a neural net solves for a function of solutions. A genetic algorithm just finds one (semi)optimal set of numbers and doesn't actually do any of the AI-ing itself. A genetic algorithm could be used to find the optimum value of each piece in a chess AI though the chess AI is primarily some type of search tree algorithm. A neural net on the other hand is a standalone AI that takes inputs and gives outputs. Neural net gives function, genetic algorithm gives a single set of solutions.

Quote:

Original post by SteadtlerThe only reason to use GA are... I cant think of one, sorry.

I'll try to think of one:

The only reason to use GA is if your parameter space is a set of Turing machines.

Staedtler,

You consistently say that genetic algorithms are regression methods, but that just isn't true. A genetic algorithm is a (weak) heuristic search method. Applied to the traveling salesman problem, for example, we're essentially looking for a best path which is a path-finding/graph problem. Or, used as a feature selection method feeding into a linear/non-linear regression, the GA is again acting as a search method.

Also, the methods you mention (for "serious" problems) are optimization methods applied to regression problems, not regression methods themselves. SVM regression is iteslf a regression method using kernal projection and quadratic programming as the optimization method.

To the OP, if you're having trouble deciding whether to use a GA or NN, you probably need to understand both more thoroughly before using either one. They are very different techniques and have their applications in potentially very different domains. There is some cross-over but his is usually the result of changing the formulation and representation to achieve the desired result.

-Kirk

[Edited by - kirkd on October 11, 2006 12:07:35 PM]

Depends on what you're doing and what you're looking for.

Neural nets are a form of regression, that is they are used for predicting a continuous value. They are a linear combination of functions (often sigmoids).

Genetic algorithms are essentially a not-so local stochastic hill-climber. Depending on the problem they can be applied in a number of ways.

Quick example:
One interesting experience I have had with both of them, is on the reinforcement learning problem: the mountain car problem.
Here we applied standard reinforcement learning techniques, with a neural net as an approximation method rather than using a Q-tabel.
We then compared it to using a GA to directly search for an optimal policy.
The GA performed much better, finding solutions that got the car out of the valley in fewer iterations.

Why did it work better in this case? Not entirely sure, but because the search space is easily described as a series of "accelerate right"'s or "accelerate left"'s, is makes sense that just looking for a good combination of these should be at least as easy as trying to fit some function.

Just remember that there's no silver bullet, nothing is going to always work better than any other thing (pretty much).

Quote:

Original post by kirkd Staedtler, You consistently say that genetic algorithms are regression methods, but that just isn't true. -Kirk

Classifying those algorithms is difficult. GA are (almost always) used to search a parameter space, looking for a better fitting, which is regression in the general sense. Hill-climbing is another example of a non-linear regression method that is also a search method. If the OP is considering using either GA or ANN, he could probably use other regression or optimization methods.

In any comparative studies of regression methods Ive seen, GA and ANN are on the bottom end, so its probably a good idea to at least look at the methods Ive mentionned.

Quote:

Original post by Steadtler Classifying those algorithms is difficult. GA are (almost always) used to search a parameter space, looking for a better fitting, which is regression in the general sense. Hill-climbing is another example of a non-linear regression method that is also a search method. If the OP is considering using either GA or ANN, he could probably use other regression or optimization methods. In any comparative studies of regression methods Ive seen, GA and ANN are on the bottom end, so its probably a good idea to at least look at the methods Ive mentionned.

I agree 100% that people should have a variety of tools available in their toolbox, and the methods you mentioned are indeed the pick of the litter for numerical optimization. But, Nelder-Mead and Levenberg-Marquardt are not strictly regression methods but rather numerical optimization methods. You could apply them to any numerical optimization problem given the requirements you described, including regression. They can also be applied to neural nets.

But I think we're mixing terms and contexts. I just want to make sure that beginners like the OP know that GAs are a search method that can be formulated as numerical optimization, but is not necessarily such a method.

-Kirk

Quote:

Original post by kirkd But I think we're mixing terms and contexts. I just want to make sure that beginners like the OP know that GAs are a search method that can be formulated as numerical optimization, but is not necessarily such a method. -Kirk

Alright, Im not arguing with that. I just wish people would take the time to consider other options instead of going straight to overhyped methods.

Quote:

Original post by Steadtler Alright, Im not arguing with that. I just wish people would take the time to consider other options instead of going straight to overhyped methods.

I couldn't agree more. GAs and NNs are very sexy, but there are so many methods out there that offer as much or more bang for the buck.

-Kirk

Quote:

Original post by Steadtler I just wish people would take the time to consider other options instead of going straight to overhyped methods.

I second and third that. Neural nets and genetic algorithms, despite their simplicity, are often overkill. Many times a weight matrix, simple search tree, or even just a set of simple hardcoded instructions will be more effecient and easier to code and read.

Though I must admit genetic algorithms are fun to play with...

Thank you very much, all your answers have helped me a lot.

Quote:

Original post by Alrecenk Many times a weight matrix, simple search tree, or even just a set of simple hardcoded instructions will be more effecient and easier to code and read.

Actually, I can write an ANN in a single line of code representing

y=W<sup>T</sup>Φ(x)

and a multilayer ANN is no more difficult than nesting additional terms in the right hand side...

...but that's not the point. Actually, I do agree with the above posts as well... that one should understand the problem first before thinking of the solution. When the problem is translated into a set of tasks, the solution methods are generally self-evident (and don't generally involve over-hyped methods like ANNs and GAs).

On the GA classification front... from my perspective...

In the strictest terms, a GA is an optimisation method relying on blind search, in that any given iteration of the algorithm improves the quality of a candidate solution (in this case a population of candidates) relative to an objective function and uses blind search to do so. Blind search being search where no differntial surface information is available, only function evaluations.

In this sense, it's not regression, since regression seeks to identify a mapping between two manifolds. In the optimisation problem, we know what the mapping is; it's the objective function. However, you can express a regression problem as an optimisation problem in the parameter space of some fixed functional form and hence solve it using a GA.

Cheers,

Timkin

Isn't that what I just said? 8^)

Seriously, well stated Timkin.

-Kirk

Yes it is. ;) Just lending my support but offering it from my perspective.