I believe that was my issue with all of this... it seems that by combining the two, you are training your fitness function with a fitness function. Why not just make the original one more accurately descriptive in the first place?

I disagree. Back propagation works about as well as permuting the weights genetically. That's all this guy is doing. Treating the weights of the connections between the nodes as components of the chromosome. The only time NN works against GA is when not only the weights but the structure of the NN is dynamic.

[quote name='sooner123' timestamp='1318439931' post='4871917']The only time NN works against GA is when not only the weights but the structure of the NN is dynamic.

Why would it works against? As stated in my previous post, structural evolution is potentially one of the key benefits of training ANNs with GA.
[/quote]

Totally agree. Start with inputs feeding directly to the output and let the structure evolve on it's own. Usually works faster than actually imposing a structure. I've used this method to build classifiers for both face detection and OCR.

That said, I want to take this opportunity to pee on the GA/ANN parade here for a second.

GA/ANN can take an insanely long time to converge, and it's impossible to calculate ahead of time when covergence will occure. In some cases we are talking about days of processing without any gaurantees of an acceptable solution.

Back propagation is usually much faster than GA/ANN provided that you have training data. It's so much faster that you can often train a few hundred ANNs with different structures and different activation functions (picking the best performer when you're all done) in a fraction of the time it would take to 'evolve' one. Below is a video I made demonstrating the back propagation learning algorithm in 'real time'. If this were done using a GA/EP/whatever we would have started out with a single 'step' function within a few generations, and then waited a long time for any additional progress.

Back prop video


If you don't have training data (like in the case of the race car) then definately using a GA is fine and dandy.

Now, all of that said, there is yet a better method. I am talking about C&RT (Brieman, et al). It's statistically sound, lightning fast, and the results are human readable to the point that a 5 year old could understand what the final algorithm is doing. There are no complicated activation functions of derivatives to worry about, no learning rates to twiddle with, inputs do not need to be normalized or altered in any way, and anyone who can use an 'if then' statement is already qualified to work with them.

If there is enough interest I can write something up about them and how to implement a simple homogenity-based learning algorithm.

The reason I bring this up is because it would be much easier to hand-code a C&RTree to navigate a car around a track, and then let the machine learning tool of choice improve on it.

Okay... I'm actually surprised to see this discussion having gone on to this depth. There're loads of ways to train NN's, and GAs are a great one... but why are you even doing this? Why not just neural nets? Why not only GAs?

I think the bottom line here is that these algorithms are just too complex for what you are trying to do with them. Even if your problem space wasn't large and your heuristic (fitness function) was more specific (it seems rather vague), it would still take ages for this thing to to run it's course, and even THEN you wouldn't have any guarantee that your network is well-trained. Even less so that these AI opponents are fun / worthwhile to race against.

I think your idea sounds cool, but if you said you were investigating NNs trained by GAs in this problem space, I would think it a fantastic idea. But as you say you wish to develop a game using this, I cannot help but think that you learnt about these algorithms and are trying to apply them in a space where they do not belong.

I still don't know what sort of "neural network" is being used as a race car controller. What are its inputs (range, speed, acceleration sensors) and outputs (steering, throttle, brakes)? Does it hold some internal state? Is it really a neural network, or thinking of it as a generic concise representation of a lookup table would be more appropriate? What constraints (e.g. symmetry between left and right) can be imposed on its structure?

Without assurance that good car controllers are possible, I'm strongly inclined to attribute any problems with cars that keep crashing around to fundamentally inadequate architecture (mainly ignoring important inputs), not to insufficient or unsophisticated training.

He specified the inputs: they are range to wall, and he has four of them. Range to wall is meaured in distance I'm assuming, and not 'time to collision'.

Good car controllers are definately possible. Below are some decent ones.

a. This one made by evolving a neural network (video)
b. A prettier one using Neuroevolution
c. Even prettier one

It's not really that complicated a problem when there are no other cars on the track. With enough time the neural network would just memorize the optimal path.

I can't speak for the OP, but one reason to use a neural network (MLP) is that they can approximate any function 'as is' without special coding considerations. Changing the inputs/outputs doesn't mean re-writing the neural network code.


He is definately missing some inputs if he only has 4 direction sensors. Speed of car would be an important one.