Sign in to follow this  
vokuhila

Neural network for racing AI

Recommended Posts

Hello our current project is an arcade racing game. The simulation of the cars is physics based and practically impossible to predict. Right now the AI just tries to follow a target point that get's pulled along the race track in front of the car. I tweaked this system a lot to make the AI faster and more competitive, but now I'm in doubt that will ever work. Especially since we added a track with very little grip where you have to slide through the corners, my approach fails badly. That's why I want to try neural networks to train the AI on every track. My hope is that someone can recommend an easy to use library to me where I can setup and train a NN with just a few lines of code. I don't have practical experience with NNs. I think I need two input variables which indicate the car's position on the track, and three output variables for steering, throttle and brakes. Would this work? And what happens if the car spins, get's stuck or dynamically placed obstacles get in the way? My current approach can handle these situations (well it tries at least), so in such cases the AI would be able to fall back. I hope these are not too many questions. I'm primarily looking for a simple to use library so that I can instantly experiment a bit, instead of trying to get back-propagation to work for a week.

Share this post


Link to post
Share on other sites
Neural networks *could* work, but i'm not sure that they're the best tool for this job. racing car AI has been around forever (at least since pole position on the atari) and I'm not aware of any commercial release that used neural networks. It'd be pretty hassling to set one up and train it to drive; and would be almost impossible to tune. NNs aren't magic AI entites that can learn anything and perform well at all tasks; think of them more as pattern recognizers or classifiers. There are elements of pattern recognition and classification in a race situation, but I think it's better solved with a little pre-computed or designer placed data.

You should check out the Article & Resources section of this site. There is a whole series of articles about writing a race-car AI system.

-me

Share this post


Link to post
Share on other sites
Thank you for your replies.

Well I thought about the problem again. Until now I refused to implement an optimal racing line, but I guess it would help quite a bit.
Maybe that will even be sufficiently good on tarmac tracks. However, I don't see how that could work on gravel tracks, where really aggressive sliding is required to be fast.
The interview mentioned by viblo deals with this problem, but it's very vague.
So I think I'll just have to try and find out by myself. In the other thread about Neural Nets "FANN" (http://leenissen.dk/fann/) is mentioned, and from their tutorial page it looks like it's just what I searched for.

AI is a lot more work than I expected but at least it's interesting, too :)

Share this post


Link to post
Share on other sites
Quote:
Original post by vokuhila
Thank you for your replies.

Well I thought about the problem again. Until now I refused to implement an optimal racing line, but I guess it would help quite a bit.
Maybe that will even be sufficiently good on tarmac tracks. However, I don't see how that could work on gravel tracks, where really aggressive sliding is required to be fast.
The interview mentioned by viblo deals with this problem, but it's very vague.
So I think I'll just have to try and find out by myself. In the other thread about Neural Nets "FANN" (http://leenissen.dk/fann/) is mentioned, and from their tutorial page it looks like it's just what I searched for.

AI is a lot more work than I expected but at least it's interesting, too :)


If you are unable to describe the behavior you want, it is unlikely that you will be able to make any method works, machine learning even less.

Take some time to think and write the behavior you want to see. You want it to run ahead in lines, break before curves (what is each curve ideal entry speed?) slide in gravel (what are the mechanics of sliding? Facing vector vs speed vs velocity?), avoid other cars, pass other cars...

Only after you know exactly what you want, will it be time to choose a method to implement these behaviors.

Share this post


Link to post
Share on other sites
Hello Steadtler

I thought about what you wrote, and my conclusion was that I'm not able to write down rules that would let the AI drive fast enough. It's just too complicated because it depends on many parameters. Every turn on every track is different. They have different ground materials, different decline, some have to be cut, some must not be cut. etc. etc.
So the strange thing is that I perfectly know how to drive around a track, but I cannot formulate rules which describe how to drive.

That's why I implemented a system today which simply collects data while I'm driving along the track:
- positon of the car
- velocity
- heading vector

The AI simply tries to make its car imitate these parameters. This works much better than I expected. The AI can slide through corners nicely and is just 2% slower than me which means that it's almost impossible to catch up if you are a bit behind :-)

So in the end, I used a neuronal net, but not an artificial one. I drove arround the tracks hundreds of times, training the thing between my ears without any overfitting or generalisation problems. And now I just need to harvest the results.
Hm. Pretty easy and probably done a thousand times before, but I'm not that much familiar with AI techniques.

Thanks all. I think with some more tuning my problem is solved.

Share this post


Link to post
Share on other sites
Quote:
Original post by vokuhila

That's why I implemented a system today which simply collects data while I'm driving along the track:
- positon of the car
- velocity
- heading vector

The AI simply tries to make its car imitate these parameters.


Thats pretty much the same as following a well-defined raceline, dont you think?

Share this post


Link to post
Share on other sites
Quote:
Original post by vokuhila
I thought about what you wrote, and my conclusion was that I'm not able to write down rules that would let the AI drive fast enough. It's just too complicated because it depends on many parameters. Every turn on every track is different. They have different ground materials, different decline, some have to be cut, some must not be cut. etc. etc.
So the strange thing is that I perfectly know how to drive around a track, but I cannot formulate rules which describe how to drive.


That's because you're 'over-thinking' the problem. A very simple rule for turning a vehicle through an arc of any radius: apply a force to the vehicle directed radially inward, with sufficient magnitude to achieve the desired turn rate.

Because you want the vehicle to rotate as it changes velocity, you need to ensure that you generate torque on the vehicle body (that's why we have steerable wheels at one or both ends of a vehicle, rather than in the middle). If you have a tight turn, then the radial force generated by the steerable wheels is insufficient and you must align the drive wheels with the direction of resultant force required (remembering that there are two components... the radial component for cornering and the tangential component for overcoming friction and maintaining tangential speed of the vehicle). You can do this by initiating a skid and using the steerable wheels to control the angle of the skid (and hence the angle of force application on the road)... and the drive wheels to control the magnitude of the applied force. The two together give you the applied force vector. For any turn its fairly easy to determine the turning force required given the entry point and forward speed at that point, mass and rotational inertia of the vehicle and the friction properties of the road surface.

As for a controller for this problem: a simple bang-bang control scheme on the steering wheel and constant drive wheel speed should do the trick.

Cheers,

Timkin

Share this post


Link to post
Share on other sites
Quote:
Original post by Palidine
Neural networks *could* work, but i'm not sure that they're the best tool for this job. racing car AI has been around forever (at least since pole position on the atari) and I'm not aware of any commercial release that used neural networks.
-me

Colin McRae Rally 2 (I think that's what it was called) used neural nets for AI, and the lead AI programmer went on to write a book (AI Techniques for Game Programming) that was a very good primer on neural nets.

@OP
Neural Nets are not a magic bullet. Those "couple of inputs" that you listed will quickly show their limitations, and things can get very very difficult to get the cars acting in a way that you like. I did a project while in school where I made a 2D racetrack, and even then it was very difficult to get consistently good racers.

If you want to give it a try, check out the book I referenced above. But be warned that you may just be replacing your current difficulties with a new set of less obvious problems.

Share this post


Link to post
Share on other sites
1. Those who understand the tradeoffs between NNs, bayesian networks, naive bayesian networks, ridge regression, linear regression, logistic regression, etc.

2. Those who think that "neural network" sounds all cool and brainy.

One of these groups should use NNs where they're appropriate. The other group needs to pick up a textbook on artificial intelligence, and another on machine learning.

Now, neural networks are actually potentially not a bad tool for racing AI. But they're probably not the best tool for a job. Until you understand NNs in a ML context you really shouldn't be prescribing them for yourself.

Share this post


Link to post
Share on other sites
As an aside, I found that the "driving license" mode of GT4 does an excellent job of explaining proper cornering technique, in easy terms.

Share this post


Link to post
Share on other sites
Quote:
Original post by Steadtler
Thats pretty much the same as following a well-defined raceline, dont you think?
Yes, of course, I admit it. It just took me some time to find out how to do it exactly.


Quote:
Original post by Sneftel
1. Those who understand the tradeoffs between NNs, bayesian networks, naive bayesian networks, ridge regression, linear regression, logistic regression, etc.

2. Those who think that "neural network" sounds all cool and brainy.
Well it's not that I don't know anything about NNs. We used them for pattern classification at uni and I have a feeling for what they can and can not. Just wanted to give it a try, but since the AI is fast enough with my current approach NNs aren't needed anymore anyways.

Share this post


Link to post
Share on other sites
Quote:
Original post by vokuhila
Well it's not that I don't know anything about NNs.

It's not a matter of knowing anything about NNs. NNs are a pretty simple concept. It's a matter of knowing about the alternatives to NNs, and the conceptual framework in which NNs are relevant.

Share this post


Link to post
Share on other sites
I recommend this link about an interview with Jeff Hannan, creator of the Artificial Intelligence for the excellent Colin McRae Rally 2.0. A "commercial game" that uses neuronal networks to implement a rally car controller:

http://www.generation5.org/content/2001/hannan.asp

Share this post


Link to post
Share on other sites
A waypoint system which defines a best line for the ai cars to follow is usually suitable for most racing games. Which is actually what was used in Colin Mcrae 2. Great game by the way.

As used in my game "Auto Cross Racing" available at www.kjmsoftware.co.uk/
to see the system in action.

Each ai car aims for the next waypoint and moves on to the next in the sequence when it comes into range of it's target waypoint.

Each waypoint can also have a desired speed variable, which the ai cars would aim to achieve when they reach that waypoint. This way the ai cars are give direction and speed inputs. The ai cars outputs are steering and throttle / brake.

A simple Dot product of two vectors will tell each ai car if it is either to the left or right of each waypoint.

Direction vector 1 = Unit vector rotated by the cars yaw orientation
Direction vector 2 = Unit vector from cars position to waypoint position

The sign of the Dot product between both vectors should tell you to which side of the waypoint the car is pointing. eg, Left = negative, Right = positive.

Hope that helps

KJM

Share this post


Link to post
Share on other sites
Quote:
Original post by K_J_M
A waypoint system which defines a best line for the ai cars to follow is usually suitable for most racing games.


I think it's worth stressing that although in some cases waypoint systems are appropriate, that isn't always the case.

If you use just a waypoint system you are implicitly relying on the fact that the vehicle controller can accurately track linear segments, and that the dynamics effects of switching between segments are negligible. For some vehicles this may be acceptable, and for others it might not.

There are lots of examples where a level designer / artist could place either too few or too many waypoints without having a sufficiently good understanding of the underlying dynamics to predict what the results will be. Think about sharp 'S' bends and the results of under or over damped response tracking a linear segment.

Waypoint based planning has it's merits, but there are alternative path representations (including splines), which may be more appropriate depending on dynamics and control.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this