Jump to content

  • Log In with Google      Sign In   
  • Create Account

AI for continuous snake game (trying NEAT algorithm)


Old topic!

Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.


  • You cannot reply to this topic
15 replies to this topic

#1   Members   

179
Like
3Likes
Like

Posted 12 May 2014 - 02:55 PM

Hi!

 

I'm trying to evolve an AI for a game of 'continuous' snake.

Check this vid to understand the game:

Basically, the snake drives a bit like a car and the first one of the 2 players to crash into his trail or the other's trail loses the game.

What ideas would you have guys to design an AI for this ?

The state space is huge: ideally there'd be 60fps per second and every frame, the snake can steer from -1 (left) to +1 (right) including all the values in between.

So A* wouldn't cut it.

 

My current try is to use the NEAT algorithm (http://www.cs.ucf.edu/~kstanley/neat.html).

I have the framework working and the AI is starting to show some traces of intelligence.

The fitness value doesn't go up much though and it tends to converge toward the spiral pattern. It might be a good strategy though

The inputs I've chosen so far are:

- the opponent angle relative to us

- the opponent distance

- the opponent heading relative to us

- the eyes: inputs representing how far a certain number of ray casts can go before hitting a wall (represented on the vid)

 

I've attached the code to this post if anyone wants to try it.

You just need to install the SharpDX packages using nuget and hit F5.

 

Basically I'm interested in knowing what you think, and what other ideas you could give me for coding an AI for this game.

 

Thanks!

 

edit:

Added the exe

Numpad 4 and 6 : turn left and right

Space : Restart a game against the new champion

E : Pause/Unpause evolution. Useful to get a responsive UI. Pauses after the current iteration (8s on my machine)

Attached Files



#2   Members   

5753
Like
1Likes
Like

Posted 12 May 2014 - 05:34 PM

Interesting, seems to work okay with what you have?

 

Some other options:  

 

Some basic avoidance steering AI (it would be somewhat stupid, but it might work just fine)

Marco Pinter's method, which is A* + grid + dubin's car: http://www.gamasutra.com/view/feature/131505/toward_more_realistic_pathfinding.php?print=1



#3   Members   

179
Like
1Likes
Like

Posted 12 May 2014 - 06:20 PM

Ah, very interesting article thanks, I'll read it carefully tomorrow for insights.

Basic avoidance steering would work to avoid obstacles, but it'd be pretty stupid indeed.

 

A good player needs to:

1) Avoid walls. Your two suggestions address that.

2) Notice occasions when it can 'cut short' the other one (when next to him a bit ahead).

3) Avoid getting 'cut short'.

4) Also ideally a great player would need an idea of the topology of the current 2d space to try to enclose the other one in a smaller space / safeguard himself a bigger space.

 

My approach seems to have potential indeed as it starts to exhibit 1), 2) and 3). I wonder what other kind of inputs I could give the neural network though...

Also for some reason, my networks do not evolve any hidden neurons so far. I probably need to get more familiar with the NEAT algorithm.



#4   Members   

5753
Like
1Likes
Like

Posted 13 May 2014 - 08:57 AM

Ah, very interesting article thanks, I'll read it carefully tomorrow for insights.

Basic avoidance steering would work to avoid obstacles, but it'd be pretty stupid indeed.

 

A good player needs to:

1) Avoid walls. Your two suggestions address that.

2) Notice occasions when it can 'cut short' the other one (when next to him a bit ahead).

3) Avoid getting 'cut short'.

4) Also ideally a great player would need an idea of the topology of the current 2d space to try to enclose the other one in a smaller space / safeguard himself a bigger space.

 

My approach seems to have potential indeed as it starts to exhibit 1), 2) and 3). I wonder what other kind of inputs I could give the neural network though...

Also for some reason, my networks do not evolve any hidden neurons so far. I probably need to get more familiar with the NEAT algorithm.

 

You might be able to simulate some of the 2nd and 3rd points with basic steering.  If the AI is ahead of the opponent, instead of avoidance, it could have an attractive parallel force, while if it is behind someone, it could have a repulsive force to the player.  Though there would be no way to do 4.

 

Or it might just end up a fiddly mess =)

 

I look forward to seeing how the NEAT algorithm does though, and it looks promising.


Edited by ferrous, 13 May 2014 - 09:02 AM.


#5   Members   

179
Like
6Likes
Like

Posted 13 May 2014 - 07:08 PM

*
POPULAR

Yes, this sounds fiddly indeed smile.png But I appreciate the input and the interest.

 

I've tuned the AI some more and got something that starts to be a bit decent.

Here, I'm playing against it (I'm yellow, the AI is blue). It's actually pretty fun:) I was quite surprised.

As you can see, it's still severely lacking in number 4) above (topological analysis) but I do have some ideas for it now.

I'll try adding some inputs representing the shape of the topology toward some angles based on the results of some forward search on a grossly discretized state space. It's not yet clear in my mind. I know I need something somewhat continous for the AI to learn from it. Also again it evolves this spiral pattern which is actually not that bad.

Surprisingly, as you can see, the evolved neural network has only 2 of the inputs connected (or I don't understand the NEAT algorithm yet)

I can't wait to get more time on it. If someone wants to play it, I'll be happy to upload an exe (it's very simple to use)


Edited by vlad2048, 13 May 2014 - 07:09 PM.


#6   Members   

583
Like
0Likes
Like

Posted 17 May 2014 - 09:37 AM

I can't contribute sadly having never heard of NEAT before, but i'd be interested in reading about your progress smile.png

(also the exe would be nice)


Edited by SerialKicked, 17 May 2014 - 09:37 AM.


#7   Members   

179
Like
1Likes
Like

Posted 18 May 2014 - 08:45 AM

Sure, I've added the .exe in the first post. Let me know how it works for you !

 

I've not made huge progress really, I've been mostly playing with the parameters.

I got some ideas for new inputs, I'll post again when I implement them!



#8   Members   

583
Like
0Likes
Like

Posted 19 May 2014 - 12:59 PM

Thanks, It worked perfectly (Win7 4core AMD CPU -> used 100% with evo on)

 

A few things i noticed, no idea if it's of any use to you:

 

I've not played extensively and in general it tries to get the same spiral pattern you described earlier.

 

However, on my first go, its behavior was pretty weird, maybe because I died a few times stupidly at the beginning of the round (getting used to the lag, a sleep somewhere would help a bit). Maybe it reached an evolutionary dead end, I dunno. Anyway, even after 150+ generations its go-to behavior was to draw a simple circle and kill itself. After a few rounds it tried to go for a spiral again and / or follow me, failed and back to the circle for a few more rounds (rinse and repeat).

 

The graph was still getting fairly complicated, with all top nodes connected to the bottom ones and several additional nodes created in the middle. I probably should have taken a screenshot, sorry i forgot.

 

When it gets a good generation, it can be a fun opponent indeed, much less predictable than what you'd usually expect in this kind of game.


Edited by SerialKicked, 19 May 2014 - 01:35 PM.


#9   Members   

179
Like
3Likes
Like

Posted 19 May 2014 - 06:20 PM

Cool! thanks for trying.

Yes, you're right in your descriptions:

- you need to be a bit lucky to make it break from its spiral pattern

- it does now evolve hidden neurons and cyclic connections (I got the parameters right)

- you can remove the lag by pressing 'E'. And at the end of the current generation, it will pause the evolution algorithm so you can play without lag. Press 'E' again to resume evolution.

Also, it does not learn from you playing against it. It just learns by playing against different versions of itself in the background.

 

I've implemented my first idea of the new inputs. I call them smart rays, it's a bit hard to explain in detail, but basically they do a depth first search on certain paths and it does improve the AI a lot. Here's a vid of me (yellow) playing against it:

Notice I've evolved it to about the 30th generation (took a while) before taking the vid. And it even beat me a few times fair and square. These smart rays do give it a bit of an indication about the topology of the playfield. When it gets good at making sense of the smartrays, it seems it forgets a bit how to cut me short when it has the chance though.

 

I've added an .exe for it (_V2) in the first post.

A few notes about it in no particular order:

- When you start it, you'll need to wait about 2min for the 1st gen to finish before you can play (quad core intel i5-3570k 3.40GHz). Then press space to play against the AI.

- I've disabled multithreading because it causes exceptions I can't explain. So you should have a mostly lag free experience but the evolving is core times slower.

- It's again doing the silly spiral pattern at the beginning, but did evolve out of it for me.

- The .exe is exactly the same as the one I used for producing the video this time (uses Randoms() based on time seeds, so might be different every time)

- It does lag a lot when the smart rays all get blocked, but that doesn't happen too often. Might be a bug on my part.

- I had lots of fun playing it cool.png

 

I now think a good AI for it will come from switching between different AIs with a state machine (got some more ideas).

Again, if you can try it, I'd love your feedback!



#10   Members   

687
Like
0Likes
Like

Posted 19 May 2014 - 10:23 PM

If there is only 1 AI snake on the gamefield and no players, what the longest survival time it can reach?


Edited by tom_mai78101, 19 May 2014 - 10:23 PM.


#11   Members   

179
Like
0Likes
Like

Posted 20 May 2014 - 12:28 PM

1 min 11s

Really silly though as that's not what I'm trying to optimize it for, and it's not fun.

It'd probably kick a human ass at that game though



#12   Members   

687
Like
0Likes
Like

Posted 20 May 2014 - 07:04 PM

From the video I can see two places where you can probably just tweak it a bit:

  1. Directions - I could suggest that you add in a randomly generated value representing 3 states (left, straight, right), it may do unexpectedly good behaviors for the A.I. snakes. This can be applied if it is close in contact with the obstacles.
  2. Minimum distance - I see that you didn't calculate 2 sides (left and right sides) of the A.I. snake when both sides have obstacles near its head. Perhaps, I can suggest that you can take the mean average distance between the left and right side, and try to make the A.I. snake aim at that spot, so that it can barely touch the obstacles and can still keep going.


#13   Members   

179
Like
0Likes
Like

Posted 22 May 2014 - 12:46 PM

Hey Tom thanks! It's a bit hard to understand your grammar but here's my interpretation:

1. You're suggesting that I add a random input when the snake is near obstacles ? That sounds like a bad idea, how could it learn from it ?

2. You're suggesting that I find the direction of the middle of the hole in front of the snake and use that as an input ? The intuition is ok on that one, but this sounds quite fiddly. Also I would think that my smart ray pointing straight ahead already does that.

 

Of course I'm quite eager for more suggestions, silly or not, thanks!



#14   Members   

687
Like
0Likes
Like

Posted 26 May 2014 - 08:35 PM

Sorry for not replying soon enough. We had drills needed to do here.

 

  1. Not really. When there aren't any obstacles nearby, it keeps going in a straight line. Once it detects a presence of an obstacle, it randomly chooses to go either left or right of the obstacle. The moment the obstacle is outside of its detection, it goes in a straight line, and repeat the procedures if it detects another presence again and avoids it before colliding it.
  2. After 2 or more tries of avoidance, it learns by delaying the turning, so that it can do sharp bank turns or be couragious when it comes close to an obstacles.
  3. If the snake detects 2 or more than 2 obstacles within its detection range, take the mean average of two closest obstacles to the head, and make the head go straight towards it, and then return to its original state. I need a GIF to demonstrate.

uzWsKlv.gif



#15   Members   

179
Like
1Likes
Like

Posted 27 May 2014 - 05:45 PM

Cheers, love the animated gif:)

 

1. 'it randomly chooses to go either left or right of the obstacle'

Ok, imagine I look at a bunch of pixels in front of the snake, if I see some becoming non empty, then it chooses to go left or right of the obstacle. This is very fiddly, or at least your explanation would need a lot of work to make it into something programmable. If I look at a line of pixels in front of the snake and detect some pixels as occupied (not necessarily 1 interval, but possibly any combination of those pixels, edit: OK this is probably addressed in your 3.), then how exactly do I decide to go left of them ? Just go completely left/right of its field of vision. Also the turning radius needs to be taken into account, so not only this pixel line need to be taken into account, but also all the pixels in my planned trajectory. Also this algorithm doesn't look at anything after this line. Anyway I think about it, this doesn't sound programmable actually.

 

2. 'After 2 or more tries of avoidance, it learns by delaying the turning'

Maybe this addresses some of the shortcomings in 1. But I'm not exactly sure how

 

3. Ok this means go in the middle of the obstacles in the front if it can't go left or right. But again this has no knowledge of anything else than the line in front of you so it wouldn't be very smart. Here I included an image (excuse the drawing skills) showing that the snake could go straight into an obstacle in that case. I know it's a very specific example but I'm pretty sure an algorithm like that just wouldn't work in the general case

 

Again thanks for the input! But I think your approach is too simplistic.

I tried a very interesting different approach a few years ago that I'll post here (didn't manage to export the gmail thread today).

Attached Thumbnails

  • snake_heading.png

Edited by vlad2048, 27 May 2014 - 05:47 PM.


#16 rouncer   Members   

294
Like
-2Likes
Like

Posted 24 June 2014 - 09:42 PM

i (as in myself) wouldnt use neat, i dont believe in genetic algorythms.

 

What I would do, is put some "sight" into a set of inputs, like your doing now, with those hairs,  its best it always knew where the opponent was, maybe even what part of the screen he was.  the difficult thing would hes only aware of some of the lines there.

 

If youd like to do it like this yet still speculation of mine ->

 

now youve got that,  just randomize a function that leads to left and right directions.

 

ok, if you do that, youll have to wait ten million years for him to learn a single step,  cause you cant justify any of its learning yet.   its based apon success of wins, so youd have to wait for a whole game to finish to even judge a single randomization.

 

what you have to do is form a rule for your play, in the form of a function watching your outputs, but giving you the sight that he has,  cause he pretends you play like him, except you cheat and see the whole thing in detail.

 

ok, to actually justify the mutation, you have to play games in high speed of him, and the function that is "dummying" you,  the more you speed it up the quicker hell be able to justify its behaviour as successful or a failure.  the length of time it takes for him to lose or win (on a pole) would be how you would score it.






Old topic!

Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.