21 replies to this topic

[Teddybot]

The algorithm to make a light cycle play more like a human has been eluding us for a long time.

Most or all action games only become more challenging by speeding up game play or number of enemies.

With the Tron light cycle game the challenge increases with speed.
We want the challenge to increase along with the computers strategic progression.

Possible help in this area:
A technology which stores huge numbers of games and their outcomes along with quick search of the games data streams.
A technology which is able to quickly identify a current game and generate a list of similar games.
Ability to improvise another game into this one by handling the interpolation, transposition and scaling of different games.

This is an implementation of my variation of the cortical algorithm PDF psuedocode:


More vids:
http://www.youtube.c...er/htmtutor#p/u

The problems with seeing games at various scales, knowing which games the current game is like with low processing power are very well handled by this.
There are additional hurdles and I believe we are at a point where they can be overcome.

If you are not excited about math you will like this new biological based emergent algorithm very much.
It offers ways to have improvised motor output and lots more.

How is this received so far? With an open mind, objections etc??

Thanks!

[IADaveMark]

Huh?

[Olof Hedman]

If not even IADaveMark has any idea what you are talking about, maybe you need to explain yourself better

I don't even begin to understand what problem you are trying to solve...
And can't really comment anything on the algoritm without any description of how it works or what it solves...

[willh]

I read your description and watched the video. Is this what you are saying?

- You find a list of past games that seem similar, in some respect, to the current game.

- Using the past games as a model you attempt to find a new action that is optimal.

- Optimal, in this case, is a decision that appears human like in its pursuit of a win.

- The difficulty of the CPU opponent is increased by giving it more past games to compare to.

I don't know anything about HTMs, but from what you're saying they sound like a way of storing and retrieving events from a collection of records based on some similarity match.

Am I in the right ball park?


It would be helpful if you could post a video of a light-cycle game without and with your HTM implementation. Could you compare this method to some existing ones? I've seen some really good light-cycle AIs built using genetic algorithms.

[Teddybot]

See numenta.com to get familiar with this general algorithm of our neocortex.
It has a psuedocode pdf you can download and implement to your liking.

- You find a list of past games that seem similar, in some respect, to the current game.
The algorithm may recall from millions of games within 100 memory steps the best and worst matching games for example.
The important part here is we have an algorithm that is alreadying 'seeing' with 100 neuron steps like our mind.
This example is just one possible use.

>>a way of storing and retrieving events from a collection of records based on some similarity match.

***Yes but it does it over planes of generalization and scaling concepts within a game***
It is able to predict what you will do on large and small scales and even is able to predict next game with minimum CPU.

Check out the vision videos and algorithms at Numenta.com - what they do was impossible before it came out.

[IADaveMark]

Oh good, you're back.

[willh]

The important part here is we have an algorithm that is alreadying 'seeing' with 100 neuron steps like our mind.
This example is just one possible use.

>>a way of storing and retrieving events from a collection of records based on some similarity match.

***Yes but it does it over planes of generalization and scaling concepts within a game***
It is able to predict what you will do on large and small scales and even is able to predict next game with minimum CPU.

That could mean anything. Seriously, anything. Genome clustering? Database optimization? Compression algorithm?

[Olof Hedman]

That could mean anything. Seriously, anything. Genome clustering? Database optimization? Compression algorithm?

The pdf on numenta seem to suggest some sort of neural network.
Have only read the intro so far, a bit interesting.

[LorenzoGatti]

I don't think lightcycles are a suitable problem domain for techniques that employ very little abstraction and directly reproduce the moves that gave good results in cases that match the current situation:

• How do you maintain high level plans? Even if you figure them out from your examples, how do you ensure that you stick to the same plan when your system extracts other examples?
  Trapping an opponent can be a very long process: in a traditional racing game stringing together good short term reactions keeps you close to the ideal trajectory, but in lightcycles the only meaningful short term decision is avoiding crashes: clearly not enough to win, since the benefit of building walls is usually reaped much later.
• Minute geometrical differences (e.g. whether one more lightcycle path can fit in a certain gap) can have a great importance (e.g.whether the lightcycle can come back through the gap or gets trapped), but not always (e.g. you might not want to go in and out of that gap in the first place). How can you be confident that an example is relevant or irrelevant?
  The state space given by lightcycle walls is huge, probably condemning your example-based approach by sheer curse of dimensionality, and decent ways to simplify it are hard to find and time-varying.

[Álvaro]

There was a Google AI contest about this. This is a great read about the winning entry. I would start there. This is probably overkill for beating humans, so you can dumb it down to make it fun.

[Teddybot]

Everything in the last two posts has been extremely considered and gone over for years before making this thread.

Thank you for your replies,
Teddybot

[Teddybot]

http://www.youtube.com/results?search_query=jeff+hawkins&oq=jeff+hawkins&aq=f&aqi=g5&aql=&gs_sm=e&gs_upl=1200l3123l0l3866l12l11l0l3l3l1l220l1304l0.6.2l8l0

Jeff Hawkins

Derivatives of this technology will be the foundation for tomorrows software.
The first person or group to sucessfully apply HTM technology to gaming will recorded in history.

[Álvaro]

What's the point of this thread? Are you just telling us how excited you are about Numenta's technology?

If you want to have good AI for a light cycle game, concentrate on making AI for a light cycle game, and don't think that a generic solution based on not-quite-ready technology is going to give you good results.

I did see a Numenta presentation video 4 years ago and, although I have some sympathy for the people that want to understand what the neocortex is doing, I don't think they have delivered anything of substance. They mostly have managed to resurrect the excitement that ANNs generated decades ago. I don't see anything to be too excited about.

[Teddybot]

The core of the technology is required to accomplish proper AI for a light cycle game.
It is possible to add fundamentals to the technology and create a box for specific solutions.

[Álvaro]

The core of the technology is required to accomplish proper AI for a light cycle game.

That's an unsubstantiated claim. You seem to be so excited about this technology that you can't possibly imagine a different solution, even when faced with the fact that a recent Google AI contest was won by programs that did not use the technology, even though it was available.

[IADaveMark]

The core of the technology is required to accomplish proper AI for a light cycle game.

Aaaaannndd.... you're wrong.

[Teddybot]

I appreciate your opinions.

Thanks,
Teddybot

[willh]

TeddyBot: I read the PDF on Numentas site. I'm still not sure what it's supposed to do. Unless I missed something, the video linked in their site doesn't include a demonstration if the technology in action.

What's the advantage of a HTM over other methods? What sorts of problems can it solve that traditional ML technologies struggle with?

Would you consider IBMs Watson a "proper" AI? How about the AI that drives many virtual pets? Or the AI that detects spam before it gets delivered to your inbox? The AI that detects credit fraud? What makes these things less proper than the AI you envision?

[wodinoneeye]

Are you talking about 'situational factoring' where you analyze 'game' results and try to figure out which factors can be used to define the situation and then correlate that situation with the action and results (themselves subject to factoring) so that you can predict outcomes and try to carry out 'correct' actions on future situations to have a favorable outcome ???

Such 'factoring' is part of the 'cognitive' area of AI (I recall reading about it 25 years ago).

Static system situations are usually fairly easy to solve, but the problem becomes MUCH more difficult when you have a dynamic system with temporal effects (when you have to figure out which factors from the past in a sequence of actions were the cause of the results).

Of course as usual any gathered information where uncertainty (missing info) is involved adds at least a magnitude more difficulty to solving.

[willh]

... the problem becomes MUCH more difficult when you have a dynamic system with temporal effects (when you have to figure out which factors from the past in a sequence of actions were the cause of the results).

This is one of the things that hidden markov models aim to solve. The Viterbi algorithm allows you to determine the most likely sequence of events that could have resulted in a particular outcome.

ANNs may have a bad reputation, but there have been some very interesting developments relating to neural networks recently (see Geoffrey Hinton and U. of Toronto). It's plausible that HTMs are something new and fascinating. If only someone who has used them could explain what they do...... <*cough*teddybot*cough*>