Archived

This topic is now archived and is closed to further replies.

Jotaf

Storing logic information using neural networks

Recommended Posts

Has anyone ever tried to use neural networks to store information like "A is B" etc? I mean, like the part of our brain that stores that a cow is a mammal and a mammal is an animal, not the part that finds out that if this is true then a cow is an animal (this would be the "logic unit" - although if you know anything about that it would be great if you shared that too ). I already know everything (or most of) what I need about recognizing simple patterns, I wanted to know if this is possible, in the works, or if it has been done [edited by - Jotaf on May 27, 2004 12:19:21 PM]

Share this post


Link to post
Share on other sites
quote:
Original post by Jotaf
Has anyone ever tried to use neural networks to store information like "A is B" etc?

I mean, like the part of our brain that stores that a cow is a mammal and a mammal is an animal, not the part that finds out that if this is true then a cow is an animal (this would be the "logic unit" - although if you know anything about that it would be great if you shared that too ). I already know everything (or most of) what I need about recognizing simple patterns, I wanted to know if this is possible, in the works, or if it has been done




Yes, some things along these lines have been done. The experiments I''ve seen using MLPs (multilayer perceptrons) didn''t seem terribly promising. Some work was done at some point using SDMs (sparse distributed memory), where the network is fired multiple times, chaining one iteration off the last. AFFM (adaptive fuzzy feature maps) also represent connections among distinct items.

Frankly, though, I would imagine that this sort of operation (if I understand your meaning) would be more efficiently constructed using a relational database or Prolog.

-Predictor
http://will.dwinnell.com




Share this post


Link to post
Share on other sites
I would use semantic networks...

Semantic network is sort of an array of elements,
each element has two arrays: one that stores the elements it is connected to, and the other stores the type of connection.

check out www.generation5.org

Share this post


Link to post
Share on other sites
Yes, a relational database, or semantic network, is the right tool for the job. But our brain can do it using neurons, so it should be possible to create a construct that works like a semantic network, but instead of being hard-wired, it uses artificial neurons.

Why am I not surprised that multilayer perceptrons aren't too good for this? They're easy to code but a brain doesn't work like that (this is too limiting, real neurons are much more flexible), it's more like a neuron can accept input from any other neuron and send output to any other neuron - is that the "sparse distributed memory" you're talking about? It would be great if you had some links to papers describing this (and even greater if they weren't too academic and lenghty, thanks ).

AFFM could be useful to find out what neurons in a flexible network are more likely to relate to each other, right?

[edited by - Jotaf on May 28, 2004 12:00:35 PM]

Share this post


Link to post
Share on other sites
Hello...
This is an area of programming that I am very interested in, but haven't actually got around to studing yet... but..I think what you want to do(simulate human memory and knowledge) may be very difficult, or impossible with todays technology. The problem that I see right away is that our brains never turn off. therefore we have been colecting, and processing, and storing info since even before we were born. That is why I think it would be hard to do this in a computer because of the fact that sometime the computer will go off, and all the info you have stored in neurons will be lost.

That said, I still think this is something we should research, and figure out..... maybe find a way to store the info held to disk and load it next time. But there is tons of information in our brian. Would be a good challenge to overcome, that's for sure

Any other ideas or suggestions?

Mike

[edited by - mrhodes on May 28, 2004 12:20:25 PM]

Share this post


Link to post
Share on other sites
One of the grad students I shared an office with when I was doing my PhD developed a feed-forward ANN based on constraint satisfaction nodes that could perform this sort of operation trivially. If I recall correctly it could also be trained in forward mode (no need for back-prop)! Last I heard her model was being patented but there was some opposition from some ''big wig'' members of our faculty who didn''t think what she was doing was reasonable (I don''t think they like being shown up by a grad student). I think she did publish a paper on it... or at least she was off at a conference delivering a paper when I finished up... I''ll see what I can track down for you...

Timkin

Share this post


Link to post
Share on other sites
I''ve been doing some conceptuals on this and while trying to find some info about this on the net i ran into this gamedev post. I''m glad I''m not crazy enough to be the only one who thought about this while reading about how the brain stores it''s memories. My idea is to use a core module which then stores it''s information as a neural network. So if the core module can train a neuron to recognize red, then it is a memory place where the color red is stored so it doesn''t have to be stored anymore and all of the objects that the system remembers as being specifically red will be connected to this.

I don''t know where i''m going to so I''m reading about this on the net after some marihuana (jk)

Share this post


Link to post
Share on other sites
A researcher by the name of Ben Goertzel had an idea he called Hebbian Logic Networks, which he felt would work for this, but it''s never been implemented (he''s working on another method most of the time).

The paper can be found here.

Keep in mind this is way beyond the level of difficulty of usual AI''s for games (or any other purpose, for that matter).

Share this post


Link to post
Share on other sites
It''s a pretty interesting paper, it is quite high level in its content but I''m sure we can take it to much simpler terms. As the paper mentions it is only an outline of what can be the real thing. It''s great to know someone else thought about a theory you were unsure off yourself, this opens up many ideas in my head and encourages. Just gotta make it better!

THanks Flarelocke

Share this post


Link to post
Share on other sites
I did a little follow up to my last post but was not able to find any of Bernadette's publications online. If you're particularly interested in this research, drop me an email and I'll give you her email address. I don't want to post it pubicly as you never know what sort of spam she might end up getting!

Cheers,

Timkin

[edited by - Timkin on May 30, 2004 9:04:34 PM]

Share this post


Link to post
Share on other sites
Thanks for your replies

mrhodes that''s not really a problem, you can simply store them on disk But you have a point there, we have a lot more data on our daily lives than any other neural network, and duplicating that is really hard. But if a neural network like this works, at least it has the potential to develop an understanding of the world (logic and relations) similar to a young human.

Timkin thanks, I already sent you an e-mail

lukar I''m not sure it works like that. The idea I had looked a bit more like a semantic network. A neural network trained specially for this would look nothing more than a database program that works with a semantic network - it has inputs that tell it to store new information (and inputs that can be used to tell it what kind of information it is), or retrieve it (returning the desired information through the outputs). A complete brain could use this "module" to do tons of stuff that plain pattern recognizers can''t. Am I right?

Flarelocke I''m not really interested in the application of this for games, it''s something else I got interested in, and I thought that the gurus here would have some infor on it Thanks for the link, I''ll check it out later when I have the time

Share this post


Link to post
Share on other sites
Well im trying to put together now concepts for a pattern recognizer, a pattern generator and a priorities module all with neural nets. Why? The human brain has these three to do its learning. Now I know this isn''t possible with normal cpu''s so im giving also a read to NNP''s (Neural Network Processors). I''m currently on my first semesters of mechatronics and I want to do further research on this while im on my final project.

It is very important for a system that can recognize patterns to generate it''s own... this is where imagination comes from (to my thinking) and also for it to set priorities. Of course this won''t be a very precise machine... but it can get better by time.

Share this post


Link to post
Share on other sites