🔒 Locked

Idea for "learning" AI

Started by tetelestia Oct 3, 2005 at 10:06 PM 9 replies 2.9k views

Original Post

Oct 3, 2005 at 10:06 PM

A year or two back, I found the A.L.I.C.E. bot ( http://www.alicebot.org/ ) and found it quite fascinating. A short while later, I was thinking that it could be possible to make those bots "smarter." ALICE has no recollection of previous conversations, limiting herself a lot. The AIML code is very static, she cannot add code in herself. But what if she could? If, the bot could understand and then categorize information that is given to it, it could use more and more information as the basis for everything it says. Being a computer program, it would have vast amounts of space to store the information it learns. I had forgotten about this for quite a while, but decided to throw it up amongst a bunch of programmers and see if they think it is a plausible new face of AI. I'll try to explain my ideas more indepth about the "brain" of the bot. It would have a couple different categories of information in xml databases, for example, an English dictionary, fact dictionary, opinion, user specific dictionary and questions/unknown facts. Here's an example of how a conversation could work... User (Matt): I just got a new dog today. I named him Oliver. Bot: (think)From Dictionary- Dog=Furry animal; From Fact Dictionary- Dog=commonly held as pet, Stinks when wet, many different species; Into User Dictionary- Matt=Got pet dog on Oct 3/05, Dog's name is Oliver; Into Question Dictionary- What species of dog? Where did you get it? What colour? (REPLY): That's cool, what kind of dog is it? User (Matt): It's a Black Lab, we also got a red collar and leash for it. Bot: (think) From Dictionary- Black Lab=Species of Dog; Into User DIctionary- Matt=Dog is Black Lab, has red collar and red leash; Question queue- Where did Matt get dog from? What colour is Matt's dog? (REPLY): What colour is the dog and where did you get it from? That's an example of a pretty basic conversation, but I think the basic theory could work in something larger scale. Eventually it oculd get to the point of being able to have a semi intelligent conversation with someone or be able to have an article or something like that be pasted and pull facts out of it... I just want to know what some of you guys think about it, particularly if it could work or not. I've got quite limited programming knowledge (HTML and a tiny bit of C++), so I wouldn't be able to follow up on this by myself. Thanks for any input

mnansgar

Oct 3, 2005 at 10:20 PM

It *can work*, but only for very specific types of input and where your English phrases are not ambiguous. For instance:

(1) Take your example -- Black Lab. You used an abbreviation! Will your dictionary have all possible abbreviations as well (this is getting to be pretty large!)? Is this a species of dog, or is it a black laboratory (you could say this can be inferred from the question, but what if the user just types he has a black lab without context?)?

(2) Note that you've defined all of your words in terms of other words! How can the computer "understand" a word if each is defined by similar words that it doesn't know? Now you might refute this by saying that it will match words together; however, it is very likely that a different set of words are used in the definition than are needed to actually tell that it fits that definition, correct? If you then decided to apply a thesaurus, we'll easily run into a situation where we are likely recursive and take forever to make decisions!

(3) Parsing sentences is also a big probelm. Sure, we can deal with "Bill is red." (just assign the quality 'red' to 'Bill'), but we can't deal with natural language. Even the sentences you posted have imprecise grammar (bad comma usage mostly).

(4) How does the computer deal with actions which require visualizing the situation? Or with things which require human experiences to really understand?

... and so on. The point is, yes it is possible for a very limited subset of the English language. I think the more interesting problem lies in designing a language close to English but more amenable for computer parsing.

Nypyren

Oct 3, 2005 at 10:56 PM

It's possible to parse grammars even if there would be ambiguity in constructing parse trees - google "GLR parser". The nastiest ambiguous parts of English are prepositional phrases; the rest isn't bad. The problem I had when I was writing this kind of parser is that I couldn't find any reference material that describes English grammar in anything remotely close to BNF.

Ok, so let's say you manage to get past that part, and have a parse forest output by a GLR parser that's stored in a graph (by folding the common parts of each tree - there is a good description of this algorithm somewhere, but it's been years since I've dealt with it).

The main problem is: what the heck do you do with the graph? I think probably the most productive thing you can do is compare the graph to other preconstructed graphs (the knowledge base of your program). My idea was that you can take two graphs, connect the matching leaves end-to-end, and start marking any nodes that match, reducing both graphs to what did and didn't match. (I use this same technique for finding intrinsic functions like strcpy in optimized assembly). Ideally, you can associate some kind of metadata with each graph (or possibly metadata for each major section of the graph) that more effectively represents concepts for your AI.

Then you could finally do some grammar-based analysis of the matching and remainder sections of the graphs to figure out what the sentence means in terms of your knowledge base. You would still have to come up with a way to add new graphs to your knowledge base, do some decision making to come up with a reply/question to the user, and be able to formulate sentences that don't seem "stale" to a person.

Nypyren

Oct 3, 2005 at 11:38 PM

Here's my partial grammar which I was using to test graph generation...
Note - I came up with this thing myself, so if it looks like a piece of crap, it's because I'm not a linguistics expert.

It currently completely ignores punctuation.

Terminals:lambdadeterminernounpronounverbadjectiveadverbprepositionparticipleinterjectioncrdconj (coordinating conjunction)subconj (subordinating conjunction)$ (end of input)Nonterminals:S    (start)NP   (noun phrase)VP   (verb phrase)AdjP (adjective phrase)AdvP (adverb phrase)PrpP (preposition phrase)GP   (gerund phrase)AppP (appositive phrase)AbsP (absolute phrase)InfP (infinitive phrase)ParP (participle phrase)IC   (independent clause)DC   (dependent clause)RC   (relative clause)NFC  (nonfinite clause)IngC (-ing clause)EdC  (-ed clause)InfC (infinitive clause)NDets  (noun determiners)NPres  (noun premodifiers)NHeads (noun heads)NPosts (noun postmodifiers)NComp  (noun complement)VAux  (verb auxiliary)AdvComp (adverb complement)AdjHead (adjective head)AdjPostmod (adjective postmodifier)PrpComp (preposition complement)

And if the list of terminals and nonterminals didn't scare you, here are some productions (nonfinal experimental list that works decently for a lot of my testing input):

S -> NPS -> NP VPS -> VP// Noun PhrasesNP -> NP crdconj NPNP -> NDets NPres NHeads NPostsNP -> GPNDets -> determiner NDetsNDets -> lambdaNPres -> AdjP ParPNPres -> AdjPNPres -> ParPNPres -> lambdaNHeads -> noun NHeadsNHeads -> nounNHeads -> pronounNHeads -> adjectiveNPosts -> NCompNPosts -> NFCNPosts -> NFC NCompNPosts -> RCNPosts -> RC NCompNPosts -> RC NFCNPosts -> RC NFC NCompNPosts -> PrpPNPosts -> PrpP NCompNPosts -> PrpP NFCNPosts -> PrpP NFC NCompNPosts -> PrpP RCNPosts -> PrpP RC NCompNPosts -> PrpP RC NFCNPosts -> PrpP RC NFC NCompNFC -> IngCNFC -> EdCNFC -> InfC// TODO: Noun ComplementVP -> VP crdconj VPVP -> VP NP// Phrasal verbsVP -> VP NP PrpPVP -> VP NP AdvPVP -> VP PrpP NPVP -> VP AdvP NPVP -> VP PrpPVP -> VP AdvP// Standard verbs// TODO: VP -> VAux formsVP -> verb// TODO: Auxilliary verbs// Adjective PhrasesAdjP -> AdjP crdconj AdjPAdjP -> AdvP AdjHead AdjPostmodAdjHead -> adjective AdjPAdjHead -> adjectiveAdjHead -> participle AdjPAdjHead -> participleAdjPostmod -> adverb PrpPAdjPostmod -> adverb InfCAdjPostmod -> adverbAdjPostmod -> PrpPAdjPostmod -> InfC// Adverb Phrases (highly crappy ambiguous production setup here)AdvP -> AdvP adverb adverb AdvCompAdvP -> AdvP adverb adverbAdvP -> AdvP adverb AdvCompAdvP -> AdvP adverbAdvP -> adverb adverb AdvCompAdvP -> adverb adverbAdvP -> adverp AdvCompAdvP -> adverbAdvComp -> PrpP InfCAdvComp -> PrpPAdvComp -> InfC// Prepositional PhrasesPrpP -> preposition PrpCompPrpP -> prepositionPrpComp -> adverbPrpComp -> NPPrpComp -> NP IngCPrpComp -> RC// TODO: reduce ambiguous crap as much as possible

Yeah... And that's incomplete and really horribly unoptimized (parsing with that grammar results in horribly bloated parse graphs).

But that should give you a general idea about how freaking horrible even a partial natural language grammar looks.

Nypyren

Oct 3, 2005 at 11:42 PM

Quote:
Original post by tetelestia
It would have a couple different categories of information in xml databases, for example, an English dictionary, fact dictionary, opinion, user specific dictionary and questions/unknown facts. Here's an example of how a conversation could work...

You probably want to stay away from XML for data storage on this one. You don't want to have to wait an hour for your program to do matching on a single graph just because it's having to parse the XML multiple times because you can't store the knowledge database in RAM, would you?

fur

Oct 4, 2005 at 1:12 PM

I have a system like this I've been developing for the NPC brains in our game. It's similar in concept to the ALICE bots you speak of. It has the same goal you've mentioned of being able to create and save out constructs so it 'learns' over time.

However, I use JSON instead of XML, which is much faster/easier to work with! JSON can do anything XML does. http://www.crockford.com/JSON/index.html

Cheers,
Roy
http://www.p2pmud.com

tetelestia

Oct 4, 2005 at 10:52 PM

@mnangsar - The basic idea of all of your points is that it would have trouble understanding the complex nature of the english language and replying correctly. Have you looked at the ALICE bot? My (not so) new idea would basically expand on that program's base understanding of english. Including abbreviations in the dictionary wouldn't add much space at all, ALICE understands "black lab" and even "lol." It's knowledge base could potentially be enormous...

@Nypren - Yes, XML would be pretty inefficent, it could run at a tolerable speed if it were sorted/categorized enough, still a bit of a problem (maybe it'll just have to wait for quantum computing ;))

@fur - That's pretty cool, how far has it come along? Would you mind letting me see what you've got done?

mnansgar

Oct 5, 2005 at 8:06 AM

ALICE doesn't understand anything. It seems to understand "lol" because the programmers specifically told it to look out for that word and how to respond. Otherwise, ALICE is just a clever scheme which looks for simple sentence structures and responds with preformed sentences. The AIML code is basically a list of templates to watch out for. It would be very difficult for the computer to generalize random English sentences which are spoken to it when if you look around, very few sentences have the EXACT same structure (and those which do are probably already coded in!).

Vampus

Oct 7, 2005 at 11:50 PM

One problem I came across when contemplating this was that it’s one thing to have the program understand a “perfect sentence”, but the problem is, people don’t use perfect sentences, and even fewer use them while typing!

Rhalin

Oct 8, 2005 at 8:11 PM

As far as not understanding imperfect grammar, spelling, and even abbrieviations, is there any reason that the bot/ai can't do what a human could do when he/she doesn't understand something?

That is, why can't you make it ask questions about what it doesn't understand, based on what it _can_ understand about the context and structure of the sentance? Granted, I understand some sentances are completely unrecoverable. ;)

Still, allowing it to ask a series of questions might help (and letting it just give up eventually could be an option). Also, for abbrieviantions, you are left with the option of scanning the dictionary for the abbrieviated part, and finding options that may match, and asking more specific questions. "Lab? Are you talking about a dog, or a place where you blow things up?"

Granted, these options may not be _fast_, but I hope this helps you out somehow!

Oct 9, 2005 at 12:48 AM

I think the AI will have to form contexts about each person/thing based on its interaction with it just like we do. We're not born with extended vocabulary nor meaning of words. That comes later when we attach names to contexts/things. Because of this I believe that our brain is large association network. Say you have a picture frame and things in it. Then those things are associated with other things with other picture frames and who knows how else it's all connected. I don't know whether the logic comes from the frames or if this is something that uses/consumes the frames then making new ones with solutions in them. I also think that our subconciousness is part of this process because when I problem solve, I come up with solution without coming up with steps necessary for solving it. The solution just pops up in my head sometimes and I scrounge or try to trace back things I though about and how they lead me to the solution. I do know that past experience plays into that and perhaps it was stored as frames that were referenced during problem solving process. New frames created after problem was solved. The frames came to me when I was thinking of how early humans used tools. Perhaps they saw a big rock rolling down hill and crushing an animal. That would be stored as a frame in memory. In that frame we have a rock, crushing part, animal, etc. So next time we see an animal in another frame we can back reference to the rock+crushing and figure out that we can use tools to crush a new animal. Though nothing is perfect and if it's a turtle then it will be hard to crush. Thus we make a new frame, remembering to put in body armor and "hard to crush" scenario. So after zillion of these frames it would be easy to ping pong between them to formulate some kind of solution or step forward, though it might be a wrong one. Nothing says we are perfect all the time.

Topic Locked

This topic has been locked by a moderator. New replies are not allowed.