I just completed two quarters of Dr. Hecht-Nielsen's course, and he recently went public with his new theory for cognition. You can see the release here. The technical paper abstract is here. Doesn't look like the paper is freely available. The first link has a link to the video of his lecture where he introduced the paper. Using his methods, I was able to build a pretty simple machine that would reasonably fill-in the fifth word of a sentence given the first four words. This machine had no knowledge of grammar, and did not do a search of learned text--some of the sentences filled-in were completely novel.

I guess I'll preface this by saying that I am always completely skeptical (and thereby mildy argumentative) whenever anyone says "this is how <insert brain behavior here> works".

I think that reads as a pretty cool theory. It seems to amount to: "brains think by pattern recognition", right? I mean essentially the article describes his theory as saying that the brain thinks through pattern completion and not through an algorithmic processing of the possible solutions. Is this really new? Fundimentally that's how fully connected neural networks operate, and I thought that was the general concensus on how high level memory and other cortical processes work. Essentially certain brain patterns trigger other brain patterns to complete, ad infinitum. There is no "algorithmic processing" of the possible results, but rather the network settles into the local holding pattern that is closest to the inputs received. The process of remembering has been suggested to being something akin to a spontaneous reconstruction of a memory triggered by present patterns that resemble the patterns laid down during the initial experience. Basically think of a memory as a pattern of neural activation, if the current experience of the organism happens to create a pattern of activation that resembles the pattern of the memory, then the memory pattern will be triggered b/c of the bias in the neural weighting.

I guess I just missed the part where people thought that the brain operated by looking at a possible solution set and running some kind of probablility test on the results. Or have I missed the point entirely and is this article about a revolution in the software simulation of cognition and not about a revolution in the theory of how the brain works?

-me

[Edited by - Palidine on March 17, 2005 12:32:56 PM]

Quote:Original post by Palidine
I guess I'll preface this by saying that I am always completely skeptical (and thereby mildy argumentative) whenever anyone says "this is how <insert brain behavior here> works".

Very healthy. I too remain skeptical.

Quote:
I think that reads as a pretty cool theory. It seems to amount to: "brains think by pattern recognition", right? I mean essentially the article describes his theory as saying that the brain thinks through pattern completion and not through an algorithmic processing of the possible solutions. Is this really new?

The novelty of the theory, from what I understand, is that he presents a way for complicated pattern recognition to be broken down into a massive set of co-occurrences. Only two things are checked against each other at a time. Neurons can't do much--they fire or don't fire, and fire with a certain strength.

From what he said, most of the current AI field believes the brain works to solve the problem p(e|abcd). That is, if I know the current facts a,b,c and d, what's the most probable event e? And this is devolved into smaller problems using Bayesian analysis.

His approach is the opposite. What's the probability of observing facts a,b,c and d given event e? p(abcd|e). He calls this the cogency. The event e that maximizes this for known facts abcd is the one our brain "picks".

His justification for this is that p(abcd|e) is sort-of related to (not even proportional to--he goes into great depth in his paper): p(a|e)p(b|e)p(c|e)p(d|e). In other words, the product of the co-occurrences of each fact and event. A machine that would do this would--in parallel--look up all the pairwise co-occurrences and formulate the product (or sum the log). This is a process he calls confabulation. His argument is that the brain is such a machine--there's evidence that neurons do this sort of thing, and they can't do much else so this must be the way it works.

Quote:
Fundimentally that's how fully connected neural networks operate, and I thought that was the general concensus on how high level memory and other cortical processes work. Essentially certain brain patterns trigger other brain patterns to complete, ad infinitum. There is no "algorithmic processing" of the possible results, but rather the network settles into the local holding pattern that is closest to the inputs received.

Yes, I think that's right, although I don't know about neural networks. His main complaint is that "people who do AI don't pay any attention to neurobiology". So even though that's the consensus on how the brain works, that's not how neural nets work. His words, not mine; I'm not knowledgable enough on the subject.

Remember, this isn't some neural-net know-nothing. He was a little modest in class about his accomplishments, but his company (HNC) was acquired by Fair Isaacs, he's currently a VP of R&D there, and he's credited with creating the first neurocomputer. He's a pioneer in the field.

That said, he makes extraordinary claims that require extraordinary proof. I was personally impressed with the results, but I am new to the field.

Quote:
I guess I just missed the part where people thought that the brain operated by looking at a possible solution set and running some kind of probablility test on the results. Or have I missed the point entirely and is this article about a revolution in the software simulation of cognition and not about a revolution in the theory of how the brain works?
-me

Right--his complaint isn't about neuroscience, but that neurcomputing pays no attention to neuroscience. Apparently most traditional AI work by some means of Bayesian application of probabilities to the problem set, and he thinks this is absolutely backward, but nobody's come up with a way to do it "right" until now. He claims that cogent confabulation is the right way.

Quote:Original post by Stoffel
His main complaint is that "people who do AI don't pay any attention to neurobiology".

I'm curious as to his source of information on this matter. I for one am a person trained in AI and who has worked in neuroscience research and I can tell you that there are hundreds, nay thousands of researchers out there who regularly broach this division of the sciences on a daily basis.

What he apparently fails to recognise - or refuses to admit - is that most people who work on AI aren't trying to mimic brain function, but are rather trying to produce rational and/or logical behaviour by artificial means. It sounds to me like he's doing a bit of a beat-up of the AI community to promote the importance/novelty of his own work.

Personally, I've seen many 'models' of cognition proposed. As yet, none has solidified itself as an accepted theory by more than a handful of researchers. I guess we'll wait and see what the jury verdict on this one is.

Cheers,

Timkin

Quote:Original post by Timkin

Quote:Original post by Stoffel
His main complaint is that "people who do AI don't pay any attention to neurobiology".

I'm curious as to his source of information on this matter. I for one am a person trained in AI and who has worked in neuroscience research and I can tell you that there are hundreds, nay thousands of researchers out there who regularly broach this division of the sciences on a daily basis.

Personal experience, it sounded like. I didn't subscribe to his "me against the world"-isms--I chalked it up to his eccentricity.

Quote:
What he apparently fails to recognise - or refuses to admit - is that most people who work on AI aren't trying to mimic brain function, but are rather trying to produce rational and/or logical behaviour by artificial means. It sounds to me like he's doing a bit of a beat-up of the AI community to promote the importance/novelty of his own work.

Could very well be.

Quote:
Personally, I've seen many 'models' of cognition proposed. As yet, none has solidified itself as an accepted theory by more than a handful of researchers. I guess we'll wait and see what the jury verdict on this one is.

Agreed. This is the very first glimpse of this theory, so it's very early in the game. His approach seems to be, "It sounds crazy because it's different than anything you've ever seen, but it works and here's how you do it." I know of a number of people who are already digging into it, so I think we'll see if the proof is in the pudding within the next 5 years.

World that HAS come ...

www.imagination-engines.com aka IEI

Dr. Stephen Thaler of Imagination Engines Inc. seems to have clearly established these "new" concepts in several "concrete" ways over quite a long span of time. I am perplexed as to why it becomes "news" when a "high profile" individual suddenly "repackages" another company's established methods.

In the context of Imagination Engines Inc. and Dr. Thaler, I must say "established" due to the fact that, not only does IEI have historical references to these concepts, going back many years, in addition, the company holds patents (>20 in total) relating to these (and other) advanced AI/neural network methodologies.

Ultimately IEI and Dr. Thaler can point to "concrete customers" and their successful applications as capability going beyond simply theoretical postulation. One has only to peruse IEI's web site to confirm this case, which I would assume would be the minimal, fair "due diligence" that should be afforded Dr. Thaler.

I believe it would be in everyone's best interest and in the interest of fairness to give credit to those that clearly have established a case for "credit where credit is due"!

Perhaps a corresponding article pertaining to IEI technology could be presented citing specific cases with very pragmatic details as to how these "confabulation-like" methods are already working, would be of great interest to your group. (I know the "ultra-pragmatic" tend to "lurk" here). Not just vague theories but actual "case histories" of real world application of these techniques!

In fact, unapologetically, Dr. Thaler would certainly state that "human level" computer based intelligence has been demonstrably "achievable" for some time now. (BTW it might be a "good thing" to ask how does a guy get the "balls" to make such an apparently absurd claim - it's the "been there, done that" substantiation with REAL customers).

There are well established groups of companies and government entities that have and will continue to successfully apply and reap the benefits of this technology. These companies stand as evidence that these concepts are "alive and well" and poised to seed incredible changes in the synthetic intelligence landscape of the present and foreseeable future of technological achievement!

Please assist in propagating the appropriate source and proper recognition of this technology!

Quote:Original post by dmvieau
Perhaps a corresponding article pertaining to IEI technology could be presented citing specific cases with very pragmatic details as to how these "confabulation-like" methods are already working, would be of great interest to your group. (I know the "ultra-pragmatic" tend to "lurk" here). Not just vague theories but actual "case histories" of real world application of these techniques!

By all means, please present your article--right here would be fine, or just post a new topic. If you'd like to post your methods (they're patented so there's no IP involved, correct?), please do so.

Thanks for the comments, mnansgar. I agree with your assessment of his presentation style, and it's something that was off-putting to me during class as well.

As for research, he has been mostly in industry--he's an adjunct at UCSD, meaning it's not a pure research paper. He founded a company (HNC) that was acquired by Fair Isaacs where he employs neurocomputing to do all sorts of secret things (probably to figure out your credit risk). He is generally credited with creating the first neurocomputer. So though he's been tangentially involved in academia, he's not a pure researcher. I didn't realize he was so poorly published.

Thanks for your comments--nice to hear from someone who tried it out.