Biological vs. Artificial Neural Networks

Quote:

Original post by chadjohnson My idea (maybe this has been done before) is to make a threaded neural network, where each neuron is a separate thread. That way, a neuron could process incoming signals and fire WHEN it receives them rather than when the program gets to it.

And what would be the benefit of doing it like this? I'm asking sincerely.

ANNs are not entirely meant to model biological neural networks. Modern mathematical biological neuron modelling is far too complex to be carried out in real-time anyways, so I wouldn't worry about it.

Look here: http://www.neuron.yale.edu/neuron/
That's one of the standard neuron modelling software applications.

Quote:

Original post by chadjohnson My idea (maybe this has been done before) is to make a threaded neural network, where each neuron is a separate thread. That way, a neuron could process incoming signals and fire WHEN it receives them rather than when the program gets to it.

What would be the purpose of this? On single processor machine, it wouldn't be true multitasking so it would still have to "wait" until execution reaches it. Also on most threading systems, it is difficult or impossible to know ahead of time which thread will get execution when, which might complicate things some. It would probably be easier (and possibly more efficient) just to do it the normal way.

First, neural networks, while inspired by biological neurons, have very little to do with them. Most artifical neural networks are some sort of algorithm related to statistical pattern recognition. Forget the biology connection...I don't think it's particularly helpful. Artificial neural networks aren't meant for modeling the brain..they're underpowered for the task, and our understanding of the brain doesn't allow for anything but the most superficial models.

Now, regarding threading vs iterative simulation. On a single processor machine, it turns out to be the same thing. With threading you're just letting the OS do the work of pretending things happen simultaneously. This is a problem in all types of simulations. You can't have a true dynamic system on a single processor...now with multiple processors this might be possible, but you'd require a processor per neuron.

-Alex

I really can't say what the benefit of a multi-threaded ANN would be. It was just a thought, but you guys are right. It would take too much processing power, and synchronization would be difficult. There probably wouldn't be much benefit. And also, human brains have billions of neurons, so essentially they have billions of processors. Most animals don't have nearly that many. So I guess the only way we could truly model a human brain would be to use a quantum computer. Imagine the speed...biological neurons (from what I've read) are much slower than computer processors. Such a system would be thousands of times faster than a human brain.

I know that ANNs aren't entirely meant to model biological neural networks. But I've had this idea floating around. I read on Slashdot that some students came up with a method to allow a computer to infer grammatical structure. Since then I've been thinking: how could you design a system where the computer could "read" the text of a story and correctly answer questions that a user might ask? For instance: "Jack got into his truck. He put on his seatbelt, and he drove away." If a user asked the question, "What was Jack driving?" the computer would be unable to answer with "a truck" unless it knew that a truck was able to be driven. Using current natural language processing techniques, there is no way (as far as I know) to have such a system.

To have such a system would require a knowledge base that uses associative memories. The computer would need to be able to model real life concepts (e.g., a truck and its capabilities), and also need to understand context. I believe this could be done using an expert system that incorporates an associative memory model.

Do you think an ART-type network would work for this? And what do you think about the prototype vector being distributed across multiple neurons rather than being local to a single neuron? I think that memory = synaptic weights, and memory recall = a set of outputs via calculations of signals and connection weights. In an ART, a new neuron (which includes a prototype vector) is created for each new concept. I believe that a memory exists across multiple neurons instead of pointing to just one. This would allow you to manifest a memory which is very similar to another (almost identical), except that it contains a little bit different information (our minds do this). I mean there's just no way that an ART network could account for all the memories that would be necessary in such a system. And biologically, I would argue that a single memory does not constitute memory. A memory consists of multiple neurons, and it can change slightly just by incorporating more neurons.

The main reason that I argue this is this: if you have an input vector (biologically speaking) and you change one of the values in that vector just slightly, it is going to change how the signal propagates through the network. The slightest change could in fact make the signal become just strong enough to exceed yet another neuron's threshold value and make it fire (when that neuron would not have fired before). This would in turn make neuron's output (and possibly more) also be included in the final output vector (when it was not before).

What do you think? Please let me know.

I understand ANNs aren't meant to model biological networks, but just go with me on this. I think I might have something here.

[Edited by - chadjohnson on January 1, 2006 2:26:09 AM]

Yes the human brain has billions of neurons indeed. One idea I had was to look at mathematical models for biological neurons, and design a chip per neuron. I doubt that a neuron needs the full capabilities of a modern processor, and a small chip should do the trick. Just plugging these chips into each other could create an artificial neural network. Just an idea...