Jump to content
  • Advertisement
Sign in to follow this  
chadjohnson

Biological vs. Artificial Neural Networks

This topic is 4702 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

If anyone has any ideas about this, PLEASE post! And please correct me wherever I may be wrong. I've looking into the way biological neural networks function, and I'm starting to think that the current ANN models are not as accurate as they should be. First of all I think the ART (adaptive resonance theory) network is a good representation of memory storage and recall. It maps an input vector to an output vector. In this model, each neuron in the storage layer contains a prototype vector. I personally think a memory is an output vector obtained though the propagation of a set of input signals and the connection weights between neurons (and chemicals also take part in this). I think that the storage of the prototype vectors in the ART is incorrect. Rather than storing a separate vector in each neuron, these vectors should each be represented by sets of neurons. So basically the prototype vector is distributed across multiple neurons rather than being local to a single neuron. Second, I've been thinking about the way neurons are represented. In most models a network of neurons is represented by a graph, and each neuron is just another instantiation of an object. With these models, neurons do not operate simultaneously. The program must loop through each individual neuron, calculating their activation levels one by one. My idea (maybe this has been done before) is to make a threaded neural network, where each neuron is a separate thread. That way, a neuron could process incoming signals and fire WHEN it receives them rather than when the program gets to it. What do you think?

Share this post


Link to post
Share on other sites
Advertisement
Quote:
Original post by chadjohnson
My idea (maybe this has been done before) is to make a threaded neural network, where each neuron is a separate thread. That way, a neuron could process incoming signals and fire WHEN it receives them rather than when the program gets to it.


And what would be the benefit of doing it like this? I'm asking sincerely.

Share this post


Link to post
Share on other sites
ANNs are not entirely meant to model biological neural networks. Modern mathematical biological neuron modelling is far too complex to be carried out in real-time anyways, so I wouldn't worry about it.

Look here: http://www.neuron.yale.edu/neuron/
That's one of the standard neuron modelling software applications.

Share this post


Link to post
Share on other sites
Quote:
Original post by chadjohnson
My idea (maybe this has been done before) is to make a threaded neural network, where each neuron is a separate thread. That way, a neuron could process incoming signals and fire WHEN it receives them rather than when the program gets to it.


What would be the purpose of this? On single processor machine, it wouldn't be true multitasking so it would still have to "wait" until execution reaches it. Also on most threading systems, it is difficult or impossible to know ahead of time which thread will get execution when, which might complicate things some. It would probably be easier (and possibly more efficient) just to do it the normal way.

Share this post


Link to post
Share on other sites
First, neural networks, while inspired by biological neurons, have very little to do with them. Most artifical neural networks are some sort of algorithm related to statistical pattern recognition. Forget the biology connection...I don't think it's particularly helpful. Artificial neural networks aren't meant for modeling the brain..they're underpowered for the task, and our understanding of the brain doesn't allow for anything but the most superficial models.

Now, regarding threading vs iterative simulation. On a single processor machine, it turns out to be the same thing. With threading you're just letting the OS do the work of pretending things happen simultaneously. This is a problem in all types of simulations. You can't have a true dynamic system on a single processor...now with multiple processors this might be possible, but you'd require a processor per neuron.

-Alex



Share this post


Link to post
Share on other sites
I really can't say what the benefit of a multi-threaded ANN would be. It was just a thought, but you guys are right. It would take too much processing power, and synchronization would be difficult. There probably wouldn't be much benefit. And also, human brains have billions of neurons, so essentially they have billions of processors. Most animals don't have nearly that many. So I guess the only way we could truly model a human brain would be to use a quantum computer. Imagine the speed...biological neurons (from what I've read) are much slower than computer processors. Such a system would be thousands of times faster than a human brain.

I know that ANNs aren't entirely meant to model biological neural networks. But I've had this idea floating around. I read on Slashdot that some students came up with a method to allow a computer to infer grammatical structure. Since then I've been thinking: how could you design a system where the computer could "read" the text of a story and correctly answer questions that a user might ask? For instance: "Jack got into his truck. He put on his seatbelt, and he drove away." If a user asked the question, "What was Jack driving?" the computer would be unable to answer with "a truck" unless it knew that a truck was able to be driven. Using current natural language processing techniques, there is no way (as far as I know) to have such a system.

To have such a system would require a knowledge base that uses associative memories. The computer would need to be able to model real life concepts (e.g., a truck and its capabilities), and also need to understand context. I believe this could be done using an expert system that incorporates an associative memory model.

Do you think an ART-type network would work for this? And what do you think about the prototype vector being distributed across multiple neurons rather than being local to a single neuron? I think that memory = synaptic weights, and memory recall = a set of outputs via calculations of signals and connection weights. In an ART, a new neuron (which includes a prototype vector) is created for each new concept. I believe that a memory exists across multiple neurons instead of pointing to just one. This would allow you to manifest a memory which is very similar to another (almost identical), except that it contains a little bit different information (our minds do this). I mean there's just no way that an ART network could account for all the memories that would be necessary in such a system. And biologically, I would argue that a single memory does not constitute memory. A memory consists of multiple neurons, and it can change slightly just by incorporating more neurons.

The main reason that I argue this is this: if you have an input vector (biologically speaking) and you change one of the values in that vector just slightly, it is going to change how the signal propagates through the network. The slightest change could in fact make the signal become just strong enough to exceed yet another neuron's threshold value and make it fire (when that neuron would not have fired before). This would in turn make neuron's output (and possibly more) also be included in the final output vector (when it was not before).

What do you think? Please let me know.

I understand ANNs aren't meant to model biological networks, but just go with me on this. I think I might have something here.

[Edited by - chadjohnson on January 1, 2006 2:26:09 AM]

Share this post


Link to post
Share on other sites
Guest Anonymous Poster
ANNs are not very much like real neurons at all. If you think multiple artificial neurons are needed to store a memory, go ahead and devise or use such a system. I think there are some types of ANNs that already do this.

One of the advantages of ANNs for some tasks is that they are highly modular so hardware implementations that can propagate signals in parallel are possible. But for software, there would be too much overhead to make it worthwhile, even with multiple processors. Larger networks could be split across multiple processors.

Share this post


Link to post
Share on other sites
Yes the human brain has billions of neurons indeed. One idea I had was to look at mathematical models for biological neurons, and design a chip per neuron. I doubt that a neuron needs the full capabilities of a modern processor, and a small chip should do the trick. Just plugging these chips into each other could create an artificial neural network. Just an idea...

Share this post


Link to post
Share on other sites
Sign in to follow this  

  • Advertisement
×

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

We are the game development community.

Whether you are an indie, hobbyist, AAA developer, or just trying to learn, GameDev.net is the place for you to learn, share, and connect with the games industry. Learn more About Us or sign up!

Sign me up!