🔒 Locked

Supervised Neural Network Adaptation Problem

Started by 00702000744 Mar 14, 2004 at 1:46 PM 2 replies 900+ views

Original Post

00702000744

Mar 14, 2004 at 1:46 PM

Hi: I have been reading on SNN recently and getting the big picture of it. It seems that SNN is suitable for pattern association. I am sure that as SNN keeps adjusting its weight values in effort of matching inputs to output, the end result of that particular pattern association will be quite accurate. My question is how well will the old patterns be recognized. Say if for the ith time, SNN has weight values such that inputs A, B, C map perfectly to X; now, for the (i+1)th time, we want to map inputs A,B,D to output Y. That new pattern could greatly alter the existing weight values. So, although SNN can recognize (i+1)th pattern perfectly this time, it may likely forget ith pattern as well. Is this a real problem in SNN or is there something I misunderstood, or there is common way to get around it? Any suggestion will be greatly appreciated! Thanks for help in advance

joanusdmentia

Mar 14, 2004 at 3:39 PM

While someone more knowledgable on the subject will probably have to answer, I think it depends greatly on the back propogation algorithm you use and the layout of the network. I''d be very surprised if the neural net forgot the ith pattern when it learnt the (i+1)th pattern, but would be less surprised if it forgot the ith pattern when it learnt the (i+10)th for example.

Predictor

Mar 15, 2004 at 4:40 AM

quote:
Original post by 00702000744
It seems that SNN is suitable for pattern association. I am sure that as SNN keeps adjusting its weight values in effort of matching inputs to output, the end result of that particular pattern association will be quite accurate. My question is how well will the old patterns be recognized. Say if for the ith time, SNN has weight values such that inputs A, B, C map perfectly to X; now, for the (i+1)th time, we want to map inputs A,B,D to output Y. That new pattern could greatly alter the existing weight values. So, although SNN can recognize (i+1)th pattern perfectly this time, it may likely forget ith pattern as well. Is this a real problem in SNN or is there something I misunderstood, or there is common way to get around it? Any suggestion will be greatly appreciated!

The more instances there are of cases with similar inputs and different outputs, the harder the data will be to learn. If the neural network is updated incrementally, after each individual examples is stored, the "forgetting" you describe may become a problem. This is why it is a good idea to some how mix up the order of the examples and why neural networks are often trained in "batch", meaning after all examples have been seen.

-Predictor
http://will.dwinnell.com

stevie56

Mar 15, 2004 at 1:51 PM

When using a neural net to recognise patterns (like OCR), it is most unusual to present a single pattern, backpropagate the error and then present a new pattern and backpropagate again.

As has been stated already, the network is presented with a batch of input-to-output patterns and the total error over the whole training sample is backpropagated.

It is also recommended that on each iteration in the training, the patterns be shuffled, to remove any bias that might happen if the early samples bear more heavily on the errors.

On the occasions when we want to introduce a new pattern to a semi-trained network, we never present the pattern singly, but include it inside the batch and iterate again until the total error is within acceptable limits.

Stevie

Don''t follow me, I''m lost.

Topic Locked

This topic has been locked by a moderator. New replies are not allowed.