Jump to content
  • Advertisement
Sign in to follow this  
sidhantdash

Training Error vs Testing Error

This topic is 4437 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

Hi all, I have trained three different kinds of networks for a Time series problem, Real Time Recurrent Learning (RTRL), 1 hidden layer Feedforward Network, 1 Hidden layer Elman Network. Upon training these nets I find that after the training process, the MSE comes out to be in the range of 1e-5~1e-4. But no matter how much I train, and irrespective of the order of presentation of the inputs (it is randomized), the error on the test data is kind of stagnant at abt 1e-3. Sometimes when I train my RTRL longer, the error on the test data actually increases (as vs lesser training). Is this because of the weights getting stuck at some local minima? I may also add that the data I am using (currency exchange rate data) doesnt really seem to have any underlying pattern (this is what I gathered from a plot of the data). Also, the data has *not* been preprocessed. My particular investigation is really abt Neural net structure, and preprocessing the data might bias my findings. So I havent done anything abt it. What do you recommend? thanks Sidhant

Share this post


Link to post
Share on other sites
Advertisement
If you're noticing performance degrade as the length of training increases, then that suggests your network is over-fitting the training data.

As for the RMSE of your test data, you should expect it to be worse than your training data as a function of the cumulative distance between each training set datum and test set datum. That is, you don't expect the network to represent the data perfectly (particularly in your domain where there is a significant element of noise in the data), so you expect that it will not perform as well on data it has not seen before.

Since you're dealing with time series data, you might consider a hybrid approach whereby you pre-filter the data and the filter parameters are chosen so as to minimise the RMS of the test set.

Cheers,

Timkin

Share this post


Link to post
Share on other sites
Sign in to follow this  

  • Advertisement
×

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

We are the game development community.

Whether you are an indie, hobbyist, AAA developer, or just trying to learn, GameDev.net is the place for you to learn, share, and connect with the games industry. Learn more About Us or sign up!

Sign me up!