Jump to content
  • Advertisement
Sign in to follow this  
Idov

MLP + overfitting

This topic is 4095 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

Hi! I'm using MATLAB to simulate MLP. so i'm using a validation set, but when i draw the error graph, I see the the error of the validation set is going down and then flattens. I DO want it to go up after a while... but it doesn't. Anybdoy knows why it can happen? Do you have a link or code example? thanks!

Share this post


Link to post
Share on other sites
Advertisement
That can happen because the model doesn't have enough parameters to allow overfitting. You'll probably see the effect you expect if you add more neurons to the hidden layer(s).

Share this post


Link to post
Share on other sites
Just a question here - why do you want it to exhibit overtraining? Isn't overtraining a bad thing?

Share this post


Link to post
Share on other sites
It is, but i need it for an exercise i'm doing.
plus it helps me know when I should training the network :)

Share this post


Link to post
Share on other sites
Have you tried a lower learning rate? You may be pushing the training to the nearest local minimum. You might see what you're looking for if you decrease the learning rate and run for a higher number of iterations. Also keep the number of hidden nodes on the high side and you should see a badly overtrained network.

Seems weird to offer suggestions on how to overtrain. 8^O

Share this post


Link to post
Share on other sites
Ok, thanks! It goes up a little bit.

But I have another question...
I'm tryinf to reduce the variance error by adding more "experts" but as I add more and more of them the variance doesn't go down and even worse - the validation set error GOES UP! (it's not going down and goes up, it's more like a high straight line)

It's very strange because i'm pretty sure i calculated the variance well.

can you tell me why?

Share this post


Link to post
Share on other sites
When you say "adding more experts" do you mean that you are developing an ensemble model? This would involved training multiple, independent models (neural nets in your case) and combining the results. Is this correct?

-Kirk

Share this post


Link to post
Share on other sites
quote]Original post by kirkd
When you say "adding more experts" do you mean that you are developing an ensemble model? This would involved training multiple, independent models (neural nets in your case) and combining the results. Is this correct?

-Kirk[/quote

yes :)

[Edited by - Idov on September 27, 2007 2:33:16 PM]

Share this post


Link to post
Share on other sites
Sign in to follow this  

  • Advertisement
×

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

GameDev.net is your game development community. Create an account for your GameDev Portfolio and participate in the largest developer community in the games industry.

Sign me up!