Jump to content
  • Advertisement
Sign in to follow this  
Geometrian

Deleting a Large "list",

This topic is 2646 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

Hi,

I've got a very large list of pointers. The pointers point to new-d objects. I need to free all the objects and then delete the list (which is itself new-d). My current technique looks like:[source lang="cpp"]for (list<Token*>::iterator iter=_tokens->begin();iter!=_tokens->end();++iter) {
delete (*iter);
}
delete _tokens;[/source]Unfortunately, the "delete _tokens;" line takes a very long time (about ten times longer than loading the data into the list in the first place, doing lexical analysis, and parsing, all put together). Clearly, this is problematic. I have tried "_tokens->clear();" before the delete, but then that line takes forever instead. I have also tried manually removing all the elements one by one with a loop, but then that loop takes too long.

Help? Thanks,
-G

Share this post


Link to post
Share on other sites
Advertisement

Hi,

I've got a very large list of pointers. The pointers point to new-d objects. I need to free all the objects and then delete the list (which is itself new-d). My current technique looks like:[source lang="cpp"]for (list<Token*>::iterator iter=_tokens->begin();iter!=_tokens->end();++iter) {
delete (*iter);
}
delete _tokens;[/source]Unfortunately, the "delete _tokens;" line takes a very long time (about ten times longer than loading the data into the list in the first place, doing lexical analysis, and parsing, all put together). Clearly, this is problematic. I have tried "_tokens->clear();" before the delete, but then that line takes forever instead. I have also tried manually removing all the elements one by one with a loop, but then that loop takes too long.

Help? Thanks,
-G


You have to release that memory, unless you want to leak it. There is no way around it. Your way of doing it seems fine (although I wouldn't use raw pointers in containers): If you want it to be faster then you need to start profiling your code. Some hints:

-You haven't told us what very large means to you
-What is your destructor doing?
-What compiler (gcc/msvc, debug/release, what optimizations), etc...
-Have you tried another data structure? (std::vector comes into mind)
-Have you looked into using memory pools (eg. custom allocators, etc..)?

*edit* You should think about your code before randomly writing code. Calling _tokens->clear() is WRONG. Your list is empty afterwards and therefore the loop doesn't do anything and therefore you are now leaking memory.


Share this post


Link to post
Share on other sites
You have to release that memory, unless you want to leak it. There is no way around it.[/quote]Righto.
-You haven't told us what very large means to you[/quote]193,409 elements.
-What is your destructor doing?
...
-Have you looked into using memory pools?[/quote]This is the STL list class. I'm not sure what its destructor does.
-What compiler (gcc/msvc, debug/release, what optimizations), etc...[/quote]Visual Studio 2010 Ultimate's built-in compiler; full optimizations/release mode.

Thanks,
-G

Share this post


Link to post
Share on other sites
Is there a good reason for using a list instead of a vector?
How long is "a very long time"? Are we talking seconds or microseconds?

Share this post


Link to post
Share on other sites
Try running your application outside of visual studio. When running under the debugger (even in a release build), the memory manager is replaced with a debug one that can be heinously slow on very large numbers of deletes. At least that's what some testing earlier this year seemed to bear out.

Share this post


Link to post
Share on other sites
I actually meant the destructor of your Token class. Calling delete(*iter) first calls the destructor of your Token object and then frees the memory associated with it. Depending on the Token object and the amount of work the destructor must do, it can take some time. But this totally depends. If you want to continue using the list class then I suggest you look into custom allocators and memory pooling. You could advise the pool to reserve a large amount of memory upfront (for example 10mb) and then use that pool instead of the default allocator (which simply uses new for allocating a node within a list).

Another option would be to skip visual studio's memory allocator alltogether and use a better one: http://www.hoard.org/ We are using hoard at work and it has proven to be several times faster than the one visual studio provides.

Share this post


Link to post
Share on other sites
I second the suggestion that you should consider std::vector. std::list<>'s theoretical advantages are usually more than offset by the performance implications of marching randomly across memory, and the many allocation/deallocation calls it makes.

Share this post


Link to post
Share on other sites
Sign in to follow this  

  • Advertisement
×

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

GameDev.net is your game development community. Create an account for your GameDev Portfolio and participate in the largest developer community in the games industry.

Sign me up!