Jump to content

  • Log In with Google      Sign In   
  • Create Account


Deleting a Large "list",


Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.

  • You cannot reply to this topic
6 replies to this topic

#1 Geometrian   Crossbones+   -  Reputation: 1495

Like
0Likes
Like

Posted 21 September 2011 - 12:26 AM

Hi,

I've got a very large list of pointers. The pointers point to new-d objects. I need to free all the objects and then delete the list (which is itself new-d). My current technique looks like:[source lang="cpp"]for (list<Token*>::iterator iter=_tokens->begin();iter!=_tokens->end();++iter) { delete (*iter);}delete _tokens;[/source]Unfortunately, the "delete _tokens;" line takes a very long time (about ten times longer than loading the data into the list in the first place, doing lexical analysis, and parsing, all put together). Clearly, this is problematic. I have tried "_tokens->clear();" before the delete, but then that line takes forever instead. I have also tried manually removing all the elements one by one with a loop, but then that loop takes too long.

Help? Thanks,
-G
And a Unix user said rm -rf *.* and all was null and void...|There's no place like 127.0.0.1|The Application "Programmer" has unexpectedly quit. An error of type A.M. has occurred.

Sponsor:

#2 SiS-Shadowman   Members   -  Reputation: 359

Like
0Likes
Like

Posted 21 September 2011 - 12:34 AM

Hi,

I've got a very large list of pointers. The pointers point to new-d objects. I need to free all the objects and then delete the list (which is itself new-d). My current technique looks like:[source lang="cpp"]for (list<Token*>::iterator iter=_tokens->begin();iter!=_tokens->end();++iter) { delete (*iter);}delete _tokens;[/source]Unfortunately, the "delete _tokens;" line takes a very long time (about ten times longer than loading the data into the list in the first place, doing lexical analysis, and parsing, all put together). Clearly, this is problematic. I have tried "_tokens->clear();" before the delete, but then that line takes forever instead. I have also tried manually removing all the elements one by one with a loop, but then that loop takes too long.

Help? Thanks,
-G


You have to release that memory, unless you want to leak it. There is no way around it. Your way of doing it seems fine (although I wouldn't use raw pointers in containers): If you want it to be faster then you need to start profiling your code. Some hints:

-You haven't told us what very large means to you
-What is your destructor doing?
-What compiler (gcc/msvc, debug/release, what optimizations), etc...
-Have you tried another data structure? (std::vector comes into mind)
-Have you looked into using memory pools (eg. custom allocators, etc..)?

*edit* You should think about your code before randomly writing code. Calling _tokens->clear() is WRONG. Your list is empty afterwards and therefore the loop doesn't do anything and therefore you are now leaking memory.




#3 Geometrian   Crossbones+   -  Reputation: 1495

Like
0Likes
Like

Posted 21 September 2011 - 12:40 AM

You have to release that memory, unless you want to leak it. There is no way around it.

Righto.

-You haven't told us what very large means to you

193,409 elements.

-What is your destructor doing?
...
-Have you looked into using memory pools?

This is the STL list class. I'm not sure what its destructor does.

-What compiler (gcc/msvc, debug/release, what optimizations), etc...

Visual Studio 2010 Ultimate's built-in compiler; full optimizations/release mode.

Thanks,
-G
And a Unix user said rm -rf *.* and all was null and void...|There's no place like 127.0.0.1|The Application "Programmer" has unexpectedly quit. An error of type A.M. has occurred.

#4 Hodgman   Moderators   -  Reputation: 29493

Like
0Likes
Like

Posted 21 September 2011 - 01:13 AM

Is there a good reason for using a list instead of a vector?
How long is "a very long time"? Are we talking seconds or microseconds?

#5 latent   Members   -  Reputation: 139

Like
0Likes
Like

Posted 21 September 2011 - 01:20 AM

Try running your application outside of visual studio. When running under the debugger (even in a release build), the memory manager is replaced with a debug one that can be heinously slow on very large numbers of deletes. At least that's what some testing earlier this year seemed to bear out.

#6 SiS-Shadowman   Members   -  Reputation: 359

Like
0Likes
Like

Posted 21 September 2011 - 01:25 AM

I actually meant the destructor of your Token class. Calling delete(*iter) first calls the destructor of your Token object and then frees the memory associated with it. Depending on the Token object and the amount of work the destructor must do, it can take some time. But this totally depends. If you want to continue using the list class then I suggest you look into custom allocators and memory pooling. You could advise the pool to reserve a large amount of memory upfront (for example 10mb) and then use that pool instead of the default allocator (which simply uses new for allocating a node within a list).

Another option would be to skip visual studio's memory allocator alltogether and use a better one: http://www.hoard.org/ We are using hoard at work and it has proven to be several times faster than the one visual studio provides.



#7 rip-off   Moderators   -  Reputation: 8115

Like
0Likes
Like

Posted 21 September 2011 - 02:48 AM

I second the suggestion that you should consider std::vector. std::list<>'s theoretical advantages are usually more than offset by the performance implications of marching randomly across memory, and the many allocation/deallocation calls it makes.




Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.



PARTNERS