Jump to content

  • Log In with Google      Sign In   
  • Create Account

We're offering banner ads on our site from just $5!

1. Details HERE. 2. GDNet+ Subscriptions HERE. 3. Ad upload HERE.


TinyXML++ memory leaks


Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.

  • You cannot reply to this topic
20 replies to this topic

#1 leelouch   Members   -  Reputation: 122

Like
0Likes
Like

Posted 24 March 2009 - 07:13 AM

Hi I have huge memory leak issue using tinyxml with this code : any idea please ?
ticpp::Document   xml;
ticpp::Node     * _top;

filename="myfile";


xml.LoadFile(filename);
_top = xml.FirstChild(false);

ticpp::Element * _atomsib (std::string name) {
	for (ticpp::Element* pElem = _top->FirstChildElement(); pElem; ) {
		if (pElem->Value() == name) {
			return pElem;
		}
		ticpp::Element * tmp = pElem;
		pElem= pElem->NextSiblingElement("atom");
		tmp->Clear();
		delete tmp;
	};
	return 0;
}

while (1) {
	ticpp::Element * n;
	n = _atomsib("socket");
	n->Clear();
	delete n;
};





[Edited by - leelouch on March 24, 2009 7:18:32 PM]

Sponsor:

#2 NotAYakk   Members   -  Reputation: 876

Like
0Likes
Like

Posted 24 March 2009 - 08:12 AM

delete pElem;
...
pElem = (ticpp::Element*) pElem->NextSiblingElement("atom")

I see a problem. And it isn't a memory leak.

#3 zedz   Members   -  Reputation: 291

Like
0Likes
Like

Posted 24 March 2009 - 12:13 PM

I dont know about the above code, but tinyxml did used to leak years ago,
do u have the latest version?

#4 leelouch   Members   -  Reputation: 122

Like
0Likes
Like

Posted 24 March 2009 - 01:07 PM

Quote:
Original post by NotAYakk
delete pElem;
...
pElem = (ticpp::Element*) pElem->NextSiblingElement("atom")

I see a problem. And it isn't a memory leak.


Yes of course, this is a problem, I figured out that, and even with the fix, it does not change the behaviour !!
maybe I missed something on what I have to do to free the memory allocated by the iterator, other than the Clear() method and delete on Element * ???

and Yes I have the last version .

What is the correct code to search an element in a xml hierarchy, without memory leak ??

Thanks for help.

#5 Codeka   Members   -  Reputation: 1157

Like
0Likes
Like

Posted 24 March 2009 - 02:07 PM

You don't have to do anything to free memory. When the XML document itself is unloaded, all elements are freed as well.

#6 leelouch   Members   -  Reputation: 122

Like
0Likes
Like

Posted 24 March 2009 - 08:40 PM

Quote:
Original post by Codeka
You don't have to do anything to free memory. When the XML document itself is unloaded, all elements are freed as well.


What if I do not want to unload the XML document ? (take it as a database)
in this case, the while loop above will increase the memory till saturation (is this normal ?).

valgrind mention that blocks allocated by tinyxml are still reachable.

anyway to avoid that ?

#7 Codeka   Members   -  Reputation: 1157

Like
0Likes
Like

Posted 24 March 2009 - 10:29 PM

Quote:
Original post by leelouch
What if I do not want to unload the XML document ? (take it as a database)
in this case, the while loop above will increase the memory till saturation (is this normal ?).
No, tinyxml allocates memory for the whole document up-front. As you enumerate elements, it just returns its internal pointers. So the same element will return the same pointer each time.

#8 leelouch   Members   -  Reputation: 122

Like
0Likes
Like

Posted 24 March 2009 - 11:21 PM

Quote:
Original post by Codeka
Quote:
Original post by leelouch
What if I do not want to unload the XML document ? (take it as a database)
in this case, the while loop above will increase the memory till saturation (is this normal ?).
No, tinyxml allocates memory for the whole document up-front. As you enumerate elements, it just returns its internal pointers. So the same element will return the same pointer each time.


Ok this is good to know, so normaly if I remove my delete and clear, memory must not increase, but this it'is not the case ! ->
pointer change !

very simple example : you can try it :)
pElem change ->
0x8d7dfd8
0x8d7dfe8
0x8d7dff8
0x8d7e008
0x8d7e018
0x8d7e028
0x8d7e038
0x8d7e048

(see also valgrind output below)


#include <ticpp.h>
#include <ostream>

using namespace std;

int main() {
ticpp::Document xml;
string topName;

ticpp::Node * _top;

xml.LoadFile("test.xml");
_top = xml.FirstChild(false); // throwIfNoChildren=false
topName = ((ticpp::Element*)_top)->GetAttribute("atom");

cout << topName << endl;

while (1) {
ticpp::Element * pElem = _top->FirstChildElement();
cout << pElem << endl;
};
}





test.xml:

<params atom="unit_2ddfba91" name="top">
<atom name="base_22475976">
<step value="0">
<param name="enRead" type="Boolean" value="True"/>
<param name="enWrite" type="Boolean" value="True"/>
</step>
</atom>
<atom name="base_22111976">
<step value="0">
<param name="enRead" type="Boolean" value="False"/>
<param name="enWrite" type="Boolean" value="True"/>
</step>
</atom>
</params>





valgrind output:
valgrind --log-file=val --leak-resolution=high --leak-check=full --show-reachable=yes test.out


==28684== 624 bytes in 12 blocks are still reachable in loss record 38 of 40
==28684== at 0x4004790: operator new(unsigned) (vg_replace_malloc.c:164)
==28684== by 0x80665A5: TiXmlElement::Parse(char const*, TiXmlParsingData*, TiXmlEncoding) (in test.out)
==28684== by 0x806623D: TiXmlElement::ReadValue(char const*, TiXmlParsingData*, TiXmlEncoding) (in test.out)
==28684== by 0x806670D: TiXmlElement::Parse(char const*, TiXmlParsingData*, TiXmlEncoding) (in test.out)
==28684== by 0x806623D: TiXmlElement::ReadValue(char const*, TiXmlParsingData*, TiXmlEncoding) (in test.out)
==28684== by 0x806670D: TiXmlElement::Parse(char const*, TiXmlParsingData*, TiXmlEncoding) (in test.out)
==28684== by 0x806623D: TiXmlElement::ReadValue(char const*, TiXmlParsingData*, TiXmlEncoding) (in test.out)
==28684== by 0x806670D: TiXmlElement::Parse(char const*, TiXmlParsingData*, TiXmlEncoding) (in test.out)
==28684== by 0x8066B6A: TiXmlDocument::Parse(char const*, TiXmlParsingData*, TiXmlEncoding) (in test.out)
==28684== by 0x8063333: TiXmlDocument::LoadFile(_IO_FILE*, TiXmlEncoding) (in test.out)
==28684== by 0x80634E9: TiXmlDocument::LoadFile(char const*, TiXmlEncoding) (in test.out)
==28684== by 0x804DFB4: ticpp::Document::LoadFile(char const*, TiXmlEncoding) (in test.out)
==28684==
==28684==
==28684== 8,388,608 bytes in 1 blocks are still reachable in loss record 39 of 40
==28684== at 0x4004790: operator new(unsigned) (vg_replace_malloc.c:164)
==28684== by 0x805C5BB: std::vector<ticpp::Base*, std::allocator<ticpp::Base*> >::_M_insert_aux(__gnu_cxx::__normal_iterator<ticpp::Base**, std::vector<ticpp::Base*, std::allocator<ticpp::Base*> > >, ticpp::Base* const&) (in test.out)
==28684== by 0x8054304: ticpp::Node::FirstChildElement(char const*, bool) const (in test.out)
==28684== by 0x805443E: ticpp::Node::FirstChildElement(bool) const (in test.out)
==28684== by 0x804AD95: main (params.cc:97)
==28684==
==28684==
==28684== 12,748,332 bytes in 1,062,361 blocks are still reachable in loss record 40 of 40
==28684== at 0x4004790: operator new(unsigned) (vg_replace_malloc.c:164)
==28684== by 0x8053EC5: ticpp::Node::FirstChildElement(char const*, bool) const (in test.out)
==28684== by 0x805443E: ticpp::Node::FirstChildElement(bool) const (in test.out)
==28684== by 0x804AD95: main (params.cc:97)






#9 Tunah   Members   -  Reputation: 126

Like
0Likes
Like

Posted 25 March 2009 - 02:11 AM

Quote:
Original post by leelouch

#include <ticpp.h>
#include <ostream>

using namespace std;

int main() {
ticpp::Document xml;
string topName;

ticpp::Node * _top;

xml.LoadFile("test.xml");
_top = xml.FirstChild(false); // throwIfNoChildren=false
topName = ((ticpp::Element*)_top)->GetAttribute("atom");

cout << topName << endl;

while (1) {
ticpp::Element * pElem = _top->FirstChildElement();
cout << pElem << endl;
};
}




How are you stopping your program if you have an infinite loop at the end? Try just running that last loop a couple of hundred times and let the program terminate itself.
Otherwise, ticpp::Document xml won't go out of scope and be destroyed. You're probably just seeing memory that was allocated by the Document-object, but its destructor isn't called when your terminate the program with Ctrl-C.
That's my guess anyway.

#10 leelouch   Members   -  Reputation: 122

Like
0Likes
Like

Posted 25 March 2009 - 03:10 AM

Quote:
Original post by Tunah


How are you stopping your program if you have an infinite loop at the end? Try just running that last loop a couple of hundred times and let the program terminate itself.
Otherwise, ticpp::Document xml won't go out of scope and be destroyed. You're probably just seeing memory that was allocated by the Document-object, but its destructor isn't called when your terminate the program with Ctrl-C.
That's my guess anyway.


This is just an example !
Let's say that xml is a global variable and I do not want to destroy it and reload it again on each time I need an attribute or something else.
The while statement here is only to simulate several access to the xml document ! Access to xml database must not reallocate a new element on each time you access to it. it must have like a cache on element already allocated and return cached pointer.

the while loop must not create an infinite allocation behavior. memory allocation size must remains constant during the loop where here is not the case !



#11 Tunah   Members   -  Reputation: 126

Like
0Likes
Like

Posted 25 March 2009 - 03:27 AM

Quote:
Original post by leelouch
Quote:
Original post by Tunah


How are you stopping your program if you have an infinite loop at the end? Try just running that last loop a couple of hundred times and let the program terminate itself.
Otherwise, ticpp::Document xml won't go out of scope and be destroyed. You're probably just seeing memory that was allocated by the Document-object, but its destructor isn't called when your terminate the program with Ctrl-C.
That's my guess anyway.


This is just an example !
Let's say that xml is a global variable and I do not want to destroy it and reload it again on each time I need an attribute or something else.
The while statement here is only to simulate several access to the xml document ! Access to xml database must not reallocate a new element on each time you access to it. it must have like a cache on element already allocated and return cached pointer.

the while loop must not create an infinite allocation behavior. memory allocation size must remains constant during the loop where here is not the case !


Fair enough, I guess. But you posted example code and valgrind output after it, so it's only logical to assume that you ran it like that.


#12 leelouch   Members   -  Reputation: 122

Like
0Likes
Like

Posted 25 March 2009 - 03:34 AM

Quote:
Original post by Tunah

Fair enough, I guess. But you posted example code and valgrind output after it, so it's only logical to assume that you ran it like that.


valgrind support Ctrl-C interrupt, and show you still-reachable pointer
otherwise the linux "top" command works very well :)


#13 Kylotan   Moderators   -  Reputation: 3338

Like
0Likes
Like

Posted 25 March 2009 - 04:15 AM

If it's any consolation, someone else here reported seeing memory leaks with their TinyXML++ code and their code seemed fine.

#14 leelouch   Members   -  Reputation: 122

Like
0Likes
Like

Posted 25 March 2009 - 04:19 AM

Quote:
Original post by Kylotan
If it's any consolation, someone else here reported seeing memory leaks with their TinyXML++ code and their code seemed fine.


Yes, it's not the first time, I submit an issue in the TinyXml++ project :
http://code.google.com/p/ticpp/issues/list
waiting to see if I missed something or not !
Thanks for everybody

#15 Codeka   Members   -  Reputation: 1157

Like
0Likes
Like

Posted 25 March 2009 - 09:32 AM

Yeah, I just downloaded the TinyXML++ source code and it looks like it's allocating a "wrapper" object every time you call FirstChildElement or NextChildElement, etc. It seems a bit silly to me, but you can call DeleteSpawnedWrappers at any time to free up the memory used by these wrapper objects. The memory is also free when you destroy the main object.

Personally, I use the regular TinyXML. I don't think TinyXML++ provides much over the base TinyXML anyway. The whole point of TinyXML is that it's light-weight :-)

#16 Codeka   Members   -  Reputation: 1157

Like
0Likes
Like

Posted 25 March 2009 - 09:37 AM

Quote:
Original post by leelouch
valgrind support Ctrl-C interrupt, and show you still-reachable pointer
otherwise the linux "top" command works very well :)
This is not really a valid test, though. Pressing Ctrl-C will not call the destructor of the Document object and so it would not have a chance to free the memory. As I said, the memory is freed when you release the top-level document object.

It's a bit silly (in my opinion) that they allocate a new "wrapper" object every time you call NextSiblingElement, etc, but I guess they're not really expecting it to be used like how you use it (I'm guessing they expect you to just parse the document into your own internal types and then that's it).

#17 leelouch   Members   -  Reputation: 122

Like
0Likes
Like

Posted 25 March 2009 - 10:39 AM

Quote:
Original post by Codeka
Quote:
Original post by leelouch
valgrind support Ctrl-C interrupt, and show you still-reachable pointer
otherwise the linux "top" command works very well :)
This is not really a valid test, though. Pressing Ctrl-C will not call the destructor of the Document object and so it would not have a chance to free the memory. As I said, the memory is freed when you release the top-level document object.

It's a bit silly (in my opinion) that they allocate a new "wrapper" object every time you call NextSiblingElement, etc, but I guess they're not really expecting it to be used like how you use it (I'm guessing they expect you to just parse the document into your own internal types and then that's it).


I agree, it's a bit silly, and it's not usable in my case. it is difficult, because it's completely used as a database that can contains a lot of different stuff depending on the reader ...


#18 Codeka   Members   -  Reputation: 1157

Like
0Likes
Like

Posted 25 March 2009 - 11:47 AM

Quote:
Original post by leelouch
I agree, it's a bit silly, and it's not usable in my case. it is difficult, because it's completely used as a database that can contains a lot of different stuff depending on the reader ...
Would it make more sense for you to load the entire document into your own data structure, and query that at runtime? That way, you only keep the XML document loaded for the short time it takes to parse it, and when you're finished parsing, and the memory will be freed. It also means you don't end up doing O(n) loops through all your XML nodes every time you want to do a query and you can use more efficient data structures.

#19 leelouch   Members   -  Reputation: 122

Like
0Likes
Like

Posted 25 March 2009 - 08:40 PM

Quote:
Original post by Codeka
Would it make more sense for you to load the entire document into your own data structure, and query that at runtime? That way, you only keep the XML document loaded for the short time it takes to parse it, and when you're finished parsing, and the memory will be freed. It also means you don't end up doing O(n) loops through all your XML nodes every time you want to do a query and you can use more efficient data structures.


it's not so simple, the xml structure is not uniq, can be anything, only the reader module knows what it's in side,
what I wanted to do is a simple way to access the xml structure using an hierarchical access. for ex:
<top>
<a>
<b value=True type=bool>
<c value=True type=int>
<d type=array len=3 type=string>
<item0 value="x">
<item1 value="y">
<item2 value= "45">
<d>

</top>

then you can access easily :
top.a.b -> value of b !
top.d.len -> length
top.d.0 -> item 0

etc etc
so a generic structure would be a map<map<map ... <string,string>>>>... :)
which is not practical !

Thanks anyway


#20 leelouch   Members   -  Reputation: 122

Like
0Likes
Like

Posted 26 March 2009 - 01:22 AM

lookin to the code of TiXml++ in ticpp.h
I found this !!

They create a new wrapper on each time and they store it in vector of (Base *) (spawnedWrappers), which is very very weird !!!
this vector will be bigger and bigger each time accesing a new sibling !

wondering why ? any idea ?

Element* temp = new Element( sibling );
sibling->m_spawnedWrappers.push_back( temp );

why not storing one and only wrapper ?





Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.



PARTNERS