Skip to main content
GameDev.net gamedev.net
🔒 Locked

Anybody had crashes with boost::unordered_map?

Started by Kylotan Nov 25, 2009 at 7:40 AM 31 replies 14.3k views
Original Post
Kylotan
Kylotan
Basically, I have some code where I have 15,000 elements in a std::map. I thought a direct switch to boost::unordered_map would yield lookup and space savings, but unfortunately what happens is that it crashes inside boost::unordered_map::find with a division by zero error. The only division in that area of code that I can see involves the number of buckets, but I know that is non-zero (something like 23000). I'm using Visual C++ 2008 with Boost 1.36. (I know it's old, but that's not something I can change right now. There's nothing in the changelog suggesting a fix since then however.) Has anybody else seen anything like this?
swiftcoder
swiftcoder
Quote:
Original post by Kylotan
Has anybody else seen anything like this?
Ja, I tried unordered_map exactly once, and went back to std::map. This was Mac (Leopard) + gcc + Boost 1.4, and while I don't remember the exact nature of the crash, it was deep within the unordered_map implementation.
Tristam MacDonald. Ex-BigTech Software Engineer. Future farmer. [https://trist.am]
alvaro
alvaro
I understand unordered_map is essentially what in g++ used to be called hash_map. I've used hash_map in g++ for a long time and I never had a problem with it. I'm surprised that implementations of unordered_map are not solid.

You should probably try to reduce your program to a minimal program that still shows the problem. Then you can send that to the compiler's writers and you can post a copy here as well.

Kylotan
Kylotan
Unfortunately that's probably not practical due to the size of the program I'm working with. But it is interesting to see that someone else gets something similar too, with a different compiler but the same library.
Rattrap
Rattrap
If you can get a minimal example, you might also send it over to the guys at boost through the boost user forums. They are usually pretty good at responding pretty quickly.
"I can't believe I'm defending logic to a turing machine." - Kent Woolworth [Other Space]
Kylotan
Kylotan
Quote:
Original post by snake5
How about fixing the bug yourself? :) I'm sure it wouldn't be hard. You already have the place in your code where it happens, right?

Unfortunately I'm finding this in optimised code where the relevant stuff has all been inlined. It could be a compiler bug. But those are even rarer than Boost bugs.

snake5
snake5
Do you mean you don't get the bug in the debug version of your code?
Then you can check this out: http://www.flounder.com/debug_release.htm
And some bugs from my experience:
- some .obj files are old and the compiler thinks that they shouldn't be rebuilt when they actually should; solution - rebuild all
- code that is inside an empty macro (like assert in release build) will not be executed; solution - move the code out of that macro.
Kylotan
Kylotan
It's nothing to do with macros - this is quite simple code that works perfectly for std::map and stops working for boost::unordered_map. Anyway, I'm running in debug mode - it's just that we're inlining this code even in debug builds. I may try turning inlining off for the overnight build and see what happens tomorrow.
Sneftel
Sneftel
Any chance your hashes are changing underneath you? That seems like the only thing (other than a library or compiler bug) that could cause unordered_map to hiccup.
Kylotan
Kylotan
The key type is std::string (passed by value), the hash function is the default one provided, so I don't see how that could happen?
Sneftel
Sneftel
Yeh, that sounds stable. Weird. I suppose it could be a wild memory write somewhere else in the application, which tends to poop on the unordered_map because it's the unordered_map that has a bunch of memory allocated? Kind of a Hail Mary theory, but you might try peppering your code with _CrtCheckMemory() and see if anything fires.
popsoftheyear
popsoftheyear
Sorry if this is a naive suggestion and I don't realize it... but if you're using Visual Studio 2008 why not use std::tr1::unordered_map? It's worked great for me although I don't believe I've reached so many elements as 15000.
Washu
Washu
Quote:
Original post by popsoftheyear
Sorry if this is a naive suggestion and I don't realize it... but if you're using Visual Studio 2008 why not use std::tr1::unordered_map? It's worked great for me although I don't believe I've reached so many elements as 15000.


As has been noted previously the implementation of TR1 that comes with Visual Studio 2008 SP1 (Dinkumware's) is not optimal.
In time the project grows, the ignorance of its devs it shows, with many a convoluted function, it plunges into deep compunction, the price of failure is high, Washu's mirth is nigh.
Kylotan
Kylotan
In this case, std::tr1::unordered_map works fine where boost::unordered_map crashes. Intriguing. (std::tr1::unordered_map appears to cost about 2Mb more than std::map for this data, however.)
Kylotan
Kylotan
I changed it to supposedly only inline functions that were "__inline only" and it ignored that, so I'm giving up on boost::unordered_map and going back to std::map.
brainydexter
brainydexter
I have recently started using boost::unordered_map, where I use it to as a lookup for a pair.

I don't understand what is the problem, for it works absolutely fine sometimes and burps on me otherwise. I had a duck model loading in and it worked fine. It had a lot of vertices in it(don't remember the exact count). On the other hand, I tried loading in another model, and it gave me an error inside the boost::unordered_map::find method.

Do you guys think it could be an issue with the boost implementation ?

Also, please correct me on this one.. I used boost::unordered_map , since each of the vertices have no sense of ordering between them and std::map is (I think) an associative container. Thus, I went ahead with boost::unordered_map. Do you think I should switch to std::map ?
I also came across std::hash_map, but then I read, it is compiler specific. VS has its own files and gcc its own. That was another reason I chose boost.
outRider
outRider
Quote:
Original post by Kylotan
Quote:
Original post by snake5
How about fixing the bug yourself? :) I'm sure it wouldn't be hard. You already have the place in your code where it happens, right?

Unfortunately I'm finding this in optimised code where the relevant stuff has all been inlined. It could be a compiler bug. But those are even rarer than Boost bugs.


Do you have a core dump of some sort (whatever the Windows equivalent is), or the assembly code generated for the inner-most function containing the crashing code? If there's only one division in the source it shouldn't be too hard to find where the 0 comes from.
Kylotan
Kylotan
brainydexter, that seems like an odd application (why does a vertex need to know its own index?) but it sounds like it should work, providing you have a hashing function for your Vertex class. The idea is that std::map does everything that an unordered_map does, plus maintains the ordering, and therefore it is likely to be slower due to this extra requirement. That means unordered_map is preferable if you don't need the ordering - and if it doesn't crash when you use it, obviously. It's interesting if you're seeing divide by zero errors in boost::unordered_map::find like I am.

outRider, that code is all inlined so unfortunately it is quite difficult for me to work out where the zero came from. I have a good idea what variable it's supposed to be (ie. the number of buckets in the bucket manager) but earlier in the code that was over 23,000.
outRider
outRider
Quote:
Original post by Kylotan
outRider, that code is all inlined so unfortunately it is quite difficult for me to work out where the zero came from. I have a good idea what variable it's supposed to be (ie. the number of buckets in the bucket manager) but earlier in the code that was over 23,000.


Right. You can always try a watchpoint on the number of buckets variable. You can also print the number of buckets before each find call. I doubt it's a compiler bug if it occurs in both Debug and Release builds, but it's not impossible either. Maybe you can try a different compiler to rule it out.

Topic Locked

This topic has been locked by a moderator. New replies are not allowed.

Sign in to reply to this topic.