Correctly using google sparse hash maps in MS VC++?

This topic is 2861 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

Recommended Posts

I seem to be having some issues in my program... I'm probably doing something wrong. Right now I'll declare the hash map as thus:
google::sparse_hash_map<const char*, unsigned long, stdext::hash_compare<const char*>> MyHash;


My goal here is to be able to retrieve/store unsigned long values by using a (const char *) key For example
MyHash["Pizza"] = 42;


etc. I attempt to see if a value is in the hash like so
if(!MyHash["Pizza"]){
//Pizza is not in the table!
}else{
//Pizza is in the table!
}


Where am I going wrong? sometimes the Hash lookup messes up strings. Most of the time actually. As a for instance, I'd keep trying to see if Pineapple was in the table but it keeps returning Pizza. Any advice would be very appreciated :) [Edited by - Foofles on March 18, 2010 5:27:21 PM]

Share on other sites
What does stdext::hash_compare<const char*> do?

Does it hash the pointer or the text?

Share on other sites

Hmmm that didn't occur to me before. I did try std::string instead but I'm still having the same problem.

This may sound silly but I'm not even sure what stdext::hash_compare does... it's a templated class buried in templated classes. So I'm not sure what ultimately happens when it gets to the const char * or std::string.

Is there a way to use my own compare function?

Share on other sites
Quote:
 Original post by FooflesThanks for the reply :) Hmmm that didn't occur to me before. I did try std::string instead but I'm still having the same problem.This may sound silly but I'm not even sure what stdext::hash_compare does... it's a templated class buried in templated classes. So I'm not sure what ultimately happens when it gets to the const char * or std::string. Is there a way to use my own compare function?

Probably. I don't know what stdext is (or what compiler/stdlib you're using). You'll have to look it up in your library's documentation.

Anything I tell you is going to come via Google.

Share on other sites
Well thanks for the reply anyhow :)

I'm still stumped :( ... hash_compare defaults to a "less than" comparison function, I tried plugging in my own like:

struct eqstr{  bool operator()(const char* s1, const char* s2) const  {    return strcmp(s1, s2) == 0;  }};google::sparse_hash_map<const char*, unsigned long, stdext::hash_compare<const char*, eqstr>> MyHash;

But it still doesn't work. Maybe I'm incorrectly accessing whether a key is in the map or not. Anybody have any experience with google:sparse_hash_map ? or maybe even stdext::hash_map? This is giving me a world of trouble. :/

Share on other sites
Try!

FWIW, I've had good experiences with boost's unordered_map.

Share on other sites
Quote:
 Original post by the_eddTry!FWIW, I've had good experiences with boost's unordered_map.

Same here. You can also use the standard unordered_map with any compiler that has tr1 (Visual Studio 2008, for example). I believe Boost's version uses the standard one if it exists, and otherwise it's own implementation (which is similar).

Share on other sites
I've been staring at that documentation page for days, tried that page's eqstr function and everything else on it as well. Nothing seems to work!

But this unordered_map is actually news to me. I will look into it right now. Thanks so much :)

Edit: OOps. Here I was so confident I had solved the problem. I haven't. Any advice on hash_map , google::sparse_hash_map or unordered_map is still greatly appreciated. :)

[Edited by - Foofles on March 18, 2010 5:01:41 PM]

Share on other sites
Post the actual code you're using.

Share on other sites
Quote:
 Original post by FooflesI've been staring at that documentation page for days, tried that page's eqstr function and everything else on it as well. Nothing seems to work!

What do you mean by "work"? Have you copy-and-pasted the example on that page in to a text editor and got it to compile? Was there any part of *that specific example* that you didn't understand or your compiler wouldn't accept? Perhaps the part about tr1/ext::has/__gnu_cxx::hash was confusing? Which compiler are you using, anyway?

Concrete details are needed.

Quote:
 Edit: OOps. Here I was so confident I had solved the problem. I haven't. Any advice on hash_map , google::sparse_hash_map or unordered_map is still greatly appreciated. :)

Again, Google is a wonderful thing. Enter "boost" and "unordered_map". Take it from there.

Share on other sites
I'm using Microsoft Visual C++ express, the newest edition.

Yes I copied exactly, I apologize, let me explain more clearly:

When I say "It doesn't work", I mean it will associate a string with an existing value when the string really shouldn't be in the hash table. Very well, allow me to post the exact code that's being used at the moment even though it's been changed a little. If there's anything particularly silly, please don't hesitate to yell at me for it :)

The point of the hash tables at the moment is to not have to redundantly load textures or geometry data. For example, if I have brickwall.jpg and then another model uses brickwall.jpg in a material, it will just pass an index to an existing texture resource... I am using DirectX 10.

struct eqstr{  bool operator()(const char* s1, const char* s2) const  {    return strcmp(s1, s2) == 0;  }};google::sparse_hash_map<const char*, unsigned long, stdext::hash_compare<const char*, eqstr>> TexHash;std::vector<ID3D10ShaderResourceView*> TexSRVArray;// cArtManager is a class that deals with art resources...// I am aware this function is potentially dangerous since// it can just plain fail loading a texture, but right now// I am using break points to see if it's going to !TexHash[pTextureName]// pFilePath is the full file path// I only save the "Brickwall.jpg" for instance.unsigned long cArtManager::AddTexture(const char * pFilePath, ID3D10Device * pd3dDevice){	char * pTextureName = NULL;	pTextureName = (char *)strrchr ( pFilePath, '\\' )+1;	if(pTextureName == NULL || pTextureName == (char*)0x00000001){ 		pTextureName = (char *)strrchr ( pFilePath, '//' )+1;	}	if(!TexHash[pTextureName]){ // if we don't already have this, add it		ID3D10ShaderResourceView* pSRV;		D3DX10CreateShaderResourceViewFromFileA( pd3dDevice, pFilePath, NULL, NULL, &pSRV, NULL );		TexSRVArray.push_back(pSRV);		pSRV = NULL;		TexHash[pTextureName] = (TexSRVArray.size() - 1);	}	return TexHash[pTextureName];}

Now the problem here... Like I said, sometimes it'll incorrectly assume one filename to be another. Not all Filenames either. Which makes it even weirder.

For example I have a coin ... and a torch. Let's say one has Coin.jpg and the torch has Torch.jpg ... no matter what, it always seems to swap. If I load the torch first, the coin will have torch texture. If I load the coin first, the torch will have the coin texture.

This tells me there is some weird issue with the way it's checking strings. Or something... help me please? :) Much appreciated btw everyone sticking with me

Share on other sites
First things first: Use std::string, not char pointers. If you have a character pointer in your code for representing a string, chances are you have a bug.

Secondly: If it's misbehaving despite your usage appearing correct, then chances are you have a bug ELSEWHERE. Try using the debugger.

Finally: generally you don't have to specify all of the template parameters (assuming you're using std::string), as it will appropriately derive the correct template parameters based on the initial one... Also, you should use boost::unordered_map over the one that comes with vs2008, as the one in 2008 has a rather nasty bug in it.

Share on other sites
I have also tried it with std::string instead of char * ... when I use breakpoints pTextureName appears as it should. I will investigate more thoroughly, thank you for the response.

I guess it does make sense to start actually using C++ functionalities...

Share on other sites
Quote:
 Original post by FooflesI'm using Microsoft Visual C++ express, the newest edition.Yes I copied exactly

So which hash did you use? What command line or project did you use to compile it? What was the output?

In other words, did the absolute minimal example posted on that page do something sensible? If not, there's absolutely no point in trying to do anything more with it, until you get over that problem.

Share on other sites

google::sparse_hash_map<const char*, unsigned long, stdext::hash_compare<const char*>> TexHash;

Now I am trying the Boost libraries and std::string but now rather than mixing up entries I get duplicate entries, for example I'll add coin.jpg once and then the second time it comes up the hash find will fail and it'll add it a second time, but after the second time it's fine.

For boost I am doing:

#include <boost/unordered_map.hpp>typedef boost::unordered_map<std::string, unsigned long> MyHash;MyHash TexHash;// ... unsigned long cArtManager::AddTexture(const char * pFilePath, ID3D10Device * pd3dDevice){	std::string TextureName(pFilePath);	if(!TexHash[TextureName]){ // if we don't already have this, add it		ID3D10ShaderResourceView* pSRV;		D3DX10CreateShaderResourceViewFromFileA( pd3dDevice, pFilePath, NULL, NULL, &pSRV, NULL );		TexSRVArray.push_back(pSRV);		pSRV = NULL;		TexHash[TextureName] = (TexSRVArray.size() - 1);	}	return TexHash[TextureName];}

This duplicate effect happens even if I use something like this in the constructor

std::string MyString("Hello");if(!MyHash[MyString]){    //Something.}

At least now it isn't confusing entries randomly...

Share on other sites
Quote:
 Original post by Fooflesif(!TexHash[TextureName])

Thats not how you check if an object already exists in an unordered_map. What that will do is try to find the value with key of "TextureName" and if it doesnt find it, it will default construct one.

You should use:

MyHash::const_iterator value = TexHash.find( TextureName );if ( value == TexHash.end( ) ){    // TextureName was not found.}return value->second; // TextureName was found, so the second object in the pair will be the value.

Share on other sites
Quote:
 Original post by diablos_bladeYou should use:*** Source Snippet Removed ***

More accurately:
std::string key(pFilePath);MyHash::const_iterator iter = TexHash.find( key );if ( iter == TexHash.end() ){  ID3D10ShaderResourceView* pSRV;  D3DX10CreateShaderResourceViewFromFileA( pd3dDevice, pFilePath, NULL, NULL, &pSRV, NULL );  size_t value = v.size();  TexSRVArray.push_back(pSRV);  iter = TexHash.insert(iter, std::make_pair(key, value));}return iter->second;

Share on other sites
Quote:
 Original post by WashuFinally: generally you don't have to specify all of the template parameters (assuming you're using std::string), as it will appropriately derive the correct template parameters based on the initial one... Also, you should use boost::unordered_map over the one that comes with vs2008, as the one in 2008 has a rather nasty bug in it.

I'm curious, what exactly is this "nasty" bug? I've been trying to get us to move away from hash_map for a while (since it's the only stdext thing we use) and this info would help my cause [grin]

Share on other sites
Quote:
Original post by Zipster
Quote:
 Original post by WashuFinally: generally you don't have to specify all of the template parameters (assuming you're using std::string), as it will appropriately derive the correct template parameters based on the initial one... Also, you should use boost::unordered_map over the one that comes with vs2008, as the one in 2008 has a rather nasty bug in it.

I'm curious, what exactly is this "nasty" bug? I've been trying to get us to move away from hash_map for a while (since it's the only stdext thing we use) and this info would help my cause [grin]

This one, maybe? I was curious too and that's the best I could come up with after a couple of minutes of googling.

Share on other sites
I'm guessing he's referring to the performance characteristics we noticed in this thread.

Share on other sites
Quote:
Original post by Antheus
Quote:
 Original post by diablos_bladeYou should use:*** Source Snippet Removed ***

More accurately:*** Source Snippet Removed ***

Aha, thank you so much :) I knew it had to be some ridiculous mistake in me checking for an entry. using this Iterator / find method is working perfectly.

Thank you everyone, I am going to assume that if I were to use google::sparse_hash_map then I'd also use a similar route, I just couldn't make heads or tails of how it worked.