Jump to content
  • Advertisement
Sign in to follow this  
Osmo Suvisaari

Challenging STL magic

This topic is 4019 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

I have used STL widely in a large WinXP project. Last week I decided to try again STLport (version 5.1.3). I have been using the STL that comes with VS2005 and it has worked quite well... However, as I expected, STLport is quite a lot faster and I would like to keep using it, but: While all else seems to work. Something goes wrong with the vector normals that are calculated for bumpmap textures. Now I am narrowing down the possible causes: Visual inspection and logfile written values suggest that bumpmap textures have some correct values and repeated patterns of wrong values. Hardcoding some patterns to bumpmap generating function seems to work sometimes. Before that failed values come from sinf and cosf using calculations. After that the values go are stored in an array and put into DirectX 9 managed textures using LockRect and memcpy (no STL there). I have multiple threads, but this all is done in a single thread without interference. I have very few pointers in the whole program. Turning STLport debug range checking does not catch any out-of-range uses. STL is not used to generate the data. However std::map is used to store information about bumpmap texture piece updates. (Not actual data, just updated index numbers for both key and value.) Everything works ok in debug mode and also in release mode if I disable VS2005 compiler optimizations. If I do not disable optimizations but change floating point model from the VS2005 project settings, then the wrong values change and sometimes work but not reliably. Without STLport everything works, but is slower. Questions: Are other people using STLport with a VS2005 project? Have you encountered similar problems? Good ideas about finding a bug that does not show in debug mode at all? Thanks in advance for help and encouragement... [cool] Osmo Suvisaari waspgames.com

Share this post


Link to post
Share on other sites
Advertisement
I looked at STL port some time back. The supported compilers list stops at VC6 and gcc 3.0.

Then I stopped looking into it.

I'm not saying that that list is correct, or definitive, but MVS 7,8 and 9 are perhaps worth mentioning if you're dealing with anything even remotely C++ related. It's not like they are new products. VC6 is only 10 years old now.

Share this post


Link to post
Share on other sites
Quote:
Original post by Osmo Suvisaari
I have very few pointers in the whole program.
Turning STLport debug range checking does not catch any out-of-range uses.
STL is not used to generate the data.
However std::map is used to store information about bumpmap texture piece updates. (Not actual data, just updated index numbers for both key and value.)
Hmm... Are you changing the the data values (std::map <key_type, data_type, [comparison_function]>) inside the map without first removing them? This is at least one way of possibly corrupting the data as the inner data structure invariant doesn't necessarily hold in that case any more. (I'd bet it's a tree and changing some internal node in which the ordering is based doesn't trigger any tree rotations.)

This is something VC++ compiler didn't catch earlier at least whereas some g++ compilers will report as an error when compiled with strict enough compiler options.
Quote:
Original post by Osmo Suvisaari
Questions:
Are other people using STLport with a VS2005 project?
Have you encountered similar problems?
Good ideas about finding a bug that does not show in debug mode at all?
Those pesky heisenbugs. ;) But anyway, that is the only thing I can think of and that is a pretty common data structure related error.

Share this post


Link to post
Share on other sites
Quote:
Original post by Antheus
I looked at STL port some time back. The supported compilers list stops at VC6 and gcc 3.0.

Then I stopped looking into it.

I'm not saying that that list is correct, or definitive, but MVS 7,8 and 9 are perhaps worth mentioning if you're dealing with anything even remotely C++ related. It's not like they are new products. VC6 is only 10 years old now.
STLport-5.1.3 seem to be supporting also the compiler shipping with VS2005. Some release notes. And besides, I tested the new TR1 stuff with my version of VS2005 in order to do some algorithms regarding automata minimisation (pesky coursework I want to do in environment initially Windows and not in some *nix) since I need hashmaps. [smile]

Share this post


Link to post
Share on other sites
I think that is a bug in my own code or configuration, and STLport is ok...

But it is certainly possible that there is something wrong with using STLport with VS2005. It would be good to know for certain before I spend days and nights debugging this... [cool]

This bug is probably a pointer bug, out-of-range bug or a multithreading bug in my own code, or in the configuration.

It is ironic that changing STL broke something that is the least STL using part of the project. That means probably that the error is somewhere else but only causing problems at that point due memory corruption, or... a STLport compatibility issue?

-Osmo Suvisaari
waspgames.com

Share this post


Link to post
Share on other sites
Progress update, pragmatic approach:
I have not found any bugs in my own code that would cause this.
However I have isolated the code that causes errors when optimized using

#pragma optimize("", off)
and
#pragma optimize("", on)




Everything works ok if I disable optimization before the end of one class declaration in one class header. I turn the optimization back on right after that like this:

#pragma optimize("", off)
}; //class declaration ends here
#pragma pack(pop)
#pragma optimize("", on)




I wonder how that does affect the code as the disabled part does not contain any functions? I think it should not have any effect as Visual Studio documentation states that:
Quote:

The optimize pragma must appear outside a function and takes effect at the first function defined after the pragma is seen.


Only special thing in that class is the use of #pragma pack lines before and after class declaration.
And removing that does not fix the problem, and I have used #pragma pack in similar fashion in 5 classes.

Conclusion:
The problem is now gone, but I am not satisfied by the solution, because it is probable that the bug is still there and this has only hidden it...

-Osmo Suvisaari
waspgames.com

Share this post


Link to post
Share on other sites
Ok, here is the whole header file ActiveMapBlock_logical.h:


#pragma once

#include "Globals.h"
#include "ActiveMapBlock.h"

#pragma pack(push)
#pragma pack(1)

// Logical map block used in game mechanical calculations
class CActiveMapBlock_logical : public CActiveMapBlock
{
unsigned short CalculatedRelativeMovementCost;

char Rainfall;
char Warmth;

unsigned char DefaultPopulation; //read from defaults, not saved
unsigned char DefaultTechnologyLevel; //read from defaults, not saved
int DefaultFlags; //read from defaults, not saved

unsigned char Population;
unsigned char TechnologyLevel;
unsigned char Roughness;

int Flags;
int PreviousFlags;

public:

void SetFlagOn( CActiveMapBlock::enumFlag gFlag)
{
Flags |= gFlag;
}

void SetFlagOff( CActiveMapBlock::enumFlag gFlag)
{
Flags &= (0xffffffff ^ gFlag);
}

void SetPreviousFlags(int gFlags) {PreviousFlags=gFlags;}
void SetFlags(int gFlags) {Flags=gFlags;}

int GetCurrentFlags(float gPhase) {if(gPhase<=0) return PreviousFlags; else return Flags;}
int GetFlags() {return Flags;}
int GetPreviousFlags() {return PreviousFlags;}

virtual void LoadDefaultFrom(CBuffer & gBuffer);
virtual void SaveDefaultInto(CBuffer & gBuffer);

virtual void LoadFrom(CBuffer & gBuffer);
virtual void SaveInto(CBuffer & gBuffer);

void SetRainfall(float gValue)
{
Rainfall=static_cast<char>(gValue*127.0f);
}
float GetRainfall()
{
return Rainfall/127.0f;
}

void SetWarmth(float gValue)
{
Warmth=static_cast<char>(gValue*127.0f);
}
float GetWarmth()
{
return Warmth/127.0f;
}

float GetCalculatedRelativeMovementCost()
{
return static_cast<float>(CalculatedRelativeMovementCost)*0.01f;
}

void SetCalculatedRelativeMovementCost(float gValue)
{
if(gValue<=0.01f) CalculatedRelativeMovementCost = 1;
else if(gValue>=255.0f) CalculatedRelativeMovementCost = 25500;
else
{
CalculatedRelativeMovementCost=static_cast<unsigned short>(gValue*100.0f);
}
}

float GetPopulation() {return static_cast<float>(Population);}
float GetTechnologyLevel() {return static_cast<float>(TechnologyLevel)/255.0f;}
float GetRoughness() {return static_cast<float>(Roughness);}

void SetPopulation(float gPopulation)
{
if(gPopulation<=0) Population=0;
else if(gPopulation>255) Population=255;
else Population=static_cast<unsigned char>(gPopulation);
}
void SetTechnologyLevel(float gTechnologyLevel)
{
if(gTechnologyLevel<=0) TechnologyLevel=0;
else if(gTechnologyLevel>=1) TechnologyLevel=255;
else TechnologyLevel=static_cast<unsigned char>(gTechnologyLevel*255.0f);
}

void ImproveTechnologyLevel(float gImprovement)
{
SetTechnologyLevel( GetTechnologyLevel()+gImprovement );
}

void SetRoughness(float gRoughness)
{
if(gRoughness<=0) Roughness=0;
else if(gRoughness>255) Roughness=255;
else Roughness=static_cast<unsigned char>(gRoughness);
}

void CalculateRelativeMovementCost();

static float GetRoadMovementCost(float gPopulation, float gTechnology, float gRoughness);
float GetRoadMovementCost();

CActiveMapBlock_logical(void);
virtual ~CActiveMapBlock_logical(void);

#pragma optimize("", off)
};
#pragma pack(pop)
#pragma optimize("", on)




-Osmo Suvisaari
waspgames.com

Share this post


Link to post
Share on other sites
How about removing the pragma pack(1) and the corresponding pop statement? It could be possible that your other code excepts to see some other (default) alignment than the one you have you have explicitly declared. Moreover -- I can't know for sure -- it could be that your packs and pops interleave so that something goes wrong.

Anyway, it feels at least that the bugs that appear during optimisations are due to the fact that the optimisations rely on some other data alignment than the one you have explicitly set in your code. The other parts can't possible know that you have set some other member alignment than the default. Please see pack (C/C++) at MSDN for more information.

[EDIT]Consider, for instance, the one-line functions that will most likely be inlined during the optimisations. Now there's a real change the other code is doing something with your data that is aligned differently than it excepts it to be (due to the fact that there are private member with different alignment involved there).[/EDIT]

[Edited by - Naurava kulkuri on September 22, 2007 1:38:23 PM]

Share this post


Link to post
Share on other sites
Sign in to follow this  

  • Advertisement
×

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

We are the game development community.

Whether you are an indie, hobbyist, AAA developer, or just trying to learn, GameDev.net is the place for you to learn, share, and connect with the games industry. Learn more About Us or sign up!

Sign me up!