• Advertisement
Sign in to follow this  

GetOpenFileName stack corruption

This topic is 3674 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

Hi, I'm having some ridiculous trouble with GetOpenFileName. Here is my code:
memset(&ofn, 0, sizeof(ofn));

buffer[0] = 0;

ofn.lStructSize = sizeof(ofn); 
ofn.hwndOwner = DXUTGetHWND();
ofn.lpstrFilter = TEXT("AVI Files (.avi)\0*.avi\0Quicktime Files (.mov)\0*.mov\0\0");
ofn.lpstrFile = buffer;
ofn.nMaxFile = MAX_PATH;
ofn.lpstrDefExt = TEXT("avi");
If I select a file and click ok, I get a crash later on in the program. If I click cancel, no crash I've checked that GetOpenFileName is not overwriting more memory than it should in buffer by putting specific values at the end of the buffer and checking them after the call. No problem there. If I comment out the line that sets the edit box text, it still crashes. The same issue happens with GetSaveFileName. Any ideas? I must be making some horrible stupid mistake here, or windows has a serious bug in these functions.

Share this post

Link to post
Share on other sites
It is way, way later. The later crash occurs when I recreate my direct3d device. The reset works fine, as long as you haven't clicked browse to open the file dialog.

Honestly I know how ridiculous that sounds because the stack obviously works fine for a while (until you reset the device), but I have no other explanation. It was a bitch to track down but I'm quite sure that it is precisely opening a file dialog and nothing else that causes the device reset to fail.

More specifically, it is calling D3DXCreateEffectFromFile that fails. What has me concerned is that D3DXCreateEffectFromFile uses a fixed string in the code, as does my dialog code. I noticed that lpstrFilter is not const, so if GetOpenFileDialog is messing with that string, overrunning it, and going into another string, it could affect the fixed string I use for D3DXCreateEffectFromFile. But, it shouldn't even compile if that member wasn't really const. The HRESULT from D3DXCreateEffectFromFile is D3DXERR_INVALIDDATA.

Even if I have my dialog related code do NOTHING except open the dialog, it still causes the crash, so its not the code that is run as a result of browsing for a file.

I almost feel stupid asking this question because I don't see how these could possibly be related (aside from that hairbrained string table nonsense), but I've searched the web and noticed a few others with somewhat similar problems (deep hard to find errors when using GetOpenFileName)

Share this post

Link to post
Share on other sites
Original post by AnAss
I've figured this stupid problem out. GetOpenFileName changes the current directory.

With the right flag, no.

Share this post

Link to post
Share on other sites
Sign in to follow this  

  • Advertisement