Jump to content

  • Log In with Google      Sign In   
  • Create Account


D3D9 CreateDevice crash with D3DCREATE_HARDWARE_VERTEXPROCESSING


Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.

  • You cannot reply to this topic
8 replies to this topic

#1 ~0ul   Members   -  Reputation: 607

Like
0Likes
Like

Posted 12 July 2013 - 01:10 AM

Hi,

 

I'm seeing a crash inside CreateDevice in an old code base when initializing a d3d device with the D3DCREATE_HARDWARE_VERTEXPROCESSING flag. The d3d initialization is contained in its own thread which is contained in a native dll being called from managed code. I am able to run with D3DCREATE_SOFTWARE_VERTEXPROCESSING or D3DCREATE_MIXED_VERTEXPROCESSING but D3DCREATE_HARDWARE_VERTEXPROCESSING always crashes inside CreateDevice. These flags are always OR'd with D3DCREATE_MULTITHREADED.

 

There are no error messages returned as the code crashes inside the function. Sometimes in gdi32.dll (and a few times in nvd3dum.dll). It is always a stack overflow exception caught by a chckstck in the thread running the dll. I don't have source for these so don't know whats going on. I'm on the latest NVIDIA drivers and have tried both debug and release d3d9 runtimes (Note: This used to work on older/last year's drivers).

 

I know the card (GTX 460 SE) supports hardware vertex processing and I am able to get the code with the same creation parameters to work inside the d3d sample demo app. I tried increasing the thread stack size from the default 1mb on windows but this had no effect on the crash either.

 

Anyone seen anything like this before or know what could be causing this ? or anything else I can do to track the problem ?

 

Thanks!

 



Sponsor:

#2 Tom KQT   Members   -  Reputation: 1508

Like
0Likes
Like

Posted 12 July 2013 - 02:47 AM

Did you anyway (even if you know the card should support HW vertex processing) try to verify it by checking for D3DDEVCAPS_HWTRANSFORMANDLIGHT in GetDeviceCaps? Just for sure ;)



#3 ~0ul   Members   -  Reputation: 607

Like
0Likes
Like

Posted 12 July 2013 - 11:53 AM

Thanks for the suggestion. Yup, just tried it:

D3DCAPS9 Caps;
sys->GetDeviceCaps(D3DADAPTER_DEFAULT, D3DDEVTYPE_HAL, &Caps);
if( Caps.DevCaps & D3DDEVCAPS_HWTRANSFORMANDLIGHT )
     return true
return false;

returns true.

 

What is D3DCREATE_HARDWARE_VERTEXPROCESSING anyway ? Does this mean the entire geometry stage (vertex fetch, vertex shader etc) runs in software or just the old fixed function stages ? I can even get it to work with D3DCREATE_MIXED_VERTEXPROCESSING which makes the CPU run a lot faster so I'm assuming its doing all the heavy lifting on the gpu. I just don't know why its crashing with D3DCREATE_HARDWARE_VERTEXPROCESSING. 



#4 Tom KQT   Members   -  Reputation: 1508

Like
0Likes
Like

Posted 12 July 2013 - 01:12 PM

Hm that's a good question :) I also don't know what exactly does HARDWARE_VERTEXPROCESSING include. Vertex transformations and fixed-function lighting for sure, that's the good old HW T&L, I remember when this came out and some games then started to require GPUs capable of it :)

Probably if you use D3DCREATE_SOFTWARE_VERTEXPROCESSING with a vertex shader, then the shader is executed on the CPU?



#5 ~0ul   Members   -  Reputation: 607

Like
0Likes
Like

Posted 12 July 2013 - 11:42 PM

Yeah, that's what I'm assuming. Software VP would mean vertex shader emulated on the CPU whereas mixed picks hardware if it can otherwise fall backs to software ?

 

Maybe that's why it fixed my crash, by delaying some initialization to happen sometime later which causes the crash to not happen.



#6 BMW   Members   -  Reputation: 160

Like
-2Likes
Like

Posted 13 July 2013 - 02:03 AM

If you are getting a stack overflow, increase the size of the stack.



#7 mhagain   Crossbones+   -  Reputation: 7468

Like
1Likes
Like

Posted 13 July 2013 - 05:15 AM

If you are getting a stack overflow, increase the size of the stack.

 

And then the next time you get an overflow you'll also increase it, and you'll find youself in an arms race with your code.

 

Better to find out what's causing the overflow and fix it, such as by running your code in a debugger, which will break on the overflow and let you see the call stack, inspect variables, etc in order to determine the cause.

 

OP - you've determined that the overflow happens inside either GDI or a driver DLL; that doesn't mean that they're the direct cause of this bug (otherwise it would affect every program) - far more likely is that you've built up such a huge stack in your own code that by the time it comes to call into these it's triggering the overflow.  The fix is to not build up such a huge stack in your own code.

 

Two common causes of stack overflows are recursion gone wild or declaring excessively large local variables.  For example, declaring "float stuff[512 * 512]" as a local variable will cause a stack overflow with the default 1mb stack.  I'd advise that you check your code up to device creation point for these two.

 

A third possible cause is passing large structs by value, but that's less frequently seen as most people know that they should be passed by reference or const reference.

 

So the first thing to do is set a breakpoint on the first line of your device creation function, run in the debugger, see the call stack up to that point, and check the functions in that call stack for anything that may be contributing to excessive stack usage.


It appears that the gentleman thought C++ was extremely difficult and he was overjoyed that the machine was absorbing it; he understood that good C++ is difficult but the best C++ is well-nigh unintelligible.


#8 ~0ul   Members   -  Reputation: 607

Like
0Likes
Like

Posted 13 July 2013 - 01:25 PM

Thanks for the reply mhagain. I did take a quick look and the code around the device creation looked ok, but I'll check again ( I also tried increasing the thread stack just as a test but I still got the crash, so I was thinking something might just be blowing the stack somewhere (infinite recursion, or huge allocation on the stack) ). Yeah, I'm assuming the fault is in our code and not in the drivers. I got the sample app to work correctly, however I haven't tried spawning a thread from a native dll called from managed code and creating a device from there yet.

 

I was curious if the CreateDevice function might have some callbacks into user code ? If so, what would these callbacks be. for eg: the d3dxeffect system has user callbacks which get called from inside the d3d dll's. Maybe create device is calling such a callback which is causing the stack overflow. If so what kind of usage would trigger a callback from inside CreateDevice ? I can't step into this function so I'm not sure how to find out.



#9 ~0ul   Members   -  Reputation: 607

Like
0Likes
Like

Posted 15 July 2013 - 03:55 PM

actually its probably not a callback blowing the stack as I would see it in the callstack, still could be a stomp though.






Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.



PARTNERS