Direct3D11 dev->CreateVertexShader() crashes the program.

Started by
14 comments, last by unbird 10 years, 11 months ago

Good. But you don't check the shader compilation actually. Check that too. If the compilation fails, vBlob is NULL. Is it ? Because, if so, it's not even the CreateVertexShader that crashes, but dereferencing the NULL pointer with GetBufferPointer().

Also: Do you know how to use the interactive debugger ?

Advertisement

Good. But you don't check the shader compilation actually. Check that too. If the compilation fails, vBlob is NULL. Is it ? Because, if so, it's not even the CreateVertexShader that crashes, but dereferencing the NULL pointer with GetBufferPointer().

I can't really check it if it's NULL or not with a "==" operator. And yes you are correct, It crashes on


vBlob->GetBufferPointer();

Also: Do you know how to use the interactive debugger ?

No sad.png

Hmmm, now I realize that there's even more to it, since you dont initialize your pointers to anything. Always do that IMO:

    ID3D10Blob *vBlob = NULL;
Or nullptr if you're in for modern C++.

Anyway, if not it could be anything. In debug mode they get initialized to the magic number 0xCCCCCCCC (see here).

Checking for NULL should work with

    if(vBlob == NULL)
or even (Edit: corrected)

    if(!vBlob)
Why does it not work for you ?

Sooo, the why is it NULL (because I think it is) ? If the shader compilation fails, it will be set so (Note: I don't use VS 2012 but 2010, but I can reproduce that with the old D3DX function here).

Checking for the compilation works something like this (careful, I can't really check this, I just transliterated it for you):

ID3D10Blob *vBlob = NULL;
ID3D10Blob *errors = NULL;
HRESULT hr;
hr = D3DCompileFromFile(L"Shaders.hlsl",NULL,NULL,"VShader","vs_5_0",0,0,&vBlob,&errors);
if(FAILED(hr))
{
    if(errors)
    {			
        MessageBoxA(0, (LPCSTR)errors->GetBufferPointer(), "Error", 0);
        errors->Release();
    }
    else
    {
        MessageBoxA(0, "Compilation failed. Is the file where it should be ?", "Error", 0);
    }
    exit (EXIT_FAILURE);
}
If that code does not work, maybe someone else is so kind to correct it (Edit:Found already a typo, vBlob, not pBlob. Corrected)

So either your app cannot find your shader file, or there's an error in there.

Interactive debugging. Well, you could hit the 'Break' button when the crash occurs, you should be on the exact line where it happens. Move your mouse over the variables and be amazed (Hint: vBlob).

Thanks unbird.

It turns out to be the Shaders.hlsl file needed to be in the same directory as the "whatever.exe" and when I think about it from what you said earlier it does make sense since compilation is on runtime, right ?

But it's damn stupid, It crashed without the file at all, when I put it in the project source files (see videos) it compiled fine and even produced somekind of "Shaders.cso" file inthe directory of the "whatever.exe" yet crashed. How on earth was I supposed to know about this madness happening in a correct way even those something as being compiled angry.png

So yeah the problem is solved with moving "Shaders.hlsl" file. Thanks again.

By the way, I believe you over-thought(or forgot how C++ works) a bit when saying to check the Blob for NULL since you posted:


 if(vBlob == NULL)

Which compares the address of the pointer to NULL since we do no deference and I can't imagine the address itself ever being NULL from a typical pointer declaration when the pointer get's created under some address upon declaration and simply using it's name refers to that address.

Thanks unbird.

It turns out to be the Shaders.hlsl file needed to be in the same directory as the "whatever.exe" and when I think about it from what you said earlier it does make sense since compilation is on runtime, right ?

But it's damn stupid, It crashed without the file at all, when I put it in the project source files (see videos) it compiled fine and even produced somekind of "Shaders.cso" file inthe directory of the "whatever.exe" yet crashed. How on earth was I supposed to know about this madness happening in a correct way even those something as being compiled angry.png

So yeah the problem is solved with moving "Shaders.hlsl" file. Thanks again.

By the way, I believe you over-thought(or forgot how C++ works) a bit when saying to check the Blob for NULL since you posted:


 if(vBlob == NULL)

Which compares the address of the pointer to NULL since we do no deference and I can't imagine the address itself ever being NULL from a typical pointer declaration when the pointer get's created under some address upon declaration and simply using it's name refers to that address.

His point is that you should set a pointer to NULL when you initialize it(unless it's a class member then use the initialization list.) If you check it for NULL then later, that tells you if it was ever allocated to something else. Using something like new on a pointer allocates new memory and sets it to point to that, so it would no longer be null.

If it's still set to NULL then obviously whatever call you used to allocate it earlier in the program never happened. You could also use an assert technically if it SHOULDN'T be null at that point.

That said if you don't initialize all your pointers to NULL then they're going to contain some crap value that will be interpreted as an intentional address and thus NULL will never trigger.

Precisely. Admittedly, I made a mistake earlier, the statements are not equivalent, it should be if(!vBlob). Corrected.

Fun fact: The Visual Studio debugger will complain when using an unitialized/unused pointer (it likely checks for that that magic number). Also: It was really meant as a question or suggestion to try out yourself (because asserts or - as in this case - proper error checking are the way to go). If you want us to help you give us feedback and try our approaches. Remote debugging through a forum is not simple. Making a video is overkill, and Zaoshi Kaba was right: The video did not show more than what we already knew and suggested approaches.

Which brings me back to the other question: You really need to learn to use the interactive debugger. I already started fabricating a screenshot but heck, we're in the age of
">youtube

This topic is closed to new replies.

Advertisement