Hi, I'm trying to use a simple hlsl file in the rendering, so I compile my .fx with the fxc.exe as directed on msdn, it produces a clean object with no errors. I read it into memory using msdn's method, but for some reason this function still comes back with E_FAIL. Here's some of my code:

[source]ifstream is(TEXT("DX10RendererEffect.fxo"), ios::binary);
is.seekg(0, ios_base::end);
streampos pos = is.tellg();
is.seekg(0, ios_base::beg);
char * effectBuffer = new char[pos];
is.read(effectBuffer, pos);
HRESULT hr;
if (FAILED(hr = D3D10CreateEffectFromMemory((void*)effectBuffer, pos, 0, softObjPtr->pD3D10Device, NULL, &softObjPtr->pD3D10Effect))){
return;

}[/source]

Also, I've created my device using the D3D10DEBUG_DEVICE flag (something like that), but the function doesn't say anything besides E_FAIL, that on msdn says that means the debug layer is not installed.

I don't know if this matters but I'm on a fresh in stall of Windows 7 and I just installed Windows 8.1 SDK. It's a hassle because all the D3DX helper functions are removed, so I can't compile from file, and must use a build rule on visual studio to compile my fx files manually. And the D3D sdk is completely merged with Windows SDK, so I can't tell if the debug layer was installed or not, it should have been right? Any suggestions?

You don’t release effectBuffer on failure, which is a memory leak.

Did you compile the effect with the correct version?
Did you verify that it compiled correctly?
Did you verify that the file exists?
Did you add your program to the debug-layer list of executables and enable debugging of it?


L. Spiro

I'm trying to get it to work, so memory leaks are second priority right now.

Here is my compilation string in the build tool:

"C:\Program Files (x86)\Windows Kits\8.1\bin\x86\fxc.exe" /Od /Zi /T fx_4_0 /Fo "C:\Users\Tim\Documents\Visual Studio Projects\SoftwareRasterizer\DX10RendererEffect.fxo" "C:\Users\Tim\Documents\Visual Studio Projects\SoftwareRasterizer\DX10RendererEffect.fx"

this is the end of my .fx file

[source]

[/source]

I'm not sure how to add the program to the dubug layer though, that might be helpful if it gives me a more elaborate fail string. How exactly do I do that?

Can you try to retrieve the HRESULT error string?

Something like this:

void PrintDebugOutput(const LPCWSTR pText, const HRESULT pHr)
{
	OutputDebugString(pText);
	OutputDebugString(DXGetErrorString(pHr));
	OutputDebugString(L", ");
	OutputDebugString(DXGetErrorDescription(pHr));
	OutputDebugString(L"\n");
}

Can you try to retrieve the HRESULT error string?

Something like this:

void PrintDebugOutput(const LPCWSTR pText, const HRESULT pHr)
{
	OutputDebugString(pText);
	OutputDebugString(DXGetErrorString(pHr));
	OutputDebugString(L", ");
	OutputDebugString(DXGetErrorDescription(pHr));
	OutputDebugString(L"\n");
}

So I get "E_FAIL" out geterrorstring
and "An undetermined error occurred" at the errordescription

Another thing is that I installed the wrong SDK originally. I got the windows 8.1 sdk, even though I'm on windows 7. I'm not sure if it's a big deal but just to be safe I uninstalled the windows 8.1 sdk and installed windows 7.1 instead, along with June 2010 DirectX SDK. Otherwise the above function you suggested doesn't even exist as windows 8 merged dx with the os, so you have to use the regular os error reporting function. Anyways with that taken care of, it's the same issue and I STILL can't get DX to spit out debug information. What am I doing wrong??? This is how I create my device:

[source]if (FAILED(D3D10CreateDeviceAndSwapChain(NULL, D3D10_DRIVER_TYPE_HARDWARE, NULL, D3D10_CREATE_DEVICE_DEBUG, D3D10_SDK_VERSION, &softObjPtr->D3D10SwapChainDesc, &softObjPtr->pD3D10SwapChain, &softObjPtr->pD3D10Device)))
{
ZeroMemory(&softObjPtr->D3D10SwapChainDesc, sizeof(DXGI_SWAP_CHAIN_DESC));
return;
}[/source]

I also selected the program "SoftwareRasterizer" in my debug folder as the target in the DX Control Panel, should I instead select the VS 2012 executable?

I'm trying to move my direct draw rasterizer to DX10 and this is proving to be quite a hassle, lol.

I'm going to try using the old DX functions to compile my effect instead of the FXC and I'll update on how it goes, but I would still appreciate the help in trying to get my debug output to work ))

OK so rewrote the code with D3DX10CreateEffectFromFile and it seems to work, so I'm thinking something was off about the object created by fxc compiler, judging from another forum where someone had a similar issue it pads the size of the file, but not sure, it was a post from 2008. Thanks for the help.