Compiling a Shader, at runtime, in D3D9, without D3DX

Started by
6 comments, last by RonHiler 13 years, 4 months ago
Well, the title says it all.

I figured I could use D3DCompile() from the D3DCompiler.lib (as I do with DX10+), but apparently I cannot (it compiles the shader without error, but fails with a 'D3DERR_INVALIDCALL' when you try to create the shader via CreateVertexShader()). I guess D3DBlob can only be used with DX10+ (according to the docs).

The shader itself works fine under the other renderers, so there is nothing wrong with it. It's just a matter of getting it to compile under D3D9

So what alternative do I have here?
Creation is an act of sheer will
Advertisement
Other than using D3DX or not compiling at runtime (i.e. loading in compiled shaders from disc) I don't think there is another option for D3D9.

D3DX should get installed with your game anyway, or you can statically link an ancient version. Why do you want to avoid using it?
I wonder whether your problem stems from the fact that you are using raw shaders. It would be interesting to know what the debug runtimes have to say about that invalid call, that is hopefully more conclusive.

I'm telling you this because I successfully switched to D3DCompile (in SlimDX, on a Windows XP) to create blobs for the DX 9 effect system using "fx_2_0" as target profile. Haven't used raw shaders yet, so bear with me, but I'd be surprised if it's only working for full effect blobs. Chances are you got some flags wrong. Can you show us the shader code in question - or tell more about it's characteristics - and how you actually compile and create it ?

Also: Consider using the command line compiler (or even alternatives like Cg) in a sub-process, if that is applicable in your context.
Quote:Original post by unbird
I wonder whether your problem stems from the fact that you are using raw shaders. It would be interesting to know what the debug runtimes have to say about that invalid call, that is hopefully more conclusive.

Yeah, I can do that. I'll load them up and see what they say.

Quote:I'm telling you this because I successfully switched to D3DCompile (in SlimDX, on a Windows XP) to create blobs for the DX 9 effect system using "fx_2_0" as target profile.

Really? Hmmmm, that's very interesting.

Quote:Haven't used raw shaders yet, so bear with me, but I'd be surprised if it's only working for full effect blobs. Chances are you got some flags wrong. Can you show us the shader code in question - or tell more about it's characteristics - and how you actually compile and create it ?

Well, okay, but it's a bit convoluted. I'll do my best. I'll show you the vertex shader, since that's the first one created (and thus the one that is failing). This shader works fine under D3D10 and higher.

String DefaultVertexShader("cbuffer cbPerObject{float4x4 WorldMatrix;};cbuffer cbPerViewport{float4x4 ViewMatrix;float4x4 ProjectionMatrix;};void VS(float3 Position: POSITION, float4 Color: COLOR, out float4 oPosition: SV_POSITION, out float4 oColor: COLOR){oPosition = mul(float4(Position, 1.0f), mul(mul(WorldMatrix, ViewMatrix), ProjectionMatrix));oColor = Color;}");

Nothing fancy, just a default test shader. This string is sent to the compiler through my wrapper manager class, like such:

switch (CurrentRenderer){  case ENZ_DIRECT3D9:  {    Flags = 0;    Result = DX9Wrapper::CompileShader(Source, &EntryFunction, &ShaderProfile, Flags, reinterpret_cast<ID3DBlob**>(CompiledShader), &DXErrors);    if (Result != S_OK)    {      Size = DXErrors->GetBufferSize();      Buffer.Reserve(Size);      Buffer = static_cast<char*>(DXErrors->GetBufferPointer());      OutputDebugString(Buffer.c_str());      DXErrors->Release();    }  return;}


I'm not sending any flags at all for the DX9 compiler (for the DX10+, I send D3D10_SHADER_ENABLE_STRICTNESS).

HRESULT DX9Wrapper::CompileShader(String *Source, String *MainFunction, String *ShaderProfile, unsigned int Flags, ID3DBlob **Shader, ID3DBlob **Errors){return D3DCompile(Source->c_strA(), Source->Size(), nullptr, nullptr, nullptr, MainFunction->c_strA(), ShaderProfile->c_strA(), Flags, 0, Shader, Errors);}


And that is the actual compile function. The creation of the vertex shader is the same sort of thing.

switch (CurrentRenderer){  case ENZ_DIRECT3D9:  {    Result = DX9Wrapper::CreateVertexShader(static_cast<IDirect3DDevice9*>(Device),         static_cast<const DWORD*>(static_cast<ID3DBlob*>(CompiledShader)->GetBufferPointer()),         reinterpret_cast<IDirect3DVertexShader9**>(Shader));    if (Result != S_OK)      Assert(0, "CreateShader: Failure to create vertex shader");    return;  }


HRESULT DX9Wrapper::CreateVertexShader(IDirect3DDevice9 *D3D9Device, const DWORD *CompiledShader, IDirect3DVertexShader9 **VertexShader){return D3D9Device->CreateVertexShader(CompiledShader, VertexShader);}


Note that I'm going to be changing the whole switch-based part of the wrapper manager over to a pointer system, but that is neither here nor there, I only mention it so no one yells at me about it :)

[EDIT: sorry about the formatting, I tried to clean it up a bit]

[Edited by - RonHiler on December 10, 2010 9:49:42 AM]
Creation is an act of sheer will
Thanks.

Ok, what profile (ShaderProfile) do you use ? I guess: Since you are using cbuffer and system value semantics (SV_POSITION) you are compiling against (at least) shader model 4. Such a shader cannot be loaded by a DX 9 device, it has to be SM 3.0 or lower. SM 4 is DirectX 10 upwards. This translates to the vs_3_0 profile for your vertex shader. I can figure that invalid call.

If you want to use DX 9 you will have to rewrite your shader(s). But first check if my approach also works for raw shaders before you waste time for a dead end.
Yes you are right, I am using vs_4_0. Okay, so I'll have to write a shader that will compile under vs_3_0. That's easy enough. I didn't realize DX9 wouldn't take 4.0. I guess I should have looked into that before assuming.

So many different things to take into consideration when using different rendering APIs, heh.
Creation is an act of sheer will
Unfortunately you're going to have to a lot of #define'ing to get code compiling as both *s_3_0 and *s_4_0. D3D10_SHADER_ENABLE_BACKWARDS_COMPATIBILITY allows you to use some of the older semantics in *s_4_0, but I don't think it covers everything.

I remember there was a presentation way back when D3D10 had first come out about targeting both API's...I think it was by the guys who did Hellgate: London. It was probably GameFest.
I got the vertex and pixel shaders such that they will compile under 3.0, 4.0, and 4.1. So everything is good now. All of the renderers are working again except D3D9 (which I have to do a little more work on in other areas to finish up).

But that's just because they are such simple basic shaders. I suspect when things start getting a little complex, I'm going to have to break them apart into separate shaders rather than try to compile one shader under all three models. That's probably a better idea anyway, since otherwise I'd essentially be crippling 4.1 shaders just for them to be 3.0 compatible. And there is no reason to do that.

Thanks for the advice guys.
Creation is an act of sheer will

This topic is closed to new replies.

Advertisement