Jump to content

  • Log In with Google      Sign In   
  • Create Account


#Actualxoofx

Posted 11 October 2012 - 06:38 AM

And even then, you can still use the FX file format - the difference is you'd have an offline compiler tool that parses the format for techniques/passes in order to acquire the necessary shader profiles + entry points. So with the profile/entry point in hand, you can run the FX file through the D3DCompiler to get a ShaderByteCode object for each shader, then use that to create a reflection object to query all your meta data. Then write out the reflected meta data to a file, that gets consumed by your application at runtime - which would be your own implementation of Effects11 (or something completely different, either way...you use the meta data to automatically setup your constant buffers, bind resources, and manage the shader pipeline by directly using the Direct3D11 shader interfaces).

For parsing, you can use something like Tiny Parser Generator, which is a really neat little tool if you know how to create a grammar. This is a problem that I've been working on myself for my own software and the approach has been working out pretty well (and frankly, I love TinyPG). I also believe the MonoGame folks have adopted this method, so that may be a good place to gather some ideas.

You are right, that is basically what has been done recently in SharpDX.Toolkit to support parsing of HLSL FX shaders:

- I decided to almost parse fx file format (only the content of technique/passes) in order to keep things quite compatible with legacy shaders and ease porting (though the syntax is slightly simplidied and different, check for example BasicEffect.fx link below)
- Write a parser, done by hand (nothing more than a couple of hundred of lines) and is mainly parsing technique/passes and the content of a pass, but the syntax supported is slightly closer to D3D9, producing an AST.
- Write a custom metadata fileformat (EffectData in the toolkit) that is able to save the whole description of a shader (shader bytecodes, constant buffers, value & resource variables, techniques/passes), nice thing is that It supports multiple effects inside a same archive that can be merged (like a zip of effects)
- Write a compilerthat takes the AST produced by the parser and compile all shaders and put them in the metadata format.
- A command line tool "fxc.exe" like called "tkfxc.exe" which basically allow to generate an archive of effects (unlike legacy FX, I can then put several effects inside the same tkfxo file), it is also outputing a friendly colored output as the original fxc.exe.
- Then you have to rewrite the whole Effect framework

Getting all this efficiently written is really really a long way (correct handling of constant buffers update, minimize calls to XXSetShaderResource/SetSamplers...for each shader stages...etc.), but in the end the result is pretty close to XNA effects (thus, I have been able to reuse all StockEffects from XNA, for example BasicEffect.fx is very similar to the original one, only techniques/passes declaration is slgihtly changing) but a bit more flexible as It can provide some extras features like linking of shader declared from another shader (export/import of a particular shader shareable between effects), or constant buffer pooling, or automatic access to all non-builtin variable declared in the effect pass at runtime (if you declare Toto = 1.0f; in the effect pass, you will be able to get this value from C#).

#1xoofx

Posted 11 October 2012 - 06:35 AM

And even then, you can still use the FX file format - the difference is you'd have an offline compiler tool that parses the format for techniques/passes in order to acquire the necessary shader profiles + entry points. So with the profile/entry point in hand, you can run the FX file through the D3DCompiler to get a ShaderByteCode object for each shader, then use that to create a reflection object to query all your meta data. Then write out the reflected meta data to a file, that gets consumed by your application at runtime - which would be your own implementation of Effects11 (or something completely different, either way...you use the meta data to automatically setup your constant buffers, bind resources, and manage the shader pipeline by directly using the Direct3D11 shader interfaces).

For parsing, you can use something like Tiny Parser Generator, which is a really neat little tool if you know how to create a grammar. This is a problem that I've been working on myself for my own software and the approach has been working out pretty well (and frankly, I love TinyPG). I also believe the MonoGame folks have adopted this method, so that may be a good place to gather some ideas.

You are right, that is basically what has been done recently in SharpDX.Toolkit to support parsing of HLSL FX shaders:

- I decided to almost parse fx file format (only the content of technique/passes) in order to keep things quite compatible with legacy shaders and ease porting (though the syntax is slightly simplidied and different, check for example BasicEffect.fx link below)
- Write a parser, done by hand (nothing more than a couple of hundred of lines) and is mainly parsing technique/passes and the content of a pass, but the syntax supported is slightly closer to D3D9, producing an AST.
- Write a custom metadata fileformat (EffectData in the toolkit) that is able to save the whole description of a shader (shader bytecodes, constant buffers, value & resource variables, techniques/passes)
- Write a compilerthat takes the AST produced by the parser and compile all shaders and put them in the metadata format.
- A command line tool "fxc.exe" like called "tkfxc.exe" which basically allow to generate an archive of effects (unlike legacy FX, I can then put several effects inside the same tkfxo file), it is also outputing a friendly colored output as the original fxc.exe.
- Then you have to rewrite the whole Effect framework

Getting all this efficiently written is really really a long way (correct handling of constant buffers update, minimize calls to XXSetShaderResource/SetSamplers...for each shader stages...etc.), but in the end the result is pretty close to XNA effects (thus, I have been able to reuse all StockEffects from XNA, for example BasicEffect.fx is very similar to the original one, only techniques/passes declaration is slgihtly changing) but a bit more flexible as It can provide some extras features like linking of shader declared from another shader (export/import of a particular shader shareable between effects), or constant buffer pooling, or automatic access to all non-builtin variable declared in the effect pass at runtime (if you declare Toto = 1.0f; in the effect pass, you will be able to get this value from C#).

PARTNERS