Sign in to follow this  
InfoGeek

Compiling and using shaders

Recommended Posts

InfoGeek    244

hi all.

 

i do not understand compiling and using shaders good enough. which i tried to ask here . but i guess it is a question for a new thread.

 

I tried to find a a tutorial to explain how switching of shaders and their compiling works, like what gets replaces when switching a vertex shader, what happens to global variables declared in one shader file when you switch to another and just how it all comes together.

 

i did no find such a tutorial. i tried the official msdn site Programming Guide for HLSL and Compiling Shaders or get general overviews from googling Introduction to DirectX Shader. nothing explains that.

 

can anyone please point to a resource or explain?

Share this post


Link to post
Share on other sites
SyncViews    734

I am not sure what you mean? If just talking about the compiling and loading of a shader.

 

Pre-compiled shaders:

  1. Creating shaders in VS2013 by adding a new item, Visual C++ -> HLSL then picking the type of shader I want.
  2. These will compile to .cso files
  3. Loading the .cso file into memory
  4. Using ID3D11Device::CreateVertexShader etc. to turn that loaded cso file into one of the shader objects such as ID3D11VertexShader.

Runtime-compiled shaders:

  1. Get/load your shader code
  2. Use D3DCompile or D3DCompile2 to compile the shader to byte code (the same as the .cso file)
  3. Use ID3D11Device::CreateVertexShader etc. to create your shader object.

 

By global variables I guess you means constants (ID3D11Context::VSSetConstantBuffers etc.)?  In which case the constants for each type of shader exist separately from whatever the current shader is. If you change shaders and have different constants, you need to set them again.

 

If you are using the effects system, use D3DX11CreateEffectFromMemory instead of the CreateVertexShader etc. I understand that ID3D11XEffectPass will call all the individual ID3D11Context:: set methods automatically, including putting the effects variables back as the GPU's active constants/variables/globals.

Edited by SyncViews

Share this post


Link to post
Share on other sites
HappyCoder    5053

When a shader is compiled, it references texture slots, not textures. Same thing applies for constant buffers and samplers. The shader is compiled with code that references a slot. You then attach textures, buffers, and samplers to these slots. So changing a shader does nothing to change a texture.

 

Even though this is specifically a DirectX question I feel like I should mention OpenGL. It works a little differently. In OpenGL, any uniforms are bound to the actual program and change when you change a program but texture slots and constant buffers (aka uniform block) work the same.

Share this post


Link to post
Share on other sites
InfoGeek    244

what happens if you have 2 shaders for normal rendering and 2 shaders for font rendering? like... what happens to global declarations which as HappyCoder said refer to slots which you bound actual resources to later?
 
right now i have 1 file with two shader(following a tutorial):

cbuffer CBPerObject
{
	float4x4 WVP;
};


Texture2D objTexture;
SamplerState objSamplerState;

struct VS_OUTPUT
{
	float4 position : SV_POSITION;
	float2 texturePosition : TEXPOS;
};

// Vertex Shader.
VS_OUTPUT VS(float4 inPos : POSITION, float2 inTexPos : TEXPOS)
{
	VS_OUTPUT output;
	output.position = mul(inPos, WVP);
	output.texturePosition = inTexPos;
	return output;
}


// Pixel Shader.
float4 PS(VS_OUTPUT input) : SV_TARGET
{
	float4 textureColor = objTexture.Sample(objSamplerState, input.texturePosition);
	return textureColor;
}

if i split it into 3 files, 1 normal PS, 1 normal VS and 1 font PS. i would only change the PSes, where would i declare this:

Texture2D objTexture;
SamplerState objSamplerState;

struct VS_OUTPUT
{
	float4 position : SV_POSITION;
	float2 texturePosition : TEXPOS;
};

? and at which point do they get compiled and how... do they get compiled everytime i compile a shader... i don't even know?

Share this post


Link to post
Share on other sites
megadan    789

You can place code used by multiple shaders into header files and then include them in each shader that uses it.  So for example, you could place the VS_OUTPUT struct into some header file like "ShaderCommon.hlsli" and then add #include "ShaderCommon.hlsli" to each file that needs access to it.  It will then be compiled into each shader.

Share this post


Link to post
Share on other sites
InfoGeek    244

You can place code used by multiple shaders into header files and then include them in each shader that uses it.  So for example, you could place the VS_OUTPUT struct into some header file like "ShaderCommon.hlsli" and then add #include "ShaderCommon.hlsli" to each file that needs access to it.  It will then be compiled into each shader.

 

is that how everything is done by standard/industry?

 

Edit: wait a minute... do shader files allow pre-processor directives? i'm compiling using D3DCompileFromFile.

Edited by InfoGeek

Share this post


Link to post
Share on other sites
megadan    789

Yes it's pretty common to use includes to share code between the shaders. You can also use it for any shared functions.

 

Pre-processor directives are allowed. See http://msdn.microsoft.com/en-us/library/windows/desktop/bb943993(v=vs.85).aspx

 

Defines can be set with the pDefines parameter of D3DCompileFromFile. See http://msdn.microsoft.com/en-us/library/windows/desktop/hh968107(v=vs.85).aspx for an example.

Edited by megadan

Share this post


Link to post
Share on other sites
InfoGeek    244

wait... ultimately it's the same as simply copy pasting the shared code/declarations. so you should treat each shader as a separate program with copies of all declarations?

Share this post


Link to post
Share on other sites
megadan    789

Yeah it will end up being compiled to the same code so it could just be copied instead of included, but then if you need to make a change to the copied code, you'll need to change every place that uses it. It kind of depends on the extent of the shared code and the likelihood of it to change.

Share this post


Link to post
Share on other sites
InfoGeek    244


Yeah it will end up being compiled to the same code so it could just be copied instead of included

 

no mechanism to have shared declaractions across shaders?

Share this post


Link to post
Share on other sites
megadan    789

What did you have in mind? If you were to declare a cbuffer or something in a header and then include it in multiple shaders, while the shader compiler will ultimately copy the declaration to both shaders, the actual resource can be shared from C++ like this:
 

context->PSSetConstantBuffers( 0, 1, &buffer );
context->PSSetShader( shader1, NULL, 0 );
... do some drawing ...
context->PSSetShader( shader2, NULL, 0 );
... do some more drawing ...

Here the buffer is bound to slot 0, which will apply to any activated pixel shader until PSSetConstantBuffers is called again. The result would be the same if you called PSSetConstantBuffer again with the same buffer before using shader2, but it is better to try to reduce API calls by preventing redundant state changes.  Setting up a system to handle checking for redundancies and only calling API functions when necessary is ideal.

Edited by megadan

Share this post


Link to post
Share on other sites
InfoGeek    244

sorry for dragging this thread for a bit.

 

 

Defines can be set with the pDefines parameter of D3DCompileFromFile. See http://msdn.microsoft.com/en-us/library/windows/desktop/hh968107(v=vs.85).aspx for an example.

 

 

there was no preprocessor/include related code in example .hlsl files, so if i go the runtime compiling of shader way i would have to pass a: 

D3D_SHADER_MACRO Shader_Macros[1] = { "FONT_COMMON", "1"  };

having something like:

 

font.hlsli

#if DEFINED(FONT_COMMON)

struct commonStruct
{
   int whatever;
   float whatever2;
}

#endif

to every compiling call of a shader that includes that common include like so:

 

font_ps.hlsl

#include "font.hlsli"

float4 PS(VS_OUTPUT input) : SV_TARGET
{
   // do pixel shader stuff...
}

Call (Second parameter Shader_Macros): 

D3DCompileFromFile(L"font_ps.hlsl", Shader_Macros, 0, "PS", "ps_5_0", 0, 0, &PS_Buffer, 0);

right?

 

because 

#ifndef COMMON_H
#define COMMON_H

struct commonStruct
{
   //... whatever ....
}

#endif

would not make sense because it's run-time right?

 

 

If so seems a bit messy.

Share this post


Link to post
Share on other sites
Buckeye    10747

because

#ifndef COMMON_H
#define COMMON_H

struct commonStruct
{
//... whatever ....
}

#endif

would not make sense because it's run-time right?

 

Two sides of the same coin. If the above code appears in a file (either in the file or #include'd), you could define COMMON_H to exclude the code blink.png , which is counter-intuitive. (Not sure what you mean by "run-time.") FYI, though that's often used in header files to prevent (well.. attempt to avoid) multiple defines, it's still just a ifndef/endif construct.

Edited by Buckeye

Share this post


Link to post
Share on other sites
InfoGeek    244

Two sides of the same coin. If the above code appears in a file (either in the file or #include'd), you could define COMMON_H to exclude the code , which is counter-intuitive. (Not sure what you mean by "run-time.") FYI, though that's often used in header files to prevent (well.. attempt to avoid) multiple defines, it's still just a ifndef/endif construct.

 

i have no idea what you tried to say here. what's the point of "#if DEFINED(FONT_COMMON)   //....    #endif" then?

 

run-time as in program run-time compilation of shaders instead of program compile time.

Edited by InfoGeek

Share this post


Link to post
Share on other sites
Buckeye    10747


i have no idea what you tried to say here. what's the point of "#if DEFINED(FONT_COMMON) //.... #endif" then?

 

Just observing that #ifndef COMMON_H is the same in kind as an #ifdef FONT_COMMON. Interpretation takes place at shader compile time, whenever that occurs.

Share this post


Link to post
Share on other sites
megadan    789

so if i go the runtime compiling of shader way i would have to pass a ... to every compiling call of a shader that includes that common include like so:


Generally, this is correct.  Each shader will have its own set of defines.  The same is true for C++.  The compiler passes some set of defines to each individual file as it gets compiled.  

 

In C++,#ifndef/#define is used to prevent a cpp file from including two headers that both include the same header which would lead to something being defined multiple times in the cpp file.  The same is true for shaders.  So if your shader header is being included in other headers that are then being included in your shader, it's a good idea to put an #ifndef/#define to prevent multiple declarations in the same file.  If you are just including the header file directly in your shader, then it's not totally necessary, although it might still be good practice.

 

As to which method you described you should use, it depends on what you want to accomplish.  If you want to prevent things from being multiply defined, use #ifndef/#define.  If you want to conditionally use one struct or another struct, or one block of code or another block of code, use the #ifdef with the define passed to the compile function.

 

One usage example for passing a define to the compile function would be to use a different vertex layout struct for passing into the vertex shader.  You can define two structs, say one with normals and the other without normals and use an #ifdef to control which is used.  Then compile the shader twice, one with the value defined, and the other without. Now you have two shader objects, one that can handle vertex inputs with normals, and one without.  This allows you to write one shader that you can use in multiple ways with different types of geometry.

Share this post


Link to post
Share on other sites
InfoGeek    244

thanks for the explanation again megadan, i knew some of it but it helps to be reassured.

 


If you want to prevent things from being multiply defined, use #ifndef/#define.

 

i want to follow your suggestion to include common header in shaders, but if i use #ifndef/#define at runtime how will it even work?

 

i mean at program compile time(as you said, i think...) the compiler keeps track of the #define so the #ifndef/#define method works and duplicates are excluded. if it is program runtime call to compile would it always include all files anyway because it has no way of tracking the #define ?

Share this post


Link to post
Share on other sites
megadan    789

Ah ok, I think I see what you're asking now.  Compiling a shader at your program runtime with D3DCompileFromFile is actually no different from compiling the shader offline from the command line using fxc.exe.  The same steps are performed and the same shader bytecode is generated.  So using #ifndef/#define will work exactly the same in both cases.

 

Edit: The shader compilation step (either through D3DCompileFromFile or fxc.exe) is what handles processing the defines.  So the defines work the same both ways.  Your actual application runtime or compilation doesn't affect it.

Edited by megadan

Share this post


Link to post
Share on other sites
InfoGeek    244

yeah but...

 

say i have:

 

common.hlsli

#ifndef POOP_H
#define POOP_H

// whatever shared code goes here.

#endif

and i have two shader that use that include:

 

shader1.hlsl

#include "common.hlsli"

// Whatever entry.

shader2.hlsl

#include "common.hlsli"
#include "common.hlsli"

// Whatever entry.

then:

D3DCompileFromFile(L"shader1.hlsl", 0, 0, "PS", "ps_5_0", 0, 0, &PS_Buffer, 0);

D3DCompileFromFile(L"shader2.hlsl", 0, 0, "PS", "ps_5_0", 0, 0, &PS_Buffer2, 0);

this is all runtime, so will the common.hlsli be included 3 times ? 2 ? 1? how does this happen? would it be different if it were at compile time?

Share this post


Link to post
Share on other sites
megadan    789

shader1.hlsl and shader2.hlsl will both have exactly one copy of common.hlsli.  This works because D3DCompileFromFile is performing all of the steps necessary to make that happen.  

 

It opens shader1.hlsl, parses the #include, opens common.hlsli, defines POOP_H, copies the contents into shader1.hlsl, and then compiles the shader and generates the bytecode which is returned in PS_Buffer.  

 

For shader2.hlsl, it parses the first #include, opens common.hlsli, defines POOP_H, copies the contents into shader1.hlsl, parses the second include, POOP_H is already defined so the contents are not copied again, and then compiles the shader to bytecode.

 

This is a slight simplification from the actual steps, but this is essentially what happens.  Each shader file compilation has its own local set of defines which is why POOP_H wasn't defined at first for shader2.hlsli and the contents were able to be copied.

Share this post


Link to post
Share on other sites
InfoGeek    244

great stuff megadan laugh.png i am very grateful for your patience with me. i believe this thread has all the answers to compile and use shaders. i'm sure it will be useful to any beginner who comes here googling.

 

once again thanks everyone!

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this