• Advertisement
Sign in to follow this  

DX11 DX11 Shader Generation

This topic is 1797 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

This is just a question of curiosity by the way. wink.png

 

So some few weeks ago I was curious on how to render an object multiple times with different to archive different results, and the only problem is that if I try to use additive blending, I get a black object.  Now to the point,  because of this, a guy said to me:

 

"You could also just write a single shader that performs the appropriate computations for the diffuse term, direction lighting term, and reads the color from the read and modulates it by the lighting terms."

 

Now this is a good idea, but i wanted to give my engine the freedom to render any shaders together (render 1, render 2, etc...), and by that it would be easier! But he is right...

 

So I though, why not generate a shader, like:

String Diff  = "color = somecolor"
String Light = "...color += something..."

PixelShader += Diff;
PixelShader += Light;

WholeShader = Buffers + Layout + Vertex + Pixel;

Compile WholeShader

Now could something like this be made, due to flexibility, and is this how real material editors work? (E.g. UDK Material Editor)

 

Thank You

Share this post


Link to post
Share on other sites
Advertisement

Yes, that is the basic idea behind node-based shader editors like the one in UDK. Once you start implementing it though I think you'll find that it can get pretty hairy, due to complex interactions between the various features that you support. Personally I think it's easier (and better overall) to go with an ubershader approach where you just turn features on and off.

Share this post


Link to post
Share on other sites

Yes, that is the basic idea behind node-based shader editors like the one in UDK. Once you start implementing it though I think you'll find that it can get pretty hairy, due to complex interactions between the various features that you support. Personally I think it's easier (and better overall) to go with an ubershader approach where you just turn features on and off.

 

How exacly does this ubershader work(I've heard of it, but never wen't into it)? Is it a framework?

Share this post


Link to post
Share on other sites

"Ubershader" is just a name for a general pattern for authoring shader permutations. Typically you will write one large vertex and pixel shader with all possible features implemented, and then you will wrap the corresponding code for a certain features in "#ifdef" preprocessor statements. This allows you to compile different permutations of your shader with various features enabled or disabled by passing different macro definitions when compiling the shader. Here's a really simple example:

 

 

#ifdef ENABLE_COLOR_TEXTURE
Texture2D ColorTexture;
SamplerSTate ColorTextureSampler;
#endif

struct PSInput
{
    float4 VertexColor : VTXCOLOR;

#ifdef ENABLE_COLOR_TEXTURE
    float2 UV          : UV0;
#endif
};

float4 PSMain(in PSInput input) : SV_Target0
{
    float4 color = input.VertexColor;

#ifdef ENABLE_COLOR_TEXTURE
    color *= ColorTexture.Sample(ColorTextureSampler, input.UV);
#endif

    return color;
}


This is pixel shader for an ubershader that has 2 permutations: one with a color texture enabled, and one without. So you would compile one where you'd specify that you want ENABLE_COLOR_TEXTURE defined as an extra preprocessor macro, and one where you wouldn't. Then you can switch between the two at runtime depending on what you need for a particular mesh. Or alternatively you can have your features be options when authoring a material, and then you can specifically compile a shader permutation for that material. Typically you'll have lots of options that you want to disable, such as lighting, normal mapping, skinning, etc. Supporting all possible permutations of a bunch of on/off options then requires you to compile 2^N shaders, where N is the number of options you want to support. A common way index into all of these shaders is with a bitfield, where each the value of each bit corresponds to a feature being turned on or off. This gives you a simple hash that you can use to lookup into std::map or a similar data structure.

You can also support enabling or disabling features by wrapping them in regular if statements, and passing a bool through a constant buffer to specify whether you want them on or off. This saves you from having to compile different shaders and switch between them at runtime, which can save you build time and can also potentially save you some CPU time due to driver overhead from switching shaders. However in generally it will result in less optimal compiled shader code, since the compiler will be unable to fully optimize out the code in a branch that's not taken (or any code that produces results required for the branch not taken). This can result in extra instructions being executed, and potentially higher register usage which reduces thread occupancy. There's also some cost to actually executing a branch instruction, although this is generally small on modern DX11-capable hardware. The one major limitation of branches is that you can't use them to disable vertex shader inputs. Here's a simple example showing what I mean:

 

cbuffer VSConstants
{
    float4x4 World;
    float4x4 WorldViewProj;
}

#ifdef ENABLE_SKINNING
static const uint MaxBones = 256;

cbuffer SkinningConstants
{
    float4x4 Bones[MaxBones];
}

struct VSInput
{
    float3 Position : POSITION;
    float3 Normal : NORMAL;

#ifdef ENABLE_SKINNING
    uint4 SkinIndices : SKININDICES;
    float4 SkinWeights : SKINWEIGHTS;
#endif
};

struct VSOutput
{
    float4 Position : SV_Position;
    float3 Normal : NORMAL;
};

VSOutput VSMain(in VSInput input)
{
    VSOutput output;

#ifdef ENABLE_SKINNING
    float3 position = 0.0f;
    float3 normal = 0.0f;

    [unroll]
    for(uint i = 0; i < 4; ++i)
    {
        float4x4 bone = Bones[input.SkinIndices[i]];
        float weight = input.SkinWeights[i];
        position += mul(float4(input.Position, 1.0f), bone).xyz * weight;
        normal = mul(float4(normal, 0.0f), bone).xyz * weight;
    }
#else
    float3 position = input.Position;
    float3 normal = input.Normal;
#endif

    output.Position = mul(float4(position, 1.0f), WorldViewProj).xyz;
    output.Normal = mul(float4(normal, 0.0f), World).xyz;

    return output;
}


If you were to try to implement this feature using runtime branching instead of preprocessor macros, you'd have the problem that you wouldn't be able to disable the SkinWeights and SkinIndices vertex inputs. This means that even when skinning is disabled the shader would still expect those elements to be provided in a vertex buffer, so you'd either need to pad out your vertex buffers or pass dummy vertex buffers as a separate stream. It also means that the shader will expect the "Bones" constant buffer to be bound, which means you'll get spammed with warnings from the debug device (although the shader will still work fine as long as you don't actually use anything from that constant buffer). Edited by MJP

Share this post


Link to post
Share on other sites
Sign in to follow this  

  • Advertisement
  • Advertisement
  • Popular Now

  • Advertisement
  • Similar Content

    • By AxeGuywithanAxe
      I wanted to see how others are currently handling descriptor heap updates and management.
      I've read a few articles and there tends to be three major strategies :
      1 ) You split up descriptor heaps per shader stage ( i.e one for vertex shader , pixel , hull, etc)
      2) You have one descriptor heap for an entire pipeline
      3) You split up descriptor heaps for update each update frequency (i.e EResourceSet_PerInstance , EResourceSet_PerPass , EResourceSet_PerMaterial, etc)
      The benefits of the first two approaches is that it makes it easier to port current code, and descriptor / resource descriptor management and updating tends to be easier to manage, but it seems to be not as efficient.
      The benefits of the third approach seems to be that it's the most efficient because you only manage and update objects when they change.
    • By evelyn4you
      hi,
      until now i use typical vertexshader approach for skinning with a Constantbuffer containing the transform matrix for the bones and an the vertexbuffer containing bone index and bone weight.
      Now i have implemented realtime environment  probe cubemaping so i have to render my scene from many point of views and the time for skinning takes too long because it is recalculated for every side of the cubemap.
      For Info i am working on Win7 an therefore use one Shadermodel 5.0 not 5.x that have more options, or is there a way to use 5.x in Win 7
      My Graphic Card is Directx 12 compatible NVidia GTX 960
      the member turanszkij has posted a good for me understandable compute shader. ( for Info: in his engine he uses an optimized version of it )
      https://turanszkij.wordpress.com/2017/09/09/skinning-in-compute-shader/
      Now my questions
       is it possible to feed the compute shader with my orignial vertexbuffer or do i have to copy it in several ByteAdressBuffers as implemented in the following code ?
        the same question is about the constant buffer of the matrixes
       my more urgent question is how do i feed my normal pipeline with the result of the compute Shader which are 2 RWByteAddressBuffers that contain position an normal
      for example i could use 2 vertexbuffer bindings
      1 containing only the uv coordinates
      2.containing position and normal
      How do i copy from the RWByteAddressBuffers to the vertexbuffer ?
       
      (Code from turanszkij )
      Here is my shader implementation for skinning a mesh in a compute shader:
      1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 struct Bone { float4x4 pose; }; StructuredBuffer<Bone> boneBuffer;   ByteAddressBuffer vertexBuffer_POS; // T-Pose pos ByteAddressBuffer vertexBuffer_NOR; // T-Pose normal ByteAddressBuffer vertexBuffer_WEI; // bone weights ByteAddressBuffer vertexBuffer_BON; // bone indices   RWByteAddressBuffer streamoutBuffer_POS; // skinned pos RWByteAddressBuffer streamoutBuffer_NOR; // skinned normal RWByteAddressBuffer streamoutBuffer_PRE; // previous frame skinned pos   inline void Skinning(inout float4 pos, inout float4 nor, in float4 inBon, in float4 inWei) {  float4 p = 0, pp = 0;  float3 n = 0;  float4x4 m;  float3x3 m3;  float weisum = 0;   // force loop to reduce register pressure  // though this way we can not interleave TEX - ALU operations  [loop]  for (uint i = 0; ((i &lt; 4) &amp;&amp; (weisum&lt;1.0f)); ++i)  {  m = boneBuffer[(uint)inBon].pose;  m3 = (float3x3)m;   p += mul(float4(pos.xyz, 1), m)*inWei;  n += mul(nor.xyz, m3)*inWei;   weisum += inWei;  }   bool w = any(inWei);  pos.xyz = w ? p.xyz : pos.xyz;  nor.xyz = w ? n : nor.xyz; }   [numthreads(1024, 1, 1)] void main( uint3 DTid : SV_DispatchThreadID ) {  const uint fetchAddress = DTid.x * 16; // stride is 16 bytes for each vertex buffer now...   uint4 pos_u = vertexBuffer_POS.Load4(fetchAddress);  uint4 nor_u = vertexBuffer_NOR.Load4(fetchAddress);  uint4 wei_u = vertexBuffer_WEI.Load4(fetchAddress);  uint4 bon_u = vertexBuffer_BON.Load4(fetchAddress);   float4 pos = asfloat(pos_u);  float4 nor = asfloat(nor_u);  float4 wei = asfloat(wei_u);  float4 bon = asfloat(bon_u);   Skinning(pos, nor, bon, wei);   pos_u = asuint(pos);  nor_u = asuint(nor);   // copy prev frame current pos to current frame prev pos streamoutBuffer_PRE.Store4(fetchAddress, streamoutBuffer_POS.Load4(fetchAddress)); // write out skinned props:  streamoutBuffer_POS.Store4(fetchAddress, pos_u);  streamoutBuffer_NOR.Store4(fetchAddress, nor_u); }  
    • By mister345
      Hi, can someone please explain why this is giving an assertion EyePosition!=0 exception?
       
      _lightBufferVS->viewMatrix = DirectX::XMMatrixLookAtLH(XMLoadFloat3(&_lightBufferVS->position), XMLoadFloat3(&_lookAt), XMLoadFloat3(&up));
      It looks like DirectX doesnt want the 2nd parameter to be a zero vector in the assertion, but I passed in a zero vector with this exact same code in another program and it ran just fine. (Here is the version of the code that worked - note XMLoadFloat3(&m_lookAt) parameter value is (0,0,0) at runtime - I debugged it - but it throws no exceptions.
          m_viewMatrix = DirectX::XMMatrixLookAtLH(XMLoadFloat3(&m_position), XMLoadFloat3(&m_lookAt), XMLoadFloat3(&up)); Here is the repo for the broken code (See LightClass) https://github.com/mister51213/DirectX11Engine/blob/master/DirectX11Engine/LightClass.cpp
      and here is the repo with the alternative version of the code that is working with a value of (0,0,0) for the second parameter.
      https://github.com/mister51213/DX11Port_SoftShadows/blob/master/Engine/lightclass.cpp
    • By mister345
      Hi, can somebody please tell me in clear simple steps how to debug and step through an hlsl shader file?
      I already did Debug > Start Graphics Debugging > then captured some frames from Visual Studio and
      double clicked on the frame to open it, but no idea where to go from there.
       
      I've been searching for hours and there's no information on this, not even on the Microsoft Website!
      They say "open the  Graphics Pixel History window" but there is no such window!
      Then they say, in the "Pipeline Stages choose Start Debugging"  but the Start Debugging option is nowhere to be found in the whole interface.
      Also, how do I even open the hlsl file that I want to set a break point in from inside the Graphics Debugger?
       
      All I want to do is set a break point in a specific hlsl file, step thru it, and see the data, but this is so unbelievably complicated
      and Microsoft's instructions are horrible! Somebody please, please help.
       
       
       

    • By mister345
      I finally ported Rastertek's tutorial # 42 on soft shadows and blur shading. This tutorial has a ton of really useful effects and there's no working version anywhere online.
      Unfortunately it just draws a black screen. Not sure what's causing it. I'm guessing the camera or ortho matrix transforms are wrong, light directions, or maybe texture resources not being properly initialized.  I didnt change any of the variables though, only upgraded all types and functions DirectX3DVector3 to XMFLOAT3, and used DirectXTK for texture loading. If anyone is willing to take a look at what might be causing the black screen, maybe something pops out to you, let me know, thanks.
      https://github.com/mister51213/DX11Port_SoftShadows
       
      Also, for reference, here's tutorial #40 which has normal shadows but no blur, which I also ported, and it works perfectly.
      https://github.com/mister51213/DX11Port_ShadowMapping
       
  • Advertisement