How to use external variables in HLSL?

Started by
4 comments, last by AvengerDr 9 years, 9 months ago

I'd like to have a master HLSL file, with some functions in it, but these functions require a lot of variables to be passed in. It'd be *nice*, but not absolutely necessary, if my master .fx file could reference "extern" variables, and at compile time, resolve those to the ones in my cbuffer in the non-master file. Is this possible?

to restate, I have two files, "master.hlsl" and let's say "blur.hlsl". I want blur.hlsl to reference master.hlsl with a #include. I want blur.hlsl to define a cbuffer with some varaibles (let's say float fBlurSizeH and float fBlurSizeW), and inside master.hlsl, it defines these variables as "extern float fBlurSizeH" (etc)... Maybe I'd have a function like void DoBlur( Texture2D image ), which references fBlurSizeH. When I try this, it tells me my variable is defined twice. Stupid compiler! :-)

I don't see any reason why this couldn't happen, but I don't know the syntax of how to do it. It just seems cleaner.

One problem I have is that I have multiple HLSL files wanting to reference this master HLSL file, so I can't reference a specific constant buffer from within the master HLSL file, I need the variables it references to be "generic".

thanks

Advertisement
Shaders have only 1 translation unit; there is no concept of “external” variables.
The results you are getting make perfect sense. An #include is simply copying the contents of a file in place of the #include command, making 1 single long shader file.
If you declare fBlurSize in both files then of course you are going to get an error.

Put declarations in header files and #include those when you want to use them across various shaders.
Or just redeclare them in your new file. It may be slightly redundant but if your buffers are declared specifically for each purpose rather than made to be “general-use” you will reduce bandwidth and increase performance, generally.


Also:

and at compile time, resolve those to the ones in my cbuffer in the non-master file. Is this possible?

Separating this issue from the rest, the solution is simple:
#1: Declare the same cbuffer in both files, using the same offsets for all members.
#2: Make 1 cbuffer object that matches the layout in both shader files.
#3: Set that cbuffer as active for both shaders before rendering.
It has nothing to do with the compiler etc. You create cbuffers on your own outside of the shader. If you want to share those cbuffers between shaders…
…then do it.

You don’t make a copy of a texture for every shader that uses it, you just make one texture and activate it on every shader.
Nothing changes when working with cbuffers.


L. Spiro

I restore Nintendo 64 video-game OST’s into HD! https://www.youtube.com/channel/UCCtX_wedtZ5BoyQBXEhnVZw/playlists?view=1&sort=lad&flow=grid

Can you just do:

cbuffer blah {};
 
#include "master.h"
 
void main() {};

If you have time to spare you could also go the totally overkill route as I did. I wrote a Shader Generation tool that allows me to define shaders as tree of "nodes". For example here is a Gaussian Blur shader.


DeclarationNode nColor = DeclarationNode.InitNode("color", Shaders.Type.Float4, 0, 0, 0, 0);
ArrayNode fOffsetWeight = new ArrayNode() {Input = fOffsetsAndWeights, Index = "i"};

AdditionNode nBlur = new AdditionNode
{
    PreCondition = new ForBlock()
    {
        PreCondition = nColor,
        StartCondition = new ScalarNode(){Value = 0},
        EndCondition = new ScalarNode(){Value = 15}
    },
    OpensBlock = true,
    Input1 = nColor,
    Input2 = new MultiplyNode()
    {
        Input1 = new TextureSampleNode()
        {
            Texture = tDiffuse,
            Sampler = sLinear,
            Coordinates = new AdditionNode()
            {
                Input1 = new ReferenceNode() {Value = InputStruct[Param.SemanticVariables.Texture]},
                Input2 = new SwizzleNode(){Input = fOffsetWeight, Swizzle = new []{Swizzle.X, Swizzle.Y}},
            }
        },
        Input2 = new SwizzleNode() { Input = fOffsetWeight, Swizzle = new[] { Swizzle.Z } },
    },
    ClosesBlock = true,
    IsVerbose = true,
    Declare = false,
    AssignToInput1 = true,
    Output = nColor.Output
};

OutputStruct = Struct.PixelShaderOutput;
Result = new PSOutputNode
{
    FinalColor = nBlur,
    Output = OutputStruct
};

And the resulting shader code (cbuffer declarations omitted)


PSOut GaussianBlurPS(VSOut input) : SV_Target
{
    float4 color = float4(0, 0, 0, 0);
    for (int i = 0; i < 15; i++)
    {
        color += tDiffuseMap.Sample(sLinear, input.Texture + fOffsetsAndWeights[i].xy) * fOffsetsAndWeights[i].z;
    }
    PSOut output;
    output.Color = color;
    return output;
}

The issue is that shaders need to be declared as combinations of node. There's no graphical editor as of yet (someday!). On the other hand this allows me to tailor it to the necessities of my engine. Since I can annotate every variable with the corresponding engine references. For example if a cbuffer requires a World matrix, the corresponding variable is tagged with that reference. So the shader initializer system when encountering that reference will automatically apply the correct data without needing to initialize a shader on a ad hoc basis.

Further, I can have "function" nodes such as one representing your "DoBlur" nodes (either as combination of other nodes or as plain text methods). When the shader is generated it will contain only the code strictly necessary (and nothing else) for that shader to work. Without messy includes and whatnot. But wait, there's more! The generated shaders are part of a "ShaderCollection" structure that holds various technique mappings. For example technique A might use SM4, while technique B might use SM5. So a ShaderCollection object can hold different versions of the same shader type, making the choice of the correct version a simpler one. AND it also comes with shader preview functionality (which is still experimental).

I just noticed that I forgot to add it to my GitHub repository but it will be when I go back home.

--Avengers UTD Chronicles - My game development blog

@AvengerDR: That looks like an interesting approach (although you wrote much more C++ code than just writing the shader by hand :P ). Do you have any further information about the system written up?

Indeed, that's very true unsure.png If some day I'll find the time to build a graphical editor, that should address it. However, the main reason I did it was the above mentioned automatic initialization. Now I don't have to worry about that anymore biggrin.png And I really don't like all the #include stuff. It's easy to manage for a small demo but a real project will have dozens of shaders and it can get messy really fast.

I didn't write any tutorial on how to use it (yet). Being a lone developer most of the code is unfortunately uncommented. I have now uploaded the source code to the repository. Look up the Odyssey.Daedalus project in OdysseyTools.sln. There are more examples in the Shaders/Techniques folder (though some might not work as I added a lot more functionalities and I need to check what happened to the older ones). If you are interested in the initialization code, look at the variable factory classes. Some of the static properties defined there instantiate objects that are tagged with the relevant engine value to use. Then in the Odyssey.Talos project (yes i like fancy names biggrin.png , it's my ECS framework) you can see how it works inside the Initializers folder.

--Avengers UTD Chronicles - My game development blog

This topic is closed to new replies.

Advertisement