Skip to main content
GameDev.net gamedev.net
🔒 Locked

HLSL Data Type Conversion in a Compute Shader

DX11
Started by ucfchuck Apr 24, 2018 at 12:17 AM 4 replies 15.6k views
Original Post
ucfchuck
ucfchuck

I am feeding in 16 bit unsigned integer data to process in a compute shader and i need to get a standard deviation.

So I read in a series of samples and push them into float arrays 


        float vals1[9], vals2[9], vals3[9], vals4[9];
    int x = 0,y=0;
    for ( x = 0; x < 3; x++)
    {
        for (y = 0; y < 3; y++)
        {
            vals1[3 * x + y] = (float) (asuint(Input1[threadID.xy + int2(x - 1, y - 1)].x));
            vals2[3 * x + y] = (float) (asuint(Input2[threadID.xy + int2(x - 1, y - 1)].x));
            vals3[3 * x + y] = (float) (asuint(Input3[threadID.xy + int2(x - 1, y - 1)].x));
            vals4[3 * x + y] = (float) (asuint(Input4[threadID.xy + int2(x - 1, y - 1)].x));
        }
    }

I can send these values out directly and the data is as expected 



         
    Output1[threadID.xy] = (uint) (vals1[4] );
    Output2[threadID.xy] = (uint) (vals2[4] );
    Output3[threadID.xy] = (uint) (vals3[4] );
    Output4[threadID.xy] = (uint) (vals4[4] );

however if i do anything to that data it is destroyed.

If i add a

vals1[4] = vals1[4]/2;

or a

vals1[4] = vals[1]-vals[4];

the data is gone and everything comes back 0.



How does one go about converting a uint to a float and performing operations on it and then converting back to a rounded uint?

pcmaster
pcmaster

Can you show us the declarations of Input1-4 and Output1-4 as well? Also their descriptors used to create the SRV and UAV.

ucfchuck
ucfchuck 

Texture2D Input1 : register(t0);
Texture2D Input2 : register(t1);
Texture2D Input3 : register(t2);
Texture2D Input4 : register(t3);  

RWTexture2D<uint> Output1 : register(u0);
RWTexture2D<uint> Output2 : register(u1);
RWTexture2D<uint> Output3 : register(u2);
RWTexture2D<uint> Output4 : register(u3);      
                               



D3D11.Texture2DDescription t = new D3D11.Texture2DDescription();
t.Width = CurrentGeometry.SamplesPerLine/2;
t.Height = CurrentGeometry.LineCount/2;
t.MipLevels = t.ArraySize = 1;
t.Format = DXGI.Format.R16_UNorm;
t.SampleDescription = new DXGI.SampleDescription(1, 0);
t.Usage = D3D11.ResourceUsage.Default;
t.BindFlags = D3D11.BindFlags.ShaderResource;
t.CpuAccessFlags = D3D11.CpuAccessFlags.None;


Set(ref D1Tex, new D3D11.Texture2D(ComputeDevice, t) { DebugName = "D1Tex" });
Set(ref D1View, new D3D11.ShaderResourceView(ComputeDevice, D1Tex) { DebugName = "D1View" });  


...


t.BindFlags = D3D11.BindFlags.ShaderResource | D3D11.BindFlags.UnorderedAccess;


Set(ref T1Out, new D3D11.Texture2D(ComputeDevice, t) { DebugName = "t1outTex" });
Set(ref T1OutView, new D3D11.UnorderedAccessView(ComputeDevice, T1Out) { DebugName = "t1outView" });

...

the Set() is just to help track memory leaks. 



ComputeDevice.ImmediateContext.ComputeShader.Set(SD);
ComputeDevice.ImmediateContext.ComputeShader.SetShaderResource(0, D1View);
ComputeDevice.ImmediateContext.ComputeShader.SetShaderResource(1, D2View);
ComputeDevice.ImmediateContext.ComputeShader.SetShaderResource(2, D3View);
ComputeDevice.ImmediateContext.ComputeShader.SetShaderResource(3, D4View);
ComputeDevice.ImmediateContext.ComputeShader.SetUnorderedAccessView(0, T1OutView);
ComputeDevice.ImmediateContext.ComputeShader.SetUnorderedAccessView(1, T2OutView);
ComputeDevice.ImmediateContext.ComputeShader.SetUnorderedAccessView(2, T3OutView);
ComputeDevice.ImmediateContext.ComputeShader.SetUnorderedAccessView(3, T4OutView);
ComputeDevice.ImmediateContext.Dispatch((int)(CurrentGeometry.SamplesPerLine / 32), (int)(CurrentGeometry.LineCount / 32), 1);

ComputeDevice.ImmediateContext.CopyResource(T1Out, SD1Tex);
ComputeDevice.ImmediateContext.CopyResource(T2Out, SD2Tex);
ComputeDevice.ImmediateContext.CopyResource(T3Out, SD3Tex);
ComputeDevice.ImmediateContext.CopyResource(T4Out, SD4Tex);



btw this is using sharpdx for the c#

ucfchuck
ucfchuck

So if I ignore the incoming data actually being a uint and just treat everything as floats the whole way through then it works fine...

MJP
MJP

So your input textures have a format of R16_UNORM. This means that the texels in memory will store a [0.0, 1.0] value in a fixed-point integer representation, where 0x0000 corresponds to 0.0 and 0xFFFF corresponds to 1.0. When you sample one of these textures in a shader through a Texture with a float template type (Texture2D is implictly Texture2D), the fixed-point integer data will be automatically converted to floating-point by the texture unit. This allows your shader to work with 32-bit single-precision floats, without needing to manually decode the texture data.

Your shader code is taking the decoded floating-point value and bit-casting it to a uint, which means that you'll end up with the raw binary representation of that floating point value. You then cast back to a float, which will try to represent your integer as 32-bit float, probably destroying the information in the process due to imprecision. You then store that in a RWTexture2D, which will cause implicit casting of that floating-point value back to an unsigned integer. Note that using a RWTexture2D to write to R16_UNORM is invalid: you need to use a float type to do this (specifically you want RWTexture2D for this case). If you enable the debug validation layer, it will complain if you try to do this.

As you've already found out, the solution is to just use "float" everywhere in your shader. If you do that, your UNORM data will be automatically converted to a float when you read from your input textures, and then converted back to 16-bit UNORM representation when you write to the output texture.

Topic Locked

This topic has been locked by a moderator. New replies are not allowed.

Sign in to reply to this topic.