Followers 0

# f16tof32 f32tof16 doesn't work correctly

## 7 posts in this topic

Hi,

I am trying to read&modify the data from my vertex buffer in the compute shader. the position in the vertex buffer has the format of R16G16B16A16. (there are some other elements in the vertex buffer as well: normal, color, texcoord...)

uint nOffsetPosZW = nOffsetPosXY + 4;
float4 vInPos;
vInPos.x = f16tof32(nXY);
vInPos.y = f16tof32(nXY>>16);
vInPos.z = f16tof32(nZW);
vInPos.w = f16tof32(nZW>>16);

// some modification for the position
vInPos.z = 10; // value doesn't behave correctly

// write into another UAV buffer
uint nXYNew = f32tof16(vInPos.x)|(f32tof16(vInPos.y)<<16);
uint nZWNew = f32tof16(vInPos.z)|(f32tof16(vInPos.w)<<16);
gVertexBufferDSTUAV.Store(nOffsetPosXY, nXYNew);
gVertexBufferDSTUAV.Store(nOffsetPosZW, nZWNew);

I am sure the offset is correct since if i set x,y,z individually to 0, i can see the value in the corresponding axis is set to 0 visually. but if i set the value other than 0, the value doesn't behave correctly. (although the sign is correct.)

My feeling is that the f16tof32 f32tof16 doesn't work correctly. (or maybe i did something stupid. )

Any help or suggestion would be really appreciated.

By the way, i couldn't find any tool to debug compute shader on PC, PIX & VS2012 graphics debugger don't work. Is there any good tool for debugging the compute shader?

Thanks!!!

Edited by MJP
0

##### Share on other sites

I've never used f32tof16... but I know you can't shift a float or at least if you do you completely mess it up, you wont get the result you were expecting. Unless there's some support for it in shaders that I didn't know of.

Binary operators weren't meant for floats

I would look up on binary operators if I were you. 1 << 8 will give you 256, but 1.0f << 8 ( assuming it compiles ) will give you something MUCH different.

The same goes for every other operator, 2 | 3  gives you 3: 2.0f | 3.0f gives you god only knows what.

and i would say the only reason why it works when you give it 0 is because 0 << anything is always 0.

Edited by Muzzy A
0

##### Share on other sites

Then that function name is misleading, 'f32' to 'f16'.

I just wanted to do this to make it more readable :)


uint nOffsetPosZW = nOffsetPosXY + 4;
float4 vInPos;
vInPos.x = f16tof32(nXY);
vInPos.y = f16tof32(nXY>>16);
vInPos.z = f16tof32(nZW);
vInPos.w = f16tof32(nZW>>16);

// some modification for the position
vInPos.z = 10; // value doesn't behave correctly

// write into another UAV buffer

uint nXYNew = f32tof16(vInPos.x)|(f32tof16(vInPos.y)<<16);
uint nZWNew = f32tof16(vInPos.z)|(f32tof16(vInPos.w)<<16);
gVertexBufferDSTUAV.Store(nOffsetPosXY, nXYNew);
gVertexBufferDSTUAV.Store(nOffsetPosZW, nZWNew);


Edited by Muzzy A
0

##### Share on other sites

Are you using DXGI_FORMAT_R16G16B16A16_FLOAT for the position element in your vertex buffer? You said the format is R16G16B16A16, but there are several varieties of that format.

For debugging compute shaders the only tools you can use are Nsight (Nvidia) and GPU PerfStudio (AMD). However you don't necessarily need any tools to debug this...you can copy your buffer to a staging buffer that you can MAP on the CPU so that you can print out the values or inspect them in a debugger.

Edited by MJP
1

##### Share on other sites
Without taking an in-depth look, I spotted these two:
vInPos.x = f16tof32(nXY);
vInPos.z = f16tof32(nZW);
Which could be wrong. They could be:
vInPos.x = f16tof32(nXY & 0x0000ffff);
vInPos.z = f16tof32(nZW & 0x0000ffff);
You should also check that nStride is correct, and that vDispatchThreadId.x*nStride actually results in what you expect it to be.
0

##### Share on other sites

Thanks a lot guys for the relies!

Muzzy: Thx for cleaning up the code, it is much more readable. yeah, i agree, the name is a bit misleading.

MJP: Yes, i am using DXGI_FORMAT_R16G16B16A16_FLOAT. Ok, i will give Nsight a try, thx for the suggestion. Yes, i could map the buffer, but it would be much easier if i could debug the shader directly, even with assembly. ;) I did similar things as you suggested: draw the mesh later and using pix to inspect the vertex buffer, i can see the result is wrong but not sure which step is causing problem. it would be nice to be able to debug step by step.

Matias: according to the document, f16tof32 Converts the float16 stored in the low-half of the uint to a float. so should be fine. I've indeed tried the one with &0x0000ffff, still the same result.

to remove the dependency of nStride, i've set the numthreads to [1,1,1] with dispatch(1,1,1), so vDispatchThreadId.x would be always 0, and only the first vertex is touched. so i think the nStride is not the problem.

Edited by zhugel
0

##### Share on other sites

MJP: Actually, you are right! the problem is because the vertex position format is DXGI_FORMAT_R16G16B16A16_SNORM!

So any value outside of the range of [-1, 1] would not work.

Edited by zhugel
1

## Create an account

Register a new account