Jump to content
Subscribe to GameDev.net Direct to receive the latest updates and exclusive content.
Posted 08 July 2012 - 05:17 AM
Posted 08 July 2012 - 05:29 AM
You mean that you have a float depth value, then you split it into e.g. a float4 for writing into a texture (which you later re-assemble back into a float)?
if I sample the colours in photoshop the value decreases but at the edge of the bands it jumps up!?!?!
am storing it across all for channels for precision
Edited by Hodgman, 08 July 2012 - 05:31 AM.
Posted 08 July 2012 - 06:02 AM
Posted 08 July 2012 - 06:45 AM
Posted 08 July 2012 - 08:51 AM
N.B. at the end of the GPU version, after your pixel shader runs, the hardware's ROP stage takes your float4 output and quantises it down to a byte4 (assuming a 4-channel 8-bit texture format). If emulating your code on the CPU to test correctness, make sure that you perform this same float->byte->float quantisation to emulate the value being written to a texture and read back out again.
Simple but seems to produce very accurate results when I am testing it on the CPU, very hard to tell if its the GPU where this issue lies of if the problem is fundamental
Edited by Hodgman, 08 July 2012 - 08:54 AM.
Posted 08 July 2012 - 12:26 PM
Posted 08 July 2012 - 01:22 PM
I'm not sure if the quantisation process is actually documented anywhere... I guess it would be byteValue = round(input*255), but it might also use floor/truncation... maybe test both approaches in your CPU test and see if it affects your results?
Posted 08 July 2012 - 01:45 PM
Posted 08 July 2012 - 04:28 PM
Posted 09 July 2012 - 01:06 AM
GameDev.net™, the GameDev.net logo, and GDNet™ are trademarks of GameDev.net, LLC.