I use FNV-1a for most things as it's super cheap and produces great distribution.
Check out these for a good collection of options though:
http://aras-p.info/blog/2016/08/02/Hash-Functions-all-the-way-down/
http://aras-p.info/blog/2016/08/09/More-Hash-Function-Tests/
To reduce a 32bit has to a 16bit hash (or 16 bit to 8bit), I usually just XOR the top half of the value over the bottom half.
uint16_t hash16 = (uint16_t)(((hash32>>16)^hash32)&0xffff);
uint8_t hash8 = (uint8_t )(((hash16>> 8)^hash16)&0xff );
However, if you've got a good hashing algorithm, then you should even just be able to slice part of the result off and get decent results, e.g.
uint16_t hash16 = hash32 & 0xffff;
If you've got a 64bit value where significance matters and you need to reduce it to a 32bit value, the xor trick won't work, because it combines, for example, bit #31 and bit #63. In these situations I've sometimes just thrown out every second bit like this:
u32 KeepEverySecondBit(u64 key)
{
key &= 0x5555555555555555;//throw out every second bit -- 0a0b0c0d0e0f0g0h0i0j0k0l0m0n0o0p0A0B0C0D0E0F0G0H0I0J0K0L0M0N0O0P
key = (key | (key >> 1)) & 0x3333333333333333;// 00ab00cd00ef00gh00ij00kl00mn00op00AB00CD00EF00GH00IJ00KL00MN00OP
key = (key | (key >> 2)) & 0x0f0f0f0f0f0f0f0f;// 0000abcd0000efgh0000ijkl0000mnop0000ABCD0000EFGH0000IJKL0000MNOP
key = (key | (key >> 4)) & 0x00ff00ff00ff00ff;// 00000000abcdefgh00000000ijklmnop00000000ABCDEFGH00000000IJKLMNOP
key = (key | (key >> 8)) & 0x0000ffff0000ffff;// 0000000000000000abcdefghijklmnop0000000000000000ABCDEFGHIJKLMNOP
key = (key | (key >>16)) & 0x00000000ffffffff;// 00000000000000000000000000000000abcdefghijklmnopABCDEFGHIJKLMNOP
return (u32)key;
}
If you want to do this without actually throwing away any entropy, I guess you could use this to XOR together pairs of bits (e.g. bit #62 with bit #63):
u32 hash32 = KeepEverySecondBit(hash64)^KeepEverySecondBit(hash64>>1);
If you know your depth values are always positive (distances should be), then their bitwise representations will always be nicely ordered, so you can just bitwise cast/reinterpret those floats to an int, and then put them in the most significant bits of your key. Doing a bitwise negation on that value reverses the ordering.
e.g.
u32 intDepth = *(u32*)&floatDepth;
Back to front:
key = ((~(u64)intDepth)<<32) | KeepEverySecondBit(key);
Front to back:
key = (( (u64)intDepth)<<32) | KeepEverySecondBit(key);
Alternatively, you can normalize your depth values into a zero to one range, and then quantize them.
float normalized = depth / far_clip_plane;
int quantized_8bit = Clamp( (int)(normalized * 255 + 0.5), 0, 255 );
And then put these in the most significant bits of your key. e.g. you can just shift the key down to lose the least significant sorting data:
key = (((u64)quantized_8bit)<<56) | (key>>8);
For transparent passes you typically want to do an accurate (low/no quantization) back to front sorting.
For opaques that occur after a Z-pre-pass, you don't care about depth -- just sort by state changes.
For opaques that occur without a Z-pre-pass, do a very coarse front-to-back sorting (e.g. quantize depth to 4 bits!).