Use:
float scale = 1.0 / 255.0;
Secondly are you aware that DitherByChance() returns values from the range of 0-255?
Reading from the texture in the shader gives you a value between 0 and 1.
Then you multiply that by 255.
For example, if the texture gave you 0.52 for red as per your example, you then do the following to it:
double rPercentChance = frac(oldColor.r / scale);Which evaluates to the following:
double rPercentChance = frac(0.52 / 0.0039215686274509803921568627451);Which evaluates to:
double rPercentChance = frac(132.59999999999999999999999999993);
Is this what you were expecting?
If so, why would you not just do:
double rPercentChance = frac(oldColor.r * 255.0);instead? Multiplication is faster than division.
L. Spiro