I am writing I SlimDX application to test color contrast sensitivity thresholds of the human visual system. The subject is presented with a large letter that is at low contrast with with the background and asked to identify the letter. However I need the ability to display more colors then is available with a simple 8bit per channel color depth. (i.e. rgb values 0-255). The way I plan to due this is through a simple "dithering" algorithm implemented as an HLSL pixel shader. Basically if request slimDX to render the text to a surface with a color of Color4( 0.55f, 0.55f, 0.55f ) which corresponds to a rgb value of ( 140.25 , 140.25 , 140.25 ) I want each color channel of each pixel to have a 25% chance of being set to 141 and a 75% chance of getting set to 140. This should (in the limit of many pixels) lead to a letter that appears as a shade of gray one 1/4 of the way in-between 140 and 141.
I am however getting errors in my prototype shader code when trying to compile with fxc. I am getting an Illegal character code when I compile and I don't know why. Also if you are a HLSL guru please look over my code and make any comments that strike you. Note I got the HLSL random function from an answer to this question.
http://stackoverflow.com/questions/5149544/can-i-generate-a-random-number-inside-a-pixel-shader

Below is my shader code. Please forgive me if there a many mistakes as its my first HLSL code:


float4 DitherByChance(float2 coords : TEXCOORD0) : COLOR
{
float4 newColor; // The pixel color to return
float4 oldColor = tex2D(s0, coords);

// Note I know that as the code stands now rCutOff = gCutOff = bCutOff I will sort a fix for this out later
float rCutOff = random(coords); // A random float that determines if the red channel will be rounded up or down
float gCutOff = random(coords); // A random float that determines if the green channel will be rounded up or down
float bCutOff = random(coords); // A random float that determines if the blue channel will be rounded up or down
float rPercentChance = frac(oldColor.r * 255); //Chance to round red channel up
float gPercentChance = frac(oldColor.g * 255); //Chance to round green channel up
float bPercentChance = frac(oldColor.b * 255); //Chance to round blue channel up

//In the code below (1/255) is the floating point represntaion of an incress of one on the 0-255 RGB scale

if (rCutOff <= rPercentChance) newColor.r = oldColor.r + ((1 - rPercentChance) / 255); //Bump up one r value
else newColor.r = oldColor.r - rPercentChance * (1 / 255); //Bump down to ensure r is not rounded up

if (gCutOff <= gPercentChance) newColor.g = oldColor.g + ((1 - gPercentChance) / 255); //Bump up one g value
else newColor.b = oldColor.b - bPercentChance * (1 / 255); //Bump down to ensure g is not rounded up

if (bCutOff <= bPercentChance) newColor.b = oldColor.b + ((1 - bPercentChance) / 255); //Bump up one b value
else newColor.b = oldColor.b - bPercentChance * (1 / 255); //Bump down to ensure b is not rounded up

return newColor;
}
// Input: It uses texture coords as the random number seed.
// Output: Random number: [0,1), that is between 0.0 and 0.999999... inclusive.
// Author: Michael Pohoreski
// Copyright: Copyleft 2012 :-)
float random( vec2 p )
{
// We need irrationals for pseudo randomness.
// Most (all?) known transcendental numbers will (generally) work.
const vec2 r = vec2(
23.1406926327792690, // e^pi (Gelfond's constant)
2.6651441426902251); // 2^sqrt(2) (Gelfond–Schneider constant)
return fract( cos( mod( 123456789., 1e-7 + 256. * dot(p,r) ) ) );
}


I'm not sure why you'd get an "illegal character" error. Perhaps the encoding of the file you saved is incorrect? I think fxc can only work with plain ANSI text files.

Anyway I had to make a few changes to get your shader to compile. You need to declare a sampler2D named s0, since you try to use one. The random function needs to be declared before DitherByChance, since that function uses it. Also the random code you got was in GLSL, and had to be converted. Here's a new version:


sampler2D s0;
// Input: It uses texture coords as the random number seed.
// Output: Random number: [0,1), that is between 0.0 and 0.999999... inclusive.
// Author: Michael Pohoreski
// Copyright: Copyleft 2012 :-)
float random( float2 p )
{
// We need irrationals for pseudo randomness.
// Most (all?) known transcendental numbers will (generally) work.
const float2 r = float2(
23.1406926327792690, // e^pi (Gelfond's constant)
2.6651441426902251); // 2^sqrt(2) (Gelfond–Schneider constant)
return frac( cos( fmod( 123456789., 1e-7 + 256. * dot(p,r) ) ) );
}
float4 DitherByChance(float2 coords : TEXCOORD0) : COLOR
{
float3 newColor; // The pixel color to return
float4 oldColor = tex2D(s0, coords);
// Note I know that as the code stands now rCutOff = gCutOff = bCutOff I will sort a fix for this out later
float rCutOff = random(coords); // A random float that determines if the red channel will be rounded up or down
float gCutOff = random(coords); // A random float that determines if the green channel will be rounded up or down
float bCutOff = random(coords); // A random float that determines if the blue channel will be rounded up or down
float rPercentChance = frac(oldColor.r * 255); //Chance to round red channel up
float gPercentChance = frac(oldColor.g * 255); //Chance to round green channel up
float bPercentChance = frac(oldColor.b * 255); //Chance to round blue channel up
//In the code below (1/255) is the floating point represntaion of an incress of one on the 0-255 RGB scale
if (rCutOff <= rPercentChance) newColor.r = oldColor.r + ((1 - rPercentChance) / 255); //Bump up one r value
else newColor.r = oldColor.r - rPercentChance * (1 / 255); //Bump down to ensure r is not rounded up
if (gCutOff <= gPercentChance) newColor.g = oldColor.g + ((1 - gPercentChance) / 255); //Bump up one g value
else newColor.b = oldColor.b - bPercentChance * (1 / 255); //Bump down to ensure g is not rounded up
if (bCutOff <= bPercentChance) newColor.b = oldColor.b + ((1 - bPercentChance) / 255); //Bump up one b value
else newColor.b = oldColor.b - bPercentChance * (1 / 255); //Bump down to ensure b is not rounded up
return float4(newColor, 1.0f);
}


Thank you so much! My file was indeed encoded in UTF-8 which does blow up fxc. Your conversion of the GLSL code fixed the rest of my problems. I am struggling right now with figuring out how to use my effect when rendering 2d textures with spritebatch but if I can't figure it out I will post my problems here again. Thanks you again for your help.