Original post by TheAdmiral
Better colour-spaces exist for this kind of business. In particular, I'd recommend YCC. A weighted Euclidean distance-function in this space would give far better results than anything linear in RGB or even HSL.
The most important coordinate is luma. I couldn't say anything certain about the two chroma components, but I suspect the red is more significant than blue (at middling intensities). IIRC, the the eye is far more sensitive to intensities in the blue hue at very low exposure, green at typical-to-high exposure, with red being fairly constant. I don't really know how or if this translates to hue-sensitivity. I'm sure Google knows a lot more, but you'll probably want to take that into consideration if working in HDR. Otherwise you may have success with something along the lines of:float3 ycc1 = RGBToYCrCb(rgb1);
float3 ycc2 = RGBToYCrCb(rgb3);
float3 ycc_dist = abs(ycc1 - ycc2);
const float3 WEIGHTS = float3(0.814, 0.453, 0.362);
float perceived_distance = dot(ycc_dist, WEIGHTS);
Understand that the weights there came entirely off the top of my head (I normalised 9:5:4), and that better values have almost certainly been researched, if you know where to look. The implementation of the RGB-YCC conversion is linear (a matrix multiplication); details here.
Which method and color-space would be most effective for comparing gray-scale colors?