Color grading shader

Started by
5 comments, last by Ruggostein 9 years, 10 months ago

I'm trying to implement a shader that allows to do color grading using a 2D texture.

2D because I need it to run on all mobile devices capable of OpenGL ES 2.0.

The lookup table textures used for color grading look like this.

RGBTable16x1.jpg

The Y axis contain the green values, the X axis contains both R and B.

This is the same format used by Unreal engine for color grading.

My shading works on all of my mobile devices, works on my desktop with an AMD card.

Does not work properly in my laptop that has an ATI card though, some pixels get completly wrong colors, most look perfecty good..

This is shader, can someone find any problem with it?


    uniform sampler2D color_table_texture;          // LUT texture with nearest filter, mipmaps off
    uniform mediump float color_table_elements;  //  LUT.Height
    uniform mediump float color_table_scale;         // 256 / LUT.Height
    uniform mediump vec3 color_table_clamp;      // vec3(LUT.Height-1)

    lowp vec3 ColorTableLookup(mediump vec3 color)    
{
mediump float base_value = 255.0;
    mediump vec3 rescaler = vec3(base_value, base_value, base_value);
    color *= rescaler;   // convert to 0..255 range
    color /= color_table_scale;        // convert to LUT range (0..LUT.HEIGHT)

    mediump vec3 delta = fract(color);  // get interpolation value
    mediump vec3 rgb1 = floor(color);   // get lower color
    mediump vec3 rgb2 = ceil(color);    // get higher color
    rgb2 = min(rgb2, color_table_clamp);   // clamp to LUT.height-1

  // loop up first color
    mediump vec2 ofs = vec2(rgb1.r + rgb1.b * color_table_elements, rgb1.g);
    ofs.x /= (color_table_elements*color_table_elements);
    ofs.y /= color_table_elements;
    mediump vec3 temp = texture2D(color_table_texture, ofs).rgb;

  // loop up second color
    ofs = vec2(rgb2.r + rgb2.b * color_table_elements, rgb2.g);
    ofs.x /= (color_table_elements*color_table_elements);
    ofs.y /= color_table_elements;
    mediump vec3 temp2 = texture2D(color_table_texture, ofs).rgb;

  // interpolate
  return mix(temp, temp2, delta);    
}

Note that while here I'm doign two lookups and interpolating, I also tried a trimmed down version that only did one lookup, no interpolation, the problem still happens.

I've tried so many things, it seems that some of the texture reads result in wrong lookups.

I don't know if this is a precision issue during generation of the offsets for lookup.

Is there a way to access a texture using integer pixel coordinates (1..Texture.Size) instead of 0...1?

Or what else could be the problem?

Advertisement

You can use texelFetch() for integral values, but not with regular textures

See: http://www.opengl.org/sdk/docs/man/html/texelFetch.xhtml

(assuming it exists in OpenGL ES, which I don't know anything about)

If you know what the expected result is, it should be straightforward to output each stage of the conversion to see where the hickup is.

Verify the inputs, check that they are received properly in the shader.

Also you can simplify a little: vec3(float) == vec3(float, float, float)

I also wonder about that ceil(), don't you want it to always be (int)x + 1? Maybe it's not necessary here.

It also seems to me that you can put the lookups into functions, because they are basically the same code

Hi, thanks, did not know about textureFetch(), however it does not existe in GL ES 2.0.

By the way, I found the problem, and it was really stupid...

Disabling ATI Catalyst AI setting fixes it!

I guess this setting was making the texture into a 16 bit texture instead of 32 bit or something else really funky that did mess with the results...

Anyone has a clue what could cause it, and if there is a way to hint the drivers to not mess with the textures?

There's a few issues that will affect quality in your shader.

1) You should be using linear filtering instead of nearest, so that each of your two samples is the bilinear interpolated result of the nearest 4 texels in the LUT. The mix at the end should only be based on the fractional blue value, as your 2D texture means that red & green will be usnig HW bilinear filtering, but blue won't so needs to be emulated.

i.e. when using a 3D LUT with linear filtering, the HW mixes 8 texels. Using the above scheme, the HW mixes 4 texels, 2 times, and then you lerp them together to get the exact same result. With your curent code, you only fetch 2 source texels and then mix them, which will give a much lower quality result.

2) You need to be very precise with your texture coordinates. The centre of the leftmost texel is at U = 0.5/256 (or U = 0.5/16 if the texture was 16 pixels wide).

Your current code is acting like, for the bottom layer (blue=0% layer) the leftmost texel (the red=0% column) is centred at 0.0 (0/256) and the rightmost (the red=100% column) at 0.0625 (16/256). Instead you should use the range 0.5/256 texels to 15.5/256 texels.

Likewise for height / green - it should be from 0.5/16 to 15.5/16 texels.

The artefact caused by this will be a very, very small loss of quality / a small rounding towards 0/255 and 255/255 / rounding away from 128/255.

You can verify this by saving a screenshot with and without your color grading applied - using a completely standard LUT texture, both screenshots should be bit for bit exact. If there's any changes at all, then your code or your LUT are slightly wrong.

Thanks Hodgman, thats lots of useful information there.

However that bit about using linear vs nearest, there was reason why I did not use linear (I might be wrong though, correct me if thats the case).

This 2D texture is basically emulating a 3D texture, so we can think of it as a series of slices (arranged horizontally).

If the shader tries to lookup a color in the LUT that falls near the "slices" borders, it will interpolate between two slices, and that will produce very wrong results (eg: mixing very light colors with dark colors).

What solution you can think to avoid this problem when using linear filtering?

But the problem why I created the topic is that in my laptop, the results are completly wrong for some of the pixels (eg: some pixels appeared magenta when they should appear dark blue). In all other devices, from PCs running Windows, OSX and lots of mobile devices, the results at least it looks good to the eyes (might be not exactly correct if we compare the exact in and out RGB values, as you said, this shader is missing some details.

However as I said in my previous post, this particular issue was fixed just by disable Catalyst AI option (this laptop has a ATI card, around 4 years old).

Might be old drivers, or just the way the "catalyst AI" setting works, I'm not sure, but it somehow was altering the texture pixels/quality/size.

What solution you can think to avoid this problem when using linear filtering?

What Hodgman has mentioned in "2) You need to be very precise with your texture coordinates." already: Use the center of the texels! When e.g. red input is 0, and you address the LUT at 0/16 (or 0/256 for 2D), you hit the left border of the texel, and the sampler will interpolate 50% to 50%. However, with an offset of 0.5/16 (or 0.5/256 for 2D), you hit the center of the texel instead, and the sampler will interpolate with 100% to 0%. So a span ranges from 0.5/16 to 15.5/16 (or 0.5/256 to 15.5/256 for 2D) for an color channel input range of 0 to 15. Hence interpolation will be done inside a slice, but not crossing slice boundaries.

BTW: This is true not only for the 2D arrangement, but also for a real 3D LUT.

However that bit about using linear vs nearest, there was reason why I did not use linear

If you use nearest neighbor interpolation, you effectively reduce your amount of colors to 16*16*16 = 4096. I.e. a kind of posterize effect.

Ah yes, I overlooked that part about precise coordinatse.

If you use nearest neighbor interpolation, you effectively reduce your amount of colors to 16*16*16 = 4096. I.e. a kind of posterize effect.

Yes true, but I'm not just looking up one single color at nearest, I'm interpolating between two colors, so I think I get much more than that.

I've got this shader running in my game, I applied it to everything from GUIs to scene drawing and did not notice any color loss, especialy nothing similiar to a posterize effect.

However as Hodgman said, I understand now that in the 2d LUT case this manual interpolation should be done only for the blue component, as is the only one that cannot really be interpolated via texture filtering.

This topic is closed to new replies.

Advertisement