Sign in to follow this  
medevilenemy

Color Keying issue

Recommended Posts

Hello all, I've been working on drawing a sprite system for a new game, but I seem to have run into a bit of a snag. I designated the color (255,0,255) as the transparent color key and in my loader I change the alpha for any pixel with that color to 0 (normal alpha is 255). As expected due to this method, this works except for the border between transparent key and opaque pixels. In an attempt to rectify this, I attempted to use glAlphaFunc(GL_GREATER, 0.0) however this did not work. It only started to work at around glAlphaFunc(GL_GREATER, 0.7), but the farther up I made the requirement, the more distorted my source image became (not blurryness, but rather shapes would be more stretched or flat than they should have been). Is there any way to eliminate the odd color bleed while avoiding distortion? Thanks again

Share this post


Link to post
Share on other sites
Use a properly designed alpha channel and a properly designed color image. The bleeding you're seeing comes from filtering in the key color border neighborhood, where samples are blended between a keyed color and a non-keyed color.

One solution is, after replacing the keyed pixels alpha, to replace the color of the image as well. Replace it with the color of a non-keyed neighboring pixel (or the avegare of all non-keyed neoghbors). That way, the key color will not bleed, because the color is the same as the surrounding area, instead of the key color. You will still have bleeding, but you will not notice it now, because the bleed color is the same as the rest of the texture, not the key color.

But as I said, the proper way it to not use color keying in the first place, but a proper alpha channel.

Share this post


Link to post
Share on other sites
One of these days I'll switch file formats to something with inherent alpha support, but at the moment I'm still using .bmp. I can't use the solution you suggested because that would cause problems on its own once actual art assets are used (there will be some relatively high-detail art). So, I guess I'll look into switching file formats.

Share this post


Link to post
Share on other sites
At work today I wrote up a .tga loader. As an experiment I converted the old test sprite I was using into a .tga and changed pixels with my color key into fully transparent pixels. Checking to make sure the appropriate code was changed I just tested the new configuration and I still seem to be seeing some bleeding.

Here are a few relevant enables I have set up:
glEnable(GL_BLEND);
glBlendFunc(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA);
glEnable(GL_DEPTH_TEST);
glShadeModel(GL_SMOOTH);
glEnable(GL_ALPHA_TEST);
glAlphaFunc(GL_GREATER, 0.0);

I have tried alphafunc with higher reference values to no apparent effect. What should I do?

Share this post


Link to post
Share on other sites
Apologies, I seem to have jumped the gun a bit. After digging around in the forums for a few moments, and digging around in my loader code, I have solved the problem. It was a matter of setting:
glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_NEAREST);
glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_NEAREST);

Now I have a nice clear image, with no artifacts or abnormalities of any kind. It seems changing file types really was a good idea. Thanks.

Share this post


Link to post
Share on other sites
If you see bleeding with an explicit alpha channel, it most likely means you didn't change the color of the transparent part of the image.

The transparent parts, most importantly the border pixels, cannot just have any arbitrary color. You need to make sure the border pixels have some color that can be blended into without causing bleeding. That is why you should not use color keying, because you need both transparency and full control over the color; the color key can only provide one of them in the border case. And adding the alpha is only half-way, you need to adjust the color of the border pixels as well.

Share this post


Link to post
Share on other sites
I presume, then, that I should give the transparent sections a color somewhat like the nearest opaque sections, then. That shouldn't be terribly complicated. Should I leave filtering on nearest? or is there a better setting.

Share this post


Link to post
Share on other sites
Reading the man page made it seem to me that GL_LINEAR_MIPMAP_LINEAR was the overall highest quality of the filtering options (along with GL_LINEAR for the mag filter) and so this is what I have switched it to. I am relatively unconcerned by the speed of the setting at the moment, but may change it if it causes a problem.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this