Properly colour-keying anti-aliased images?

Started by
10 comments, last by programering 15 years, 8 months ago
I've made a few sprites of a character that have anti-aliased edges. So obviously, when I put them in front of a pink background for colour-keying and try to use them in a game they end up with a pink-ish border around them where the semi-transparent edge of the sprite covers the pink background. How should I best get rid of that? I've tried a few things and the best result so far is saving the sprite sheet in Photoshop with a transparent background as a GIF and then setting the colour key to white. It comes out ok, but some of the edges are a bit jaggy. Any better way apart from just getting rid of the anti-aliasing altogether?
Advertisement
Which graphics API (OGL, DX, SDL, ...)?

You'll need partial transparency instead of color keying. You should google what the Alpha channel is about, aswell as blending.
I see... I'll have a look around then.

Using SDL BTW. Maybe there's something helpful in the documentation.
Can you post an explanation of how your image loading function works (or better, the actual code?).
You can't mix color-keys and anti-aliasing.

You can either solve this by tweaking the art (it seems this is what you're already doing), or by changing the way that your code handles transparency (i.e. use an alpha channel).

An alternative method (to your save-as-gif method) that I've used with Photoshop in the past is to record a macro that duplicates the transparent layer and then merges the duplicate with the original. Then record another macro that plays this first macro 100 times. This should make any semi-transparent pixels become fully opaque.

If you're going to use an alpha channel, ctrl+click on your transparent layer (this creates a selection mask from the layer, respecting semi-transparent pixels), then go to the Channels tab and click the "Save Selection an Channel" button (Pic of a square with a circle over it).
Quote:Original post by Hodgman
You can't mix color-keys and anti-aliasing.


Uhm, yes, you can. That's a totally separate topic. It's just that programmers don't do this, because if you use the alpha channel, there's no need for color-keying anymore.
Quote:Original post by Eskapade
Quote:Original post by Hodgman
You can't mix color-keys and anti-aliasing.
Uhm, yes, you can. That's a totally separate topic. It's just that programmers don't do this, because if you use the alpha channel, there's no need for color-keying anymore.
...I was talking about from an art creation perspective - i.e. if you do this in your art package then your color-key color will bleed into the border pixels.
e.g. :
Quote:they end up with a pink-ish border around them where the semi-transparent edge of the sprite covers the pink background.


[Edited by - Hodgman on August 9, 2008 9:31:12 AM]
Quote:Original post by Hodgman
Quote:Original post by Eskapade
Quote:Original post by Hodgman
You can't mix color-keys and anti-aliasing.
Uhm, yes, you can. That's a totally separate topic. It's just that programmers don't do this, because if you use the alpha channel, there's no need for color-keying anymore.
...I was talking about from an art creation perspective - i.e. if you do this in your art package then your color-key color will bleed into the border pixels.
e.g. :
Quote:they end up with a pink-ish border around them where the semi-transparent edge of the sprite covers the pink background.


The color key blends into the anti-aliased edges? How should that happen? Bad compression? It can't be filtering, that's after the color-keyed color is taken out of consideration.
Quote:Original post by Eskapade
The color key blends into the anti-aliased edges? How should that happen?


If your transparent color is, say, pink, and you have an anti-aliased green sphere in the image, then there will be pixels around the edge of the sphere which are a mix of green and pink (which is what the whole anti-aliasing thing does).
When blitting such an image with color-key transparency, only pixels which are pure pink will not be shown. This still shows the partial pink edges.

Million-to-one chances occur nine times out of ten!
Quote:Original post by Eskapade
Quote:Original post by Hodgman
...from an art creation perspective ... in your art package...
The color key blends into the anti-aliased edges? How should that happen? Bad compression? It can't be filtering, that's after the color-keyed color is taken out of consideration.
Stop thinking about it as a graphics programmer, as I already said, I was talking about from an artists perspective.

Look, this is what happens IN YOUR ART PACKAGE when you draw using AA:

This topic is closed to new replies.

Advertisement