Best fit in a paletted anti-aliaser

Started by
7 comments, last by FReY 23 years, 11 months ago
Hi, I''m writing a screen anti-aliaser for a 256 palette mode. Problem is, when I average the colours, and find the ACTUAL colour of what I need, the ''fitting'' process does not come up with the right colour. At the moment my best fit colour routine looks like this: byte fitcol(char *pal, byte r,byte g,byte b) { byte pr,pg,pb; int diff,sdiff=256,scol=0; for (int c=0;c<256;c++) { pr=pal[c*3]; pg=pal[1+c*3]; pb=pal[2+c*3]; diff=(abs(r-pr)+abs(g-pg)+abs(b-pb)); if (diff<sdiff) { sdiff=diff; scol=c; } } return scol; // returns the index in the palette passed to it of the // best colour in that palette } Can anyone help? Thanks FReY
do unto others... and then run like hell.
Advertisement
Try setting sdiff 256*3.
You see, the color black (0,0,0) and white (255,255,255) would have a difference of 765 (255 times 3), so you need a higher default value.

lntakitopi@aol.com | http://geocities.com/guanajam/
Ah, sorry, I should''ve mentioned that my palette data uses only 0-63 in value for each of it''s RGB, so theoretically sdiff would be <192 at max.

Anyone else have a better algorithm that takes into account the closeness of Red, Green, Blue, specific to that colour?

Thanks and HELP!!!! once again,
FReY
do unto others... and then run like hell.
I can''t remember the algorithm right off the top of my head, but I know that a couple of years ago, I found a really good example of how to do this in the Allegro source. You could try looking there to see how they did it. The only details that I really remember at the moment is that the colors (red, green, blue) were weighted unevenly to account for the fact that the human eye is most perceptive of shades of certain colors over others.

Anyway, check out the Allegro source code and you should find a really good example of how to do it.
Maybe we can hunt down the problem.
Is the color blatantly WRONG, (i.e. green instead of blue), or is it just not close enough?

lntakitopi@aol.com | http://geocities.com/guanajam/
Well, lets just say I tried to antialias some trees, and they came out blue.

The problem is, that in my palette, the difference between the desired colour and multiple colours in my palette is the same.

I''ve used this algorithm before and it "seemed" to work then, but maybe it''s just a problem with my palette. It''s definitely nothing wrong with the anti-aliaser. It''s just the "best-fit" routine. . . Ugh!!

Greetings,
FReY

PS. I will take a look in the Allegro source though

do unto others... and then run like hell.
The problem is your algorithm is finding the closest brightness, not color.

You need to keep each color channel evaluated seperately.

You could try squaring or cubing the differences, and adding those up. That may give better results.

Or, you could try evaluating based only on the largest of the three differences.

These are just some suggestions.
Darnit, I feel so stupid, Anon...

lntakitopi@aol.com | http://geocities.com/guanajam/
I don''t think that squaring/cubing them will work, though... you have to have an independant difference for R, G and B.

lntakitopi@aol.com | http://geocities.com/guanajam/

This topic is closed to new replies.

Advertisement