I understand that when I linearly filter my sprites, the border pixels will get mixed with the color keyed transparent sprites. There doesn't seem to be any way to avoid this problem when using linear filtering, so what I did was add an alpha stencil and block off these partially transparent pixels from getting rendered. Looks good enough... except now there's a very dark border around my sprite, and it's very visible with lighter backgrounds. I don't know why it's there or what to do about it. And it's really quite ugly and if I can't fix it its enough to make me switch to point filtering which I'd rather not have to do, cause linear filtered sprites look pretty. Anyway, here is an example of what I mean: http://sqpat.com/project/mysample37-uglyborders.PNG At first I thought it was cause the borders of the sprite were a darker color, but then I edited the hair in the second sprite so the border was the same color, and you can still see that dark border (perhaps even more clearly). Then i figured my alpha stencil was maybe too lenient so i made it more strict, with a higher cutoff: http://sqpat.com/project/mysample38-noBordersEvenWorse.PNG Yeah... the really pixellated edges dont mesh with the filtered style for one, and then the fact that it looks like garbage after a zoom out (due to the same reason) Any ideas on how to make this still look right with linear filtering? Maybe something with texture stages, like a custom alpha channel or something crazy? I really don't know, haha. I get the feeling I'll have to switch to point filtering unless someone has a very clever idea... I may as well ask, though! Don't all help me out at once, now ;)

Premultiplied alpha is likely what you want.

Basically, you take normal alpha blending:

SrcBlend = source.a (multiply the source color by the source alpha)
DestBlend = 1-source.a (multiply the destination color by the inverse of the source alpha)

Basically, you (in a preprocess) multiply the color in the pixel by the alpha. What that does is make all pixels with 0 alpha be transparent black (and fades out any others). If, as it appears, your sprites are only ever alpha=1 or alpha=0, just make sure that the clear color is transparent black.

You then change the blend mode to:

SrcBlend = 1 (this is okay, the source has already been multiplied by the alpha)
DestBlend = 1-source.a (same as before)

What this does is, if you want to look at this way, darken the destination color to match the alpha and then add the entirety of the source to it.

It's very effective for eliminating unwanted borders on the edges of sprites.

Give it a go :)

[EDIT] long post and mostly nonsense, i answered these questions on my own [/EDIT]

[Edited by - sqpat on August 20, 2008 3:56:05 AM]

1) dont use color keying
2) use an alpha channel
3) extend the rgb pixels on the sprite a bit around all edges (can be done simply using the photoshop filter "maximum"

Quote:Original post by Matt Aufderheide
1) dont use color keying
2) use an alpha channel
3) extend the rgb pixels on the sprite a bit around all edges (can be done simply using the photoshop filter "maximum"

you'd still get partly transparent pixels on the border due to filtering which would mess with the zbuffer and look weird, i'm pretty sure.

ok i racked my brain on this all day and here's what i came up with

a. i realize i can have gimp or photoshop or something do pre-multiplied alpha for me ahead of time. but im not sure how to write the dx code to make it work to render these premultiplied alpha images

b. i tried to do the premultiplied alpha stuff in the code itself. this is what i ended up with:

(note: alpha blending is enabled)

p_dx_Device->SetRenderState(D3DRS_SRCBLEND, D3DBLEND_ONE);
p_dx_Device->SetRenderState(D3DRS_DESTBLEND, D3DBLEND_INVSRCALPHA);

[EDIT] okay i had a bunch of texturestagestates here that dont do anything. [/EDIT]

i tried both the old green XRGB color key and the transparent black ARGB color key with a bit of editing of the .png sprite sheet and either way i get these black lines:

http://sqpat.com/premultalpha.bmp

by the way i've also seen some stuff about the compressed textures... DXT4 mentioned alpha blending. but i wasn't sure if they were lossless or not (kind of important when talking about sprite accuracy). Should i be looking at something like that or is it lossy?

[Edited by - sqpat on August 20, 2008 3:08:05 AM]

Yup - using d3dxcreatetexturefromfileexa (whew that's a long one)

Hodgman, your solution seems to be the exact same one as Matt's, which won't work. It still has the z buffer artifact problem, and I don't want to order sprites and primitives to both be rendered back to front cause that'd get rid of all my batching and slow down the pipeline, quite likely.

You're also incorrect on your statement because pre-multiplied alpha blending, when shooting for opaque only sprites, seems quite feasible to get away with without z-buffer artifacts, unless I'm misinterpreting drillan's explanation.

I dont understand.. you cant have alpha blending AND use the Z-buffer.. it doesnt work that way (unless..you use alpha-to-coverage).

Normally with transparent sprites you just sort back to front..this can be easily batched by using D3DXSPRITE..

Why do people insist on making things harder than they should be?

EDIT:

There are some tricks to get what you want and use the Z-buffer so you dont have to sort if for some reason you dont want to.

option 1: a two-pass method... just render each sprite twice, the first with alpha bledning and soft edges, the second time with z-buffer and pixel perfect hard edges..this give you the look of anti aliased sprites, with mostly correct z-sorting.

option 2: render all sprites to a second render target twice the size of your main window, with z-buffer enabled and hard edges. Then after all else is rendered, just render this texture over the background on a screen-aligned quad with filtering, and you have perfect anti-aliased sprites.

option 3: i mentioned alpha-to-coverage.. this render state allows you fake alpha blending by using antialiasing and screendoor transparency.. this is not ideal in my opinion and not all cards support it.