Alpha Testing: I manage to "scorch" my edges

Started by
25 comments, last by phil_t 11 years, 2 months ago

As I understand it, Alpha Testing will simply ignore those pixels, that do not pass the test.

I have made an alpha-test with renderstates [D3DRS_ALPHAFUNC, D3DCMP_GREATER] and [D3DRS_ALPHAREF, 128].

Since the problem arose, I have simplified my texture, making the rgb-layer completely white, while preserving the alpha-layer.

I have further simplified my shader, so that it now simply outputs the texture raw, and that is all. No lighting or any other kinds of calculations.

The result is still scorched edges, as can be seen below:

alphatest_scorched.jpg

I would have expected a completely white plant, with no grey-scales (partial alpha) and no black edges.

How come, that the alpha is burned into the texture like that?

As can be seen, the pixel shader is extremely simple (I removed the argument list because it is irrelevant (except a_texcoord0)).


float4 PS_light_SHADOW(...) : COLOR
{
    return tex2D(g_texturemap_sampler, a_texcoord0);
}

My renderstates are as follows:


SetRenderState(D3DRS_ZENABLE, D3DZB_TRUE)
SetRenderState(D3DRS_ZWRITEENABLE, TRUE)
SetRenderState(D3DRS_CULLMODE, D3DCULL_NONE)
SetRenderState(D3DRS_ALPHATESTENABLE, TRUE)
SetRenderState(D3DRS_ALPHAFUNC, D3DCMP_GREATER)
SetRenderState(D3DRS_ALPHAREF, 128)

Thank you in advance.

Advertisement

Your opaque areas seem to have an RGB value of white, but your transparent areas seem to have an RGB value of black.

You've set your alpha cut-off point at 50%, which means that due to bilinear filtering, the RGB value at that point will be 50% white and 50% black -- grey -- this is expected behaviour.

You need to fix your texture (not your code) so that the RGB values in the transparent areas is similar to their neighbouring opaque areas. If a transparent area is next to a green leaf, then it should also be green.

This is also very important when generating mip-maps, because they will also 'drag in' the black RGB values to the opaque areas...

What program are you using to create the textures, and what texture formats do you use?

I use Paint Shop Pro 7 (ya, I know, ancient, but simple and wonderfully fast) and Photoshop CS4.
I do all alpha work in Photoshop.
The texture is in this case .tga. I use many kinds of formats. .tga usually when testing something with alpha, although I would use .dds "live".

I will upload the original texture. Be aware, that it will only use the upper-left quarter of the texturemap.
The texture should be professional level.

I had to pack the texture in a .zip, as the server refused to have it downloaded.
http://www.samtalebasen.dk/test.zip
Btw, I whited out the entire rgb layer - completely, meaning across borders - of the texture in the opening post. That should ensure, that the colors around the borders were the same.
Are you not reading Hodgman's post?

Textures get filtered. When you render them, their values come from more than a single pixel. It doesn't matter what color they are, the surrounding pixels will have an affect unless you are using NEAREST or POINT filtering. So 1 pixel in the image will have alpha 1, and the next will have alpha 0. But when interpolated during rendering, you will get some that average out to 0.5.

Using a similar color on the surrounding transparent pixels will help a bit.

Using the pre-multiplied alpha technique will help a lot more.

http://blogs.msdn.com/b/shawnhar/archive/2009/11/06/premultiplied-alpha.aspx

Your texture looks good -- the transparent areas do have colours similar to the opaque areas...

For the benefit of others, it looks like this:

z8pX2PY.png

How are you loading the texture? Perhaps your texture loader is changing the transparent areas to black?

Using the pre-multiplied alpha technique will help a lot more

This is good advice -- this black border artefact simply doesn't occur with pre-multiplied alpha blending.

However, that requires the use of alpha-blending, not just alpha testing that the OP is using, so it might not be applicable (e.g. if the correct sort order can't be guaranteed).

Btw, I whited out the entire rgb layer - completely, meaning across borders - of the texture in the opening post. That should ensure, that the colors around the borders were the same.


When do you do this? In Photoshop or during texture load time? Many photo editors have the nasty habit of zeroing out RGB values for 0-alpha pixels when they export an image.

Have you checked the texture in PIX?
A good trick to fix black borders especially when storing in 1bit DXT1 is to do the following in the shader.

color.rgb /=color.a;

So as you are start filtering towards a edge pixel you renormalize the colours.

I tried to alter the texture filtering in the shader to Point, with this result:
alphatest_scorched_filteringpoint.jpg

As can be seen, this does not solve my problem. The result is just pixelated.

This is how the sampler looks in the shader:


sampler2D g_texturemap_sampler =
sampler_state
{
	Texture = <g_texturemap>;
	MinFilter = Point;
	MagFilter = Point;
	MipFilter = Point;
};

I also tried to plug my texture into an ATI-demo called "Alpha to Coverage", replacing it with one of the other textures. The demo also draws AlphaTesting, which I set it to in the screenshot.

This is how it looks in the ATI demo:

alphatest_scorched_ATI_demo.jpg

There is no flaws, so I believe that this is not a texture problem - at least not at the file level.

Maybe in my creation?


D3DXCreateTextureFromFileExW(   m_device,
                                "../database/texture/foliage/gniff.tga", // gniff.tga is just a testname - it is the right one
                                D3DX_FROM_FILE,			// width
                                D3DX_FROM_FILE,			// height
                                D3DX_DEFAULT,			// miplevels
                                0,				// usage
                                D3DFMT_A8R8G8B8,		// format
                                D3DPOOL_MANAGED,		// pool
                                D3DX_DEFAULT,			// filter
                                D3DX_DEFAULT,			// mipfilter
                                0,				// colorkey
                                NULL,				// srcinfo
                                NULL,				// palette
                                a_textureDX))			// texture

If it is not any of these, could it then be a renderstate, that Im unaware of? Is there any renderstate, that can cause this?

Thank you for your help guys.

As eppo said, you can use PIX to capture a frame of your game while it's running, and then view the it's resources. That will let you see what your texture looks like after it's been loaded into the GPU.

God knows what D3DXCreateTextureFromFileEx does internally... From the looks of it, it is "pre-multiplying" your RGB values with your alpha values... which as Daaark said, is helpful if you want to use alpha blending, but isn't helpful if you just want to use alpha-testing.

You could try loading the TGA file yourself instead of using the D3DX helper library to do it...
http://nothings.org/stb_image.c
http://nehe.gamedev.net/tutorial/loading_compressed_and_uncompressed_tga's/22001/

This topic is closed to new replies.

Advertisement