Jump to content

  • Log In with Google      Sign In   
  • Create Account


Alpha Testing: I manage to "scorch" my edges


Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.

  • You cannot reply to this topic
26 replies to this topic

#1 QNAN   Members   -  Reputation: 223

Like
0Likes
Like

Posted 04 February 2013 - 11:28 PM

As I understand it, Alpha Testing will simply ignore those pixels, that do not pass the test.

I have made an alpha-test with  renderstates [D3DRS_ALPHAFUNC, D3DCMP_GREATER] and [D3DRS_ALPHAREF, 128].

 

Since the problem arose, I have simplified my texture, making the rgb-layer completely white, while preserving the alpha-layer.

 

I have further simplified my shader, so that it now simply outputs the texture raw, and that is all. No lighting or any other kinds of calculations.

 

The result is still scorched edges, as can be seen below:

 

alphatest_scorched.jpg

 

I would have expected a completely white plant, with no grey-scales (partial alpha) and no black edges.

How come, that the alpha is burned into the texture like that?

 

As can be seen, the pixel shader is extremely simple (I removed the argument list because it is irrelevant (except a_texcoord0)).

float4 PS_light_SHADOW(...) : COLOR
{
    return tex2D(g_texturemap_sampler, a_texcoord0);
}

My renderstates are as follows:

SetRenderState(D3DRS_ZENABLE, D3DZB_TRUE)
SetRenderState(D3DRS_ZWRITEENABLE, TRUE)
SetRenderState(D3DRS_CULLMODE, D3DCULL_NONE)
SetRenderState(D3DRS_ALPHATESTENABLE, TRUE)
SetRenderState(D3DRS_ALPHAFUNC, D3DCMP_GREATER)
SetRenderState(D3DRS_ALPHAREF, 128)

 

Thank you in advance.



Sponsor:

#2 Hodgman   Moderators   -  Reputation: 28615

Like
3Likes
Like

Posted 04 February 2013 - 11:46 PM

Your opaque areas seem to have an RGB value of white, but your transparent areas seem to have an RGB value of black.

You've set your alpha cut-off point at 50%, which means that due to bilinear filtering, the RGB value at that point will be 50% white and 50% black -- grey -- this is expected behaviour.

 

You need to fix your texture (not your code) so that the RGB values in the transparent areas is similar to their neighbouring opaque areas. If a transparent area is next to a green leaf, then it should also be green.

This is also very important when generating mip-maps, because they will also 'drag in' the black RGB values to the opaque areas...

 

What program are you using to create the textures, and what texture formats do you use?


Edited by Hodgman, 04 February 2013 - 11:50 PM.


#3 QNAN   Members   -  Reputation: 223

Like
0Likes
Like

Posted 05 February 2013 - 12:16 AM

I use Paint Shop Pro 7 (ya, I know, ancient, but simple and wonderfully fast) and Photoshop CS4.
I do all alpha work in Photoshop.
The texture is in this case .tga. I use many kinds of formats. .tga usually when testing something with alpha, although I would use .dds "live".

I will upload the original texture. Be aware, that it will only use the upper-left quarter of the texturemap.
The texture should be professional level.

I had to pack the texture in a .zip, as the server refused to have it downloaded.
http://www.samtalebasen.dk/test.zip

Edited by QNAN, 05 February 2013 - 12:18 AM.


#4 QNAN   Members   -  Reputation: 223

Like
0Likes
Like

Posted 05 February 2013 - 12:20 AM

Btw, I whited out the entire rgb layer - completely, meaning across borders - of the texture in the opening post. That should ensure, that the colors around the borders were the same.

Edited by QNAN, 05 February 2013 - 12:22 AM.


#5 MrDaaark   Members   -  Reputation: 3551

Like
2Likes
Like

Posted 05 February 2013 - 12:36 AM

Are you not reading Hodgman's post?

Textures get filtered. When you render them, their values come from more than a single pixel. It doesn't matter what color they are, the surrounding pixels will have an affect unless you are using NEAREST or POINT filtering. So 1 pixel in the image will have alpha 1, and the next will have alpha 0. But when interpolated during rendering, you will get some that average out to 0.5.

Using a similar color on the surrounding transparent pixels will help a bit.

Using the pre-multiplied alpha technique will help a lot more.

http://blogs.msdn.com/b/shawnhar/archive/2009/11/06/premultiplied-alpha.aspx

#6 Hodgman   Moderators   -  Reputation: 28615

Like
1Likes
Like

Posted 05 February 2013 - 01:52 AM

Your texture looks good --  the transparent areas do have colours similar to the opaque areas...

For the benefit of others, it looks like this:

z8pX2PY.png

 

How are you loading the texture? Perhaps your texture loader is changing the transparent areas to black?

 

Using the pre-multiplied alpha technique will help a lot more

This is good advice -- this black border artefact simply doesn't occur with pre-multiplied alpha blending.

However, that requires the use of alpha-blending, not just alpha testing that the OP is using, so it might not be applicable (e.g. if the correct sort order can't be guaranteed).



#7 eppo   Crossbones+   -  Reputation: 2342

Like
1Likes
Like

Posted 05 February 2013 - 03:13 AM

Btw, I whited out the entire rgb layer - completely, meaning across borders - of the texture in the opening post. That should ensure, that the colors around the borders were the same.


When do you do this? In Photoshop or during texture load time? Many photo editors have the nasty habit of zeroing out RGB values for 0-alpha pixels when they export an image.

Have you checked the texture in PIX?

Edited by eppo, 05 February 2013 - 03:16 AM.


#8 AliasBinman   Members   -  Reputation: 420

Like
2Likes
Like

Posted 05 February 2013 - 11:22 AM

A good trick to fix black borders especially when storing in 1bit DXT1 is to do the following in the shader.

color.rgb /=color.a;

So as you are start filtering towards a edge pixel you renormalize the colours.

#9 QNAN   Members   -  Reputation: 223

Like
0Likes
Like

Posted 05 February 2013 - 03:52 PM

I tried to alter the texture filtering in the shader to Point, with this result:
alphatest_scorched_filteringpoint.jpg
 
As can be seen, this does not solve my problem. The result is just pixelated.
 
This is how the sampler looks in the shader:

sampler2D g_texturemap_sampler =
sampler_state
{
	Texture = <g_texturemap>;
	MinFilter = Point;
	MagFilter = Point;
	MipFilter = Point;
};

 

I also tried to plug my texture into an ATI-demo called "Alpha to Coverage", replacing it with one of the other textures. The demo also draws AlphaTesting, which I set it to in the screenshot.

This is how it looks in the ATI demo:

alphatest_scorched_ATI_demo.jpg

 

There is no flaws, so I believe that this is not a texture problem - at least not at the file level.

 

Maybe in my creation?

D3DXCreateTextureFromFileExW(   m_device,
                                "../database/texture/foliage/gniff.tga", // gniff.tga is just a testname - it is the right one
                                D3DX_FROM_FILE,			// width
                                D3DX_FROM_FILE,			// height
                                D3DX_DEFAULT,			// miplevels
                                0,				// usage
                                D3DFMT_A8R8G8B8,		// format
                                D3DPOOL_MANAGED,		// pool
                                D3DX_DEFAULT,			// filter
                                D3DX_DEFAULT,			// mipfilter
                                0,				// colorkey
                                NULL,				// srcinfo
                                NULL,				// palette
                                a_textureDX))			// texture

 

If it is not any of these, could it then be a renderstate, that Im unaware of? Is there any renderstate, that can cause this?

 

Thank you for your help guys.


Edited by QNAN, 05 February 2013 - 03:57 PM.


#10 Hodgman   Moderators   -  Reputation: 28615

Like
3Likes
Like

Posted 05 February 2013 - 04:30 PM

As eppo said, you can use PIX to capture a frame of your game while it's running, and then view the it's resources. That will let you see what your texture looks like after it's been loaded into the GPU.

God knows what D3DXCreateTextureFromFileEx does internally... From the looks of it, it is "pre-multiplying" your RGB values with your alpha values... which as Daaark said, is helpful if you want to use alpha blending, but isn't helpful if you just want to use alpha-testing.

You could try loading the TGA file yourself instead of using the D3DX helper library to do it...
http://nothings.org/stb_image.c
http://nehe.gamedev.net/tutorial/loading_compressed_and_uncompressed_tga's/22001/

Edited by Hodgman, 05 February 2013 - 04:34 PM.


#11 AliasBinman   Members   -  Reputation: 420

Like
2Likes
Like

Posted 05 February 2013 - 05:54 PM

float4 PS_light_SHADOW(...) : COLOR
{
return tex2D(g_texturemap_sampler, a_texcoord0);
}

Change the body to

float4 col = tex2D(g_texturemap_sampler, a_texcoord0);
return float4(col.rgb/col.a, col.a);

#12 proanim   Members   -  Reputation: 440

Like
0Likes
Like

Posted 05 February 2013 - 05:56 PM

I don't know if the alpha texture is pre-made or you made it your self. If you made it your self try saving the texture as grayscale and not RGB, if possible when you convert your image to grayscale one save it with DXT format that uses alpha channel, and than see what happens. Also if that doesn't help, then try to use that alpha texture (grayscale one) with one of the shaders from above to see if that fixes the problem, for you.



#13 ankhd   Members   -  Reputation: 1195

Like
0Likes
Like

Posted 06 February 2013 - 03:04 AM

Hello I think i may be self shadowing ???



#14 Norman Barrows   Crossbones+   -  Reputation: 1991

Like
0Likes
Like

Posted 06 February 2013 - 07:00 AM

This looks like the classic problem of background colors getting blended into the edges of mipmaps. I spent a significant amount of time on this topic when it arose in my project. In short, what happens is that directx blends the background pixels at the edge of the visible part of the image into the edge pixels of the visible part of the image, resulting in a usually black or dark border around the visible part of the image. It appears to be a weakness or flaw in the mipmap generation algorithm in DirectX. Only pixels with alpha > threshold value should be blended. I'll dig up my notes and post the results. I tested just about every possible combo. point sampling removes the edges, but as i recall, the image disappears at certain viewing angles. this effect can be seen in some plants in Oblivion by Bethesda Softworks. I'll check my notes and post my findings. While my work is done in retained mode, not HLSL, you should be able to easily implement it in HLSL, perhaps with some help from folks here.

 

Norm Barrows

Rockland Software Productions

"Building PC games since 1988"


Norm Barrows

Rockland Software Productions

"Building PC games since 1988"

 

rocklandsoftware.net

 


#15 osmanb   Crossbones+   -  Reputation: 1500

Like
0Likes
Like

Posted 06 February 2013 - 07:23 AM

As eppo said, you can use PIX to capture a frame of your game while it's running, and then view the it's resources. That will let you see what your texture looks like after it's been loaded into the GPU.

God knows what D3DXCreateTextureFromFileEx does internally... From the looks of it, it is "pre-multiplying" your RGB values with your alpha values... which as Daaark said, is helpful if you want to use alpha blending, but isn't helpful if you just want to use alpha-testing.

You could try loading the TGA file yourself instead of using the D3DX helper library to do it...
http://nothings.org/stb_image.c
http://nehe.gamedev.net/tutorial/loading_compressed_and_uncompressed_tga's/22001/

 

QFT. Given the texture that we've seen (which does have correctly painted RGB channels extending outside the opaque portions), I strongly suspect D3DX is either explicitly pre-multiplying the alpha channel *or* the texture is ending up as DXT1 (BC1), which implicitly does the same thing.



#16 Norman Barrows   Crossbones+   -  Reputation: 1991

Like
0Likes
Like

Posted 06 February 2013 - 07:50 AM

From "Norm's DirectX Tips":

 

 

textures with transparent backgrounds and mipmapping:
 
use point sampling for the mipfilter when loading the texture.  D3DX_FILTER_NONE  is probaly undefined as a filter for loading textures and generating mipmaps, as its the flag used to turn mipmaps OFF!   using D3DX_FILTER_NONE results in directx copying a portion of each mipmap (the UL 1/4 ?) to the next level down, with no scaling or filtering or anything. this somehow results in textures that appear and disappear as their triangles move in relation to the camera. you can see this effect on some plants in oblivion. the upside to this is that with no filtering or sampling, no background color gets blended in, and you dont get any dark edges around the sprite images. 
so here's the deal with dark edges, sprites, and mipmapping:
sprite textures drawn without mipmaps tend to "sparkle" as the camera moves. the non-top level mipmaps in a sprite texture tend to have the background color blended into the edges of the image, either when directx makes the mipmaps, or when it samples them to texture a triangle.
when directx creates and / or uses mipmaps, it has a tendancy to blend the keycolor or background color into the pixels at the edge of the sprite image. linear, box, and triangle filters do this quite noticeably, point filtering only somewhat. playing with alpaha testing doesnt help much, as it only controls how diretcx draws these already messed up textures. weither the texture is colorkeyed, or has a black and white alpha channel or a grayscale alpha channel makes little difference. making your own mipmaps with black and white alpha helps very little, as directx still blends between miplevels. 
about the best you can do, is set the alpha test high (i use 128), and use textures that aren't too dark. the high alpha test filters out the black edges pretty well.
but ideally the apha test should be "if greater than 0, draw". for this, you'd have to go to a LOD scheme, where you had textures at 256,128,64, etc sizes, and draw sprite textured meshes with no mipmaps, using a different size texture based on range (IE do the mipmap manually).  but there, its possible you'd still get background color blended into the edges as directx scaled the texture. but maybe not, beacuse you get no black edges on the topmost sprite tex. you only get it on lower mipmap levels. 
another technique that showed promise was a texture that filled a 256x256 (stone in my case), and an alpha channel that had a white silouhette of the object (a distant rock). all mipmaps of all levels of this texcture were solid stone, with no background color, guaranteeing that the background color couldn't get sampled and blended in.
setting min and mag filters to point filtering  as opposed to anisotropic filtering may also help make sprite textures look better.
i was using sprite textures to draw billboards of distant rocks and plants. i kept moving the clip distance for meshes further out. i started at meshes up to 100 range, then billboards til 350 range. i got as far as meshes to 250 or 300 range and the rest billboards, and decided to just get rid of the billboards and draw all meshes out to 350 range.
 
=================================================================================
 
 
here's what i do in my current project:
 
// load sprite texture
void Zloadspritetex3(char *s)      //  as loadtex, w/ color key, and point mipfilter
{
HRESULT h;
char s2[100];
h=D3DXCreateTextureFromFileExA(Zd3d_device_ptr,s,
                           D3DX_DEFAULT,D3DX_DEFAULT,             // width,height
                           0,                                     // mip lvls. 0 or default = full chain, 1= no mip maps.
                           0,                                     // usage: not dynamic, not render tgt.
                           D3DFMT_A8R8G8B8,D3DPOOL_DEFAULT,
                           D3DX_FILTER_NONE,                      // image filter
                           D3DX_FILTER_POINT,                     // mip filter.  none=vanishing mipmaps problem. point=wrap problems. 
                                                                  // linear,tri,and box blend the keycolor into the edges of the image (dark edges problem).  
                                                                  // no mips causes sparkle problem.
                           0xFF000000,                                     //  0XFF000000, color key   
                           NULL,NULL,&(Ztex[numtextures].tex));
if (h != D3D_OK) { strcpy_s(s2,100,"Error loading "); strcat_s(s2,100,s); Zmsg2(s2); exit(1); }
strcpy_s(Ztex[numtextures].name,100,s);
numtextures++;
}
 
 
 
 
 
 
int Zalphatestlvl=128;
 
 
 
 
// turn alphatesting on/off
void Zalphatest(int onoff)
{
if (onoff==1)
  {
  Zd3d_device_ptr->SetRenderState(D3DRS_ALPHATESTENABLE, TRUE); 
  Zd3d_device_ptr->SetRenderState(D3DRS_ALPHAFUNC, D3DCMP_GREATEREQUAL);       //   D3DCMP_NOTEQUAL
  Zd3d_device_ptr->SetRenderState(D3DRS_ALPHAREF,(DWORD)Zalphatestlvl);        //    (DWORD)0x00000000
  }
else
  {
  Zd3d_device_ptr->SetRenderState(D3DRS_ALPHATESTENABLE,FALSE); 
  }
}
 
 
 
unless you want to generate your own mipmaps that don't blend the background into the visible image edges, this is about the best you can do.  
 
I also read about one game that first did alphatest pass, then an alphablend pass to draw the fine edges of the visible image that get truncated in a simple alphatest.
 
Hope this helps!
 
Norm Barrows
Rockland Software Productions - Building PC games since 1988

Norm Barrows

Rockland Software Productions

"Building PC games since 1988"

 

rocklandsoftware.net

 


#17 DrColossus   Members   -  Reputation: 469

Like
0Likes
Like

Posted 06 February 2013 - 07:56 AM

Do you render the alpha tested geometry before or after you render opaque geometry?

 

If you render it first, it'll still write the depth values for pixels with alpha < 1.0. The terrain would then fail the depth comparison for those pixels and your clearcolor would become visible.



#18 AliasBinman   Members   -  Reputation: 420

Like
0Likes
Like

Posted 06 February 2013 - 11:48 AM

Do you render the alpha tested geometry before or after you render opaque geometry?
 
If you render it first, it'll still write the depth values for pixels with alpha < 1.0. The terrain would then fail the depth comparison for those pixels and your clearcolor would become visible.

This is only true for alpha blending. For alpha testing each pixel is either opaque or clipped. Of course its possible to enable both in which case your point is valid.

#19 QNAN   Members   -  Reputation: 223

Like
0Likes
Like

Posted 06 February 2013 - 09:09 PM

I am hard at work testing different approaches. I got drowned in suggestions here smile.png

My first attempt will be to write my own loader, so I can be sure D3DX is not messing with me.

 

I will get back with a full report when I come out of the pixel mines again smile.png

 

 

EDIT:

Actually Im gonna install PIX first, it looks like it can save me the trouble, if D3DX is actually doing its job properly.


Edited by QNAN, 06 February 2013 - 09:20 PM.


#20 phil_t   Crossbones+   -  Reputation: 3234

Like
1Likes
Like

Posted 06 February 2013 - 09:13 PM

My first attempt will be to write my own loader, so I can be sure D3DX is not messing with me.

Much easier than writing your own loader would be to just take a few minutes and take frame capture in PIX to see the exact pixel values of the source texture and make sure they're right.




Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.



PARTNERS