• Advertisement

Archived

This topic is now archived and is closed to further replies.

Anti-aliasing problems DirectX

This topic is 5091 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

Hi, My game is nearly finished but I''m having some problems with my anti-aliasing of the tiles, unfortunatly. I''ve been rendering my objects in 3DSMax, and just found out a week ago about TGA files and an Alpha Channel. I''ve used Colorkeys before, but it made the objects look aliased around it (the black border), like this: With Alpha Channels it got a lot better, but unfortunatly I''m stuck with an annoying problem. There is some grey pixels coming around my mirrors, caused by the anti-aliasing. Although barely visibile in PSP, it is clearly visible in-game. It produces results like: Now, simply turning off anti-aliasing works quite well, but then I have to filter the texture, which causes some grey pixels again. Also, when I rotate a mirror on 45 degrees or something, it will look terribly aliased, like you can see here ( in the full game it is more visible, but if you look closely, you will see what I mean ): I''ve spent like four full evenings trying to get this solved, without having to edit ALL my tiles by hand, to remove this grey stuff. I''ve also tried rendering it 64x64 and scaling it down to 32x32. didn''t exactly work well either. Searching all the Gamedev forums which are somehow related to this issue, didn''t give much of a help either . I''ve been googling around as well, for quite some keywords, but still no solution. FSAA and stuff is not an option Can someone maybe shed some light which anti-aliasing method to use in 3DSMax, or some tips how to get rid of these grey surroundings? (Without having to manually fix all my tiles...)

Share this post


Link to post
Share on other sites
Advertisement
Max has a preference setting, "Don''t antialias against background". Maybe that is just the thing you need

-Nik

Share this post


Link to post
Share on other sites
Been there . It results in choppy mirrors, well, just the same happens as mentioned above. I''m pretty clueless. But some people in commerical games obviously get it right, and I''m sure someone could explain how exactly . Well I hope

Share this post


Link to post
Share on other sites
A suggestion:
Use a colorkey as usual to make the background color completely transparent (i.e. 0 alpha). Then, for every pixel in your tile, calculate the distance between it and the nearest background pixel, and use this as an estimator for determining the pixel''s alpha value.

Let''s call the distance from the current pixel to the nearest background pixel "dist". Let''s say we''re going to blend pixels that are less than 6 pixels away from background pixels.

pixel.alpha = min( 1, dist/6 )

That way, after you enable alpha blending, pixels that are away from background pixels by less than 6 pixels are "blended" with the background, increasingly as you go near the background (i.e. near the edges of the tile).

This can reduce the aliasing you''re seeing, although I can''t gurantee it''ll look right. Anyway, you can easily try this on a couple of tiles, and if the result is satisfactory, apply it to all others. Make sure you play with the "6" value until you find what''s best for your tiles.

Muhammad Haggag

Share this post


Link to post
Share on other sites
Hmm, that''s an interesting theory. I miht try that. Still, I think there should be a more simple solution somehow

Share this post


Link to post
Share on other sites
Render your disc against a black background with antialiasing enabled and whatever t takes to generate alpha in the image. Then set the texture stage state to use a color op of D3DTOP_BLENDTEXTUREALPHAPM.

Share this post


Link to post
Share on other sites
Hm, then it got all white, but with the mirror shape. Then I tried without the "M" and got the same situation



The right images get''s all weird grey stuff. This is just the rotated version (rotating the vertex quad) of the left one..

Share this post


Link to post
Share on other sites
You''ve got a problem with whatever tool you''ve used to generate the alpha values for the disc. If you ask the tool to antialias against a background and it is generating alpha values, it should scale the alpha values near the edges by the same amount that it has scaled the color values to perform the blending.

This evening I generated a similar disc using Rhino3D. Rhino won''t generate alpha values when rendering so I had to improvise.

I rendered two images of the disc with antialiasing enabled; one over a black background and then another over a white background. Neither of these images contain any alpha information.

I then wrote a small app that constructed an alpha map using the two images comparing white and black pixels and pixels that were identical between the two images.

I used the resulting alpha map with the image that I rendered over the black background in the MFC Texture DXSDK sample, using the premultiplied blending mode I mentioned before, and it worked perfectly.

I don''t know how to post images here otherwise I''d show you the series of 64x64 bitmaps I generated.

There''s got to be a better way to do this than having to write code to perform 2D post-processing of rendered images. You might want to ask in the Visual Arts forum.

Share this post


Link to post
Share on other sites
quote:
I don''t know how to post images here otherwise I''d show you the series of 64x64 bitmaps I generated.

You can go here: http://www.freepgs.com/cgi-bin/easyhost.cgi
Login:
User: MHaggag
password: gdupload

Upload whatever you wish, and directly link to it from here (img src="bla bla bla" stuff).

And yes, anyone wishing to upload can use this. It''s a 20 MB free space, with a limit of 20 MB daily transfer, 2 GB monthly (I *think*).

Muhammad Haggag

Share this post


Link to post
Share on other sites
Here''s an more detailed explanation on how I used a post-process technique to generate the appropriate alpha data.

First I generated two 64x64 pixel images of a disc using Rhino3D. Both images were rendered with antialiasing enabled. The only difference between the two images is the color of the background (black vs. white):





Next I wrote a small app that parsed the pixel data of each of these two bitmaps to generate an alpha mask. White pixels in the image with a white background, that were also black in the image with a black background are transparent. Pixels that are the same color between the two images are opaque. The remaining pixels have some degree of translucency. Here is an image showing the output of the first stage of this process. Black pixels are transparent, white pixels are opaque, and green pixels are translucent:




In order to get the various values of the translucent pixels you can use the typical alpha blending equation and substitute portions of the equation with the color values of pixels in the images with the white and black background:

color = (src_color * alpha) + dest_color * (1.0 - alpha)

In the image with the black background, the dest_color is 0.0 (black). This simplifies the equation to:

color = (src_color * alpha)

We know the value of ''color'' in the equation above already since it''s the pixel color in the image with the black background.

We need to find the other two unknowns, src_color and alpha. (Actually if we''re eventually going to be doing premultiplied blending, we only need the alpha value). To find the alpha value, we need to bring in the pixel from the image with the white background where dest_color in the alpha equation is 1.0 (white).

pixel_from_white_bkgnd_image = (src_color * alpha) + 1.0 - alpha

Substituting in the pixel value from the image with the black background for (src_color * alpha) and a little shuffling we get:

alpha = 1.0 - (pixel_from_black_bkgnd_image - pixel_from_white_bkgnd_image)

The following image is the completed alpha map with the green pixels replaced by the appropriate alpha values:




Once you have the alpha value you can solve for the src_color value in the alpha blending equation and produce a texture map that contains the proper color and alpha values. I used the premultiplied alpha blending mode I mentioned in previous posts (this is the image with the black background with the computed alpha map added) to generate the following image of a disc rendered on a gouraud shaded quad. As you can see, there are no gray pixels or odd halos around the edges of the disc and the image is antialiased quite nicely:





I hope that explains the process and you can gather something from it. It really sounds like whatever tool you''re using to generate your alpha mask isn''t generating the correct values. If you get really stuck you can use the technique I''ve described above to generate the correct values. Good luck.

Share this post


Link to post
Share on other sites
Wow! That sure looks great. I hope it works well with rotating too. Are you adding that alpha texture as alpha layer to the actual texture, or simply using it as a second texture while rendering? . That is the only thing that isn''t completely clear

I''m going to check this out tommorow, it''s quite late overhere now . Again, thanks!

Share this post


Link to post
Share on other sites
Well I don't know what you're looking for when it's rotated because once you get things right it shouldn't matter how it's translated or rotated. Here are some rotated images. The first is rotating the texture mapped quad, and the second is performing a rotation with a texture transformation. Looks OK to me.






There was no point in keeping the alpha map as a separate texture so I merged it with the texture containing the color data. No need to burn an extra texture stage if it's not necessary.



[edited by - don on February 9, 2004 6:58:51 PM]

Share this post


Link to post
Share on other sites
Well, I managed to work the alpha "processor" out . Looks exaclty the same as your picture. Unfortunatly, I have programmed so much (deadlines ) today that my arm and hand hurts . So, I will try the actual texture stuff tommorow or Friday

Share this post


Link to post
Share on other sites
Hmm, unfortunatly.. still problems



Looks alright..



With the PM colorop



Without the PM colorop.

I'm not sure what I'm doing wrong. I've added the first picture as alpha channel to my mirror picture.


[edited by - ajoling on February 12, 2004 2:23:44 PM]

Share this post


Link to post
Share on other sites
If you can generate the 32-bpp bitmaps of the images with a black and with a white background and post them on a website where I can download them, I''ll be happy to have a look at them and attempt to generate the proper alpha channel.

Share this post


Link to post
Share on other sites
I have included the graphics - as generated by 3dsmax -in the zip file, together with my alpha processor (It's VB code though )
zip file

For people who are interested in the source code: It will be put on my site eventually, so you don't have to download this not much commented version..

The pictures in the exe are "hardcoded" right now, btw. But I bet your program generates exactly the same alpha-map.

Well, I wonder what for result you get Thanks in advance!

[edited by - ajoling on February 12, 2004 3:58:00 PM]

Share this post


Link to post
Share on other sites
Thanks. I''ll look at it later after I get home from work.

BTW, are you sure you''re using the correct texture map in the image you posted marked "With the PM colorop"? That image is way off and looks like you tried to draw some sort of antenna or raygun instead of a mirror (disc).

Share this post


Link to post
Share on other sites
I merged your alpha map and image with black background to create a D3DFMT_A8R8G8B8 texture map, then rendered it (rotated as you prefer) using D3DTOP_BLENDTEXTUREALPHAPM and it worked fine. I''ve posted two images below, one with point sampling and one with linear sampling drawn using the reference rasterizer. Neither image displays the artifacts in the images that you''ve posted in this thread.








Share this post


Link to post
Share on other sites
Well, I guess it might be some other renderstates then.. or.. could it be that I''m running 16 bit mode? . But then again, the PM version simply doesn''t show anything, except for a white mirror.

I used a simple tutorial from DirectX4Vb.com to make sure my game wasn''t doing something weird. Same result:

small zip file (click to exit). I''m not sure if you are able to use VB, but if you see the code, but in Notepad you''ll simply see it as well, and how I have my renderstate set.

(Oh: and that "antenna" is my temporary mouse cursor . I have to reset the texture coords back so it is pointing top-left)

Thanks again!

Share this post


Link to post
Share on other sites
Disable alpha blending (D3DRS_ALPHABLENDENABLE = FALSE) and see if your problems go away.


Share this post


Link to post
Share on other sites
Then it''s just one white square .

Hm. The test thing you made yourself, If it''s C(++) or something, I can probably read it .

And I''m going to test it on a different computer (which is a few generations younger than this one ) later today when I''m back from work.

Share this post


Link to post
Share on other sites
Not sure if you already know this but I thought I'd try to help anyways:

3dsmax automatically generates an alpha channel that is PERFECTLY what you need. Just make sure you save the image in a format that supports alpha (eg. TGA is the easiest). Also in Max, MAKE SURE that in the format settings ("Setup" button in save dialog I think), you DISABLE "pre-multiplied alpha" (ie. clear the checkbox).

Then your program should load it all in correctly and display it perfectly. I've never had a problem with this technique, except as I stated: that pre-multipled alpha is bad, bad, bad


Also note that PSP has a problem saving TGA files in that it ALWAYS premultiplies the alpha... there is a workaround (through a plugin), that I can get the link for when I get home if you wish. Realize that for your image to look correct with the alpha channel, it will look BAD without it. Premultiplied alpha tries to correct this problem, but as you have noted, it tends to mess up some other things.

[edited by - AndyTX on February 13, 2004 11:37:00 AM]

Share this post


Link to post
Share on other sites
Hmm, that could be the problem. But how should the renderstates be then? I''m tryin every combination right now. The one time, the mirror is alright, but with a black background (no change). And the other time it''s a mirror.. with a white surrounding! I never get it to show the mirror only

Share this post


Link to post
Share on other sites
Don could you maybe send me that little demo you made? I''m curious if it works overhere. I just made a demo myself which tries 6 different renderstate combinations, one row is a mirror, with a white surrounding (just the black turning white)

Now, while I typed this, I was wondering about alpha testing. Which seems to do the trick. But then, I''m still having a white surrounding . So I''m wondering if you could e-mail it to me or something (see my profile). Would be great!

Share this post


Link to post
Share on other sites
Rather than send you my code, I''ve modified one of the D3D9 SDK tutorial samples to load the .tga texture generated by using your black background image and alpha map, and render it on a centered quad. I''ve put the merged 32x32 tga texture in the zip file.

Zip file here.

Share this post


Link to post
Share on other sites

  • Advertisement