Sign in to follow this  
Shenjoku

DirectX 9 Image Masking Without Shaders

Recommended Posts

Shenjoku    161
I'm trying to figure out what the most efficient method is of applying an alpha transparency mask to a DirectX 9 texture in memory without using shaders. The current idea for implementing it is to do something like the following:

[CODE]
LPDIRECT3DTEXTURE9 texture, mask;

// Get the texture and the mask to apply from somewhere.

texture->Lock();
mask->Lock();

// Copy only the alpha channel from the mask into the texture, effectively applying the mask.
for (int32u i = 0; i < textureWIdth; ++i)
{
textureBits.alpha = maskBits.alpha;
}

texture->Unlock();
mask->Unlock();
[/CODE]

As you can see, this is a rather brute force method that would end up being extremely slow on large images, but I don't know of anything in DirectX 9 for doing this besides using shaders. So my question to all of you: Are there any DirectX helper function that can be used to obtain the same results?

I'm not sure if this changes anything, but the texture I'm trying to apply the mask to is being used in off-screen rendering, so it is a render target. Edited by Shenjoku

Share this post


Link to post
Share on other sites
Nik02    4348
This is possible in fixed function texture stages; sample the color texture in the first stage, and in the second stage, get the alpha from the second texture but take the color from the previous stage (D3DTA_CURRENT). The texture stage setup runs on the GPU hardware, unlike your proposed approach.

A pixel shader would be more simple here, though [img]http://public.gamedev.net//public/style_emoticons/default/smile.png[/img] Edited by Nik02

Share this post


Link to post
Share on other sites
Shenjoku    161
[quote name='Nik02' timestamp='1340262057' post='4951237']
This is possible in fixed function texture stages; sample the color texture in the first stage, and in the second stage, get the alpha from the second texture but take the color from the previous stage (D3DTA_CURRENT). The texture stage setup runs on the GPU hardware, unlike your proposed approach.
[/quote]

That sounds interesting, but I have no idea how to do something like that. Does this have to be done every frame before rendering or something? If you have any articles or anything on it or could post some pseudo code that would be very appreciated [img]http://public.gamedev.net//public/style_emoticons/default/smile.png[/img] I'll try to do a bit of research on my own though.

[quote name='Nik02' timestamp='1340262057' post='4951237']
A pixel shader would be more simple here, though [img]http://public.gamedev.net//public/style_emoticons/default/smile.png[/img]
[/quote]

I know it would be, but the engine I'm using has no support for shaders whatsoever and it would be rather difficult to add it, especially with the time constraints on the project.

EDIT: Well I looked into it and I sort of understand what to do, but I can't figure out the mixture of texture stage states to get it working. This is what I have currently, which is resulting in no change:

[CODE]
device->SetTexture(0, fTexture);
device->SetTexture(1, fMaskTexture);

// Only use the color from the first texture.
device->SetTextureStageState(0, D3DTSS_COLOROP, D3DTOP_MODULATE);
device->SetTextureStageState(0, D3DTSS_COLORARG1, D3DTA_TEXTURE);
device->SetTextureStageState(0, D3DTSS_COLORARG2, D3DTA_DIFFUSE);
device->SetTextureStageState(0, D3DTSS_ALPHAOP, D3DTOP_MODULATE);
device->SetTextureStageState(0, D3DTSS_ALPHAARG1, D3DTA_TEXTURE);
device->SetTextureStageState(0, D3DTSS_ALPHAARG2, D3DTA_DIFFUSE);

// Then use the color from the previous texture, and the alpha from the mask.
device->SetTextureStageState(1, D3DTSS_COLOROP, D3DTOP_SELECTARG1);
device->SetTextureStageState(1, D3DTSS_COLORARG1, D3DTA_CURRENT);
device->SetTextureStageState(1, D3DTSS_COLORARG2, D3DTA_CURRENT);
device->SetTextureStageState(1, D3DTSS_ALPHAOP, D3DTOP_MODULATE);
device->SetTextureStageState(1, D3DTSS_ALPHAARG1, D3DTA_TEXTURE);
device->SetTextureStageState(1, D3DTSS_ALPHAARG2, D3DTA_DIFFUSE);
[/CODE] Edited by Shenjoku

Share this post


Link to post
Share on other sites
Shenjoku    161
Just rendering it to a primitive right below those calls with:

[CODE]
device->SetSamplerState(0, D3DSAMP_MAGFILTER, fD3DTextureFilterType);
device->SetSamplerState(0, D3DSAMP_MINFILTER, fD3DTextureFilterType);

device->SetStreamSource(0, fVertexBuffer, 0, sizeof(DXUVert_TEXTURED));

device->DrawPrimitive(D3DPT_TRIANGLEFAN, 0, 2);
[/CODE]

Share this post


Link to post
Share on other sites
Shenjoku    161
Currently they are being set to:

[CODE]
device->SetRenderState(D3DRS_SRCBLEND, D3DBLEND_SRCALPHA);
device->SetRenderState(D3DRS_DESTBLEND, D3DBLEND_INVSRCALPHA);
device->SetRenderState(D3DRS_ALPHABLENDENABLE, TRUE);
[/CODE]

Maybe there's something wrong with the mask I'm using? Here are the images I'm testing with:

The main image to apply the mask to.
[attachment=9596:test.png]

The image mask to apply. It has a green color just to make it more obvious if there was a problem with the color part. The empty areas have zero alpha, which needs to be applied to the first image somehow.
[attachment=9597:testMask.png]

This is what the result should be.
[attachment=9598:testResult.png]

Share this post


Link to post
Share on other sites
Shenjoku    161
Well I sort of figured out what the problem is but I don't know how to fix it. After tinkering a bit I noticed that for some reason it's not picking up the alpha in the mask at all. If I set the states to:

[CODE]
// Combine the colors and use alpha from the mask only.
device->SetTextureStageState(1, D3DTSS_COLOROP, D3DTOP_MODULATE);
device->SetTextureStageState(1, D3DTSS_COLORARG1, D3DTA_CURRENT);
device->SetTextureStageState(1, D3DTSS_COLORARG2, D3DTA_TEXTURE);
device->SetTextureStageState(1, D3DTSS_ALPHAOP, D3DTOP_SELECTARG2);
device->SetTextureStageState(1, D3DTSS_ALPHAARG1, D3DTA_CURRENT);
device->SetTextureStageState(1, D3DTSS_ALPHAARG2, D3DTA_TEXTURE);
[/CODE]

The texture gets tinted green as you would expect, except that the areas that are supposed to have zero alpha are also tinted green. Those areas should remain the same shouldn't they? The only logical reason why that would be happening would be because the alpha in the mask isn't getting applied or is getting ignored for some reason.

EDIT: After even more investigation I've narrowed it down to what the problem is. For some reason it's only using a single pixel, the top-left pixel at (0, 0), from the mask to apply to the entire texture. Anyone know why it would be doing this?

EDIT2: Think I figured it out! Took a lunch break to clear my head and realized something extremely important was mising; A second set of texture coordinates in the vertex data. I haven't tested it yet but I'm pretty sure this is why it isn't working. I'll post again once I confirm. Edited by Shenjoku

Share this post


Link to post
Share on other sites
Shenjoku    161
Well that did it, now it's working perfectly. Thanks a lot for you help Nik02. In case anyone else needs help here's some pseudo code for how to get this working:

[CODE]
// The vertex struct to use has two sets of texture coordinates, one for the main texture and one for the mask.
struct tDXVertexTextureMask
{
FLOAT x, y, z;
D3DCOLOR color;
FLOAT u1, v1;
FLOAT u2, v2;
};

// Define the FVF to use.
#define D3DFVF_TEXTURE_MASK (D3DFVF_XYZ | D3DFVF_DIFFUSE | D3DFVF_TEX2)

// Create your vertex buffer like usual using the above struct and FVF. When applying the texture coordinates,
// make sure to set u1 = u2 and v1 = v2, so the textures will line up exactly.

// Time to render! Setup the FVF and texture states.
device->SetFVF(D3DFVF_TEXTURE_MASK);

device->SetTexture(0, fTexture);
device->SetTexture(1, fMaskTexture);

device->SetTextureStageState(0, D3DTSS_COLOROP, D3DTOP_MODULATE);
device->SetTextureStageState(0, D3DTSS_COLORARG1, D3DTA_TEXTURE);
device->SetTextureStageState(0, D3DTSS_COLORARG2, D3DTA_DIFFUSE);
device->SetTextureStageState(0, D3DTSS_ALPHAOP, D3DTOP_MODULATE);
device->SetTextureStageState(0, D3DTSS_ALPHAARG1, D3DTA_TEXTURE);
device->SetTextureStageState(0, D3DTSS_ALPHAARG2, D3DTA_DIFFUSE);

// Use the color from the previous texture, and blend the alpha from the mask.
device->SetTextureStageState(1, D3DTSS_COLOROP, D3DTOP_SELECTARG1);
device->SetTextureStageState(1, D3DTSS_COLORARG1, D3DTA_CURRENT);
device->SetTextureStageState(1, D3DTSS_ALPHAOP, D3DTOP_MODULATE);
device->SetTextureStageState(1, D3DTSS_ALPHAARG1, D3DTA_TEXTURE);
device->SetTextureStageState(1, D3DTSS_ALPHAARG2, D3DTA_CURRENT);

// Render your primative and voila! Masked texture!
[/CODE] Edited by Shenjoku

Share this post


Link to post
Share on other sites
Shenjoku    161
Oh that's cool! That would be helpful if I needed them to be exactly the same, which I do for now, but eventually they are going to have to be calculated separately and will have different values potentially.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this