Hi,

While trying to debug my performance issue, I noticed that in release mode alphablending is not working. Every alpha blendet vertex I draw fills the pixels with alpha < 1.0 with a random color (usually grey/black). I am using a deferred renderer, hdr (tonemap) and postprocessing. Of course I'm using custom shaders, some with premultiplied alpha, some with normal blending. Any idea what causes this/how to fix it? Thanks!

Hi,

While trying to debug my performance issue, I noticed that in release mode alphablending is not working. Every alpha blendet vertex I draw fills the pixels with alpha < 1.0 with a random color (usually grey/black). I am using a deferred renderer, hdr (tonemap) and postprocessing. Of course I'm using custom shaders, some with premultiplied alpha, some with normal blending. Any idea what causes this/how to fix it? Thanks!

Did you try the directx debug runtimes, even in release mode? There must be some problem in your code making it behave differently in debug and release modes (that's unfortunate, but not that uncommon ) and the directx debug output maybe could show you the problem. Maybe.

PIX can also be useful for debugging pixels.

I bet you have an uninitialised variable somewhere that's getting filled with random garbage in release, and 0xcccccccc or some other recognisable pattern in debug.

@Tom KQT:

Unfortunately, there isn't any helpful output from directx debug in release mode other than quite a lot of redundant state changes (which occour in debug too of course)..

@Evil Steve:

PIX is not working for me, at least frame-capture. It crashes upon pressing F12 with that log:
C, 0x0013FD60, 0x0013FD54, D3DCOLOR_ARGB(0xff,0xff,0xff,0xff))
Frame 000172 ........PRE: <this=0x056bac58>ID3DXSprite::Draw(0x0570FAF0, 0x0013FD7C, 0x0013FD60, 0x0013FD54, D3DCOLOR_ARGB(0xff,0xff,0xff,0xff))
Frame 000172 ............PRE: <this=0x0570faf0>IDirect3DTexture9::GetLevelDesc(0, 0x0013FBB4)
Frame 000172 ............POST: <S_OK><this=0x0570faf0> IDirect3DTexture9::GetLevelDesc(0, 0x0013FBB4)
Frame 000172 ............PRE: <this=0x0570faf0>IDirect3DTexture9::AddRef()
Frame 000172 ............POST: <26><this=0x0570faf0> IDirect3DTexture9::AddRef()
Frame 000172 ........POST: <S_OK><this=0x056bac58> ID3DXSprite::Draw(0x0570FAF0, 0x0013FD7C, 0x0013FD60, 0x0013FD54, D3DCOLOR_ARGB(0xff,0xff,0xff,0xff))
Frame 000172 ........PRE: <this=0x056bac58>ID3DXSprite::Draw(0x0570FAF0, 0x0013FD7C, 0x0013FD60, 0x0013FD54, D3DCOLOR_ARGB(0xff,0xff,0xff,0xff))
Frame 000172 ............PRE: <this=0x0570faf0>IDirect3DTexture9::AddRef()
Frame 000172 ............POST: <27><this=0x0570faf0> IDirect3DTexture9::AddRef()
Frame 000172 ........POST: <S_OK><this=0x056bac58> ID3DXSprite::Draw(0x0570FAF0, 0x0013FD7C, 0x0013FD60, 0x0013FD54, D3DCOLOR_ARGB(0xff,0xff,0xff,0xff))
Frame 000172 ........PRE: <this=0x056bac58>ID3DXSprite::End()
Frame 000172 ............PRE: <this=0x056c5ae8>IDirect3DDevice9::SetVertexDeclaration(0x056BA7C8)
Frame 000172 ............POST: <S_OK><this=0x056c5ae8> IDirect3DDevice9::SetVertexDeclaration(0x056BA7C8)
Frame 000172 ............PRE: <this=0x056c5ae8>IDirect3DDevice9::SetIndices(0x057093C8)
Frame 000172 ............POST: <S_OK><this=0x056c5ae8> IDirect3DDevice9::SetIndices(0x057093C8)
Frame 000172 ............PRE: <this=0x056c5ae8>IDirect3DDevice9::SetStreamSource(0, 0x05709550, 0, 24)
Frame 000172 ............POST: <S_OK><this=0x056c5ae8> IDirect3DDevice9::SetStreamSource(0, 0x05709550, 0, 24)
Frame 000172 ............PRE: <this=0x056c5ae8>IDirect3DDevice9::SetTexture(0, 0x0570FAF0)
Frame 000172 ............POST: <S_OK><this=0x056c5ae8> IDirect3DDevice9::SetTexture(0, 0x0570FAF0)
Frame 000172 ............PRE: <this=0x05709550>IDirect3DVertexBuffer9::Lock(0, 0, 0x0013FD8C, D3DLOCK_NOOVERWRITE)
Frame 000172 ............POST: <S_OK><this=0x05709550> IDirect3DVertexBuffer9::Lock(0, 0, 0x0013FD8C, D3DLOCK_NOOVERWRITE)
Frame 000172 ............PRE: <this=0x0570faf0>IDirect3DTexture9::Release()
Frame 000172 ............POST: <26><this=0x0570faf0> IDirect3DTexture9::Release()
Frame 000172 ............PRE: <this=0x0570faf0>IDirect3DTexture9::Release()
Frame 000172 ............POST: <25><this=0x0570faf0> IDirect3DTexture9::Release()
Frame 000172 ............PRE: <this=0x0570faf0>IDirect3DTexture9::Release()
Frame 000172 ............POST: <24><this=0x0570faf0> IDirect3DTexture9::Release()
Frame 000172 ............PRE: <this=0x0570faf0>IDirect3DTexture9::Release()
Frame 000172 ............POST: <23><this=0x0570faf0> IDirect3DTexture9::Release()
Frame 000172 ............PRE: <this=0x0570faf0>IDirect3DTexture9::Release()
Frame 000172 ............POST: <22><this=0x0570faf0> IDirect3DTexture9::Release()
Frame 000172 ............PRE: <this=0x0570faf0>IDirect3DTexture9::Release()
Frame 000172 ............POST: <21><this=0x0570faf0> IDirect3DTexture9::Release()
Frame 000172 ............PRE: <this=0x0570faf0>IDirect3DTexture9::Release()
Frame 000172 ............POST: <20><this=0x0570faf0> IDirect3DTexture9::Release()
Frame 000172 ............PRE: <this=0x0570faf0>IDirect3DTexture9::Release()
Frame 000172 ............POST: <19><this=0x0570faf0> IDirect3DTexture9::Release()
Frame 000172 ............PRE: <this=0x0570faf0>IDirect3DTexture9::Release()
Frame 000172 ............POST: <18><this=0x0570faf0> IDirect3DTexture9::Release()
Frame 000172 ............PRE: <this=0x0570faf0>IDirect3DTexture9::Release()
Frame 000172 ............POST: <17><this=0x0570faf0> IDirect3DTexture9::Release()
Frame 000172 ............PRE: <this=0x0570faf0>IDirect3DTexture9::Release()
Frame 000172 ............POST: <16><this=0x0570faf0> IDirect3DTexture9::Release()
Frame 000172 ............PRE: <this=0x0570faf0>IDirect3DTexture9::Release()
Frame 000172 ............POST: <15><this=0x0570faf0> IDirect3DTexture9::Release()
Frame 000172 ............PRE: <this=0x0570faf0>IDirect3DTexture9::Release()
Frame 000172 ............POST: <14><this=0x0570faf0> IDirect3DTexture9::Release()
Frame 000172 ............PRE: <this=0x0570faf0>IDirect3DTexture9::Release()
Frame 000172 ............POST: <13><this=0x0570faf0> IDirect3DTexture9::Release()
Frame 000172 ............PRE: <this=0x0570faf0>IDirect3DTexture9::Release()
Frame 000172 ............POST: <12><this=0x0570faf0> IDirect3DTexture9::Release()
Frame 000172 ............PRE: <this=0x0570faf0>IDirect3DTexture9::Release()
Frame 000172 ............POST: <11><this=0x0570faf0> IDirect3DTexture9::Release()
Frame 000172 ............PRE: <this=0x0570faf0>IDirect3DTexture9::Release()
Frame 000172 ............POST: <10><this=0x0570faf0> IDirect3DTexture9::Release()
Frame 000172 ............PRE: <this=0x0570faf0>IDirect3DTexture9::Release()
Frame 000172 ............POST: <9><this=0x0570faf0> IDirect3DTexture9::Release()
Frame 000172 ............PRE: <this=0x0570faf0>IDirect3DTexture9::Release()
Frame 000172 ............POST: <8><this=0x0570faf0> IDirect3DTexture9::Release()
Frame 000172 ............PRE: <this=0x0570faf0>IDirect3DTexture9::Release()
Frame 000172 ............POST: <7><this=0x0570faf0> IDirect3DTexture9::Release()
Frame 000172 ............PRE: <this=0x0570faf0>IDirect3DTexture9::Release()
Frame 000172 ............POST: <6><this=0x0570faf0> IDirect3DTexture9::Release()
Frame 000172 ............PRE: <this=0x0570faf0>IDirect3DTexture9::Release()
Frame 000172 ............POST: <5><this=0x0570faf0> IDirect3DTexture9::Release()
Frame 000172 ............PRE: <this=0x0570faf0>IDirect3DTexture9::Release()
Frame 000172 ............POST: <4><this=0x0570faf0> IDirect3DTexture9::Release()
Frame 000172 ............PRE: <this=0x0570faf0>IDirect3DTexture9::Release()
Frame 000172 ............POST: <3><this=0x0570faf0> IDirect3DTexture9::Release()
Frame 000172 ............PRE: <this=0x05709550>IDirect3DVertexBuffer9::Unlock()
Frame 000172 ............POST: <S_OK><this=0x05709550> IDirect3DVertexBuffer9::Unlock()
Frame 000172 ............PRE: <this=0x056c5ae8>IDirect3DDevice9::DrawIndexedPrimitive(D3DPT_TRIANGLELIST, 0, 2084, 96, 3126, 48)
Frame 000172 ............POST: <S_OK><this=0x056c5ae8> IDirect3DDevice9::DrawIndexedPrimitive(D3DPT_TRIANGLELIST, 0, 2084, 96, 3126, 48)
Frame 000172 ............PRE: <this=0x056c5ae8>IDirect3DDevice9::SetTexture(0, 0x05710E60)
Frame 000172 ............POST: <S_OK><this=0x056c5ae8> IDirect3DDevice9::SetTexture(0, 0x05710E60)
Frame 000172 ............PRE: <this=0x05709550>IDirect3DVertexBuffer9::Lock(0, 0, 0x0013FD8C, D3DLOCK_NOOVERWRITE)
Frame 000172 ............POST: <S_OK><this=0x05709550> IDirect3DVertexBuffer9::Lock(0, 0, 0x0013FD8C, D3DLOCK_NOOVERWRITE)
Frame 000172 ............PRE: <this=0x05710e60>IDirect3DTexture9::Release()
Frame 000172 ............POST: <1><this=0x05710e60> IDirect3DTexture9::Release()
Frame 000172 ............PRE: <this=0x05709550>IDirect3DVertexBuffer9::Unlock()
Frame 000172 ............POST: <S_OK><this=0x05709550> IDirect3DVertexBuffer9::Unlock()
Frame 000172 ............PRE: <this=0x056c5ae8>IDirect3DDevice9::DrawIndexedPrimitive(D3DPT_TRIANGLELIST, 0, 2180, 4, 3270, 2)
Frame 000172 ............POST: <S_OK><this=0x056c5ae8> IDirect3DDevice9::DrawIndexedPrimitive(D3DPT_TRIANGLELIST, 0, 2180, 4, 3270, 2)
Frame 000172 ............PRE: <this=0x056c5ae8>IDirect3DDevice9::SetTexture(0, 0x0570FAF0)
Frame 000172 ............POST: <S_OK><this=0x056c5ae8> IDirect3DDevice9::SetTexture(0, 0x0570FAF0)
Frame 000172 ............PRE: <this=0x05709550>IDirect3DVertexBuffer9::Lock(0, 0, 0x0013FD8C, D3DLOCK_NOOVERWRITE)
Frame 000172 ............POST: <S_OK><this=0x05709550> IDirect3DVertexBuffer9::Lock(0, 0, 0x0013FD8C, D3DLOCK_NOOVERWRITE)
Frame 000172 ............PRE: <this=0x0570faf0>IDirect3DTexture9::Release()
Frame 000172 ............POST: <2><this=0x0570faf0> IDirect3DTexture9::Release()
Frame 000172 ............PRE: <this=0x0570faf0>IDirect3DTexture9::Release()
Frame 000172 ............POST: <1><this=0x0570faf0> IDirect3DTexture9::Release()
Frame 000172 ............PRE: <this=0x05709550>IDirect3DVertexBuffer9::Unlock()
Frame 000172 ............POST: <S_OK><this=0x05709550> IDirect3DVertexBuffer9::Unlock()
Frame 000172 ............PRE: <this=0x056c5ae8>IDirect3DDevice9::DrawIndexedPrimitive(D3DPT_TRIANGLELIST, 0, 2184, 8, 3276, 4)
Frame 000172 ............POST: <S_OK><this=0x056c5ae8> IDirect3DDevice9::DrawIndexedPrimitive(D3DPT_TRIANGLELIST, 0, 2184, 8, 3276, 4)
Frame 000172 ............PRE: DebugSetMute(TRUE)
Frame 000172 ............POST: <> DebugSetMute(TRUE)
Frame 000172 ............PRE: <this=0x0571b9b8>IDirect3DStateBlock9::Apply()
Frame 000172 ............POST: <S_OK><this=0x0571b9b8> IDirect3DStateBlock9::Apply()
Frame 000172 ............PRE: <this=0x05714d00>IDirect3DStateBlock9::Apply()
Frame 000172 ............POST: <S_OK><this=0x05714d00> IDirect3DStateBlock9::Apply()
Frame 000172 ............PRE: <this=0x0571ec30>IDirect3DStateBlock9::Apply()
Frame 000172 ............POST: <S_OK><this=0x0571ec30> IDirect3DStateBlock9::Apply()
Frame 000172 ............PRE: DebugSetMute(FALSE)
Frame 000172 ............POST: <> DebugSetMute(FALSE)
Frame 000172 ........POST: <S_OK><this=0x056bac58> ID3DXSprite::End()
Frame 000172 ........PRE: <this=0x056c5ae8>IDirect3DDevice9::SetTransform(D3DTS_PROJECTION, 0x0013FE70)
Frame 000172 ........POST: <S_OK><this=0x056c5ae8> IDirect3DDevice9::SetTransform(D3DTS_PROJECTION, 0x0013FE70)
Frame 000172 ........PRE: <this=0x056c5ae8>IDirect3DDevice9::EndScene()
Frame 000172 ........POST: <S_OK><this=0x056c5ae8> IDirect3DDevice9::EndScene()
Frame 000172 ........PRE: <this=0x056c5ae8>IDirect3DDevice9::Present(NULL, NULL, NULL, NULL)


[quote name='Evil Steve']I bet you have an uninitialised variable somewhere that's getting filled with random garbage in release, and 0xcccccccc or some other recognisable pattern in debug. [/quote]

I though so too, but it is unlikely. Here is for example my shader for drawing the menue (on which the bug occours to):

/*

% Description of my shader.
% Second line of description for my shader.

keywords: material classic

date: YYMMDD

*/

struct VS_INPUT
{
float4 Position : POSITION;
float2 Texture : TEXCOORD0;
};

struct VS_OUTPUT
{
float4 Position : POSITION;
float2 Texture : TEXCOORD0;
};

float4x4 WorldViewProj : WorldViewProjection;
float4x4 World : World;
float Alpha;

texture2D DiffuseMap;

sampler Sampler = sampler_state
{
Texture = <DiffuseMap>;

AddressU = Wrap;
AddressV = Wrap;

MagFilter = Linear;
MinFilter = Linear;
MipFilter = Linear;
};

VS_OUTPUT mainVS( in VS_INPUT In ){
VS_OUTPUT Out = (VS_OUTPUT)0;

Out.Position = mul(In.Position, WorldViewProj);
Out.Texture = In.Texture.xy;

return Out;
}

// Pixel shader input structure
struct PS_INPUT
{
float4 Position : POSITION;
float2 Texture : TEXCOORD0;
};


// Pixel shader output structure
struct PS_OUTPUT
{
float4 Color : COLOR0;
};


PS_OUTPUT mainPS(in PS_INPUT In) : COLOR {
PS_OUTPUT Out;

Out.Color = tex2D(Sampler, In.Texture);
Out.Color.a = Alpha;
Out.Color.rgb *= Out.Color.a;

return Out;
}

technique build {
pass p0 {
VertexShader = compile vs_3_0 mainVS();
PixelShader = compile ps_3_0 mainPS();

CullMode = none;
Srcblend = One;
Destblend = InvSrcAlpha;
AlphaBlendEnable = true;
AlphaTestEnable = true;
ZEnable = false;
ZwriteEnable = false;
ColorWriteEnable = red | green | blue;
}
}


There isn't really anything that could not be initialized, couldn't it? float Alpha is 100% initialized as in debug mode, I use it to fade out a picture on the title screen. This does happen too on shaders without float alpha as input. Is there anything else that could cause such and alpha blending bug upon not being initialized? Note: the pattern stays the same all the time. Even on computer restart or so. It's always the same color for the same pictures..

No ideas? I think I've got one: Could that issue come from that I'm using a deferred rendering system? I've read here and there that deferred rendering doesn't work too well with alpha blending, however it did do a good job even in release mode until a few weeks ago..

Ok, after a long time I finally found out what was the problem: I noticed that alpha blending was working basically, but just for gui-elements, it wasn't. So I checked my debug-log, and noticed one thing: In debug mode, for all gui elements the effect file "gui.fx" was loaded. However in release mode, "effect.fx", my model rendering effect, was loaded. What the..? I double checked my code, but there is no evidence in the code where instead of "gui.fx" "effect.fx" could be loaded. Here is the important source code:

void CHUD::SetSprite(LPCWSTR FileName, D3DXVECTOR3 Position, bool AlwaysVisible, D3DCOLOR ColorKey)
{
wstring Dir = L"HUD/";
Dir += FileName;
int ID = m_Elements.size();
m_Elements.resize(ID+1);
m_Elements[ID].SetModel(*m_ModelManager->GetMesh(L"HUD.x"));
m_Elements[ID].SetAlwaysVisible(AlwaysVisible);
//load gui effect
m_ModelManager->GetMesh(L"HUD.x")->SetEffect(L"gui.fx"));
m_Elements[ID].SetDevice(m_lpDevice);
m_Elements[ID].SetTexture(m_Manager->GetTexture(Dir.c_str(), ColorKey));
m_Elements[ID].SetPosition((Position.x - m_ScreenSize.x/2)/m_Scale, -(Position.y - m_ScreenSize.y/2)/m_Scale, Position.z);
}

Where the effect gets loaded:

LPD3DXEFFECT CResources::GetEffect(LPCWSTR lpEffectName)
{
map<LPCWSTR, LPD3DXEFFECT>::iterator result;

result = m_Effects.find(lpEffectName);

if(result == m_Effects.end())
{
AddEffect(lpEffectName);
return GetEffect(lpEffectName);
}

return result->second;
}

void CResources::AddEffect(LPCWSTR lpFileName)
{
wstring Dir = L"../../Dead Mountains/Data/Effects/";
Dir += lpFileName;
LPD3DXBUFFER Errors;
HRESULT hr = D3DXCreateEffectFromFile(m_lpDevice, Dir.c_str(),NULL, NULL, 0, m_EffectPool, &m_Effects[lpFileName], &Errors);
if(FAILED(hr))
{
DebugOutput("Failed to load effect ", false);
}
else
{
DebugOutput("Successfully loaded effect ", false);
}
DebugOutputW(lpFileName);
}

Any ideas of what could be wrong? In release mode, when calling m_Manager->Geteffect("gui.fx"), the output says "Successfully loaded effect Effect.fx". Note that in release mode effects get loaded numerous times anyway. Could there be something wrong with the std::map.find()?

Note that a map<LPCWSTR, LPD3DXEFFECT> will compare pointers as the key and not text strings. You probably want a map<wstring, LPD3DXEFFECT>

Note that a map<LPCWSTR, LPD3DXEFFECT> will compare pointers as the key and not text strings. You probably want a map<wstring, LPD3DXEFFECT>

Thanks, that was actually one part of the issue! I alread stored textures in my resource manager using a map<wstring, LPDIRECT3DTEXTURE9>, so I'm wondering why I still used LPCWSTR for the effects. Anyway, as I said, thats not the only issue I had:

- First was the storing & accessing of the effect file. Obviously in debug mode find() worked with LPCWSTR, in release mode it didn't. So what happend in release mode is that every time I called GetEffect() a new effect was created and returned. In some obscure cases like with my gui.fx-file, find() returned the wrong effect. Using wstring fixed this. However, the problem wasn't solved. Instead of alpha not working for my gui, I got a mirrored copie of my scene at the position of every hud-element...

- Second issue, as I found out, was the assigning of my effects to the gui-elements. I did it this way:

m_Elements[ID].SetModel(*m_ModelManager->GetMesh(L"HUD.x"));
m_Manager->GetMesh("HUD.x")->SetEffect("gui.fx");

Where

CMesh* CHUDElement::SetModel(CMesh& Mesh);
void CManager::GetMesh(LPCWSTR lpFileName);

So what I found out is that the effect file simply was set for the model in the manager - but not for the one in the gui-element! Changing it to this worked:

m_Elements[ID].SetModel(*m_ModelManager->GetMesh(L"HUD.x"));
m_Elements[ID].GetModel()->SetEffect("gui.fx")

So while this fixed the problem I was still not satisfied. What I figured out is that SetModel() instead of storing a pointer created a copy of the object. I changed it to that:

void SetModel(CMesh* Model);

within all my code. Now only the pointer of a mesh is shared, and not a copy for every object (this also impacts other objects than the gui), which speeded up my engine by about 20% (27->33%) on my notebook.

My question is just: Why does that happen? I use code like this all the time to share classes, and there it works. Why is passing a mesh by reference and then storing in a pointer not working here?