Jump to content

  • Log In with Google      Sign In   
  • Create Account

FREE SOFTWARE GIVEAWAY

We have 4 x Pro Licences (valued at $59 each) for 2d modular animation software Spriter to give away in this Thursday's GDNet Direct email newsletter.


Read more in this forum topic or make sure you're signed up (from the right-hand sidebar on the homepage) and read Thursday's newsletter to get in the running!


stencil not working when rendering intances


Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.

  • You cannot reply to this topic
7 replies to this topic

#1 lomateron   Members   -  Reputation: 363

Like
0Likes
Like

Posted 06 March 2013 - 10:37 PM

Has anyone had this problem?

 

I am rendering many point instances to random locations of a 2Dtexture(different points can land in the same texel), if i render them using drawInstance() the stencil buffer doesnt updates(a texel's stencil value is increased when the point lands on it)

if i draw them using a draw() inside a loop the stencil updates its value.


Edited by lomateron, 06 March 2013 - 10:46 PM.


Sponsor:

#2 lomateron   Members   -  Reputation: 363

Like
0Likes
Like

Posted 07 March 2013 - 05:54 AM

Is there any change in the stencil configuration between draw() and drawInstanced()?



#3 lomateron   Members   -  Reputation: 363

Like
0Likes
Like

Posted 07 March 2013 - 08:34 AM

I just found the root of my problem(not the solution) ...Weirdest of all i have ever had: 

for(int rrr=0;rrr!=1;rrr++)
{
	g_pd3dDevice->ClearDepthStencilView( g_pDepthStencilViewSpace3D, D3D10_CLEAR_STENCIL, 0.0f, 0 );
	g_pd3dDevice->OMSetDepthStencilState(g_pDepthStencilStateSpace3D,1);
	     if(rrr==0){g_pd3dDevice->OMSetBlendState(g_pBlSRed	        , 0, 0xffffffff);}
	else if(rrr==1){g_pd3dDevice->OMSetBlendState(g_pBlSGreen	, 0, 0xffffffff);}
	else if(rrr==2){g_pd3dDevice->OMSetBlendState(g_pBlSBlue	, 0, 0xffffffff);}
	else if(rrr==3){g_pd3dDevice->OMSetBlendState(g_pBlSAlpha	, 0, 0xffffffff);}

	g_pT->SetFloat((float)rrr);
	g_pTechnique3DSpace->GetPassByIndex( 0 )->Apply( 0 );
	g_pd3dDevice->DrawInstanced( 1, 1, 0, 0);
}

The principal problem is that its passing the stencil test in the 2nd, 3rd, 4th and on an on loop, and appart from that in those same loops there are other weird things happening (i don't know why) that may be related to why it passes the stencil test.

 

I am rendering to 1 texel of a texture using a point vertex

The default value of all texels in texture is float4(-1,-1,-1,-1)

The stencil test passes if EQUAL, OMSetDepthStencilState() sets to 1 and ClearDepthStencilView() sets the other to 0, it will have to fail always

the depth test is disabled, if depth test fails nothing happens

if the stencil test passes or fails the stencil value is incremented

The blending just mask a color(the color in the name)

The value rendered to the texel is float4(rrr,rrr,rrr,rrr)

 

If the loop does 1 loop: for(int rrr=0;rrr!=1;rrr++)

-nothing is write to the texel, it failed the stencil test.

 

if the loop does 2 loops: for(int rrr=0;rrr!=2;rrr++)

-In the first loop nothing is rendered, it failed the stencil test.

-In the second loop it passes the stencil test(I dont know why) so it renders to the texel float4(0,0,0,0)-->it should be float4(1,1,1,1) but its not (i don't know why)

and it is masked by the unupdated red mask(i dont know why)-->it should be the green mask, the final texel value is float4(0,-1,-1,-1)-->g_pT is 0 not 1(i don't know why)

 

if the loop does 3 loops: for(int rrr=0;rrr!=3;rrr++)

-In the first loop nothing is rendered, it failed the stencil test.

-In the second loop it passes the stencil test(I don't know why), it renders to the texel float4(0,0,0,0)-->it should be float4(1,1,1,1) but its not (i don't know why)

and  it is masked by the red mask(i don't know why)-->it should be the green mask, the final texel value is float4(0,-1,-1,-1)-->g_pT is 0 not 1(i don't know why)

-In the third loop it passes the stencil test(I don't know why) so it renders to the texel float4(1,1,1,1) -->it should be float4(2,2,2,2) but its not (i don't know why)

and  it is masked by the green mask(i don't know why)-->it should be the blue mask, the final texel value is float4(0,1,-1,-1)

-->g_pT is 1 not 2(i don't know why)

 

In the end if i replace drawInstance() by a draw() inside a loop it all runs correctly but i cant because in the real code i run aroun 5000 instances


Edited by lomateron, 07 March 2013 - 12:49 PM.


#4 Seabolt   Members   -  Reputation: 633

Like
0Likes
Like

Posted 07 March 2013 - 09:12 AM

Should you by clearing the DepthStencilView each iteration of the loop? Everytime you clear your entire stencil buffer, and you'll never have anything to test it with, (essentially you're only testing against 0). Not to mention it's terribly slow.


Perception is when one imagination clashes with another

#5 lomateron   Members   -  Reputation: 363

Like
0Likes
Like

Posted 07 March 2013 - 12:36 PM

I know that its going to happen, i made it that way, its a test code that shows how i discovered the root of my problem, and the problem is that it passes in the second loop the stencil test when it shoul never ever pass it, but its passing the test and i want to know why it is passing the test in the 2nd, 3rd, 4th, and on and on, that's the weird problem i am refering to.


Edited by lomateron, 07 March 2013 - 12:41 PM.


#6 Seabolt   Members   -  Reputation: 633

Like
0Likes
Like

Posted 07 March 2013 - 02:36 PM

I can't honestly say I completely understand what's happening here. It seems like if it's passing the stencil test on the 2nd, 3rd, and 4th iterations, then you're not writing what you expect into the stencil buffer. Break out PIX, (or equivalent), and make sure all the states are where you think that they should be, and that the stencil buffer is what you think it should be.


Perception is when one imagination clashes with another

#7 Cthuga   Members   -  Reputation: 132

Like
0Likes
Like

Posted 08 March 2013 - 05:18 AM

I'm trying to understand Your situation.

 

I suppose that Your DepthStencilBuffer is enabled, his test is LESS THAN (standard one), and it MASK_ALL - so You make standard depth stencil test. Am I right ?

 

Could You explain am I right:

 

The stencil test passes if EQUAL, OMSetDepthStencilState() sets to 1 and ClearDepthStencilView() sets the other to 0,

 

So You want to set Stencil buffer to 1 there where You draw Your Instance ?

 

But then You wrote:

 

if the stencil test passes or fails the stencil value is incremented

 

So I see here two different things: Do You set it to 1, or increment in every iteration ?

 

______________________________________________________________________________________________________________

 

Geek Message Blog

The steps I take to make a dream come true:

 

http://geekmessage.wordpress.com


Edited by Cthuga, 08 March 2013 - 05:19 AM.


#8 lomateron   Members   -  Reputation: 363

Like
0Likes
Like

Posted 08 March 2013 - 06:50 AM

I think I just found a directx bug, can someone try this to see if its really a bug, it's simple:

 

create a texture with 1 texel....create it's render target..create its depthstencil with this DXGI_FORMAT_D24_UNORM_S8_UINT

create depthstencil state with this description:

D3D10_DEPTH_STENCIL_DESC dSDd;
dSDd.DepthEnable = FALSE;
dSDd.DepthWriteMask = D3D10_DEPTH_WRITE_MASK_ZERO;
dSDd.DepthFunc = D3D10_COMPARISON_ALWAYS;

dSDd.StencilEnable = TRUE;
dSDd.StencilReadMask = D3D10_DEFAULT_STENCIL_READ_MASK;
dSDd.StencilWriteMask = D3D10_DEFAULT_STENCIL_WRITE_MASK;

dSDd.FrontFace.StencilDepthFailOp	= D3D10_STENCIL_OP_KEEP;
dSDd.FrontFace.StencilFailOp		= D3D10_STENCIL_OP_INCR;
dSDd.FrontFace.StencilPassOp		= D3D10_STENCIL_OP_INCR;
dSDd.FrontFace.StencilFunc		= D3D10_COMPARISON_EQUAL;

dSDd.BackFace.StencilDepthFailOp	= D3D10_STENCIL_OP_KEEP;
dSDd.BackFace.StencilFailOp		= D3D10_STENCIL_OP_KEEP;
dSDd.BackFace.StencilPassOp		= D3D10_STENCIL_OP_KEEP;
dSDd.BackFace.StencilFunc		= D3D10_COMPARISON_NEVER;

 

create 2 techniques with this shaders, first technique uses VS1 and second technique uses VS2

 

 

struct VS_A
{
    float4 Pos : POSITION;
    uint Id : SV_InstanceID;
};

struct PS_A
{
    float4 Pos : SV_POSITION;
    float4 Texl : TTTTT;
};

PS_A VS1( VS_A i  )//stencil will not work as it should
{
	PS_A output;

	output.Pos=float4(0.0f,0.0f,0.0f,1.0f);
	output.Texl=float4(1.0f,1.0f,1.0f,1.0f);

	return output;
}

PS_A VS2( float4 Pos : POSITION  )//stencil will work correctly
{
	PS_A output;

	output.Pos=float4(0.0f,0.0f,0.0f,1.0f);
	output.Texl=float4(1.0f,1.0f,1.0f,1.0f);

	return output;
}

float4 PS( PS_A a) : SV_Target
{
	return a.Texl;
}

 

 

now to render

set the render target with its depthstencil, set its viewport and layout and finally render it like this:

 

g_pd3dDevice->OMSetDepthStencilState(g_pDepthStencilState,1);

g_pd3dDevice->ClearDepthStencilView( g_pDepthStencilViewSpace3D, D3D10_CLEAR_STENCIL, 0.0f, 0 );
g_pTechnique1->GetPassByIndex( 0 )->Apply( 0 );
g_pd3dDevice->DrawInstanced( 1, 1, 0, 0);

g_pd3dDevice->ClearDepthStencilView( g_pDepthStencilViewSpace3D, D3D10_CLEAR_STENCIL, 0.0f, 0 );
g_pTechnique1->GetPassByIndex( 0 )->Apply( 0 );
g_pd3dDevice->DrawInstanced( 1, 1, 0, 0);

 

 

 

the first draw will fail the stencil test and will not draw nothing,

the second draw will pass the stencil test and will draw ----------->This is the problem, i don't know why it passes the test

 

Now change the technique so it uses the VS2 and render using: g_pd3dDevice->Draw( 1, 0);

The two draws will fail the stencil test, thats how it should work

 

The use of SV_Instance in VS1 i think is the cause of the problem, i dont know more


Edited by lomateron, 08 March 2013 - 07:00 AM.





Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.



PARTNERS