Have problem with stencil buffer.

Started by
3 comments, last by Gao Lu 18 years ago
first time using stencil buffer, faced a problem which trapped me for two days. I want to render an box which floating in water surface and seperated the processing into two passes. First passes i want to render the under water part of the box and river bed into a render target(not the device default one) and at the same time write those correspond pixels into stencil buffer with value of 0x1. Then in the second pass, set render target back to device default one and render the water surface using a big screen-aligned quad with the texture just filled in the previous pass and the stencil buffer. But the render result was messed with a lot of pixels which i don't know how they were coming out. It seems that the stencil buffer have affect the result but not correct. Here are the Effect codes of render states of the two passes mention above:

pass Render2Stencil
{
     VertexShader = compile vs_1_1 COMMON_VS();
     PixelShader = compile ps_1_3 COMMON_PS();
        
     ZEnable = true;        
     ZWriteEnable = true;

     StencilEnable = true;
     StencilRef = 0x1;
     StencilMask = 0xffffffff;
     StencilWriteMask = 0xffffffff;
     StencilPass = REPLACE;
     StencilFunc = ALWAYS;
     StencilZFail = KEEP;
     StencilFail = KEEP; 
        
     AlphaBlendEnable = false;
                   
}
	
pass FinalWater
{
    VertexShader = compile vs_1_1 FINAL_VS();
    PixelShader = <FINAL_PS_ASM>;
		
    Texture[0] = <TexBump>;
    MinFilter[0] = Linear;
    MagFilter[0] = Linear;
    MipFilter[0] = Linear;
    AddressV[0] = border;
    AddressU[0] = border;	
    BORDERCOLOR[0] = 0x0;  
	    
    Texture[1] = <TexUnderWater>;
    MinFilter[1] = Linear;
    MagFilter[1] = Linear;
    MipFilter[1] = Linear;
    AddressV[1] = border;
    AddressU[1] = border;	
    BORDERCOLOR[1] = 0x0; 
	    
    ZEnable = false;
    ZWriteEnable = false; 
    AlphaBlendEnable = false;
    DestBlend = ONE;
    SrcBlend = ZERO; 
        
    StencilEnable = true; 
    StencilRef = 0x1;
    StencilMask = 0xffffffff;
    StencilWriteMask = 0xffffffff;
    StencilPass = KEEP; 
    StencilFail = KEEP; 
    StencilFunc = EQUAL;
    StencilZFail = KEEP;
      	
}

Anyone have experience of using stencil buffer should be helpful and thanks in advance. ps. forgive my poor english [Edited by - Gao Lu on March 30, 2006 3:26:43 AM]
Advertisement
And here are the screens. Left one is the problem one, seemed the stencil actually did some thing because of the red lines and those pixels scattered around, but what i want is the right one.

http://bbs.gameres.com/upload/sf_2006330172953.jpg

by the way, i'm new to this forum. anyone bothered to tell me how to post image, thanks a lot.
basic question: did you clear the stencil buffer before rendering to it?
Yes i did.

And i have tried disabling changing render target during rendering and the result seemed all right.Why?
I seperated the processing of water surface pass into two passes. One for color, and another for stencil. That will work but also waste a lot of render time. Could just say that every water intersected object will used at least two render passes. Is there any better way to make them sqeezed into one pass?

This topic is closed to new replies.

Advertisement