HLSL: Create own back face cull function

This topic is 3471 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

Recommended Posts

Hi, been using the site and forum for a while but first post so, hello :) I have a question reguarding back face culling when using a HLSL fx file for vertex/pixel shaders. I know I can use the CULLMODE = CW or CCW function in the technique (which works), but I'm wanting to create my own function. I understand about transforming the positions of the veticies, getting their normal and dotting this against the eye vector to see if its front/back facing, what I don't understand is what to do when you have found this out. I have the following structs being used as the inputs to the vertex (a2v) and pixel (v2p):
// Structs
struct a2v
{
float4 pos			: POSITION;
float3 normal			: NORMAL;
float2 texCoord			: TEXCOORD0;
float3 tangent			: TEXCOORD1;
};

struct v2p
{
float4 pos			: POSITION;
float3 normal			: TEXCOORD0;
float2 texCoord			: TEXCOORD1;
float3 light			: TEXCOORD2;
float3 eye			: TEXCOORD3;
};

v2p Vs( in a2v IN )
{...}

float4 Ps ( in v2p IN ) : COLOR
{...}

In the vertex shader I planed to do the normal dot eye calculation, if it's front faceing use the functions I have to transform and pass to the pixel for rendering as normal. If it was back facing I was thinking about setting the v2p to NULL then in the pixel shader noticing the input was null, returning NULL also, though thinking about this it doesn't really achive much (doesn't compile anyway) after a bit more thinking and a fair bit of searching I'm still stuck. I will also be including view frustrum culling which I'm guessing will use a similar method of removing the face/part of the face which is not visiable and not passing it to the pixel shader. Any help is greatly apprichiated Thanks - Tranq

Share on other sites
As far i understand, you cant remove vertex or faces within vertex/pixel shaders. newers API version allow a new module called "Geometry shaders" which is used for add primitives within gpu; however i am not sure if you can remove primitives.

[Edited by - tpascal on December 18, 2008 10:49:35 AM]

Share on other sites
It IS possible to backface cull in the vertex shader but highly inefficient. Each vertex structure must contain the other 2 vertices the triangle will reference and if backfaced put the vertex behind the camera. The explosion in memory is incrediable though (each vert is now per triangle unique, i.e. indexing does not save you anything) and going to kill your performance, let the hardware do it. (Or use a geometry shader)

Share on other sites
Check the vFace register in the pixel shader. It's positive for a front faces and negative for a backface, if I remember correctly. Check the HLSL documentation for the details. Compare to the register's value and clip() accordingly. The register is only available from shader model 3.0 upwards.

Share on other sites
Thanks for the replies
I'm using shader v2.0 so the vFace idea isn't an option :( **edit (using DX 9) edit**
I have quickly tried moving the position of the vertex to the eye position, though this causes problems when only 1 of the verticies of the face is back facing (on the view edge of a cylinder/sphere for example) drawing a poly from the other vert to the eye position.
what does the CULLMODE = CCW function in the technique actually do? Looking in PIX the postVS is the same with it on and off, just that the back faces are not drawn, I guess I'm trying to recode that in essance

I shall look into the geometry shader idea and see if that will fit into the application easily. If I don't move to the geometry shader idea, sticking with the custom vertex I have now, should I think about finding the backfaces pre the shader? ie before the DrawPrimitive() function? Though guessing this would also be slow and cause massive problems if it breaks a face up.. :(

Thanks again

- Tranq

[Edited by - Tranq on December 18, 2008 11:29:45 AM]

Share on other sites
Geometry Shaders are new for SM4.0, so I don't think you're going to be using those either.

Share on other sites
Quote:
 Original post by TranqI'm using shader v2.0 so the vFace idea isn't an option :( **edit (using DX 9) edit**

Shader 3.0 is available in DX9. Unless you want to support ancient cards there is no reason not use it.

Share on other sites
Quote:
Original post by Matt Aufderheide
Quote:
 Original post by TranqI'm using shader v2.0 so the vFace idea isn't an option :( **edit (using DX 9) edit**

Shader 3.0 is available in DX9. Unless you want to support ancient cards there is no reason not use it.

the project requires DX9 and SM 2.0 at the moment sadly. Still haven't figured out how to do it.. is there a way to do it in SM 3.0? Might be able to up the shader model to it if it does, but the DX version can't be changed for this program.

• 18
• 11
• 20
• 9
• 52
• Forum Statistics

• Total Topics
631397
• Total Posts
2999812
×