bool GetFacetWinding(IN IFacet * facet)
{
	TReal area = 0;
	TReal p0x, p0y, p1x, p1y;
	IMesh* m = facet->ownermesh;

	IVector3D vNormal = CalculateFacetNormal(facet);

	int iAxis0, iAxis1;

        //project the triangle
	IMath::GetMinorAxes(vNormal, iAxis0, iAxis1);

	for(int i = 0; i < facet->indexes.size() - 1; i++)
		{
		p0x = m->vertices[facet->indexes[i    ]][iAxis0] - m->vertices[0][iAxis0];
		p0y = m->vertices[facet->indexes[i    ]][iAxis1] - m->vertices[0][iAxis1];

		p1x = m->vertices[facet->indexes[i + 1]][iAxis0] - m->vertices[0][iAxis0];
		p1y = m->vertices[facet->indexes[i + 1]][iAxis1] - m->vertices[0][iAxis1];

		area += (p0x * -p1y) + (p1x * p0y);
		}

	return(area < 0);
}

I'm getting opposite winding for opposite sides of a cube. This can't be right as the cube renders just fine. Either I'm missing something super obvious here or I need to get some rest.

Sorry for being vague.

After building a box from faces that should be winding the same way (or else they'd get culled), I would expect all faces to have the same winding (either clockwise or counter-clockwise). This function should return either true or false for all of them. It returns true for faces facing in one direction and false for faces facing in the opposite direction. Here's an example:

Normal Winding

(0, 0, -1) 1

(0, 0, +1) 0

(-1, 0, 0) 1

(+1, 0, 0) 0

(0, -1, 0) 0

(0, +1, 0) 1

Looking at my code now, I think I see why the results are wrong - the faces facing in opposite directions aren't being affected by the normal while physically the order indeed is opposite. I'm not sure how to fix it, though.

The way you determine the normal already depends on the winding (order of vertices), since the two triangle-edge vectors you use to get the normal already have a specific order in your "facet", so the normal has nothing to do with it.

Also, I think (please correct me if I'm wrong) the formula you're using determine the winding only works for 2d triangles but your code also allows higher-order polygons, so your m->vertices should already be transformed into projection space or reference plane, and it should contain only triangles.

In fact, in your case you don't even need to use the area formula, as the sign of the z from the normal will give you the winding (assuming that the original vectors used in the normal cross-product calculation had x and y set to the pre-transformed 2d coordinates and z=0) - i.e, the normal will be pointing either towards positive z or negative z, and the x and y components of the normal will be 0.

If you do use transformed vertices, keep in mind that using a perspective projection matrix will give you the same winding for left& right faces of of a cube when the cube's front/back faces are perfectly parallel to the projection plane. The same goes for the top and bottom faces. Only the front and back faces will have different winding. In an orthogonal projection, all faces that are opposite to each other will have opposite windings, unless the faces are perpendicular to the projection plane, in which case you can't determine the winding (because the z value of the normal will be 0).

bool GetFacetWinding(IN IFacet * facet)

bool GetFacetWinding(IN IFacet * facet)

This does not make sense. There is no 'right' winding order for any triangle or polygon.

You're right - the name of the function should read as IsFacetWindingClockwise().

This is exactly what I'm trying to accomplish. Point in case - the faces of a cube should be wound consistently. My test fails to show that. Note that I'm aiming to determine the winding of a 3D polygon.

In short - if I pass in a set of polygons taken from a closed mesh or polytope - even if it's concave -, I want them all to consistently return the same winding.

The lines:

int iAxis0, iAxis1;

//project the triangle

IMath::GetMinorAxes(vNormal, iAxis0, iAxis1);

do exactly that. They use the normal to identify the minor axes (eg the two non-major axes) and the rest of the code uses the triangle that is projected onto this plane as baseline.

Hm - unless I'm misunderstanding you, you're suggesting the following code should basically give me the winding?

bClockwise = vNormal[iMajorAxis] > 0;

Where iMajorAxis is the predominant axis of the vector, eg z for (0, 0, -1). I'll be honest, my brain's a little garbled at the moment but at least one case I can see where it fails is for faces that are diagonal with respect to all major planes, eg faces with (unnormalized) normals like (1, 1, 1), (-1, 1, -1) etc. That is, cases where no major axis exists (come to think of it my original code doesn't account for these either).

You're right - the name of the function should read as IsFacetWindingClockwise().

In short - if I pass in a set of polygons taken from a closed mesh or polytope - even if it's concave -, I want them all to consistently return the same winding.