Ack! should have actually looked at your code...
Well then i can give you the solution that i came up with when i was fed up with that algorithm. It should work for any polygon so long as the center is within the polygon.
In addition to recording the minimum and maximum shadows for the SAT collision, record the vertices that were associated with the shadow, in your code:
// Get the projection of the next point dotProduct = D3DXVec2Dot(&pVerts[count], &_vAxis); // Update the min/max if (dotProduct < min) min = dotProduct; vmin=count; else if (dotProduct> max) max = dotProduct; vmax=count;
if the new mtd turns out to be the shortest mtd, record the vertex used in the mtd, as well as the polygon the vertex was on:
if (intervalDistance < minIntervalDistance) { minIntervalDistance = intervalDistance; translationAxis = axis; mtd_polygon=edgeIndex<_pPoly1->GetNumVertices()?_pPoly1:_pPoly2; mtd_vertex=//the vertex of the minimum or maximum shadow, whichever was shorter; D3DXVECTOR2 d = _pPoly1->GetPosition() - _pPoly2->GetPosition(); if (D3DXVec2Dot(&d, &translationAxis) < 0) translationAxis = -translationAxis; }
The actual min/max vertex that you choose will have to be flipped depending on if you're checking the vertices of polygon A or B.
My code for determining the mtd looks like:
d0=min1-offset+max2;//signs are probably messed upd1=min2-max1+offset;//signs are probably messed upif (d0>0.0f || d1>0.0f){return}if (d0>mtd_dist)//since they are negative, greater is smaller{ mtd_polygon=...whichever one isn't creating the axis; mtd_vertex=min_vert;//might be max, depends on which polygon is being checked mtd_dist=d0;}if (d1>mtd_dist)//since they are negative, greater is smaller{ mtd_polygon=...whichever one isn't creating the axis; mtd_vertex=max_vert;//might be min, depends on which polygon is being checked mtd_dist=d1;}
when you believe you have the mtd vector, change the current vector flipping check to:
if ((mtd_polygon->vertex[mtd_vertex]-mtd_polygon->center).dot(mtd_normal)<0.0f){mtd_normal=-mtd_normal;}
This assumes that everything is in world coordinates. If you're using rotation matrices you're going to have to apply them to the normal or the vertex.
Most of this is off the top of my head, i'm just trying to explain the concept.