Mostly correct.
You will first calculate the normal and plug it in the a,b,c parameters in the equation.
Then replace x,y,z by one of the points on the plane(one of the points of the triangle) and you have got d.
The reset is correct.
Two things to note:
1. if (D * n) = 0 it means that the ray and the plane are parallel hence no intersection.
2. The algorithm described in the linked paper performs ray/triangle intersection yet what I've described only performs ray.plane intersection.
Which means that after you calculated the intersection point of the ray and the plane you still have to check if it lies within the triangle.

Suspecting my last attempt was a fail, here is a new attempt that I think is correct:


void OpenGLControl::hitTest()
{

float t;
float a, b;
vector3 b, pointOnPlane, tempPoint;
float distance;


for(int i = 0; i < hitTestPoints.size(); i++)
{
for(int j = 0; j < model->triangles.size(); j++)
{
//N.D normal . (0,1,0)
a = model->triangles[j].normal * rayOrientation;
//if plane is paralel to ray
if(a == 0)
{
continue;
}
//N.v3 normal . one point on plane (one of the vertices)
//ax + by + cz + d = 0 wich can be rewritten as N.v3 + d = 0 => d = -(N.v3)
distance = -(model->triangles[j].normal*model->triangles[j].v3);
//origin . normal + distance ( N.O + d )
b = hitTestPoints * model->triangles[j].normal + distance;
t = (-b)/a;
//intersection is behind the ray
if(t < 0)
{
continue;
}
//calculate the point
tempPoint = rayOrientation*t;
pointOnPlane = hitTestPoints + tempPoint;
}
}
}


1. calculate Vd: normal . orientation (orientation is (0,1,0))
2. check if Vd == 0, then go to next iteration - ray is paralel
3. calculate distance: d = abs((vertex3 - rayOrigin) . normal); vertex3 is one of the 3 vertices on the plane
4. calculate V0: -(normal . rayOrigin + distance)
5. calculate t: t = V0/Vd
6. check if t < 0, then behind the ray, go to next iteration
7. calculate the point on plane: pointOnPlane = t . orientation + rayOrigin

calculate distance: d = abs((vertex3 - rayOrigin) . normal); vertex3 is one of the 3 vertices on the plane[/quote]
So, d should be either vertex3 * normal or -(vertex3 * normal) depending how you wrote the plane equation.

calculate V0: -(normal . rayOrigin + distance)[/quote]
Again V0 should be either d - rayOrigin * normal or rayOrigin * normal - d.

Make sure you understand the equations and follow the calculation.

Taking the plane equation as ax + by + cz - d = 0
we get that d = ax + by + cz
so by substituting a,b,c by the plane normal and x y z by some point on it we get d.
Then we know that:
t = (d - O*n) / (D * n)
where t is the distance along the ray direction, O is the ray origin, D is the ray direction, n is the plane normal and d is the value we calculated above.

calculate distance: d = abs((vertex3 - rayOrigin) . normal); vertex3 is one of the 3 vertices on the plane

So, d should be either vertex3 * normal or -(vertex3 * normal) depending how you wrote the plane equation.

calculate V0: -(normal . rayOrigin + distance)[/quote]
Again V0 should be either d - rayOrigin * normal or rayOrigin * normal - d.

Make sure you understand the equations and follow the calculation.
[/quote]

Im little confused: Isn't d distance from an origin point to the plane? Btw, I am very grateful that you are helping me despite my stupidity.