When I do raycasting, I can get the intersected triangle and the hit point, I want to place a bullet hole on the hit point.

How do I calculate the billboard position from the hit point and intersected triangle?

I'm not sure how I should start, also I'm not really good in mathematics.

Here is what I got:

D3DXVECTOR3 triangleV1, triangleV2, triangleV3;
D3DXVECTOR3 intersectedPoint;

Let me know EXACTLY how do I get:

D3DXVECTOR3 bulletBillboardV1, bulletBillboardV2, bulletBillboardV3, bulletBillboardV4;

I am not good at math either, but try this:

D3DXVECTOR3 GetTriNormal(const D3DXVECTOR3& v0, const D3DXVECTOR3& v1, const D3DXVECTOR3& v2)
{
    D3DXVECTOR3 edge1 = v1 - v0;
    D3DXVECTOR3 edge2 = v2 - v0;
    D3DXVECTOR3 normal;
    D3DXVec3Cross(&normal, &edge1, &edge2);
    return normal;
}
...
D3DXVECTOR3 up = triangleV1 - intersectedPoint;
D3DXVECTOR3 nrm = GetTriNormal(triangleV1, triangleV2, triangleV13);
D3DXVECTOR3 right;
D3DXVec3Cross(&right, &nrm, &up);
D3DXVec3Normalize(&up, &up);
D3DXVec3Normalize(&nrm, &nrm);
D3DXVec3Normalize(&right, &right);
...
float size = 0.3f;
D3DXVECTOR3 horizontal = right * size;
D3DXVECTOR3 vertical = up * size;
bulletBillboardV1 = intersectedPoint - horizontal + vertical; // up-left
bulletBillboardV2 = intersectedPoint + horizontal + vertical; // up-right
bulletBillboardV3 = intersectedPoint + horizontal - vertical; // down-right
bulletBillboardV4 = intersectedPoint - horizontal - vertical; // down-left

no guarantee that it will work. Maybie someone could correct me if i am wrong.

EDIT: It works actually, just tried it with my old "triangle picking" project. Here on the image i just draw lines around "billboards":

If it doesn't show for you or triangles are missing, make sure you got your winding/indices right.

@belfegor: It works, however, I have a problem when I shoot another side of a box:

[attachment=15008:wr.png]

How do you obtain your triangle points?

For testing, I tried to draw 3 spheres on the triangle points, it works very well with the terrain, but it doesn't work correctly with the box, I get points in a completely different position.

Here is how I get the triangle:

D3DXIntersect(mesh, &rayFrom, &rayDir, &hit, &faceIndex, &pU, &pV, &distance, NULL, NULL);
if (hit)
{
    D3DXVECTOR3 V1 = pVertices[pIndices[3 * faceIndex + 0]].p;
    D3DXVECTOR3 V2 = pVertices[pIndices[3 * faceIndex+ 1]].p;
    D3DXVECTOR3 V3 = pVertices[pIndices[3 * faceIndex + 2]].p;

    // Code to draw bullet billboard here...
}

That is correct code for triangle points, but i don't know why do you get them at different position then expected.

Let me show you more code, maybe you could spot something wrong:

mesh->LockVertexBuffer(D3DLOCK_READONLY, ( void** )&pVertices);
mesh->LockIndexBuffer(D3DLOCK_READONLY, ( void** )&pIndices);

D3DXMATRIX matWorld = GetMeshWorldMatrix();
D3DXMATRIX matInverse;
D3DXMatrixInverse(&matInverse, NULL, &matWorld);

D3DXVECTOR3 rayFrom, rayDir;
D3DXVec3TransformCoord(&rayFrom, &bRayFrom, &matInverse);
D3DXVec3TransformNormal(&rayDir, &bRayTo, &matInverse);

D3DXIntersect(mesh, &rayFrom, &rayDir, &hit, &faceIndex, &pU, &pV, &distance, NULL, NULL);
if (hit)
{
    D3DXVECTOR3 V1 = pVertices[pIndices[3 * faceIndex + 0]].p;
    D3DXVECTOR3 V2 = pVertices[pIndices[3 * faceIndex+ 1]].p;
    D3DXVECTOR3 V3 = pVertices[pIndices[3 * faceIndex + 2]].p;

    // Code to draw bullet billboard here...
}

I notice that the triangles are only valid when the mesh position is XYZ: (0.0f, 0.0f, 0.0f) but when the mesh move to any other position, the results assume that the position is still 0.0f, 0.0f, 0.0f.