Hello

I have a project with box2d and directx where i would like to have the ability to create a polygon with vertices that are based on points that the mouse inputs and map them to the vert buffer. When clicked the mouse will give an x and y value from 0 to HEIGHT(800) and 0 to WIDTH(600).I managed to convert those points to object-space(-1.0 to 1.0):

float X = float((-1.0f)*((2.0f/float(SCREEN_WIDTH))*float(mouse.GetMouseX())-1.0f));
float Y = float((-1.0f)*((2.0f/float(SCREEN_HEIGHT))*float(mouse.GetMouseY())-1.0f)); 

but the problem arises when i add the World, View and Projection matrix i end up with inaccurate vertices( not where the mouse clicked). Basically i'm clicking in object-space when i should be clicking in projection-space.

So my question is is there a way to scale (convert) these vertices given by the mouse so that when they are in

projection-space they match the position of the mouse.

Any articles or tutorials on this or transformations would be great.

Thanks

Yeah i forgot to state that the depth is 1.0f for all vertices .

Also how exactly should i go about multiplying the project(object) space. My position matrix looks like this

WVP = World * camView * camProjection; 

Should i use XMMatrixInverse(camView * camProjection) and just multiply it with the whole position matrix?

So this is what i came up with. I used code from http://www.braynzarsoft.net/index.php?p=D3D11PICKING

I modified it so that instead of a ray i'm working with a point(i commented out pickRayInWorldSpacePos).

i use mouse input to determine (mouseX, mouseY and 1.0f for Z) the position(a vector) in Projection-space and then apply the inverse of the Projection*View matrix.

Then i just get the x,y,z from the vector and then map a vertex with these x,z,y coord to the vertex buffer.

It is drawing the vertices but not in the location i clicked.

I think the problem is that i'm not applying the inverse of the World matrix but i don't know how to get it because it's different for every vertex.

Or maybe the ray method is the way to go?

DirectX::XMFLOAT3 D3DGraphics::pick(float mouseX, float mouseY)
{
	//DirectX::XMVECTOR pickRayInWorldSpacePos; 
	DirectX::XMVECTOR pickRayInWorldSpaceDir;

	DirectX::XMVECTOR pickRayInViewSpaceDir = DirectX::XMVectorSet(0.0f, 0.0f, 0.0f, 0.0f);
	//DirectX::XMVECTOR pickRayInViewSpacePos = DirectX::XMVectorSet(0.0f, 0.0f, 0.0f, 0.0f);

	DirectX::XMFLOAT3 pickRayViewSpace;
pickRayViewSpace.x = ((2.0f * float(mouseX))/float(SCREEN_WIDTH) - 1.0f) / ReturnProjection(0,0);
pickRayViewSpace.y = -((2.0f * float(mouseY))/float(SCREEN_HEIGHT) - 1.0f) / ReturnProjection(1,1);
pickRayViewSpace.z = 1.0f;

	pickRayInViewSpaceDir = DirectX::XMVectorSet(pickRayViewSpace.x,pickRayViewSpace.y,pickRayViewSpace.z, 0.0f);

	// Transform 3D Ray from View space to 3D ray in World space
	DirectX::XMMATRIX pickRayToWorldSpaceMatrix;
	DirectX::XMVECTOR matInvDeter;	//We don't use this, but the xna matrix inverse function requires the first parameter to not be null

	pickRayToWorldSpaceMatrix = DirectX::XMMatrixInverse(&matInvDeter,camView*camProjection);

	//pickRayInWorldSpacePos = DirectX::XMVector3TransformCoord(pickRayInViewSpacePos, pickRayToWorldSpaceMatrix);
	pickRayInWorldSpaceDir = DirectX::XMVector3Transform(pickRayInViewSpaceDir, pickRayToWorldSpaceMatrix);

	DirectX::XMFLOAT3 Vertex;

	Vertex.x = DirectX::XMVectorGetX(pickRayInWorldSpaceDir);
	Vertex.y = DirectX::XMVectorGetY(pickRayInWorldSpaceDir);
	Vertex.z = DirectX::XMVectorGetZ(pickRayInWorldSpaceDir);

	return Vertex;
}

XNA? You're using DirectXMath, are you not? The docs say "This parameter can be a nullptr."

Not constructive to the posted problem, but XNAMath/DirectXMath (both use an XM prefix, so it could be either iteration) have some incorrect documentation as to when a parameter can or can't accept nullptr. I wish I had the code at my fingertips so I could give you some specific examples, but I've definitely run into issues where documentation says it will accept a nullptr on an "optional" parameter and in reality that crashes, or where it says that it requires valid memory and nullptr works just fine.

Not to say that you shouldn't pass nullptr in the case the documentation is correct, but it's possible this is being done with good reason.