Hi guys

Im curious, do HLSL floats have different numerical ranges to C++ floats? As far as I recall, C++ floats can have a range of 1.2E-38 to 3.4E38.

I am getting a odd response from a geometry shader here. Here is the vertex XYZ values after being multiplied by the world matrix on the C++ side:

62909.550781f, 43.891846f, 129223.367188f

and here is the code in the geometry shader:

	float4 vert = mul(sprite[0].pos, g_world);
	
	float4 center = float4(vert.xz*0.000043192f, 0, 1);

	//top right
	if(abs(center.x) < 1.0f && abs(center.y) < 1.0f)
	{
		v.p = center+float4(0.007702941f, 0.002502837f, 0, 0);	
		triStream.Append(v);
		//bottom right
		v.p = center+float4(0.004760679f, -0.006552513f, 0, 0);
		triStream.Append(v);
		//top
		v.p = center+float4(0, 0.008099352f, 0, 0);	
		triStream.Append(v);
		//bottom left
		v.p = center+float4(-0.004760679f, -0.006552513f, 0, 0);
		triStream.Append(v);
		//top left
		v.p = center+float4(-0.007702941f, 0.002502837f, 0, 0);
		triStream.Append(v);
		triStream.RestartStrip();
	}

If we take the Z value of 129223.367188 (which is getting shifted to the Y value for the display) and multiply by 0.000043192f, I should get a value of 5.5814 etc which would put the vertex well outside of the viewport limit of +-1.0f. However, despite the limiting if statement in the above code which prevents a triangle from being sent out if it is greater than 1.0f, the vertex still gets drawn around 0.5f-0.6f on the display.

Any ideas as to why? Transposing the world matrix has no effect. This just makes no sense.

Yes my GS input is sprite[1].

I tried setting up my own mul function after verifying which values go where when comparing the world matrix on the C++ side with the HLSL one. But it still gives the same results. Tried moving the positioning to the vertex shader and leaving the geometry shader to only output the triangles. No luck.

FWIW here is my own MUL function.

float4 MUL(float3 v, float4x4 m)
{
	float x = (m[0][0]*v[0]) + (m[0][1]*v[1]) + (m[0][2]*v[2]);
	float y = (m[1][0]*v[0]) + (m[1][1]*v[1]) + (m[1][2]*v[2]);
	float z = (m[2][0]*v[0]) + (m[2][1]*v[1]) + (m[2][2]*v[2]);
	return float4(x, y, z, 1);
}

I cant use shader debuggers as the program is running both DX11 and DX9. They cant seem to work in such an environment. However I did tests with each element of the world matrix and the vectors to make sure I am not making any incorrect assumptions before I created the above function. All tests returned the expected values. So the only conclusion I can come to is something is happening to the values after I set the vertex buffer and give the draw call. I have no way to determine what.

Its a 2D display. I am ignoring the Y axis in the source verts and putting the Z axis into the Y to get a top down view.

This line:

float4 center = float4(vert.xz*0.000043192f, 0, 1);

scales the values down.

The vertex values are local space only. The world matrix repositions them. There is no view or projection matrix because as I said earlier it is a 2D display so I am only scaling down the values to fit inside the +-1.0f viewport space with the intention of 0 being the center of the display.

As I stated already I have done tests where I transpose the world matrix before sending it to the shader but this has no effect. Whether I use my own MUL function or the intrinsic mul the result is always the same. I only created my own MUL to verify the results that I would based on tests to determine the positions in the matrix of the various values.

Bottom line, no matter what I do, the picture I get on the display doesn't correspond with pen and paper based calculations based on the vertex*world output to file that I posted.

(62909.550781f, 129223.367188f)*0.000043192f should give an output of (2.7171893f, 5.5814156f). Yet on screen the vertex is showing up below and right of the display center when it should be completely offscreen. If you make the integer parts of the value 0 so it sits between +-1.0f the vertex should appear above and right of the center as far as I recall. For some strange reason, it is just totally defying logic.

Well will test things without the GS. However I would be curious to know if there is any setting on the C++ side that could have this effect. If I think about it, the output on the screen does correlate with (2.7171893f-2.0f, (5.5814156f-5.0f)-1.0f) which gives (0.7171893f, -0.4185844f). Almost as if a modulo is being performed on the vertex so that it always appears on screen instead of culling it. Its not quite a modulo (otherwise the Y axis would be appearing above and right of the center instead of below and right) but I am not quite sure how else to describe it. Almost as if the viewport gets shifted up and to the right until the vertex appears.

Well I finally got fedup with the whole thing, decided to grab hold of a little D3D10_1 app that I had from the old days of learning DX and edited it to use my shader. Then I grabbed the vertices and world matrices from the program I have been struggling with and saved them to a file. Loaded that file in the app and am making use of the VS2012 HLSL debugger. The only problem now is...I am using the ID3D10Effect system for my shaders. And I cant seem to find any details on setting up the HLSL debugger to recognize my HLSL source.

Any one got a crash course on how to set the thing up? MSDN is of absolutely no help here.