Jump to content
  • Advertisement
Sign in to follow this  
yk_cadcg

[hlsl dx10]can't convert 23-bit int correctly

This topic is 4015 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

[dx10, hlsl, vista, gpgpu] Hi, for a int x (x in [0, MAX)), I first pass (float)x/MAX into the layout { "POSITION", 0, DXGI_FORMAT_R32G32_FLOAT, 0, 0, D3D10_INPUT_PER_VERTEX_DATA, 0 }, and in vs void vs(in float4 vert: POSITION, out float4 pos: SV_Position) { pos = vert * MAX / _screenSize; } I let x be (0,0),(1,1),(2,2),(3,3) and expect 4 points in a line on screen. for MAX=4, i'm right. for MAX=1<<23, which i think should be correctly converted between int and float, but they all overlap to 1 point in (0,0). why's that? i'm dying. thanks!

Share this post


Link to post
Share on other sites
Advertisement
In addition to Adam_42's comment, you realise you have a mismatch between your IA<->VS linkage? Widening is not an error, but your VS should really be float2 rather than float4...

Also, a little more whitespace in your post might be beneficial - I had to read your question a couple of times before I picked up all the details [smile].


Jack

Share this post


Link to post
Share on other sites
Thanks Adam_42, 1<<20... all lead to the same result.

Thanks Jack, if I use
in float2 vert: POSITION, 
..
pos = float4(vert, 0, 0);

instead of float4 in VS, then i can't get 4 points in a line, but a overlapped point.

btw, i can't find reference saying that we should only use "float" rather than "int" as vs's positions, like
in int4 vert: POSITION, 
, could anyone point to reference? thanks.


Quote:
Original post by jollyjeffers
In addition to Adam_42's comment, you realise you have a mismatch between your IA<->VS linkage? Widening is not an error, but your VS should really be float2 rather than float4...

Also, a little more whitespace in your post might be beneficial - I had to read your question a couple of times before I picked up all the details [smile].


Jack


[Edited by - yk_cadcg on September 26, 2007 7:21:02 AM]

Share this post


Link to post
Share on other sites
Shader Semantics (DirectX HLSL) -> The 'System-Value Semantics' section lists SV_POSITION (==POSITION) as float4.

I'm not sure about this:

pos = float4(vert, 0, 0);

But try:

pos = float4(vert.x, vert.y, 0, 0);

instead. I'm not sure there is a (float2,float,float) overload to the float4 constructor - I thought it was only (float,float,float,float) or (float3,float). Could well be your odd results are due to some sort of silent casting/promotion error...

hth
Jack

Share this post


Link to post
Share on other sites
thanks a lot!
i tried your suggestion and the same overlappe result.

i let color = pos in ps, staged rendertarget, and find that if we use your mentioned "widening" (IA is float2 but vs input is float4), then the final output pos is made up with "0, 1": (x, y, 0, 1).

this means, system default widening is "0,1". In light of this, i tried
pos = float4(vert.x, vert.y, 0, 1);
or
pos = float4(vert, 0, 1);
both get what i want (4 points in a line).

so i recalled there's a /w operation after vs. Digged out sdk entry "Rasterizer Stage (Direct3D 10)":
Quote:
the rasterizer always performs clipping, a perspective divide to transform the points into homogenous space, and maps the vertices to the viewport.

Vertices (x,y,z,w), coming into the rasterizer stage are assumed to be in homogenous clip-space. In this coordinate space the X axis points right, Y points up and Z points away from camera.

thanks a lot!



col = (int4)(pos.w, 255,255,255);
Quote:
Original post by jollyjeffers
Shader Semantics (DirectX HLSL) -> The 'System-Value Semantics' section lists SV_POSITION (==POSITION) as float4.

I'm not sure about this:

pos = float4(vert, 0, 0);

But try:

pos = float4(vert.x, vert.y, 0, 0);

instead. I'm not sure there is a (float2,float,float) overload to the float4 constructor - I thought it was only (float,float,float,float) or (float3,float). Could well be your odd results are due to some sort of silent casting/promotion error...

hth
Jack


Share this post


Link to post
Share on other sites
thanks a lot!
i tried your suggestion and the same overlappe result.

i let color = pos in ps, staged rendertarget, and find that if we use your mentioned "widening" (IA is float2 but vs input is float4), then the final output pos is made up with "0, 1": (x, y, 0, 1).

this means, system default widening is "0,1". In light of this, i tried
pos = float4(vert.x, vert.y, 0, 1);
or
pos = float4(vert, 0, 1);
both get what i want (4 points in a line).

so i recalled there's a /w operation after vs. Digged out sdk, found the pipeline architecture diagram, found that vs->(gs)->raster->ps, then found the entry "Rasterizer Stage (Direct3D 10)":
Quote:
the rasterizer always performs clipping, a perspective divide to transform the points into homogenous space, and maps the vertices to the viewport.

Vertices (x,y,z,w), coming into the rasterizer stage are assumed to be in homogenous clip-space. In this coordinate space the X axis points right, Y points up and Z points away from camera.

thanks a lot!

Quote:
Original post by jollyjeffers
Shader Semantics (DirectX HLSL) -> The 'System-Value Semantics' section lists SV_POSITION (==POSITION) as float4.

I'm not sure about this:

pos = float4(vert, 0, 0);

But try:

pos = float4(vert.x, vert.y, 0, 0);

instead. I'm not sure there is a (float2,float,float) overload to the float4 constructor - I thought it was only (float,float,float,float) or (float3,float). Could well be your odd results are due to some sort of silent casting/promotion error...

hth
Jack
[/quote]

Share this post


Link to post
Share on other sites
Sign in to follow this  

  • Advertisement
×

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

We are the game development community.

Whether you are an indie, hobbyist, AAA developer, or just trying to learn, GameDev.net is the place for you to learn, share, and connect with the games industry. Learn more About Us or sign up!

Sign me up!