Jump to content
  • Advertisement
Sign in to follow this  

Iterate texels in shader [fixed]

This topic is 3663 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

hi there the following pixel shader is not for practical use : it just add up positions stored in a texture to verify the position sum is the same value produced by its cpp counterpart.

// D3DFMT_A32B32G32R32F, 128 x 128, stores required positions
texture texPosition;
sampler2D sampPosition : sampler_state
    Texture = <texPosition>;
    MinFilter = POINT;
    MagFilter = POINT;
    MipFilter = NONE;				
    AddressU = Clamp;
    AddressV = Clamp;    

// how many positions to be used for calculation (not using all the 128x128 texels, but the first n texels)
int n;

float4 pshader(vs_output i) : COLOR
    float3 sumPosition = float3(0, 0, 0);
    float2 uv;

    for (int i = 0; i < n; ++i)
        // calculate coordinates to fetch position at i, texture sizes are 128x128
        float fac = i / 128.0;

        float x = frac(fac);
        float y = fac - frac;
        // uv coordinates
        uv.x = x + 1.0 / 256.0;
        uv.y = y / 128.0 + 1.0 / 256.0;

        float4 temp = tex2D(sampPosition, uv);
        sumPosition += temp.xyz;
    return float4(sumPosition, 1);

there must be something wrong with the code cause it always produce wrong sum. after a few tests i found : when n is 256 (note the texture width is 128), the sum will become <0, 0, 0, 1> as if the sum is reset, (any value as long as n < 256 it will be correct, but n should be more than 1024 in real calculation). i am guessing the iterating method is wrong but not sure. Any suggestion and reply would be really appericiated. thanks a lot. PS. it is compiled in ps_3_0 PS. hi Hodgman, i have changed the code to use texel center [Edited by - Pet123 on June 10, 2008 10:36:50 PM]

Share this post

Link to post
Share on other sites
Would like to help but I am a bit confused as what you are trying to do.

What is your sampPosition Sampler state doing? As thats where you are getting your color values from (the ones that you are summing).

I also (even though I don't quite get what your doing) wonder why you don't just do a nested loop rather than the modulus mathematics.

It also seems that you will be looping over 128 values of uv.x (0-1.0 in 1/128th jumps) but only 0 and 1 for your uv.y value (as your setting 1/128 to an int)

Also aren't there 16384 pixels in your texture rather than 1024? I may just not be getting what your trying to do, but I hope I helped some.

Share this post

Link to post
Share on other sites
AFAIK, your algorithm for constructing UV coordinates is a little bit off. You want to find the center of each texel, not the edge.

The tex-coord for the left-most pixel is not 0, but 0+texelSize/2. Likewise, the right-most pixel is not 1, but 1-texelSize/2:
uv.x = x / 128.0 + 1/256.0;
uv.y = y / 128.0 + 1/256.0;

Share this post

Link to post
Share on other sites
thanks for all the replys.

1. the sampler state is now added (sorry it was missing cause at first i thought it was not the main point here :P)

2. here using a while-loop and not a nested for-loop is because only the first n texels of the 128x128 textures used, using a for-loop would produce a lot empty cycles. besides, it will crash FXComposer if change to a nested-for loop.
[edit] FXComposer crashed was because the code i wrote will force FXComposer to un-roll the loop.

the most confusing thing is when n is 256 (the texture width is 128), the sum will become <0, 0, 0, 1> as if it is reset (by who? how?)

[edit] wow, not only 256 the sum will be reset, but also 512, 768, 1024 (all these values are multiple of 256).

[Edited by - Pet123 on June 10, 2008 9:09:18 AM]

Share this post

Link to post
Share on other sites
another thing i found : everything will be ok if i run the shader in REF mode, what does this mean?

Share this post

Link to post
Share on other sites
ok, i've found the reason, here is the compiled code :

// Generated by Microsoft (R) HLSL Shader Compiler 9.22.949.2248
// Parameters:
// int nov;
// sampler2D sampPosition;
// Registers:
// Name Reg Size
// ------------ ----- ----
// nov i0 1
// sampPosition s0 1

def c0, 0, 0.0078125, 0.00390625, 1
dcl_2d s0
mov r0, c0.x
rep i0 // <---- this line
mul r1.x, r0.w, c0.y
frc r1.y, r1.x
add r1.x, r1.x, -r1.y
mad r1.y, r0.w, c0.y, -r1.x
add r1.y, r1.y, c0.z
mad r1.z, r1.x, c0.y, c0.z
texld r1, r1.yzzw, s0
add r0.xyz, r0, r1
add r0.w, r0.w, c0.w
mov oC0.xyz, r0
mov oC0.w, c0.w

// approximately 17 instruction slots used (1 texture, 16 arithmetic)

the rep/endrep pair is the reason, here is what the dx doc. says about the instrcution :

rep - ps
Start a rep...endrep - ps block.

rep i#
where i# is an integer register that specifies the repeat count in the .x component. See Constant Integer Register.

i#.x specifies the iteration count. The legal range is [0, 255]. Note that this instruction does not increment or decrement the value of i#.x.

"i#.x specifies the iteration count, the legal range is [0, 255]", so i'll have to break the original loop into small batches to meet this requirement.

Share this post

Link to post
Share on other sites
Sign in to follow this  

  • Advertisement

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

Participate in the game development conversation and more when you create an account on GameDev.net!

Sign me up!