Hey,

I have a very strange problem. I've been struggling with SSAO self-occlusion (http://www.gamedev.net/topic/595679-ssao-problem-with-self-occlusion/) and finally decided to see what is actually being done under the hood of the shader. So I used PIX to debug some selected pixel of the screen which in my opinion should be white (unoccluded) and which ends up being pretty dark. Look at the code:

float occlusion = 0.0f;
float3 pixelPosition = tex2D(positionsSampler, input.texCoord).xyz;
float3 pixelNormal = tex2D(normalsAndDepthsSampler, input.texCoord).xyz;
float pixelDepth = -tex2D(normalsAndDepthsSampler, input.texCoord).w;

for (int i = 0; i < 2; i++)
{
float3 randomVector = tex2D(randomVectorsSampler, input.texCoord * (7.0f + (float)i)).xyz;
randomVector = randomVector * 2.0f - 1.0f;

for (int j = 0; j < 8; j++)
{
float3 offsetVector = reflect(kernelVectors1[j], randomVector);

if (dot(pixelNormal, offsetVector) < 0.0f)
offsetVector *= -1.0f;

float3 samplePosition = pixelPosition + offsetVector * radius * pixelDepth / 10.0f; // HERE
samplePosition.z *= -1.0f;
samplePosition = projectToScreen(samplePosition);

float sampleDepth = -tex2D(normalsAndDepthsSampler, samplePosition.xy).w;

occlusion += occlusionFunction0(samplePosition.z - sampleDepth);
}
}

During stepping through each instruction, the debugger does 3 steps in the line commented with "HERE". And during these steps I see in the Variables table in the debugger, that the value of offsetVector changes (!). How can this be? offsetVector is not changed anywhere.
I though that maybe offsetVector is "updated" with "radius * pixelDepth / 10.0f" term. But this doesn't match. Before the change offsetVector is:

offsetVector ( 0.079, -0.016, -0.012 ) float3

After the change it becomes:

offsetVector ( 0.008, -0.002, -0.001 ) float3

where the term "radius * pixelDepth / 10.0f" is equal to 0.09633 in this case. It is not only that the scale of .x component doesn't match (0.079*0.09633!=0.008). Other components are scaled by a different value than the .x component! So if the .x component changes from 0.079 to 0.008 via some "magic constant" k, the relationships -0.016*k equal to -0.002 and -0.012*k equal to -0.001 do not hold. So every component of offsetVector is scaled by a different value.

Who is guilty? PIX or me cause I do not something very obvious? I'd appreciate your help on that issue.

Yeah, I did disable optimizations. And I looked closer at what is happening during asm instruction exectuion. Indeed, the debugger much more pays attention to the asm code rather than HSLS one. Huh, shame a bit that you cannot rely on what the debugger displays about high level variables. That makes the debugging much harder. Anyway, thanks for your help.