Hi,

I try to do omnidirectional shadows, but I have problems comparing the depth values stored in the cube map with the frament positions.

To fill the cube map with the depth values I render my scene in each face with the differend view matrices (in world space) and use the hardware depth buffer. I checked the resultig cube map and it seems to have the right values stored in it.

Then I render my scene and look up the depth values in the cube map and try to compare them to the depth value of the projected fragment position.

The depth from the cube map is in the [0,1] range as expected, but my calcualted fragment deph (`depth_from_pos(lookup)`) returns values in the range [-1,0] and if I render them, they don't look quite right.

uniform samplerCube texShadow;

float depth_from_pos(vec3 pos) {
    // near and far used by the projection matrix for the cube map faces
    float n = 1.0;
    float f = 100.0;

    //// calculate the major depth axis -> max(abs(pos))
    vec3 abs_pos = abs(pos);
    float z = max(abs_pos.x, max(abs_pos.y, abs_pos.z));
    //// project pos to get the z and depth value
    // p_light = [[n,0,0,0], [0,n,0,0], [0,0,-(f+n)/(f-n),-2*f*n*(f-n)], [0,0,-1,0]] // right = top = 1
    // clip = p_light * vec(0,0,-z,1.0)
    // depth = (clip.z / clip.w) * 0.5 + 0.5 // divide by w and transform ndc [-1,1] to [0,1]
    float depth = -f*n/(z*(f-n)) + f/(f-n);
    return depth;
}

...

// transform light direction from view space to world space
// as the cube map faces are in world coordinates
vec3 lookup = mat3(view_inv) * vec3(lightDir);
float shadow_depth = texture(texShadow, normalize(lookup)).r
// get depth of pixel position from the light source 
float frag_shadow_depth = depth_from_pos(lookup);
float visibility = 1.0;
if (shadow_depth < frag_shadow_depth) {
    visibility = 0.0;
}

...

This is the rendered `frag_shadow_depth+1` of a scene with a plane with some cubes seen from above. The light is a bit over the plane in the center of the image.

[attachment=30131:bad depth.png]

I would have suspected something which is dark in the center and gets brighter to the edges without this 'x'.

I suspect the error in the depth_from_pos function, but I don't know what I'm doing wrong.

Does anybody have some ideas?

Thanks in advance

It took me much longer than I'd like to admit, but I solved it.

I thought that lightDir was simply lightPos - fragmentPos, but in fact it was normalized. So calculating depth from a normlaized vector won't work.

Now it works as expected :)