I'm using an orthographic projection matrix for the light and it doesn't seem correct unless in the depth vertex shader I set gl_Position.w = 1; (I just noticed in my other project that I had to set a drawing orthographic projection matrix gl_Position.w = 1; to make it pixel-perfect with the dimensions I specified when constructing the matrix, and that the shadow actually moved if I moved the light, which shouldn't happen for a directional/orthographic projection light, so I noticed today that I had to set gl_Position.w = 1; for the light orthographic matrix too.)
attribute vec4 position;
uniform mat4 projection;
uniform mat4 model;
uniform mat4 view;
attribute vec2 texCoordIn0;
varying vec2 texCoordOut0;
void main(void)
{
gl_Position = projection * (view * (model * position));
gl_Position.w = 1; // sun light is orthographic directional light
texCoordOut0 = texCoordIn0;
}
However that causes the shadow to show up wrong. lpos is the vertex in the light's clip-space on a range [0,1] (scaled by 0.5 and translated 0.5 for texture mapping).
Heightmap vertex shader:
void main(void)
{
//vec4 vpos = (view * (model * position));
vec4 vpos = model * position;
//vec4 vpos = position;
//vpos.w = 1;
lpos = lightMatrix * vpos;
lpos.w = 1;
gl_Position = projection * (view * (model * position));
//gl_Position.w = 1;
Setting lpos.w = 1 doesn't help.
Heightmap frag shader:
void main (void)
{
//....
vec3 smcoord = lpos.xyz / lpos.w;
//vec3 smcoord = lpos.xyz;
float shadow = max(0.6,
float(smcoord.z <= texture(shadowmap, smcoord.xy).x));