Calculate Normals from a displacement map

You should calculate the normal map once if the heightmap is static. To do it in the shader you would need to sample the height map in an area to calculate the normal, not sure what the most efficient way of calculating normals from a height map is.

What you are looking for is a Sobel filter, for 3 x 3 kernel sizes. Other alternative is the 5 x 5 kernel mask based on Canny filter coefficients. Former is faster, but can be sensitive to noise. The latter gives you a smoother normal map, because the 5 x 5 kernel mask incorporates a Gaussian filter. Start with the simpler first.

but i don't understand what he means with putting the "result into a render target" [/quote]

Think "render target" as a Frame Buffer Object which is can be a texture target. Therefore, you need to create a FBO in OpenGL, attach a texture ID to it, then render the normal map result in there.

or how such a pixel shader could look like by only having access to one pixel[/quote]

What you need to do is represent the displacement/height map as a separate grey-scale texture. You bind that texture to a sampler in the fragment shader. In the fragment shader you need to find the 3x3 kernel offsets for current displacement map texture coordinates. Something like:

//Max 3x3 kernel offsets
vec2 d = vec2(dFdx(gl_TexCoord[0].s), dFdy(gl_TexCoord[0].t));

vec2 TexCoord[3];
TexCoord[0] = gl_TexCoord[0].st - d;
TexCoord[1] = gl_TexCoord[0].st;
TexCoord[2] = gl_TexCoord[0].st + d;


You sample the kernels using those texture offsets. Note that TexCoord[0] represents the bottom left kernel cell, TexCoord[1] is central, and TexCoord[2] is top right. Naturally, all the other kernel coords will be the combination of TexCoord components.

or how to run this pre-calculation before rendering the actual object...[/quote]

1. Create a RGB FBO texture, where you will store the normal map.
2. Bind the displacement map as a grey-scale texture.
3. Bind the FBO texture.
4. User the fragment shader to compute the normal map from the displacement map, and render the result into the FBO texture.
5. Use the FBO texture to manipulate geometry via a vertex texture unit. How you do this will be interesting, because OpenGL ES 2 is quite limited and I'm not sure whether it supports vertex texture units.


The real question is, do you need to generate the normal map one-off, or do you have to compute it frequently? If it's one-off, you might as well do it on the cpu.

Ok i'm trying to understand what sobel is doing
But doesn't sobel need access to the neighbour pixels? I still don't understand how to access the texture coords of the neighbour vertices within the fragment shader... because the offset don't have to be the same everytime?

Maybe you can explain the sobel-image a bit more... I see the more steeply the slope between some pixels of the heightmap (or a polygon after displacement) is, the brighter a pixel in the sobel-texture is,
but I still don't understand how to calculate a vec3 normal from the sobel-image-texture?

Can you point out the correlation between your code and the Gx/Gy, Sx/Sy and A from Wikipedia ?
http://en.wikipedia..../Sobel_operator

Right now my Heightmap is an Greyscale texture and im just taking the brightness of a texel to displace the vertex along his normal
But after that the normals isn't valid anymore and my light calculations are returning incorrect results.

CPU rendering is damn slow in JavaScript, especially image pixel access

Here is a complete frag shader from my implementation. Its targeted for OpenGL 2.0, GLSL 1.20, therefore i don't know whether it will work in ES. This thing will convert any input texture into a normal map.

//-- Fragment shader for the normal map filter --

#version 120

//The heightmap
uniform sampler2D Texture;

//Scharr operator constants combined with luminance weights
const vec3 Sobel1 = vec3(0.2990, 0.5870, 0.1140) * vec3(3.0);
const vec3 Sobel2 = vec3(0.2990, 0.5870, 0.1140) * vec3(10.0);

//Luminance weights, scaled by the average of 3x3 normalised kernel weights (including zeros)
const vec3 Lum = vec3(0.2990, 0.5870, 0.1140) * vec3(0.355556);

//Blur level (mip map LOD bias)
const float Blur = 0.5;

void main(void)
{
vec2 Coord[3], d;
vec4 Texel[6];
vec3 Normal;

//3x3 kernel offsets
d = vec2(dFdx(gl_TexCoord[0].s), dFdy(gl_TexCoord[0].t));
Coord[0] = gl_TexCoord[0].st - d;
Coord[1] = gl_TexCoord[0].st;
Coord[2] = gl_TexCoord[0].st + d;

//Sobel operator, U direction
Texel[0] = texture2D(Texture, vec2(Coord[2].s, Coord[0].t), Blur) - texture2D(Texture, vec2(Coord[0].s, Coord[0].t), Blur);
Texel[1] = texture2D(Texture, vec2(Coord[2].s, Coord[1].t), Blur) - texture2D(Texture, vec2(Coord[0].s, Coord[1].t), Blur);
Texel[2] = texture2D(Texture, vec2(Coord[2].s, Coord[2].t), Blur) - texture2D(Texture, vec2(Coord[0].s, Coord[2].t), Blur);

//Sobel operator, V direction
Texel[3] = texture2D(Texture, vec2(Coord[0].s, Coord[0].t), Blur) - texture2D(Texture, vec2(Coord[0].s, Coord[2].t), Blur);
Texel[4] = texture2D(Texture, vec2(Coord[1].s, Coord[0].t), Blur) - texture2D(Texture, vec2(Coord[1].s, Coord[2].t), Blur);
Texel[5] = texture2D(Texture, vec2(Coord[2].s, Coord[0].t), Blur) - texture2D(Texture, vec2(Coord[2].s, Coord[2].t), Blur);

//Compute luminance from each texel, apply kernel weights, and sum them all
Normal.s = dot(Texel[0].rgb, Sobel1);
Normal.s += dot(Texel[1].rgb, Sobel2);
Normal.s += dot(Texel[2].rgb, Sobel1);

Normal.t = dot(Texel[3].rgb, Sobel1);
Normal.t += dot(Texel[4].rgb, Sobel2);
Normal.t += dot(Texel[5].rgb, Sobel1);

Normal.p = dot(texture2D(Texture, Coord[1], Blur).rgb, Lum);

gl_FragColor = vec4(vec3(0.5) + normalize(Normal) * 0.5, 1.0);
}


Couple of notes, the texture origin is assumed to be top-left (as opposed to bottom left according to OpenGL convention). This is due to the requirements of my implementation. Therefore you will need to invert Normal.t. The final value passed to the frag colour is scaled and shifted from [-1, 1] into the range of [0, 1].

Right now my Heightmap is an Greyscale texture and im just taking the brightness of a texel to displace the vertex along his normal
But after that the normals isn't valid anymore and my light calculations are returning incorrect results.[/quote]
Ok, when you are doing the geometry pass, use the normal map as a vertex attribute array, similar in the way you use the height-map to displace the vertices in your vert shader. Therefore, you'll use the normal map entries to substitute the vertex normals. In order to make the lighting look correct, you have to do two things with the normals. 1.) Convert from range [0, 1] back to [-1, 1]; 2.) Multiply the result with the normal transformation matrix.

That said, if you do a one off normal map calculation, it might be worthwhile to build a new VBO, with the normal map converted to proper vertex normals. This will eliminate the need of passing the normal map as a vertex attribute array.

Thanks for your effort!

Sorry i've never done rendering with multiple targets...
How do i start the normal-map-creation-rendering? Usually i just call drawElements or drawArrays with a vertice buffer... but now i don't have an object... i just want to write some memory ?

How can i pass it as an attribute array? I thought the normal-map is inside a framebuffer in the graphic memory ?
Still i don't have a clue how to calculate a normal from a one-dimensional 0 .. 255 grey value of a pixel in the sobel-generated normal map to a 3-dimensional normal vector

How do i start the normal-map-creation-rendering? Usually i just call drawElements or drawArrays with a vertice buffer... but now i don't have an object... i just want to write some memory ?[/quote]
What you can do is you bind the height map texture to a quad polygon object. You render that quad into the FBO, using the normal map fragment shader. Naturally you will have to create a separate viewport matching the normal map texture resolution (aka the FBO), and create an orthographic projection so that the quad will fill the entire viewport when rendering.

How can i pass it as an attribute array? I thought the normal-map is inside a framebuffer in the graphic memory ?[/quote]
This is where things get tricky. Perhaps you can read the FBO normal map out into an external buffer and pass that as an attribute array. Another (much better) alternative is maybe bind the FBO normal map in the vertex shader and access texture directly per vertex (like you would in a frag shader) - you can do the same with the height map texture as well. Although I'm not sure whether ES would support that.

Still i don't have a clue how to calculate a normal from a one-dimensional 0 .. 255 grey value of a pixel in the sobel-generated normal map to a 3-dimensional normal vector[/quote]
You will have to read the Sobel article I linked earlier, and the wiki page, which explains mathematically how it works. Basically it is a gradient filter. Say you have a height map. As you examine each pixel, you want to extract the horizontal gradient (Gx) and the vertical gradient (Gy) at that location. This implies that you want to know the difference in height between the central pixel you are examining, and its immediate neighbours.

The resulting gradient [Gx, Gy] will correspond to the 2D direction of the normal vector at that pixel. To compute the gradient [Gx, Gy], you create two 3x3 kernel masks around the central pixel you are examining. To find Gx, you sample the neighbouring pixels around the central one, multiply with the weights specified by its mask and add them together. Repeat for Gy and use the other mask. In my frag shader, you can see them in action under the "Sobel operator, [...] direction" comment.

There is other stuff going on in the frag code; for example it allows you to use RGB textures as a height map. It simply converts them to greyscale in the Sobel operation, hence the constants, such as vec3(0.2990, 0.5870, 0.1140) is combined with the kernel weight Sobel1 and Sobel2. It works just the same if your texture is already greyscale. Note that unlike in the articles, i'm using kernel weights 3 and 10, as opposed to 1 and 2. This gives better contrast for edges, but you can revert that to 1 and 2.

Thanks for your detailed answer, but I think you got me wrong

i understand what sobel does, but i don't know what to do with it's result image?

I pass it into the vertex shader, together with the height map and the object to be manipulated.. and then?
How do i calculate the actual normal vector?

The entries in the normal map is the normal vector.

ah ... ok now i see ;D
gl_FragColor = vec4(vec3(0.5) + normalize(Normal) * 0.5, 1.0);
[font="Arial"]It seems that WebGL doesn't support dfdx() & dfdy()... even if i enable the necessary opengl feature.
But isn't d just the offset to the neighbour pixel and can be calculated with 1/texture_width ?
What about the pixels on the edges?
[/font]

But isn't d just the offset to the neighbour pixel and can be calculated with 1/texture_width ?[/quote]

Yep pretty much, dFdx() and dFdy() is just an automated method. You can define the offset as a uniform if you wish.

What about the pixels on the edges?[/quote]

OpenGL will handle that automatically, depending on your clamping parameters for the texture, see glTexParameterf().