First off, I would recommend that you implement your normals using textures instead, this will allow you to vastly improve the graphical quality of your terrain, without increasing the vertex count. You can easily use normal maps with 2-3 times higher resolution (than the heightmap), and it's really really cheap. I was personally amazed at how low heightmap quality one could get away with as long as the normal maps where of high quality. I don't have my old comparison images available, but I assure that it is definately worthwhile if you have the time.

The only downside to using textures for normals is that pointy edges will cause the normals to bleed over the edge incorrectly... however, you should never have that steep heightmaps anyway if you're at all concerned about the visual quality.

Anyway, as for your actual problem, it seems like you forgot to consider the X/Y-scaling of the different heightmap LODs as you compute your normals... remember, one LOD further away, means each quad is twice as wide (but not twice as high!), which also means you have to consider this when computing your normals.

EDIT: Also, this depends entirely on your what your purpose is, but note that the transition area requires that you consider the zig-zag pattern of vertices when blending between the higher and lower-resolution heightmap, you cannot use the default interpolation if you want 100% seamless transitions (you may see very slight popping of certain vertices on steep terrain), mostly though this will not be noticeable.

EDIT: I'm not sure what's going on in your wireframe-screenshot... it look likes the LODs get coarser and coarser, and suddenly, some of them get finer... and then coarser again?... those look like the LODs that are incorrectly colored as well.

Hi Syranide! Thanks for the reply!
[quote name='Syranide' timestamp='1308258148' post='4824226']
Anyway, as for your actual problem, it seems like you forgot to consider the X/Y-scaling of the different heightmap LODs as you compute your normals... remember, one LOD further away, means each quad is twice as wide (but not twice as high!), which also means you have to consider this when computing your normals.

You've spotted the problem precisely! The XY-scaling doubles as the LODs get farther away. My ScaleFac variable doesn't take it into account. Unfortunately the error was only in the problem description I've posted here. My code snippet was wrong. Sorry! In fact I calculate the ScaleFac as
uniform float2 ScaleFac = float2(-0.5/2^L,-0.5/2^L); // L is the clipmap index increasing for the LODs farther away

For the posted image I'm drawing 4 LODs with the squares of the closest LOD being 1x1 units big (2x2 units for the next LOD, than 4x4 and finally 8x8).

First off, I would recommend that you implement your normals using textures instead, this will allow you to vastly improve the graphical quality of your terrain, without increasing the vertex count. You can easily use normal maps with 2-3 times higher resolution (than the heightmap), and it's really really cheap.

I'm not sure how to combine this LOD method with normal maps. As the LOD heightmaps are calculated at run time on the GPU with only incremental updates for being provided by the CPU as the camera moves. Normal-mapping the LODs with the double or tripple resolution would require some efficient update scheme. Right now I'm just trying to get the basic method work.

EDIT: I'm not sure what's going on in your wireframe-screenshot... it look likes the LODs get coarser and coarser, and suddenly, some of them get finer... and then coarser again?... those look like the LODs that are incorrectly colored as well.

The LODs don't get finer, only coarser. I'm drawing 4 LOD rings. For each successive ring the size of the squares is doubled. They may seem to get finer but that's just the screenshot.The incorrectly lit area is exactly at the border between 2 rings.

Anyway after you hint I've double checked my normal scaling factors. They seem to be correct but the problem is not solved. I'm still looking for help.
[/quote]

Using textures for normals is really simple, just treat them as you do with the heightmap... of course there are some things to take care of, such as there being additional LODs for normals and normal LODs "moving faster". But performance shouldn't really be a problem, I did a naive implemention myself for this, and while I could see the CPU spiking with really high detail normal maps being streamed to the GPU. That had a lot more with the unrealistically fast flying movement I had, which may not be obvious at first, my issue really was because I was literally streaming tens of megabytes per second because I was moving so fast (although it didn't seem like it).

Anyway, your choice, but I really would recommend them in the long run... if you want to keep it simple for now, you could just copy the heightmap code and use the same resolution for heightmaps and normalmaps (literally copy-paste, although you need to compute the normal maps). You could even bake the heightmap and normalmap into the same texture, although, it would only be a temporary solution.

Also, looking at the incorrectly colored image above... something else seems amiss... I seems to me that there are two distinct blue colors up there, not one. So the normals seems correct (the shading of it all), but it seems to transition between two different colors? Or rather... like the it transitions from some mixture of RGB, to just B. Messed up the RGB-components somewhere?

Additional info: normal maps calculated for all LODs. The X and Y components are encoded in the red and green color channels. The Z component is supposed to be always 1.

[attachment=3421:normals_level3.png] [attachment=3420:normals_level2.png] [attachment=3419:normals_level1.png] [attachment=3418:normals_level0.png]
It is obvious that all the textures except the first one have a strange discontinuity along the border. The reason for this is still not clear to me.

I would assume the discontinuity is because you are computing the normals, but you don't actually sample outside of the boundaries specified by the target texture (as is required)... although, the pattern on the edges doesn't really seem ordinary... perhaps the edge normals get computed from some irrelevant data outside the texture? I'm not sure if you've noticed it, but the edge at the top and the bottom are equal or near equal... it would seem like the edges wrap around when sampling the normals?That would also explain why the first image looks correct, because it tiles.

[quote name='Syranide' timestamp='1308824101' post='4826703']
I would assume the discontinuity is because you are computing the normals, but you don't actually sample outside of the boundaries specified by the target texture (as is required)... although, the pattern on the edges doesn't really seem ordinary... perhaps the edge normals get computed from some irrelevant data outside the texture? I'm not sure if you've noticed it, but the edge at the top and the bottom are equal or near equal... it would seem like the edges wrap around when sampling the normals?That would also explain why the first image looks correct, because it tiles.

Well, that it is quite likely that the sampling outside the boundaries gets messed up. I'm not sure that I understand the paper correctly at this point. It is necessary to sample the neighboring points in order to compute the normal to at some clipmap point. But where should the data come from for the points along the boundaries? I have no idea right now.

No I haven't noticed that the texture tiles and the edges at the top and bottom are nearly equal. Thanks for the tip!

I have rewritten my code a little bit. I can now use (almost) arbitrary clipmap resolution. I'm going to make the textures 8 x 8 instead of 128 x 128 and check manually the calculation of the edge texel values.
[/quote]

Well, you can't compute normals around the edges of a heightmap texture... however, you are computing normals for a subset of a larger heightmap so you should actually have the data needed for computing the normals around the edges... however, you cannot first crop the heightmap and then compute the normals from that, you need to compute the normals from the "uncropped" heightmap, which should also include some amount of additional heightmap data outside of the immediate "rectangle of interest".

That is, if you want to compute the normalmap for a small rectangle in the center of a large heightmap, you cannot crop the heightmap and then compute the normalmap from that, you need to sample directly from the larger heightmap (for each pixel in the target normalmap, sample from the large heightmap, individual samples should have no knowledge of the normalmap boundaries).