One strange thing I noticed that those samples do:

float3 nor = tex2D(NormalSampler, texCoord).xyz;
nor = nor * 2.0 - 1.0;
nor.y = -nor.y;

What could be the possible reason for inverting the up component of the normal?

The normal is still in tangent space so Y is not the "Up" direction, Z is.

You wouldn't do this normally no but sometimes due to the way the normalmap is baked out, you might need to flip the X or Y component to get correct lighting.

The normal is still in tangent space so Y is not the "Up" direction, Z is.

You wouldn't do this normally no but sometimes due to the way the normalmap is baked out, you might need to flip the X or Y component to get correct lighting.

Thanks, it makes sense. After further investigation I did notice that the "Green" part of the normal map is flipped. I guess I need to add an option to the content creator to fix such maps so that I don't have to maintain multiple versions of the same shaders.

Anyway, after months of study and work this is my result:

http://dl.dropboxusercontent.com/u/45638513/rs02.png

How does that look? This is the best visual result I ever got and the shaders support (non optimally) any number of light on a forward path. I used the higher resolution textures found in that blog post, but the albedo texture is disabled. It is just a normal map and an optional AO map, which is turned on in that screenshot. This is how an untextured object will look in the engine.

I read in one of the physically based rendering papers that an object should look good with only lighting, and I think truer words have never been spoken (in the domain of rendering).

The curving artifacts are still there, but depend on texture. This high resolution one has 0 curving or artifacts. High contrast normal maps under 512x512 will curve the shit out.

There are two more things I need to master and I'm done.

1. Toksvig AA, precomputed or not. I discovered Tokvig a while ago, but I'm always having a problem with it. In all the reference applications, Tokvig does stabilize the specular shimmering and gives some AA, but it also changes the perceived roughness of the surface a bit and spreads out the marginal highlights, basically altering the shape. So the choices are normal shimmering specular of different shaped non-shimmering ones. But whenever I "implement" Toksvig, and I put implement in quotes because it is just basically and extra multiplication with a simple expression based on normal length, I do not get that shape change. I'm comparing it right now with the results in FX Composer and things are slightly different. You do need a magnification glass to see the difference often, but it is there.

2. AA. That screenshot has some nasty shader surface aliasing. And from some angles it can be a lot worse. I'm guessing that the high contrast normal texture is partially to blame for increasing the surface aliasing:

http://dl.dropboxusercontent.com/u/45638513/rs04.png

That surface aliasing is pretty bad. The best SMAA is not good enough to fix this. Maybe with temporal reprojection it would help a bit.

What can be done about it?

One thing that I'll try it supersampling. I know of a technique used only for specular highlight supersampling. I never tried it on the entire lighting process, so there is no telling what results it will give. It is a constant backbufffer supersampling solution, meaning that 4x does not increase the resolution 4 times nor does it drop the performance by 75%. But there is still a steep performance drop.

On the other hand, more and more modern games do come with a SSAA solution out of the box. My GTX 660 TI is not strong enough for that, but I guess top end cards are so strong today that you can run some games with SSAA?

Anyway, I'll post my SSAA result here in a few days.