The question is how to get an decreasing shadow effect with distance from sun?

Some background:

• I am using shadow maps, a separate render pass with the camera up in the sky in using orthogonal projection with the depth buffer saved in a bitmap for the final render phase.
  
• My world is infinite in all dimensions. There will be objects above ground, unlimited. In theory, that means that there should always be shadows from the sun. But in practice, I want nice views instead of totally realistic views.
  
• The idea is to let the effect of the shadow decrease with distance between the object doing the shadow and the surface that is shadowed. Beyond a certain limit, there would be no shadow at all.
  
• It is easy to get a value for the distance, from the object casting a shadow to the surface, in the fragment shader.
  

This is where the problems start. The shadow map records the distances to the objects closest to the sun. If there are two objects casting a shadow on the same surface, the distance computation algorithm would be based on the object closest to the sun. This is not the result I want. I want the near distance object to cast a dark shadow, and the far distance object to cast a weaker shadow, outside of the shadow of the near distance object.

Edit: Of course I want decreasing shadow effects with distance from the sun, not increasing.

You can render two shadow maps (min/max), one standard and one reverted (revert culling and z-tests). The latter one can be used for calculating the distance to the closest surface. You can't use only the second one.

Though you could try to write it in a single pass (render the second value to a standard render target), but this could be slower than a two pass solution, because of turning down all the hardware z-buffer optimsation stuff. On the other hand the fragment shader would be pretty fast, at least I would test out both solution.

Ashaman, thanks for the suggestion. But I am starting to believe it is simply not possible with a technique based on shadow maps.

In the creation of the shadow map, the camera is moved "near" the sun, looking in the same direction as the sun light is going. For every point in the shadow map, there can now be many objects that are hit by the same ray of light. In principle, I think one depth value is needed for every possible object in that direction. But one depth buffer can only hold one depth value for a point.

The "decreasing with distance" behaviour of real shadows is due to the penumbra. Standard shadow mapping techniques are only good for drawing the umbra, and soft-shadow extensions are usually hacks to generate a visually-plausible (not physically correct) penumbra. e.g. a common approach is to use a larger "softening" filter for depth-samples that are a long distance from the shadow-receiver.


The basic shadow mapping technique can be extended to allow for real penumbras, but a general solution will be extremely slow/expensive.

Ashaman's suggestion to use 2 shadow maps is a trade-off -- instead of the up-to-infinite depths required, it's limited to 2, which is still more information than 1 ;)

There's a general rendering technique known as "Depth Peeling", which allows you to store many layers of information per pixel, which can be applied to shadow mapping, but of course the memory and processing requires go up as you increase the amount of layers that are required.
For an example, look up "deep shadow maps", but be aware that solving this problem in general isn't easy and will involve quite a bit of research.

In the creation of the shadow map, the camera is moved "near" the sun, looking in the same direction as the sun light is going.

Another version of Ashaman's suggestion would be to render one shadow map like this, and then to move the camera "opposite" the sun (looking back towards the sun) and render a 2nd shadow map. This gives you a kind of minimum and maximum distance for occluders from the sun. You could use one map to draw really soft shadows, and the other to draw hard shadows.

Thanks for the elaborate description!

I think I get it; using Ashaman's suggestion will not give a perfect penumbra, but it may be a good enough approximation. Which is what much about game graphics is about...