The deferred renderer I've made combines shadow information in the lighting pass, additively blending the inverse of the shadow maps and multiplying that with the light. Completely opaque shadows look very harsh and not so realistic so there's a user-definable darkness setting that can make shadows completely opaque to completely invisible. But can something be done to select the right amount of darkness? Like if shadow darkness depends on the time of day, or more on how much reflected light hits other surfaces.

I guess some sort of basic GI technique is in order, but what are some other approaches? Also I've realized that shadows are not always a result of multiplying some grayscale value but there's a subtle hue to them in certain lighting situations. Not sure if this is the result of ambient lighting reflected by objects or something else.

Also I've realized that shadows are not always a result of multiplying some grayscale value but there's a subtle hue to them in certain lighting situations. Not sure if this is the result of ambient lighting reflected by objects or something else.

This gets into painting territory, James Gurney has an excellent book on lighting that covers a lot of issues like this. Shadows are the absence of a light source contextual to surrounding areas that are fully or partially lit. Therefore, if there are other light sources in the area, the shadows can take on the other light source's hues, or splash color from surrounding elements. If I light a box with a blue gel on the left, and a red gel on the right, then in the red light's shadow, you'll just get a blue color, and vice versa.

Unfortunately it really complicates the lighting algorithms and shader calculations to try and replicate bounce light and overlapping shadow maps, so it becomes a question of how real or "artsy" you want your result to look. I re-read your post, and the additive blend sounds like it comes close to approximating most shadow-mixing scenarios.

If you want the simplest solution, just add an ambient value to everything (shadows and brightly lit areas). You could vary this depending on the time of day.

If you want the simplest solution, just add an ambient value to everything (shadows and brightly lit areas). You could vary this depending on the time of day.

Sounds like good compromise with simplicity and added realism. Using cubemaps to capture some ambient lighting as shown here sounds like a good approach.

Yeah, I would start trying to work in some sort of ambient term. You can start simple, with just a single color. Then you get a little fancier and try some basic hemispherical lighting, with two colors instead of one (usually you have one color for the sky and one for the ground). Then you can move on to an irradiance environment map stored as a cube map. Then you can move on to compressing that using spherical harmonics, and adding more than one probe location. Or you can explore pre-calculating diffuse bounce lighting into a lightmap or vertex data, or using real-time GI solutions.

Yep, I've noticed in places where usually the sky is a clear blue and not much overcast.

Looks like I will use environment maps for the ambient lighting. Not only for shadows but for all my geometry as well. So basically I just need to use the normalized values of the normal component of the G-buffer and sample the cube texture with that vector's coordinates?