I'm working with diffuse and specular irradiance environment maps, and the results look great, some of the time. Unfotunatly, on some types of objects, the lack of visibility calculation is overwhelmingly apparent and completely destroys the effect. It's very apparent on complex objects when the environment has strong contrast, eg. in my image you see the head with a very bright source of light on the opposite side.
What are some ways in which this can be handled?