AO is an approximation in itself, so there's not necessarily a "right" and "wrong" way to use. However there are ways that make more sense than others.

Normal AO is essentially your total visibility to the sky about the hemisphere. Thus it makes more sense to apply it to things like a flat ambient term or an IBL diffuse term, since those aren't inherently directional. It probably makes less sense to apply it to a local light source, since those are directional. For instance with AO a corner will be very dark, but if you aimed a a spot light at a corner it should actually be very bright. One thing you can do is store your AO in a basis that has directional information, like SH. This gives you a sort-of "directional AO" which encodes different occlusion values for different directs, which can give you a better approximation for both IBL lighting and local lighting (I believe Crytek does something like this now with their SSAO).

Thanks for the suggestions!

I know this is off topic, but maybe you can/care to answer. What is the point of radiosity normal mapping? As the lightmaps are pre-computed, why isn't the normal mapped lighting calculation baked in there as well. I can only think of 2 possible answers:

• It would allow to change the normal maps after lighting calculation, but that does not really make sense, because changing the material would "invalidate" the pre computed lighting.
• It allows to the keep the light map fairly low res, while still having high res normal mapped static lighting. (Is this even true?)

It's the latter. Typically light maps have significantly lower texel density compared to normal maps...I think for Half Life 2 their luxels were around 4"x4". This is because a normal map can be tiled and reused many places throughout a level, while a light map has to be completely unique for all surfaces. So by encoding your lighting in a basis, you store not just irradiance for a single normal direction on a surface but rather an entire hemisphere of irradiance that you can query at runtime. This lets you get high-frequency lighting variation while still having low-resolution lightmaps (or even vertex lighting).