Hmmm, now that I think it through... I assume you'd want to go with a cascaded shadow map approach so as not to have your results just snap into reality, and since they're going to be done from a fixed position at all times you'd get it go much faster with something like: http://advances.realtimerendering.com/s2012/insomniac/Acton-CSM_Scrolling%28Siggraph2012%29.pdf which is a caching scheme for cascaded shadow maps.
And really what you've got here is a hacky version of shadowing a hemispherical skylighting term. Which is nice all around, especially if it's ever used for some sort of highly dynamic environment where things are going to get moved around a lot. You could also advance your shadow term even more by using all four shadow maps and something like: http://people.cs.clemson.edu/~geist/public_html/public_html/seminar/SoftShadowMapping-egsr06.pdf Which uses a shadow maps as a scene proxy to trace back to an area lightsource. With four shadow maps instead of one I'd imagine you'd avoid some of the drawbacks of such (holes and etc.) and with modern compute shaders you'd probably pick up more performance if you did it right.
Well, whatever, it's a neat idea any way you strike it!