QFE. The tl;dr version is that Blinn-Phong is actually an approximation to evaluating a Gaussian centered on the halfway vector.
Phong was intentional, since it's more accurate (I am aware it's slower).
Actually Blinn-Phong produces more accurate results (try both at glancing angles and you'll see how bad Phong looks!), to even better results try Enery conserving Blinn-Phong.
In more plain English, you're using some statistics hacks to guess what fraction of the total surface of the area to be shaded is angled in such a way to bounce light towards you/give you laser eye surgery if it starts out coming from the light in question.
EDIT: And for extra credit, use Toksvig filtering to account for actual texture detail in the normal map!
EDIT 2: Also
float NdL = max(0.0f, dot(Normal, LightVector));makes me really, really angry. You wouldn't like me when I'm angry. Do
float NdL = saturate(dot(Normal, LightVector));instead to avoid my wrath.
For clarification, you're wasting precious GPU time with those max() operations that you could be getting for free with a saturate modifier. You might think that the compiler can optimize this. You'd be wrong, though-- remember that the dot product itself does not have a defined range and that the compiler generally lacks sufficient context to know that you're dotting normalized vectors.