You mentioned that you are using a shader for this. To me it looks like maybe there is a conditional switch causing a sharp cutoff for the specular highlights.
Look for a conditional like this in the fragment shader. Get rid of the 'if' statement if it's there, the highlights will look better and the shader will run faster as well.
if(diffuse > 0.5)
specular = dot(halfVec, shininess);
Also, turn down shininess, it will spread the highlight out a bit so it spreads out around bumps a bit more, it'll flicker less.
If you are using normal maps, blur them a bit in a 2D image editor and this will reduce the harsh flickering even more.