Background

I am working on a physically based shading system in Unity. Among the many components needed by this system is way to compensate for specular aliasing. Our chosen method involves calculating Toksvig AA from our normal maps, pre-correcting our specular powers, and storing the result for use at runtime.

Problem

We are dealing with ARGB32 precision normal maps as inputs, and are encountering the problem that some of our artists tools don't quite output unit length normals in this precision. Unfortunately, this has dramatic consequences when applied to our pre-correction of specular powers, even for "flat" normals. So materials that are supposed to be smooth end up getting dramatically more rough.

Questions

1.) Is there a way to deal with this besides just using higher precision storage for our normal maps?

2.) One of our tactics to try to deal with this problem is offering the artist a bias factor that adjusts the calculated variance down slightly to compensate. Is this a valid solution, or will it break horribly down the line?

3.) Is there some other trick I could try that I have overlooked?

Thanks for any help you can provide.