I can give some advice, but it depends on what level of hardware you're targeting. Do you have any idea what your target spec is for your game/engine?

Sorry for the delay. In terms of performance of a single shader, statically compiling different permutations of that shader will always win over having a runtime branch. When you use a branch the compiler will need to allocate registers for any code inside of the branch, and those additional registers can limit your shader occupancy even if the branch is never taken. There is also some cost associated with executing the branch instruction at all, although this should be fairly minimal for the hardware that you're targeting.

In practice, you'll probably need to find a balance between permutations and branches. Adding permutations for everything tends not to scale too well if you add too many, since you can spend a lot of time just compiling them all (this can be especially problematic when using OpenGL since you can only pre-compile shaders for a given hardware/driver combination). You can also hit additional overhead from switching shaders too many times during a frame. So in general I would say you should use permutations when you need to, and use branches when you can get away with it. The best candidates for using permutation are:

1. Complex features that could increase register pressure

2. Features that are difficult to localize inside of a single branch

3. Features that require the vertex shader to output additional values (since you can't avoid this with branching, and additional VS outputs can degrade performance even if the fragment shader doesn't use them).

Hope this helps!

It may be instructive to look at how this kind of problem was solved in the past - Doom 3. There a decision was made that every lit surface has the same combination of shaders with the same inputs and outputs. If the surface didn't have a normal map then a generic one was provided, likewise with specular/etc. This could be as basic as providing a 1x1 texture procedurally generated during startup.

True, it targetted a different class of hardware and branching wasn't possible in the shading language it used, but if there are some of your current branch/permute cases which you can use this approach on (your diffuse/specular/normal basic setup seems a candidate, for example) it may help with keeping things more manageable.

Using default textures is an interesting option -- despite the number of texture fetches being the same, fetching from a 1x1 texture will be faster on average (than fetching from a large texture), because every pixel is fetching the same value, meaning it will very likely be cached.

If you happen to be doing your lighting in tangent-space (which I admit, is rarer these days), then normal mapping vs not normal mapping has no cost difference either (besides the texture fetch).

One issue that I've had with it though, is that the common way of decoding normals from a texture doesn't allow you to ever represent a perfectly flat surface (i.e. a normal of [0,0,1]).

The usual process of decoding a normal map is ((float)8bitTexValue/255.0f)*2-1, and if you put 0 through the reverse of that, it's encoding is 127.5, which either gets rounded to 127 or 128, slightly tilting your normal map in one direction or the other... it's so small that you might not notice, but I have seen it reported as a bug in an engine that followed this design.

It's a tradeoff - if you compress all of the RGB values in the same way, in the old days it would be faster to decompress them all in your shader; nowadays you may even be able to get away with just storing 2 values (at the cost of some extra math ops shader-side) or using something like a 10/10/10/2 format (those extra 2 bits of precision can make a huge difference).

Many of the hacks in Doom 3 should really come with a health warning that they were a valid tradeoff in the days of 128mb gfx cards, but that these days burning the extra video RAM may be a better choice. At the same time, the generalization of surfaces and use of default textures still seems like a good approach.