I tried to implement the algorithm by putting everything in two lookup tables textures, and multiplying them together in a pixel shader. It worked, but i had some artifacts and could not see how to fix them.
The main problem was color banding in the gradiants. I tracked it down to the multiplication of the two textures. Since they're both 8 bits, the result is loosing precision.. to increase the brightness, i also had to scale the result by a constant. It cannot be done before the lookups, because they only store results in a limited range (0-1) and at a low precision (8 bits per component). Honnestly, the result looked like it was displayed on a 16-bits total color buffer.
Although i was calculating the angles (input of the lookup textures) per-pixel, it was still somewhat dependant on the tesselation. Better than vertex shading, but definately not perfect.
So, i tried an hybrid solution, by storing only one lookup texture, and calculating half of the scattering equation in a vertex shader, at a high precision. It worked, and the quality was definately better, but it was still not as smooth as i was expecting from a 32 bits color buffer. I could still see some banding in the sky colors.. and the tesselation artifacts remained.
Now, i'm trying to implement the whole equation in the vertex shader. It will be slower, but the artifacts and the banding will disappear. I'm not happy with the performance, since the vertex shader has to compute a lot of terms and to do an expensive ray/sphere intersection test, but i have no choice.
The following screenshot shows a buggy atmosphere, but it looked somewhat cool, so i thought i'd post it while waiting for a correct one.