In particular, I'm unsure what this formula is actually calculating:
float t = -r * mu - sqrt(r * r * (mu * mu - 1.0) + Rg * Rg);
where:
c = camera position relative to the planet's center r = length(c) mu = dot(x,v)/r Rg = radius of the planet, in KM
And what I'm attempting to do, is modify T based on the depth of terrain as sampled from the depth buffer. What I currently have, is the point of the terrain which is directly facing the sun ends up overbright, almost white, which washes out the diffuse colours. If I revert to the basic T formula as shown above, the terrain facing the sun is a typical blue colour which is not overbright but of course it doesn't reflect the changes in terrain.
Short version: Can anyone help me work out what T is actually calculating so I can adjust it to incorporate altitude changes of my terrain?
I've attached an example of my current (clearly broken) method, bottom left area shows terrain without atmosphere effect applied.






