If your planet sphere vertices have normals, you can just render a larger version and dot the normal with the normalized (eye -> worldposition) vector to determine how much the rendered geometry is "facing" the camera. Stuff that's directly facing the camera will be invisible, stuff that's directly perpendicular (the edges of the atmosphere as seen from the viewer) will be the most visible.
Essentially, the dot product will give you a number between 0 and 1 (well, -1 and 1, but negative stuff is facing away and won't be rendered if culling is enabled). Then you can map that 0 to 1 range to whatever "alpha" you need... you'll probably want an exponential function in there, since the atmosphere would only tend to be truly visible at values close to 0.
I use something similar to render the atmosphere here: