OK, well it looks like I solved it.
It seem that the order of operations of glClear() and glDepthMask(GL_TRUE) actually really matters. Depth mask needs to be true BEFORE you clear. It makes sense, because it allows writing the depth buffer. If it's false when you clear, you won't clear the depth buffer.
I was trying to have the depth mask true for my geometry pass, but false for light and stencil passes. Without realizing the implications I just put the depth mask true after the clear.
Hopefully this will help anyone with a similar issue.
Mods can close the thread I guess?