Is it always safe to generate the bitangent from the normal and the tangent? I can imagine that with some funky UV mapping it'd be possible for your bitangent to be incorrect. I seem to remember that to be safe when generating an NBT you're best off with a tangent, a bitangent and a sign bit/scalar with which to generate the normal in the correct direction.
That said, I'd be surprised if that particular floor mesh did have the sort of odd UV mapping that would have caused this, so perhaps the problem lies elsewhere.
The Image loader I use (SDL_Image) loads images upside down so I flip the uv-coords to compensate, might that have something to do with it? also as a more accurate description of the problem, the effect is correct on the right side of the player but wrong on the left side and this will anomaly will remain even as the player moves around always on their left.