1ms difference sounds to me though like quite a large difference to begin with.There is actually no way it could be any different.
Sounds about right. Did you work out whether switching FBO was faster than switching attachments? I'm curious to know.
When you bind a different FBO, the driver internally does something like the equivalent of changing a pointer. When you switch attachments, the FBO is in an unknown/undefined state. The driver must run a number of checks on both the attachment and the FBO (render target size, bit depth, framebuffer completeness, and what not). What's worst, it has little choice to defer those checks and batch them together.
If a FBO is not bound, the driver can, as an optimization, choose to do the completeness check no earlier than either you ask for completeness or attempt to bind it. If it is bound already, the driver must do the check every single time you touch an attachment.
EDIT: Besides you can't change attachments on an FBO if it is not bound so it is not possible to optimized the attachment process of an FBO if it is not bound.