Is binding a resource very time consuming? I assume they are already on the GPU so shouldn't be much right?
It depends very, very much on the hardware, but in general you should minimize state changes. E.g. when binding a resource the caches are invalidated and several things are validated (are all mipmaps available (for textures), have the formats changed etc). Probably even worth is changing a shader, since it internally changes much more states, potentially invalidating various caches (including the texture cache) as well as inlining of functions in case you use dynamic shader linkage etc.
Writing a system that keeps on the CPU side track of the states and binds them when you submit a draw or dispatch call, isn’t worth it btw, since the GPU already does that for you.
We have many experts here in the forum, so if you open up a new topic, you can get much more (and better ) information on this.