the library itself is usually considered a completely separate project, benefits from things like stable versioned releases and regression testing, and can be separately distributed so it can be shared with friends and colleagues.
This is one end of the spectrum, and the appropriate approach in some cases where the library project is well defined and reasonably large. I've found myself on the other end, where I first tended to write the same code (e.g vector math) over and over again, and then moved to lots of copy-and-paste over and over again, and finally to GIT. There's no real "library" project in there, no library "team" and definitely no planned work devoted just for library code. There's just a lot of code reuse that's evolving over time. Moving to GIT (or any other good version control system) is the right tool for such an agile approach where you always work on real products and any shared code is generated as a byproduct of the refactoring steps.