There's not too much GPU memory management you can do on PC, besides just trusting the driver
Actually there is. Strategy on timing of creation and deletion of data can help out the driver.
Fragmentation is a tricky problem; there's a trade-off with performance. If data are allocated and deallocated all the time in arbitrary order, gaps in allocated memory can appear. If you are able to stick a small piece of data in a gap, it makes it less likely to lead to memory fragmentation down the road. However, tracking all those gaps makes allocation an N^2 operation with the number of allocated data chunks, which can kill performance. Strategies like compaction can affect performance as well. Therefore, there's typically a complex strategy to memory management that balances performance with minimizing fragmentation. On a desktop PC which has lots of speed and memory, the algorithms do pretty well, but they might not do so well on a smaller device with less available RAM and throughput.
But there are things you can do to help keep it simple for the driver. The most obvious is LIFO (last in first out), always deallocating data in the opposite order they were allocated in, which would completely eliminate gaps. Since I can't do that any more, I am thinking of how I can still keep it kind of simple, so that when the game transitions to phones and such, there are no unexpected crashes or missing textures.