That article is a fantastic read! Thank you!
On consoles you generally get full control of the video memory allocation strategy, and that's where things get interesting. Broadly, if you're on a system that lets you fully manage video memory yourself you have two main options, both of which have merits and are perfectly viable options used in shipped games:
Unfortunately I'm on the PC. I hope that someday, PCs will be able to have a unified memory architecture like the current batch of next gen consoles.
1. Use pools of fixed sized blocks to limit fragmentation. This is simple and means you can give clear budgets to artists.
2. Leave a chunk of memory unused (25% perhaps), and have the system continuously move assets around to defragment. 25% sounds like a lot of memory to leave on the table, but the pool approach will likely waste just as much if not more from having fixed sized blocks that don't fit tightly.
This makes a lot of sense. I can still use a system like this even without complete memory management access, if I have one giant vertex buffer divided logically into 4k chunks for example, with a bitmap to see what's allocated.
Also I just read that the cost to set the active vertex buffer is fairly minimal, so maybe it was a bit foolish of me to immediately start thinking about optimizing. I had assumed that the cost to bind a vertex buffer was on the same order of magnitude as the cost to bind a texture, but after some further testing it seems to be much faster.
I wish there was a giant test database of different video cards and how long they take to complete certain OpenGL commands.