A thanks for the recommendations. Looks like I'm off to implement nice batching then. I thought so, I was just befuddled at what a difference it makes, the little bit of text would cause such tremendous slowdowns.
@NightCreature83: I was creating a buffer for every quad. So that's quite a few reuses per frame for a few lines of text
Its the number of buffers here that are the problem, you are usually better of creating a fewer vertex buffers and having a few large ones than really small ones, the CPU overhead will cost more on the small buffers than on the large ones.