It was all down to the fonts for which I use freetype. When I create the font I create a bunch of opengl textures(128) and use FT_Bitmap etc to make and font texture for each glyph. Anyway I was missing a simple FT_Done_Glyph. Bloody hard to find and simple to fix *sigh*
If I hear another cover song I'm going to have to start to hurt people.