For some kind of errors, using some kind of intercepting tool can be pretty useful. In the past I have used both glIntercept (https://code.google.com/p/glintercept/) and gDEBugger (http://developer.amd.com/tools-and-sdks/archive/amd-gdebugger/).
I also have some builds of my applications which, at every frame, check the date of the shader files and, if they are newer than the ones that were loaded (I store a timestamp in my Shader class) they are loaded again.
This way I can have the application in one screen, the shaders in Notepad++ in the other, and everytime I save in Notepad++ I see the results in the application. For tuning shaders this is awesome. It makes the application somewhat more unstable and of course it is not efficient, so I would never leave this in my production release...
EDIT:
Oh, I also have a macro for checking glError. It is a macro called __GL which is blank in Release mode and in Debug it basically calls a function that calls glError, and asserts that the error is GL_NONE. When I am desperate enough I just fill the code with calls to that macro in the sections I suspect.
And finally, the new OpenGL 4.1 (I believe) comes with the Debug context, which is a God-sent. More info in: http://www.altdevblogaday.com/2011/06/23/improving-opengl-error-messages/