There are multiple ways to do this. I personally prefer the cheap and easy way of doing it. Take a look at this tutorial written by swiftless: http://www.swiftless.com/tutorials/opengl/camera3.html
It uses legacy OpenGL (i.e. glTranslatef and glRotatef), but if you already have your own matrix rotation functions, you can easily replace the legacy/deprecated stuff.
If you prefer a quaternion based camera, take a look at this tutorial by dhpoware: http://www.dhpoware.com/demos/glThirdPersonCamera1.html (IIRC, it's quaternion based, at least the D3D version is).
Hope that helps.