There are several ways, but the two most common are probably:

1. Use OpenGL 'feedback' mode. Set up a 'picking matrix' with gluPickMatrix() (or whatever the function is); this will restrict the frustum to a small area around the mouse position, more or less. Then do a 'fake' rendering pass, where you don't actually draw onscreen. Anything that intersects the frustum will trigger a callback and return the ID of the object (which you assign).

2. Use gluUnproject() and a little vector math to create a picking ray. Intersect geometrically with your objects or their bounding volumes to find which should be selected.

The first option might be easier if you don't have much experience with collision detection. More info here, here, and in the Red Book.

Or you could do multiple render target or use the stencil buffer to store the object id in the buffer. Then you simply readback the id on the mouse screen position and check whats the id