This is something of an oversimplification, but a rough answer would be that a compositing manager handles display of the window images whereas a window manager handles the windows themselves. While on earlier GUI systems the window manager was sufficient to do rendering as well, due to the lack of effects such as transparency, on more recent GUI systems such as Aqua, there''s sufficient extra work to be done that putting management of those functions in its own subsystem is justifiable.

IIRC, (and I may not, the last time I read up on the OS X window management system was many moons ago), the composition manager is not just a renderer, it also pre-processes some input before that information is passed to the window manager. But it''s primary usage is to do the pretty special effects like transparency or the weird genie animation, etc.