I am trying to make a simple turret system without using skeletal animation.

I have 2 meshes, turret base and the turret itself. I am trying to attach a particle emitter to the ends of the turret where the cube is like this:
http://imgur.com/tbely

I just rotate the turret mesh around the y axis and it looks like its aiming for something, but I tried to rotate the cube around the turret using the same rotation matrix and it never aligns properly

Is there some different way I should be doing this? do I really need to use bones and animation files for this? there must be a simpler way of handling this.

Here's what I do:

parentMatrix = the transform of the turret's base (the part that does not rotate)
targetPos = world position of the target
turretPos = world position of the turret

direction = Normalize(targetPos - turretPos);

// Optionally constrain the direction to a plane if you only want the turret piece to rotate on a plane.

direction = parentMatrix.InverseTransformDirection(direction); // convert world direction to local direction

// Generate a quaternion from the direction and local up vector. (my solution is Unity-specific using the LookRotation method)
// Interpolate the turret's orientation towards the quaternion. (my solution is Unity-specific using the RotateTowards method)