Problem is, I also need to calculate the intersection point. I am using that information to position my 3D cursor. So color picking doesn't really apply here.
You can still use rasterization to find the triangle at the specific point, then you can use math to find the exact intersection point between the selection ray and that specific triangle.
That said, I think it is more reasonable to make distinct separation between working with dense meshes (sculpting) and "regular" meshes modeled by hand. This will make it easier for you as a programmer, and users will not object to having two different workflows for both kinds of models, as that's generally the standard approach.