The void pointer thing is troublesome. I may be missing some of the fine points, but to me, the point of having an interface like IMeshLoaderPlugin, and a method like IMeshLoaderPlugin::loadFile, is that, given an IMeshLoaderPlugin, you can call loadFile() without knowing what kind of plugin you have. In this case, that is lost; if you have an assimp plugin, you have to give it an AssimpDesc or it will fail. Worse, it will fail at run-time, not at compile-time.
So, my question to you: Is there some good reason to have a generic IMeshLoaderPlugin::loadFile method, and the related IMeshLoader::load function? Why not just have separate AssimpLoader::loadAssimpFile and HeightmapLoader::loadHeightmapFile functions? You may well be getting some other benefit from the generic IMeshLoaderPlugin::loadFile method that I'm just not seeing. However, if I were you, I'd be looking at my class abstraction carefully. If you really have plugins that are not interchangeable, then you probably don't need any of this enum stuff in the first place.
Anyways, I hope that's a little helpful. I've probably asked more questions than supplied answers. Good luck!