Technically you don't need all models to have their hierarchies kept, the hierarchy is only for animation, so if a static model (like a building exterior) had an animated component, like a flag waving in the wind, you would still need the hierarchy. And of course most actor's are going to need the hierarchy.
However for other types of models you can indeed pre-transform all the vertices and optimize the scene graph a bit. Search for Scene Graph Optimizers and Render Queue Sorting for reference.
EDIT: I don't know a programmatic way to detect AnimationSets in an X file, but if the X file is in text format, you could just parse for the word "AnimationSet".
Edited by Steve_Segreto, 16 January 2013 - 07:14 PM.
Unfortunately to do the sophisticated kinds of things you want, you will need to move away from the X file and associated loaders.