I then have many cpp files that implement then stuff in these headers, e.g. I'd have a single Device.h, and then Device_D3D11.cpp, Device_D3D9.cpp, etc...
This is what we do at work.
Maybe function pointers come handy at this situation . Instead of havina a launcher the application can in realtime assign appropriate functions that the platform supports to the pointers . And then application moves on with calling only these function pointers .
That would be a great way to make a mess of your code, and since the application would link to both Direct3D 11 and Direct3D 12, only people running Direct3D 12 could run it, making Direct3D 11 a useless parasitic twin.
If you took anything away from what Hodgman said let it be the part regarding multiple executables. This is standard in the industry for a reason.
Not entirely true. I've had success in the past with Direct3D 9 and 10 in the same executable by marking the D3D dlls as being delay loadable, and then checking for which version of Direct3D the user's system was capable of running and loading that rendering path.