The approach I take is that I have header files full of platform-agnostic structures/enum/interfaces/functions, e.g.
• an enum of comparison modes for depth testing,
• a struct describing the depth/stencil state,
• a device class, contexts, states, etc
• a CreateDevice function
I then have many cpp files that implement then stuff in these headers, e.g. I'd have a single Device.h, and then Device_D3D11.cpp, Device_D3D9.cpp, etc...
I then compile the exe using only one set of these cpp files - e.g. I'd end up with Game_D3D11.exe, Game_D3D12.exe, etc...
You can then have a 'launcher' exe choose the right one automagically -- when the user runs Game.exe, it will check for D3D 12, 11 and 9 compatibility and then run the appropriate exe.
As for your actual platform-agnostic API, you need to take care in how you design it. If you model it on just one back-end-API, then that implementation will be a simple wrapper, but the implementations for other back-ends may involve complicated emulation.
You need to find a common abstraction that's easily implemented on all of your target back-ends.
My engine has 7 rendering back-ends, with plans to support 4 more, including D3D12