In general I try to limit platform-specific code to low-level implementations of free functions. This allows the majority of code to be shared between platforms, since your high-level classes and whatnot can just be implemented in terms of your shared low-level interface. However for me this a guideline and not a rule. In practice I often have to consider things on a case-by-case basis, with my decision of where to split off platform code being based on the particulars of what I need to do as well as what platforms I'm working with. In cases where the functionality and optimal usage patterns vary greatly between platforms, I often have to split off into platform-specific code at much higher level. For instance, on PS3 it is common to perform vertex pre-processing on the SPU's as an optimization. This is generally not something you can do just by implementing a thin wrapper around a D3D/OpenGL-style rendering API, since it requires diverging at a higher level.

If you've ever written (or tried writing) a GL-to-D3D wrapper you'll be fully aware of how nasty things get when you go too low. I'd strongly caution against abstracting down to the level of individual API calls; while there is a lot of good commonality between the modern versions of both APIs there are still significant differences that throw things out of whack; vertex format specification and GL's bind to create/modify paradigm are two examples that can cause huge trouble (although GL4.3 will make the former easier when drivers become common).