KulSeran raises some good points, and it would certainly be worth looking into the examples he's provided. I've built a framework in C# (using Mono) so that I can deploy games across different platforms. In doing so I have dealt with, at least, a subset of this problem. I haven't provided access to all File IO methods, but I have created a limited interface that has served my purposes.
My approach has been to employ a number of handlers for file operations, each being extended with platform-specific implementation. I've been primarily interested in simple IO, reading and writing with path conversion, but the approach might be applicable to your problem too. Firstly, I have a number of base classes.
public class AssetIOHandler... public class StorageIOHandler... public class FileSystemHandler... public class SystemPathValidator...
The first handles input-only from the asset file structure. In the case of Android this would include bundled assets, for example. The second handles read and write access to storage, useful for writing save and configuration data. The third handles checking for existence, directory operations, etc. This seems to be the crux of your question. The fourth takes in a generic file-path (in my case using the Windows standard), and converts it to a platform-specific file path.
Currently instances of these classes are maintained in a static class that automatically handles converting paths before sending them to IO or Directory methods, checks for illegal states, etc. It could easily be handled through your application or game class or wrapped in an IOHandler class instance, however, if you'd rather avoid statics/globals. This approach allows me to define default behaviour, even if that's throwing an exception, and cleanly build the functionality for each platform without impacting on the interface or employing too many preprocessor directives. I simply then set the correct functionality when the game is launched prior to any IO operations.