FileSystem Cross-Platform

Started by
25 comments, last by NightCreature83 9 years, 12 months ago

Hi all,

Here what I made :


#ifdef WINDOWS
  #include <Windows.h>
#else
  #include <direct.h>
  #include <sys/types.h>
  #include <sys/stat.h>
#endif // WINDOWS

bool CreateFolder( const CString& Path )
{
#ifdef WINDOWS
  return CreateDirectoryA( Path.GetData(), NULL ) == TRUE;
#else
  return mkdir( Path.GetData() ) == 0;
#endif // WINDOWS
}

bool RemoveFolder( const CString& Path )
{
#ifdef WINDOWS
  return RemoveDirectoryA( Path.GetData() ) == TRUE;
#else
  return rmdir( Path.GetData() ) == 0;
#endif // WINDOWS
}

bool RenameFileFolder( const CString& Old, const CString& New )
{
  return rename( Old.GetData(), New.GetData() ) == 0;
}

bool ExistFileFolder( const CString& Path )
{
#ifdef WINDOWS
  return GetFileAttributesA( Path.GetData() ) != INVALID_FILE_ATTRIBUTES;
#else
  stat Stats;
  return stat( Path.GetData(), &Stats ) == 0;
#endif // WINDOWS
}

Is it the correct way to have it cross-platform ?

Thanks

Advertisement

Well, you're quite plainly missing handling of paths. They're very different between platforms (eg. 'c:\\users\kulseran\my documents' vs '/home/kulseran/' ).

Most applications can't just create and remove folders anywhere, they need permissions. You'll need the platform specific lookups for where to put your game data.

Most games don't want to allow creating/removing files from any old code. You'll want to check the paths against known good paths to edit. You'll want to restrict different parts of your game (eg scripts) from having access to the raw functions without protection.

I suggest you take a look at something like PhysFS for a robust implementation of cross-platform file operations focusing on a game environment. Otherwise, look into boost::filesystem.

Hi Alundra,

KulSeran raises some good points, and it would certainly be worth looking into the examples he's provided. I've built a framework in C# (using Mono) so that I can deploy games across different platforms. In doing so I have dealt with, at least, a subset of this problem. I haven't provided access to all File IO methods, but I have created a limited interface that has served my purposes.

My approach has been to employ a number of handlers for file operations, each being extended with platform-specific implementation. I've been primarily interested in simple IO, reading and writing with path conversion, but the approach might be applicable to your problem too. Firstly, I have a number of base classes.


public class AssetIOHandler...
public class StorageIOHandler...
public class FileSystemHandler...
public class SystemPathValidator...

The first handles input-only from the asset file structure. In the case of Android this would include bundled assets, for example. The second handles read and write access to storage, useful for writing save and configuration data. The third handles checking for existence, directory operations, etc. This seems to be the crux of your question. The fourth takes in a generic file-path (in my case using the Windows standard), and converts it to a platform-specific file path.

Currently instances of these classes are maintained in a static class that automatically handles converting paths before sending them to IO or Directory methods, checks for illegal states, etc. It could easily be handled through your application or game class or wrapped in an IOHandler class instance, however, if you'd rather avoid statics/globals. This approach allows me to define default behaviour, even if that's throwing an exception, and cleanly build the functionality for each platform without impacting on the interface or employing too many preprocessor directives. I simply then set the correct functionality when the game is launched prior to any IO operations.

Unless there is a different CString somewhere I'm not aware of the choice of that as a string class already invalidated all possibility of platform-independent code. That said, the choice of strings is actually rather complex. On Windows you are stuck with either ANSI or you need UTF32, all other current operating systems I'm aware of use UTF8.

Although I'm not completely happy with it (some weird design choices and sub-par documentation) I would strongly suggest to use boost::filesystem for anything not completely trivial. Writing something new truly platform-independent will not be easy and will require quite a bit of continuous testing on all platforms as well as a decent level of previous experience on those platforms.

The last time I looked into PhysFS I was very unhappy with it. It enforced some weird singletonitis and did not really offer much functionality outside of reading files (admittedly from several sources like an archive format).

You definitely need to handle paths in some platform independent way too.

Not only might paths be different in different platforms, some platforms will not let you write anywhere so you also need to have separate paths for Read-Only resources, and for any files you need to write to (such as save files and settings).

An example of such platform is OSX if you want to release in the app store.

It's also true for most mobile platforms, which should be considered before it approaches platform independence in my mind... (and also rules out using boost)

You definitely need to handle paths in some platform independent way too.

Since all path use '/', it's a platform independant way right ? Maybe you mean a check should be added to change "\\" to '/'.

On Windows you are stuck with either ANSI or you need UTF32, all other current operating systems I'm aware of use UTF8.

This is not a problem, my string class stores UTF8 data but can convert to UTF32 string.

Most applications can't just create and remove folders anywhere, they need permissions.

This is why I have boolean output for each function to know if it succeed or not, but maybe error value is better or enum to be more clean.

---

Why the way I do it is not robust since function I use are from Windows API and standard c++ ?

--

Another question is : Is the only way to remove part of file to remove the old file and rewrite completly the file without this part ?

To do this remove I know the offset and the size of the part.


On Windows you are stuck with either ANSI or you need UTF32, all other current operating systems I'm aware of use UTF8.

This is close, and unicode will definitely need to be considered because file names and folders can be in any language. In Windows you must use utf-16 for this. Ascii won't cut it. In most other OSes you can choose between utf-8 and utf-32 but utf-8 is usually the simplest (and at least Windows has functionality to convert utf-8 to utf-16 in order to use it's wide-character functions).

Again, there is no utf-32 in Windows.

Another question is : Is the only way to remove part of file to remove the old file and rewrite completly the file without this part ?

To do this remove I know the offset and the size of the part.

For Windows, see SetEndOfFile.

(and at least Windows has functionality to convert utf-8 to utf-16 in order to use it's wide-character functions)

Right, this is why it's not a problem (You have to take care to use -1 to avoid problems) :


int NumCodeUnits = MultiByteToWideChar( CP_UTF8, 0, utf8_data, -1, NULL, 0 );
wchar_t* wchar_data = new wchar_t[NumCodeUnits];
MultiByteToWideChar( CP_UTF8, 0, utf8_data, -1, wchar_data, NumCodeUnits );
// ...
delete[] wchar_data;

Why the way I do it is not robust since function I use are from Windows API and standard c++ ?


Because you are not using standard C++ functions (C++ does not have functions to interact with the file system). You are using either Windows API function or POSIX functions. POSIX functions are not part of standard C or C++. Linux distributions are not in general POSIX-certified (although will be usually compliant enough for most use cases).

If you want something using standard C++ the closest you can get at the moment would be boost::filesystem. It still seems to be a candidate to be included in tr2.

Edit: And it's actually included in MSVC 2013, although I would still suggest to use the Boost version because it's unclear if there will be breaking interface changes with the standard library implementation.


Since all path use '/', it's a platform independant way right ? Maybe you mean a check should be added to change "\\" to '/'.

No thats not enough.

For example on OSX, you have to put all writable data in /Users/<username>/Library/<appname>/, and all resources should be bundled within the .app bundle, which could be located anywhere.

Both of those paths have to be queried from the system and prepended on all paths before you can send it to fopen, etc.

If you have no clear divide between writable and readonly resources, and separate methods to get their paths, you are kind of screwed.

Also bitmaster makes a good point with fopen being POSIX (a platform) and not platform independent. It's just available in many operating systems.

Not all though, and sometimes they have subtle differences.

You definitely should avoid using fscanf if you do, that one is particularly hard to wrap if you need to port to a platform where no such method exists (such as Android when reading resource files from the .apk)

This topic is closed to new replies.

Advertisement