I am currently building a 2D game engine, written in C++ and aiming to be cross platform (at least for Mac OS X and Windows XP) and still in its early stages. Currently I am trying to figure out the best way to deal with file i/o in a portable way. Since I will have to implement some kind of filename mangler to access the right place for game files, I thought it might be time to integrate a file i/o abstraction library such as PhysicsFS. I have skimmed through the PhysicsFS documentation and I think I understand the basics of how it works, but I have a question about the best way to access file data. From what I have read PhysicsFS provides a very low level mechanism for accessing file data, loading data in specified chunk sizes into a specified buffer in memory. This would not be too challenging if I were to write my own parsers for the data types I wish to use, but I prefer to use other external libraries to do the dirty work of converting file formats to usable data. Most of these libraries tend to have file interfaces that require a filename (usually in the form of a pointer to char) or a stdio C style file object (pointer to FILE). If I were to use a file i/o system like PhysicsFS is there a convenient way to adapt libraries that expect a stdio FILE pointer or filename to work with the system, or should I look for libraries that can accept pointers to blocks in memory instead? Additionally, is there an easy way to handle buffers that represent files in memory, or is just using a std::vector type array what's expected? For every data structure type problem I need to solve there always seems to be a handy managed package out there somewhere that does what I need it to do, but I never seem to find out about it until I've already written my badly implemented version so I thought I'd ask first this time [grin]. [Edited by - Trapper Zoid on May 5, 2007 1:46:48 AM]

After a second look through the header files of the libraries I'm using I've found that nearly all of them do, in fact, have an interface that can deal with raw memory instead of files. SDL_image, SDL_mixer and FMOD all have a mechanism for reading memory, although in some cases it involves a bit of digging to figure out what it was. I'm not that sure about TinyXML, but given that library's liberal license I can hack my way around that one.

I'd still like to know though if there's a way I could emulate a file stream from memory to get around this problem if it crops up later, as well as if there's a really simple way to deal with buffers of file data that I've overlooked before I start writing my own interfaces.

The libraries I'm working with (Ogg, Vorbis, libJPEG, libPNG) all allow you to specify the functions to use to read data, so instead of fread and such, they would use your I/O functions. TinyXML just lets you give it a string to parse from memory.
I think building a working FILE * that fread and friends can work with would be quite non-portable, but you might want to take a loop at memory mapped I/O.

Quote:Original post by DaBono
The libraries I'm working with (Ogg, Vorbis, libJPEG, libPNG) all allow you to specify the functions to use to read data, so instead of fread and such, they would use your I/O functions. TinyXML just lets you give it a string to parse from memory.
I think building a working FILE * that fread and friends can work with would be quite non-portable, but you might want to take a loop at memory mapped I/O.

Thanks - I was starting to think the portability would be too much of an issue if I went with a hack to get around that. It's probably easier to avoid any third party libraries that don't have an interface to read from memory. It's mainly the more complicated data structures such as images and audio that I wouldn't want to have to write my own parsers for, and there should be workable solutions for all of those.

check out John Ratcliff's page. He modified tiny so it could work with memory buffers instead: http://codesuppository.blogspot.com/2007/02/tinyxml.html

at the risk of self pimping GTL can be compiled with PhysFS support in to directly read image files from a PhysFS file pointer.

Self pimping is appreciated - I'll see if I can use your library or at the least learn from it how to implement my own generic file loader using PhysFS.