Sign in to follow this  

File I/O Methods: Using 'streambuf'?

This topic is 4483 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

I have two methods that can be used for file I/O in C++: Simply using file streams (fstream), or using virtual files (the file in RAM [char* & size_t]). Question is, what is the most efficient way I can combine those two and easily use them? One thought I had was using STL I/O streams, and if it was a simple file, use 'init(&file)', and if it was a virtual file, somehow put that virtual file data into the stream (without buffering?). The reason why is, I just want to use a stream to read data by either way, instead of having to use two conditional blocks, one with simple RAM reading, and another with 'fstream'. This is the crappy function I have so far:
//Should it be a pointer to a pointer for more options?
bool [Class]::GetFile(iostream *pOut) {
 //Note: If it's a virtual file, there is double buffering slow-down
 if (!loaded) return false;
 if (IsVirtual()) {
  char *fdata; size_t fsize;
  if (!GetVirtualFile(&fdata,fsize)) return false;
  pOut->write(fdata,fsize); //How efficient is this?
  return true;
 }
 else {
  fstream file;
  //'fname' is a member of the class
  file.open(fname.c_str(),ios::in | ios::binary);
  if (file.fail()) return false;
  pOut->init(&file);
  //Will it automatically close the file once it is destroyed?
  return true; //GAHHH!!!
 }
}



Also note that a virtual file is just a single file stored inside of another file, utilizing more management and grouping. In "[Class]::GetVirtualFile(char**,size_t*)", I simply read that 'file' and return it's data. UPDATE: New question about 'streambuf' class... [Edited by - deadimp on September 7, 2005 12:36:25 PM]

Share this post


Link to post
Share on other sites
This is just a guess, but perhaps using a stringstream to store the virtual file could allow you to use it as an iostream (hence allowing you to use iostream-based code for both real and virtual files).

Share this post


Link to post
Share on other sites
Thanks.
I've looked a little more into the iostream library and found 'streambuf', and I have decided to go with that (using 'streambuf::pubsetbuf(...)') for virtual files. However, will the 'fstream' still allow me to use 'read' without opening an actual file? Also, will the 'streambuf' automatically deallocate the memory I passed off to it, or will I have to do that myself. Or does it copy/buffer the memory I provide it (I hope not)?

Share this post


Link to post
Share on other sites
Quote:
Original post by deadimp
I have two methods that can be used for file I/O in C++: Simply using file streams (fstream), or using virtual files (the file in RAM [char* & size_t]). Question is, what is the most efficient way I can combine those two and easily use them?


There are a couple of ways of going about this but let me it say something fundamental about the C++ iostreams themselfs, they are parameterized polymorphic types meaning they are class templates and also have just one virtual member function (that is just the destructor first declared in std::ios_base). What that means is you can have reference to std::basic_i/o(io)stream that can refer to any sub-type of those said classes hence one way of dealing with this is to refactor out specific iostream sub-types (like std::basic_i/o(io)fstream) and push specific operations out to clients to deal with, like open an a file stream e.g.

Instead of this:


#include <fstream>

void foo(std::fstream& fref) {
if(!fref.is_open())
fref.open(...);
....
}


Do this:


#include <iostream>

template < typename CharT, typename Traits >
void foo(std::basic_iostream<CharT, Traits>& io_ref) {
// just use the stream...
}

....

std::fstream fs(....);

foo(fs);


Also take note that the C++ iostreams are nothing more than an interface & mediator to nearly the entire architecture of C++ iostreams & locales. They don't really do that much of the work, the main participants of a stream are:


  • The locale, facets, and stream buffer iterators to deal with localized/internalization parsing/formatting.

  • Stream buffer that deals with the transportation layer, they abstract platform specific I/O devices (or anything that acts like one) and may also preform buffering depending on the context.



That only skims the surfaces, there are othing things involved too like formatting states, stream streams etc. The framework is easilly extended with minimal fuss (once some of the architecture and conventions used is understood) via a thew different methods, not just via inheritance.

Quote:
Original post by deadimp
One thought I had was using STL I/O streams, and if it was a simple file, use 'init(&file)', and if it was a virtual file, somehow put that virtual file data into the stream (without buffering?). The reason why is, I just want to use a stream to read data by either way, instead of having to use two conditional blocks, one with simple RAM reading, and another with 'fstream'.
This is the crappy function I have so far:


Well you could do what i mentioned earlier, another options is change to a stream's stream buffer associated with it with any other instance of a sub-type of std::basic_streambuf, you can set a stream's buffer via the rdbuf method which also returns the address of the original stream buffer.

The only problem if you are dealing with std::basic_i/o(io)fstream directly this will not wont work (or works but not portable) because there stream buffers usually have extended methods like open/is_open (which is not guarneeted) not in std::basic_streambuf's interface.

So the safest thing to do is make instance of (or have a reference to) std::basic_i/o(io)stream (not a file stream) and give it an address of an instance of any sub-type of std::basic_streambuf e.g.



#include <iostream> // basic_iostream
#include <fstream> // basic_fstream
#include <sstream> // basic_stringstream

....
std::stringstream ss(....);
std::fstream f(...);

// must be done on construction for basic_i/o(io)stream
std::basic_iostream<char> foo(f.rdbuf()); // file stream buffer

....

// but you can change it later on if you want.
foo.rdbuf(ss.rdbuf()); // string stream buffer.

....


Quote:
Original post by deadimp
I've looked a little more into the iostream library and found 'streambuf', and I have decided to go with that (using 'streambuf::pubsetbuf(...)') for virtual files.


well its actually std::basic_streambuf (its also a parameterized polymorphic type) and i think you should read this first.

Quote:
Original post by deadimp
However, will the 'fstream' still allow me to use 'read' without opening an actual file?


I think i've answered this above.


Quote:
Original post by deadimp
Also, will the 'streambuf' automatically deallocate the memory I passed off to it, or will I have to do that myself. Or does it copy/buffer the memory I provide it (I hope not)?


I think you might be slightly confused i think the best thing to do is go through Aneglika Langer's online IOstream & Locale articles especially the one on stream buffers & stream buffer iterators. That should clear up a thew things.

[Edited by - snk_kid on September 7, 2005 2:16:33 PM]

Share this post


Link to post
Share on other sites

This topic is 4483 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this