Sign in to follow this  

istream_iterator for raw bytes?

This topic is 4111 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

Is there not a way to read the raw bytes of a file using an istream_iterator? I would like to be able to read the raw bytes, modify them, and then write them to another file. I tried this, but it skips white space
typedef unsigned char uchar_t;

vector<uchar_t> data;
ifstream in("in_file.txt", ios_base::in | ios_base::binary);
ofsteram out("out_file.txt", ios_base::out | ios_base::binary);

// read the raw bytes from the file
copy(istream_iterator<uchar_t>(in), istream_iterator<uchar_t>(), back_inserter(data));

// transform the raw bytes
transform(data.begin(), data.end(), data.begin(), some_func);

// write the raw bytes to another file
copy(data.begin(), data.end(), ostream_iterator<uchar_t>(out));

Share this post


Link to post
Share on other sites
Quote:
Original post by ToohrVyk
Initialize the vector to the size of the file and use istream::read to read the binary data in one sweep. It's faster, and requires less code.


istream::read() takes a char*, not a vector<T>::iterator

Share this post


Link to post
Share on other sites
The following snippit is from this thread:
http://www.gamedev.net/community/forums/topic.asp?topic_id=355078


inline std::string loadtext(char * filename) 
{
std::ifstream fin(filename);
return std::string((std::istreambuf_iterator<char>(fin)), std::istreambuf_iterator<char>());
}


I admit, it doesn't load it into a vector, but if I know my STL good enough, you shouldn't no problem replacing std::string with std::vector<char>
This function preserves whitespace and all the other bytes.

Share this post


Link to post
Share on other sites
Quote:
Original post by scottdewald
Quote:
Original post by ToohrVyk
Initialize the vector to the size of the file and use istream::read to read the binary data in one sweep. It's faster, and requires less code.


istream::read() takes a char*, not a vector<T>::iterator


std::vector guarantees that its storage is contiguous. Therefore, you can use &(*vector.begin()) as an argument to istream::read() (applying any required reinterpret casting if the type of the vector is not compatible with char*).

Share this post


Link to post
Share on other sites
Quote:
Original post by ToohrVyk
Quote:
Original post by scottdewald
Quote:
Original post by ToohrVyk
Initialize the vector to the size of the file and use istream::read to read the binary data in one sweep. It's faster, and requires less code.


istream::read() takes a char*, not a vector<T>::iterator


std::vector guarantees that its storage is contiguous. Therefore, you can use &(*vector.begin()) as an argument to istream::read() (applying any required reinterpret casting if the type of the vector is not compatible with char*).


I've always read that vector does not guarantee contiguous storage, however, most libraries implement it that way. If I am wrong, could you please point me to the section in the standard that states that it used contiguous memory?

Share this post


Link to post
Share on other sites
Quote:

vector - standard library template providing contiguous storage, re-sizing and the useful push_back() functions for adding elements at the end. Vector is the default container. See also: map, multimap, list, deque. TC++PL 3.7.1, 16.3.

I think that this guy knows! =)

Share this post


Link to post
Share on other sites
Quote:
Original post by scottdewald
Is there not a way to read the raw bytes of a file using an istream_iterator?


No. An istream is a formatter that lies on top of a stream. What you want is an istreambuf_iterator. That's just what it's designed for.

Share this post


Link to post
Share on other sites
Quote:
Original post by scottdewald
I've always read that vector does not guarantee contiguous storage, however, most libraries implement it that way. If I am wrong, could you please point me to the section in the standard that states that it used contiguous memory?


Quote:
23.2.4 [lib.container.adaptors], paragraph 1:
The elements of a vector are stored contiguously, meaning that if v is a vector<T, Allocator> where T is some type other than bool, then it obeys the identity &v[n] == &v[0] + n for all 0 <= n < v.size().


It's been there since the 2003 Technical Corrigendum.

Share this post


Link to post
Share on other sites
Thanks for the clearing me up on vector being contiguous.

Are istreambuf_iterator's extremely slow? I ran some simple tests by reading in a file with istreambuf_iterator's and writing them out with ostream_iterator's vs. using fread() and fwrite() with a single char, and the latter was 5x faster. Is there any way to speed up iterators?

Share this post


Link to post
Share on other sites
Check your compiler settings. If you're using (certain versions of) MSVC, iterators are "checked" by default, even in release mode, but you can disable that.

Share this post


Link to post
Share on other sites
Quote:
Original post by Zahlman
Check your compiler settings. If you're using (certain versions of) MSVC, iterators are "checked" by default, even in release mode, but you can disable that.

Thanks. I didn't realize it, but with my compiler, I had to explicitly turn on inlining. After doing that and turning on optimizations (-O), I got much better performance.

Share this post


Link to post
Share on other sites

This topic is 4111 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this