Jump to content
  • Advertisement
Sign in to follow this  
weasalmongler

Long Text File corruptions

This topic is 4905 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

Hi everyone, I've just put level scripts into my game engine and I've run into a bizarre problem. It appears that roughly 3% of the end of longer text files that I load get corrupted. It is about 3% no matter what the size of the text file rather than being a fixed number of bytes. The files that I am loading are about 2kb. I am using ifstream to load my files and I load them all into a big buffer before parsing them in memory. Has anyone ever had a problem like this? Anyone know any causes and how to fix it? Thanks, - James

Share this post


Link to post
Share on other sites
Advertisement
Never had that problem.

Are you sure that you're not overwriting the bounds of your buffer?

Share this post


Link to post
Share on other sites
I'm pretty sure that I am not, I don't write to the buffer once it's been loaded and the buffer is the same size as the file. Here is some of the code for loading :



m_hFile.open(filename, ios::in);

if(!m_hFile.is_open())
return false;

//Get length
m_hFile.seekg(0, ios::beg);
l = m_hFile.tellg();
m_hFile.seekg (0, ios::end);
m = m_hFile.tellg();

m_iFileLen = m - l;

//Make sure we are at the start
m_hFile.seekg(0, ios::beg);

m_cFileBuffer = new char[m_iFileLen+1];

memset(m_cFileBuffer, 0, m_iFileLen+1);

m_hFile.read(m_cFileBuffer, m_iFileLen);

//Set our incremental pointer to the beginning of the file
m_cPtr = m_cFileBuffer;

m_hFile.close();




Any suggestions?

- James

Share this post


Link to post
Share on other sites
Actually, I just printed out the file just after it had been loaded and it worked fine, but when the parser gets to the end of the file in the debugger, the data is corrupted, but I don't believe that I write to the buffer at all, is it possible for something to be causing this?

- James

Share this post


Link to post
Share on other sites
Try calling istream::gcount after you call read.

That won't solve anything, but it should help you to figure out if the data is being read then corrupted or not being read at all.

Is what you're doing working well with smaller text files?

Share this post


Link to post
Share on other sites
You can't expect m-l to tell you the correct length of a text file. They go through extra processing, so the number of characters you read doesn't necessarily add up the the number of bytes you've gone through.

Share this post


Link to post
Share on other sites
You have 2 options, either read the file as binary, in which case your current reading functions will work fine, or use ascii specific functions. The reason being that 1 character in ascii might not be 1 byte (I found out the hard way).

Share this post


Link to post
Share on other sites

#include <iostream>
#include <fstream>
#include <string>
#include <algorithm>
#include <iterator>

int main()
{
std::ifstream file("message.txt");

if(!file.good())
{
std::cout << "Oopsies!" << std::endl;
return -1;
}

std::string my_text;

file.unsetf(std::ios::skipws); // Don't skip whitespace.

std::copy(std::istream_iterator<char>(file),
std::istream_iterator<char>(),
std::back_insert_iterator<std::string>(my_text));

std::cout << "The file contained:\n----\n" << my_text << "\n----" << std::endl;
return 0;
}

Share this post


Link to post
Share on other sites
Thanks guys! It was the filesize problem, I changed my loader to count the number of characters rather than m-l and now everything is running fine.

Happy coding :)

- James

Share this post


Link to post
Share on other sites
Sign in to follow this  

  • Advertisement
×

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

We are the game development community.

Whether you are an indie, hobbyist, AAA developer, or just trying to learn, GameDev.net is the place for you to learn, share, and connect with the games industry. Learn more About Us or sign up!

Sign me up!