File I/O

Started by
9 comments, last by Drew_Benton 18 years, 11 months ago
First off, greetings... Post Count = 1; Read the short or the long... whatever suits you. Short: I'm working on a program to do some "simple" text manipulation. It takes in a file and creates new files based on the data inside. I have a problem where I get an infinite loop after i make about 18 files... which is nowhere what I need. I would upload the code... but don't have it here. Long: Since that helped probably nobody, here's the long. I created a pdf in Adobe Acrobat back a few months that used forms to fill in the data for a 1099-S form(taxes in case you were wondering). The database that I am using outputs the data that I want into line separated entries. Each entry contains strings in quotes and separated by commas. ie "string1","string2","string3",etc. The first entry is always 'Y' or 'N'. The program's function is to take the data exported from the database and turn it into fdf's, three entries per file. If the first entry is Y then add the entry, else go to the next one. I am having problems with an infinite loop around the 18th file that I create. I also would like to confirm that I am doing this correctly. Sorry no real code to look at, not at that computer right now. But this is the basic structure(mix of pseudo and real): main() {Sets up files for use... opens and closes output file until writeFile returns false} bool writeFile(ofstream &fout, ifstream &fin) {writes initial data to file then loops until it finds three 'Y' or eof is reached. Then writes final data to file. Returns true if it writes 3 records. Makes calls to writeRecord} void writeRecord(ofstream &fout, ifstream &fin) {writes initial data to record, then loops using writeData to write rest of input line to file} void writeData(ofstream &fout, ifstream &fin) {writes to file until a " is found.} I'm using fin.get() to grab the next letter from the file and looking at it. Would it be better to getLine and then use that array? When i'm creating the new files... I new an ofstream, open the file, write, close the file, delete the ofstream. I'm not worried about efficiency... obviously don't want a O(2^N) but can manage. Less than a thousand entries are put into the export with most being N. Any help would be greatly appreciated.
Advertisement
Hi and welcome.

We obviously can't fix your problem without seeing the source (the data would help too, but I can understand why you wouldn't want to provide it [smile]). The only suspicious bit in your description would be "writes to file until a " is found." - that's the only way I can see an infinite loop happening, a priori.

Since your data file is line-oriented, you should really write your functions to process it line by line. std::getline(std::istream&, std::string&) does wonders. Then you take that line and either use it to initialize a std::stringstream or feed it to a parser. Taking that approach allows you to recover more easily if the parsing of one line fails.

Which reminds me that you should always check if your operations fail (e.g. if you try to read a number, but the data contains text instead, the read operation will fail and, if you're doing so in a loop, you will get an infinite loop, for the offending characters are not consumed by the failed read). Functions like std::ifstream::fail(), std::ifstream::ignore() and std::ifstream::clear() can prove useful there.

That being said, C++ sure is not the best language to do data munging in. Python (or, if you lean that way, Perl) is more convenient and, as you mentioned, you're not worried about performance.
"Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it." — Brian W. Kernighan
Ok I've got the code now... Butcher it up:

// File:             // Author:           David Culpepper// Created:          05/09/05// Last Modified:    05/10/05// DESCRIPTION //////////////////////////////////////////////////////////////////* * */// END DESCRIPTION /////////////////////////////////////////////////////////////// INCLUDES ////////////////////////////////////////////////////////////////////#include <iostream>#include <fstream>using namespace std;// FUNCTION PROTOTYPES /////////////////////////////////////////////////////////void writeData(ofstream &, ifstream &);void writeRecord(int, ofstream &, ifstream &);bool writeFile(ofstream &, ifstream &);void processLine(ifstream &fin);// main ////////////////////////////////////////////////////////////////////////int main(){   ifstream* fin;   ofstream* fout;   char outFileName[27] = "Z:\\1099\\Forms\\1099-000.fdf";   fin = new ifstream();   cout << outFileName << endl;   fin->open("Z:\\1099\\1099.txt");   int i = 0;   int temp;   fout = new ofstream();   do   {      ++i;      fout->close();      delete fout;      fout = new ofstream();      temp = i;      outFileName[21] = (char)(48 + i % 10);      i /= 10;      outFileName[20] = (char)(48 + i % 10);      i /= 10;      outFileName[19] = (char)(48 + i % 10);      i /= 10;      i = temp;      cout << outFileName << endl;      fout->open(outFileName);   }while(writeFile(*fout,*fin));   fout->close();   fin->close();   delete fout;   delete fin;   return 0;}// End main// writeData ///////////////////////////////////////////////////////////////////void writeData(ofstream &fout, ifstream &fin){   int letter;   fin.get();   fin.get();   letter = fin.get();   while(letter != '"')   {      fout << (char)letter;      letter = fin.get();   }}// End writeData// writeRecord /////////////////////////////////////////////////////////////////void writeRecord(int record, ofstream &fout, ifstream &fin){   fout << "<< /V (George Culpepper\\rPO Box 1184\\rMeridian, MS 39302\\r601-693-2055)/T (Edit1a)>> \n"           "<< /V (64-0740444)/T (Edit" << record << "b)>> " << endl;   for(int i = int('c'); i < int('l'); ++i)   {      if(i != int('j'))      {         fout << "<< /V (";         writeData(fout, fin);         fout << ")/T (Edit" << record << char(i) << ")>>" << endl;      }// End if      else      {         fout << "<< /V (";         writeData(fout, fin);         fout << "\\r";         writeData(fout, fin);         fout << ")/T (Edit" << record << char(i) << ")>>" << endl;      }// End else   }// End for      // Process to the end of the line   processLine(fin);}// End writeRecord// writeFile ///////////////////////////////////////////////////////////////////bool writeFile(ofstream &fout, ifstream &fin){   char letter;   int i = 0;   fout << "%FDF-1.2 \n"           "%âãÏÓ \n"           "1 0 obj \n"           "<< \n"           "/FDF << /Fields [ " << endl;   while(i < 3)   {      if(!fin.eof())      {         fin.get();         fin.get(letter);         if(letter != 'Y')         {            processLine(fin);         }// End if         else         {            ++i;            fin.get();            writeRecord(i,fout,fin);         }// End else      }// End if      else      {         fout << "<< /V (Filer's Name, Address, Phone)/T (Info 1)>> \n"                 "<< /V (Property Address)/T (Info 10)>> \n"                 "<< /V (Buyer's Part of Tax)/T (Info 11)>> \n"                 "<< /V (Filer's ID)/T (Info 2)>> \n"                 "<< /V (Transferor's ID)/T (Info 3)>> \n"                 "<< /V (Transferor's Name)/T (Info 4)>> \n"                 "<< /V (Transferor's Address)/T (Info 5)>> \n"                 "<< /V (City, State, Zip)/T (Info 6)>> \n"                 "<< /V (Account Number \\(Optional\\))/T (Info 7)>> \n"                 "<< /V (Closing Date)/T (Info 8)>> \n"                 "<< /V (Proceeds)/T (Info 9)>> \n"                 "] \n"                 "/F (/VIVIANCOMPUTER/DOCUMENTS/Active/Office Administration/Forms/1099-S Form\\.pdf)\n"                 "/ID [ \n"                 "<28bf4e5e4e758a4164004e56fffa0108>\n"                 "<45bc4a2d33af487cf0757290bf5b9617> \n"                 "] \n"                 ">> \n"                 ">> \n"                 "endobj\n"                 "trailer\n"                 "<< /Root 1 0 R >>\n"                 "%%EOF\n"                 "";         return false;      }// End else   }// End while   fout << "<< /V (Filer's Name, Address, Phone)/T (Info 1)>> \n"           "<< /V (Property Address)/T (Info 10)>> \n"           "<< /V (Buyer's Part of Tax)/T (Info 11)>> \n"           "<< /V (Filer's ID)/T (Info 2)>> \n"           "<< /V (Transferor's ID)/T (Info 3)>> \n"           "<< /V (Transferor's Name)/T (Info 4)>> \n"           "<< /V (Transferor's Address)/T (Info 5)>> \n"           "<< /V (City, State, Zip)/T (Info 6)>> \n"           "<< /V (Account Number \\(Optional\\))/T (Info 7)>> \n"           "<< /V (Closing Date)/T (Info 8)>>\n"           "<< /V (Proceeds)/T (Info 9)>> \n"           "] \n"           "/F (/VIVIANCOMPUTER/DOCUMENTS/Active/Office Administration/Forms/1099-S Form\\.pdf)\n"           "/ID [ \n"           "<28bf4e5e4e758a4164004e56fffa0108>\n"           "<45bc4a2d33af487cf0757290bf5b9617> \n"           "] \n"           ">> \n"           ">> \n"           "endobj\n"           "trailer\n"           "<< /Root 1 0 R >>\n"           "%%EOF\n"           "";   return i == 3;}// End writeFile// processLine /////////////////////////////////////////////////////////////////void processLine(ifstream &fin){   while(fin.get() != int('\n'))   {   }// End while}// End processLine// END OF CODE


I can give some sample data... if that would help... but yeah sry no actual data.
"Y","555-55-5555","Andy J. Roberts","Address","City","State","Zip","Date","Proceeds","PropAdd1","PropAdd2""Y","555555555","Bobby Smith","Address","City","State","Zip","Date","Proceeds","PropAdd1","PropAdd2""N","555 55 5555","Kenny","1145 South Park Ave.","South Park","IN","78041","041712","40000.00","1145 South Park Ave.","South Park, IN 78041""Y","555 55 5555","Kenny","1145 South Park Ave.","South Park","IN","78041","041712","40000.00","1145 South Park Ave.","South Park, IN 78041"

Pretend that each entry is all on one line. It is in the txt file. The file ends with a blank line.

btw... kinda a noob on forums... how do you get the code to appear in a scrollable text box?

I guess I need to get use to std::string. I just got through with my first and second c++ classes in college and the professor refused to let us use string, or vector, or anything "easy" like that. The other c++ class was graphics so it was basic coding. Time to read some documentation i guess.
Ok, I'll have a look.

Quote:btw... kinda a noob on forums... how do you get the code to appear in a scrollable text box?


GDNet Forums FAQ
"Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it." — Brian W. Kernighan
Here, all fixed up. And plenty for you to learn. [grin]

// File:             // Author:           David Culpepper// Created:          05/09/05// Last Modified:    05/11/05 by Fruny// DESCRIPTION //////////////////////////////////////////////////////////////////***/// END DESCRIPTION /////////////////////////////////////////////////////////////// INCLUDES ////////////////////////////////////////////////////////////////////#include <iostream>#include <fstream>#include <sstream>#include <iomanip>#include <string>#include <limits>using namespace std;// FUNCTION PROTOTYPES /////////////////////////////////////////////////////////void   writeRecord(int, ostream &, istream &);int    writeFile(ostream &, istream &);string readField(istream &);// main ////////////////////////////////////////////////////////////////////////int main(){      ifstream fin("Z:\\1099\\1099.txt");      int outFileNumber = 1;        while(!fin.eof())   {      stringstream outFileNameStream;      outFileNameStream << "Z:\\1099\\Forms\\1099-"                         << setfill('0') << setw(3) << outFileNumber                         << ".fdf";      ofstream fout( outFileNameStream.str().c_str() );      int numRecords = writeFile(fout,fin);      cout << outFileNameStream.str() << " has " << numRecords << " records" << endl;                                    ++outFileNumber;   }      return 0;}// End main// writeRecord /////////////////////////////////////////////////////////////////void writeRecord(int recordNumber, ostream &os, istream &is){   os << "<< /V (George Culpepper\\rPO Box 1184\\rMeridian, MS 39302\\r601-693-2055)/T (Edit1a)>> \n"      << "<< /V (64-0740444)/T (Edit" << recordNumber << "b)>> " << endl;      for(char fieldCode = 'c'; fieldCode < 'l'; ++fieldCode)   {      switch(fieldCode)      {      case 'j':         os << "<< /V (" << readField(is) << "\\r" << readField(is)            << ")/T (Edit" << recordNumber << fieldCode << ")>>" << endl;         break;              default:                  os << "<< /V (" << readField(is)             << ")/T (Edit" << recordNumber << fieldCode << ")>>" << endl;         break;            }   }// End for   }// End writeRecordconst string fdfHeader =   "%FDF-1.2 \n"                           "%???? \n"                           "1 0 obj \n"                           "<< \n"                           "/FDF << /Fields [ ";                  const string fdfTrailer =  "<< /V (Filer's Name, Address, Phone)/T (Info 1)>> \n"                           "<< /V (Property Address)/T (Info 10)>> \n"                           "<< /V (Buyer's Part of Tax)/T (Info 11)>> \n"                           "<< /V (Filer's ID)/T (Info 2)>> \n"                           "<< /V (Transferor's ID)/T (Info 3)>> \n"                           "<< /V (Transferor's Name)/T (Info 4)>> \n"                           "<< /V (Transferor's Address)/T (Info 5)>> \n"                           "<< /V (City, State, Zip)/T (Info 6)>> \n"                           "<< /V (Account Number \\(Optional\\))/T (Info 7)>> \n"                           "<< /V (Closing Date)/T (Info 8)>> \n"                           "<< /V (Proceeds)/T (Info 9)>> \n"                           "] \n"                           "/F (/VIVIANCOMPUTER/DOCUMENTS/Active/Office Administration/Forms/1099-S Form\\.pdf)\n"                           "/ID [ \n"                           "<28bf4e5e4e758a4164004e56fffa0108>\n"                           "<45bc4a2d33af487cf0757290bf5b9617> \n"                           "] \n"                           ">> \n"                           ">> \n"                           "endobj\n"                           "trailer\n"                           "<< /Root 1 0 R >>\n"                           "%%EOF\n"                           "";// writeFile ///////////////////////////////////////////////////////////////////int writeFile(ostream &os, istream &is){   const int numRecordsPerFile = 3;   int numWrittenRecords = 0;      os << fdfHeader << endl;      while( !is.eof() && numWrittenRecords != numRecordsPerFile != 3 )   {      string line;      getline(is, line);            stringstream lineStream(line);            string field = readField(lineStream);            if (field == "Y")      {         ++numWrittenRecords;         writeRecord(numWrittenRecords, os, lineStream);      }   }      os << fdfTrailer;      return numWrittenRecords;   }// End writeFilestring readField(istream &is){   // Ignore all the characters up to the first encountered double quote   is.ignore(numeric_limits<int>::max(), '"');      // Read all the characters up to the first encountered double quote   string field;   getline(is, field, '"');      return field;   }// END OF CODE


Note that the functions all take istream and ostream parameters instead of ifstream and ofstream. Unless the functions are actually manipulating the streams as files (i.e. opening/closing them), it is better to use the base class, since it allows you to pass in other types of streams. readField can read from a stringstream as well as from an ifstream or even from cin (which is an istream) if you want to. Same for writing - you could write to cout (e.g. for debugging purposes) instead of sending the data to a file.

If you have questions, do ask.
"Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it." — Brian W. Kernighan
Minor revision to the code... I needed to concatenate a couple of fields instead of creating a new line.
// File:             // Author:           David Culpepper// Created:          05/09/05// Last Modified:    05/11/05 by Fruny// DESCRIPTION //////////////////////////////////////////////////////////////////***/// END DESCRIPTION /////////////////////////////////////////////////////////////// INCLUDES ////////////////////////////////////////////////////////////////////#include <iostream>#include <fstream>#include <sstream>#include <iomanip>#include <string>#include <limits>using namespace std;// FUNCTION PROTOTYPES /////////////////////////////////////////////////////////void   writeRecord(int, ostream &, istream &);int    writeFile(ostream &, istream &);string readField(istream &);// main ////////////////////////////////////////////////////////////////////////int main(){      ifstream fin("Z:\\1099\\1099Test.txt");      int outFileNumber = 1;        while(!fin.eof())   {      stringstream outFileNameStream;      outFileNameStream << "Z:\\1099\\Forms\\1099-"                         << setfill('0') << setw(3) << outFileNumber                         << ".fdf";      ofstream fout( outFileNameStream.str().c_str() );      int numRecords = writeFile(fout,fin);      cout << outFileNameStream.str() << " has " << numRecords << " records" << endl;                                    ++outFileNumber;   }      return 0;}// End main// writeRecord /////////////////////////////////////////////////////////////////void writeRecord(int recordNumber, ostream &os, istream &is){   os << "<< /V (George Culpepper\\rPO Box 1184\\rMeridian, MS 39302\\r601-693-2055)/T (Edit1a)>> \n"      << "<< /V (64-0740444)/T (Edit" << recordNumber << "b)>> " << endl;      for(char fieldCode = 'c'; fieldCode < 'k'; ++fieldCode)   {      switch(fieldCode)      {      case 'j':         os << "<< /V (" << readField(is) << "\\r" << readField(is)            << ")/T (Edit" << recordNumber << fieldCode << ")>>" << endl;         break;      case 'f':         os << "<< /V (" << readField(is) << ", " << readField(is)            << "   " << readField(is) << ")/T (Edit" << recordNumber             << fieldCode << ")>>" << endl;         break;      default:                  os << "<< /V (" << readField(is)             << ")/T (Edit" << recordNumber << fieldCode << ")>>" << endl;         break;            }   }// End for   }// End writeRecordconst string fdfHeader =   "%FDF-1.2 \n"                           "%???? \n"                           "1 0 obj \n"                           "<< \n"                           "/FDF << /Fields [ ";                  const string fdfTrailer =  "<< /V (Filer's Name, Address, Phone)/T (Info 1)>> \n"                           "<< /V (Property Address)/T (Info 10)>> \n"                           "<< /V (Buyer's Part of Tax)/T (Info 11)>> \n"                           "<< /V (Filer's ID)/T (Info 2)>> \n"                           "<< /V (Transferor's ID)/T (Info 3)>> \n"                           "<< /V (Transferor's Name)/T (Info 4)>> \n"                           "<< /V (Transferor's Address)/T (Info 5)>> \n"                           "<< /V (City, State, Zip)/T (Info 6)>> \n"                           "<< /V (Account Number \\(Optional\\))/T (Info 7)>> \n"                           "<< /V (Closing Date)/T (Info 8)>> \n"                           "<< /V (Proceeds)/T (Info 9)>> \n"                           "] \n"                           "/F (/VIVIANCOMPUTER/DOCUMENTS/Active/Office Administration/Forms/1099-S Form\\.pdf)\n"                           "/ID [ \n"                           "<28bf4e5e4e758a4164004e56fffa0108>\n"                           "<45bc4a2d33af487cf0757290bf5b9617> \n"                           "] \n"                           ">> \n"                           ">> \n"                           "endobj\n"                           "trailer\n"                           "<< /Root 1 0 R >>\n"                           "%%EOF\n"                           "";// writeFile ///////////////////////////////////////////////////////////////////int writeFile(ostream &os, istream &is){   const int numRecordsPerFile = 3;   int numWrittenRecords = 0;      os << fdfHeader << endl;      while( !is.eof() && numWrittenRecords != numRecordsPerFile != 3 )   {      string line;      getline(is, line);            stringstream lineStream(line);            string field = readField(lineStream);            if (field == "Y")      {         ++numWrittenRecords;         writeRecord(numWrittenRecords, os, lineStream);      }   }      os << fdfTrailer;      return numWrittenRecords;   }// End writeFilestring readField(istream &is){   // Ignore all the characters up to the first encountered double quote   is.ignore(numeric_limits<int>::max(), '"');      // Read all the characters up to the first encountered double quote   string field;   getline(is, field, '"');      return field;   }// END OF CODE


When I concatenate these, it is doing it in reverse order. In other words if i want to make it city, state, zip... it is putting in zip, state, city. Also the other special case is only showing one line... Sry for being a pain. I'll have a few questions about some of the string stuff and the readField... but just trying to get working before 5.
One thing I thought about this is the basics of file IO - load the data in and get it into a format that is easy to work with. So here is what I would do to load the data in:

#include <fstream>#include <iostream>#include <string>#include <vector>// Added the ignore1 just for this example ;)std::vector<std::string> tokenize( const std::string &str, const std::string &delim, const std::string &ignore1 = "" ){	using namespace std;	vector<string> tokens;	size_t p0 = 0, p1 = string::npos;	while(p0 != string::npos)	{		p1 = str.find_first_of(delim, p0);		if(p1 != p0)		{			string token = str.substr(p0, p1 - p0);			if( token != ignore1 )				tokens.push_back(token);		}		p0 = str.find_first_not_of(delim, p1);	}	return tokens;}int main( int argc, char*argv[] ){	std::ifstream IF("data.txt");	std::string curLine;	int recordCount = 0;	while( std::getline( IF, curLine ) )	{		// Tokenize the line of data. 'Ignore' the delimiting commas as well		// (Note that I could not use the comma with the quote as a delim because of this line: "South Park, IN 78041" )		std::vector<std::string> elements = tokenize( curLine, "\"", ",");		// Just show the contents of each element		for( int x=0;x<elements.size();x++)		{			std::cout << elements[x] << std::endl;		}		std::cout << std::endl << std::endl;		recordCount++;	}	IF.close();	return 0;}


Well you will need to add in the processing specific code. I still can not figure out everything that is going on with processing, even with Fruny's example [smile] I see it works, but I don't get what's being done. Just shows how much I know about taxes [lol].

Anyways with an input file of this:

"Y","555-55-5555","Andy J. Roberts","Address","City","State","Zip","Date","Proceeds","PropAdd1","PropAdd2""Y","555555555","Bobby Smith","Address","City","State","Zip","Date","Proceeds","PropAdd1","PropAdd2""N","555 55 5555","Kenny","1145 South Park Ave.","South Park","IN","78041","041712","40000.00","1145 South Park Ave.","South Park, IN 78041""Y","555 55 5555","Kenny","1145 South Park Ave.","South Park","IN","78041","041712","40000.00","1145 South Park Ave.","South Park, IN 78041"


You will get an output of:
Y555-55-5555Andy J. RobertsAddressCityStateZipDateProceedsPropAdd1PropAdd2Y555555555Bobby SmithAddressCityStateZipDateProceedsPropAdd1PropAdd2N555 55 5555Kenny1145 South Park Ave.South ParkIN7804104171240000.001145 South Park Ave.South Park, IN 78041Y555 55 5555Kenny1145 South Park Ave.South ParkIN7804104171240000.001145 South Park Ave.South Park, IN 78041Press any key to continue


Now each line represents the index in the vector. So if you want to process the 'Y' or 'N' your processing code will look like this:
// If you don't want to add this then continue onif( elements[0] == "N" || elements[0] == "n" )	continue;


Y                         // elements[0]555 55 5555               // elements[1]Kenny                     // elements[2]1145 South Park Ave.      // elements[3]South ParkIN7804104171240000.001145 South Park Ave.South Park, IN 78041      // elements[10]


Now that will work on a per element basics, it just reads in the input file, then you do all the processing with that line, but you can work with it on a field by field basics. That is assuming your input line contains the same formatted input line, ie each line has 11 total items.

Not sure if you want to use something like this if you gotta get done by 5pm, but just an idea [wink]
Figured out why I was getting reverse locations. Forgot that insertion operator parsed from right to left on the line. So my last call to the function was being called first. At least that's what I think was happening. Note to self... try to stay down to one function call per line of code. I have one bug left. Otherwise it works perfect. No more sitting at a typewriter for me! For the second field of data it cuts off the data after six letters... And i just fixed it. Error in my pdf. Was limiting that field to six characters. Thanks a lot Fruny. Now for the tough task... Making the coworkers keep the database accurate. I'm gonna read through some documentation on strings and such then post any questions i have regarding your changes Fruny. Thanks again.
So Drew... Are you suggesting that I format it to that and use a vector of vectors. Then use a double for loop to process everything. If so, I agree that would have made this a ton easier. Just need to learn to think about a problem more abstractly I guess.
Quote:Original post by NightStalker
So Drew... Are you suggesting that I format it to that and use a vector of vectors. Then use a double for loop to process everything. If so, I agree that would have made this a ton easier. Just need to learn to think about a problem more abstractly I guess.


Well, I assumed you had a good reason for using that input format. I essentially reworked the code from that, but, as Drew indicated, a well-behaved input makes parsing easier. That really is the important bit when designing file formats.
"Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it." — Brian W. Kernighan

This topic is closed to new replies.

Advertisement