Sign in to follow this  

[Solved] An easy way to read and process a text file

This topic is 3812 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

Hello. Recently I was tying to make something in C++ to read a simple text file that consists of a bunch of variables into memory. I have successfully imported the file line by line into a vector<string> array. Now the first few lines are the header of the text file, and the rest of the data that follows are numbers, mostly floats. Now I'm pretty much stuck with my own C++ knowledge because I can't get a good idea how to read all those variables into actual memory. The thing is that the number of rows and columns are variable. Rows are easy for all I have to do is use a loop through the vector<string>, but what I can't solve is how to read the actual data from each line into memory. I have poked around a bit and found some ways to read static formatted data, but none that can read variable/dynamic formatted data. There is a line in the header of the text file that defines the number of records/lines/rows and columns in the data section. This line always consists of 4 integers and I have used the following thing to read the forst two: sscanf(datafile[1].c_str(), "%i %i %*i %*i", &records, &variables) The data section of the text looks like this: -1.23 -1.23 6.05 -0.66 -0.66 0.94 3.99 66 4 -2.46 -1.23 13.62 -1.32 -0.66 1.38 3.97 66 5 -1.9 -1.23 9.8 -1.03 -0.66 1.12 3.93 66 6 (continues)... Each line/row basically has a variable number of floats followed by a char and an int, so the format would be float[], char, int. Here I obviously can't use sscanf function because it requires a number of parameters (variable references) - a number that I do not know at compile-time. Is there some way, perhaps a modified/alternative version of sscanf to use, or maybe even some other method for reading this kind of data? Thanks for any suggestions. [Edited by - Blednik on July 12, 2007 12:54:29 AM]

Share this post


Link to post
Share on other sites
Forget any functions with "scan" in the name.

All you need to do is use the operator>> of the stream.
"Wait, what stream?" you say. "I already stored the line of the file into a string!"

Well, that's actually a good idea - you'll make the process more robust that way. But, C++ provides a way to treat strings as streams - using what is appropriately named a stringstream. This is an object that provides exactly the same interface as a file stream (or for that matter, one of the console streams), but takes its data from (or writes to) a std::string.

Also, there's no reason to accumulate the strings in a vector. Just use each one as you read it.

Of course, you still need some kind of structure to represent the line. Since we don't know ahead of time how many floats there are, the natural thing to do is to use a vector *here*. (What's this float[] nonsense? :) ) We'll make a struct that wraps up a line's contents. Also, it looks like you don't have a char in each line - in the sense of a character; you have an integer that simply happens to be in the range represented by characters. This will require a bit of trickery in our loading code, because operator>> assumes that you want to read an actual text character when you supply a char variable as the target.


struct lineData {
std::vector<float> myFloats;
char myChar;
int myInt;
};


And a function that reads a line and produces lineData:


// We'll need this include in addition to what you already have:
#include <sstream>

// We'll accept a lineData wherein the vector has been sized appropriately,
// and populate it with data from the line (overwriting whatever might have
// been in there before). We'll return whether or not we were successful in
// interpreting the line.
bool parseLine(const std::string& line, lineData& result) {
std::istringstream iss(line); // we specify that we only want to read from it.
float f;
for (int i = 0; i < result.myFloats.size(); ++i) {
iss >> result.myFloats[i];
}
int c;
iss >> c;
result.myChar = static_cast<char>(c);
iss >> result.myInt;
return iss; // in a boolean context, an input stream is interpreted as true if
// all reads so far have succeeded.
}


And then we can create a vector of those structs:


int main() {
std::ifstream data("foo.txt");
int floatsPerLine;
// read all the header info and set floatsPerLine
string line;
// Again, in a boolean context etc. etc. That's how we can easily read
// "each line" in a file: read until the operation "read a line" fails.
// Don't check .eof()! This won't return true until after an operation already
// failed. This typically results in looping one extra time, parsing the
// last line of the file twice.
std::vector<lineData> results;
lineData tmp;
tmp.myFloats.resize(floatsPerLine);
while (std::getline(data, line)) {
if (parseLine(line, tmp)) {
results.push_back(tmp);
} else {
std::cerr << "oh noes, the file is corrupt" << std::endl;
return; // of course, we should do a better job of handling this :)
}
}
}

Share this post


Link to post
Share on other sites
I would start with a boost::spirit expression representing what you need. For instance,
rule<> line = 
(strict_real_p % (*space_p)) >>
*space_p >>
anychar_p >>
*space_p >>
int_p;


Then, I would create some functors for the semantic actions:

std::vector<float> floats;
char character;
int integer;

rule<> line =
(strict_real_p[push_back_a(floats)] % (*space_p)) >>
*space_p >>
anychar_p[assign_a(character)] >>
*space_p >>
int_p[assign_a(integer)];



And simply apply the result...

Share this post


Link to post
Share on other sites

This topic is 3812 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this