Jump to content

  • Log In with Google      Sign In   
  • Create Account


#ActualServant of the Lord

Posted 25 August 2012 - 09:16 PM

When I don't care too much about speed, I handle that type of thing in distinct re-usable functions (from a lack of deep familiarity with proper stream usage).

In your case, I might do it like this:
//Load the file into a single string.
std::string fileContents = LoadFileAsString("text.txt");


//Remove comments.
fileContents = String::RemoveComments(fileContents, "//");
fileContents = String::RemoveComments(fileContents, "/*", "*/");

//Seperate the string into lines; this also eliminates empty lines via a callback function.
StringList lines = String::Seperate(fileContents, '\n', RemoveEmptyLines);
if(lines.empty())
{
	 //The file (barring empty lines and comments) is empty!
	 //Handle the error.
}

//Get the first line.
int line = 0;
int numberOfFiles = StringToInt(lines[line++]); //You can also use lines.at() and catch the exception for malformed files.

if(lines.size() > (line + numberOfFiles))
{
	 //Malformed file. Not as many lines as promised by 'numberOfFiles'.
	 //Handle the error.
}

//Find the iterators to the range of files we want.
auto beginningOfFiles = (lines.begin() + line);
auto endOfFiles = (beginningOfFiles + numberOfFiles);

//Copy the range of elements to their own vector.
StringList files;
std::copy(beginningOfFiles, endOfFiles, files.begin());

//Continue processing the file...
line += numberOfFiles;

//...more file processing...

Most of those functions exist already exist in my code base through repeated needs, though RemoveComments() does not (I've never had to deal with multi-line comments before, but that's easy enough to solve - and once solved, becomes a permanent tool in your library, ready to use).

String::Seperate() function. Currently just takes a boolean to check for empty lines, but after writing this post I realize I need a callback function to check for non-empty but whitespace-filled lines. I also have it on my todo list (but as low priority) to make a version for if 'divider' is a single char (which would be alot faster).
StringList is just a typedef for std::vector<std::string> since I use it so often.
	//Divides up a string into multiple segments seperated by 'divider', and returns each segment in a StringList.
	//If any segment is empty, and if 'ignoreEmptySegments' is true, the empty segments are not added to the StringList.
	StringList Seperate(const std::string &str, const std::string &divider, bool ignoreEmptySegments)
	{
		StringList stringList;
		//Check for empty string.
		if(str.empty() || divider.empty())
			return stringList;
		size_t start = 0;
		size_t end = str.find(divider, start);
		//Keep looping, as long as there are more segments.
		while(end != std::string::npos)
		{
			std::string subString = str.substr(start, end - start);
			if(subString.size() > 0 || ignoreEmptySegments == false)
			{
				stringList.push_back(subString);
			}
			start = end + 1;
			end = str.find(divider, start);
		}
		//Get the final (or the only) segment.
		std::string subString = str.substr(start, str.size() - start);
		if(subString.size() > 0 || ignoreEmptySegments == false)
		{
			stringList.push_back(subString);
		}
		return stringList;
	}

LoadFileAsString() function.
//Loads the entire file, newlines and all, and returns its contents as a string.
//Returns an empty string if the file doesn't exist.
std::string LoadFileAsString(const std::string &filename, ReadMode::Enum readMode)
{
	//Open the file for reading.
	std::ifstream file;
	if(readMode == ReadMode::Binary)
		file.open(filename.c_str(), std::ios_base::in | std::ios_base::binary);
	else
		file.open(filename.c_str(), std::ios_base::in);
	if(!file)
	{
		Log::Message(MSG_SOURCE("FileFunctions", Log::Severity::Error)) << "Failed to load '" << Log_HighlightCyan(GetFilenameFromPath(filename)) << "' at " << Log_DisplayPath(filename)
																		<< "\nDo I have sufficient privileges? Does the file even exist?" << Log::FlushStream;
		return "";
	}
	//Read the file into the stringStream.
	std::ostringstream stringStream;
	stringStream << file.rdbuf();
	//Convert the stringStream into a regular string.
	return stringStream.str();
}


Observant programmers will notice that most of my code suffers from Shmuel Schlemiel the Painter algorithms. Many of the functions could be individually optimized, and the use of them together could further be optimized into a stand-alone function, like a "LoadFileAsStringList()" since I frequently go from File to String to StringList (easy enough to add).
I typically prefer clean code that works before I start optimizing code, and I don't do the actual optimizations until I find them necessary (otherwise I find myself wasting inordinate amounts of time of things that don't matter much). With file parsing, when I occasionally need files parsed fast, I load it as binary and don't bother with text parsing anyway.

#2Servant of the Lord

Posted 25 August 2012 - 09:15 PM

When I don't care too much about speed, I handle that type of thing in distinct re-usable functions.

In your case, I might do it like this:
//Load the file into a single string.
std::string fileContents = LoadFileAsString("text.txt");


//Remove comments.
fileContents = String::RemoveComments(fileContents, "//");
fileContents = String::RemoveComments(fileContents, "/*", "*/");

//Seperate the string into lines; this also eliminates empty lines via a callback function.
StringList lines = String::Seperate(fileContents, '\n', RemoveEmptyLines);
if(lines.empty())
{
     //The file (barring empty lines and comments) is empty!
     //Handle the error.
}

//Get the first line.
int line = 0;
int numberOfFiles = StringToInt(lines[line++]); //You can also use lines.at() and catch the exception for malformed files.

if(lines.size() > (line + numberOfFiles))
{
	 //Malformed file. Not as many lines as promised by 'numberOfFiles'.
	 //Handle the error.
}

//Find the iterators to the range of files we want.
auto beginningOfFiles = (lines.begin() + line);
auto endOfFiles = (beginningOfFiles + numberOfFiles);

//Copy the range of elements to their own vector.
StringList files;
std::copy(beginningOfFiles, endOfFiles, files.begin());

//Continue processing the file...
line += numberOfFiles;

//...more file processing...

Most of those functions exist already exist in my code base through repeated needs, though RemoveComments() does not (I've never had to deal with multi-line comments before, but that's easy enough to solve - and once solved, becomes a permanent tool in your library, ready to use).

String::Seperate() function. Currently just takes a boolean to check for empty lines, but after writing this post I realize I need a callback function to check for non-empty but whitespace-filled lines. I also have it on my todo list (but as low priority) to make a version for if 'divider' is a single char (which would be alot faster).
StringList is just a typedef for std::vector<std::string> since I use it so often.
	//Divides up a string into multiple segments seperated by 'divider', and returns each segment in a StringList.
	//If any segment is empty, and if 'ignoreEmptySegments' is true, the empty segments are not added to the StringList.
	StringList Seperate(const std::string &str, const std::string &divider, bool ignoreEmptySegments)
	{
		StringList stringList;
		//Check for empty string.
		if(str.empty() || divider.empty())
			return stringList;
		size_t start = 0;
		size_t end = str.find(divider, start);
		//Keep looping, as long as there are more segments.
		while(end != std::string::npos)
		{
			std::string subString = str.substr(start, end - start);
			if(subString.size() > 0 || ignoreEmptySegments == false)
			{
				stringList.push_back(subString);
			}
			start = end + 1;
			end = str.find(divider, start);
		}
		//Get the final (or the only) segment.
		std::string subString = str.substr(start, str.size() - start);
		if(subString.size() > 0 || ignoreEmptySegments == false)
		{
			stringList.push_back(subString);
		}
		return stringList;
	}

LoadFileAsString() function.
//Loads the entire file, newlines and all, and returns its contents as a string.
//Returns an empty string if the file doesn't exist.
std::string LoadFileAsString(const std::string &filename, ReadMode::Enum readMode)
{
	//Open the file for reading.
	std::ifstream file;
	if(readMode == ReadMode::Binary)
		file.open(filename.c_str(), std::ios_base::in | std::ios_base::binary);
	else
		file.open(filename.c_str(), std::ios_base::in);
	if(!file)
	{
		Log::Message(MSG_SOURCE("FileFunctions", Log::Severity::Error)) << "Failed to load '" << Log_HighlightCyan(GetFilenameFromPath(filename)) << "' at " << Log_DisplayPath(filename)
																		<< "\nDo I have sufficient privileges? Does the file even exist?" << Log::FlushStream;
		return "";
	}
	//Read the file into the stringStream.
	std::ostringstream stringStream;
	stringStream << file.rdbuf();
	//Convert the stringStream into a regular string.
	return stringStream.str();
}


Observant programmers will notice that most of my code suffers from Shmuel Schlemiel the Painter algorithms. Many of the functions could be individually optimized, and the use of them together could further be optimized into a stand-alone function, like a "LoadFileAsStringList()" since I frequently go from File to String to StringList (easy enough to add).
I typically prefer clean code that works before I start optimizing code, and I don't do the actual optimizations until I find them necessary (otherwise I find myself wasting inordinate amounts of time of things that don't matter much). With file parsing, when I occasionally need files parsed fast, I load it as binary and don't bother with text parsing anyway.

#1Servant of the Lord

Posted 25 August 2012 - 09:10 PM

When I don't care too much about speed, I handle that type of thing in distinct re-usable functions.

In your case, I might do it like this:
//Load the file into a single string.
std::string fileContents = LoadFileAsString("text.txt");


//Remove comments.
fileContents = String::RemoveComments(fileContents, "//");
fileContents = String::RemoveComments(fileContents, "/*", "*/");

//Seperate the string into lines; this also eliminates empty lines via a callback function.
StringList lines = String::Seperate(fileContents, '\n', RemoveEmptyLines);


//Get the first line.
int line = 0;
int numberOfFiles = StringToInt(lines.at(line++)); //You can catch the exception for malformed files.

assert(lines.size() > (line + numberOfFiles)); //I'd actually have error handling, not an assert here, but for the sake of succinctness...

//Find the iterators to the range of files we want.
auto beginningOfFiles = (lines.begin() + line);
auto endOfFiles = (beginningOfFiles + numberOfFiles);

//Copy the range of elements to their own vector.
StringList files;
std::copy(beginningOfFiles, endOfFiles, files.begin());

//Continue processing the file...
line += numberOfFiles;

//...more file processing...

Most of those functions exist already exist in my code base through repeated needs, though RemoveComments() does not (I've never had to deal with multi-line comments before, but that's easy enough to solve - and once solved, becomes a permanent tool in your library, ready to use).

String::Seperate() function. Currently just takes a boolean to check for empty lines, but after writing this post I realize I need a callback function to check for non-empty but whitespace-filled lines. I also have it on my todo list (but as low priority) to make a version for if 'divider' is a single char (which would be alot faster).
StringList is just a typedef for std::vector<std::string> since I use it so often.
    //Divides up a string into multiple segments seperated by 'divider', and returns each segment in a StringList.
    //If any segment is empty, and if 'ignoreEmptySegments' is true, the empty segments are not added to the StringList.
    StringList Seperate(const std::string &str, const std::string &divider, bool ignoreEmptySegments)
    {
	    StringList stringList;
	    //Check for empty string.
	    if(str.empty() || divider.empty())
		    return stringList;
	    size_t start = 0;
	    size_t end = str.find(divider, start);
	    //Keep looping, as long as there are more segments.
	    while(end != std::string::npos)
	    {
		    std::string subString = str.substr(start, end - start);
		    if(subString.size() > 0 || ignoreEmptySegments == false)
		    {
			    stringList.push_back(subString);
		    }
		    start = end + 1;
		    end = str.find(divider, start);
	    }
	    //Get the final (or the only) segment.
	    std::string subString = str.substr(start, str.size() - start);
	    if(subString.size() > 0 || ignoreEmptySegments == false)
	    {
		    stringList.push_back(subString);
	    }
	    return stringList;
    }

LoadFileAsString() function.
//Loads the entire file, newlines and all, and returns its contents as a string.
//Returns an empty string if the file doesn't exist.
std::string LoadFileAsString(const std::string &filename, ReadMode::Enum readMode)
{
    //Open the file for reading.
    std::ifstream file;
    if(readMode == ReadMode::Binary)
	    file.open(filename.c_str(), std::ios_base::in | std::ios_base::binary);
    else
	    file.open(filename.c_str(), std::ios_base::in);
    if(!file)
    {
	    Log::Message(MSG_SOURCE("FileFunctions", Log::Severity::Error)) << "Failed to load '" << Log_HighlightCyan(GetFilenameFromPath(filename)) << "' at " << Log_DisplayPath(filename)
																	    << "\nDo I have sufficient privileges? Does the file even exist?" << Log::FlushStream;
	    return "";
    }
    //Read the file into the stringStream.
    std::ostringstream stringStream;
    stringStream << file.rdbuf();
    //Convert the stringStream into a regular string.
    return stringStream.str();
}


Observant programmers will notice that most of my code suffers from Shmuel Schlemiel the Painter algorithms. Many of the functions could be individually optimized, and the use of them together could further be optimized into a stand-alone function, like a "LoadFileAsStringList()" since I frequently go from File to String to StringList (easy enough to add).
I typically prefer clean code that works before I start optimizing code, and I don't do the actual optimizations until I find them necessary (otherwise I find myself wasting inordinate amounts of time of things that don't matter much). With file parsing, when I occasionally need files parsed fast, I load it as binary and don't bother with text parsing anyway.

PARTNERS