• Advertisement
  • entries
    1212
  • comments
    1738
  • views
    1143657

STL string tokenizer

Sign in to follow this  

96 views

STL is without a doubt a Good Thing. Much like C++ ten years ago, though, you've gotta change your thinking about 90 degrees to get the most out of it. Also like C++, its harshest critics are those who don't know enough about it. Anyway, I'd been looking for a decent string tokenizer in STL. I'd looked around a bit, thinking that there was probably two lines of code that could do it. I couldn't find those two lines, though, so I wrote my own. It's a bit different from other tokenizers I've seen in that it's a vector of strings. It tokenizes the whole thing in the constructor, then you can use the standard vector functions to grab out the results. It's about as simple a tokenizer as I've ever seen, so I'm gonna share it.

Note that the tokenize function itself is borrowed from someone's code I found on the web. If it's yours and you want credit, lemme know.



class StringTokenizer : public vector
{
public:
StringTokenizer(const string &rStr, const string &rDelimiters = " ,\n");
};

StringTokenizer::StringTokenizer(const string &rStr, const string &rDelimiters)
{
string::size_type lastPos(rStr.find_first_not_of(rDelimiters, 0));
string::size_type pos(rStr.find_first_of(rDelimiters, lastPos));
while (string::npos != pos || string::npos != lastPos)
{
push_back(rStr.substr(lastPos, pos - lastPos));
lastPos = rStr.find_first_not_of(rDelimiters, pos);
pos = rStr.find_first_of(rDelimiters, lastPos);
}
}
Sign in to follow this  


1 Comment


Recommended Comments

Guest Anonymous Poster

Posted

Here is an even better way to tokenize a string!

http://www.kuro5hin.org/comments/2005/11/23/192825/08/5?mode=alone;

Share this comment


Link to comment

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

  • Advertisement