• Advertisement
  • entries
  • comments
  • views

STL string tokenizer

Sign in to follow this  


STL is without a doubt a Good Thing. Much like C++ ten years ago, though, you've gotta change your thinking about 90 degrees to get the most out of it. Also like C++, its harshest critics are those who don't know enough about it. Anyway, I'd been looking for a decent string tokenizer in STL. I'd looked around a bit, thinking that there was probably two lines of code that could do it. I couldn't find those two lines, though, so I wrote my own. It's a bit different from other tokenizers I've seen in that it's a vector of strings. It tokenizes the whole thing in the constructor, then you can use the standard vector functions to grab out the results. It's about as simple a tokenizer as I've ever seen, so I'm gonna share it.

Note that the tokenize function itself is borrowed from someone's code I found on the web. If it's yours and you want credit, lemme know.

class StringTokenizer : public vector
StringTokenizer(const string &rStr, const string &rDelimiters = " ,\n");

StringTokenizer::StringTokenizer(const string &rStr, const string &rDelimiters)
string::size_type lastPos(rStr.find_first_not_of(rDelimiters, 0));
string::size_type pos(rStr.find_first_of(rDelimiters, lastPos));
while (string::npos != pos || string::npos != lastPos)
push_back(rStr.substr(lastPos, pos - lastPos));
lastPos = rStr.find_first_not_of(rDelimiters, pos);
pos = rStr.find_first_of(rDelimiters, lastPos);
Sign in to follow this  

1 Comment

Recommended Comments

Guest Anonymous Poster


Here is an even better way to tokenize a string!


Share this comment

Link to comment

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

  • Advertisement