Jump to content
Site Stability Read more... ×
  • Advertisement
  • entries
  • comments
  • views

STL string tokenizer

Sign in to follow this  


STL is without a doubt a Good Thing. Much like C++ ten years ago, though, you've gotta change your thinking about 90 degrees to get the most out of it. Also like C++, its harshest critics are those who don't know enough about it. Anyway, I'd been looking for a decent string tokenizer in STL. I'd looked around a bit, thinking that there was probably two lines of code that could do it. I couldn't find those two lines, though, so I wrote my own. It's a bit different from other tokenizers I've seen in that it's a vector of strings. It tokenizes the whole thing in the constructor, then you can use the standard vector functions to grab out the results. It's about as simple a tokenizer as I've ever seen, so I'm gonna share it.

Note that the tokenize function itself is borrowed from someone's code I found on the web. If it's yours and you want credit, lemme know.

class StringTokenizer : public vector
StringTokenizer(const string &rStr, const string &rDelimiters = " ,\n");

StringTokenizer::StringTokenizer(const string &rStr, const string &rDelimiters)
string::size_type lastPos(rStr.find_first_not_of(rDelimiters, 0));
string::size_type pos(rStr.find_first_of(rDelimiters, lastPos));
while (string::npos != pos || string::npos != lastPos)
push_back(rStr.substr(lastPos, pos - lastPos));
lastPos = rStr.find_first_not_of(rDelimiters, pos);
pos = rStr.find_first_of(rDelimiters, lastPos);
Sign in to follow this  

1 Comment

Recommended Comments

Guest Anonymous Poster


Here is an even better way to tokenize a string!


Share this comment

Link to comment

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
  • Advertisement

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

GameDev.net is your game development community. Create an account for your GameDev Portfolio and participate in the largest developer community in the games industry.

Sign me up!