• 11
• 9
• 11
• 9
• 11
• entries
1212
1738
• views
1143657

# STL string tokenizer

96 views

STL is without a doubt a Good Thing. Much like C++ ten years ago, though, you've gotta change your thinking about 90 degrees to get the most out of it. Also like C++, its harshest critics are those who don't know enough about it. Anyway, I'd been looking for a decent string tokenizer in STL. I'd looked around a bit, thinking that there was probably two lines of code that could do it. I couldn't find those two lines, though, so I wrote my own. It's a bit different from other tokenizers I've seen in that it's a vector of strings. It tokenizes the whole thing in the constructor, then you can use the standard vector functions to grab out the results. It's about as simple a tokenizer as I've ever seen, so I'm gonna share it.

Note that the tokenize function itself is borrowed from someone's code I found on the web. If it's yours and you want credit, lemme know.

class StringTokenizer : public vector{	public:		StringTokenizer(const string &rStr, const string &rDelimiters = " ,\n");};StringTokenizer::StringTokenizer(const string &rStr, const string &rDelimiters){	string::size_type lastPos(rStr.find_first_not_of(rDelimiters, 0));	string::size_type pos(rStr.find_first_of(rDelimiters, lastPos));	while (string::npos != pos || string::npos != lastPos)	{		push_back(rStr.substr(lastPos, pos - lastPos));		lastPos = rStr.find_first_not_of(rDelimiters, pos);		pos = rStr.find_first_of(rDelimiters, lastPos);	}}

## 1 Comment

Here is an even better way to tokenize a string!