void System::Tokenize(const std::string &string, std::vector<std::string> &result)
{
std::string temp;
//"/msg bob"
for (int i = 0; i < string.size(); ++i)
{
if (string == ' ')
{
result.push_back(temp);
temp.clear();
}
else
{
temp += string;
}
}
}
help with simple "tokenize" function
hi,
im not even sure if this is the correct terminology, but... im trying to break a string into pieces based on whitespace. so that "hello there" turns into 2 strings, "hello" and "there". heres the function:
its not working though.. can anyone help?
Looks right to me, but maybe I'm missing something obvious.
Are you clearing 'result' before passing it to the function? Are you sure that 'string' contains what you think it does?
Anyway, how is it not working?
Are you clearing 'result' before passing it to the function? Are you sure that 'string' contains what you think it does?
Anyway, how is it not working?
void tokenize(const std::string &string, std::vector<std::string> &result, char delim = ' ') { std::string::const_iterator pos(string.begin()); while(true) { for(; pos != string.end() && *pos == delim; ++pos) {} // Skip delimiters. if(pos == string.end()) return; std::string::const_iterator end(std::find(pos, string.end(), delim)); result.push_back(std::string(pos, end)); pos = end; } }
3rd time's the charm.
std::string::find_first_ofstd::string::find_first_not_ofstd::string::substr
That's all you need. To whit:
std::vector<std::string> tokenize(const std::string & str, const std::string & delimiters){ using namespace std; vector<string> tokens; size_t p0 = 0, p1 = string::npos; while(p0 != string::npos) { p1 = str.find_first_of(delim, p0); if(p1 != p0) { string token = str.substr(p0, p1 - p0); tokens.push_back(token); } p0 = str.find_first_not_of(delim, p1); } return tokens;}
Quote:Original post by Oluseyi
*** Source Snippet Removed ***
That's pretty darn nifty if I might say Oluseyi. Thanks! Works like a charm.
@smart_idiot: Linky
- Drew
[edit] Added link
[Edited by - Drew_Benton on February 22, 2005 11:15:25 PM]
As Oluseyi has shown, at first glance the primary issue is that you aren't returning a value. Remember that in your original code, 'temp' is scoped to the function being called. After the function returns, temp will go out of scope and essentially be deleted.
If you're creating an object with your function that is going to be used, make sure to return it instead of using the keyword 'void'.
Another possibility would be to set a parameter to take an argument by reference, thus editing the original argument passed, and remaining in the scope of the original function that created it.
If you're creating an object with your function that is going to be used, make sure to return it instead of using the keyword 'void'.
Another possibility would be to set a parameter to take an argument by reference, thus editing the original argument passed, and remaining in the scope of the original function that created it.
Quote:As Oluseyi has shown, at first glance the primary issue is that you aren't returning a value. Remember that in your original code, 'temp' is scoped to the function being called. After the function returns, temp will go out of scope and essentially be deleted.I'm curious as to what you mean. Are you talking about the OP's function? It looks to me like he's doing pretty much what you describe. The function takes as an argument a non-const reference to a std::vector<string>. temp is of course local, but on completion of each token is pushed onto the vector, where presumably a copy is made. The function returns void because there's nothing to return.
If you're creating an object with your function that is going to be used, make sure to return it instead of using the keyword 'void'.
Another possibility would be to set a parameter to take an argument by reference, thus editing the original argument passed, and remaining in the scope of the original function that created it.
If you were talking about something else, never mind.
Anyway, as the previous posts have shown, there are certainly other, and probably better, ways to parse a string. But I'm still curious as to why the OP's code doesn't work.
If you only need to tokenize on whitespace, I'd just use a stringstream:
vector<string> getWords(const string& input) { vector<string> result; stringstream ss(input); string word; // We exploit the default stream behaviour whereby reading into a string // variable reads one "word". while (ss >> word) result.push_back(word); return result;}
This topic is closed to new replies.
Advertisement
Popular Topics
Advertisement