const map<string, string> ParsePropertyString(string s)
{
using namespace boost;
using namespace std;
map<string, string> propertyMap;
vector<string> propertyList;
string a, b;
tokenizer<escaped_list_separator<char> > commaTokenizer(s);
for(tokenizer<escaped_list_separator<char> >::iterator i=commaTokenizer.begin(); i!=commaTokenizer.end();++i)
{
propertyList.push_back(*i);
}
char_separator<char> eqlsDelimiter(" =", "", drop_empty_tokens);
int end = propertyList.size();
for (int j = 0; j < end; j++)
{
tokenizer<char_separator<char> > eqlsTokenizer(propertyList.pop_back(), eqlsDelimiter);
a = *(eqlsTokenizer.begin());
b = *(eqlsTokenizer.end());
propertyMap.insert(a, b);
}
return propertyMap;
}
Help with boost::tokenizer
Author
Argh...I've spent over an hour trying to figure this out, but I just can't get it. I'll give you some background information before posting code.
I'm creating a resource manager class (very similar to the ClanLib resource manager). The manager takes a reads a script file with resource IDs and their properties. Part of the resource script would look like this:
CloudTexture (type = Texture, file = clouds.tga, filetype = targa);
Basically, the resource id comes first, then there is a list of properties inside parenthesis. Sounds simple...but my code is not working. I have to go right now, so I'll just dump it. Any help would be appreciated. The function should return a map where the keys are the property name and the values are the property values. The ()'s will be removed by the time this function is called.
EDIT: Changed source. What you saw before was a debugged-version, wouldn't work anyway. Basic idea the same, I just had some strings plugged in to try to see if that was the problem.
Doesn't matter anyway. The response from Fruny should clear it up. Thanks.
EDIT2: Response from Fruny didn't clear it up. Scroll down for newest source posting.
[edited by - Neosmyle on December 5, 2002 10:17:35 PM]
[edited by - Neosmyle on December 5, 2002 10:56:28 PM]
Have you even looked into the separator funtions? Of course it''s not going to work if you don''t specify what characters you want to use to separate your tokens.
Basically, though, you''ll need to use space, commas, and equal signs to separate your tokens. You can use one of the three recommended pre-made functions (there''s fourth, but it''s depreciated), or make your own. Either way, you''re going to need to look at the Tokenizer documentation more. Quick link:
http://www.boost.org/libs/tokenizer/index.htm
Basically, though, you''ll need to use space, commas, and equal signs to separate your tokens. You can use one of the three recommended pre-made functions (there''s fourth, but it''s depreciated), or make your own. Either way, you''re going to need to look at the Tokenizer documentation more. Quick link:
http://www.boost.org/libs/tokenizer/index.htm
b = *(eqlsTokenizer.end()); Boom !
You can''t dereference an end() iterator.
Additionally, tokenizer iterators are forward iterators, not random access, so you need to walk through them one by one ( itor = tok.begin(); a = *itor; ++itor; b = *itor; )
Finally, you can build your vector like this : vector<string> propertyList( commaTokenizer.begin(), commaTokenizer.end() );, unless your STL implementation is deficient.
The second tokenizer function could be escaped_list_separator<char>( "\\", "=", "\"" );
And, finally, propertyMap[ a ] = b; can work too.
Documents [ GDNet | MSDN | STL | OpenGL | Formats | RTFM | Asking Smart Questions ]
C++ Stuff [ MinGW | Loki | SDL | Boost. | STLport | FLTK | ACCU Recommended Books ]
You can''t dereference an end() iterator.
Additionally, tokenizer iterators are forward iterators, not random access, so you need to walk through them one by one ( itor = tok.begin(); a = *itor; ++itor; b = *itor; )
Finally, you can build your vector like this : vector<string> propertyList( commaTokenizer.begin(), commaTokenizer.end() );, unless your STL implementation is deficient.
The second tokenizer function could be escaped_list_separator<char>( "\\", "=", "\"" );
And, finally, propertyMap[ a ] = b; can work too.
Documents [ GDNet | MSDN | STL | OpenGL | Formats | RTFM | Asking Smart Questions ]
C++ Stuff [ MinGW | Loki | SDL | Boost. | STLport | FLTK | ACCU Recommended Books ]
Author
This is still giving me problems. I am no longer trying to derefenence .end() but it is just not working. Here is the function:
I marked the problem area. I have been getting very erratic compiler behavior. Maybe it is VC++, but who knows? Can someone please try to compile this? I have been getting internal compiler errors, things like "cannot convert from std::basic_string to std::_Tree" in the call to propertyMap.insert(), and also when I Rebuild All, there are no errors, but if I erase one letter and put it back (so the file has been modified) and do a regular build, there are errors.
This is driving me crazy!
map<string, string> ParsePropertyString(string s){ using namespace boost; using namespace std; map<string, string> propertyMap; vector<string> propertyList; string a, b; tokenizer<escaped_list_separator<char> > commaTokenizer(s); for(tokenizer<escaped_list_separator<char> >::iterator i=commaTokenizer.begin(); i!=commaTokenizer.end();++i) { propertyList.push_back(*i); } char_separator<char> eqlsDelimiter(" =", "", drop_empty_tokens); int end = propertyList.size(); for (int j = 0; j < end; j++) { // I have narrowed the problem down to these // lines: // ----------------------------------------- tokenizer<char_separator<char> > eqlsTokenizer(propertyList.pop_back(), eqlsDelimiter); tokenizer<char_separator<char> >::iterator i; i = eqlsTokenizer.begin(); a = *i; i++; b = *i; propertyMap.insert(a, b); // ----------------------------------------- } return propertyMap;} I marked the problem area. I have been getting very erratic compiler behavior. Maybe it is VC++, but who knows? Can someone please try to compile this? I have been getting internal compiler errors, things like "cannot convert from std::basic_string to std::_Tree" in the call to propertyMap.insert(), and also when I Rebuild All, there are no errors, but if I erase one letter and put it back (so the file has been modified) and do a regular build, there are errors.
This is driving me crazy!
No version of map::insert takes key and value parameters.
You have map::insert( value_type ) and map::insert( iterator, value_type )
where value_type is a std:
air holding both key and value.
Documents [ GDNet | MSDN | STL | OpenGL | Formats | RTFM | Asking Smart Questions ]
C++ Stuff [ MinGW | Loki | SDL | Boost. | STLport | FLTK | ACCU Recommended Books ]
You have map::insert( value_type ) and map::insert( iterator, value_type )
where value_type is a std:
air holding both key and value. Documents [ GDNet | MSDN | STL | OpenGL | Formats | RTFM | Asking Smart Questions ]
C++ Stuff [ MinGW | Loki | SDL | Boost. | STLport | FLTK | ACCU Recommended Books ]
Well, this code should not even compile. The whole template thing sometimes leaves VC++ 6.0 quite confused though. Unfortunate that it does not catch the error.
Your basic problem is that vector<> pop_back() does not return anything. You need to copy the last element off into a temporary before you pop_back().
After doing that it works flawlessly on my side.
Here is the modified code:
Your basic problem is that vector<> pop_back() does not return anything. You need to copy the last element off into a temporary before you pop_back().
After doing that it works flawlessly on my side.
Here is the modified code:
map<string, string> ParsePropertyString(string s){ map<string, string> propertyMap; vector<string> propertyList; string a, b; tokenizer<escaped_list_separator<char> > commaTokenizer(s); for(tokenizer<escaped_list_separator<char> >::iterator i=commaTokenizer.begin(); i!=commaTokenizer.end();++i) { propertyList.push_back(*i); } char_separator<char> eqlsDelimiter(" =", "", drop_empty_tokens); int end = propertyList.size(); for (int j = 0; j < end; j++) { string tmp = propertyList.back(); propertyList.pop_back(); tokenizer<char_separator<char> > eqlsTokenizer(tmp, eqlsDelimiter); tokenizer<char_separator<char> >::iterator i; i = eqlsTokenizer.begin(); a = *i++; propertyMap[a] = *i; } return propertyMap;}
Author
quote:Original post by Premandrake
After doing that it works flawlessly on my side.
Does it? I'm still having problems. Still the very erratic compiler behavior. Are you using VC6?
Any other suggestions? The source that Premandrake posted does not compile for me.
UPDATE: The problem is with the lines:
for(tokenizer >::iterator i=commaTokenizer.begin(); i!=commaTokenizer.end();++i)
and
tokenizer >::iterator i;
but I have no idea why. I am getting an internal compiler in the file <boost/iterator_adaptors.hpp> in the dereference operator function for iterator_adaptor. Could this have something to do with the order I am including my files?!?
Here it is, just in case:
#include <vector>
#include <map>
#include <string>
#include <boost/tokenizer.hpp>
This is very frustrating...can anyone else try to compile it?
[edited by - Neosmyle on December 6, 2002 11:30:03 PM]
Okay, with VC++ 6, I am able to reproduce your problem of hitting space then compiling again and getting the internal compiler error. Rebuild all works correctly. You can also delete the .obj for the file manually and it will work (perhaps do this as a prebuild step in a batch file). This is a very unfortunate bug that I''m quite surprised hasn''t been fixed yet.
Anyhow, the following code *does* work, just it will give an internal compiler error when you try to compile after already compiling it.
Anyhow, the following code *does* work, just it will give an internal compiler error when you try to compile after already compiling it.
#include <boost/tokenizer.hpp>#include <map>#include <vector>#include <string>#include <iostream>using namespace std;using namespace boost;map<string, string> ParsePropertyString(string s){ map<string, string> propertyMap; vector<string> propertyList; string a, b; tokenizer<escaped_list_separator<char> > commaTokenizer(s); for(tokenizer<escaped_list_separator<char> >::iterator i=commaTokenizer.begin(); i!=commaTokenizer.end();++i) { propertyList.push_back(*i); } char_separator<char> eqlsDelimiter(" =", "", drop_empty_tokens); int end = propertyList.size(); for (int j = 0; j < end; j++) { string tmp = propertyList.back(); propertyList.pop_back(); tokenizer<char_separator<char> > eqlsTokenizer(tmp, eqlsDelimiter); tokenizer<char_separator<char> >::iterator it; it = eqlsTokenizer.begin(); a = *it++; propertyMap[a] = *it; } return propertyMap;}int main(){ map<string,string> prop = ParsePropertyString("temp = Blah, asdf = fdf"); cout << prop["temp"] << " " << prop["asdf"] << endl; return 0;} This topic is closed to new replies.
Advertisement
Popular Topics
Advertisement
