What the heck is "cnp'ed"? I've found that MinGW doesn't like using C-style string functions (basically <cstring>)
Back on track, what would be the most effective way for me to pass in a list of keywords for the lexer to look out for? I had a go and just got the Windows Error report dialog T_T
Still, I did get it to keep track of the line number it is on, and it tokenizes every plain-text file I feed into it
Problems with parsing
orcfan, can you tell me exactly what you did? I'd like to have a look to see if I can sort it out in case it's a problem with my code.
All I did was take the lexer code [lexer.h and lexer.cpp] and added to a blank project (because I'm interested in writing my own someday and I would like to study how they work). Anyways, I went like this in main.cpp:
It crashes like that. But like I commented earlier, I'm not exactly fond of classes.
P.S. cnp = copy n pasted
P.S.S. Please don't take it the wrong way, I'm not trying to steal your code.
#include "lexer.h"// The file test.txt only contains the following text:// "string literal"int main(){ std::ifstream f("test.txt"); base_lexer lexer(f); // I assume this is how this works, I'm not exactly fond of classes tok_type t = lexer.NextToken(); // Read 1 token std::cout<<t.token<<" : "<<t.value<<"\n"; std::cin.get(); return 0;}
It crashes like that. But like I commented earlier, I'm not exactly fond of classes.
P.S. cnp = copy n pasted
P.S.S. Please don't take it the wrong way, I'm not trying to steal your code.
which compiler are you using? I always use MinGW with Code::Blocks IDE.
Your source is almost exactly what I used to test the lexer myself.
here's my main.cpp file:
and test02.txt;
I haven't tried it with any other compiler, so it could be an incompatibility there, although I'd have thought that would be unlikely nowadays.
Also, I'm more than happy for you to use this code ( assuming you can get it working :P ), it'd be nice to think that my code can help.
PS; try adding in the other includes at the top of main.cpp again. it's just a stab in the dark, mind, but it might sort it
Your source is almost exactly what I used to test the lexer myself.
here's my main.cpp file:
#include <iostream>#include <fstream>#include <string>#include "lexer.h"using namespace std;int main(){ ifstream in("test02.txt"); cout << "Hi!!" << endl << endl; base_lexer bl(in); while(!in.eof()) { tok_type tt=bl.NextToken(); cout <<tt.token << tt.value << endl; } system("pause"); return 0;}
and test02.txt;
prog main: if(a) return else print "how are you?" end if "of course, you can put \"absolute gibberish in if you want.The lexer only splits the file into tokens. It doesn't try to makesense of them...For an example, run miniScript.exe, or just look in its test file 'test.txt'#this comment should appear in the string"#this comment should not appearend main
I haven't tried it with any other compiler, so it could be an incompatibility there, although I'd have thought that would be unlikely nowadays.
Also, I'm more than happy for you to use this code ( assuming you can get it working :P ), it'd be nice to think that my code can help.
PS; try adding in the other includes at the top of main.cpp again. it's just a stab in the dark, mind, but it might sort it
I'm using (was, at least) VS 2005 with a win32 project. Only thing is I can't test your code right now because of a computer breakdown... Anyways, the code I used looks very similar except for the fact that I only read 1 token. However, I ran one test earlier before posting that code that did the while loop and printed the token and value. It still crashed on the string literals when I did it that way. So I'm guessing it's my compiler..?
I'm not familiar with Visual Studio beyond version 6 :/
when you say you use a win32 project, shouldn't you be using a console project instead?
when you say you use a win32 project, shouldn't you be using a console project instead?
Not to be rude, but you do have a test.txt file in the folder, right?If it can't find the file, it just continuously prints a single number to the consolein a never-ending(i.e.; until you break) loop.
Does it compile, and crash when you try to run it? I had a similar problem to that once, turned out to be a nasty little virus called 'w32.jeefo'(at least, that's what Norton calls it, I won't say what I call it ;)
My lexer seems to dislike having two comment lines one after the other, I'll see if I can change that, now
Edit: thinking about it, the lexer might not like strings on the first line, either. I'll check it out now T_T
Edit 2: nope, they work fine on my comp...
[Edited by - webwraith on October 15, 2006 10:15:14 PM]
Does it compile, and crash when you try to run it? I had a similar problem to that once, turned out to be a nasty little virus called 'w32.jeefo'(at least, that's what Norton calls it, I won't say what I call it ;)
My lexer seems to dislike having two comment lines one after the other, I'll see if I can change that, now
Edit: thinking about it, the lexer might not like strings on the first line, either. I'll check it out now T_T
Edit 2: nope, they work fine on my comp...
[Edited by - webwraith on October 15, 2006 10:15:14 PM]
This topic is closed to new replies.
Advertisement
Popular Topics
Advertisement