Sign in to follow this  
Funkymunky

Flex/Bison help

Recommended Posts

I've gone through the Flipcode Scripting Engine tutorial, and implemented the basic scripting engine demonstrated there. Currently it only really works with strings. If my script contains the statement,
a = "hello globe";
Then the code figures out that a is a new symbol, and assigns it the string constant specified. I want to add a second variable type for a 2 dimensional point. I'd like to be able to do this:
a = "hello globe";
b = point(4, 7);
And have the code treat the first statement the same, but figure out that the second statement is for a different variable type. Can anyone help me with this? Suggestions/advice would be awesome, flex/bison code would be greatly appreciated.

Share this post


Link to post
Share on other sites
Posting here. I've been analyzing the flex input and bison input files for a few hours now, but I don't see how to make it so that my assignment expression can be treated as more than just one variable type. I'm also unsure how to make the analyzer treat "point(4, 7)" as a variable declaration. Down the road I'm going to wonder how to make the "." operator, such that i can do "b = point(4, 7); b.x += 10;"

I'm not trying to just dump my problem here and get free work done for me. I'm actively scouring examples on Google and reading the help files; I was just hoping someone had some insight from experience.

Share this post


Link to post
Share on other sites
I have used flex/bison a few times before and I always managed to figure out how to do anything I needed. Unfortunately, it's been several years since the last time I used these tools, and I would have to spend a couple of hours to refresh my memory.

Perhaps you should start from a less trivial example.

Share this post


Link to post
Share on other sites
Allow me to rephrase: it's easier to modify something than come up with something from scratch. If you post something you've tried, even if you think it's crap, it'd be easier to show you how to modify that into working flex/bison code than if you posted none of your own code.

Share this post


Link to post
Share on other sites
my code right now is literally the code from the flipcode tutorial, unmodified. I understand why you're asking that, but I don't even have an attempt at this to start from. I was hoping for someone with some flex/bison experience in using different types of variables, even if it isn't the "point(4, 7)" format that I specifically want. Even just:
a = "hello globe";
b = 4;

Share this post


Link to post
Share on other sites
Break it down by parts. What are the tokens that make up "b = point(4, 7)"? What are the regular expressions that express each of those tokens?

Share this post


Link to post
Share on other sites
Alright I got it worked out to where it accepts strings or numbers. Originally I had:
// lexer rules
{STR} {StringConstant (); /* string constant: copy contents */
return STRING;}

// yacc token definitions
%token <str> ID STRING
// yacc rule type definitions
%type <symbol> identifier string
// yacc grammar rules
simple_expression
: identifier {$$ = new TreeNode (IDENT_EXPR); $$->symbol = $1;}
| string {$$ = new TreeNode (STR_EXPR); $$->symbol = $1;}


(with a definition for "identifier" and "string", as per the flipcode tutorials.)

I modified this to be:
// lexer rules
{STR} {StringConstant (); /* string constant: copy contents */
return STRING;}
{NUM} {NumValue (); /* number: copy value */
return NUMBER;}

// yacc token definitions
%token <str> ID STRING NUMBER
// yacc rule type definitions
%type <symbol> identifier string number
// yacc grammar rules
simple_expression
: identifier {$$ = new TreeNode (IDENT_EXPR); $$->symbol = $1;}
| string {$$ = new TreeNode (STR_EXPR); $$->symbol = $1;}
| number {$$ = new TreeNode (NUM_EXPR); $$->symbol = $1;}


(with an additional definition for "number")

This works; I can now accept numbers as well as strings.
Now, what I'm noticing, is that my virtual machine will have to have variables that can be either strings or numbers. For example, if the script is like this:
a = "hello";
b = 42;

I'll get symbols for a, b, "hello", and 42. The virtual machine will get virtual assembly like this:
 1: OP_NOP 
2: OP_PUSH strconst1
3: OP_GETTOP a
4: OP_DISCARD
5: OP_PUSH numvar1
6: OP_GETTOP b
7: OP_DISCARD

So at runtime, whenever I OP_GETTOP, I'd have to figure out the type of the variable and then assign it to a or b as such. This is fairly inefficient for my purposes; I'd like to know from the get-go what type of values the variables can hold. Essentially, I want to do type checking, so that my script will have to be as such:
string a = "hello";
int b = 42;

But I'm struggling with getting that to work. Right now, I could have a script like this:
a;

And it would happily recognize 'a' as a symbol. Instead, I want to have to specify a type for a so that I know what it can be assigned.

... any help on that?

[Edited by - Funkymunky on March 12, 2010 12:30:21 PM]

Share this post


Link to post
Share on other sites
Again, break it down by parts. What are the tokens in 'string a = "hello";' and what would regular expressions for those be?

Share this post


Link to post
Share on other sites
I have broken it down. The fact that it accepts just "a;" means it doesn't care what types of symbols it accepts. How do I enforce a rule paradigm of type checking?

Look please don't be offended, but I'm not looking for help with my problem solving process. The fact that after a few days I found the answer to my initial question should demonstrate that. I'm looking for assistance from someone with specific knowledge of lex/yacc (flex/bison).

Share this post


Link to post
Share on other sites
Dude, seriously, break it down into tokens.

string a = "hello";

KEYWORD_STRING IDENTIFIER EQUALS STRING_LITERAL SEMICOLON

Does that look anything like it would accept just "a;"? Doesn't that look something suspiciously like you could plug in as a parser rule?

Share this post


Link to post
Share on other sites
SiCrane, that's exactly what I'm asking how to do. How do you make it enforce the use of "KEYWORD_STRING" in your example. Furthermore, as it takes each token as one symbol at a time, how do I know when I get to "IDENTIFIER" what "KEYWORD_STRING" has been used? Can you tell me how to do it using Yacc grammar rules instead of just throwing me generalizations like I'm an idiot?

Share this post


Link to post
Share on other sites
I'm not writing your code for you. If you want to try posting something you've tried that doesn't work, I'll help you fix it. But you have to post something you've tried first. If you say you've broken it down by tokens then show something, anything, where you've tried to make a rule involving those tokens. If you don't care enough to bother trying something, why should anyone bother helping you?

In any case, by the KEYWORD_STRING token I don't mean any keyword I mean literally the keyword "string". For the int example, it would start with KEYWORD_INT, though obviously you can choose different names if you want. Now, think what the regular expression for the word "string" would look like. Turn that into a lex rule. Then take the next step. You've already got a lex rule for identifiers, so after that think of what the lex rule for "=" would look like, and so on.

Share this post


Link to post
Share on other sites
SiCrane, if you can't help with my specific question, why are you replying with general advice? Is it to bolster your ego as a coder who can answer any question? I'm not asking anyone to write my code for me, I'm asking if anyone has experience with lex/yacc and can answer my specific question. You could point me toward resources on yacc type checking, or offer snippets from code of your own. Instead you're treating me like an imbecile. You've effectively wasted my time for your own benefit; I haven't gotten any useful advice from this thread.

...

Thanks anyway. Bye.

Share this post


Link to post
Share on other sites
I'm not treating you like an imbecile; I'm treating you like a lazy bum. Get off your ass and actually try something before complaining about what kind of advice you're getting. You can't even be bothered to try writing a yacc rule when I spelled out token by token what would go in it. Which, incidentally, is specific advice.

Share this post


Link to post
Share on other sites
SiCrane I've been working on this since the time of my original post, as is evidenced by my 4th posting. I was even gracious enough to post up all of my work in case anyone in the future comes across this thread and can gain something from it. You haven't given me specific advice; you've told me to look at the tokens. That's an entry-level generalization on lexical analysis, not specific yacc grammar advice.

Forget it. I've been with this site for a looooong time, but maybe it's time I move on. I've gradually watched the quality of help from these forums degrade, and this right here is ridiculous. You're a staff member.

Share this post


Link to post
Share on other sites
Dude, if you've actually been trying stuff, then go ahead and post it! What yacc rules have you tried to use to incorporate types on your variables? It doesn't matter if it doesn't even compile, just show your work.

Share this post


Link to post
Share on other sites
Your problem is that what you are trying to do you don't actually do with flex/bison. You do it later.

As said in the flipcode tut, you need to build a syntax tree then parse that to do type checking amongst other things (semantic analysis it's called). All you do with flex/bison is break down the input into tokens (flex) and ensure the tokens fit into a grammar rule (bison).

You build your syntax tree through custom code in the grammar rules. The examples of this are in that tut.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this