Sign in to follow this  
  • entries
    149
  • comments
    510
  • views
    94472

Magus Expressions

Sign in to follow this  
noaktree

81 views

Today I implemented a first version token parser. I've kept the grammar small for now and have only added support for integer variables. The main parsing function is responsible for variable management and handling keywords. The real work was in the expression function. When a possible expression is encountered in the main parsing loop, it passes the tokens and the memory map to an expression parser.

The expression parser...

The operator precedence expression parser uses two stacks (values and operators) and a precedence lookup table to determine appropriate actions and solve the expression.

"Operator precedence parsing is based on bottom-up shift-reduce parsing. As an expression is parsed, tokens are shifted to a stack. At the appropriate time, the stack is reduced, applying the operator at the top of the stack." - Thomas Niemann


As I already mentioned, I've only implemented an expression parser for integer types. I still need to settle on a final design for the expression parser function. I could make a template out of it but I'll probably end up making a different version of each data type. This may be necessary because of incompatible data types. For example I am including float2, float3, and float4 built in vector data types and while I could add an integer to a float3 I couldn't add a float3 to an integer.

Example code...

It's kind of cool to step through the parsers as they execute a script. This is an example of the kind of code it can handle as of now:

int x = 20;
int y = 4;
int z = (2 * (x - y^2)) / y;
print z;

output: 2
Sign in to follow this  


12 Comments


Recommended Comments

Good progress for a couple of days work (based on journal dates!)...

I always wanted to find a tool that'd draw (as in, to a bitmap) out expression trees and/or AST's from compiler parsing. Never managed to find one that I could get for free [smile]

Our lecturer got some sort of sick enjoyment over seeing us cramming a whole program's AST onto a single piece of A4 paper in exam conditions [rolleyes]

Jack

Share this comment


Link to comment
Very cool man. I dealt with expression parsing during the end of last semester. We had to write a compiler that compiled SIMPLE language to SML commands, and then would run it on a simulator.

The best part was definitely the expression parsing. It's a lot of fun to watch it solve complex expressions, especially ones with variables where it uses a lookup table. "Solving" in this case involved compiling it to SML commands, and this conversion was really interesting because I had to map out SML commands that would calculate the expression; I couldn't just do algebra then and there and output the solution.

I could send you the code if you want, though it's in stupid Java as that was a requirement. Sounds like you've gotten pretty far though anyway.

Share this comment


Link to comment
Quote:
Our lecturer got some sort of sick enjoyment over seeing us cramming a whole program's AST onto a single piece of A4 paper in exam conditions
Jack that sounds just evil.

Magus is a scripting language that is executed on the fly so I don't see the need to build a syntax tree from the parsed source (am I wrong?). I may build an actual compiler for it some time.
Quote:
I could send you the code if you want, though it's in stupid Java as that was a requirement.
Thanks. I may want to see that in the future. I'm interested in standard ML but have no experience with it yet. It would be interesting to see how you generated the syntax tree.

Share this comment


Link to comment
Quote:
Good progress for a couple of days work (based on journal dates!)...


That is what I am talking about too! This guy is sick. You arent human are you Neil...I bet you have a secret time device where you go program and then come out laughing mockingly evilly at all us fools....

I wish I could program that well/fast. [sad]

Share this comment


Link to comment
Quote:
This guy is sick. You arent human are you Neil.
Wow man you're too much. [headshake] I bet there are a million other programmers better than me. You're probably one of them and you're just picking on me because I suck so bad.

Share this comment


Link to comment
Quote:

Thanks. I may want to see that in the future. I'm interested in standard ML but have no experience with it yet. It would be interesting to see how you generated the syntax tree.


Unfortunately, I meant Simpletron Machine Language. I wish it was Standard ML, that would be so cool [grin] hmmm..... compiling it into a functional language. Interesting!

But it's nothing complex, so you probably wouldn't gain anything from it.

And they aren't joking, you move through projects like a fat man does pizza.

Share this comment


Link to comment
Quote:
Magus is a scripting language that is executed on the fly so I don't see the need to build a syntax tree from the parsed source (am I wrong?). I may build an actual compiler for it some time.

I suppose if it's interpretted, things like AST's are probably a bit overkill [smile]

Quote:
You're probably one of them and you're just picking on me because I suck so bad.

Don't let Daerax get to you, he's just jealous: noaktree > Daerax [razz]

Jack

Share this comment


Link to comment
Quote:
That's pretty cool. Will object orientation be available?
As of now I plan to add structs for user objects. If all goes well then I may add classes?? This is still a ways off though as I need functions working first. [smile]

Share this comment


Link to comment
Quote:
Quote:
You're probably one of them and you're just picking on me because I suck so bad.

Don't let Daerax get to you, he's just jealous: noaktree > Daerax [razz]


Yeah? Well, Daerax > jollyjeffers so take that. Yeah, thats what I thought. Foo. [grin]

Seriously though, the projects you pick up and finish just like tha t require a fair bit of knowledge, which means you are already posessed of this knowledge or are some kind of genius able to gain understanding of such topics as compiler theory in a matter of hours... Either way, the scope of knowledge is quite impressive.

Share this comment


Link to comment

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now