Symbols table and/or syntax tree

Started by
3 comments, last by TheAtom 12 years, 11 months ago
Is it possible, to use
int asCParser::ParseScript(asCScriptCode *script)
method, to only parse the whole script and obtain table of all symbols like variables and their nesting levels? I would like that this work even if there are some syntax errors in the script. I need it to my "intellisense" system which must work all the time, not only on valid scripts. Is there any good way to implement this using AngelScript mechanisms or do I have to implement some Abstract Syntax Tree (or something else) by
myself ?
Advertisement
I didn't write the parser with the intent to expose the symbol tree to the application. I don't think exposing the actual symbol tree to the application would be a good idea, as the application would be too tightly integrated with the internals of AngelScript.

Perhaps I could implement methods that would allow the application to give a script, and then have AngelScript do a quick scan to identify entities (functions, classes, etc) without actually compiling the script. This would be something that I could maintain for future releases, without the application being tightly integrated with the internals.

AngelCode.com - game development and more - Reference DB - game developer references
AngelScript - free scripting library - BMFont - free bitmap font generator - Tower - free puzzle game

That scan method would be very great and it could be really useful in making AngelScript IDE's or integrating it with Visual Studio. Can you estimate when will you have some spare time to implement that method for fast scan?
Unfortunately not. It will be a while. Right now I'm working on 2.20.2 which is going to make some bigger internal changes to optimize run-time memory usage and performance, it will probably be at least another month before I get that out of the door. I haven't decided yet which features I'll work on for 2.21.0 after that.

AngelCode.com - game development and more - Reference DB - game developer references
AngelScript - free scripting library - BMFont - free bitmap font generator - Tower - free puzzle game

One can take the advantage of similarity of AS syntax to C++ syntax and, with some tweaking, use the wide range of MSVC Intellisense features. It is possible to get (at least) syntax highlighting, pretty much complete and correct autocompletion, jumping to function definitions (even imported from a different module), jumping to classes and their members declarations, list of classes and their members in Class View, code definition lookup and listing of functions in a current module. I have a MSVC solution for my AS files, they total around 80000 lines of the code and it works very good.

The trickery used here works in MSVC 2008 Express and is known to not work in MSVC 2010 Ultimate. The following tutorial is written specifically for MSVC 2008 Express (I believe it's the same for complete MSVC 2008).

First, one needs to associate AS files with the environment and it looks that none of the following steps can be omitted:
1. The AS source extension (e.g. .as) has to be associated with vcexpress.exe.
2. The AS source extension has to be associated with C++ language:
a) Open Tools->Options->VC++ Project Settings,
b) Add the AS source extension to C/C++ Files Extensions and restart the IDE.
3. The AS source needs editing experience assigned:
a) Open Tools->Options->Text Editor,
b) Add the AS extension to the MSVC++,

Of course, all files have to be placed in some vcproj file and added to some solution.
It is very convenient, if not outright necessary, to attach a standard header to all AS module files, henceforth referred to as stdafx.h, following the MSVS convention. It should be included in #ifdef _MSC_VER [...] #endif block to make it ignored by anything but the MSVC IDE.

stdafx.h should contain declarations of all classes and functions registered by the engine for which the scripts are written for. In particular, the array<T> template (if used). Furthermore, OpAssign and other operators can be written in proper C++.

After 1., there should be syntax highlighting working. Points 2. and 3. enable Intellisense, which at this point is limited by differences between AS and C++. Now I will describe how to deal with them.

There are seven key differences that confuse the Intellisense to the point of near-uselessness, but six of them can be dealt with decisively, while the remaining one isn't very problematic to begin with.

1. The @ for handle.
Apparently, MSVC 2008 Express runtime parser ignores this sign completely (this is not the case for 2010 Ultimate). This is of course excellent - for instance, function arguments that are handles are thought to be passed by value after @ is ignored, and . membership operator is expected by autocompletion. There's not much that has to be done about it further (and it probably wouldn't be really possible anyway).

2. Allowed lack of semicolon at the end of class.
This confuses the Intellisense as it expects the class definition to be followed by either ; or an identifier, which is then assumed to be of this type.

struct T
{
} s;

Following AS code is therefore badly interpreted:

struct T
{
}
void f() { }

This is simply remedied by placing the missing semicolons.

3. T[] array definitions.
As the stdafx.h contains the array<T> template definition, it is simply the matter of changing all T[] to array<T>. It has a nice bonus: the code is more ready for the transition to AS 3.0.0 which, as far as I know, deprecates the T[] syntax for the dynamic arrays. Static arrays can be left as T[], as having the Intellisense for them may not be that important.

4. The import keyword.
Placing

#define import extern

in stdafx.h not only takes care about this unrecognized token, but furthermore, it hints the intellisense that the function definition is in some different file. It works nearly flawless.

5. The private keyword.
Solved by

#define private

in stdafx.h. Of course the Intellisense will see all class members as private everywhere (or public if in struct).

6. The interface keyword.
Solved by

#define interface class

and there is not much to tell about it.

7. #define syntax in Anthony Casteel's preprocessor.
The problem is the way in which a macro argument number is defined:

AS: #define macro #(a,b,c) (a)+(b)+(b)
C++: #define macro(a,b,c) (a)+(b)+(b)

The solution would be to modify the preprocessor source, which might not be possible for a script writer who has no access to the application source. However, this is not a source of much inconvenience, at least for me. The Intellisense does not work inside macros.

For my project, I have prepared a set of tools that automate all of these steps. They are quite project-dependent, but they are not difficult to write for general use:
1. Intellisense Creator, i.e. tool for generation of stdafx.h. In our project, we have AS compiler with set of dlls that register all API functions and classes, identical to ones that engine will actually register. These dlls are also used to simply extract those functions and classes and put them in the header file. The operators are added based on Op* functions, too. The only problem is that argument names are lost in the process.
2. Script Refactorer. As the name suggests, it's a rudimentary parser for the AS code that does the tricks explained in 2. and 3.
3. Project Updater. Puts all modules used in the application to a specified vcproj file, scans the modules code to find headers (even cares about #ifdef blocks), and puts them to vcproj as well.

This topic is closed to new replies.

Advertisement