• Announcements

    • khawk

      Download the Game Design and Indie Game Marketing Freebook   07/19/17

      GameDev.net and CRC Press have teamed up to bring a free ebook of content curated from top titles published by CRC Press. The freebook, Practices of Game Design & Indie Game Marketing, includes chapters from The Art of Game Design: A Book of Lenses, A Practical Guide to Indie Game Marketing, and An Architectural Approach to Level Design. The GameDev.net FreeBook is relevant to game designers, developers, and those interested in learning more about the challenges in game development. We know game development can be a tough discipline and business, so we picked several chapters from CRC Press titles that we thought would be of interest to you, the GameDev.net audience, in your journey to design, develop, and market your next game. The free ebook is available through CRC Press by clicking here. The Curated Books The Art of Game Design: A Book of Lenses, Second Edition, by Jesse Schell Presents 100+ sets of questions, or different lenses, for viewing a game’s design, encompassing diverse fields such as psychology, architecture, music, film, software engineering, theme park design, mathematics, anthropology, and more. Written by one of the world's top game designers, this book describes the deepest and most fundamental principles of game design, demonstrating how tactics used in board, card, and athletic games also work in video games. It provides practical instruction on creating world-class games that will be played again and again. View it here. A Practical Guide to Indie Game Marketing, by Joel Dreskin Marketing is an essential but too frequently overlooked or minimized component of the release plan for indie games. A Practical Guide to Indie Game Marketing provides you with the tools needed to build visibility and sell your indie games. With special focus on those developers with small budgets and limited staff and resources, this book is packed with tangible recommendations and techniques that you can put to use immediately. As a seasoned professional of the indie game arena, author Joel Dreskin gives you insight into practical, real-world experiences of marketing numerous successful games and also provides stories of the failures. View it here. An Architectural Approach to Level Design This is one of the first books to integrate architectural and spatial design theory with the field of level design. The book presents architectural techniques and theories for level designers to use in their own work. It connects architecture and level design in different ways that address the practical elements of how designers construct space and the experiential elements of how and why humans interact with this space. Throughout the text, readers learn skills for spatial layout, evoking emotion through gamespaces, and creating better levels through architectural theory. View it here. Learn more and download the ebook by clicking here. Did you know? GameDev.net and CRC Press also recently teamed up to bring GDNet+ Members up to a 20% discount on all CRC Press books. Learn more about this and other benefits here.

Jemgine

  • entries
    22
  • comments
    7
  • views
    77818

About this blog

Development of the Jemgine 2d game engine

Entries in this blog

Deyja

DeyjaScript 12 : Closures

The next step after lambda functions is obviously closures. Without closures, I couldn't write code like this.


var view = new View(world, new Rectangle(0,0,800,600));
bindKey("W", [void v] { view.panCamera(0.0, 1.0, 5.0); }, true);


Closures allow lambdas to use variables local to the function they are declared in. To implement this, first I need to detect when variables that need to be closed over are used. The Scope abstraction already performs variable name resolution. I hijack that, and when the scope in question is a lambda scope, and the variable found is from a higher scope, I wrap the variable up. I also prevent it from wrapping the same variable twice.


public UsableVariable resolveVariableName(bool searchUp, String name)
{
foreach (var node in variables) if (node.Name == name) return node;
if (baseScope != null)
{
var R = baseScope.resolveVariableName(false, name);
if (R != null) return R;
}

if (isLambdaFunctionScope) foreach (var node in lambdaReferencedVariables) if (node.Name == name) return node;

if (searchUp && parent != null)
{
var R = parent.resolveVariableName(true, name);
if (R != null && !(R is MemberVariable) && isLambdaFunctionScope)
{
var newLambdaVar = new LambdaVariableWrapper(R);
newLambdaVar.variableIndex = lambdaReferencedVariables.Count;
lambdaReferencedVariables.Add(newLambdaVar);
return newLambdaVar;
}
return R;
}
return null;
}


LambdaVariableWrapper disguises the variable wrapped as a member variable to some imaginary type. All of these variable references are detected in the first pass of compilation, so the list is ready to go for the second. As far as the code inside the lambda, the LambdaVariableWrapper emits bytecode to fetch the variable from an object passed on the stack. Func handles pushing. More interesting is the code that pushes the Func to the stack in the first place. It first has to create an object to hold all the closed variables, and then it needs to load all of them into this object, and then finally it passes the object to Func's constructor.


bytecode.AddBytecode(Instruction.pushNewImpleObject, (byte)localScope.lambdaReferencedVariables.Count);
bytecode.AddBytecode(Instruction.loadRegisterFromStack, (byte)3);
foreach (var boundVar in localScope.lambdaReferencedVariables)
{
boundVar.originalVariable.emitFetchByteCode(bytecode, context);
bytecode.AddBytecode(Instruction.pushRegisterToStack, (byte)3);
bytecode.AddBytecode(Instruction.loadMemberFromStack, (byte)boundVar.variableIndex);
bytecode.AddBytecode(Instruction.popN, (byte)2);
}


The lambda body will later assume this 'ImpleObject' is on the stack at position -4. It's actually a variable; localScope.addVariable(new LocalVariable("__bind", null, IndexType.Parameter)); But, even though the lambda body can resolve the name, it can't use the variable because it has no type.

This implementation closes over values, not variables. That means that whatever value a variable has when the lambda is declared, that's the value it will have in the body. Supporting closure-over-variables is simple. Notice that in the current implementation, something like


type A { int x; }
var a = new A();
a.x = 5;
var func = [void v] { doSomething(a.x); };
a.x = 2;
func();


closes over the variable x. When I call func() at the end, the value of the closed-over variable, 'a', has not changed. It still refers to the same A. And, even if you do a = new A();, the lambda is still going to use the original A. But, look at x. x has changed. And x wasn't closed over, a was. So when I call func at the end, x has changed, and the lambda sees the new value. This method of implementing closure over variables is so simple that I made it an official part of the language with this generic type.


var refCode = @"type Ref {
T value;
void construct(T value) { this.value = value; }
}";
CompileType(globalScope, refCode);


And then I can write code like this


var ogRef = new Ref();
ogRef.value = new ObjectGrid(new Rectangle(104, 4, 692, 592), [Definition d, void v]
{
ogRef.value.hide();
mousePlaceObject(view, world, d, [Location l, Facing f, void v]
{
addLocalTask(new BuildTask(d,l,f));
});
});


in which the lambda argument to the ObjectGrid constructor references the same ObjectGrid it is being passed to. It (unfortunately) compiles without the use of references, such as


var og = new ObjectGrid(new Rectangle(104, 4, 692, 592), [Definition d, void v]
{
og.hide();
mousePlaceObject(view, world, d, [Location l, Facing f, void v]
{
addLocalTask(new BuildTask(d,l,f));
});
});


but will fail at runtime because, at the time the closed values are gathered, 'og' is null. The lambda body will see null, not the new value of og. The reason the first version works is because ogRef already exists. I change the object ogRef references without creating a new Ref<> object. The lambda sees the original value of ogRef (the Ref<> instance) and gets the new value of Ref<>.value from it.

Closures are damn handy. I also implemented some helper functions to make binding things to the scripting language easier. C# reflection makes automatically wrapping arbitrary C# lambdas possible.


public void wrapSystemFunction(Scope into, String name, Delegate Func)
{
var systemFunction = new CallableFunction(CallingConvention.System, FunctionType.Free);

var invokeMethod = Func.GetType().GetMethod("Invoke");
var Parameters = invokeMethod.GetParameters();

systemFunction.ReturnType = findSystemType(invokeMethod.ReturnType);
systemFunction.ParameterTypes = new List();
foreach (var parm in Parameters) systemFunction.ParameterTypes.Add(findSystemType(parm.ParameterType));

systemFunction.ReturnTypeName = systemFunction.ReturnType.fullTypeName;
systemFunction.Name = name;
systemFunction.DecoratedName = Scope.getDecoratedName(systemFunction);
systemFunction.ParameterCount = systemFunction.ParameterTypes.Count; // parameterTypes.Length;
systemFunction.systemFunc = (vm, parms) =>
{
var arguments = new object[parms.Length];
for (int i = 0; i < parms.Length; ++i)
arguments = parms.getMember(0);
var result = Func.DynamicInvoke(arguments);
var obj = vm.createScriptObject(systemFunction.ReturnType);
obj.setMember(0, result);
return obj;
};
systemFunction.DeclarationScope = into;

into.addStaticFunction(systemFunction);
if (globalScope.IsDuplicateFunction(systemFunction)) throw new CompileError("Duplicate system func");
}


Unfortunately I still have to explicitly declare the types of parameters to the delegate when I call wrapSystemFunction.

scriptEngine.wrapSystemFunction(scriptEngine.getGlobalScope(), "hostPeerToPeerBuildSession", new Action((world, port) =>
{ activeSession = new PeerToPeerBuildSession(world, port, this); }));
Deyja
I mentioned last time that I might start integrating this scripting language with that other thing, and I did. Nothing evolves a library faster than actually using it. First, I designed the interface I wanted scripts to see. The first thing I want to use scripts for is key binding. I could enumerate all possible actions and end up with something like 'Game:bindKey(Key, Action)', but I really want something more. I want players to be able to bind complex scripts to keys. It complicates the binding of simple things a great deal, but I think it will be worth it. A representative line from the startup script is Game:bindKey("W", [void v] { Game:getActiveCamera().pan(0.0, 1.0, 5.0 * Game:deltaTime()); }, true); Eventually I want to find a way to simplify this, but first, I'll concentrate on just getting this working.

In that line I see a 'Game' type (or variable?) that has the functions bindKey, getActiveCamera, and deltaTime. I can approach this two ways. I can add global objects to the language, create a type, and create a global instance called 'Game'. Or, I can add static or 'free' functions. I decide to go with the latter as it doesn't involve me having to figure out where to store the globals. Each scope can have regular member functions or static functions. You can't actually declare them in script, there is simply no syntax for them, but you can bind them externally by calling ScriptEngine.addSystemFunction as in scriptEngine.addSystemFunction(gameType, "getActiveCamera", "Camera", (vm, parms) => { return Cam; }); Allowing scripts to call these functions is more complicated. There are three different nodes in the grammar that call functions. FunctionCallNode, MethodCallNode, and InvokeOperationNode, and FunctionCallNode is overloaded to behave like MethodCallNode in some cases. Where do I shove StaticCallNode into this? Well, I could, and then overload FunctionCallNode again, but instead I combine all these nodes into a single InvokationNode. It's... well, I don't think Apoch would like it.


using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using Irony.Interpreter.Ast;

namespace DeyjaScript
{
class FunctionCallExNode : CompilableNode
{
CallableFunction function;
UsableVariable invokeOn;
CompilableNode emitThisFirst;
bool pushImplicitThis = false;

public override void Init(Irony.Parsing.ParsingContext context, Irony.Parsing.ParseTreeNode treeNode)
{
base.Init(context, treeNode);
AddChild("Expresion", treeNode.ChildNodes[0].FirstChild);
if (treeNode.ChildNodes.Count == 2)
foreach (var parameter in treeNode.ChildNodes[1].ChildNodes)
AddChild("Parameter", parameter);
this.AsString = "Function Invokation";
}

internal void lookupFunction(ScopeStack context)
{
if (function != null) return;

var argumentTypes = CompilableNode.getArgumentTypes(ChildNodes, context, 1);

if (ChildNodes[0] is MemberAccessNode)
{
//method call or member functor; This is a shortcut to implement methods more effeciently.
var memberAccessExpression = ChildNodes[0].ChildNodes[0];
var memberAccessIdentifier = ChildNodes[0].AsString;

var invokeOnType = (memberAccessExpression as CompilableNode).getResultType(context);
invokeOn = invokeOnType.resolveVariableName(false, memberAccessIdentifier);
if (invokeOn != null)
function = invokeOn.Type.resolveFunctionOverload(false, "__invoke", argumentTypes);
else
function = invokeOnType.resolveFunctionOverload(false, memberAccessIdentifier, argumentTypes);
emitThisFirst = memberAccessExpression as CompilableNode;
}
else if (ChildNodes[0] is TypenameNode)
{
var funcName = (ChildNodes[0] as TypenameNode).removeLastToken();
if (funcName == null) throw new CompileError("How did this match if it's empty?");
var invokeOnType = (ChildNodes[0] as TypenameNode).findType(context);
if (invokeOnType == null)
{
invokeOn = context.topScope.resolveVariableName(true, funcName.identifier);
if (invokeOn != null) //Functor
{
function = invokeOn.Type.resolveFunctionOverload(false, "__invoke", argumentTypes);
if (function == null) throw new CompileError("Cannot invoke this type.");
pushImplicitThis = true;
}
else
{
function = context.topScope.resolveFunctionOverload(true, funcName.identifier, argumentTypes);
if (function != null) pushImplicitThis = true; //bare function call
if (function == null)
{
function = context.topScope.resolveStaticFunctionOverload(true, funcName.identifier, argumentTypes);
if (function == null) throw new CompileError("Could not find function " + funcName.identifier);
}
}
}
else
{
function = invokeOnType.resolveStaticFunctionOverload(false, funcName.identifier, argumentTypes);
if (function == null) throw new CompileError("Could not find function " + funcName.identifier);
}
}
else if (ChildNodes[0] is CompilableNode)
{
//Expression - must be a functor of some kind.
var invokeOnType = (ChildNodes[0] as CompilableNode).getResultType(context);
function = invokeOnType.resolveFunctionOverload(true, "__invoke", argumentTypes);
emitThisFirst = ChildNodes[0] as CompilableNode;
}
}

internal override Scope getResultType(ScopeStack context)
{
lookupFunction(context);
if (function == null) throw new CompileError("Function not found (GRT)");
return function.ReturnType;
}

internal override void gatherInformation(ScopeStack context)
{
lookupFunction(context);
if (function == null) throw new CompileError("Function not found (GI)");

if (emitThisFirst != null) emitThisFirst.gatherInformation(context);
for (int i = 1; i < ChildNodes.Count; ++i)
(ChildNodes as CompilableNode).gatherInformation(context);
}

internal override void emitByteCode(List bytecode, ScopeStack context, bool placeResultOnStack)
{
for (int i = 1; i < ChildNodes.Count; ++i)
(ChildNodes as CompilableNode).emitByteCode(bytecode, context, true);

//Can't just emit the leading expression since member access nodes are short circuited.
if (emitThisFirst != null) emitThisFirst.emitByteCode(bytecode, context, true);
if (invokeOn != null)
{
if (invokeOn is MemberVariable && pushImplicitThis)
bytecode.AddBytecode(Instruction.pushVariableToStack, (byte)(125));
invokeOn.emitFetchByteCode(bytecode, context);
function.EmitCallBytecode(bytecode, context);
}
else
{
if (function.type == FunctionType.Method && pushImplicitThis)
bytecode.AddBytecode(Instruction.pushVariableToStack, (byte)(125));
function.EmitCallBytecode(bytecode, context);
}
bytecode.AddBytecode(Instruction.popN, (byte)function.ParameterCount);
if (placeResultOnStack) bytecode.AddBytecode(Instruction.pushRegisterToStack, (byte)Register.returnValue);
}
}


}


It's big, it's ugly, and it handles every possible way of invoking a function.

Oh, I also added While loops and a floating-point type. It's called 'float' in script, it's implemented as a C# double.

Up until this point, I've had a 'test script' that I run to make sure everything works. But it's gotten too long, and I need to cut out old stuff to debug the new stuff. Suddenly I have no way to know if I've broken old functionality. Also, bits of the library are leaking out to support the testing. Bits that really shouldn't be public. So I move regression testing into the library itself, with a suite of tests. It is woefully incomplete, but as I discover bugs, I'll add tests to cover them.


executeTest("Empty Test", "pass();", Engine, VM);
executeTest("Global System Call", "pass();", Engine, VM);
executeTest("System Method Call", "var S = \"123\"; if (S.length() == 3) pass();", Engine, VM);
executeTest("Free Function Call", "void function() { pass(); } void TEST() { function(); }", Engine, VM, false);
executeTest("Method Call", "type A { void method() { pass(); }}; var a = new A(); a.method();", Engine, VM);
executeTest("Lambda Call", "var lambda = [void v] { pass(); }; lambda();", Engine, VM);
executeTest("Lambda Method", "type A { void method() { pass(); }}; var a = new A(); var lambda = a.method; lambda();", Engine, VM);
executeTest("Integer Ops", "int x = 4 * 3; int y = 16 / 8; if (x + y == 14) pass();", Engine, VM);
executeTest("Float Ops", "float x = 4.0 * 3.0; float y = 16.0 / 8.0; if (x + y == 14.0) pass();", Engine, VM);
executeTest("Order of Ops", "int x = 4 * 3 + 10 / 5 - (4 + 2) / 3; if (x == 12) pass();", Engine, VM);
executeTest("Chain Invoke", "RegressionTest:getRegTest().test();", Engine, VM);
executeTest("Expression Invoke A", "Func A() { return RegressionTest:getRegTest().test; } void TEST() { A()(); }", Engine, VM, false);
executeTest("Expression Invoke B", "Func A() { return [void v] { RegressionTest:getRegTest().test(); }; } void TEST() { A()(); }", Engine, VM, false);
executeTest("Negative Numbers", "int x = -5; if (x == -5) pass();", Engine, VM);
executeTest("Subtraction & Negate", "if (5 - 7 == 2 + -4) pass();", Engine, VM);


Most of these tests trivially test trivial bits of the language. Several of them declare variables or invoke other parts of the language they aren't directly testing. At least one of them is designed to fail - Expression Invoke A results in the compile error, 'Cannot take pointer to system function'. This represents a flaw I'll have to address at some point. You can see in the last two tests how parsing negative numbers broke everything.

It's time now to seriously consider the interface the scripting language will present to the world. So far I've just sort of written code. A lot of the types are public for no good reason. Some, like CodePointer, the outside world has no business knowing about ever. I hide them, and I resolve the inconsistent accessibility errors (An error that took me a long time to understand when I first started using C#. Now I wonder, why doesn't C++ have this same error?), and then I go through every class that's still public and make sure what's public actually should be, and everything else is internal or private. I'm left with a few core types. The engine, the virtual machine, Scope, ScriptObject, Function. The engine was pretty well buttoned up. Now Scope has public functions for looking up functions, and you can walk the owner scope tree but only up. Function exists only so you can call it. The important data is exposed, but immutable.

I decided that the most useful function the virtual machine could have is a 'run string' function. This will compile a module, execute it, and then discard it. The engine doesn't actually support discarding modules, but that's fine, I'm the library author. I can just hack it.


var wrappedCode = "void RUNSTRING(){" + code + "}";
var currentTypeCount = context.globalScope.SubordinateScopes.Count;
var module = context.CompileModule("RUNSTRING", new List { wrappedCode });
var func = module.findStaticFunctionBySimpleName("RUNSTRING");

callFreeFunction(func, context);

context.globalScope.SubordinateScopes.RemoveRange(currentTypeCount, context.globalScope.SubordinateScopes.Count - currentTypeCount);


So I'm finally using the scripting language in my game, and I want to bind a function that takes a Func as an argument. Except, I can't. Why not? Because addSystemFunction doesn't actually parse the argument types, it just looks for a type with that name. Fine, Func is named Func; except that it doesn't exist yet, because it hasn't be instantiated. This is a terrible flaw. AddSystemFunction should parse that typename and instantiate the generic. In the mean time, I provide a terrible work around, invoked like so


var FuncVoidType = scriptEngine.InstanceGenericType(scriptEngine.getGlobalScope().findGenericType("Func", false, 1),
new List { scriptEngine.getGlobalScope().findLocalType("void", false) });


I use it, it works. When I press W, scripts get run, the camera pans. The game does absolutely nothing it didn't do before, but it does it differently, and that makes all the difference.

Next time, I think I'll talk about type deduction. Not the sort the compiler uses now, which I'll term 'backward type deduction'. That is fairly trivial. Given A(B);, if I know the type of B, I can find A. Everything in the compiler currently uses backward type deduction. Instead, I'll talk about something more complicated, forward type deduction. That is, given something like A( { B.foo(); });, if we know the type of A, we can deduce the type of B. I call this 'forward' because we move forward from the deduced type, rather than backward from it. Implementing a method of forward type deduction will allow me to dispense with type arguments to Lambdas, including their return type; it's also necessary to properly support the 'null' token. See if you can figure out why.
Deyja

DeyjaScript 10 : Lambdas

I started thinking, okay, I've written this thing. Now how can I actually use it? Wasn't I supposed to be making a mud engine? Yeah, well, I already don't want to anymore. Instead, I'm going to use DeyjaScript in this thing, https://www.gamedev.net/blog/342/entry-2250747-gnome-colony-build-demo/ . I'll use it to implement an in-game console where the player can assign macros - bits of DeyjaScript code - to keys. This is also how I'll implement key binding and re-mapping. This decision led me to the next required feature in DeyjaScript - Lambdas. The goal is to be able to type something like 'bindkey [key] [lambda]' into the console.

The first thing I need to support lambdas is function pointers. A function pointer will be able to point to a method or a lambda function. To make this possible, lambdas will masquerade as methods. Like arrays, I'll first create a type-less raw system type, and then wrap it in a generic type. Since it will be impossible for scripts to do anything with this type (though they could create one, since the language lacks any sort of access control) it doesn't have any methods. It's just a data carrier. I add two instructions to create them. pushMemberFunctionPointer and pushLambdaPointer. The first looks up the member, wraps it's code pointer in the system type, and pushes it. The second takes the current code pointer, changes it's start index, wraps it and pushes it. Since lambdas will be smashed onto the end of the function they are declared in, and can't be referenced anywhere else, this works great. A final instruction, invokeFunctionPointer, takes the wrapped code pointer and jumps to it.

Next, the wrapper generic.


Func {
funcPtr raw;
object _this;
void construct(object _this, funcPtr ptr)
{
raw = ptr;
this._this = _this;
}
R __invoke(T0 t0) {
//What goes here??
}
}


There's another reserved method name. I'll use '__invoke' later to support operator(). Two places in the grammar create Func<...> objects. Accessing members, and lambda declarations. Both of these emit code to create the funcPtr type, then to create a Func<...> with the correct types, and call it's constructor, so that it appears to the script that the type of a method or lambda is 'Func<...>'.


//Inside LambdaExpressionNode.emitBytecode...
bytecode.AddBytecode(Instruction.pushVariableToStack, (byte)125); //push this
callIndex = bytecode.Count + 1;
bytecode.AddBytecode(Instruction.pushLambdaPointer, BitConverter.GetBytes((Int16)0));
bytecode.AddBytecode(Instruction.pushNewObject);
NewStatementNode.emitIndexStack(bytecode, funcType, 0, context);
var constructor = funcType.findFunctionBySimpleName("construct");
constructor.EmitCallBytecode(bytecode);

//replace the raw function pointer with the Func<..> object.
bytecode.AddBytecode(Instruction.loadRegisterFromStack, (byte)Register.returnValue);
bytecode.AddBytecode(Instruction.popN, 3); //pop Func<...>, funcPtr, this
bytecode.AddBytecode(Instruction.pushRegisterToStack, (byte)Register.returnValue);


The lambda has an address of 0 because the enclosing function needs to fill that value in later. The actual syntax of a lambda looks like this -

var func = [int a, int b, int c, int r] { return a + b + c; };

I'm not thrilled with having to specify the return value as the last type argument, or with having to name it. The name is never used. Maybe I'll find a way to deal with that later. I want to support calling Funcs with variable() syntax, so I have to throw a bunch of code all over. A new grammar construction 'expression + ( + parameter list + )' catches cases where the func is the result of an operation, but when it's just a variable, the parser mistakes it for an ordinary function call. The function call node gets to deal with it too. I could replace the function call node with the new version entirely, but only at the cost of highly ineffecient function calls (Create a function pointer, invoke it, throw it away - compared to just that middle step.)

To compile lambdas, the enclosing function keeps track of all the lambdas declared in itself, and after it compiles it's own code it compiles all of them and tacks them onto the end of it's bytecode.

I skipped over an important detail earlier, the actual implementation of Func<...>.__invoke. How, exactly, do I invoke the raw funcPtr object? I have an instruction to do it, but I need to prepare the stack and actually call it. The answer turns out to be inline bytecode. To support that, I need to build a table of all the instructions and the number and sizes of their arguments. Then I add it to the grammar and compile it. It does no checking of any sort, it just spits opcodes into the bytecode stream, so it's very dangerous. But it allows me to write this


R __invoke(T0 t0) {
inline {
pushVariableToStack 126; //push return pointer
loadRegisterFromStack 4; //Remember return pointer
pop;

pushVariableToStack 125; //push this
pushMemberToStack 0; //push raw
loadRegisterFromStack 5; //Remember func pointer
pop;

pushVariableToStack 125; //push this
pushMemberToStack 1; //push _this
loadRegisterFromStack 6; //Remember _this;
pop;

loadRegisterFromStack 0; //recover frame pointer
popN 3; //remove frame pointer, return pointer, and this pointer

pushRegisterToStack 6; //Push _this;
pushRegisterToStack 4; //Push return pointer
pushRegisterToStack 5; //Push func pointer

invokeFunctionPointer;
}
}


I re-write the top of the stack to remove the current frame, and then I invoke the function. When that function returns, it will go directly to the code that called __invoke, as if __invoke never existed at all. Since the parameters to __invoke are on the stack, I never have to touch them. I use a few magic constants (The this pointer is always at stack position 125, which is the same as -3 - it's encoded as 128 + offset, to support negative offsets), and I know where __invoke's frame pointer and return value are at on the stack. By time I invoke, the stack looks as if the code that called __invoke called the function pointed to directly, complete with it's this pointer.

So, lambdas are a go. Maybe next I'll start integrating it with that other thing. Or maybe I'll add loops. I'm not out of 'advanced' features I can implement before the basics at all, maybe this language will never have loops.

Aside : As it turns out, deducing the return type of a lambda from the contents of that lambda is impossible. Even in the trivial case of a lambda that returns an int, the system can't handle it. Does it return int? Does it return object? Either is valid. The only way to deduce the return type will be from what the lambda is assigned to, or from the type of the argument it is in a function call. Presumably, all types could be deduced this way, but it would mean type resolution becomes compilation order dependent.
Deyja
For arrays in this language, I had two requirements. First, they had to be dynamically sized, and second, they had to be type-safe. But, when I want about trying to actually implement them, I discovered that my current implementation of the type system could handle only one or the other. I could create arrays, but like in C++, the array itself could have no members. Dynamic sizing requires member functions on the array for adding and removing elements, and there is no way to add these members to the array without implementing some sort of generic type. The second option was to create a dynamically sized array that wasn't type-safe. This didn't require any special syntax since it looks like an ordinary type (called 'array'), but it did require every type in the language to have a common base. For types defined in scripts, that was simple. Their base defaults to 'object'. For primitive types, it was a bit more complicated. Instead of being a C# primitive boxed into an object, they are a C# primitive boxed into an object boxed into a ScriptObject. It is nasty, but in the long run, worth it. For either of these array implementation options, the other behavior can be built on top using generics. I decided to provide a dynamically-sized array of anything, as it should be useful for many data structures I might want to represent in the scripts.

The array isn't technically part of the language. It's one of the system types that are always available, but the script engine uses the public interface to bind them. Any client code could bind a type that behaves exactly like it.


var arrayType = addSystemType("array", objectType);
addSystemMemberFunction(arrayType, "construct", "void", (parms) =>
{
parms[0].members[0] = new List();
return null;
});
addSystemMemberFunction(arrayType, "add", "void", (parms) =>
{
(parms[1].members[0] as List).Add(parms[0]);
return null;
}, "object");
addSystemMemberFunction(arrayType, "__indexGet", "object", (parms) =>
{
return (parms[1].members[0] as List)[(parms[0].members[0] as int?).Value];
}, "int");
addSystemMemberFunction(arrayType, "__indexSet", "object", (parms) =>
{
var index = (parms[1].members[0] as int?).Value;
var list = (parms[2].members[0] as List);
var obj = parms[0];
list[index] = obj;
return obj;
}, "object", "int");


There will also have to be some methods for removal, for retrieving the count, etc, but I haven't written them. Those named '__indexGet' and '__indexSet' are special methods called by operator[]. Any type can support operator[] by implementing these functions. Function names that begin with double underscores are from this point on reserved for implementation details like this. Also notice that when a function returns void, it doesn't need to create a special ScriptObject to represent void.

I mentioned generics, and I really wanted type-safe arrays, so it was time to implement them. First, I added generic parameters to the grammar for typenames. So Foo is a valid typename, and so is Foo. Typenames became a tree of nested typenames, and, frankly, became way more complex than I expected. Parsing is not so difficult. I add generic parameters to object declarations too. Now I need to resolve these new typenames to actual types. TypenameNode does the heavy lifting.


internal static Scope findType(bool searchUp, ScopeStack context, Scope currentScope, TypenameToken of)
{
Scope nextScope = null;
if (of.hasGenericParameters)
{
var templateName = of.singleLevelName;
var instancedType = currentScope.findLocalType(templateName, searchUp);
if (instancedType != null) nextScope = instancedType;
else
{
var declarationScope = currentScope.findLocalType(of.identifier, searchUp);
var genericParameters = new List();
foreach (var parameter in of.genericParameters)
genericParameters.Add(findType(true, context, currentScope, parameter));
var Engine = context.globalScope.tag as ScriptEngine;
var Object = declarationScope.tag as ObjectDeclarationNode;
nextScope = Engine.EnqueueGenericTypeForCompilation(Object, genericParameters);
}
}
else
nextScope = currentScope.findLocalType(of.identifier, searchUp);
if (of.next == null || nextScope == null) return nextScope;

return findType(false, context, nextScope, of.next);
}


Ow! TypenameToken is a linked list of tokens, where a token is an identifier plus an optional list of generic arguments. This function does one thing of particular note; it looks for an instance of the generic type that takes the same parameter types. If it finds one, great, it just returns that one (This little detail will allow me to implement typed arrays of primitive types far more efficiently, assuming I do something to prevent a script from deriving from a primitive type). If not, it tells the script engine to compile it. The script engine will delay compilation until the current module is finished, but it will immediately create a scope for the generic type, register all it's members, and resolve all it's typenames. It has to do this right away so that whatever called TypenameNode.findType can finish it's compilation.

internal Scope EnqueueGenericTypeForCompilation(ObjectDeclarationNode decl, List genericParameters)
{
var into = new Scope();
var newDecl = decl.Clone() as ObjectDeclarationNode;
genericTypesAwaitingCompilation.Enqueue(new pendingGeneric
{
decl = newDecl,
genericParameters = genericParameters,
into = into
});

if (genericParameters.Count != decl.genericParamters.Count)
throw new CompileError("Wrong number of generic arguments.");
var newTypeName = newDecl.localScope.thisType + "<";
for (int i = 0; i < genericParameters.Count; ++i)
{
if (i != 0) newTypeName += ",";
newTypeName += genericParameters.fullTypeName;
}
newTypeName += ">";

into.thisType = newTypeName;
into.tag = newDecl;
newDecl.registerMembers(decl.localScope.ownerScope, into);

for (int i = 0; i < genericParameters.Count; ++i)
into.genericParameters.Type = genericParameters;

newDecl.resolveTypes(new ScopeStack(globalScope, into), into);
into.finalizeScope();
into.parent = null;
return into;
}


It is necessary to clone the entire abstract syntax tree for the generic type. This is because each node keeps some state that's set in the 'gather information' compile pass, and used in the 'emit bytecode' pass. Since we are just doing the first pass on the generic type, and saving the second pass for later, we need a copy of that information for each instantiation. If you tried to use TypedArray and TypedArray in the same module, whichever one was mentioned second would trample the state of the first. Wait, why can't we just compile it completely right away? Because a generic argument might be a type in the module that's using the generic type, and it's first pass isn't finished yet. Actually cloning the syntax tree is straightforward. I don't need to worry about the extra information that Irony puts into AstNodes, because it won't be used.

public object Clone()
{
var result = Activator.CreateInstance(this.GetType());
if (result is CompilableNode) (result as CompilableNode).AsString = AsString;
foreach (var child in this.ChildNodes)
{
var newInstance = (child as ICloneable).Clone();
if (child is CompilableNode) (child as CompilableNode)._cloneImpl(newInstance);
(result as AstNode).ChildNodes.Add(newInstance as AstNode);
}
return result;
}


A few types use _cloneImpl to do some extra state transfer, and a few override clone entirely. TypenameNode gets away with public object Clone() { return this; }!

After cloning the AST, I figure out the name of this instance so that later attempts to use this same generic with these same parameters won't repeat this effort. Finally, I get down to compiling. RegisterMembers, in addition to adding all the members and methods, populates a list of generic parameter names. I use this list to fill in the types with the generic parameters specified. Later, when asked for a type, Scope will look to see if the name asked for exists in this list. If it does, it won't search it's member types, it will just return whatever type is in the list. Now I resolve the generic's types. There's a terrible bug here I'll leave as an exercise to the reader, but here's a hint : Where is the module in that scope stack? I return the scope for TypenameNode to use as the generic type instance.

Finally, I can actually write TypedArray.

TypedArray {
array data;
void construct() { data = new array(); }
T __indexGet(int i) { return (data as T); }
T __indexSet(T t, int i) { return ((data = t) as T); }
void add(T t) { data.add(t); }
//etc
}


It exists in the 'Core' module, so you can use it like 'Core:TypedArray'.

Some closing thoughts
*This is probably the only scripting language to support generics before it supports for loops. But I'm glad for that, because I would have had to modify them anyway.
*When resolving the types of a function declaration, I had to reach into the function implementation and look for any generic types so they can be instanced before the function is compiled.
*You could technically use the type 'TypedArray' without any generic arguments. But none of it's methods will work, as it will never, and in fact can't, be compiled.
*These 'generics' are more like C++ templates than C# generics. However, the compilation model won't support a test-substitution without major changes so SFINAE will probably never be an option.
*The error messages produced by the compiler are entirely useless.
Deyja

DeyjaScript 8 : Modules

I try to keep the ultimate goal of this project, to be used in a mud engine, in mind when I decide what features to implement next. So I decide that modules are next. Actually compiling a self-contained module is straightforward. I changed the 'ExecutionContext' into a 'ScriptEngine'. I gave it a global scope. Each module will get it's own scope, that is contained within the global scope. They can co-exist just fine, but, they can't actually refer to each other in anyway. I need some way for types to be visible to other modules. The syntax for referring to a type in another module looks similar to C++'s namespace resolution operator. ModuleName:TypeName. I decided not to overload the . operator the way C# does because it would complicate the syntax (There are cases where it's impossible for the parser to distinguish between member access and namespace access).
I added the 'typename' to the grammar, and everywhere the grammar currently used an identifier as a typename, I replace it. And then I need to actually use the additional information.


class TypenameNode : AstNode
{
List Tokens = new List();

public override void Init(Irony.Parsing.ParsingContext context, Irony.Parsing.ParseTreeNode treeNode)
{
base.Init(context, treeNode);
foreach (var child in treeNode.ChildNodes)
Tokens.Add(child.FindTokenAndGetText());
AsString = String.Join(":", Tokens.ToArray());
}

internal Scope findType(ScopeStack context)
{
var type = context.topScope.searchForType(Tokens.ToArray());
if (type == null) throw new CompileError("Could not find type " + Tokens[0]);
return type;
}
}

//Inside Scope
public Scope searchForType(String[] tokens)
{
var type = findType(tokens[0], true);
if (type == null) return null;
if (tokens.Length > 1)
{
for (int i = 1; i < tokens.Length; ++i)
{
type = type.findType(tokens, false);
if (type == null) return null;
}
}
return type;
}


The node is not compilable. In all the cases where identifier nodes were used as typenames before, there was never any attempt to compile them. The typename node will remember the typename as specified in the source. FindType first searches up through the scopes for the first token, and then from that type down until all tokens are matched. Right now there's no way to declare a type more than one level deep, so I don't need to bother looking any further, but the functionality is there. And then I have to change everything to use this new method. Thankfully, only a few nodes use typenames. Declarations, new statements, and the as operation. Several other nodes compare typenames, and there I run into a problem. They need to know the full typename, with the module specified.
I give Scopes a 'fullTypeName' and keep track of it. Since only module and object level scopes survive the compile phase, those are the only ones I worry about. To get the built-in types to work again (Name resolution can't find them) I add some 'fake' modules.


globalScope.addType(new Scope { thisType = "int", moduleIndex = 0 });
globalScope.addType(new Scope { thisType = "string", moduleIndex = 1 });
globalScope.addType(new Scope { thisType = "void", moduleIndex = 2 });
globalScope.addType(new Scope { thisType = "bool", moduleIndex = 3 });


I should be able to add some member functions to built in types using this same technique.

There's an unrelated problem with the language as is. I can't call the base implementation of virtual functions. For virtual functions to work, the derived type's implementation must replace the base implementation in the function table. So I stick the base implementation on the end of the function table, and the derived implementation entry keeps a reference to it.

I create a special syntax for calling the base implementation. It looks like a function called 'base' is being called, but it's actually a special node. Listing the base call rule before function calls keeps them from conflicting, though it does mean it's impossible to call a function named 'base'. A bit of special-casing allows constructors to call any base constructor.


public override void gatherInformation(ScopeStack context)
{
var parameterTypes = new String[ChildNodes.Count];
for (int i = 0; i < ChildNodes.Count; ++i)
parameterTypes = (ChildNodes as CompilableNode).getResultType(context);

var containingFunction = context.topScope.searchForFunction();
var decoratedName = ScriptEngine.decorateFunctionName(containingFunction.AsString, parameterTypes);

if (containingFunction.AsString == "construct")
{
if (containingFunction.decoratedName == decoratedName)
function = containingFunction.thisFunc.BaseImplementation;
else
{
function = context.topScope.findFunctionByDecoratedName(true, true, decoratedName);
if (function != null && function.BaseImplementation != null)
function = function.BaseImplementation;
}
}
else
function = containingFunction.thisFunc.BaseImplementation;

if (function == null) throw new CompileError("Function does not exist in base");
if (ChildNodes.Count != function.ParameterCount - 1) throw new CompileError("Wrong number of arguments to base");
if (decoratedName != function.DecoratedName) throw new CompileError("Argument type mismatch");

foreach (var child in ChildNodes)
(child as CompilableNode).gatherInformation(context);
}


It's about time I implemented arrays. I think those will be next. I'm not exactly sure how I'll implement them. I want them to be dynamically sized, so they will need to have member functions. Which means they'll need to be a type. But I want to be able to create arrays of anything. I'll either have to drop a load of special case code into type resolution or implement some sort of generic type.
Deyja
This entry will be formatted as a series of features, and some notes on their implementation.

First, I created a proper scope stack. This allowed me to fix the problem of variables 'leaking out of' blocks. Code like this used to be possible.


if (condition) {
int x = 5;
}
x = 6; //Just fine!


It takes more work to prevent that sort of thing, but code like that is dangerous enough to warrant it. Lets see what could happen if I allowed it.


if (condition that is false) {
Foo foo = new Foo();
//code that uses foo
}
//many more lines of code
foo.doSomething(); //<- Sometimes foo has been initialized, sometimes it hasn't.


If I detected the use of uninitialized variables like C# does, I could make this particular case an error. But it's easier to just prevent leaking. So, each block pushes a scope onto the scope-stack. Statements will search up the scope stack for the variables they reference. When the function scope is closed, it assigns ids to all the local variables used in all it's sub scopes. It doesn't need a unique id for every variable, it only needs to know the maximum number of variables used at once.

In this example, x and y will be given the exact same id.

if (condition) {
int x;
} else {
int y;
}


All's not well, though, because a naked statement is not a block. There's no block node involved to push scopes. So if I were to do like so


if (condition)
int x;


not only would I be accomplishing absolutely nothing, I'd be leaking x into the enclosing scope. Then consider a for statement, which often has a variable declaration in it, outside the block. I would leak that as well. The solution is for these control structures to also push scopes. In the case of an if statement with a block rather than a single statement, two scopes will be pushed. One of them will remain empty, but that's fine, all of these function-level scopes are discarded when function compilation is finished.

Next I added constructors. This was simple. I modify the grammar so that a new statement expects an argument list, and the node looks for a method on the type called 'construct' and that takes those parameter types. If the constructor was found, it emits bytecode to call the method. If it wasn't and there are no arguments, that's fine. If it wasn't found and there are arguments, it's an error. So all objects have a default constructor that does nothing.

The next feature I implemented was enabled by this scope-stack addition, and it was inheritance. Inheritance can be complicated, so I started with some simple restrictions. First, objects can only derive from one base. This means that I don't need to figure out where in an object's memory layout a particular base is at. The base is always first. I also check for and prevent base loops, where B derives from A, and A derives from B. This is only even possible because I allow objects to be declared in any order. Second, all methods are virtual. There is no additional cost for a virtual function since all method calls already involve looking up the function in a table. My particular implementation, however, prevents calling the base type's version of the method (Something I may want to support, particularly for constructors).

First I inherit member variables. Memory layout is a non-issue. Members of the base are just listed first. For example


A {
int x;
int y;
}

B is an A {
int z;
}


Here, in an instance of B, z will have an index of 2. If I convert a B to an A, x and y will still be in the right place; there will just be an extra member hanging on the end. The scope keeps track of how many member variables it has total so that ScriptObject can make enough space for all of them. I modify the scope to look for variables in base types too (Variables with the same name in derived scopes will hide base members) and it works.

Next, I want to inherit functions. This is a little more complicated because, unlike variables, I actually need to be able to lookup the function at runtime. So I still list base member functions first, but I duplicate the function list into the derived scope, and then add the local functions to it. When I find a local function with the same decorated name as a base function, I replace the base function entry. Since the dynamic type of a script object is stored in the script object, and base members are listed first, I can treat a B as an A, call a method from A, and it will have the same index in B's function list.

I think I have enough now to move on. Next time, I'll see about making sure I can call base implementations of virtual functions. It might also be a good idea if not every function was virtual. And then it's time for modules.
Deyja

Source for the previous entry

I fixed those design flaws I talked about in https://www.gamedev.net/blog/342/entry-2250653-deyjascript-6-member-variables-and-design-flaws/ so here is the source. There isn't actually anything to say about it, as it doesn't do anything it didn't already do. Except string literals.

Some changes
-Functions now get their own chunk of bytecode
-Declaration nodes no longer double as scopes; they have scopes.

Some problems
-types can only exist in the global scope, or the new instruction can't find them. That's not a problem now, but if I try to support inner classes it will be a problem.

Next on the list, what I just said, and constructors.
Deyja
Installment six, in which I implement member variables and the whole thing starts to unravel. I always find that there's a certain point in a project where the abstractions that I've chosen suddenly show their flaws, and this is that point. A series of kludges, rather than proper design, makes this work.

Declaring member variables is straightforward. I add them to the grammar, and I add member access too. In order to be used as a member variable, VariableDeclarationNode needs to implement MemberNode. No big deal, that interface is simple. I need somewhere to put these values, so I create a ScriptObject class and I modify the new instruction to create it. Thinking ahead, I create an instruction for storing the top of the stack in a member, and for pushing a member onto the stack. Both need to find the this pointer on the stack.
First I modify identifier and assignment to support assignment to members with an implicit this, and here's the first hitch. I have to create a whole new class hierarchy to represent different types of variables (Types as in local, parameter, and member). To search for the variable I have to check the function scope and then the object scope. There is another hole. I should have some kind of scope-stack instead. Once I have the variable I am assigning to, the only way to tell if it is a member variable, and I need to push the this parameter, is a type-check.
Some of the problems are caused by Irony itself. I don't see any way for a rule to create one node type in some cases and another in others, so nodes either have to do double duty, or I have to duplicate rules.

No source this time. But here's almost all of the code I had to add to support member variables.


case Instruction.pushMemberToStack:
{
var stackTopPointer = getTopPointer();
var @this = stack.fetch(stackTopPointer - 1) as ScriptObject;
var memberIndex = codePointer.bytecode[codePointer.index + 1];
stack.store(stackTopPointer - 1, @this.members[memberIndex]);
codePointer.index += 2;
}
break;

case Instruction.loadMemberFromStack:
{
var stackTopPointer = getTopPointer();
var @this = stack.fetch(stackTopPointer - 1) as ScriptObject;
var value = stack.fetch(stackTopPointer - 2);
var memberIndex = codePointer.bytecode[codePointer.index + 1];
@this.members[memberIndex] = value;
codePointer.index += 2;
}
break;


For next time, I will be creating a proper scope-stack. This might even allow me to support objects-in-objects, or objects-in-functions. The only catch being that, for the new instruction to be able to create them, they need to be collapsed into a single list of types somewhere. See you then.

Edit - It looks like someone was paying attention and changed the community journal listing so it would only show the latest entry from any given user. Bravo; now I can throw these out lightning quick without feeling guilty about it.
Deyja
I have to say, after getting to the fifth installment in about a week, that I like this style of 'journaling' 'tutorials'. I'm not writing what you should do to create a scripting language, instead I'm just saying what I have done. It also forces me to find a single isolated feature and implement it. I did something similar once before, except then I wrote what I was going to do, and then did it, rather than the other way around. It seems to work well either way. Code gets written, information gets dispensed, certain people hate it.

The goal all along has been to create an object-oriented language. Last time, we ended with a list of functions that could all call each other. There was a lot crammed into that last very short installment, more than I realized. The basic function calling mechanism is in place. Now I transition from a list of functions to an object with methods, and then I go immediately to a list of these objects. The grammar is simple.


objectDeclaration.Rule = identifier + "{" + memberList + "}";
objectList.Rule = MakeStarRule(objectList, objectDeclaration);
newStatement.Rule = ToTerm("new") + identifier;
methodCall.Rule = expression + "." + identifier + "(" + parameterList + ")";


Oh yeah, and new statements and method calls. Notice that objects just have a name, there's no way to pass arguments when calling new, etc. Those are all features I'll add in time. I was a little worried that the methodCall rule would cause a conflict, but Irony hasn't complained and seems to be parsing it correctly.

I create an object list node that acts as the new root node of the program, and allows me to find types by name. Now the execution context (perhaps I should separate execution and compilation contexts?) has three scopes, global, object, and function. I need to change this to some sort of scope stack. That would help me resolve the identifier leaking problem, too. Excuse me for rambling, but take a look at this -


int foo(int x)
{
if (x == 5)
{
int bar = 6;
}
return bar;
}


That compiles. And runs. And if x happens to be 5, it actually works. But if x isn't five, it will return an uninitialized value, which usually results in the VM dieing suddenly and violently.

The object list node is not a compilable node. That abstraction leaked far enough. Instead, it supplies a single 'compile' function, since it has (almost) all the information it needs to compile already. This compile function first gathers up the names of all the objects and their methods, then it allows all it's children CompilableNodes to gather the information they need to compile, and then it allows them all to emit bytecode. The list to emit the bytecode into isn't provided to the function. Instead, each object gets it's own bytecode (methods are still all smashed into one big buffer). I made this change so I could more easily support modules later.

It means a major change in the VM, though. Instead of just a set of bytecode and an instruction pointer, it needs multiple sets of bytecode and instruction pointers. Since the stack is a stack of Objects, when I call a function, instead of pushing just the return address, I push the bytecode/return address pair. The callMethod instruction works basically the same as callFunction.


case Instruction.callMethod:
{
var stackTopPointer = getTopPointer();
var @this = stack.fetch(stackTopPointer - 1) as ObjectDeclarationNode;
stack.store(stackTopPointer, codePointer);
setTopPointer(stackTopPointer + 1);
var methodIndex = BitConverter.ToInt16(codePointer.bytecode, codePointer.index + 1);
codePointer.index += 3;

codePointer = new CodePointer
{
bytecode = @this.bytecode,
index = @this.functions[methodIndex].firstInstruction
};
}
break;


Special thanks to MikeP for teaching me about @ and identifiers in C#, even if he did describe this project as a complete waste of everyone's time. I'll abuse @ in his honor.

Now I have to implement MethodCallNode, which will be an almost direct copy of FunctionCallNode. Combining them would be difficult enough that I didn't. First, I make changes to FunctionCallNode. Since everything is a method, if the function called isn't a system function, it needs an implicit this parameter. Thankfully, I already added a 'this' parameter to every function declaration automatically, and it's always the last parameter, so it's always at index -3. (-2 is the return point, -1 is the old framepointer, 0 is the first variable declared in the function) So when the function being called isn't a system function, FunctionCallNode pushes this and treats it like a method call.

MethodCallNode requires an expression first to tell which object to call the function on. The callMethod instruction assumes that the object the method is being invoked on is at the top of the stack, so first I gather all the parameters, and then I evaluate the leading expression. There is, of course, static type checking when I lookup the function.


internal FunctionDeclarationNode lookupFunction(ExecutionContext context)
{
var invokeOn = (ChildNodes[0] as CompilableNode).getResultType(context);
var objectDeclaration = context.currentGlobalScope.findObject(invokeOn);
if (objectDeclaration == null) throw new CompileError("Can't invoke methods on this type.");

var argumentTypes = new String[ChildNodes.Count - 1];
for (int i = 1; i < ChildNodes.Count; ++i)
argumentTypes[i - 1] = (ChildNodes as CompilableNode).getResultType(context);

var func = objectDeclaration.findFunction(AsString, argumentTypes);
return func;
}

internal override string getResultType(ExecutionContext context)
{
var func = lookupFunction(context);
if (func == null) throw new CompileError("Function not found (GRT)");
return func.getResultType(context);
}

public override void gatherInformation(ExecutionContext context)
{
var func = lookupFunction(context);
if (func == null) throw new CompileError("Function not found (GI)");
function = func;
foreach (var child in ChildNodes)
(child as CompilableNode).gatherInformation(context);
}

public override void emitByteCode(List bytecode, ExecutionContext context, bool placeResultOnStack)
{
for (int i = 1; i < ChildNodes.Count; ++i)
(ChildNodes as CompilableNode).emitByteCode(bytecode, context, true);
(ChildNodes[0] as CompilableNode).emitByteCode(bytecode, context, true);
function.EmitCallBytecode(bytecode);
bytecode.AddBytecode(Instruction.popN, (byte)ChildNodes.Count); //pop parameters
if (placeResultOnStack) bytecode.AddBytecode(Instruction.pushRegisterToStack, (byte)Register.returnValue);
}


An aside, apparently C# supports zero-length arrays.

To actually test this, I need objects to invoke methods on. So I create a new instruction. For now, it just shoves the object declaration onto the stack, effectively making every object static. The next step is probably to have actual instances. Afterwards, I will have to eventually support static method invocation.

For testing, I write a quick way to invoke methods on a script object.


public void callFunction(VirtualMachine VM, ExecutionContext context, int func)
{
var wrapperCode = new List();
wrapperCode.AddBytecode(Instruction.callMethod, BitConverter.GetBytes((Int16)func));
wrapperCode.AddBytecode(Instruction.exit);
VM.registers[(int)Register.stackTopPointer] = 1;
VM.registers[(int)Register.framePointer] = 0;
VM.stack.store(0, this);
VM.execute(wrapperCode.ToArray(), context, 0, true);
}


It takes advantage of the change to multiple bytecode buffers. That last argument to execute tells the VM not to setup the stack or registers. One last thing, I get rid of the callFunction instruction and the function table at the start of the bytecode entirely, as they are no longer used.
Deyja

DeyjaScript 4 : Functions

In this installment I continue to reinvent the wheel by implementing function calls. First I write the grammar, and quickly implement calling system functions, since they already exist.


parameterList.Rule = MakeStarRule(parameterList, ToTerm(","), expression);
functionCall.Rule = identifier + "(" + parameterList + ")";
parameterDeclaration.Rule = identifier + identifier;
parameterListDeclaration.Rule = MakeStarRule(parameterListDeclaration, ToTerm(","), parameterDeclaration);
functionDeclaration.Rule = identifier + identifier + "(" + parameterListDeclaration + ")" + block;
member.Rule = functionDeclaration;
memberList.Rule = MakeStarRule(memberList, member);
returnStatement.Rule = ToTerm("return") + (Empty | expression);


In order to call functions defined in scripts, I need to be able to define functions in script. The root node of the grammar becomes a list of function definitions instead of a list of statements. I also added parsing of return statements.
Calling a function is fairly straightforward. I create a 'callFunction' instruction that does the important dirty work. It looks up the function in the function table (which I stash at the beginning of the bytecode), pushes the return address, and jumps to the first instruction of the function. A companion return instruction pops the return address from the stack and jumps to it.

case Instruction.callFunction:
{
var stackTopPointer = getTopPointer();
stack.store(stackTopPointer, instructionPointer + 3);
setTopPointer(stackTopPointer + 1);
var functionIndex = BitConverter.ToInt16(bytecode.GetRange(instructionPointer + 1, 2).ToArray(), 0);
var functionFirstInstruction = BitConverter.ToInt16(bytecode.GetRange(functionIndex * 2, 2).ToArray(), 0);
instructionPointer = functionFirstInstruction;
}
break;

case Instruction.returnFromFunction:
{
var stackTopPointer = getTopPointer();
var returnAddress = (stack.fetch(stackTopPointer - 1) as int?).Value;
setTopPointer(stackTopPointer - 1);
instructionPointer = returnAddress;
}
break;


To implement parameters, I do much the same as I did for variables inside the functions and assign ids to them all in advance. In this case, the ids are negative (which also means I need to offset them when writing the bytecode since the bytecode is all unsigned bytes) because they will be on the stack below the frame pointer.
Every function needs a chunk of entrance and exit code to maintain the frame pointer and make space for variables.


context.currentFunctionScope = this;
this.firstInstruction = bytecode.Count;
bytecode.AddBytecode(Instruction.pushRegisterToStack, (byte)Register.framePointer); //Remember the previous frame pointer
bytecode.AddBytecode(Instruction.pushRegisterToStack, (byte)Register.stackTopPointer); //Set our frame pointer
bytecode.AddBytecode(Instruction.loadRegisterFromStack, (byte)Register.framePointer);
bytecode.AddBytecode(Instruction.pop); //pop the new framepointer value - At this point, the old framebuffer is the stack top
if (localVariableCount > 0) bytecode.AddBytecode(Instruction.pushN, (byte)localVariableCount); //push space for variables
(ChildNodes[1] as CompilableNode).emitByteCode(bytecode, context, false); //function implementation

int returnDestination = bytecode.Count;

bytecode.AddBytecode(Instruction.pushRegisterToStack, (byte)Register.framePointer);
bytecode.AddBytecode(Instruction.loadRegisterFromStack, (byte)Register.stackTopPointer); //pops everything above the framepointer
bytecode.AddBytecode(Instruction.loadRegisterFromStack, (byte)Register.framePointer);
bytecode.AddBytecode(Instruction.pop); //pop framepointer from stack
bytecode.AddBytecode(Instruction.returnFromFunction);

foreach (var ret in returnStatements)
{
var relativeJump = returnDestination - ret;
var bytes = BitConverter.GetBytes((Int16)relativeJump);
bytecode[ret + 1] = bytes[0];
bytecode[ret + 2] = bytes[1];
}


To implement return statements, I keep track of every return statement I encounter. The return is implemented as a simple constantRelativeJump to the end of the function. I just have to go back and fill in the jump distance.

I mentioned a function table earlier. What I did was assign all the functions an ID, and smack their first instruction into a table. But I think I'm going to change that, it's not going to work so well when I start doing objects and inheritance and virtual functions.
Deyja
Time now for control structures. But first, I made some changes. I implemented registers, and I stashed the frame pointer and the stack pointer in there. The registers, like the stack, is just an array of objects. It's inefficient for integral types, but it allows the VM to pretty much ignore types. It still assumes that the frame pointer and stack pointer registers contain ints, of course. I just have to make sure I don't emit any bytecode that violates this. I also add some instructions for dealing with them, such as loadRegisterFromVariable. I'm using long names for my instructions because there's no reason to use short, unreadable nonsense like you would see in an assembly language. So now I have three places the bytecode can store data. Variables, the top of the stack, and registers.
I also go ahead and define a function calling convention. The only functions being called right now are operators, but having a clearly defined calling convention will be important shortly. I don't know if this convention is named. Here it is - Parameters are pushed onto the stack in order. They are not popped by the calling function, and the return value is placed in the returnValue register. A bit of parameterization of the ExpressionNode's emit function lets me tell any arbitrary node whether or not to push it's return value onto the stack, but that push is now a separate instruction.
Now on with the grammar. I separate the idea of a list of statements and a block. I allow a code block to be used as a statement, but not an expression. For control structures, I'll need a jump instruction. Just two, actually. constantRelativeJump and constantRelativeJumpIfNotTrue. Why jump on false? So I can emit the then block first. Those are straightforward. I also add an operator== for ints so that I can actually try it out. Finally, the grammar, and the first real hurdle.


ifStatement.Rule = ToTerm("if") + "(" + expression + ")" + statement;
ifElseStatement.Rule = ToTerm("if") + "(" + expression + ")" + statement + this.PreferShiftHere() + "else" + statement;


I'd stumbled directly into something called the 'if-then-else ambiguity'. A bit of research told me why Irony was complaining, and led to that call to PreferShiftHere which took care of the problem. Here's some more information about it. http://docs.freebsd.org/info/bison/bison.info.Shift_Reduce.html This really hi-lites something I should probably mention, and that is that I have no idea what I am doing. That's part of the reason for blogging this whole experience. If I'm doing something obviously wrong, I'd hope someone would let me know.
So all that's left is to actually implement IfStatementNode.


public override void emitByteCode(List bytecode, ExecutionContext context, bool placeResultOnStack)
{
var hasElseClause = ChildNodes.Count == 3;
(ChildNodes[0] as ExpressionNode).emitByteCode(bytecode, context, false); //Leaves result in returnValue register
int jumpInstruction = bytecode.Count;
int elseSkipJumpInstruction = 0;
bytecode.AddBytecode(Instruction.constantRelativeJumpIfNotTrue, (byte)Register.returnValue, (byte)0, (byte)0);
(ChildNodes[1] as ExpressionNode).emitByteCode(bytecode, context, false);
if (hasElseClause)
{
elseSkipJumpInstruction = bytecode.Count;
bytecode.AddBytecode(Instruction.constantRelativeJump, (byte)0, (byte)0);
}
var jumpDistance = bytecode.Count - jumpInstruction;
var bytes = BitConverter.GetBytes((Int16)jumpDistance);
bytecode[jumpInstruction + 2] = bytes[0];
bytecode[jumpInstruction + 3] = bytes[1];
if (hasElseClause)
{
(ChildNodes[2] as ExpressionNode).emitByteCode(bytecode, context, false);
jumpDistance = bytecode.Count - elseSkipJumpInstruction;
bytes = BitConverter.GetBytes((Int16)jumpDistance);
bytecode[elseSkipJumpInstruction + 1] = bytes[0];
bytecode[elseSkipJumpInstruction + 2] = bytes[1];
}
}


It has to go back and fill in the jump instructions once it knows just how far to jump.
Deyja

DeyjaScript 2 : Variables

Next I'm going to add variables. First I'll add identifiers and variable declarations to the grammar. I won't be supporting any sort of initialization yet, so I don't need to add that to the grammar. I also go ahead and create a statement rule, and a statement block. Later I added assignment statements.


var identifier = TerminalFactory.CreateCSharpIdentifier("identifier");
identifier.AstNodeType = typeof(IdentifierNode);

var variableDeclaration = new NonTerminal("Variable Declaration", typeof(VariableDeclarationNode));
var statement = new NonTerminal("Statement");
var statementBlock = new NonTerminal("Statement Block", typeof(StatementBlockNode));
var assignment = new NonTerminal("Assignment", typeof(AssignmentNode));

expression.Rule = binaryOperation | numberLiteral | parenExpression | identifier;
assignment.Rule = identifier + "=" + expression;
variableDeclaration.Rule = identifier + identifier + ";";
statement.Rule = (assignment + ";") | variableDeclaration;
statementBlock.Rule = MakeStarRule(statementBlock, statement);


Hey, I got the featured article. On the very first one. The day after I posted it. I also noticed the website just takes the beginning of the article and uses it on the front page, so I tried to put something on topic right at the beginning instead of this.

Variables will be stored on the stack. In order to know how much stack space to reserve, I need to count how many variables are declared. I can do this in one pass or two passes. I choose two because it seems more natural to me, because it was the first way to occur to me, and because I didn't think of the other way until just now. So I split the compilation into two phases.


public class ExpressionNode : AstNode
{
public virtual void gatherInformation(ExecutionContext context) { throw new NotImplementedException(); }
public virtual void emitByteCode(List bytecode, ExecutionContext context) { throw new NotImplementedException(); }
internal virtual String getResultType(ExecutionContext context) { throw new NotImplementedException(); }
}


To count the variables, and to look up variables by name, I need to keep track of them. I created a 'Scope' class with information about the variables, and gave context a value 'currentScope'. In the new StatementBlockNode, I set the currentScope in the first compilation path. VariableDeclarationNodes add variables to it, and IdentifierNodes lookup variables by name. These are the two compilation phases for the IdentifierNode, which will just push the correct variable onto the top of the stack.


public override void gatherInformation(ExecutionContext context)
{
var variable = context.currentScope.findVariable(AsString);
if (variable == null) throw new CompileError("Undefined variable");
variableIndex = variable.variableIndex;
}

public override void emitByteCode(List bytecode, ExecutionContext context)
{
bytecode.Add((byte)Instruction.pushVar);
bytecode.Add((byte)variableIndex);
}


I can now declare variables and use them in expressions, but if I actually try to I will trigger an exception. As of right now, trying to use an unitialized variable will make the virtual machine throw a null reference exception, and there isn't actually any way to initialize a variable. I need an assignment operator. Assignment is technically a binary operation, but it's handled very differently, so I won't implement it as one. First, the left-side of an assignment operator must be an 'LValue'. So I create an LValue interface. Any ExpressionNode that implements this interface is an LValue, and can be assigned to.


internal interface LValue
{
void emitAssignmentByteCode(List bytecode, ExecutionContext context);
}


Because only ExpressionNodes will ever be LValues, I will assume the node does error checking for both usages in gatherInformation. This is true of IdentifierNode, and if it proves an impossible assumption I will change it. I implement this interface in IdentifierNode.


void LValue.emitAssignmentByteCode(List bytecode, ExecutionContext context)
{
bytecode.Add((byte)Instruction.popVar);
bytecode.Add((byte)variableIndex);
}


And then I move on to AssignmentNode. It looks very similar to BinaryOperationNode, except that I never call emitByteCode on the first child. I emit the bytecode for the second child instead, and then I call emitAssignmentByteCode on the first child. I assume that the second child leaves it's result on the stack.


public override void gatherInformation(ExecutionContext context)
{
var LValue = ChildNodes[0] as LValue;
if (LValue == null) throw new CompileError("Only assignment to LValues is allowed.");
if ((ChildNodes[0] as ExpressionNode).getResultType(context) != (ChildNodes[1] as ExpressionNode).getResultType(context))
throw new CompileError("Assignment type mismatch");
(ChildNodes[0] as ExpressionNode).gatherInformation(context);
(ChildNodes[1] as ExpressionNode).gatherInformation(context);
}

public override void emitByteCode(List bytecode, ExecutionContext context)
{
(ChildNodes[1] as ExpressionNode).emitByteCode(bytecode, context);
(ChildNodes[0] as LValue).emitAssignmentByteCode(bytecode, context);
}


Implementation of the new instructions is straight forward. The biggest change to the virtual machine is that I change how the stack is addressed, and I add a frame pointer which, for now, is always 0. I write a new test script - "int foo; foo = 6 * 6;" and compile and run it. Afterwards, the stack is empty, but since the values are never actually cleared I can still pull the value of foo out of the stack and confirm that it is, in fact, 36.

There's a few gaps in assignment, however. I can't chain assignments. Foo = bar = 5; will not compile. Also, I can't use assignments as an expression. It might be a good idea not to support the latter, but I want to support it. There's another problem; expressions can leave garbage on the stack. It's not possible now, but only because I only allow declaration and assignment in statements. It will be a problem when I implement functions. I'll clean those up but I probably won't mention them in the next installment, because next time, I'm going to work on embedded code blocks, the first step in implementing control structures.

DeyjaScript02.zip
Deyja
Lets make a mud! Why? Why not? I love muds, which is very odd because one thing I do not love is actually playing muds. They are sort of a poor-man's MMORPG, an MMO you actually can run out of your living room. It amazes me that there aren't more people making them. I've actually built a mud engine from scratch before. Four times, in fact, which will make this the fifth. This time around, I'm going to do it in the style of an LPMud. In an LPMud, each object in the game world is a 'class' in a script file. The LPMud is really a shared terminal to a compiler and interpreter. The game is a library written in the LPMud language. This won't be quite so primitive, but it will need a scripting language of some kind. So, at the very least, this is an excuse to write a scripting language, and I might as well start there.

So first I create some projects, and I download Irony from codeplex - http://irony.codeplex.com/ . Then I start defining the grammar in Irony.


var numberLiteral = TerminalFactory.CreateCSharpNumber("number");
numberLiteral.AstNodeType = typeof(IntLiteralNode);

var expression = new NonTerminal("Expression");
var parenExpression = new NonTerminal("Paren Expression");
var binaryOperation = new NonTerminal("Binary Operation", typeof(BinaryOperationNode));
var _operator = ToTerm("+") | "-" | "*" | "/";

expression.Rule = binaryOperation | numberLiteral | parenExpression;
binaryOperation.Rule = expression + _operator + expression;
parenExpression.Rule = ToTerm("(") + expression + ")";


I start with the most basic constructs in the language. This can parse basic math expressions using the +, -, * and / operators and integer literals. There's a little more Irony needs to parse it correctly that I won't go over here. When Irony parses my script, it creates an Abstract Syntax Tree. The tree is made of BinaryOperationNodes and IntLiteralNodes. Each of these is an expression node.


public class ExpressionNode : AstNode
{
public virtual void compile(List bytecode, ExecutionContext context) { throw new NotImplementedException(); }
internal virtual String getResultType(ExecutionContext context) { throw new NotImplementedException(); }
}


At this point, I could lay out a set of operation codes that includes things like 'add' and 'mul', but instead I am going to implement a 'system_call' operation code. I'll need some way to register 'system functions'.


public class ExecutionContext
{
internal Dictionary systemFunctions = new Dictionary();
internal List systemFunctionTable = new List();

internal String decorateFunctionName(String name, params String[] parameterTypes)
{
return name + String.Join("@", parameterTypes);
}

public void addSystemFunction(
String name,
String returnType,
Func implementation,
params String[] parameterTypes)
{
var systemFunction = new SystemFunction();
systemFunction.name = name;
systemFunction.returnType = returnType;
systemFunction.parameterTypes = parameterTypes;
systemFunction.systemFunc = implementation;
systemFunctions.Add(decorateFunctionName(name, parameterTypes), systemFunction);

systemFunction.funcTableId = systemFunctionTable.Count;
systemFunctionTable.Add(systemFunction);
}
}


The 'system functions' take an array of objects and return an object. They are terribly generic. My language's stack is a stack of objects. This makes implementing the 'systemcall' opcode fairly straightforward.


case Instruction.systemcall:
{
var _funcID = BitConverter.ToInt32(bytecode.GetRange(instructionPointer + 1, 4).ToArray(), 0);
var _func = context.systemFunctionTable[_funcID];
var parameterArray = new object[_func.parameterTypes.Length];
for (int i = 0; i < parameterArray.Length; ++i)
parameterArray = stack.fetch(parameterArray.Length - i);
for (int i = 0; i < parameterArray.Length; ++i)
stack.pop();
stack.push(_func.systemFunc(parameterArray));
instructionPointer += 5;
}
break;


It looks up the function, then it copies the arguments out of the stack, pops the arguments off the stack, and pushes the result. The index '1' always points to the top of the stack, '2' points to the next thing down, and so forth, and it assumes the arguments were pushed in order, so the first argument is at 'argument count' and the next one is at 'argument count - 1'. The BinaryOperationNode will use this opcode to implement itself.


public override void compile(List bytecode, ExecutionContext context)
{
var func = context.lookupFunction(AsString, (ChildNodes[0] as ExpressionNode).getResultType(context),
(ChildNodes[1] as ExpressionNode).getResultType(context));
if (func == null) throw new InvalidOperationException("Operator not defined");
(ChildNodes[0] as ExpressionNode).compile(bytecode, context);
(ChildNodes[1] as ExpressionNode).compile(bytecode, context);
bytecode.Add((byte)Instruction.systemcall);
bytecode.AddRange(BitConverter.GetBytes(func.funcTableId));
}


First it makes sure the operation actually exists. Then it compiles each expression. It assumes these expression leave their result on the stack. So after two run, there should be two results on the stack. I do need to report errors better.

So I can parse simple expressions, and compile them into bytecode, and execute that bytecode. So the last thing to do is test it. First I setup a context and define some operators.


var context = new DeyjaScript.ExecutionContext();

context.addSystemFunction("*", "int", (parms) => { return (parms[0] as int?).Value * (parms[1] as int?).Value; }, "int", "int");
context.addSystemFunction("/", "int", (parms) => { return (parms[0] as int?).Value / (parms[1] as int?).Value; }, "int", "int");
context.addSystemFunction("-", "int", (parms) => { return (parms[0] as int?).Value - (parms[1] as int?).Value; }, "int", "int");
context.addSystemFunction("+", "int", (parms) => { return (parms[0] as int?).Value + (parms[1] as int?).Value; }, "int", "int");


This is a terrible way to implement basic integer math operators but they work. Did I mention this is terrible? Next I parse the script and compile it.


var grammar = new DeyjaScript.Grammar();
var parser = new Irony.Parsing.Parser(grammar);
var program = parser.Parse("(4 * 4) / (8 - 6) + 6 - 3 * 4");
var rootNode = program.Root.AstNode as DeyjaScript.ExpressionNode;

var bytecode = new List();
rootNode.compile(bytecode, context);


And finally I can execute it. There should be a single thing left on the stack, the result of the expression.


var VirtualMachine = new DeyjaScript.VirtualMachine();
VirtualMachine.execute(bytecode, context);
Console.WriteLine("Result : " + VirtualMachine.getStackTop().ToString());


The result? 4!

Wait.

So I'll stick the source code so far on this somehow. And next time I'll expand on the language a bit with, I don't know; lets say variable declarations. It's hard to add just one thing at this point, the language sort of needs everything at once to do anything interesting.
Deyja
I've copied this from my 'real blog', at jemgine.omnisu.com, as I almost always do. Perhaps, though, I should claim this as the 'real blog', since it has far more readers. This is about my 2d game engine, Jemgine, which I talk about all the time and nobody cares about. It's written in C# with XNA, and I guess I sort of assumed anyone reading this already knew a bit about the engine. In Jemgine, entities are made up of many elements grouped together. Mostly polygons and sprites, but also occasionally lines and circles and 'quads', which are sort of a hybrid of a polygon and a sprite. Anyway, the original blog entry...


Compiling an entity into a single vertex buffer-


This is something I've wanted to do in Jemgine for a long time. Currently (Well, until last night) to draw an entity (And, I mean every entity) meant iterating over all of an entity's attached elements, discovering if they were renderable, and then drawing them. Drawing each attached element usually involved at least a texture switch, and a draw call - sometimes more than one! Drawing a lot of entities was not an option. As it stands, it never became a problem for me. It still worked plenty fast enough. But I didn't like it, and I imagined having a static background entity made of thousands of sprites, something quite impossible with the existing method. I already knew a better way, it's just a matter of actually doing it. And, actually, it's not that hard at all. Keep in mind that this is optional, as it breaks many things. Any given entity can be compiled to a single vertex buffer, or drawn using the brute force method.

Here's how it's done in basic steps:
At build time,
-Gather all the textures used by the entity. One thing I did not account for in this stage (or any others) is elements whose' texture changes. Texture-switching effects will have to be achieved using the old brute-force rendering method. I also didn't account for Lines and Splines having a default texture, or round ends. These textures won't be included.
-Combine all of those textures into a single atlas texture. There's a problem here, if the texture created exceeds the maximum size the hardware can support. It also potentially wastes, via duplicating textures, video memory. Not much to do about it. I could build a whole-map texture atlas, but imported maps would still have their own atlas, with their own potentially duplicated textures; and the whole-map atlas would be much more likely to be too big. To pack the textures, I use a simple BSP subdivision scheme. There are better algorithms. I especially like the one from this paper http://clb.demon.fi/files/RectangleBinPack.pdf which subdivides the free space left from each placement into two overlapping rectangles. I might implement something like this in the future.
-Now I go through every element and get from it a vertex list in the form of a triangle strip. I use a strip because most of the elements were already rendered using one, so the data was there ready to go. The vertex indexes are implicit in the triangle strip as well. Each element also has a color and a bone id. To the engine, these properties apply to the whole element. For the final shader, they have to be duplicated across every vertex that makes up that element. I use the texture atlas to re-map texture coordinates.

-Now, this list of verticies and indexes, and the texture atlas, are serialized to disk. The texture atlas has not been rendered yet, it's just a set of filenames and rectangles so far. I do this partly because I don't know how to get XNA's content pipeline to store generated textures and vertex buffers (I know it can) and partly because the texture atlas is much smaller stored this way.

At load time,
-Deserialize everything.
-Load the individual textures in the atlas and use a render target and spritebatch to compose them into a single texture.
-Create the vertex buffer and index buffer.

Each instance of an entity will share the atlas texture and the vertex buffers, assuming they have been instanced properly. Though, I should check on that. I might not be creating the objects at all for imported maps!

At render time,
-Collect bone transforms. An entity can still have multiple skeletons, but it must have no more than 32 bones total. This is a mostly arbitrary constant. To continue running on shader model 2.0, and thus in the reach profile, I can't have more than 256 shader constants. Each bone transform represents, I think, 4 constants.
-Set the world matrix to the entity's transform.
-Draw the entity in a single draw call!

Deyja

Immediate Mode Gui

So I wanted a simple GUI library for XNA, but all the ones I found sucked. Let me preface this by saying that there are some very good gui libraries out there for XNA, if what you want is a set of traditional controls. I don't. Some of these libraries suck, some of them don't, but what they all have in common is their complexity and depth. This is not one of the cases where depth is good for me. I've built 'traditional' guis before, I even wrote a very large gui library in SDL. I called it gum (because gum is gooey, get it?) and used it for several things.

jileed.jpg

It had just about every control imaginable. So with that experience, I know that that is not the road I want to go down. Instead I want to explore a new fad called an Immediate Mode gui. That other kind, with it's hierarchy of controls, is called Retained Mode, because the state is retained in the controls. The controls are actually objects that continue to exist, and to change the gui, you manipulate this set of objects. The biggest difference in an Immediate Mode gui is that there are no controls.

Some of the advantages an Immediate Mode gui has over the other way of doing things are, in a large app with a complicated gui like a map editor or Word, nothing at all. For these big complicated maps, the Retained Mode methodology of using nested controls and event callbacks works very well and there's no need to shake things up. Immediate Mode guis do not scale well to more complex projects. They can't hide the complexity.

That's enough rambling about theory. There's plenty of that on the internet already. Most of the things that made the retained mode libraries I examined so complex no longer matter. I don't need fancy controls (I might only need buttons), I don't need floating windows that the user can drag and drop. I don't even need to be able to skin it, instead I just draw it. What's seriously lacking on the internet is implementations, and since I need so little, the implementation is simple, too.

I'm going to start with a simple main menu. It has a bunch of buttons on the left. Clicking a button changes the panel on the right (Opening the options screen, for example). I don't know how to present this besides dumping some code. So here is some code.


var Imgui = new Gui.Driver(prevMouseState, mouseState);
var BorderLayout = new Gui.BorderLayout(Graphics.Viewport.Bounds, 20);

Imgui.Button(BorderLayout.PositionItem(Gui.BorderLayout.Sides.West, 276), (A, B) =>
{
spriteBatch.DrawBorder(background, A, new Rectangle(33, 33, 190, 9), Color.White);

var Layout = new Gui.FlowLayout(A, 10);

Imgui.Button(Layout.PositionItem(256, 77),
(C, D) => {
//Draw the button.
spriteBatch.Draw(buttonPlayAlone, C, D ? Color.Plum : Color.White); },
() => { /* Do Stuff if the button is clicked */ });
Layout.ForceNewLine();
Imgui.Button(Layout.PositionItem(256, 77),
(C, D) => { spriteBatch.Draw(buttonHostOnline, C, D ? Color.Plum : Color.White); },
() => { SelectedWindow = Windows.Host; });
Layout.ForceNewLine();
Imgui.Button(Layout.PositionItem(256, 77),
(C, D) => { spriteBatch.Draw(buttonJoinOnline, C, D ? Color.Plum : Color.White); },
() => { SelectedWindow = Windows.Connect; });
Layout.ForceNewLine();
Imgui.Button(Layout.PositionItem(256, 77),
(C, D) => { spriteBatch.Draw(buttonOptions, C, D ? Color.Plum : Color.White); },
() => { SelectedWindow = Windows.Options; });
Layout.ForceNewLine();
Imgui.Button(Layout.PositionItem(256, 77),
(C, D) => { spriteBatch.Draw(buttonQuit, C, D ? Color.Plum : Color.White); },
() => { Quit(); });

},
() => { });


I'm using XNA for rendering (obviously), and lots of lambdas. The result of this is

imgui.png

All of the types involved (FlowLayout, BorderLayout, Driver) operate independently. They can be used individually, but together they allow complex arrangements. Unique to my technique is, I believe, the lack of an 'item id'. It still exists, in a sense, in the form of the 'SelectedWindow', but instead of assigning an id to every item, I require the client code to deal with it when it's appropriate.

What I want to know is, what do you want to be able to do with an immediate mode gui? If you've reviewed the material available on the internet, what don't you understand about how they work?
Deyja

Animation in Jemgine

animed3.png

I have no idea how to change the size of the image when I add it to the post. I don't see any way to edit a post as HTML, either. I'm sure there's a way to do it, but oh well.

More on the skeletal animation I've added to Jemgine at http://jemgine.omnisu.com
Deyja
I want to teach my wife how to program. So I'm going to.

Step One : Trick her into writing code by making it a game. A week or so ago, I started writing a MUD, and I made her play it. She seems to enjoy it. She's not much of a video game fan, but a MUD is nice and slow paced, and she doesn't need to learn how to press buttons or move sticks. She just types, she can type just fine. We get to build the world together, and that is nice too.
I'll expose her to concepts in whatever order the creation of the MUD allows me. The MUD itself presents the first concept. The MUD has a state. You enter commands which modify the state, and the next command will operate on the new state. Each command is the line in an imperative program.
More to the point of this, I'm going to introduce her to variables, and I'm going to do it by creating an object.

[quote]
Creating an object : Create Adjectives Noun. Adjectives are optional. Most objects shouldn't have more than one. Don't use auxiliary words like 'an' or 'and'.

Fixing Common Grammatical Errors -
You might see an object listed as 'a name of object' when it should be 'an name of object'. To fix this error, set the object's A attribute to 'an '. The phrase '' is replaced with the object's SHORT attribute when the text is displayed. Objects whose name is a proper noun also need to have their A attribute set, usually to just ''. Set their SHORT attribute too, so that their name is properly capitalized.
If the object isn't pluralized by adding an s, set the object's PLURAL attribute to the correct pluralization.


Many simple objects should be instance objects. If you want to put other objects on, in, or under the object, it can't be an instance object. To make an object into an instance object, use the command 'makeinstance #ObjectID'. Instance objects can be in many places at once so it appears that there are many copies of them.

If you want to be able to place things on an object, give it the attribute 'allow_on'. Similarly, if you want to be able to put things in it, give it the attribute 'allow_in'.
[/quote]


Did you catch the variable? It's that '' bit. She won't notice, but she'll be using variables.
Deyja
I'm cross-posting this to my journal on Gamedev.net because the journals there are now apparently free. A long time ago I had gdnet+ and I had a journal there, where I talked about some sort of mud engine I was working on or something. I'm sure that mud engine totally sucked, and the one I spent the last week working on is a thousand percent more awesome. (It's pretty amazing how much you can get done in a few days if it's interesting enough.)

So, GDNet people : I know you have no idea what I'm talking about. I've got this 2d engine called Jemgine. It's on codeplex at http://jemgine.codeplex.com And, well, it's pretty much the most awesome thing ever made.

I haven't updated in a while. I have a good excuse, but no, I won't tell you what it is. I spent some time creating a model editor, and an animation editor, so I could create some sort of skeletally animated 2d models. What I discovered, however, was that they totally blew. The whole time I was thinking, I'd really like to just animate the whole entity. Move the chunks and splines and whatever around. So that's what I decided to do.

I'd already written a working skeletal animation system, so it was pretty easy to do. Now I can animate the entire entity. It's a pain, because I can't see the skeleton in the map editor, but I'll fix that.

It needs a lot of polish before I can show it off, but the results are already pretty astounding.


Meanwhile, I've been writing a MUD engine for some reason. I know nobody plays MUDs anymore, but I had an itch to write one so I scratched it. It's my first time using SQL too, so that's been interesting. Lets look at what I think is the most interesting part - the command matcher. Here is the definition for the command 'look'

Parser.AddCommandSet("LOOK", new Matchers.SequenceMatcher(new Matchers.KeyWordMatcher("IN", false),
new Matchers.ObjectMatcher(true, true, true,
new Matchers.OSAnd(new Matchers.OSContents(0, "WORN"), new Matchers.OSShortHand()))),
new Commands.LookIn());
Parser.AddCommandSet("LOOK", new Matchers.KeyWordMatcher("HERE", true), new Commands.LookHere());
Parser.AddCommandSet("LOOK", new Matchers.SequenceMatcher(
new Matchers.KeyWordMatcher("AT", true),
new Matchers.ObjectMatcher(false, false, false, new Matchers.OSShortHand())),
new Commands.Look());
Parser.AddCommandSet("LOOK", new Matchers.RestMatcher(false), new Commands.SimpleEcho("I don't see that.\n"));


These four definitions will be attempted, in order, until one matches the input string. The second argument to AddCommandSet is a 'TokenMatcher'. It takes a token stream, and tries to interpret it in some specific way. KeyWordMatcher, for example, only succeeds if the next token is the keyword (Unless it's optional, then it will always match). Each token matcher does not just return true or false; it returns a list of every possible match. This is key to how the matchers work. The command parser explores every possible match at once. SequenceMatcher will try it's second argument for every single possible match of it's first argument. So, if a KeyWordMatcher is optional, and the keyword is present, it will return two possible matches. One with the keyword consumed, one without. This is basically how a certain kind of regex matcher works.