Hello there. I decided to design a simple virtual machine for fun. This is my 4th attempt. This one is designed to be practical... So that it is possible to easily compile C/C++ like code to bytecode that will run in the VM. When it is complete, I will probably make it open-source, and perhaps make a tutorial about how it was made, and how this can be used for game-scripting. The purpose of this thread is to gain feedback. For this VM, I decided to go with a register-less approach. Essentially, all variables will be on the stack. I also decided to go with a unified Virtual RAM memory space. My previous VM had a heap, a stack and registers all in 3 separate areas, but this made it very tedious and impractical to operate on data that is beyond the level of complexity of integers and floats (ie: C++ objects). The previous VMs also couldn't have pointers to data on the stack, which seemed rather restrictive to me, as it would have forced me to implement my language in a java-like fashion, where all objects must always be on the heap. I don't know if any of you ever did MIPS assembly. This design was partly inspired from it. Except that bytecode instructions will not always be the same size. VRAM layout: ############# ************ (VRAM upper boundary) ------------ ... Heap ------------ (Stack base + stack size) ... Stack ------------ (Stack base) ... Data segment (globals) ------------ ************ (address 0 - not addressable) The data segment (globals) are directly addressable. The code segment is not stored into the VRAM to prevent exploits. The main reason for this addressability model is that a contiguous memory system greatly simplifies addressing. It will be possible, for example, to have pointers to data in the stack or in the heap. The upper boundary of the VRAM can also be extended if there is a need for more heap space. This VM design offers the possibility of operating on all local stack variables directly, of moving objects between the stack and the heap all at once, and should have a bytecode that is reasonably suitable for eventual JIT compilation. I have also chosen to make the stack management implicit. You can request stack space to store more variables, or shrink your stack frame without touching a "stack pointer". The VM will also do bounds checking on the stack. It will also handle memory allocation in the heap. For flexibility, the heap is at the top of the memory, so that if the VM ever runs out of heap space, the VRAM can be resized at runtime. By default, it might start with a VRAM size of 4MB or so. At the bottom of the VRAM, the data segment of compiled programs is copied. This is so that you can store global variables, text strings, all kinds of datas in compiled bytecode files, and they are readily accessible to you. So you can have pointers to them. They actually start at address 1, so they are always in the same location. This data can also be modified at runtime. For obvious reasons, the code segment is not copied into the VRAM and executed from there. Instructions: ############# Memory management: movss (move stack->stack) movsg (move stack->global) movgs (move global->stack) movgg (move global->global) alloc (allocate heap bloc) free (free heap block) push (grow stack frame) pop (shrink stack frame) gsr (get stack reference) Integer math: isft iadd isub imul idiv Floating-point math: fadd fsub fmul fdiv Logic: or (bitwise or) and (bitwise and) xor (bitwise xor) gtui (greater than - uint) gti (greater than - int) gtf (greater than - float) eq (equality testing) Flow-control: jnoz jisz jump Subroutines: call ret System calls: sysi (obtain system call index from name) sysc (perform system call from index) The system call instructions are to allow developers to "bind" their own functions inside the VM. Essentually, what this will do is allow you to register a name for a function, and then give you access to a part of the stack frame of the calling function in the VM when your function is called, essentially allowing you to pass parameters. Your functions will also be able to return parameters that will be placed on the stack of the VM. This will be used for things like the "standard library", which are utility functions (for things like string manipulation and console output), but would also allow you to interface the VM with a 3D engine and provide access to its functions for game scripting. It will not provide direct access to external data, however.

Surely someone has some comment of some kind ;)

Oh well. I am thinking of also adding instructions to perform operations in "immediate mode", where the value is supplied in the instruction. This might be useful in making this faster since the compiler will most likely implement the instruction case switch as branching address table... So the VM would be advantaged to be designed for CISC, instead of trying to run a ton of small instructions.

Although I suppose I can start with a "plain" instruction set and then add on later. I could probably start by adding an instruction to set a 32-bit word value in a stack memory location.

Instructions like:
setws (set word - stack)
setwg (set word - global)

If you can regester function Foo such that your bytecode can "call Foo", then will you be able to regester funtion Foo such that you can call a function running in your scripts? ie "VM->Call("Foo")"

On the same lines, it could be usefull to provide a similar interface for variables. So that you can regester a named variable inside the VM for access outside,
or to regester a variable outside the VM for use inside.

Quote:Original post by KulSeran
If you can regester function Foo such that your bytecode can "call Foo", then will you be able to regester funtion Foo such that you can call a function running in your scripts? ie "VM->Call("Foo")"

.

Well. The VM doesn't have any concept of functions because it works at the bytecode level. It only has subroutines... But it might be possible for the VM to pass "addresses" of its functions to engine functions, and have the engine control the VM to call them.

Quote:On the same lines, it could be usefull to provide a similar interface for variables. So that you can regester a named variable inside the VM for access outside, or to regester a variable outside the VM for use inside.

Well, if you have functions you can call, you can access data through them. It's safer like that.

Quote:It sounds like a good foundation, however I don't recommend having a dynamically sizable heap, or VRAM in general. Because if this VM running a program in the background of a game and all of a sudden the heap needs to be resized, you're looking at allocating a single large block of memory (which takes time), and then copying your existing VRAM to this new location. Even if you go with an allocation scheme were you double the size of the VRAM, you go from not having enough memory to having too much memory. A better idea might be to have a fixed VRAM size, but let the code determine the size of this space before execution. So if a piece of code needs more space to run it can get it, without putting a run-time burden on the VM to meet memory demands.

Well, I've thought about giving my bytecode program a header that specifies such parameters, and it's not such a bad idea, for the most part. However, I would much rather have the VM freeze for 0.01 second while it expands its memory from 4 to 8 MB than having it crash in the middle of something. I also don't really like the idea of programs specifying their memory requirements because then people could put any kind of crazy number that will just uselessly tax your system.

As for the addressing mode, I am planning on having offsets in them all, so it can be used to access objects (and the multiple variables on the stack)... Basically, it will be a pair 32bit address + a 16bit offset for the source and one for the destination, along with a 16bit size tag in "global addressing", and just two 16bit source and destination offsets with a 16bit size tag in "stack addressing", which will be used to manipulate local variables without knowledge of the stack frame pointer. But it will be possible to use the gsr instruction to get a pointer to data on the stack that will be usable in global addressing mode.

One change I would make - instead of having conditional jumps, I would give each instruction a 3-bit 'condition' field that could be 0 for 'always', 1 for 'when equal flag set' etc. This would allow you to do conditional everything, and you would only need to make a 'jump' command that is controlled by the condition field.
In the assembly, I'd specify it like 'ceq' for 'when equal flag is set', 'cgt' for 'greater than' etc so in the end a conditional jump might become 'ceq jump Label1' and a conditional move could be 'ceq move A, B' etc

Because you only have 30 instructions (after eliminating all but 1 jump), you can fit that into 5 bits and the condition field into the remaining 3 for an 8 bit byte. Of course, if you were planning to pack the operands in there, it might end up messier.

Quote:Original post by Extrarius
One change I would make - instead of having conditional jumps, I would give each instruction a 3-bit 'condition' field that could be 0 for 'always', 1 for 'when equal flag set' etc. This would allow you to do conditional everything, and you would only need to make a 'jump' command that is controlled by the condition field.
In the assembly, I'd specify it like 'ceq' for 'when equal flag is set', 'cgt' for 'greater than' etc so in the end a conditional jump might become 'ceq jump Label1' and a conditional move could be 'ceq move A, B' etc

Because you only have 30 instructions (after eliminating all but 1 jump), you can fit that into 5 bits and the condition field into the remaining 3 for an 8 bit byte. Of course, if you were planning to pack the operands in there, it might end up messier.

I won't pack anything. Each instruction will have its own parameters depending on the opcode. Ie:

iadd (8bit) | index_srca (16bit) | index_srcb (16bit) | index_dst (16bit)
jump (8bit) | dest_index (32bit)
sysi (8bit) | name_string_addr (32bit) | index_dst(16bit)

I'm thinking of making the opcodes just 8 bit since I don't want too many instructions. As for the 16 bit stack indices, you might think that's big... But in the end, it means each stack frame can have 64KB of data in it, and it will eliminate alot of push/pop instructions. I could also have some arithmetic operations that operate directly on the operands on top of the stack, eventually, to make this even more lightweight.

As for everything being conditional... I think that would just slow things down. it's just doesn't work well with the hardware... And what people really want is branching.

personally I'd stick with a stack machine approach. I'd mix it in with lisp oriented cons cells and stop and copy gc. than I'd add a dash of salt and make a language like dylan.

Quote:Original post by Name_Unknown
personally I'd stick with a stack machine approach. I'd mix it in with lisp oriented cons cells and stop and copy gc. than I'd add a dash of salt and make a language like dylan.

What do you call a stack machine?

Quote:My instincts just cringe at the thought of allocating so much memory at once and then copying 4MB from somewhere else, at run-time.

Copying memory, even 32MB, on a modern system, should actually be rather fast.