Jump to content
  • Advertisement
Sign in to follow this  
irreversible

Exposing multiple objects to Lua from C++

This topic is 1471 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

I've been scouring the web all day and I can't figure this out.

 

1) does the below code make sense?

2) how would I go about extending it to support multiple exposed classes?

//less relevant functions have been omitted below

static int loadEntity(IN lua_State *L)
{
        //QUESTION: I'm assuming Lua automatically manages memory and it's safe to call
        //lua_newuserdata without worrying about reallocation/freeing. This seems weird,
        //but I haven't found a way around it
	IEntity** ptr = (IEntity**)lua_newuserdata(L, sizeof(IEntity*));
	*ptr = entityptr;
	luaL_getmetatable(L, "IEntity");
	lua_setmetatable(L, -2);

	return 1;
}


static int loadGame(IN lua_State *L)
{
	IGame** ptr = (IGame**)lua_newuserdata(L, sizeof(IGame*));
	*ptr = gameptr;
	luaL_getmetatable(L, "IGame");
	lua_setmetatable(L, -2);

	return 1;
}


static int AddModifier(IN lua_State * L)
{
	IGame* game = NULL;

/////////////////////////////////////////////////////////////////
/////////////////////PROBLEM COMES UP HERE///////////////////////
/////////////////////////////////////////////////////////////////

	game = checkGame(L, 2); //<- doesn't work, runs fine if I comment this out; debugging says IGame isn't even present on the stack
	IEntity* entity = checkEntity(L, 1); //works as expected
	
        ...

	return 1;
}



void luaExposeInterface(IN lua_State * L, IN const luaL_Reg listInterfaceFunctions[], IN char * sName)
{
	luaL_newmetatable(L, sName);
        //expose all the functions in the interface
	luaL_setfuncs(L, listInterfaceFunctions, 0);
	lua_pushvalue(L, -1);
	lua_setfield(L, -1, "__index");
	lua_setglobal(L, sName);
}


int main()
{
	static const luaL_Reg listGameUIFuncs[] = {
			{ "load", loadGame },
                           ...
			{ NULL, NULL }
		};

	static const luaL_Reg listEntityFuncs[] = {
			{ "load", loadEntity },
                            ...
			{ "AddModifier", AddModifier },
			{ NULL, NULL }
		};

//this should expose both IGame and IEntity
	luaExposeInterface(L, listGameUIFuncs, "IGame");
	luaExposeInterface(L, listEntityFuncs, "IEntity");

     ...

//run the script
	lua_getglobal(L, "EntryPoint");
	LUAErr(L, lua_pcall(L, 0, 0, 0));
}

Here's a relevant excerpt from the Lua code:

function EntryPoint()
        //successfully calls loadGame
	local gamestate = IGame:load()
        //sucessfully loads loadEntity
	local entity = IEntity:load()
        
        //fails if I try to access IGame
	entity:AddModifier("name", 1.0, 100)
end

To recap the code - the problem is exposing both IGame and IEntity as separate interfaces within Lua. I'll be wanting to expose more than one entity instance as well.

 

To be frank, I'm probably missing the entire point here and doing it wrong, but I can't really find concrete info on how to achieve what I really want. Which, on the Lua side, would look something like this:

function EntryPoint(entity1, entity2, gamestate)
        ...
end

Since I'm not completely sure how, or in fact what it is that I'm trying to achieve, some direction or references would be nice. I know there are several binding libraries out there, but I'm still very green when it comes to Lua and I wanted to familiarize myself with the library first. Additionally, there are some requirements that I'm not ready to look over: for instance, I've excluded luabind from consideration as I'm not using Boost and do not want to add it to my codebase.

Share this post


Link to post
Share on other sites
Advertisement

I'm sorry to ask, but heaving read your post I still don't understand: what do you want exactly? What do you mean by 'support multiple exposed classes'? Do you mean many instances of the same class or many different classes?

 

As for the question in the code - yes, lua_newuserdata will allocate a bunch of memory for you and it's garbage collected so you don't have to worry about it, if you free all C references (which is the only point of failure and possible leak of memory/object) and all Lua strong references are gone then the memory from lua_newuserdata will be garbage collected and __gc metamethod will be ran, if present. This is explained in great detail in Lua documentation.

 

As for second problem in the code - why do you think IGame would/should be on the stack?

Share this post


Link to post
Share on other sites

I originally tried to write my own implementation for binding C++ to lua to learn the lua C api.  I got it mostly working for simple classes, but quickly gave up and switched to using a library because it would just take too much time to be able to support exporting any class generically with inheritance, constructors, different parameters, return types, etc.  

 

Not to discourage you from continuing to learn the api, but if you decide to give up like me and use a library, I'd suggest LuaBridge: https://github.com/vinniefalco/LuaBridge. It's very lightweight and easy to use. The syntax is similar to Luabind, but it doesn't need boost and it can use your class member functions directly without needing to create a bunch of static wrappers:

getGlobalNamespace(L)
 .beginClass<ClassA>("ClassA")
   .addFunction("Func1", &ClassA::Func1)
 .endClass()

To your actual problem, when you call entity:AddModifier() from lua, lua pushes entity on to the stack (from the entity: part), which is why you could pop it from the stack in C++ with checkEntity(), however as FRex mentioned, the IGame object was never pushed on the stack within the call to entity:addModifier().  You would need to pass it in as a parameter.

Share this post


Link to post
Share on other sites


I'm sorry to ask, but heaving read your post I still don't understand: what do you want exactly? What do you mean by 'support multiple exposed classes'? Do you mean many instances of the same class or many different classes?

 

Both. For starters I want to expose both IGame as kind of a "namespace" to enable access to the game's internal state and IEntity as two or more separate instances to expose the internal state of several different discrete objects.

 

I realize the former can be done with globals, but that wouldn't really solve the problem.

 


As for the question in the code - yes, lua_newuserdata will allocate a bunch of memory for you and it's garbage collected so you don't have to worry about it, if you free all C references (which is the only point of failure and possible leak of memory/object) and all Lua strong references are gone then the memory from lua_newuserdata will be garbage collected and __gc metamethod will be ran, if present. This is explained in great detail in Lua documentation.

 

Yes, I figured as much. I did read the docs as well. What is kind of confusing to me is that I can't reuse the allocated memory. I'm probably running into a simple entry level learning curve here, but it would be more logical to me if I could allocated the memory once (since I'm not storing a full object, but rather a pointer) and then change that pointer as needed while having a sense of transparency in terms of what memory and how much of it is allocated. Right now I'm having trouble seeing when exactly a leak will occur and when it will be collected.

 

Then again, I'm still struggling with the basics and the fact that I can't even explain what I want in detail means that I'm probably not really getting how things should work in the first place.

 


As for second problem in the code - why do you think IGame would/should be on the stack?

 

Ah - my assumption is that after

 

1) exposing IGame's metatable

2) and loading it in response to IGame:load() (which class loadGame())

 

it would be pushed onto the stack much like IEntity is right now. Apparently my assumption is wrong :)

 


I originally tried to write my own implementation for binding C++ to lua to learn the lua C api.  I got it mostly working for simple classes, but quickly gave up and switched to using a library because it would just take too much time to be able to support exporting any class generically with inheritance, constructors, different parameters, return types, etc.  

 

Not to discourage you from continuing to learn the api, but if you decide to give up like me and use a library, I'd suggest LuaBridge: https://github.com/vinniefalco/LuaBridge.

 

Indeed - that's the impression I've gotten as well. I'm probably going to migrate to a library soon enough - I guess my first problem isn't really what the code should look like, though, but rather how it should work in the first place. Thanks for the suggestion!

 


To your actual problem, when you call entity:AddModifier() from lua, lua pushes entity on to the stack (from the entity: part), which is why you could pop it from the stack in C++ with checkEntity(), however as FRex mentioned, the IGame object was never pushed on the stack within the call to entity:addModifier().  You would need to pass it in as a parameter.

 

Hm - how so?

 

As above, doesn't calling IGame:load() (which translates to loadGame() in C++) push it onto the stack. How would I make this work?

 

 

Basically I'm trying to implement a module-based structure whereby the available modules in a Lua script would be along the lines of:

 

Discrete objects, eg

IEvent and IEntity - any number of discrete instances depending on the event

 

Interfaced internal classes, eg

IGame - expose current game state

IUI - expose UI functionality

 

I would love to pass in discrete objects (things like events and entities) as function arguments while exposing interfaced internal classes as global namespaces.

 

Does this make sense or is my logic going down the wrong path?

Share this post


Link to post
Share on other sites

If you want single IGame object, that you call methods on like game.something() then use a global table, with functions each of which has an upvalue with light userdata of your game instance in them.

If you want to be able to extend it easily with pure Lua methods and call both the same way(game:something(), no matter C or Lua method), do it with global full userdata and metatables.

As for the many instances of same class. Just push each as a separate userdata and give each the same metatable named IEntity (one metatable can apply to as many types, full userdata values or tables as you want).

You also CAN reset your pointers (or any values you stored) in full userdata, you probably shouldn't because there is no point really and it'd be weird for user values in Lua to suddenly be different instances.
 
The only way to leak memory (leak in the Lua VM, not leak as in NEVER free before exiting, VM frees all it allocated when it's closed) is to have a C reference that you throw away but forget to release.

The only things that are on Lua stack when your C code gets called are the things you passed to it, plus optional first 'self' argument from the use of : instead of . for function call.
thing:something(10,20,30) is the same as thing.something(thing,10,20,30)
except for implementation details and being evaluated once in the first case.

If you do thing:something(10,20,30) you pass to your something function 4 values: thing, 10, 20 and 30.

There is no way to access locals (debug library doesn't count for doing that, it's not meant for real code) in C function code, if you want to use something, it has to be passed in (and be on the stack), upvalue (and be on the stack under pseudo indices) or globally accessible in Lua or C (and you just lua_getglobal or access them directly as you would any other global).

Edited by FRex

Share this post


Link to post
Share on other sites
As above, doesn't calling IGame:load() (which translates to loadGame() in C++) push it onto the stack. How would I make this work?

 

 

When lua calls a C/C++ function, it pushes all arguments on the stack, but this is not a global stack.  It is a local stack to that C/C++ function, where the first item on the stack is the first parameter. When you use the syntax entity:AddModifier("name", 1.0, 100), it's equivalent to AddModifier( entity, "name", 1.0, 100).  So entity became the first argument.  On the C/C++ side, AddModifier only has visibility to the local stack, so only the 4 parameter values.  You can verify this by writing a simple C++ function to print the stack and then calling it from your C++ function before popping any values from the stack.

 

See http://www.lua.org/pil/26.html for information about the stack

and http://www.lua.org/pil/24.2.3.html for printing the contents of the stack from C/C++

 

FRex beat me to the rest. For the C/C++ AddModifier function to have access to IGame from lua, it needs to be passed as a parameter from lua, an upvalue (similar to static variables), or a lua global.  It seems like you want to use globals, so you could declare your IGame as a global in lua and then access it anywhere you want in C++ with lua_getglobal.  Alternatively, if you only have one instance of IGame in C++ which you have also exposed to lua, then C++ already has access to the instance, so you don't actually need to get it from lua anywhere.

Share this post


Link to post
Share on other sites

When lua calls a C/C++ function, it pushes all arguments on the stack, but this is not a global stack.  It is a local stack to that C/C++ function, where the first item on the stack is the first parameter. When you use the syntax entity:AddModifier("name", 1.0, 100), it's equivalent to AddModifier( entity, "name", 1.0, 100).  So entity became the first argument.  On the C/C++ side, AddModifier only has visibility to the local stack, so only the 4 parameter values.  You can verify this by writing a simple C++ function to print the stack and then calling it from your C++ function before popping any values from the stack.

 

Okay, that is much more clear.

 

The simplest solution would be to simply store sourceentity in this entity (in C++) and expose it via its own set of functions in Lua. Eg, to extend the above example, I would use entity:GetSourceModifier().

 

Or encapsulate it further by exposing it like:

 

event = IEvent:load()

source = IEntity:GetSource(event.GetID()) //uses lookup from a global event list to get the event and pushes the source entity onto the stack

target = IEntity:GetTarget(event.GetID())

 

And also call my regular functions through IEvent, which has access to all of the involved parties:

 

event:AddTargetModifier(...)

val = event:GetSourceModifier(...)

 

Or alternatively:

 

target:AddModifier(...)

source:GetModifier(...)

 

 


You also CAN reset your pointers (or any values you stored) in full userdata, you probably shouldn't because there is no point really and it'd be weird for user values in Lua to suddenly be different instances.

 

I think I'm misunderstanding this. The way I see it a pointer exists inside the userdata value for a script during the script's lifetime (not its execution time) or until it's collected. So if I run a script in response to an action that an entity performs, I want the user data to point to that particular object. I also want to use a single instance of a script for all instances of a particular entity type and I don't want to do any allocations/frees while the game is running. The most reasonable way to accomplish this would be to do it like this:

void IEntity::HandleEvent(IEvent * event)
{
   IScript* script = GetScript(event->action);
   script->Set(event); //set userdata for this particular event instance, which contains everything from the target to the source and the current game state
   script->Run();
}

In short, I can't figure out what IScript::Set() should look like. Right now I'm calling lua_newuserdata() in response to each execution of the script to set the current instance of the entity. Is this the way I should go about it?

Edited by irreversible

Share this post


Link to post
Share on other sites

The way Lua internally works it will always allocate space for something when running.

If you really are worried about allocation taking time there is a way to plug your own memory function to lua_newstate (luaL_newstate has the default malloc/free allocator).

You probably shouldn't, the standard allocators in desktop implementations are fast enough for majority of cases.

How long your Lua value lives (by lives I mean not eligible for GC collection) depends on what scope it's in and a lot of other things.

 

1. Using lua_newuserdata and then letting it get collected is probably good enough for you, unless it shows as being significant at all compared to other code that does real work in your game.

2. Another way to do it is have single entity userdata with metatable already set and just look it up via C ref and change the pointer each time you want to call your script on another entity, it's just one extremely fast (via rawgeti) look up compared to allocating new data, looking up metatable by name and setting it every time.

3. Even better would be to have each of your entities have it's own C ref that points to their userdata with set pointer, metatable and all, possibly lazily initialized. Again it'll make you just call rawgeti once, and do none of the usual: newuserdata, getmetatable, setmetatable stuff so it'll be faster.

 

Both times (both 1. vs 2. and 1. vs 3.) it's: rawgeti (raw access to array element + safety etc. checks) vs. allocation (probably doesn't matter thanks to how efficient lea, ms and other malloc/free libc allocators are..) + metatable look up by name (pretty bad, has to get length of string, hash it, find the duplicate in Lua, then using that hash look up in hash table the right table) + set it (just house keeping, bunch of pointer work, irrelevant).

 

The option 1. is what you have now, 2. is pretty weird but better than 1. if you go for micro optimizations, 3. wastes a couple bytes for one lua userdata value, pointer and int per each entity but it's pretty intuitive for each entity to hold reference to itself on Lua side (and free it when it itself dies on C++ side).

It also allows for having some data on C++ side and some (any) on Lua side (in tables, each entity would get its' own table) on per entity basis via metamethods, but it's much more advanced usage at this point.

Edited by FRex

Share this post


Link to post
Share on other sites

Actually the allocations themselves don't really worry me - the reason I'm bringing it up is that I'm using my own memory manager and it'd be great to have all memory ops confined to one framework. But that's not really crucial. In any case, I've done a little bit more poking around and have come across another problem:

 

 

I'm passing a table from C++ to Lua where each row has the format {string | float} and it seems to be working quite nicely. Except that it crashes the game after repeated use (I'm assuming it's due to the fact that I'm creating the table from scratch each time the function is called):

//In Lua:

activeItem = player:GetActiveItem()

//In C++

static int GetActiveItem(IN lua_State * L)
{
	IEntity* entity = checkEntity(L, 1);

	if(entity->activeItem)
		{
		lua_newtable(L);
		int top = lua_gettop(L);

		for(int i = 0; i < (int)entity->activeItem->modifiers.size(); i++)
			{
			IEntityModifier* m = &entity->activeItem->modifiers[i];

			lua_pushlstring(L, m->sName, STRLEN(m->sName));
			lua_pushnumber(L, m->fValue);
			lua_settable(L, top);
			}
		}

	return 1;
}

I can't yet tell what the best way to modify a table is, but I'll need a way to update the floating point values. A more logical solution would thus be:

static int GetActiveItem(IN lua_State * L)
{
	static bool bTableCreated = false;

	IEntity* entity = checkEntity(L, 1);

	if(entity->activeItem)
		{
		if(!bTableCreated)
			{
			lua_newtable(L);

			int top = lua_gettop(L);

			for(int i = 0; i < (int)entity->activeItem->entity->modifiers.size(); i++)
				{
				IEntityModifier* m = &entity->activeItem->entity->modifiers[i];

				lua_pushlstring(L, m->sName, STRLEN(m->sName) + 1);
				lua_pushnumber(L, m->fValue);
				lua_settable(L, -3/*top*/);
				}
			lua_setglobal(L, "activeItem");
			bTableCreated = true;
			}
		else
			{
			lua_getglobal(L, "activeItem");
//TODO: update fields that have changed
			}
		}

	return 1;
}

Which doesn't seem to work. I get an error that I'm trying to perform arithmetic on a field with a nil value when I try to access a valid entry in the table.

 

Once again, Google is somewhat light on concrete examples on how to go about this...

Share this post


Link to post
Share on other sites

You can (extremely easily) specify a function for memory allocation in lua_newstate.

Why does your first example crash and why is there +1 in your second example? Lua allows strings to have embedded \0s. That's probably your problem, your keys are not "key", they are "key\0" because of your +1.

Also, since it's really hard to guess from small snips of code with no context or way to run them: http://sscce.org/

Share this post


Link to post
Share on other sites
Sign in to follow this  

  • Advertisement
×

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

We are the game development community.

Whether you are an indie, hobbyist, AAA developer, or just trying to learn, GameDev.net is the place for you to learn, share, and connect with the games industry. Learn more About Us or sign up!

Sign me up!