Multiple AIs with one Lua state

This topic is 3220 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

Recommended Posts

I want the AIs of my NPCs to be stored in Lua scripts. The design I'm leaning towards is each AI type is defined in a script/file with a standard set of callbacks (e.g. "nextMove()") the engine would call. The script file may also have any global variables it needs if the AI using that script wants any state. This would be easy if each NPC had its own Lua state object. How would I do it with a single Lua state though? That is, how do I avoid function and variable names clashing between the scripts, or have NPCs with different AI state though they use the same script?

Share on other sites
Quote:
 Original post by Guy Mehhow do I avoid function and variable names clashing between the scripts,

You could use the local keyword which restricts scope.
Quote:
 have NPCs with different AI state though they use the same script?
Would you not store state in a table?

Share on other sites
Organization wise, it would probably look pretty similar to how you would do it without using Lua. You could have one big array (actually a table) at the global scope of the Lua state, called ALL_MY_NPCS or something. Inside this array, there would be a struct (also actually a table) for each NPC, which contains the state specific to that NPC.

Share on other sites
Quote:
 Original post by CmpDevYou could use the local keyword which restricts scope.

Nice. I forgot about that. :)

Quote:
 Would you not store state in a table?

Well, I suppose I could. I was originally thinking of each script being independent of each other, and each script could have whatever variables and structures it needs as long as it has the functions the game engine expects. Some AIs might not have any state, such as AIs that just make their NPCs move around randomly.

Quote:
 Original post by pinacoladaYou could have one big array (actually a table) at the global scope of the Lua state, called ALL_MY_NPCS or something. Inside this array, there would be a struct (also actually a table) for each NPC, which contains the state specific to that NPC.

So in my engine (outside Lua) the NPCs store the index their AI is kept in this Lua array? And each AI stores a table containing a reference to their functions as well as whatever state they have? I make special mention of the functions because the neither the engine nor the AI scripts make any assumptions about the different AIs. All that is known is a standard set of functions each script would have that gets called by the engine, and some of these functions do not necessarily have to do anything when called. I'm using polymorphism in a way, where there is an imaginary, pure abtract AI class and each Lua script is an implementation of this class.

Of course, I may have to rethink how I'll structure the AI scripts if I want to
jam all of them into a single Lua state.

Share on other sites
your questio is actually quite similar to my own here...

did you get it working?

Share on other sites
You should (both) take a look at this thread, and see if it resonates.

Share on other sites
thank you. i just read it, but i don 't see how this might help us. they are talking about "more elegant ways" of achieving what we want without having multiple lua states, but they don't give a clear answer how these ways look like :/

Share on other sites
Sneftel's example towards the end of the thread he linked is how you might do it. It shows how you embed all the variables you need to keep track of in a table, so they don't interfere with any other script objects.

Share on other sites
I'm not even sure if I fully understand his code.. So let's see. My base class Entity contains the following:

Entity( const string &script ){	m_strScriptFile = script;	Engine::GetScriptingSystem()->LoadScript( m_strScriptFile );	Engine::GetScriptingSystem()->RunFunction( m_strScriptFile, "Initialize" );}~Entity(){	Engine::GetScriptingSystem()->RunFunction( m_strScriptFile, "Shutdown" );}void Update(){	Engine::GetScriptingSystem()->RunFunction( m_strScriptFile, "Update" );}

When I derive classes from this base class, i'd like to be able to just provide a different m_strScriptFile, and the Initialize/Shutdown/Update functions from that script file are being run, and the variables defined in this file do not interfere with variables from other files. The scripts should, however, have access to globally defined variables and functions/classes bound from the main executable.

How exactly can I make sure this works? Thank you for your patience.

Share on other sites
As mentioned you can use the keyword local to create a variable/table/function scoped to only that module ( the script file ), but since Lua has a shared global namespace anything you put into their can be overwritten by any other script ( there are a few ways to make protected tables ). So you can't have multiple script defining the same functions say :

function executeMe(){}

and expect Lua to tell them apart. You'll need to help Lua out and distinguish them in some way. The easiest way I can think of is to declare the function local and register it with some global table, like so:

--this function is only exist in the scope of this file.local function executeMe(){}--pushes a function into the global table with unique identifier local node={source="my unique identifier, full path maybe?",func=function() executeMe(); end};gGlobalExecuteTable[node.source]=node;Now when it comes time to execute these execute functions, you can use the unique identifier to distinguish them, like so:function executeGlobal(identifer)   if (gGlobalExecuteTable[identifer]==nil) then     return;   end   gGlobalExecuteTable[identifer].func();end

This is just a simple example. That's a simple system and you don't have to mess with creating threads and stuff.

Good Luck!

EDIT:

I didn't see your latest post, but to use a single VM you have to load up the implementation of each script into the current VM using say loadfile and execute it using pcall or something, using the above registration scheme as an example, that will put that scripts main hook functions (Initialize,Shutdown,Update) into the global table, which then u can use the string m_strScriptFile as a unique identifier.

That solve the problem of registering and calling these functions but it doesn't solve the problem of storing away each AI instance state nor prevent the possibility of scripts overwriting each others global functions.

Easiest method is to have each AI instance passed into these functions and don't let the functions keep any persistent data outside of those instances. They should just be pure functions.

As for the problem of scripts overwriting each others global functions, you'll have to organize your code or perhaps look into cloning global tables.

Good Luck!

-ddn

Share on other sites
I'm starting to wonder if Lua is even appropriate for the kind of scripting I want to accomplish here.

Like the creator of the thread Sneftel linked to, I sort of want my scripts to be something that any object can just run; e.g. an NPC's think() method is runAIScript("myScript.lua"). I wanted to be a bit more elaborate by having these scripts store functions the objects could choose from, but it's the same spirit as far as I can tell. But with Lua's global name space it seems I either have to use multiple Lua states, jumble my scripts up with tables and stuff, or maybe just give up and write my entire game in Lua.

I chose Lua because it was popular, fairly minimal, and could also be used for data definition. However, is there perhaps another scripting language that would be a better fit for this structure.

Share on other sites
Quote:
 Original post by Guy MehI'm starting to wonder if Lua is even appropriate for the kind of scripting I want to accomplish here.Like the creator of the thread Sneftel linked to, I sort of want my scripts to be something that any object can just run; e.g. an NPC's think() method is runAIScript("myScript.lua"). I wanted to be a bit more elaborate by having these scripts store functions the objects could choose from, but it's the same spirit as far as I can tell. But with Lua's global name space it seems I either have to use multiple Lua states, jumble my scripts up with tables and stuff, or maybe just give up and write my entire game in Lua.

If you consider hierarchical organization with tables to be "jumbling things up", you will almost certainly be unhappy with Lua.

Share on other sites
this old post seems to deal with the exact same problem. The idea is to surround the code from a script file in a table. I just don't quite understand how to reproduce this with luabind (still pretty new to lua/luabind). and i'd like to hide all this from the enduser, of course. can you think of a way to do this?

Share on other sites
It's the same issue but slightly different take on the problem. I stuff my functions into a global table using a unique key (for dispatching later), the other adds the functions to a global table but doesn't distinguish multiple additions, they just overwrite each other, but it's simple enough to get around that problem.

You can't dispatch the function directly since multiple scripts redefine the same function and the only distinguishing factor is the file which they come from.

This question is asked so often it should be in the FAQ. Here's what I would do in this case:

Provide C or Lua function to load up and register global functions of a given script into a global dispatch table. A global function is one which isn't defined using the local keyword. Some common global functions it could define are Initialize,Shutdown,Update etc..

After loading the script up the loader function auto-registers any global functions into a dispatch function table using the scripts file as a key. This eliminates the need for the script to do this themselves. It then clears out the global functions just loaded to prevent global name space pollution.

Anyone wishing to call a registered function from a script must pass on the script file name ( or some other convenient key ) which will be used as a key to lookup that function with the global dispatch table.

Registered scripts functions are passed an instance table (anonymous tables which are used to store state about the script). Because the scripts are shared between multiple instances they can't use local data to store their state, this assumes there is a one to many relationship between the scripts and objects using them.

These anonymous tables are stuffed in the objects (which own the scripts?) using the unique key of the given script.

To call a particular function from LuaBind you would call a dispatch function which needs 3 pieces of information, the script key ( the unique identifier for a particular script), function you want to call and the object itself ( because the instance data lives in that object ).

That's how I would do it.

I can only imagine why anyone would want to do this is because they want to write scripts which are nearly wholly self contained with a common API (Initialize,Shutdown,Update etc..) so they can be auto-registered more easily.

Good Luck

-ddn

Share on other sites
ddn3, could you illustrate this with some (pseudo)code? What you're describing sounds exactly like what I'm looking for, but I can't quite follow you to the end. It's not that I want others to write my code, but I'm still pretty new to Lua, and it's C API just confuses me.

Thank you all for your patience!

Share on other sites
Better than pseduo code, I'll give u Lua code! :D Here ya go a simple example minus the C function used to load run the script.

--here is where all registered functions will liveGLOBAL_DISPATCH={}--use this function to load and register a scripts funcitonsfunction RegisterScript(scriptFile)	local key = scriptFile;		if (GLOBAL_DISPATCH[key]~=nil) then		print("script already registered");		return true;	end		local funcTable={}		--make sure these globals functions are nil before hand	Initialize=nil;	Shutdown=nil;	Update=nil;		--nfLoadAndRun is a C fucntion which loads and runs a script	--i don't know how to do  this from Lua, but in C you do 	--loadfile the do a pcall and that will load and run the script	local err = nfLoadAndRun(scriptFile)		if (err~=nil) then		print("Failed to load and execute script");		print(err);	--this should contain error message		return false;	end		--after the script has been loaded we can now store away these	--global, doesn't mater if they are nil, since dsipatcher will	--not execute nil funcitons.	funcTable["Initialize"]=Initialize;	funcTable["Shutdown"]=Shutdown;	funcTable["Update"]=Update;		--nil them out again to keep namespace clear	Initialize=nil;	Shutdown=nil;	Update=nil;		--store away our function table for later use	GLOBAL_DISPATCH[key]=funcTable;		return true;end--you'll use the resister function like soRegisterScript("Scripts/AI_Ogre")--here is an example script for say AI_Ogre (note that myData is passed in)function Initialize(myData)	myData.name="Mr Orgre";	myData.age=24;	myData.weapons={"battle axe","dagger"};	myData.dislikes={"elves","hobbits","brocolli");	myData.state="sleeping";end--note any function used by the script internally should be declared--local to prevent pollution and possible overwrite by other scriptslocal function UpdateAwake(myData)    if (math.random(0,100)==0) then	myData.state="sleeping";   endend--this is how a script uses stateful data for a particular instancefunction Update(myData)	if (myData.state=="sleeping") then		if (math.random(0,100)==0) then			myData.state="awake";		end	elseif (myData.state=="awake") then	    UpdateAwake(myData);	endend--doesnt define Shutdown but thats ok..--now to dispatch any function we need 3 thingsfunction Dispatch(funcName,scriptFile,object)	local key = scriptFile;	local funcTable = GLOBAL_DISPATCH[key];		if (funcTable==nil) then		print("cannot find function table for given script");		return false;	end		if (funcTable[funcName]==nil) then		--it's not really an error for script to not have this function?		--is it? depends on your implemenation.		return true;	end		--the object should already contain a table use to store state	--specificly for this script, but in this case we can create one 	--if it doesn't	local data = object[key];		if (data==nil) then		object[key]={};		data=object[key];	end		--now we dispatch the function with the given data table associated	--with it.	funcTable[funcName](data);		--all ok	return true;end

I didn't check if this runs, you might have to use an anonymous function instead of directly assigning the functions like so:

funcTable["Initialize"]=Initialize;

use this maybe?
funcTable["Initialize"]=function(data) Initialize(data) end;

Give it a try! :D

Good Luck!

-ddn

Share on other sites
So...if I have a script like this:
----# ai.lua# May or may not have local variables up here (which could be declared# local but might not be because I'm lazy)# And remember: more than one NPC can use this script, so the state# variables declared here would have different values for each NPC.function nextAction()    # Do cool stuff    ...   return stuffendfunction getState()    # More stuff    ...    return stateCodeend# Not called in C++, so I might want to declare it local.function helperFunc(moreStuff)    ...end----

The suggestions here have generally been to perform extra processing in the script files (this is what I called "jumbling things up"). In addition to the actual AI code, these scripts would contain processing code to differentiate each script/chunk in the Lua interpreter. Code that would probably be the same across scripts besides maybe a few name changes.

Is there no way to put this processing inside the C++ half?

Share on other sites
Problem is your witting the state into free local variables of the script. If your not going to make any concession about where you store the variables then you'll have to clone the entire script in memory and that way each instance clones all its internal state. You'll still need a mechanism to dispatch functions inside that script which are unique to it.

I believe its possible to load up a script file in Lua, stuff that compiled script into a table and its also possible to do a deep copy of a table. So you'll load up the script and when u need an instance you would clone it and stuff that clone somewhere accessible only by the object which owns it.

Just declare all functions and variables of the script local so it doesn't write into the global table overwritting each others implementation, unless that is what u want ( ie a shared global data or function between all the instances of the script ).

Again alot of convoluted steps just so you can write scripts which use exact same named functions ? Is this a concession to the designers who don't want to or know about namespace and OOP ?

Good Luck!

EDIT: ofcoruse u can put it all in C++/C Lua has a complete C api which duplicates every single facet of the normal Lua syntax, but why would u want to do that when u can dispatch any global Lua function from C/C++ with ease?

Apparently this is what the require() function already does for Lua. It loads up a script and returns a copy of it. This copy is a complete clone of the script so any locals declared inside it are unique only to it. So something like this :

local clone = require("Scripts/AI_Ogre");

now u can dispatch functions in clone ( make sure its local only to clone )

clone.Iniitlizaion();

That should work.. Good Luck! :D

http://lua-users.org/lists/lua-l/2007-08/msg00005.html

for more details...

-ddn

[Edited by - ddn3 on April 26, 2009 12:14:20 PM]

Share on other sites
Quote:
 Original post by ddn3Just declare all functions and variables of the script local so it doesn't write into the global table overwritting each others implementation, unless that is what u want ( ie a shared global data or function between all the instances of the script ).

This was sort of what I was trying to get at. NPCs with the same AI script share the functions but keep their own instances of the local variables. I suppose, to make it easier, any persistent local variables an AI would use would instead be stored in a single table instead of be scattered around the script file, though that doesn't solve the function name clashing between multiple AI scripts.

Quote:
 Again alot of convoluted steps just so you can write scripts which use exact same named functions ? Is this a concession to the designers who don't want to or know about namespace and OOP ?

Technically, yes. My scripts in general are basically implementations of C++ methods except that (a) I don't want to have to recompile and (b) I want to program in a higher level language. I'm just scared by all the solutions offered so far because they look like a lot of boilerplate code, and I wanted my scripts to be fairly minimal. Besides, I figured much of the work done in these scripts would be procedural in nature anyway (all of my OOP is in the C++ side).

Quote:
 EDIT: ofcoruse u can put it all in C++/C Lua has a complete C api which duplicates every single facet of the normal Lua syntax, but why would u want to do that when u can dispatch any global Lua function from C/C++ with ease?

So that I'd be able to put less code in my scripts. If my scripts need code to manage the scripts, I feel I might as well code my entire game in Lua.

Again, if Lua is not designed for the kind of scripting I'm trying to fit it in, is there another language that follows my way of thinking better?

</rant>

In an attempt to prove that I've learned something from this thread; if this is what my original script looked like:
local variable1 = foo...local variableN = baz# Functions called by NPC  objects in C++function nextAction(arg1, ... , argN) # Could these be local?    ...    return actionCodeendfunction currentState(arg1, ... , argN)    ...    return stateCodeend...# Functions only this script useslocal function blarg(arg1, ... , argN)    ...end...

Then I have to rewrite the script like this?
AI{    variable0 = foo,    ...    variableN = baz,    # Functions called by NPC  objects in C++    nextAction = function(arg1, ... , argN)        ...        return stateCode    end, # Since this is a table, does a comma go here?    currentState = function(arg1, ... , argN)        ...        return stateCode    end,    ...    # Functions only this script uses    blarg = function(arg1, ... , argN)        ...    end,    ...}

Here, is AI treated as a function that returns a table (in C++)? And I think I missed how to handle different NPCs that use the same script (with different values for their "local variables").

Share on other sites
Your close, but in Lua there is a better way. It involves using the metatable though, this is the way u keep per instance data but use a common implementation. It's analogous to the C++ classes and objects.

here is a sample implementation. Note this is just one way to do it, i'm sure there are many ways to do this.

This is the main program..

---------------------------------------------------------------demonstrate script loading with consistent API across all--scripts--global accessible to all scriptsglobalValue = 1.123;--original instances of these loaded scripts, we keep them around and use them to create new instances of the objects we need.local scriptA	= require "scripts/ScriptA";local scriptB   = require "scripts/ScriptB";--when we need an instance we new itlocal a = scriptA:new();local b = scriptA:new();local c = scriptB:new();--some test seting local valuesa:set(1);b:set(2);c:set(3);--setting a shared value between all instance of that scripta:setShared(7.89);--now we print out the states, note how we call functions to an object using--the : convention, this is important.a:foo();b:foo();--set global valueglobalValue = 3.124;--check on the unrelated scriptsc:foo();

This is script A

---------------------------------------------------------------------common header to all scritps using this scheme of auto registeration--dont need to change anything in this header.local Script = { }	--local table no possible namespace collisionfunction Script:new()   local __object = { }   setmetatable(__object, {__index = Script})   __object:init();   return __object; end------------------------------------------------------------------- ---------------------------------------------------------------------shared values/functions will be accessible to all instance of this particular scriptlocal sharedValue = 0;--this function is only accessible to instances of this scriptlocal function sharedFunction()	print("Shared function called");end--since this is global any script can call this, even other AIsfunction globalFunction()	print("Global function called");end---------------------------------------------------------------------init function called first thing upon object creation--any variables defined in here are only accessible to a particualr--instance, this is where all per instance state data is stored.--even though this function doesn't have the keyword local is it --local by the fact that its owner table Script is declared local above.-------------------------------------------------------------------function Script:init()	print("init called");        --define new member variables here, this is per instance data        --accessed using the self. convention.	self.x = 0;end ---------------------------------------------------------------------update function-------------------------------------------------------------------function Script:update() 	--do something?end---------------------------------------------------------------------shutdown function-------------------------------------------------------------------function Script:shutdown() 	--cleanupend---------------------------------------------------------------------misc functions accessible to this object-------------------------------------------------------------------function Script:foo()	print("local "..self.x.." global "..globalValue.." shared "..sharedValue);end function Script:set(val) 	self.x=val; endfunction Script:setShared(val) sharedValue=val; end---------------------------------------------------------------------automatically returns table for this object, in this scheme--always needs to be the last line.return Script;-------------------------------------------------------------------

This is script B

local Script = { }	--local table no possible namespace collision function Script:new()   local __object = { }   setmetatable(__object, {__index = Script})   __object:init();   return __object; end -------------------------------------------------------------------local sharedValue = 0;--init function called first thing upon object creationfunction Script:init()	print("init2 called");	self.x = 0;end function Script:update() 	--do something?endfunction Script:shutdown() 	--cleanupend--misc functions accessible to this objectfunction Script:foo()	print("local "..self.x.." global "..globalValue.." shared "..sharedValue);end function Script:set(val) 	self.x=val; end---------------------------------------------------------------------automatically returns table of this objectreturn Script;

When u run the main script you'll get something like this.

init called
init called
init2 called
local 1 global 1.123 shared 7.89
local 2 global 1.123 shared 7.89
local 3 global 3.124 shared 0

It has private instance data ( accessed through the self. notation ) shared local data and global data access. All scripts can use a common api Script:init, Script:shutdown, etc.. You can change the table name from Script to whatever you want but u need to define functions local to this script using the : notation.

That's the simplest scheme I can think of. I dont know of any other scripting language which will just let you do what you want out of the box, maybe Python but i've not used it.

Good Luck!

-ddn

Share on other sites
Quote:
 Original post by Guy MehTechnically, yes. My scripts in general are basically implementations of C++ methods except that (a) I don't want to have to recompile and (b) I want to program in a higher level language.

Why don't you simply prefix your function or use a table as name space?
C++
class NPC1 { void nextMove() {  call_lua_function("NPC1.nextMove",this->state); } ...}

Lua
NPC1 = { nextMove = function(npc1_state) ... end, ...}-- for another type of NPCNPC2 = { nextMove = function(npc2_state) ... end, ...}

the Lua functions can access global variables and still have local variables in their own name space, the object specific variables are stored in the C++ class and passed to the Lua functions. Now you just need a function like "environment()" in Lua that returns information about the rest of world relevant for the NPCs. (Or send environment as second parameter)
You can also reuse some functions easily, for example
NPC3 = { -- behaves similar to NPC1 nextMove = NPC1.nextMove, ...}

This a non-OOP solution but very minimal and simple.

Share on other sites
I found Kambiz's solution interesting. So, for each script, I'd enclose all of the functions in a table, and the table name is different for each function. This solves the problem with function names colliding, and is especially good for objects that don't need their own state. I was hoping to avoid putting any Lua AI state inside C++ since I wanted my AI's to have as few assumptions as possible so that I could experiment with any AI I wanted. Don't worry about how the AIs detect their environment; I pass in all required information when I need to.

ddn3, I noticed that your main program is another Lua script. How different should I expect it to look in C++? Also, is there any way to replicate some of the repeated code in the scripts like the header in C++ (like, say, when the script file is loaded) so that there's less work to do in the script files themselves?

Share on other sites
Quote:
 Original post by Guy MehI found Kambiz's solution interesting. So, for each script, I'd enclose all of the functions in a table, and the table name is different for each function. This solves the problem with function names colliding, and is especially good for objects that don't need their own state. I was hoping to avoid putting any Lua AI state inside C++ since I wanted my AI's to have as few assumptions as possible so that I could experiment with any AI I wanted. Don't worry about how the AIs detect their environment; I pass in all required information when I need to.
Dude, did you even read the link I posted? That was exactly what I advised you to do.

Share on other sites
Quote:
 Original post by SneftelDude, did you even read the link I posted? That was exactly what I advised you to do.

*feels embarrassed*

Share on other sites
If you don't need to store per instance data, then wrapping the functions in a table is the best way, as Sneftel suggested. If you do want per instance data I suggest you use something like I suggest, it's basically creating a class in Lua, but with exclusive file scope namespace ( so you don't have to worry about namespace collision between scripts ).

Now that I think about it, I can see some use for this methodology. I create levels in Lua and it's a pain in butt to name them uniquely (ie Level1, Level2, etc..). Using this technique i can just use a common api (ie Level:init(), etc..).

Good Luck!

-ddn