Sign in to follow this  
ZealGamedev

[Lua] Multi threading Lua...

Recommended Posts

I have been seeking help on the official Lua forums, but not many people post there, so I thought I would try my luck here. To recap my problem -

I have two C threads, and two lua_States (created via lua_newthread(mMainLuaState)). I have a function setup that ensures correct data parallelism, so in theory thread1 and thread2 should be able to both call the function at the same time (the two calls have their own environment, and only read static read only globals). However, I am getting one crash after another (no helpful information provided in the logs sorry).

Check out my original thread for more details and example code...

http://forum.luahub.com/index.php?topic=2596.0

There is no way I am the first person to try this... somebody out there must know what I am 'missing'?

*fyi I have NOT defined lua_lock/unlock, but for what I am trying to do, I should not have to right?

*here is the example code to save you from having to check out the other thread...

Lua

-- Create a unique message buffer for each thread
msgs2 = {}
msgs3 = {}

-- Declare two unique functions for each thread
function foo2() table.insert(msgs2, 777); end
function foo3() table.insert(msgs3, 777); end


C

mBarrier = new boost::barrier(3);

// The main lua state is initialize (mLua), create two 'sub states'
mLua2 = lua_newthread(mLua);
assert(lua_type(mLua, -1) == LUA_TTHREAD);
int lua2Ref = luaL_ref(mLua, LUA_REGISTRYINDEX);
assert(lua_type(mLua, -1) != LUA_TTHREAD);

mLua3 = lua_newthread(mLua);
assert(lua_type(mLua, -1) == LUA_TTHREAD);
int lua3Ref = luaL_ref(mLua, LUA_REGISTRYINDEX);
assert(lua_type(mLua, -1) != LUA_TTHREAD);

// Spawn two threads which use mLua2/mLua3 to call foo2/foo3
boost::thread thread2(boost::bind(&Sim::lua2Task, this));
boost::thread thread3(boost::bind(&Sim::lua3Task, this));

mBarrier->wait();





void call(lua_State* lua, std::string& func)
{
lua_getglobal(lua, func.c_str()); assert(lua_type(lua, -1) == LUA_TFUNCTION);
if (lua_pcall(lua, 0, 0, 0)) Aux::log(lua_tostring(lua, -1));
}

void Sim::lua2Task(void)
{
for (int i = 0; i < 10000; ++i)
call(mLua2, std::string("foo2"));
mBarrier->wait();
}

void Sim::lua3Task(void)
{
for (int i = 0; i < 10000; ++i)
call(mLua3, std::string("foo3"));
mBarrier->wait();
}


Share this post


Link to post
Share on other sites
Quote:
Original post by ZealGamedev
I have two C threads, and two lua_States (created via lua_newthread(mMainLuaState)). I have a function setup that ensures correct data parallelism, so in theory thread1 and thread2 should be able to both call the function at the same time (the two calls have their own environment, and only read static read only globals). However, I am getting one crash after another (no helpful information provided in the logs sorry).
Lua is not thread-safe. If two lua_States share the same global state (as occurs when you use lua_newthread) then you cannot use them concurrently from different C threads.

Incidentally, there is no "official Lua forum". Most users post on the lua-l mailing list.

Share this post


Link to post
Share on other sites
So youre saying I AM the first person to ever try and do this? Nobody has ever tried to do a simple data parallel task with Lua? That seems VERY strange considering how popular Lua is in the games industry.

Somebody else out there MUST have figured out a way to do this...

Share this post


Link to post
Share on other sites
Quote:
Original post by ZealGamedev
So youre saying I AM the first person to ever try and do this?
No. Where did I say that?

It's perfectly okay to multithread Lua as long as the states are completely separate (created with lua_newstate), and this is commonly done. If you need a more complete solution, Lua Lanes wraps the thread handling process and enables communication between the states, but of course you already know about Lua Lanes, since it's the very first google result for "lua multithreading".

Share this post


Link to post
Share on other sites
From http://lua-users.org/wiki/ThreadsTutorial

Quote:

Each thread in C which interacts with Lua will need its own Lua state. Each of these states has its own runtime stack. When a new C thread is started, you can create its Lua state in one of two ways. One way is to call lua_open. This creates a new state which is independent of the states in other threads. In this case, you'll need to initialize the Lua state (for example, loading libraries) as if it was the state of a new program. This approach eliminates the need for mutex locks (discussed below), but will keep the threads from sharing global data.

The other approach is to call lua_newthread. This creates a child state which has its own stack and which has access to global data. This approach is discussed here.


That sure seems to imply that two c threads can run in parallel and access global data. The article goes on to talk about implementing the lua_lock/unlock defines... however I really want to avoid locks if possible. I should have the option to avoid locks if I 'promise' lua that two threads will never step on each others data, right?

Perhaps I need to simply set a unique environment (lua_setfenv()) for each thread/function call? Shouldnt that give each thread its own global space, and then I can pass in any global data from the main environment as read only data?

Share this post


Link to post
Share on other sites
Dude, you can bargain with it all you want, you're not going to turn Lua inherently thread-safe. For the sake of GC, string interning, upvalues, etc., threads write to the global_State quite frequently, even when you aren't operating on what you think of as "global data". So yes, to do this you need locks. [See the rest of the page you quoted for details about this.] But that'll seriously hamstring performance, because of the frequency of locking and the amount of expensive operations performed while the lock is held. (It's better in 5.1 than in earlier versions, but it's still not good.) This is why people who need preemptive threading in Lua tend to use separate global states.

Share this post


Link to post
Share on other sites
Quote:
Dude, you can bargain with it all you want, you're not going to turn Lua inherently thread-safe


Dude, I am not trying to make Lua "thread-safe". I am trying to do some simple data parallelism.

Quote:

threads write to the global_State quite frequently, even when you aren't operating on what you think of as "global data".


Can you give an example? Can you point to where Lua is operating on the "global_State" in the example I posted? Unless there is something funky going on when allocating memory for tables, the two functions should never require any shared "global_State".

Quote:
...This is why people who need preemptive threading in Lua tend to use separate global states.


Good thing I am not trying to do preemptive threading eh?

Look, while I appreciate your optimism, I dont think you quite understand what I am trying to do here. Imagine each Lua function is a fragment shader. Each function will be given only thread safe (read only for example) input to perform its task, and will write its output to some unique area of memory. Once all functions have been called, the threads synchronize, so the main thread can process the output.

Share this post


Link to post
Share on other sites
Quote:
Original post by ZealGamedev
Can you give an example? Can you point to where Lua is operating on the "global_State" in the example I posted?
Sure thing. You're calling table.insert, which internally invokes lua_rawset. This changes the size of the table, resulting in the memory counters in the global_State being incremented. Non-atomically. The global lookup of msgs2 probably also puts something on the grey-list in the global_State. (EDIT: and that's assuming it doesn't happen to run an incremental collection, which'll chew on a lot more than that.)

Quote:
Quote:
...This is why people who need preemptive threading in Lua tend to use separate global states.


Good thing I am not trying to do preemptive threading eh?
Of course you are trying to do preemptive threading. That's what OS-level threading is. It's preemptive. If it wasn't preemptive, it'd be coroutines. The OS preempts a thread after its timeslice is over, in order to run a different thread. This preemption can create problems if the thread has left state which is visible to other threads in an invalid state. That's why threading is difficult. Because it's preemptive.

Seriously, do me a favor. Walk through the entire execution of a call to lua_newtable in your debugger, descending into every function call. I think it'll be eye-opening for you.

Share this post


Link to post
Share on other sites
Quote:

You're calling table.insert, which internally invokes lua_rawset. This changes the size of the table, resulting in the memory counters in the global_State being incremented. Non-atomically


Now were getting somewhere. So youre saying that even when I manipulate a data structure that is local to a functions environment, it still writes to global "memory counters"? Doesnt that strike you as kind of bad design? I see no reason why allocating local data needs to involve the global state (aside from garbage collection, but that could be solved in other creative ways).

Quote:

Of course you are trying to do preemptive threading.


What I meant is I am not yielding threads manually. I am running one thread per hardware core, and my task manager runs them cooperatively until all tasks are complete.

Share this post


Link to post
Share on other sites
Quote:
Original post by ZealGamedev
I see no reason why allocating local data needs to involve the global state (aside from garbage collection, but that could be solved in other creative ways).
Spend some time thinking about string interning and cross-coroutine upvalues, and you'll get a better idea as to why it's necessary. As for garbage collection, if you want to try implementing a thread-safe, lock-free, incremental generational garbage collector, well.... it would be a very useful thing to have, for Lua and for pretty much any other language out there.
Quote:
What I meant is I am not yielding threads manually. I am running one thread per hardware core, and my task manager runs them cooperatively until all tasks are complete.
That's what makes it preemptive: The lack of yielding threads manually.

Share this post


Link to post
Share on other sites
Quote:

Spend some time thinking about string interning and cross-coroutine upvalues, and you'll get a better idea as to why it's necessary.


Well lets just focus on "local t = {}", because that also seems to be crashing too. Why is a local allocation, that is only visible to the calling thread, not thread safe? *Why CANT it be thread safe I guess is the real question.

Quote:

As for garbage collection, if you want to try implementing a thread-safe, lock-free, incremental generational garbage collector, well.... it would be a very useful thing to have, for Lua and for pretty much any other language out there.


Well for my case, it would be sufficient if each state simply tracked its own allocations, and had its own GC. I dont plan to have these threads ever write to any true 'globals', I only want to read them.

Share this post


Link to post
Share on other sites
It sounds like you have very specific needs, and little patience for languages which do not specifically prioritize them over others' needs. I suggest that you write your own interpreter.

Share this post


Link to post
Share on other sites
Quote:
It sounds like you have very specific needs, and little patience for languages which do not specifically prioritize them over others' needs. I suggest that you write your own interpreter.


Really?


function doSomethingExpensive(gameEntity)
data = getReadOnlyGlobalData()
...
gameEntity.localState = somethingNew
end


...Is that "very specific"? Seems pretty generic to me. And VERY useful too (especially for games). There are a million cases where you might need to run a complex function on all your game entities. And if that function only relies on some thread safe data, and a unique output buffer, wouldnt it be nice to run at 8x speed?

And its not a "lack of patience", I am just surprised that Lua (the king of game scripting languages) doesnt support something like this.

Share this post


Link to post
Share on other sites
Quote:
Original post by ZealGamedev
From http://lua-users.org/wiki/ThreadsTutorial
Quote:
One way is to call lua_open. This creates a new state which is independent of the states in other threads. In this case, you'll need to initialize the Lua state (for example, loading libraries) as if it was the state of a new program. This approach eliminates the need for mutex locks (discussed below), but will keep the threads from sharing global data.

The other approach ...
^^Doesn't this sound like what you want? Completely seperate lua states without requiring mutexes?

To share the read-only global input data between each state, instead of putting it in a Lua global, you access it through a C/C++ function that you've bound to Lua.

For more info, check out the docs from the BitSquid engine, specifically section 3.5 and 3.7: http://www.bitsquid.se/files/Lua API.pdf
Quote:
Original post by Sneftel
This is why people who need preemptive threading in Lua tend to use separate global states.
So, isn't the answer to his problem to explain how to use separate global states? i.e. tell him how to do this in Lua, instead of how to do it the wrong way in a hypothetical new language that he'll develop for the purposes of not learning how to use Lua properly? Fuck...

Share this post


Link to post
Share on other sites
Quote:

^^Doesn't this sound like what you want? Completely seperate lua states without requiring mutexes?


Each thread/state needs to access a shared set of read only data (updated at a different point in the frame). Plus having to load the entire lua library, and all my game scripts for each thread seems a tad inefficient (especially when were talking about scaling to n cores).

Quote:

To share the read-only global input data between each state, instead of putting it in a Lua global, you access it through a C/C++ function that you've bound to Lua.


A lot of data (most all of the game specific data) doesnt really belong in C (C cant really do anything with it). So I am trying to avoid the overhead of moving large amounts of data to/from lua/c. Ill take a look at that link you posted though...

Share this post


Link to post
Share on other sites
Quote:
Original post by ZealGamedev
Each thread/state needs to access a shared set of read only data (updated at a different point in the frame).
Sorry, I edited in more info on this above. Store the 'global' data in the host program, not in Lua.
Quote:
Plus having to load the entire lua library, and all my game scripts for each thread seems a tad inefficient (especially when were talking about scaling to n cores).
Often, greater thread/core isolation is paid for by 'wasting' RAM. Would you rather use more RAM or use more mutexes? ;)
[edit]Also, the BitSquid guys have written several PS3/360/PC engines, so I'd trust them on this trade-off =D

Share this post


Link to post
Share on other sites
Lua was never designed to be truly multi-threaded. But it is designed to not depend on *real* global state (there are no mutable globals in Lua.dll), independent lua_States can be safely used in different threads.

This situation is a lot better than many scripting languages, Google "global interpreter lock" for more information.

Depending on what you are doing, you might be able to avoid loading all scripts into each state, and avoid copying excessive data into each.

Share this post


Link to post
Share on other sites
Quote:
Original post by Hodgman
For more info, check out the docs from the BitSquid engine, specifically section 3.5 and 3.7: http://www.bitsquid.se/files/Lua API.pdf
Quote:
Original post by Sneftel
This is why people who need preemptive threading in Lua tend to use separate global states.
So, isn't the answer to his problem to explain how to use separate global states? i.e. tell him how to do this in Lua, instead of how to do it the wrong way in a hypothetical new language that he'll develop for the purposes of not learning how to use Lua properly? Fuck...
I did, like, eight replies up. (Lua Lanes.) I'm done here.

Share this post


Link to post
Share on other sites
Quote:

I did, like, eight replies up. (Lua Lanes.)


Mentioning Lua Lanes (in the super awesome not at all condescending way that you did) hardly addressed the questions I was asking.

Quote:

I'm done here.


Awe come on dont be like that.

@Hodgman

Thanks for that bitsquid link, I just read their docs... I had contemplated solving this problem by moving all my 'global game data' from Lua to c, but I am still a tad hesitant for the following reasons - Currently the only data I manage in C is data that 'belongs' to C systems (meshes for the renderer, bodies for the physics system, ect...), while the 'game specific' data (health, ammmo, ect...) exists in Lua. Since the C systems (Graphics engine, Physics engine) have no concept of health or ammo, this makes sense. The 'game specific' systems tend to exist in Lua, so thats where I would like to keep the 'game specific' data.

However, the bitsquid guys make a couple good cases for why you may want to store all your Lua data in C. It gives you tighter control over memory allocation, and can reduce the strain on the Lua GC. So its definitely something I am considering more seriously now... It would sure make this damn problem a lot easier to solve no doubt!

But just out of curiosity, does anyone know the logic/benefit behind why even LOCAL allocations are not thread safe? Take my original example form the first post, and forget about globals. If you simply allocate a local table, the thing freaks out. I thought lua_newthread() gave you a completely independent and therefore thread safe stack at least?



Share this post


Link to post
Share on other sites
Tables are garbage collected, they aren't allocated on the stack. A reference to them is placed on the stack. Lua doesn't try to make such allocations thread safe. Another possible issue is hooks. Lua has some memory and debug hooks, some of them might be triggered by such an allocation. These would have to be thread safe too.

Share this post


Link to post
Share on other sites
Quote:
Tables are garbage collected, they aren't allocated on the stack


Really? I always assumed all local allocations avoided the GC and just got deallocated when they went out of scope. Maybe some special cases are needed for closures/upvalues? But thats not the case in my example...

So it sounds like the GC is the primary reason why there is a problem (although you mentioned there could be other problems, like hooks ect...)?

Share this post


Link to post
Share on other sites
Quote:
Original post by ZealGamedev
Quote:
Tables are garbage collected, they aren't allocated on the stack


Really? I always assumed all local allocations avoided the GC and just got deallocated when they went out of scope. Maybe some special cases are needed for closures/upvalues? But thats not the case in my example...

So it sounds like the GC is the primary reason why there is a problem (although you mentioned there could be other problems, like hooks ect...)?


I think the moral of the story here is that without intimate knowledge of Lua's inner workings you should do what the developer recommends. Ask yourself if it's really worth your time to fight it, or should you instead accept what it is and work within those constraints. Maybe you've determined that it is worth your time, I don't know...

Share this post


Link to post
Share on other sites
Quote:

I think the moral of the story here is that without intimate knowledge of Lua's inner workings you should do what the developer recommends. Ask yourself if it's really worth your time to fight it, or should you instead accept what it is and work within those constraints. Maybe you've determined that it is worth your time, I don't know...


Oh I agree. While I am 100% confident this problem could be solved (for starters, a thread safe GC like Sneftel suggested), I am also 100% confident it would require a lot of work. Like I said, I am just surprised this issue has not been addressed before (maybe it has?).

I suppose moving all my data into c and routing all communication through there is looking more and more like the best option. Perhaps the benefits (as described by the bitsquid guys) will out weigh the costs.. Still a shame though.. if only the lua GC/threads behaved slightly differently...

Share this post


Link to post
Share on other sites
Quote:
Original post by ZealGamedev
Like I said, I am just surprised this issue has not been addressed before (maybe it has?).

It was a design trade-off. There is typically no easy answer to such questions.

Making each lua_State thread safe would be very expensive in a single-threaded system. Additionally, the benefits might not be great because such locking inhibits parallelism, and such locks are likely to be highly contended.

Threading is platform dependant, Lua is AFAIK written in pure C with little or no platform specific code.

Share this post


Link to post
Share on other sites
Quote:
Original post by ZealGamedev
I suppose moving all my data into c and routing all communication through there is looking more and more like the best option. Perhaps the benefits (as described by the bitsquid guys) will out weigh the costs.. Still a shame though.. if only the lua GC/threads behaved slightly differently...


well not all your data, maybe just data that needs to be thread safe. Personally I solved the problem by exposing my event framework (which is inherently thread safe) to Lua so I can trigger events from Lua and listen for/handle events in Lua. If I need to send some data to another thread, I package it in an event and set up a listener in the other thread. I find that working through a framework is a solid way to implement threading rather than worrying about and solving the same problems in many places.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this