Sign in to follow this  
Tangletail

Just Discovered this about Lua...WTF

Recommended Posts

So... after reading so many blogs about Lua being a single threaded enviornment. I just discovered that Lua does actually support threads. And not in the co-routine fashion.

 

http://lua-users.org/wiki/ThreadsTutorial

 

 

Seriously... what the hell.

 

Has anyone had any experience with this? Because I'm practically kicking myself in the teeth after scrapping one of my previous designs to thread actors.

Share this post


Link to post
Share on other sites

The lua_lock approach? Yeah, it's not great.

 

It basically boils down to: Lua will call the lua_lock() and lua_unlock() macros around stuff it thinks needs a lock. You must re-define those macros to take an appropriate platform-specific lock in your code. This is the "lock before every access" approach to "multihreading" which can very easily result in a system that has many threads all acting serially because only thread gets the lock at a time.

 

 

In my experience, with the same amount of effort, you can devise some alternative method of solving whatever problem you've got that will be far better suited for multithreading, so I'd stop kicking yourself now.

Share this post


Link to post
Share on other sites

Honestly, I don't need the Lua lock and unlock methods. The Lua create childthread was all that I needed.

 

The old system I tried to implement mostly eliminated data races by breaking up critical functions and limiting reading and writing access.

 

The data path followed this with a job system.


UI -> Threaded AI - > Sync -> physics -> sync -> Entity State Update -> sync -> draw submit.

 

 

When the AI is running, Entity States are read only. Each AI is allowed to send messages to it's own entities, but not each other. This data is not applied until specific points in the pipeline.

 

For instance, once the AI does its updates, and the reading phase is done. We deactivate reading, switch to writing, and go ahead and allow the movement system to update it's velocities so it can follow any paths created by the AI.

The physics is ran after the syncing point. The physics system will generate a list of messages. The system will wait untill all physics jobs are done, giving the engine the go ahead to push onto the stateful updates. The messages will be pushed to Lua all at once.

 

Lua will then update the entity states (Health, animation, looking directions, etc).

 

The biggest problem would be adding data to the end of a list asynchronously. Theoretically, it might not be a problem. I think Lua uses a form of a linked list or something for arrays.

Edited by Tangletail

Share this post


Link to post
Share on other sites
Honestly, I don't need the Lua lock and unlock methods. The Lua create childthread was all that I needed.

 

 
I think you might be misunderstanding that page... or I'm misunderstanding what you mean. By "lua create childthread" do you mean lua_newthread?
 
You can't call lua_newthread if you don't provide lua_lock and lua_unlock implementations (well you can, but you'll have race conditions because no locking will be performed!). lua_newthread calls lua_lock and lua_unlock. Every lua_pushWhatever function calls lua_lock and lua_unlock.
 
Remember Lua's "threads" are not OS threads. lua_newthread is creating a child state.

Share this post


Link to post
Share on other sites

Yes, it's rather silly, and has trapped many, many people. I don't follow Lua enough to know the history behind the name, it's possible it was born when Lua got co-routine-like "threads" internally and maybe made more sense in an earlier context or something.

Edited by Josh Petrie

Share this post


Link to post
Share on other sites

Yeah, if you think you need to share Lua states among multiple threads you're in for a world of hurt. If you need to add multithreading features to your application (and have found that it needs to happen at the Lua level and not lower in your application) then the only sane way is to have one Lua state per thread and communicate between them via message passing.

 

(not talking about coroutines here)

Share this post


Link to post
Share on other sites

Actually... it turns out that there is in fact a way for external states to access a main global state from another state without messages.
 

 

Someone made something called Lua proxies. It's probably a little slower than direct access as you're using a pointer. So far I've only heard of it reading... it'd be interesting if it's capable of writing.

 

I'm not certain how safe it would be when editing certain data types however. Adding onto an array, or changing a string might cause issues. But changing a value of fixed size may not cause anything.

 

If Lua's heap handles multi threaded access it'll likely be a non issue. But I sincerely doubt it.

Edited by Tangletail

Share this post


Link to post
Share on other sites

Actually... it turns out that there is in fact a way for external states to access a main global state from another state without messages.
 

 

Someone made something called Lua proxies. It's probably a little slower than direct access as you're using a pointer. So far I've only heard of it reading... it'd be interesting if it's capable of writing.

 

I'm not certain how safe it would be when editing certain data types however. Adding onto an array, or changing a string might cause issues. But changing a value of fixed size may not cause anything.

 

If Lua's heap handles multi threaded access it'll likely be a non issue. But I sincerely doubt it.

 

Yes, it's "possible", and "may" work, but it's just the naive "lock state, do stuff, unlock state" which ends up (at best) being inefficient or (at worst) breaking your program in infinitely many subtle ways with race conditions, non-atomic operations, etc... even if you only read something, the other state might write to it which could cause your reader to read incoherent data. That way lies madness, if you want to write a multi-threaded application you need to approach the design intelligently instead of just mashing threads together, Lua doesn't change that fact :)

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this