• Advertisement
Sign in to follow this  

Concurrent programming concepts for scripts

This topic is 4170 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

Almost every day, a new person posts in the Scripting forum, asking for the best way to allow multiple scripts to run simultaneously. The 'obvious' answer is threads; yet there are so many issues there with synchronisation, and they don't scale well to thousands of objects. The more practical and typical answer is 'coroutines', which languages like Python and Lua supply, and which give you most of the advantages of threads with more control over how execution starts and stops, so you then have to go into coroutine scheduling and writing functionality to decide when to wake co-routines, adding an extra degree of complexity. Your low-level synchronisation problems are fixed since only 1 thread is executing at any one time, but higher level ones persist - if 2 scripts refer to one actor, and that actor dies during the execution of one script, what will the next script do when it is resumed? Are there any off-the-wall answers to this, allowing us to develop a new scripting language - or alter an existing one - to allow for intrinsic concurrency in a clean way? - Could you just define 'object now dead' exceptions to be thrown from within your scripts? How practical is this when that exception could be raised from any line? Can you write effective scripts when each line has to assume that it may never get executed? - Perhaps there's some sort of data model that is more robust, such as having transactions and rollbacks? (While still guaranteeing safety, liveness, and meeting all those other fun concurrent programming constraints.) - Perhaps the use of a declarative language rather than an imperative one could mean that shared references can be removed somehow? Any ideas?

Share this post


Link to post
Share on other sites
Advertisement
I think the most promising potential is either in a working, practical implementation of software transactional memory, or moving to a compartmentalized message-passing model ala Erlang. Maybe a hybrid of both, I don't know.

Either way, it seems fairly clear to me that synchronization in a predominantly-imperative, shared-state language is never going to be easy, no matter how many gimmicks are bolted on to the language itself.

Share this post


Link to post
Share on other sites
On the subject of Lua's coroutine model, it's have been formalized full asymmetric coroutines. The bible book Concepts, Techniques, and Models of Computer Programming (CTM) discusses the various models of concurrency from declarative to icky shared-state concurrency.


The best place to ask these kind of questions in the forums of lambda the ultimate

Share this post


Link to post
Share on other sites
Quote:
Original post by ApochPiQ
Either way, it seems fairly clear to me that synchronization in a predominantly-imperative, shared-state language is never going to be easy, no matter how many gimmicks are bolted on to the language itself.


Quote:
Original post by snk_kid
The best place to ask these kind of questions in the forums of lambda the ultimate


To both these points, I say:
- Are there game-specific issues that we need to bear in mind that the average concurrent programmer will not?
- Are there aspects of game development that allow us to make assumptions - thus making the language easier or safer - that the average concurrent programmer cannot make?

That's part of why I'm asking here, really.

Share this post


Link to post
Share on other sites
If there are, I've never observed them.


The only really interesting difference between "normal" programming domains and games is that games are uniquely positioned to take advantage of asynchronous processing (i.e. splitting between the CPU and GPU), but that's generally such a limited and constrained source of concurrency that it is fairly well understood how to get it done. SMP-type concurrency problems, cluster scaling, and so on are actually far more common outside games - it's been my experience that games programmers actually tend to be less familiar with concurrency issues on average.

Share this post


Link to post
Share on other sites
That was quick; I only just finished editing my post [grin].

Quote:
Original post by Kylotan
Are there game-specific issues that we need to bear in mind that the average concurrent programmer will not?


2 answers to that:


  • Making assumptions here, in general and not just game-specific "the average concurrent programmer" may not realize that there is more than one model of concurrency, when mass majority discuss concurrency they are implicitly referring to the model of shared-state concurrency which is the bottom of the pile. As i mentioned above the book I linked to discusses various models of concurrency you may be interested in reading.


  • Tim Sweeney has explored some of the game-specific issues in his paper/slide The Next Mainstream Programming Languages: A Game Developer's Perspective. There are some interesting statistics in there.




Quote:
Original post by Kylotan
- Are there aspects of game development that allow us to make assumptions - thus making the language easier or safer - that the average concurrent programmer cannot make?


I really don't think it needs to be game specific; Tim's paper might give the answers you're looking for.

Anyways it definitely doesn't hurt to ask in lambda the ultimate as well [smile]

Share this post


Link to post
Share on other sites
What you describe sounds just like Erlang:
  • Declarative.

  • Concurrent.

  • A focus on message passing for communication rather than shared state.

  • A focus on robustness and fault-tolerant systems.

  • A transactional interface to shared data with support for rollback in the event of failure.

Check it out [smile]. I also recommend CTM, but I think I'm getting a bad rep because I recommend it so often.

Share this post


Link to post
Share on other sites
Guest Anonymous Poster
Quote:
Original post by Rebooted
What you describe sounds just like Erlang:
Declarative.
Concurrent.
A focus on message passing for communication rather than shared state.
A focus on robustness and fault-tolerant systems.
A transactional interface to shared data with support for rollback in the event of failure.
Check it out [smile]. I also recommend CTM, but I think I'm getting a bad rep because I recommend it so often.


no, you are absolutely right-but on the other hand, you'd normally really not want to use a language such as Erlang as your game's embedded scripting language... so, I agree there must be something in between somewhere. Maybe it would be a good idea to start discussing this with the gamedev folks who have come up with their own scripting languages?

Share this post


Link to post
Share on other sites
Well I think the answer to your question depends on what restrictions you wish to place on the code - as in what language is the main implementation handling the scripts and how smoothly would it integrate with the scripting language, plus what platform do you wish this on? Anyway you could look into Polyphonic C# or JoCaml, both based on the join calculus model on concurrency. I too have just began tackling this and have just began reading on the topic, it looks like this technique will allow for trivial - well not trivial but not so difficult implementation of asynchronous action scripts.

I wish books were not so prohibitively expensive, its as if they dont want people to learn - Ive already overspent my book budget for the month, CTM is a no go :S.

[Edited by - Daerax on September 20, 2006 11:01:55 AM]

Share this post


Link to post
Share on other sites
I think the problems are applicable to any domain that uses embedded languages or mixed-language development, although I do have to admit that games are kind of the poster child domain for that sort of stuff.


Really, it's a simple tradeoff: languages that are really good at concurrency (i.e. typically functional/declarative or using STM) are often highly expensive in terms of space, time, or both. They also tend to be less widespread and well-taught. The net result is that even when game development teams are aware of highly concurrent languages like Erlang, they don't really have the practical support and backing needed to make them viable for embedding purposes. Contrast this with, say, Lua or Python, both of which are borderline trivial to integrate into C++ apps.

I think there's one primary reason why "better" languages don't have a lot of support in the embedding world yet: because concurrency has only just now started to become a real-world development issue. SMP has only recently reached the point where it exists in mainstream gaming hardware; stream processing is still very new outside of highly application-specific domains (where historically DSPs have dominated) and unless Sony has an ass/head extraction operation soon, it's likely to remain obscure.

All in all, there simply hasn't been widespread access to concurrency-capable hardware for very long, and so all solutions are still relatively young. This is pretty much uncharted territory, and the good solutions (I suspect) will be pretty surprising compared to what conventional wisdom of the single-processing era would have predicted.

Share this post


Link to post
Share on other sites
Guest Anonymous Poster
...and that's exactly where the Epoch language comes in ;-)

Share this post


Link to post
Share on other sites
Guest Anonymous Poster
I too thought that erlang's concurrent oriented programming would be a panacea. but after reading the link below, it turns out for large projects with erlang you just end using client/server architecture anyway.

Can someone enumerate one instance in games where concurrent scripts might be used? And how the scripts would interact with game objects such that there'd be problems? I'm have trouble imagining it....

http://www.sics.se/~joe/thesis/armstrong_thesis_2003.pdf

Share this post


Link to post
Share on other sites
Sign in to follow this  

  • Advertisement