Jump to content
  • Advertisement
Sign in to follow this  
vinnyvicious

Designing a concurrent game engine with Actor Model

This topic is 915 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

I've recently started researching the possibility of writing a game engine using the actor model, wrapped around an ECS. Systems could run concurrently and dispatch their results in messages. However:

  • Shouldn't the rendering layer be "blocking"? You must render something each frame, should you wait for all systems or just render what you have, queuing the messages from subsystems for the next frame and render then?
  • What about physics and timesteps? How would that work?
  • Does anyone know any good books or materials on the subject?

Share this post


Link to post
Share on other sites
Advertisement

if you're talking about splitting up the code into different modules or systems that communicate via a messages system, i was just thinking about that the other day.

 

for single thread, i saw no real advantage to using a message system vs using method/function calls.

 

for a concurrent system, you'd want all the stuff you could do concurrently to be a separate module - basically a stand alone process, and a message system would be used for inter-process communication, just like in a multitasking environment, such as windows, with its windows messages.   when it was time to do something that could be done concurrently, you'd fire off all the appropriate "actors" at once, then wait for them all to respond, then continue.  the advantage being that multiple actors running concurrently should all respond in a shorter amount of time than doing it non-concurrently.  not everything is parallel processing by nature. in fact it would seem that precious few things truly are in game software.   movement isn't, you can move two things at once independently and still do collision checks between them.    combat could be - simultaneous attack and resolution.   render possibly, a zbuf could handle it. but GPUs don't really support it.  input isn't enough of a bottleneck to warrant concurrent processing. so that seems to leave update. also not usually a bottleneck, unless you have very complex AI.  for update, the problem is its movement, and that means collision checks. all you could do would be to process the movements in parallel, then resolve collisions one at a time non-concurrently.  so you save a bit on new_x=current_x+speed*sin(yr)*DT and new_z=current_z+speed*cos(yr)*DT, and whatever else you do that's independent of all other update-able game objects - but that's it. odds are the message system and job dispatch overhead would make it not worth it.  concurrent processes need to be independent of each other, or they have to wait for each other, or they need to use buffers to keep each other busy all the time - similar to buffers that must be periodically filled to feed the audio card.  

Share this post


Link to post
Share on other sites
There are many concurrency models.

The one you described, where you have a task queue that workers pull from concurrently, is one of the most commonly implemented and simplest systems out there. It usually only works well when you have a large number of tiny, unrelated tasks with no interdependencies and little communications among them.

The big steps in concurrency are PCAM: Partition, Communication, Agglomeration, and Mapping. Partitioning the work into right-sized chunks is important. Your question about using an ECS system uses those entity's tasks as the level of partitioning. Other systems will partition over large loops of analysis, partition over data sets, partition across collections, or partition in other ways. Your worker systems don't really think about communications patterns but in higher-performance systems it is a major concern. Objects need to communicate with other objects and systems. They need to query for nearby targets, query for status of other items, query for game options and settings, etc. In bigger systems, understanding and planning for those communications systems is important. Next agglomeration. The common task queue doesn't care to cluster tasks back together, the first one takes the next task without regard for data locality or communications locality. Same with mapping, spread it across all processors and don't care about processor affinity.

As you're going with the Actor model, you're basically limiting yourself to only the most boring type of parallel tasks. It is often used by general gameplay development, effectively just independent tasks of the nearly-embarrassingly parallel variety.



In PC development, the biggest drawback of concurrency systems in video games stem from the fact that you don't have fixed hardware nor do you know what is in the system. You need to plan for the bad cases and not just the ideal. In game consoles you know the hardware, but in PC you get whatever the customer happens to have, which ranges from garbage to awesome.

The typical bad cases in games these days is to assume you have two processors available. While the concurrency models work great when you've got 4, 6, 12 cores available, they can help your game provide 'fluff' material like more particles and effects, but they should not be relied on for core game features because sometimes you'll only get 2 cores.

Now on to your bullet points;

* Should rendering be blocking processing?

No. Rendering and simulation should be different. Decouple your rendering from your simulating. Better engines will do this for you; your simulation will be 1-2 steps ahead of the visuals, interpolating between the results.

* Physics and timesteps?

Physics is part of simulating, and should therefore be decoupled as well. Your simulation steps should be at regular intervals, rendering should be somewhat behind and interpolating along the way.

* Good books on the subject?

The various Gems books (Game Programming Gems, GPU Gems, etc) have quite a lot, but many of the articles they contain are getting dated. Gamasutra has many good articles, the GDC Vault has much good stuff.

Always be sure to check the date, since good practices are frequently changing and patterns that worked well in one generation may work badly just a few hardware generations later. Skip five or six years and the landscape is different, with different bottlenecks and different concerns.

Share this post


Link to post
Share on other sites
I've implemented the Actor Model in am entity system before, where each component was an Actor from the Actor Model. It was nice in that components could use traditional OOP game logic, but the entity system would run them across many threads.

I don't know if this would be as beneficial for "systems", as there's a smaller number of them, so a more special-purpose results-passing and scheduling option would be feasible, rather than something as generic as the actor model.
Also, systems have a lot of internal concurrency, so geerally you want to update systems in a serial order, but have each one make use of every thread.

To get the Actor Model working for me, one big issue was determinism. With all this fine grained message passing between concurrent objects, it's easy to let the order of message execution essentially become random, which is bad.
To fix this, I broke each timestep up into "message cycles", which are like sub-frames.
The frame would start with no queued messages, so the ball go rolling by calling Update on the entities. This would then generate the first round of messages, which would go into a shared queue. After every Entity has had its update method called, the queue is synchronized/merged from all threads, and then is stable-sorted first on destination-actor-id, then by function-name, then by source-actor-id, giving a deterministic order for the messages, and grouping them by target actor. You can then partition this message list by target actor, and then let every worker thread start processing messages for a different set of actors.
While processing messages, more messages are generated, which go into a new queue, which undergoes the same sorting process at the end of the cycle (which is wheb all messages have been executed).
When a cycle results in no messages being generated, the timestep os complete.

There's a bunch more work to do in allowing message return values (futures), and resolving those futures by passing them as message arguments, which delays the carrier message by 1 or more cycles to give the future time to be calculated... And work to do in deterministic ID assignment, and multi-threaded reference counting and destruction of actors...

Share this post


Link to post
Share on other sites

Is this going to wrap all game development tasks into systems - including audio, reading user input, file I/O, etc? If so, there will be many systems that do not need to be stopped when rendering, and some which must function in the rendering thread on certain systems (looking at Windows), or which can be implemented such that there is no need for its own thread (audio, using system mixers; asynchronous I/O). There will also be systems which are also too interrelated for execution in different threads (locking overhead would be too great), but not so interrelated that one must explicitly be invoked prior to the other - an asynchronous method might be more appropriate in this case.

The best implementation might be to defer decisions about how systems should be implemented and run to the programmer using the engine. This allows you to focus on providing the abstractions and tools for the engine's user to implement their own systems, and later you could provide some generally useful system implementations as separate modules.

Your engine would then be more of a "skeleton", and the "meat" would be left for the game developer.

I know I'd be much more likely to choose such an enabling framework over one that removes optimization opportunities from my hands for the sake of simplicity.

Edited by nfries88

Share this post


Link to post
Share on other sites
researching the possibility of writing a game engine using the actor model, wrapped around an ECS

This is the solution.

 

What was the question? What lead you to ending up with "actors + ECS"?

Edited by Alberth

Share this post


Link to post
Share on other sites

note that when i said input was a poor candidate i was thinking single player.

 

in something like an MMO, input is already concurrent due to the client-server models typically used.

 

AI could be a good candidate,  you could get all multi-player input concurrently, then run all AI concurrently based on the current state of the simulation (IE where everybody is right now, before you move them), then move everything based on the input or AI results.

 

and FYI, i'm an example of one of those user with bad hardware. dual cores at 1.3Ghz.

Share this post


Link to post
Share on other sites
Sign in to follow this  

  • Advertisement
×

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

We are the game development community.

Whether you are an indie, hobbyist, AAA developer, or just trying to learn, GameDev.net is the place for you to learn, share, and connect with the games industry. Learn more About Us or sign up!

Sign me up!