Jump to content
  • Advertisement
Sign in to follow this  
Nairou

Extrapolation between game and render threads?

This topic is 3810 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

I have my game core in the main thread, running at a fixed rate for physics and input and such, and I have my renderer in a separate thread running at an unknown variable rate (though usually locked to vsync). Data is passed from the game to the renderer via a rotating set of structures, so the two never touch the same data at the same time. Since the two are processing data at different rates, I'm concerned about potential jitter. If the renderer runs faster than the game updates, it might look like it stutters a bit unless it started interpolating between each update. Likewise, if the CPU gets so bogged down that the game updates are not evenly spaced (it may later have to "catch up" by performing two updates in quick succession), then the renderer could end up needing to extrapolate beyond the last update to where it expected the next update to be. But this is starting to feel overly complex and I may be over-thinking it a bit, especially if the game updates themselves are at a high rate (i.e. 100 per second). Does anyone else run their renderer in a separate thread, and have some advice they can give on passing data back and forth regularly enough to prevent "guessing" on the part of the renderer? I know interpolation and such are used for networking, but doing this just for the renderer seems a bit crazy.

Share this post


Link to post
Share on other sites
Advertisement
Chiming in as I've been considering a similar implementation, and I'd be curious to hear people's experiences/thoughts on the matter.

The benefits of this system as I see it:
1) The renderer can safely wait for vsync without stalling the game logic
2) On multi core systems, it's simply a good way of distributing work (and to those who say rendering doesn't consume much cpu time, I'd beg to differ).
3) It "feels" right :)

However, it certainly presents some challenges. One issue is that, without interpolation, there's little point in running multiple render calls without an update in your game logic / animation / physics. Plus, as you mentioned, there might be jitter. It seems to me that this method "requires" interpolation to be successful.

Additionally, it seems like it's just not going to work if either thread gets too far in front/behind the other; you could either implement some sort of method of detecting when you're far enough ahead to "wait" for the other, or else try extrapolation.

I'm curious, on the whole, how you've found this technique (implementing it, that is). Not too much of a pain? Any hidden gotchas? Other than those mentioned in the post :)... It certainly could be argued that this is too much work for a hobby project, but I know this technique is used in many commercial engines.

Share this post


Link to post
Share on other sites
As you said, the benefits far outweigh the difficulties in getting it to work. As long as you do proper locking and data separation so that the two threads never ever touch the same thing at the same time (this usually involves a strict method of passing data from one thread to the other), then the rest is pretty simple. My current implementation is stable and runs great.

The only issue is the one I described in the first post: my renderer currently only renders the latest update from the game thread, continually re-rendering the same data until a new update is given. When game updates are 100 times a second, rendering appears smooth, but if I turn it down significantly (i.e. 10 times a second), then the jittering really stands out and the renderer just doesn't have enough data.

I'm more concerned with timing irregularities, both in the game update once it has more work to do, and just in the speed differences between the two threads, and how these differences will affect the supply of updates to the renderer, causing jitters that the renderer itself will have to compensate for. I don't know yet whether this is a real concern or not, but wanted to see if anyone else had dealt with this.

Share this post


Link to post
Share on other sites
Since no one else is replying I figured I'd pick your brain on some of the details...

Out of curiosity, how much is your rendering thread doing? Do you essentially pass render commands from the main thread and store them up in a render queue in the render thread? I.e., the render thread is pretty much just setting up shader/render states and calling draw? Or, are you doing things like scene-graph traversal... I'm assuming the former, but could be wrong...

Again, not having actually implemented this, take this all with a grain of salt, but I have been pondering it for a bit.

It seems like you're going to have to interpolate if you want to handle slower main thread updates. The question is how exactly, and the only real way that jumps out at me is to start the renderer some time after the main thread, i.e., give the main thread a head start of 33 ms or something (or whatever interval your main thread's fixed updates will be). Assuming you go that route, then you should always be able to pass two states to the renderer when the main thread updates: the current state and the previous state. Being that the renderer starts behind, it should always remain somewhere between two states, and thus be able to interpolate between them.... Unless the main thread slows down so much that it's no longer able to feed the render thread fast enough. At that point what really are your options? Certainly no good options, this is a case you should try to avoid... either extrapolate or stall the renderer. The other case (of the renderer slowing down and not being able to consume the main threads updates) seems less of a problem at the logic level, as you'd just discard older states, or buffer them up and process them when you get to them. This case is still obviously bad from the player's perspective, but somehow seems fundamentally easier to deal with...

Anyway, not sure if that adds anything that you haven't already thought of... I'd be curious as what your plans were for dealing with these issues.

Share this post


Link to post
Share on other sites
If the rendering thread knows the delay between successive data updates then it can contribute these to an overall average trend made from the previous n-updates and so choose a suitable delta to scale the interpolation. The hope is that the rendering thread will always finish interpolating just as the main thread issues its next update.
In the event that for some reason the main thread slows down suddenly, anomalous to the average trend, such that the interpolation does finish prior to the next update then the rendering thread can just be stalled, although its a much less likely occurance now.

If the main thread has suddenly slowed (rather than a gradual decline) then that might suggest a few things:
(1) There was a sudden change to the global state that requires the main thread to do more, so an extrapolation would likely be very inaccurate.
(2) There was no sudden change but the main thread has become unexpectedly busy doing a background task like maybe streaming in a new level, you could extrapolate here although you dont know how long the thread will be busy for, it would be better to offload the task to a another thread and keep the main thread speedy.
(3) The system became suddenly busy, perhaps a new process unrelated to your game; as its the system itself that has slowed down stalling the rendering thread may in-fact help alleviate this.

Share this post


Link to post
Share on other sites
dmatter:
I hadn't considered stalling the renderer to be a desired option, but you make some good points that I'll need to think over. You mentioned having the renderer keep track of the average delay between data updates, would this still be favorable to just letting the renderer know the configured update rate being used by the main game thread? Or will there always be some measure of delay that the renderer will need to account for to avoid continual extrapolation right before the update arrives? As far as significant slowdown, I guess I can just put a cap on how far the renderer will extrapolate beyond a missing update before it gives up and stalls.

emeyex:
The scenegraph is traversed in the main game thread, and the resulting transform and geometry data is put into a "sceneblock", a large structure that gets passed to (referenced by) the renderer. There are a fixed number of sceneblocks (currently 3), and they get rotated between the game thread and renderer thread.

By cycling a fixed number of sceneblocks, rather than buffering updates, I keep latency to a minimum (at most one update behind) so that the game continues to feel visually responsive to the player.

When the game thread finishes traversing the scenegraph and placing all visible data into it's sceneblock, it sets a flag to indicate to the renderer that it has finished. Once the renderer is ready to render a new frame, it looks for this flag, and if present, it rotates the sceneblock pointers so that it can read the new updated structure. The next time the game thread performs an update, it is now filling a different sceneblock structure, and the process repeats.

A third sceneblock exists in the cycle to give some breathing room, since the two threads are not running at the same speed and will not be finishing at the same time. If either thread is ready for another cycle when the other is not, they just reuse the sceneblock they already have. But as you suggested, the third sceneblock could also be used by the renderer to give two states to interpolate between: the old state, and the newly updated state. I would just need to structure them slightly different to allow easier comparison between them.

Share this post


Link to post
Share on other sites
I'm a big fan of this for RTS games, since (a) a fixed logic update speed is essential, (b) minor fudging of unit positions is fine, (c) a small amount of latency is fine, and (d) most graphical effects are framerate-independent. I'd be much more leery of the approach for something like an FPS.

Share this post


Link to post
Share on other sites
Can you clarify? My game project is an FPS of sorts. The only issue I can think of that would specifically affect FPSs would by latency, but I don't see how that is a big issue here if updates are not buffered for more than one update.

Share this post


Link to post
Share on other sites
Quote:
Original post by Nairou
dmatter:
I hadn't considered stalling the renderer to be a desired option

It isn't desired, but it may not be entirely avoidable; if not then the best you can do is minimise the chances of such occurances.

Quote:
You mentioned having the renderer keep track of the average delay between data updates, would this still be favorable to just letting the renderer know the configured update rate being used by the main game thread? Or will there always be some measure of delay that the renderer will need to account for to avoid continual extrapolation right before the update arrives?

Are you suggesting the main thread will always issue updates at a consistent frequency? My understanding of the problem was that this isn't the case and the synchronisation between the main and rendering thread can drift. This drift is a result of any difference in processing load per update and also the potential 'uneven' distribution of time allotted to a thread by the operating system.
In theory if the main thread could update at a consistent rate then using the pre-configured rate would be preferable, as the delay can change over time then you need to account for this by taking an average. Hopefully this would lead to better results as then you can preemptively scale the interpolation steps right up till the next frame arrives.
You could force the threads to stay in synch, but again only really be effectively wasting cycles, atleast by adjusting the interpolation parameter youre putting spare cycles to good use by providing more smoothly interpolated animations etc.

Quote:
As far as significant slowdown, I guess I can just put a cap on how far the renderer will extrapolate beyond a missing update before it gives up and stalls.

Certainly, a little bit of extrapolation never hurt anyone [smile]. Naturally too much is a waste since the accuracy of this tends to fall away rapidly in a complex environment i.e. you mentioned physics.

Share this post


Link to post
Share on other sites
Quote:
Original post by Nairou
Quote:
Original post by Sneftel
I'm a big fan of this for RTS games, since (a) a fixed logic update speed is essential, (b) minor fudging of unit positions is fine, (c) a small amount of latency is fine, and (d) most graphical effects are framerate-independent. I'd be much more leery of the approach for something like an FPS.

Can you clarify? My game project is an FPS of sorts. The only issue I can think of that would specifically affect FPSs would by latency, but I don't see how that is a big issue here if updates are not buffered for more than one update.

Usually RTSs use this same kind of model for their networking code (i.e. the game update cycle runs at a fixed speed, independent of rendering etc...).

This approach simplifies networking, as you can just share simulation inputs and assume everyone's sim will stay in synch (e.g. have each player collect all of their inputs in a buffer for 200ms, and then share these buffers with other players at an agreed update-step).

This doesn't work so well for FPS networking, because it causes a bit of lag (each player has to receive each other players inputs before the sim can proceed).
However, the quake 3 networking model could be thought of as a client-server modification of this same networking technique (which is optimised for FPS). With the Q3 model though, the server will need to keep a fairly large group of buffers of the previous update results (instead of the just 2 or 3 old buffers that you have now).

In other words, by taking the time to structure your sim-update code properly, you're probably going to have an easier time implementing networking later ;)

Share this post


Link to post
Share on other sites
Sign in to follow this  

  • Advertisement
×

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

Participate in the game development conversation and more when you create an account on GameDev.net!

Sign me up!