Jump to content

  • Log In with Google      Sign In   
  • Create Account


#ActualKaptein

Posted 31 August 2013 - 12:56 PM

Some multi-threading tips

One of the biggest issues in multi threading is actually benefitting from it, so keep things REALLY simple. The reason being you want there to be very few places where your concurrency issues may be occuring.

So:

 

1. An example:

I create a class thats specifically for one purpose only, for one (1) and exactly one thread to work on. That means, if i have 4 worker threads, i'd need 4 objects.

The threads will work on those objects and once they're done i'll get some result which i can use in whichever way i like.

An example here is particles. On the one side I have the particle class, and on the other i have the renderable particles list.

I can send a job to a thread to output a mesh of renderable points, which on the rendering thread really only requires one thing: a boolean variable telling the rendering thread whether or not the particles were updated. If they were, i will upload them to the graphics API. Keeping it simple and all that.

Or even simpler, have the physics thread do the particle updates, and use the same method to "send" them to the rendering thread.

 

To accomplish this I'd use a mutex to prevent a small amount of variables from being written to while i try to read them. These variables can be for example:

struct renderable_particles_t

{

   particle_vertex_t* particles;

   int renderCount;

   bool updated;

};

 

that's all i need to be able to upload the particle data to the GPU, and so the solution is probably sound smile.png

Note that this "update" needs to happen in synch with the camera, so that the particles flow perfectly. The same goes for all other synchronizations that are to be rendered.

 

2. The things that must be threaded

Typically sound and networking. Both are so much simpler when threaded, such as blocking sockets and just letting the sound system do its thing in peace.

You will need to synchronize your interactions with the threads, but that is easy enough. For sounds you only really send information one way, since you want to play sounds or music.

For networking it may be a little bit more complicated. I don't know what people usually do, but i use queues for both read and write. That's 2 queues, but only one synchronization window. Once you synchronize send as much as you can, read as much as you can, then release the lock. What you've read is now in the read queue on your side which you can do whatever you want with. While the write queue is for the network system to work with.

 

3. There are a few instances where you don't have to lock, but make sure it's not something important

Imagine that your player can crouch, and that crouching is only written to on the "physics" thread (let's just call it that). Well, heres the trick:

Only set it ONCE, as in, do every test you have to to figure out if the player is crouching, but at the end, set it once. If you do that the rendering thread can happily read from the variable, and even if it just missed it, 60fps makes sure no one cares, or will ever notice that there was a 1 frame lag between the change

The reason we can do this with many such things, such as crouching, is that it's not something that changes often. It relies heavily on human reaction times

 

Note that this opens up a can of worms called cache incoherency, which could reduce your performance if you go overboard with it

It can help you wrap your head around threading - because if one thread is reading and the other reading and writing - everything is good

The problem is if you spread these variables that you are reading from other threads all around memory then the CPU will be spending time synchronizing cache between cores. You get around this issue by collecting the data you need and place it in a "tight" container, so that the memory is linear and close. Then you synchronize only once each "round". This is essentially what is being done in no#2, and it's almost always the best solution.

I'm not saying #3 is useless, and i have absolutely no clue how bad things get when you read from multiple locations each frame.. I'd better read up on it

 

4. Lockless, double-checking, degree of parallellism, keeping it real

Lockless: The absolute nightmare, but everyone needs to know about these..

Double-checking: The absolute don't-do-this of multi-threading

Degree of parallellism: Some things are easy to implement multi-threaded, some aren't.. Personally I think really hard before I throw threads at the problem

 

Finally, keeping it real:

If your game feels laggy or stuttering, it's very likely you aren't skipping work that you don't "have" to do NOW or immediately

There are many many ways to avoid doing work in games, and they all contribute to your game feeling smooth and awesome smile.png

Some examples:

1. Doing something every Nth frame

2. Doing something every Nth * depth frame (the further away it is, the less frequently we update it)

3. Simply checking if we are close to running out of time, typically 16.7ms (0.0167 seconds), and if we are, return immediately

4. Making sure that the graphics API is able to run through a frame without waiting for something to finish

 

The last one is probably surprising to many, but very important!

 

Here is a real example of avoiding work:

I have a giant list of particles, but I measured that only 1/3 of all the particles were in the camera at time of rendering. So i took the dot product between direction(player, particle) and camera look vector, and never added particles that weren't "in front of camera" to the list. A very cheap calculation that helps me avoid rendering 66% of the particles in the simulation.

 

Finally:

When it comes to multi-threading - if there's an easier way, the easier way is the better way

And don't take my examples as being good.. it's just where im at for the moment

#3Kaptein

Posted 31 August 2013 - 12:52 PM

Some multi-threading tips

One of the biggest issues in multi threading is actually benefitting from it, so keep things REALLY simple. The reason being you want there to be very few places where your concurrency issues may be occuring.

So:

 

1. An example:

I create a class thats specifically for one purpose only, for one (1) and exactly one thread to work on. That means, if i have 4 worker threads, i'd need 4 objects.

The threads will work on those objects and once they're done i'll get some result which i can use in whichever way i like.

An example here is particles. On the one side I have the particle class, and on the other i have the renderable particles list.

I can send a job to a thread to output a mesh of renderable points, which on the rendering thread really only requires one thing: a boolean variable telling the rendering thread whether or not the particles were updated. If they were, i will upload them to the graphics API. Keeping it simple and all that.

Or even simpler, have the physics thread do the particle updates, and use the same method to "send" them to the rendering thread.

 

To accomplish this I'd use a mutex to prevent a small amount of variables from being written to while i try to read them. These variables can be for example:

struct renderable_particles_t

{

   particle_vertex_t* particles;

   int renderCount;

   bool updated;

};

 

that's all i need to be able to upload the particle data to the GPU, and so the solution is probably sound smile.png

Note that this "update" needs to happen in synch with the camera, so that the particles flow perfectly. The same goes for all other synchronizations that are to be rendered.

 

2. The things that must be threaded

Typically sound and networking. Both are so much simpler when threaded, such as blocking sockets and just letting the sound system do its thing in peace.

You will need to synchronize your interactions with the threads, but that is easy enough. For sounds you only really send information one way, since you want to play sounds or music.

For networking it may be a little bit more complicated. I don't know what people usually do, but i use queues for both read and write. That's 2 queues, but only one synchronization window. Once you synchronize send as much as you can, read as much as you can, then release the lock. What you've read is now in the read queue on your side which you can do whatever you want with. While the write queue is for the network system to work with.

 

3. There are a few instances where you don't have to lock, but make sure it's not something important

Imagine that your player can crouch, and that crouching is only written to on the "physics" thread (let's just call it that). Well, heres the trick:

Only set it ONCE, as in, do every test you have to to figure out if the player is crouching, but at the end, set it once. If you do that the rendering thread can happily read from the variable, and even if it just missed it, 60fps makes sure no one cares, or will ever notice that there was a 1 frame lag between the change

The reason we can do this with many such things, such as crouching, is that it's not something that changes often. It relies heavily on human reaction times

 

Note that this opens up a can of worms called cache incoherency, which could reduce your performance if you go overboard with it

It can help you wrap your head around threading - because if one thread is reading and the other reading and writing - everything is good

The problem is if you spread these variables that you are reading from other threads all around memory then the CPU will be spending time synchronizing cache between cores. You get around this issue by collecting the data you need and place it in a "tight" container, so that the memory is linear and close. Then you synchronize only once each "round". This is essentially what is being done in no#2, and it's almost always the best solution.

I'm not saying #3 is useless, and i have absolutely no clue how bad things get when you read from multiple locations each frame.. I'd better read up on it

 

4. Lockless, double-checking, degree of parallellism, keeping it real

Lockless: The absolute nightmare, but everyone needs to know about these..

Double-checking: The absolute don't-do-this of multi-threading

Degree of parallellism: Some things are easy to implement multi-threaded, some aren't.. Personally I think really hard before I throw threads at the problem

 

Finally, keeping it real:

If your game feels laggy or stuttering, it's very likely you aren't skipping work that you don't "have" to do NOW or immediately

There are many many ways to avoid doing work in games, and they all contribute to your game feeling smooth and awesome smile.png

Some examples:

1. Doing something every Nth frame

2. Doing something every Nth * depth frame (the further away it is, the less frequently we update it)

3. Simply checking if we are close to running out of time, typically 16.7ms (0.0167 seconds), and if we are, return immediately

4. Making sure that the graphics API is able to run through a frame without waiting for something to finish

 

The last one is probably surprising to many, but very important!

 

Finally:

When it comes to multi-threading - if there's an easier way, the easier way is the better way

And don't take my examples as being good.. it's just where im at for the moment

#2Kaptein

Posted 31 August 2013 - 12:41 PM

Some multi-threading tips

One of the biggest issues in multi threading is actually benefitting from it, so keep things REALLY simple. The reason being you want there to be very few places where your concurrency issues may be occuring.

So:

 

1. An example:

I create a class thats specifically for one purpose only, for one (1) and exactly one thread to work on. That means, if i have 4 worker threads, i'd need 4 objects.

The threads will work on those objects and once they're done i'll get some result which i can use in whichever way i like.

An example here is particles. On the one side I have the particle class, and on the other i have the renderable particles list.

I can send a job to a thread to output a mesh of renderable points, which on the rendering thread really only requires one thing: a boolean variable telling the rendering thread whether or not the particles were updated. If they were, i will upload them to the graphics API. Keeping it simple and all that.

Or even simpler, have the physics thread do the particle updates, and use the same method to "send" them to the rendering thread.

 

To accomplish this I'd use a mutex to prevent a small amount of variables from being written to while i try to read them. These variables can be for example:

struct renderable_particles_t

{

   particle_vertex_t* particles;

   int renderCount;

   bool updated;

};

 

that's all i need to be able to upload the particle data to the GPU, and so the solution is probably sound smile.png

Note that this "update" needs to happen in synch with the camera, so that the particles flow perfectly. The same goes for all other synchronizations that are to be rendered.

 

2. The things that must be threaded

Typically sound and networking. Both are so much simpler when threaded, such as blocking sockets and just letting the sound system do its thing in peace.

You will need to synchronize your interactions with the threads, but that is easy enough. For sounds you only really send information one way, since you want to play sounds or music.

For networking it may be a little bit more complicated. I don't know what people usually do, but i use queues for both read and write. That's 2 queues, but only one synchronization window. Once you synchronize send as much as you can, read as much as you can, then release the lock. What you've read is now in the read queue on your side which you can do whatever you want with. While the write queue is for the network system to work with.

 

3. There are a few instances where you don't have to lock, but make sure it's not something important

Imagine that your player can crouch, and that crouching is only written to on the "physics" thread (let's just call it that). Well, heres the trick:

Only set it ONCE, as in, do every test you have to to figure out if the player is crouching, but at the end, set it once. If you do that the rendering thread can happily read from the variable, and even if it just missed it, 60fps makes sure no one cares, or will ever notice that there was a 1 frame lag between the change

The reason we can do this with many such things, such as crouching, is that it's not something that changes often. It relies heavily on human reaction times

 

4. Lockless, double-checking, degree of parallellism, keeping it real

Lockless: The absolute nightmare, but everyone needs to know about these..

Double-checking: The absolute don't-do-this of multi-threading

Degree of parallellism: Some things are easy to implement multi-threaded, some aren't.. Personally I think really hard before I throw threads at the problem

 

Finally, keeping it real:

If your game feels laggy or stuttering, it's very likely you aren't skipping work that you don't "have" to do NOW or immediately

There are many many ways to avoid doing work in games, and they all contribute to your game feeling smooth and awesome smile.png

Some examples:

1. Doing something every Nth frame

2. Doing something every Nth * depth frame (the further away it is, the less frequently we update it)

3. Simply checking if we are close to running out of time, typically 16.7ms (0.0167 seconds), and if we are, return immediately

4. Making sure that the graphics API is able to run through a frame without waiting for something to finish

 

The last one is probably surprising to many, but very important!

 

Finally:

When it comes to multi-threading - if there's an easier way, the easier way is the better way

And don't take my examples as being good.. it's just where im at for the moment

#1Kaptein

Posted 31 August 2013 - 12:39 PM

Here's two examples

In both examples i have threads that are just dormant in the background doing exactly nothing.

One of the biggest issues in multi threading is actually benefitting from it, so keep things REALLY simple. The reason being you want there to be very few places where your concurrency issues may be occuring.

So:

 

1. I create a class thats specifically for one purpose only, for one (1) and exactly one thread to work on. That means, if i have 4 worker threads, i'd need 4 objects.

The threads will work on those objects and once they're done i'll get some result which i can use in whichever way i like.

An example here is particles. On the one side I have the particle class, and on the other i have the renderable particles list.

I can send a job to a thread to output a mesh of renderable points, which on the rendering thread really only requires one thing: a boolean variable telling the rendering thread whether or not the particles were updated. If they were, i will upload them to the graphics API. Keeping it simple and all that.

 

To accomplish this I'd use a mutex to prevent a small amount of variables from being written to while i try to read them. These variables can be for example:

struct renderable_particles_t

{

   particle_vertex_t* particles;

   int renderCount;

   bool updated;

};

 

that's all i need to be able to upload the particle data to the GPU, and so the solution is probably sound :)

Note that this "update" needs to happen in synch with the camera, so that the particles flow perfectly. The same goes for all other synchronizations that are to be rendered.

 

2. The things that must be threaded

Typically sound and networking. Both are so much simpler when threaded, such as blocking sockets and just letting the sound system do its thing in peace.

You will need to synchronize your interactions with the threads, but that is easy enough. For sounds you only really send information one way, since you want to play sounds or music.

For networking it may be a little bit more complicated. I don't know what people usually do, but i use queues for both read and write. That's 2 queues, but only one synchronization window. Once you synchronize send as much as you can, read as much as you can, then release the lock. What you've read is now in the read queue on your side which you can do whatever you want with. While the write queue is for the network system to work with.

 

3. There are a few instances where you don't have to lock, but make sure it's not something important

Imagine that your player can crouch, and that crouching is only written to on the "physics" thread (let's just call it that). Well, heres the trick:

Only set it ONCE, as in, do every test you have to to figure out if the player is crouching, but at the end, set it once. If you do that the rendering thread can happily read from the variable, and even if it just missed it, 60fps makes sure no one cares, or will ever notice that there was a 1 frame lag between the change

The reason we can do this with many such things, such as crouching, is that it's not something that changes often. It relies heavily on human reaction times

 

4. Lockless, double-checking, degree of parallellism, keeping it real

Lockless: The absolute nightmare, but everyone needs to know about these..

Double-checking: The absolute don't-do-this of multi-threading

Degree of parallellism: Some things are easy to implement multi-threaded, some aren't.. Personally I think really hard before I throw threads at the problem

 

Finally, keeping it real:

If your game feels laggy or stuttering, it's very likely you aren't skipping work that you don't "have" to do NOW or immediately

There are many many ways to avoid doing work in games, and they all contribute to your game feeling smooth and awesome :)

Some examples:

1. Doing something every Nth frame

2. Doing something every Nth * depth frame (the further away it is, the less frequently we update it)

3. Simply checking if we are close to running out of time, typically 16.7ms (0.0167 seconds), and if we are, return immediately

4. Making sure that the graphics API is able to run through a frame without waiting for something to finish

 

The last one is probably surprising to many, but very important!

 

Finally:

When it comes to multi-threading - if there's an easier way, the easier way is the better way

And don't take my examples as being good.. it's just where im at for the moment

PARTNERS