Going multi-threaded | Batches and Jobs

Started by
4 comments, last by Ravyne 8 years ago

Hello folks!

Currently I am working on my engine again and I'd love to implement a threading mechanism,

that can easily scale with the number of cores avaible. So an agent-based solution won't fit.

Once, I implemented a Thread Pool for an earlier project, which used it mainly to

save and process files in the background. However, I wondered if a threadpool

would be appropriate for multi-threaded game engine and searched for some

'additional informations' and I found a blog post by

Sean Middleditch. In his post, he explain one should settle for Thread Pools,

with 'Jobs' and 'Batches'.

After finishing reading it, I felt enlighted :), still I am having trouble to understand

the idea behind it. Basicly I don't understand what 'Batches' and 'Jobs' are.

TL;DR

What are 'Batches' and 'Jobs' when generally speaking about Thread Pooling?

What are the best way to identify a function call as a 'Batch' or either as a 'Job'?
How would one go about creating a multi-thread system?

Thanks in advance

P.S.: I hope I managed to express myself properly; Englisch ain't my native tongue.

It's not a shame to make mistakes. As a programmer I am doing mistakes on a daily basis. - JonathanKlein

Advertisement

Basicly I don't understand what 'Batches' and 'Jobs' are.

What are 'Batches' and 'Jobs' when generally speaking about Thread Pooling?

What are the best way to identify a function call as a 'Batch' or either as a 'Job'?

Hello Julien,

Essentially when he means jobs, he simply is implying work that an engine sub-system has to perform. When he says batch, it's just a conceptualization in the context of the article of a group of individual objects deli-metered based upon some logical segmentation (A BSP Tree perhaps) being submitted to a ThreadPool Queue.

How would one go about creating a multi-thread system?

To my knowledge, C/C++ by itself has no notion of multi-threading. But, here goes a pretty simple example in C, from MSDN, also I'm pretty sure the threading code here is not portable. But, I think Boost does offer a portable solution...

// sample_multithread_c_program.c
// compile with: /c
//
// Bounce - Creates a new thread each time the letter 'a' is typed.
// Each thread bounces a happy face of a different color around
// the screen. All threads are terminated when the letter 'Q' is
// entered.
//

#include <windows.h>
#include <stdlib.h>
#include <string.h>
#include <stdio.h>
#include <conio.h>
#include <process.h>

#define MAX_THREADS 32

// The function getrandom returns a random number between
// min and max, which must be in integer range.
#define getrandom( min, max ) (SHORT)((rand() % (int)(((max) + 1) - \
(min))) + (min))

int main( void ); // Thread 1: main
void KbdFunc( void ); // Keyboard input, thread dispatch
void BounceProc( void * MyID ); // Threads 2 to n: display
void ClearScreen( void ); // Screen clear
void ShutDown( void ); // Program shutdown
void WriteTitle( int ThreadNum ); // Display title bar information

HANDLE hConsoleOut; // Handle to the console
HANDLE hRunMutex; // "Keep Running" mutex
HANDLE hScreenMutex; // "Screen update" mutex
int ThreadNr; // Number of threads started
CONSOLE_SCREEN_BUFFER_INFO csbiInfo; // Console information


int main() // Thread One
{
// Get display screen information & clear the screen.
hConsoleOut = GetStdHandle( STD_OUTPUT_HANDLE );
GetConsoleScreenBufferInfo( hConsoleOut, &csbiInfo );
ClearScreen();
WriteTitle( 0 );

// Create the mutexes and reset thread count.
hScreenMutex = CreateMutex( NULL, FALSE, NULL ); // Cleared
hRunMutex = CreateMutex( NULL, TRUE, NULL ); // Set
ThreadNr = 0;

// Start waiting for keyboard input to dispatch threads or exit.
KbdFunc();

// All threads done. Clean up handles.
CloseHandle( hScreenMutex );
CloseHandle( hRunMutex );
CloseHandle( hConsoleOut );
}

void ShutDown( void ) // Shut down threads
{
while ( ThreadNr > 0 )
{
// Tell thread to die and record its death.
ReleaseMutex( hRunMutex );
ThreadNr--;
}

// Clean up display when done
WaitForSingleObject( hScreenMutex, INFINITE );
ClearScreen();
}

void KbdFunc( void ) // Dispatch and count threads.
{
int KeyInfo;

do
{
KeyInfo = _getch();
if ( tolower( KeyInfo ) == 'a' &&
ThreadNr < MAX_THREADS )
{
ThreadNr++;
_beginthread( BounceProc, 0, &ThreadNr );
WriteTitle( ThreadNr );
}
} while( tolower( KeyInfo ) != 'q' );

ShutDown();
}

void BounceProc( void *pMyID )
{
char MyCell, OldCell;
WORD MyAttrib, OldAttrib;
char BlankCell = 0x20;
COORD Coords, Delta;
COORD Old = {0,0};
DWORD Dummy;
char *MyID = (char*)pMyID;

// Generate update increments and initial
// display coordinates.
srand( (unsigned int) *MyID * 3 );

Coords.X = getrandom( 0, csbiInfo.dwSize.X - 1 );
Coords.Y = getrandom( 0, csbiInfo.dwSize.Y - 1 );
Delta.X = getrandom( -3, 3 );
Delta.Y = getrandom( -3, 3 );

// Set up "happy face" & generate color
// attribute from thread number.
if( *MyID > 16)
MyCell = 0x01; // outline face
else
MyCell = 0x02; // solid face
MyAttrib = *MyID & 0x0F; // force black background

do
{
// Wait for display to be available, then lock it.
WaitForSingleObject( hScreenMutex, INFINITE );

// If we still occupy the old screen position, blank it out.
ReadConsoleOutputCharacter( hConsoleOut, &OldCell, 1,
Old, &Dummy );
ReadConsoleOutputAttribute( hConsoleOut, &OldAttrib, 1,
Old, &Dummy );
if (( OldCell == MyCell ) && (OldAttrib == MyAttrib))
WriteConsoleOutputCharacter( hConsoleOut, &BlankCell, 1,
Old, &Dummy );

// Draw new face, then clear screen lock
WriteConsoleOutputCharacter( hConsoleOut, &MyCell, 1,
Coords, &Dummy );
WriteConsoleOutputAttribute( hConsoleOut, &MyAttrib, 1,
Coords, &Dummy );
ReleaseMutex( hScreenMutex );

// Increment the coordinates for next placement of the block.
Old.X = Coords.X;
Old.Y = Coords.Y;
Coords.X += Delta.X;
Coords.Y += Delta.Y;

// If we are about to go off the screen, reverse direction
if( Coords.X < 0 || Coords.X >= csbiInfo.dwSize.X )
{
Delta.X = -Delta.X;
Beep( 400, 50 );
}
if( Coords.Y < 0 || Coords.Y > csbiInfo.dwSize.Y )
{
Delta.Y = -Delta.Y;
Beep( 600, 50 );
}
}
// Repeat while RunMutex is still taken.
while ( WaitForSingleObject( hRunMutex, 75L ) == WAIT_TIMEOUT );
}

void WriteTitle( int ThreadNum )
{
enum {
sizeOfNThreadMsg = 80
};
char NThreadMsg[sizeOfNThreadMsg];

sprintf_s( NThreadMsg, sizeOfNThreadMsg,
"Threads running: %02d. Press 'A' "
"to start a thread,'Q' to quit.", ThreadNum );
SetConsoleTitle( NThreadMsg );
}

void ClearScreen( void )
{
DWORD dummy;
COORD Home = { 0, 0 };
FillConsoleOutputCharacter( hConsoleOut, ' ',
csbiInfo.dwSize.X * csbiInfo.dwSize.Y,
Home, &dummy );
}

So this is pretty simple. While 'q' has not been pressed, and if 'a' has been pressed, and if ntThread is less than 32; increment ntThread, pass the BounceProc function as an argument to the __beginThread() function.

insofar as how to structure this in a typical Game Engine though the article gave some high level insight on it, a deeper explanation on it however would probably be a question best reserved for some other members here, as my only real experience with Multi-Threading was with Java several years ago.

Marcus Hansen


To my knowledge, C/C++ by itself has no notion of multi-threading.

As of C++11, C++ does have native thread support.

"I can't believe I'm defending logic to a turing machine." - Kent Woolworth [Other Space]

Good to know.

That's why I try to avoid speaking in absolutes. "To my knowledge,"

The specification is always updating...

Marcus Hansen

Note to self: use less jargon, write more explanatory blog posts. :)

Here's some links to probably better articles on the topic:

http://www.gamasutra.com/view/feature/132675/sponsored_feature_doityourself_.php?page=1
https://software.intel.com/en-us/articles/using-tasking-to-scale-game-engine-systems
https://blog.molecular-matters.com/2015/08/24/job-system-2-0-lock-free-work-stealing-part-1-basics/
http://www.gdcvault.com/play/1022186/Parallelizing-the-Naughty-Dog-Engine
http://bitsquid.blogspot.com/2010/03/task-management-practical-example.html

Sean Middleditch – Game Systems Engineer – Join my team!

What are 'Batches' and 'Jobs' when generally speaking about Thread Pooling?

What are the best way to identify a function call as a 'Batch' or either as a 'Job'?
How would one go about creating a multi-thread system?

Skimming Sean's article, it looks to me to break down like this -- a batch is some portion of work as determined by a subsystem (e.g. a physics engine, rendering, resource-loaded) where different sub-systems have different needs. He uses the example of a physics system where it creates a batch for each "island" of physics objects (which is further explained to mean "nearby physics objects which potentially interact with one another, but not with other physics objects in that exist elsewhere"), because these "islands" are independent, you can actually create a batch out of each "island" and run them simultaneously on different processor cores since they don't interact with one another, giving potentially higher CPU utilization. And because the big-O cost of physics calculations usually square with the number of bodies in consideration, its also more efficient to have more smaller batches than fewer larger ones (e.g. 32+32+32 < 52+42 < 92). For other kinds of work, other batching strategies (or no particular strategy) might give best results. Rendering might create batches that use the same materials (textures+shaders+etc), a resource loader probably does one asset per "batch". A job seems to be an individual work submission -- a job is the result of batching.

As for creating the system itself, Sean posted a bunch of good links. The basic idea of a thread-pool, which seems to be the universally-preferred system today, is that you allocate a certain number of threads statically using OS mechanisms, based on the number of CPU cores (and hyperthreads) you find available. Your game logic puts jobs into a queue, and there's some mechanism that runs (could be a dedicated thread, could be a periodic or event-driven system) that moves work off the queue and onto on of those threads you allocated. There's a lot of detail I'm glossing over about what a job looks like in terms of an interface, but you can think of a job as some kind of class with access to all the operations and information that are needed to do the work, and some kind of "DoWork" method that kicks things off once it lands in one of those threads. You probably also need a way to return the results and signal when a job is done, this could be through a decoupled messaging system (like signals/slots) or a result queue -- you need to drop the result into some other delivery mechanism to free the thread as soon as possible.

throw table_exception("(? ???)? ? ???");

This topic is closed to new replies.

Advertisement