Sign in to follow this  
Techieman

Going multi-threaded | Batches and Jobs

Recommended Posts

Hello folks!

 

Currently I am working on my engine again and I'd love to implement a threading mechanism, 

that can easily scale with the number of cores avaible. So an agent-based solution won't fit. 

 

Once, I implemented a Thread Pool for an earlier project, which used it mainly to

save and process files in the background. However, I wondered if a threadpool

would be appropriate for multi-threaded game engine and searched for some 

'additional informations' and I found a blog post by

Sean MiddleditchIn his post, he explain one should settle for Thread Pools,

with 'Jobs' and 'Batches'. 

After finishing reading it, I felt enlighted :), still I am having trouble to understand

the idea behind it. Basicly I don't understand what 'Batches' and 'Jobs' are.

 

TL;DR

What are 'Batches' and 'Jobs' when generally speaking about Thread Pooling?

What are the best way to identify a function call as a 'Batch' or either as a 'Job'?
How would one go about creating a multi-thread system?

 

Thanks in advance

P.S.: I hope I managed to express myself properly; Englisch ain't my native tongue.

Share this post


Link to post
Share on other sites

Basicly I don't understand what 'Batches' and 'Jobs' are.

 

What are 'Batches' and 'Jobs' when generally speaking about Thread Pooling?

What are the best way to identify a function call as a 'Batch' or either as a 'Job'?

 

Hello Julien,

 

Essentially when he means jobs, he simply is implying work that an engine sub-system has to perform. When he says batch, it's just a conceptualization in the context of the article of a group of individual objects deli-metered based upon some logical segmentation (A BSP Tree perhaps) being submitted to a ThreadPool Queue.

 

How would one go about creating a multi-thread system?

 

To my knowledge, C/C++ by itself has no notion of multi-threading. But, here goes a pretty simple example in C, from MSDN, also I'm pretty sure the threading code here is not portable. But, I think Boost does offer a portable solution...

 

 

// sample_multithread_c_program.c
// compile with: /c
//
//  Bounce - Creates a new thread each time the letter 'a' is typed.
//  Each thread bounces a happy face of a different color around
//  the screen. All threads are terminated when the letter 'Q' is
//  entered.
//

#include <windows.h>
#include <stdlib.h>
#include <string.h>
#include <stdio.h>
#include <conio.h>
#include <process.h>

#define MAX_THREADS  32

// The function getrandom returns a random number between
// min and max, which must be in integer range.
#define getrandom( min, max ) (SHORT)((rand() % (int)(((max) + 1) - \
                               (min))) + (min))

int main( void );                    // Thread 1: main
void KbdFunc( void  );               // Keyboard input, thread dispatch
void BounceProc( void * MyID );      // Threads 2 to n: display
void ClearScreen( void );            // Screen clear
void ShutDown( void );               // Program shutdown
void WriteTitle( int ThreadNum );    // Display title bar information

HANDLE  hConsoleOut;                 // Handle to the console
HANDLE  hRunMutex;                   // "Keep Running" mutex
HANDLE  hScreenMutex;                // "Screen update" mutex
int     ThreadNr;                    // Number of threads started
CONSOLE_SCREEN_BUFFER_INFO csbiInfo; // Console information


int main() // Thread One
{
    // Get display screen information & clear the screen.
    hConsoleOut = GetStdHandle( STD_OUTPUT_HANDLE );
    GetConsoleScreenBufferInfo( hConsoleOut, &csbiInfo );
    ClearScreen();
    WriteTitle( 0 );

    // Create the mutexes and reset thread count.
    hScreenMutex = CreateMutex( NULL, FALSE, NULL );  // Cleared
    hRunMutex = CreateMutex( NULL, TRUE, NULL );      // Set
    ThreadNr = 0;

    // Start waiting for keyboard input to dispatch threads or exit.
    KbdFunc();

    // All threads done. Clean up handles.
    CloseHandle( hScreenMutex );
    CloseHandle( hRunMutex );
    CloseHandle( hConsoleOut );
}

void ShutDown( void ) // Shut down threads
{
    while ( ThreadNr > 0 )
    {
        // Tell thread to die and record its death.
        ReleaseMutex( hRunMutex );
        ThreadNr--;  
    }

    // Clean up display when done
    WaitForSingleObject( hScreenMutex, INFINITE );
    ClearScreen();
}

void KbdFunc( void ) // Dispatch and count threads.
{
    int         KeyInfo;

    do
    {
        KeyInfo = _getch();
        if ( tolower( KeyInfo ) == 'a' &&
             ThreadNr < MAX_THREADS )
        {
            ThreadNr++;
            _beginthread( BounceProc, 0, &ThreadNr );
            WriteTitle( ThreadNr );
        }
    } while( tolower( KeyInfo ) != 'q' );

    ShutDown();
}

void BounceProc( void *pMyID )
{
    char    MyCell, OldCell;
    WORD    MyAttrib, OldAttrib;
    char    BlankCell = 0x20;
    COORD   Coords, Delta;
    COORD   Old = {0,0};
    DWORD   Dummy;
    char    *MyID = (char*)pMyID;

    // Generate update increments and initial
    // display coordinates.
    srand( (unsigned int) *MyID * 3 );

    Coords.X = getrandom( 0, csbiInfo.dwSize.X - 1 );
    Coords.Y = getrandom( 0, csbiInfo.dwSize.Y - 1 );
    Delta.X = getrandom( -3, 3 );
    Delta.Y = getrandom( -3, 3 );

    // Set up "happy face" & generate color
    // attribute from thread number.
    if( *MyID > 16)
        MyCell = 0x01;          // outline face
    else
        MyCell = 0x02;          // solid face
    MyAttrib =  *MyID & 0x0F;   // force black background

    do
    {
        // Wait for display to be available, then lock it.
        WaitForSingleObject( hScreenMutex, INFINITE );

        // If we still occupy the old screen position, blank it out.
        ReadConsoleOutputCharacter( hConsoleOut, &OldCell, 1,
                                    Old, &Dummy );
        ReadConsoleOutputAttribute( hConsoleOut, &OldAttrib, 1,
                                    Old, &Dummy );
        if (( OldCell == MyCell ) && (OldAttrib == MyAttrib))
            WriteConsoleOutputCharacter( hConsoleOut, &BlankCell, 1,
                                         Old, &Dummy );

        // Draw new face, then clear screen lock
        WriteConsoleOutputCharacter( hConsoleOut, &MyCell, 1,
                                     Coords, &Dummy );
        WriteConsoleOutputAttribute( hConsoleOut, &MyAttrib, 1,
                                     Coords, &Dummy );
        ReleaseMutex( hScreenMutex );

        // Increment the coordinates for next placement of the block.
        Old.X = Coords.X;
        Old.Y = Coords.Y;
        Coords.X += Delta.X;
        Coords.Y += Delta.Y;

        // If we are about to go off the screen, reverse direction
        if( Coords.X < 0 || Coords.X >= csbiInfo.dwSize.X )
        {
            Delta.X = -Delta.X;
            Beep( 400, 50 );
        }
        if( Coords.Y < 0 || Coords.Y > csbiInfo.dwSize.Y )
        {
            Delta.Y = -Delta.Y;
            Beep( 600, 50 );
        }
    }
    // Repeat while RunMutex is still taken.
    while ( WaitForSingleObject( hRunMutex, 75L ) == WAIT_TIMEOUT );
}

void WriteTitle( int ThreadNum )
{
    enum {
        sizeOfNThreadMsg = 80
    };
    char    NThreadMsg[sizeOfNThreadMsg];

    sprintf_s( NThreadMsg, sizeOfNThreadMsg,
               "Threads running: %02d.  Press 'A' "
               "to start a thread,'Q' to quit.", ThreadNum );
    SetConsoleTitle( NThreadMsg );
}

void ClearScreen( void )
{
    DWORD    dummy;
    COORD    Home = { 0, 0 };
    FillConsoleOutputCharacter( hConsoleOut, ' ',
                                csbiInfo.dwSize.X * csbiInfo.dwSize.Y,
                                Home, &dummy );
}
 

 

So this is pretty simple. While 'q' has not been pressed, and if 'a' has been pressed, and if ntThread is less than 32; increment ntThread, pass the BounceProc function as an argument to the __beginThread() function.

 

insofar as how to structure this in a typical Game Engine though the article gave some high level insight on it, a deeper explanation on it however would probably be a question best reserved for some other members here, as my only real experience with Multi-Threading was with Java several years ago.

 

Marcus Hansen

Edited by markypooch

Share this post


Link to post
Share on other sites

What are 'Batches' and 'Jobs' when generally speaking about Thread Pooling?

What are the best way to identify a function call as a 'Batch' or either as a 'Job'?
How would one go about creating a multi-thread system?

 

Skimming Sean's article, it looks to me to break down like this -- a batch is some portion of work as determined by a subsystem (e.g. a physics engine, rendering, resource-loaded) where different sub-systems have different needs. He uses the example of a physics system where it creates a batch for each "island" of physics objects (which is further explained to mean "nearby physics objects which potentially interact with one another, but not with other physics objects in that exist elsewhere"), because these "islands" are independent, you can actually create a batch out of each "island" and run them simultaneously on different processor cores since they don't interact with one another, giving potentially higher CPU utilization. And because the big-O cost of physics calculations usually square with the number of bodies in consideration, its also more efficient to have more smaller batches than fewer larger ones (e.g. 32+32+32 < 52+42 < 92). For other kinds of work, other batching strategies (or no particular strategy) might give best results. Rendering might create batches that use the same materials (textures+shaders+etc), a resource loader probably does one asset per "batch". A job seems to be an individual work submission -- a job is the result of batching.

 

As for creating the system itself, Sean posted a bunch of good links. The basic idea of a thread-pool, which seems to be the universally-preferred system today, is that you allocate a certain number of threads statically using OS mechanisms, based on the number of CPU cores (and hyperthreads) you find available. Your game logic puts jobs into a queue, and there's some mechanism that runs (could be a dedicated thread, could be a periodic or event-driven system) that moves work off the queue and onto on of those threads you allocated. There's a lot of detail I'm glossing over about what a job looks like in terms of an interface, but you can think of a job as some kind of class with access to all the operations and information that are needed to do the work, and some kind of "DoWork" method that kicks things off once it lands in one of those threads. You probably also need a way to return the results and signal when a job is done, this could be through a decoupled messaging system (like signals/slots) or a result queue -- you need to drop the result into some other delivery mechanism to free the thread as soon as possible.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this