[D3D12] Command Queue Fence Synchronization

Graphics and GPU Programming Programming

Started by hiya83 December 08, 2015 03:20 AM

5 comments, last by MikeJM 8 years, 3 months ago

hiya83

130

Author

December 08, 2015 03:20 AM

Hi,

I am going through the d3d12 samples from microsoft; the ones at https://github.com/Microsoft/DirectX-Graphics-Samples.git, and have a question about the synchronization they are doing for their command queues. Basically in all their samples inside ::WaitForPreviousFrame(), they are doing the following:

const UINT64 fence = m_fenceValue;

ThrowIfFailed(m_commandQueue->Signal(m_fence.Get(), fence));

m_fenceValue++;

// Wait until the previous frame is finished.

if (m_fence->GetCompletedValue() < fence)

{

ThrowIfFailed(m_fence->SetEventOnCompletion(fence, m_fenceEvent));

WaitForSingleObject(m_fenceEvent, INFINITE);

}

Isn't there a possible race condition here, where the signal command on the gpu pipe is run exactly after the CPU finishes checking the if (which succeeds) but before it sets the event on completion for the fence? wouldn't CPU just wait infinitely then? Am I missing something glaring here since all the samples seem to use this pattern?

Even if SetEventOnCompletion would fire the event if the fence value is already the trigger value, wouldn't there be a race now between that event firing and CPU running WaitForSingleObject instruction?

Thanks

Hodgman

52,717

December 08, 2015 06:00 AM

So: say 'fence' is 2, and 'm_fence->GetCompletedValue()' is 1, but is immediately updated to 2 right after entering the if-block.
I would assume that 'm_fence->SetEventOnCompletion(fence, m_fenceEvent))' would immediately signal m_fenceEvent, because m_fence now contains 2, and 'fence' is 2.
Because it's been signaled already, the WaitForSingleObject call would return immediately.
So if my assumption is right, then it's perfectly safe.

Even if SetEventOnCompletion would fire the event if the fence value is already the trigger value, wouldn't there be a race now between that event firing and CPU running WaitForSingleObject instruction?

Why would there be a race? Both those calls happen on a single CPU thread.

. 22 Racing Series .

MJP

20,295

December 08, 2015 06:16 AM

Conceptually, SetEventOnCompletion works like this:

HRESULT SetEventOnCompletion(UINT64 Value, HANDLE hEvent)
{
   // Start a background thread to check the fence value and trigger the event
   CreateThread(FenceThread, Value, hEvent);
}

void FenceThread(UINT64 Value, HANDLE hEvent)
{
    while(fenceValue < Value);  // Wait for the fence to be signaled
    SetEvent(hEvent);           // Trigger the event
}

So there's no danger of "missing" the event, as you're fearing.

EDIT: changed the code to be more clear about how the checking is done in the background

The Blog | The Book

hiya83

130

Author

December 11, 2015 07:02 PM

Sorry was distracted by other stuff and just got back to this...

So: say 'fence' is 2, and 'm_fence->GetCompletedValue()' is 1, but is immediately updated to 2 right after entering the if-block.
I would assume that 'm_fence->SetEventOnCompletion(fence, m_fenceEvent))' would immediately signal m_fenceEvent, because m_fence now contains 2, and 'fence' is 2.

I don't think you can guarantee that m_fence is immediately updated to 2 right after entering the if-block since GetCompletedValue is a CPU call, but Signal is a command queued in the GPU cmd buffer?

Even if SetEventOnCompletion would fire the event if the fence value is already the trigger value, wouldn't there be a race now between that event firing and CPU running WaitForSingleObject instruction?

Why would there be a race? Both those calls happen on a single CPU thread.

Isn't m_fenceEvent set by the GPU driver/kernel thread but WaitForSingleObject being done by the CPU userland thread? I don't think they are the same CPU thread?

Conceptually, SetEventOnCompletion works like this:


HRESULT SetEventOnCompletion(UINT64 Value, HANDLE hEvent)
{
    while(fenceValue != Value);
    return SetEvent(hEvent);
}

I thought SetEventOnCompletion simply sets a callback function/event for when fence value equals the value, does it really wait for the fence value to equal the value before setting that function/event?

Thanks

Infinisearch

3,058

December 11, 2015 10:47 PM

Maybe there is a lock of some sort in the kernel that serializes the synchronization functionality of the kernel. That would allay any fears you have right?

-potential energy is easily made kinetic-

Hodgman

52,717

December 12, 2015 01:37 AM

I thought SetEventOnCompletion simply sets a callback function/event for when fence value equals the value, does it really wait for the fence value to equal the value before setting that function/event?

I don't think it waits. Pretend that MJP's code runs in a new background thread.

Isn't m_fenceEvent set by the GPU driver/kernel thread but WaitForSingleObject being done by the CPU userland thread? I don't think they are the same CPU thread?

SetEventOnCompletion and WaitForSingleObject both run on the same CPU thread.

The first call says, if/when this fence is signalled by the GPU, then pass the signal on to this event. The second call waits for the event (which means, it waits for the GPU to signal the fence).

I don't think you can guarantee that m_fence is immediately updated to 2 right after entering the if-block since GetCompletedValue is a CPU call, but Signal is a command queued in the GPU cmd buffer?

That wasn't a guarantee, that was a hypothetical situation where you would enter the if condition even though the GPU has already signalled the fence... which I thought is the situation you were concerned about. I was pointing out that even if this unlikely event does happen, everything is still safe.

. 22 Racing Series .

MikeJM

172

December 23, 2015 07:07 AM

I thought this was a race too, but from my testing I'm pretty sure the documentation is just not well specified.

Specifies an event that should be fired when the fence reaches a certain value.

from: https://msdn.microsoft.com/en-us/library/windows/desktop/dn899190(v=vs.85).aspx

The word reaches there actually means the fence value is greater than or equal to the "certain value".

And it seems the event is set immediately in the case where the fence value >= certain value before SetEventOnCompletion is called.

This is relatively easy to test out by modifying D3D12HelloTriangle.cpp.

Modify the CreateFence call to set the initial value to 100.

Then run some tests by hacking up WaitForPreviousFrame.


void D3D12HelloTriangle::WaitForPreviousFrame()
{
    ThrowIfFailed(m_fence->SetEventOnCompletion(75, m_fenceEvent));
    WaitForSingleObject(m_fenceEvent, INFINITE);
    /*** ALWAYS REACHES HERE ***/
    m_frameIndex = m_swapChain->GetCurrentBackBufferIndex();
}


void D3D12HelloTriangle::WaitForPreviousFrame()
{
    ThrowIfFailed(m_commandQueue->Signal(m_fence.Get(), 50));

    ThrowIfFailed(m_fence->SetEventOnCompletion(75, m_fenceEvent));
    WaitForSingleObject(m_fenceEvent, INFINITE);
    /*** NEVER REACHES HERE, although presumably could if you fill up the queue enough ***/
    m_frameIndex = m_swapChain->GetCurrentBackBufferIndex();
}


void D3D12HelloTriangle::WaitForPreviousFrame()
{
    ThrowIfFailed(m_commandQueue->Signal(m_fence.Get(), 75));

    ThrowIfFailed(m_fence->SetEventOnCompletion(50, m_fenceEvent));
    WaitForSingleObject(m_fenceEvent, INFINITE);
    /*** ALWAYS REACHES HERE ***/
    m_frameIndex = m_swapChain->GetCurrentBackBufferIndex();
}


void D3D12HelloTriangle::WaitForPreviousFrame()
{
    ThrowIfFailed(m_commandQueue->Signal(m_fence.Get(), 200));

    ThrowIfFailed(m_fence->SetEventOnCompletion(150, m_fenceEvent));
    WaitForSingleObject(m_fenceEvent, INFINITE);
    /*** ALWAYS REACHES HERE ***/
    m_frameIndex = m_swapChain->GetCurrentBackBufferIndex();
}


void D3D12HelloTriangle::WaitForPreviousFrame()
{
    ThrowIfFailed(m_commandQueue->Signal(m_fence.Get(), 99));

    ThrowIfFailed(m_fence->SetEventOnCompletion(100, m_fenceEvent));
    WaitForSingleObject(m_fenceEvent, INFINITE);
    /*** NEVER REACHES HERE, although presumably could if you fill up the queue enough ***/
    m_frameIndex = m_swapChain->GetCurrentBackBufferIndex();
}

Since the sample only ever increments the fence the greater than or equal behavior protects against a race.

[D3D12] Command Queue Fence Synchronization

This topic is closed to new replies.

Popular Topics

Recommended Tutorials

[D3D12] Command Queue Fence Synchronization

This topic is closed to new replies.

Popular Topics

Recommended Tutorials

Reticulating splines