D3D12 / Vulkan Synchronization Primitives

Started by
3 comments, last by Matias Goldberg 7 years, 12 months ago

Vulkan and DX12 have very similar API's, but the way they handle their synchronization primitives seem to differ in the fundamental design.

Now, both Vulkan and DirectX 12 have resource barriers, so I'm going to ignore those.

DirectX 12 uses fences with explicit values that are expected to monotonically increase. In the simplest case, you have the swap chain present barrier. I can see two ways to implement fencing in this case:

1) You create N fences. At the end of frame N you signal fence N and then wait on fence (N + 1) % SwapBufferCount.

2) You create 1 fence. At the end of each frame you increment the fence and save off the value. You then wait for the fence to reach the value for frame (N + 1) % SwapBufferCount.

In general, it seems like the "timestamp" approach to fencing is powerful. For instance, I can have a page allocator that retires pages with a fence value and then wait for the fence to reach that point before recycling the page. It seems like creating one fence per command list submission would be expensive (maybe not? how lightweight are fences?).

Now compare this with Vulkan.

Vulkan has the notion of fences, semaphores, and events. They are explained in detail here. All these primitives are binary, it is signaled once and stay signaled until you reset it. I'm less familiar with how to use these kinds of primitives, because you can't do the timestamp approach like you can with DX12 fences.

For instance, to do the page allocator in Vulkan, the fence is the correct primitive to use because it involves synchronizing the state of the hardware queue with the host (i.e. to know when a retired page can be recycled).

In order to do this, I now have to create 1 fence for each vkSubmit call, and the page allocator receives a fence handle instead of a timestamp.

It seems to me like the DirectX-style fence is more flexible, as I would imagine that internally the Vulkan fence is using the same underlying primitive as the DirectX fence to track signaling. In short, it seems like the DirectX timestamp-based fencing allows you to use less fence objects overall.

My primary concern is thinking about a common backend between Vulkan and DX12. It seems like the wiser course of action is to support the Vulkan style binary fences because they can be implemented with DX12 fences. My concern is whether I will lose performance due to creating 1 fence per ExecuteCommandLists call vs 1 overall in DirectX.

For those who understand the underlying hardware and API's deeper than me, I would appreciate some insight into these design decisions.

Thanks!

Advertisement

In order to do this, I now have to create 1 fence for each vkSubmit call, and the page allocator receives a fence handle instead of a timestamp.
...
My concern is whether I will lose performance due to creating 1 fence per ExecuteCommandLists call vs 1 overall in DirectX.

Don't you have the option of using one fence per frame on both API's? (i.e. just fence the final vkSumbit for a frame)
We already do this on D3D9/11/GL/GNM/GCM/etc... so that the user can query whether a frame is retired yet or not, so that they can implement cross-platform ring-buffers, etc (which manage memory on a per-frame basis).

DirectX 12 uses fences with explicit values that are expected to monotonically increase

That's one use-case, not a strict expectation. You can use D3D12's fences to implement equivalents of Vulkan's fences, events, and semaphores (though Vulkan's events would require a D3D backend to finish it's current command list, submit it with a fence, reset it and continue recording commands).

Don't you have the option of using one fence per frame on both API's? (i.e. just fence the final vkSumbit for a frame)

I believe nVidia explicitly calls out that you should avoid batch submitting your entire frame at the very end. I believe the idea to keep the hardware busy with ~5 vkSubmit / ExecuteCommandList calls per frame?

That said, I guess there are several ways you can pipeline your frame.

I believe nVidia explicitly calls out that you should avoid batch submitting your entire frame at the very end. I believe the idea to keep the hardware busy with ~5 vkSubmit / ExecuteCommandList calls per frame?
That said, I guess there are several ways you can pipeline your frame.

Even still, you know which is the last vkSubmit call for the frame, so you can just fence that one only.

I fail to see how:


waitOnFence( fence[i] );

is any different from:


waitOnFence( fence, i );

Yes, the first one might require more "malloc" (I'm not speaking in the C malloc sense, but rather in "we'll need more memory somewhere") assuming the second version doesn't have hidden overhead.

However since you shouldn't have much more than ~10 fences (3 for triple buffer + 6 for overall synchronization across those 3 frames + 1 for streaming) memory usage becomes irrelevant. If you are calling "waitOnFence(...)" (which has a high overhead) more than 1-3 times per frame you're probably doing something wrong and it will likely begin to show up in GPUView (unless you have carefully calculated why you are fencing more than the norm and makes sense on what you're doing).

Btw you can emulate DX12's style in vulkan with (assuming you have a max limit of what the waiting value will be):


class MyFence
{
#if VULKAN
       vkFence m_fence[N];
#else
       D3D12Fence m_fence;
#endif
       MyFence( uint maxN );

       void wait( uint value );
}
due to creating 1 fence per ExecuteCommandLists

Ewww. Why would you do that?

Fence once per frame like Hodgman said. Only exceptions are sync'ing with compute & copy queues (but keep the waits() to a minimum).

This topic is closed to new replies.

Advertisement