• Advertisement
  • Popular Tags

  • Popular Now

  • Advertisement
  • Similar Content

    • By khawk
      LunarG has released new Vulkan SDKs for Windows, Linux, and macOS based on the 1.1.73 header. The new SDK includes:
      New extensions: VK_ANDROID_external_memory_android_hardware_buffer VK_EXT_descriptor_indexing VK_AMD_shader_core_properties VK_NV_shader_subgroup_partitioned Many bug fixes, increased validation coverage and accuracy improvements, and feature additions Developers can download the SDK from LunarXchange at https://vulkan.lunarg.com/sdk/home.

      View full story
    • By khawk
      LunarG has released new Vulkan SDKs for Windows, Linux, and macOS based on the 1.1.73 header. The new SDK includes:
      New extensions: VK_ANDROID_external_memory_android_hardware_buffer VK_EXT_descriptor_indexing VK_AMD_shader_core_properties VK_NV_shader_subgroup_partitioned Many bug fixes, increased validation coverage and accuracy improvements, and feature additions Developers can download the SDK from LunarXchange at https://vulkan.lunarg.com/sdk/home.
    • By mark_braga
      I have a pretty good experience with multi gpu programming in D3D12. Now looking at Vulkan, although there are a few similarities, I cannot wrap my head around a few things due to the extremely sparse documentation (typical Khronos...)
      In D3D12 -> You create a resource on GPU0 that is visible to GPU1 by setting the VisibleNodeMask to (00000011 where last two bits set means its visible to GPU0 and GPU1)
      In Vulkan - I can see there is the VkBindImageMemoryDeviceGroupInfoKHR struct which you add to the pNext chain of VkBindImageMemoryInfoKHR and then call vkBindImageMemory2KHR. You also set the device indices which I assume is the same as the VisibleNodeMask except instead of a mask it is an array of indices. Till now it's fine.
      Let's look at a typical SFR scenario:  Render left eye using GPU0 and right eye using GPU1
      You have two textures. pTextureLeft is exclusive to GPU0 and pTextureRight is created on GPU1 but is visible to GPU0 so it can be sampled from GPU0 when we want to draw it to the swapchain. This is in the D3D12 world. How do I map this in Vulkan? Do I just set the device indices for pTextureRight as { 0, 1 }
      Now comes the command buffer submission part that is even more confusing.
      There is the struct VkDeviceGroupCommandBufferBeginInfoKHR. It accepts a device mask which I understand is similar to creating a command list with a certain NodeMask in D3D12.
      So for GPU1 -> Since I am only rendering to the pTextureRight, I need to set the device mask as 2? (00000010)
      For GPU0 -> Since I only render to pTextureLeft and finally sample pTextureLeft and pTextureRight to render to the swap chain, I need to set the device mask as 1? (00000001)
      The same applies to VkDeviceGroupSubmitInfoKHR?
      Now the fun part is it does not work  . Both command buffers render to the textures correctly. I verified this by reading back the textures and storing as png. The left texture is sampled correctly in the final composite pass. But I get a black in the area where the right texture should appear. Is there something that I am missing in this? Here is a code snippet too
      void Init() { RenderTargetInfo info = {}; info.pDeviceIndices = { 0, 0 }; CreateRenderTarget(&info, &pTextureLeft); // Need to share this on both GPUs info.pDeviceIndices = { 0, 1 }; CreateRenderTarget(&info, &pTextureRight); } void DrawEye(CommandBuffer* pCmd, uint32_t eye) { // Do the draw // Begin with device mask depending on eye pCmd->Open((1 << eye)); // If eye is 0, we need to do some extra work to composite pTextureRight and pTextureLeft if (eye == 0) { DrawTexture(0, 0, width * 0.5, height, pTextureLeft); DrawTexture(width * 0.5, 0, width * 0.5, height, pTextureRight); } // Submit to the correct GPU pQueue->Submit(pCmd, (1 << eye)); } void Draw() { DrawEye(pRightCmd, 1); DrawEye(pLeftCmd, 0); }  
    • By turanszkij
      Hi,
      I finally managed to get the DX11 emulating Vulkan device working but everything is flipped vertically now because Vulkan has a different clipping space. What are the best practices out there to keep these implementation consistent? I tried using a vertically flipped viewport, and while it works on Nvidia 1050, the Vulkan debug layer is throwing error messages that this is not supported in the spec so it might not work on others. There is also the possibility to flip the clip scpace position Y coordinate before writing out with vertex shader, but that requires changing and recompiling every shader. I could also bake it into the camera projection matrices, though I want to avoid that because then I need to track down for the whole engine where I upload matrices... Any chance of an easy extension or something? If not, I will probably go with changing the vertex shaders.
    • By Alexa Savchenko
      I publishing for manufacturing our ray tracing engines and products on graphics API (C++, Vulkan API, GLSL460, SPIR-V): https://github.com/world8th/satellite-oem
      For end users I have no more products or test products. Also, have one simple gltf viewer example (only source code).
      In 2016 year had idea for replacement of screen space reflections, but in 2018 we resolved to finally re-profile project as "basis of render engine". In Q3 of 2017 year finally merged to Vulkan API. 
       
       
  • Advertisement
  • Advertisement
Sign in to follow this  

Vulkan How To Execute Pre-Recorded Command Buffers In Vulkan?

This topic is 602 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

Hello,

 

My question sounds simple: How to execute pre-recorded command buffers in Vulkan?

 

But we'll see that the answer is not as simple as: Use Vulkan's vkQueueSubmit for primary command buffers or vkCmdExecuteCommands for secondary command buffers.

 

I'm just starting with Vulkan and I'm creating a library for the user to issue drawing commands on a 2D canvas. The context here is that the user is able to create many command buffers. The key condition is that the user is able to execute the recorded command buffers in any order and there might be pipeline state changes between the execution of each of these command buffers.

 

So, lets start with the analysis:

 

As mentioned in the VK specs, all drawing commands must live within a render pass instance. But all the work in the render pass must be done in only one command buffer because we can not end the command buffer without ending the render pass first. So, this means that we can not have a command buffer to begin the render pass, put our pre-recorded command buffers in the middle, have another command buffer to end the render pass and finally send these 3 (or more) command buffers to the queue using vkQueueSubmit. The only alternative left is to use secondary command buffers.

Secondary command buffers allow to execute command buffers within a primary command buffer. This sounds convenient because we already have a primary command buffer, the one that holds the render pass and the secondary command buffers will be the ones recorded by the user. The logical thing to do then is:

 

  1. Begin the primary command buffer using vkBeginCommandBuffer.
  2. Put a memory barrier to set the framebuffer as render target (required for drawing) in the primary command buffer using vkCmdPipelineBarrier.
  3. Begin the render pass in the primary command buffer using vkCmdBeginRenderPass.
  4. Set up the initial pipeline state object in the primary command buffer using vkCreateGraphicsPipelines.
  5. Bind the initial pipeline state object in the primary command buffer using vkCmdBindPipeline.
  6. Call vkCmdExecuteCommands to execute the pre-recorded secondary command buffers inside the primary command buffer.
  7. End the render pass in the primary command buffer using vkCmdEndRenderPass.
  8. Put a memory barrier to set the framebuffer back to its original state (required for presenting) in the primary command buffer using vkCmdPipelineBarrier.
  9. End the primary command buffer using vkEndCommandBuffer.
  10. Submit the primary command buffer to the graphics queue using vkQueueSubmit.
  11. Present the frame using vkQueuePresentKHR.

Unfortunately this won't work because when you begin a render pass in a command buffer, all the other commands in the command buffer between the vkCmdBeginRenderPass and vkCmdEndRenderPass calls must be either inlined (VK_SUBPASS_CONTENTS_INLINE) which allows execution of any commands in the primary command buffer like vkCreateGraphicsPipelines and vkCmdBindPipeline for example, or must be grouped into secondary buffers (VK_SUBPASS_CONTENTS_SECONDARY_COMMAND_BUFFERS). The problem is that in the example above we have both: Inlined commands and secondary commands, this is out of spec and if commands are inlined (first flag) then secondary commands can not be executed but if we specify the secondary flag then no inlined commands can be executed (all inlined commands are discarded except for vkCmdExecuteCommands which is used for the execution of secondary command buffers).

 

How do I deal with this?

 

Remember that we can not move inlined commands to the secondary command buffers because these are pre-recorded by the user so when a pipeline state change occurs between two secondary command buffers we can not inject vkCmdBindPipeline at the beginning of the secondary buffer.

 

The condition about exclusion between inlined and secondary commands in a render pass is canceled when either a call to vkCmdEndRenderPass is made or a call to vkCmdNextSubpass is made. Taking attention to the second call gives me the idea to use subpasses. I've never used subpasses and do not know yet how they work but if I think about it, I'll have to use the first subpass in the render pass to set up the initial pipeline state then use another subpass to execute my secondary command buffers then if a pipeline state change occurs I'll have to use another subpass to update the pipeline state and then finally use another subpass to continue executing my secondary command buffers.

 

Is this the way it is supposed to execute pre-recorded command buffers in Vulkan?

 

All the examples I've come across record a single primary command buffer every frame and do not expose this situation. This is a trivial subject but Vulkan has made it all different with the introduction of render passes.

 

Thank you guys for your time. Any help to all the Vulkan beginners will be much appreciated by all of us.

Share this post


Link to post
Share on other sites
Advertisement
s this the way it is supposed to execute pre-recorded command buffers in Vulkan?

Yes, in general. However Your use case: 

 

 

key condition is that the user is able to execute the recorded command buffers in any order and there might be pipeline state changes between the execution of each of these command buffers

is not possible. In Vulkan command buffers do not inherit pipeline state, and each command buffer must bind the proper pipeline before drawing or dispatching. Your idea seems reasonable, but would be very cumbersome to implement - just imagine the amount of state in pipeline that should be 'patched' into the command buffer. 

 

Note:

 

 

I've never used subpasses and do not know yet how they work but if I think about it, I'll have to use the first subpass in the render pass to set up the initial pipeline state then use another subpass to execute my secondary command buffers then if a pipeline state change occurs I'll have to use another subpass to update the pipeline state and then finally use another subpass to continue executing my secondary command buffers.

This is invalid. vkCmdNextSubpass invalidates the current pipeline, and another pipeline with proper VkGraphicsPipelineCreateInfo.subpass value should be bound.

 

 

 

 

 

How do I deal with this?

I guess your intention is to run same command buffer multiple times with different shaders. Instead, as long as the GPR pressure permits, you can merge the shaders together and parametrize by an uniform buffer value - sth in the spirit of 'subroutines' in OpenGL.

Edited by scygan

Share this post


Link to post
Share on other sites

cygan, wow, thank you very, very, very much for your time and your wise answer. I was pretty explicit in my post and you were able to understand my situation. I'm kind of shocked because I thought more people were going to jump to this sinking boat (a trivial Vulkan topic that almost everyone must come across)...but even more shocked by the fact that I think I got to a dead end with Vulkan...and I was just starting.

 

This is invalid. vkCmdNextSubpass invalidates the current pipeline, and another pipeline with proper VkGraphicsPipelineCreateInfo.subpass value should be bound.

Actually I could try to create a new pipeline state object at every subpass with the corresponding subpass index...lets see if the validation layer complains. This is my last hope. I just wish Khronos, AMD and Nvidia could send their engineers to my home just like the do with big game studios ;)

 

I guess your intention is to run same command buffer multiple times with different shaders. Instead, as long as the GPR pressure permits, you can merge the shaders together and parametrize by an uniform buffer value - sth in the spirit of 'subroutines' in OpenGL.

Yep, different shaders and textures. I guess I must try this as well. But the problem is that in my library I'm using a "Graphics Driver Interface". It is an interface that all graphics APIs must follow. So far I've implemented D3D11 and D3D12 and part of OpenGL 4 succesfully. So, the user just draw using my library and the interface takes care of the internal stuff no matter the selected API. The problem is that Vulkan will not behave according to the general interface thus dragging down my gfx driver interface model. This is just a personal drawback but maybe if there is a Vulkan only design it could work by merging shaders and parametrizying.

 

In D3D12 this is nothing, I did this on day one, in fact it is designed to work in this flexible model. Vulkan at the contrary has to insist in the so called "predictability" introduced as a feature when in fact is a drawback to flexibility. I understand that Vulkan is an API that has to live for another 20 years like OpenGL and this must include support for tiler devices and even devices that hasn't been invented yet but support for some devices will exclude full optimization for another kind of devices. Some graphics chip manufacturers keep telling that a forward renderer could be optimized with render passes by letting the driver to guess what you're doing and then do it for you in the background, while in my personal opinion I prefer the driver to do exactly as I say and when I say it, that's basic optimization of resources per se...and I thought Vulkan' slogan was "full control of resources".

Enough rant guys hehe. I just have to cool down and take it easy. I might drop Vulkan support for my library but then again might get back in a few years to check if this is supported in forward renderers (past, current and modern).

In the meanwhile take care guys and keep developing games and your mind.

Edited by HateWork

Share this post


Link to post
Share on other sites

Well, after a couple of days I can confirm that there's no workaround for this, it simply can't be done in Vulkan.

 

I was trying to make Vulkan behave like D3D12, that's impossible. So I had to redesign my GFX interface and make D3D12 behave like Vulkan.

 

"Don't try to bend the spoon, that's impossible...instead try to bend yourself".

Share this post


Link to post
Share on other sites
Sign in to follow this  

  • Advertisement