Vulkan Secondary command buffers & staging buffers issues

Recommended Posts

So I've been trying to implement a multi-threaded resource system w/ vulkan in my free time, where a thread can request a resource to be loaded, and it gets pushed into a queue. On another thread, the resource (as of right now, a mesh) gets loaded from a file, and I map the data to a staging buffer. The issue comes in where I record the command buffer to copy the data to a GPU buffer. I record a secondary command buffer w/ just the vkCmdCopyBuffer command, and push it to a queue to be executed from a primary command buffer on the main thread to a transfer-only queue. As far as I can tell, the staging works fine, and the mesh is drawn and looks perfectly fine, but my validation layers (VK_LAYER_LUNARG_standard_validation) spam tell me: "vkCmdBindIndexBuffer(): Cannot read invalid region of memory allocation 0x16 for bound Buffer object 0x15, please fill the memory before using," and the vertex buffer binding gives me an identical message. Both buffers were created with the proper bits, TRANSFER_SRC for the staging buffer, TRANSFER_DST for the gpu buffer (plus index and vertex buffer usage bits). I use Vulkan Memory Allocator from GPUOpen to handle buffer memory allocation, and I'm careful to make sure that the staging buffer is mapped properly and isn't deleted before the command finishes. The validation layers stop spamming telling me this error if I switch the copy commands to using primary buffers, even when recorded in the same way (i.e. just changing the level parameter), but everything I've seen recommends recording secondary command buffers simultaneously on worker threads, and submitting them on the main thread later. Any ideas on why my validation layers are freaking out, or did I just skip over something when reading the spec?

Here's some relevant code:


Recording the secondary command buffer:

// on a worker thread

VkCommandBufferAllocateInfo allocInfo = {};
allocInfo.commandPool = transferCommandPool;
allocInfo.commandBufferCount = 1;

VkCommandBuffer commandBuffer;
vkAllocateCommandBuffers(device, &allocInfo, &commandBuffer); // Need to reuse, dont' alloc each time

// Iffy about this, what's the proper way to set this up?
VkCommandBufferInheritanceInfo inherInfo = {};
inherInfo.renderPass = VK_NULL_HANDLE;
inherInfo.framebuffer = VK_NULL_HANDLE;
inherInfo.occlusionQueryEnable = VK_FALSE;

VkCommandBufferBeginInfo beginInfo = {};
beginInfo.pInheritanceInfo = &inherInfo;

vkBeginCommandBuffer(commandBuffer, &beginInfo);

VkBufferCopy vbCopyRegion = {};
vbCopyRegion.srcOffset = 0;
vbCopyRegion.dstOffset = 0;
vbCopyRegion.size = meshData.size();
vkCmdCopyBuffer(commandBuffer, stagingBuffer, meshBuffer, 1, &vbCopyRegion);


// push it to a queue

Recording the primary command buffer:

// on the main thread

// transferCommandQueuePrimaryBuffer is the primary command buffer I submit eachf rame
vkResetCommandBuffer(transferCommandQueuePrimaryBuffer, 0);

VkCommandBufferBeginInfo beginInfo = {};

vkBeginCommandBuffer(transferCommandQueuePrimaryBuffer, &beginInfo);

// transferCommandQueue being a vector of secondary command buffers
vkCmdExecuteCommands(transferCommandQueuePrimaryBuffer, transferCommandQueue.size(),;


VkSubmitInfo submitInfo = {};
submitInfo.commandBufferCount = 1;
submitInfo.pCommandBuffers = &transferCommandQueuePrimaryBuffer;

vkQueueSubmit(transferQueue, 1, &submitInfo, transferCommandQueueSubmitFence);
vkQueueWaitIdle(transferQueue); // placeholder sync while I work on making a way to properly signal the resource has been loaded (w/ fences probably)



Share this post

Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

  • Forum Statistics

    • Total Topics
    • Total Posts
  • Similar Content

    • By hiya83
      Hey, I was wondering if on mobile development (Android mainly but iOS as well if you know of it), if there is a GPUView equivalent for whole system debugging so we can figure out if the CPU/GPU are being pipelined efficiently, if there are bubbles, etc. Also slightly tangent question, but do mobile GPU's have a DMA engine exposed as a dedicated Transfer Queue for Vulkan?
    • By mark_braga
      I am working on a project which needs to share render targets between Vulkan and DirectX12. I have enabled the external memory extension and now allocate the memory for the render targets by adding the VkExportMemoryInfoKHR to the pNext chain of VkMemoryAllocateInfo. Similarly I have added the VkExternalMemoryImageCreateInfo to the pNext chain of VkImageCreateInfo.
      After calling the get win32 handle function, I get some handle pointer which is not null (I assume it is valid).
      VkExternalMemoryImageCreateInfoKHR externalImageInfo = {}; if (gExternalMemoryExtensionKHR) { externalImageInfo.sType = VK_STRUCTURE_TYPE_EXTERNAL_MEMORY_IMAGE_CREATE_INFO_KHR; externalImageInfo.pNext = NULL; externalImageInfo.handleTypes = VK_EXTERNAL_MEMORY_HANDLE_TYPE_OPAQUE_FD_BIT_KHR | VK_EXTERNAL_MEMORY_HANDLE_TYPE_OPAQUE_WIN32_BIT_KHR | VK_EXTERNAL_MEMORY_HANDLE_TYPE_OPAQUE_WIN32_KMT_BIT_KHR | VK_EXTERNAL_MEMORY_HANDLE_TYPE_D3D11_TEXTURE_BIT_KHR | VK_EXTERNAL_MEMORY_HANDLE_TYPE_D3D11_TEXTURE_KMT_BIT_KHR | VK_EXTERNAL_MEMORY_HANDLE_TYPE_D3D12_HEAP_BIT_KHR | VK_EXTERNAL_MEMORY_HANDLE_TYPE_D3D12_RESOURCE_BIT_KH imageCreateInfo.pNext = &externalImageInfo; } vkCreateImage(...); VkExportMemoryAllocateInfoKHR exportInfo = { VK_STRUCTURE_TYPE_EXPORT_MEMORY_ALLOCATE_INFO_KHR }; exportInfo.handleTypes = VK_EXTERNAL_MEMORY_HANDLE_TYPE_OPAQUE_FD_BIT_KHR | VK_EXTERNAL_MEMORY_HANDLE_TYPE_OPAQUE_WIN32_BIT_KHR | VK_EXTERNAL_MEMORY_HANDLE_TYPE_OPAQUE_WIN32_KMT_BIT_KHR | VK_EXTERNAL_MEMORY_HANDLE_TYPE_D3D11_TEXTURE_BIT_KHR | VK_EXTERNAL_MEMORY_HANDLE_TYPE_D3D11_TEXTURE_KMT_BIT_KHR | VK_EXTERNAL_MEMORY_HANDLE_TYPE_D3D12_HEAP_BIT_KHR | VK_EXTERNAL_MEMORY_HANDLE_TYPE_D3D12_RESOURCE_BIT_KHR; memoryAllocateInfo.pNext = &exportInfo; vkAllocateMemory(...); VkMemoryGetWin32HandleInfoKHR info = { VK_STRUCTURE_TYPE_MEMORY_GET_WIN32_HANDLE_INFO_KHR, NULL }; info.memory = pTexture->GetMemory(); info.handleType = VK_EXTERNAL_MEMORY_HANDLE_TYPE_D3D12_RESOURCE_BIT_KHR; VkResult res = vkGetMemoryWin32HandleKHR(vulkanDevice, &info, &pTexture->pSharedHandle); ASSERT(VK_SUCCESS == res); Now when I try to call OpenSharedHandle from a D3D12 device, it crashes inside nvwgf2umx.dll with the integer division by zero error.
      I am now lost and have no idea what the other handle types do.
      For example: How do we get the D3D12 resource from the VK_EXTERNAL_MEMORY_HANDLE_TYPE_OPAQUE_WIN32_BIT_KHR handle?
      I also found some documentation on this link but it doesn't help much.
      This is all assuming the extension works as expected since it has made it to the KHR
    • By dwatt
      I am trying to get vulkan on android working and have run into a big issue. I don't see any validation layers available. I have tried linking their libraries into mine but still no layers. I have tried compiling it into a library of my own but the headers for it are all over the place. Unfortunately , google's examples and tutorials are out of date and don't work for me. Any idea what I have to do to get those layers to work?
    • By mark_braga
      It seems like nobody really knows what is the correct behavior after window minimizes in Vulkan.
      I have looked at most of the examples (Sascha Willems, GPUOpen,...) and all of them crash after the window minimize event with the error VK_ERROR_OUT_OF_DATE either with an assertion during acquire image or after calling present. This is because we have to recreate the swap chain.
      I tried this but then Vulkan expects you to provide a swap chain with extents { 0, 0, 0, 0 }, but now if you try to set the viewport or create new image views with extents { 0, 0, 0, 0 }, Vulkan expects you to provide non-zero values. So now I am confused.
      Should we just do nothing after a window minimize event? No rendering, update, ...?
    • By mellinoe
      Hi all,
      First time poster here, although I've been reading posts here for quite a while. This place has been invaluable for learning graphics programming -- thanks for a great resource!
      Right now, I'm working on a graphics abstraction layer for .NET which supports D3D11, Vulkan, and OpenGL at the moment. I have implemented most of my planned features already, and things are working well. Some remaining features that I am planning are Compute Shaders, and some flavor of read-write shader resources. At the moment, my shaders can just get simple read-only access to a uniform (or constant) buffer, a texture, or a sampler. Unfortunately, I'm having a tough time grasping the distinctions between all of the different kinds of read-write resources that are available. In D3D alone, there seem to be 5 or 6 different kinds of resources with similar but different characteristics. On top of that, I get the impression that some of them are more or less "obsoleted" by the newer kinds, and don't have much of a place in modern code. There seem to be a few pivots:
      The data source/destination (buffer or texture) Read-write or read-only Structured or unstructured (?) Ordered vs unordered (?) These are just my observations based on a lot of MSDN and OpenGL doc reading. For my library, I'm not interested in exposing every possibility to the user -- just trying to find a good "middle-ground" that can be represented cleanly across API's which is good enough for common scenarios.
      Can anyone give a sort of "overview" of the different options, and perhaps compare/contrast the concepts between Direct3D, OpenGL, and Vulkan? I'd also be very interested in hearing how other folks have abstracted these concepts in their libraries.
  • Popular Now