I have a pretty good experience with multi gpu programming in D3D12. Now looking at Vulkan, although there are a few similarities, I cannot wrap my head around a few things due to the extremely sparse documentation (typical Khronos...)

In D3D12 -> You create a resource on GPU0 that is visible to GPU1 by setting the VisibleNodeMask to (00000011 where last two bits set means its visible to GPU0 and GPU1)

In Vulkan - I can see there is the VkBindImageMemoryDeviceGroupInfoKHR struct which you add to the pNext chain of VkBindImageMemoryInfoKHR and then call vkBindImageMemory2KHR. You also set the device indices which I assume is the same as the VisibleNodeMask except instead of a mask it is an array of indices. Till now it's fine.

Let's look at a typical SFR scenario:  Render left eye using GPU0 and right eye using GPU1

You have two textures. pTextureLeft is exclusive to GPU0 and pTextureRight is created on GPU1 but is visible to GPU0 so it can be sampled from GPU0 when we want to draw it to the swapchain. This is in the D3D12 world. How do I map this in Vulkan? Do I just set the device indices for pTextureRight as { 0, 1 }

Now comes the command buffer submission part that is even more confusing.

There is the struct VkDeviceGroupCommandBufferBeginInfoKHR. It accepts a device mask which I understand is similar to creating a command list with a certain NodeMask in D3D12.

So for GPU1 -> Since I am only rendering to the pTextureRight, I need to set the device mask as 2? (00000010)

For GPU0 -> Since I only render to pTextureLeft and finally sample pTextureLeft and pTextureRight to render to the swap chain, I need to set the device mask as 1? (00000001)

The same applies to VkDeviceGroupSubmitInfoKHR?

Now the fun part is it does not work  . Both command buffers render to the textures correctly. I verified this by reading back the textures and storing as png. The left texture is sampled correctly in the final composite pass. But I get a black in the area where the right texture should appear. Is there something that I am missing in this? Here is a code snippet too

void Init()
{
  RenderTargetInfo info = {};
  info.pDeviceIndices = { 0, 0 };
  CreateRenderTarget(&info, &pTextureLeft);
  
  // Need to share this on both GPUs
  info.pDeviceIndices = { 0, 1 };
  CreateRenderTarget(&info, &pTextureRight);
}

void DrawEye(CommandBuffer* pCmd, uint32_t eye)
{
  // Do the draw
  // Begin with device mask depending on eye
  pCmd->Open((1 << eye));
  
  // If eye is 0, we need to do some extra work to composite pTextureRight and pTextureLeft
  if (eye == 0)
  {
    DrawTexture(0, 0, width * 0.5, height, pTextureLeft);
    DrawTexture(width * 0.5, 0, width * 0.5, height, pTextureRight);
  }
  // Submit to the correct GPU
  pQueue->Submit(pCmd, (1 << eye));
}

void Draw()
{
  DrawEye(pRightCmd, 1);
  DrawEye(pLeftCmd, 0);
}