DX11 [DirectX11]How to capture the rendered scene D3D11?

Recommended Posts

I am writing code for capturing Skype video call for windows 10, I am facing an issue as following:

1). Skype use Directx11, and I am able to hook into the IDXGISwapChain::Present, and use code as below to capture the screenshot:

ID3D11Device *device = NULL;
	HRESULT hr = pswap->GetDevice(__uuidof(ID3D11Device), (void**)&device);
	if (hr != S_OK)
	{
		OutputDebugStringA("++++++++ GetDevice failed ++++++");
		return;
	}
	ID3D11DeviceContext *context;
	device->GetImmediateContext(&context);

	ID3D11Texture2D* backBuffer = nullptr;
	hr = pswap->GetBuffer(0, __uuidof(*backBuffer), (LPVOID*)&backBuffer);
	if (hr != S_OK)
	{
		OutputDebugStringA("++++++++ IDXGISwapChain::GetBuffer failed++++++");
		return;
	}

	//Save to a JPG file
    hr = SaveWICTextureToFile(context, backBuffer,
			GUID_ContainerFormatJpeg, strScreenPath);


	SafeRelease(device);
	SafeRelease(context);
	SafeRelease(backBuffer);

The problem is what I captured is either the local video frame(camera) or the remote video frame, what I want to capture is synthetic image, just like I see on the Skype window during a video call, I remember that in D3D9, there is a GetRenderTargetData() to retrieve the raw data from the back buffer, and it contains the rendered scene which is the same as I see on the Skype window during the video call, but I can't find a way to do this in D3D11....

Any advise is appreciated.

 

Share this post


Link to post
Share on other sites

Yes, actually I tried this as well, see the code below:

	ID3D11Texture2D* pbackBuffer = nullptr;
	hr = pswap->GetBuffer(0, __uuidof(*pbackBuffer), (LPVOID*)&pbackBuffer);
	if (hr != S_OK)
	{
		OutputDebugStringA("++++++++ IDXGISwapChain::GetBuffer failed++++++");
		SafeRelease(device);
		SafeRelease(pcontext);
		return hr;
	}

	ID3D11RenderTargetView *pRenderTargetView = NULL;
	// use the back buffer address to create the render target
	hr = device->CreateRenderTargetView(pbackBuffer, NULL, &pRenderTargetView);
	if (hr != S_OK)
	{
		OutputDebugStringA("++++++++ IDXGISwapChain::CreateRenderTargetView failed++++++");
		SafeRelease(device);
		SafeRelease(pcontext);
		return hr;
	}

	pcontext->OMSetRenderTargets(1, &pRenderTargetView, NULL);

	ID3D11Resource* pRenderTRes = NULL;
	pRenderTargetView->GetResource(&pRenderTRes);

    hr = SaveWICTextureToFile(pcontext, pRenderTRes,
			GUID_ContainerFormatJpeg, strScreenPath);

The image captured is still either the local camera or the remote camera, not the mixing finial image show in the call window..

 

Share this post


Link to post
Share on other sites

Oh, sorry, the "Skype" means the "Skype UWP"/"Skype For Windows 10" version instead of the desktop version.

The strange thing is the "OutputWindow" is NULL.

Edited by Jack Fu

Share this post


Link to post
Share on other sites

You're looking at a DComp app. Yes, there are swapchains involved, but no individual surface in the app contains the entire contents of the window. The reason the HWND is null is because it's a composition swapchain, not a standard HWND swapchain. See the IDXGIFactory2 CreateSwapChain methods.

The only way to capture the contents of the app is to capture the desktop and isolate the app, or to capture the individual pieces of the app and re-composite them.

Share this post


Link to post
Share on other sites

Hi SoldierOfLight, thanks for your reply.

Skype UWP doesn't use DComp libary since there is no according .dll found in the process.

Besides, it doesn't use  IDXGIFactory2 CreateSwapChain methods either, since I have hooked all the methods of IDXGIFactory2/IDXGISwapChainX, it just never called, only IDXGISwapChain is created.

Any ideas?

Share this post


Link to post
Share on other sites

I just attached a debugger to SkypeApp.exe and sure enough, DComp is loaded... I unfortunately don't have a webcam attached to be able to test out video, but I'd be fairly certain that they'll be using IDXGIFactory2::CreateSwapChainForComposition or IDXGIFactoryMedia::Create[Decode]SwapchainForCompositionSurfaceHandle.

Share this post


Link to post
Share on other sites

SkypeHost.exe is the process render the data, there is no DComp loaded, and there is no methods of IDXGISwapChainX called either...

BTW, which debugger do you use? I can't find any working debugger to debug the UWP app.

Share this post


Link to post
Share on other sites

Yes, you are right about that, I am now able to grab two instances of swap chain, one is for presenting local camera video, the other one is for presenting remote video.

The problem is I can't figure out a way to identify which swap chain is for local video, and which one is for remote video, any advise to identify this? 

P.S. the local video is always on the right corner of the skype window during the call.

Thanks in advance.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now


  • Forum Statistics

    • Total Topics
      628721
    • Total Posts
      2984394
  • Similar Content

    • By GreenGodDiary
      Having some issues with a geometry shader in a very basic DX app.
      We have an assignment where we are supposed to render a rotating textured quad, and in the geometry shader duplicate this quad and offset it by its normal. Very basic stuff essentially.
      My issue is that the duplicated quad, when rendered in front of the original quad, seems to fail the Z test and thus the original quad is rendered on top of it.
      Whats even weirder is that this only happens for one of the triangles in the duplicated quad, against one of the original quads triangles.

      Here's a video to show you what happens: Video (ignore the stretched textures)

      Here's my GS: (VS is simple passthrough shader and PS is just as basic)
      struct VS_OUT { float4 Pos : SV_POSITION; float2 UV : TEXCOORD; }; struct VS_IN { float4 Pos : POSITION; float2 UV : TEXCOORD; }; cbuffer cbPerObject : register(b0) { float4x4 WVP; }; [maxvertexcount(6)] void main( triangle VS_IN input[3], inout TriangleStream< VS_OUT > output ) { //Calculate normal float4 faceEdgeA = input[1].Pos - input[0].Pos; float4 faceEdgeB = input[2].Pos - input[0].Pos; float3 faceNormal = normalize(cross(faceEdgeA.xyz, faceEdgeB.xyz)); //Input triangle, transformed for (uint i = 0; i < 3; i++) { VS_OUT element; VS_IN vert = input[i]; element.Pos = mul(vert.Pos, WVP); element.UV = vert.UV; output.Append(element); } output.RestartStrip(); for (uint j = 0; j < 3; j++) { VS_OUT element; VS_IN vert = input[j]; element.Pos = mul(vert.Pos + float4(faceNormal, 0.0f), WVP); element.Pos.xyz; element.UV = vert.UV; output.Append(element); } }  
      I havent used geometry shaders much so im not 100% on what happens behind the scenes.
      Any tips appreciated! 
    • By mister345
      Hi, I'm building a game engine using DirectX11 in c++.
      I need a basic physics engine to handle collisions and motion, and no time to write my own.
      What is the easiest solution for this? Bullet and PhysX both seem too complicated and would still require writing my own wrapper classes, it seems. 
      I found this thing called PAL - physics abstraction layer that can support bullet, physx, etc, but it's so old and no info on how to download or install it.
      The simpler the better. Please let me know, thanks!
    • By Hexaa
      I try to draw lines with different thicknesses using the geometry shader approach from here:
      https://forum.libcinder.org/topic/smooth-thick-lines-using-geometry-shader
      It seems to work great on my development machine (some Intel HD). However, if I try it on my target (Nvidia NVS 300, yes it's old) I get different results. See the attached images. There
      seem to be gaps in my sine signal that the NVS 300 device creates, the intel does what I want and expect in the other picture.
      It's a shame, because I just can't figure out why. I expect it to be the same. I get no Error in the debug output, with enabled native debugging. I disabled culling with CullMode.None. Could it be some z-fighting? I have little clue about it but I tested to play around with the RasterizerStateDescription and DepthBias properties with no success, no change at all. Maybe I miss something there?
      I develop the application with SharpDX btw.
      Any clues or help is very welcome
       


    • By Beny Benz
      Hi,
      I'm currently trying to write a shader which shoud compute a fast fourier transform of some data, manipulating the transformed data, do an inverse FFT an then displaying the result as vertex offset and color. I use Unity3d and HLSL as shader language. One of the main problems is that the data should not be passed from CPU to GPU for every frame if possible. My original plan was to use a vertex shader and do the fft there, but I fail to find out how to store changing data betwen shader calls/passes. I found a technique called ping-ponging which seems to be based on writing and exchangeing render targets, but I couldn't find an example for HLSL as a vertex shader yet.
      I found https://social.msdn.microsoft.com/Forums/en-US/c79a3701-d028-41d9-ad74-a2b3b3958383/how-to-render-to-multiple-render-targets-in-hlsl?forum=xnaframework
      which seem to use COLOR0 and COLOR1 as such render targets.
      Is it even possible to do such calculations on the gpu only? (/in this shader stage?, because I need the result of the calculation to modify the vertex offsets there)
      I also saw the use of compute shaders in simmilar projects (ocean wave simulation), do they realy copy data between CPU / GPU for every frame?
      How does this ping-ponging / rendertarget switching technique work in HLSL?
      Have you seen an example of usage?
      Any answer would be helpfull.
      Thank you
      appswert
    • By ADDMX
      Hi
      Just a simple question about compute shaders (CS5, DX11).
      Do the atomic operations (InterlockedAdd in my case) should work without any issues on RWByteAddressBuffer and be globaly coherent ?
      I'v come back from CUDA world and commited fairly simple kernel that does some job, the pseudo-code is as follows:
      (both kernels use that same RWByteAddressBuffer)
      first kernel does some job and sets Result[0] = 0;
      (using Result.Store(0, 0))
      I'v checked with debugger, and indeed the value stored at dword 0 is 0
      now my second kernel
      RWByteAddressBuffer Result;  [numthreads(8, 8, 8)] void main() {     for (int i = 0; i < 5; i++)     {         uint4 v0 = DoSomeCalculations1();         uint4 v1 = DoSomeCalculations2();         uint4 v2 = DoSomeCalculations3();                  if (v0.w == 0 && v1.w == 0 && v2.w)             continue;         //    increment counter by 3, and get it previous value         // this should basically allocate space for 3 uint4 values in buffer         uint prev;         Result.InterlockedAdd(0, 3, prev);                  // this fills the buffer with 3 uint4 values (+1 is here as the first 16 bytes is occupied by DrawInstancedIndirect data)         Result.Store4((prev+0+1)*16, v0);         Result.Store4((prev+1+1)*16, v1);         Result.Store4((prev+2+1)*16, v2);     } } Now I invoke it with Dispatch(4,4,4)
      Now I use DrawInstancedIndirect to draw the buffer, but ocassionaly there is missed triangle here and there for a frame, as if the atomic counter does not work as expected
      do I need any additional synchronization there ?
      I'v tried 'AllMemoryBarrierWithGroupSync' at the end of kernel, but without effect.
      If I do not use atomic counter, and istead just output empty vertices (that will transform into degenerated triangles) the all is OK - as if I'm missing some form of synchronization, but I do not see such a thing in DX11.
      I'v tested on both old and new nvidia hardware (680M and 1080, the behaviour is that same).
       
  • Popular Now