Hi everybody!

I have run across a very interesting issue that I have been trying to solve for days now: rendering a mesh to an off-screen surface does not work. Rendering to the back-buffer or rendering something simple like a cube works just fine, but when I call ID3DXMesh::DrawSubset it does not work after a SetRenderTarget call.

Let me give you a little bit of background: I am porting an existing XNA 3D engine to DirectX and C++. Since I wanted to make the port as easy as possible, I am writing a series of tutorials that build upon each other, with the final result being a very lightweight OOP framework on top of DXUT that offer all the common features like AA, lighting, shadows, post-processing and physics out of the box.

The first problem was making AA available on off-screen render targets. You can either create a (non-lockable) surface with AA options and no way to access it as a texture input for post-processing, or create a texture with no AA option. My solution is kind of brute force, but it works fine: create a surface and a depth buffer that share AA options, StretchDraw the surface to a texture, use the texture in a full screen quad.

This worked out just fine while I was rendering a cube. But rendering a mesh does not work. I did a lot of testing and found out that the depth buffer is updated, yet no pixels appear. I would suspect some issue with the bliting operations, but if I replace the mesh draw with a cube rendered with DrawPrimitive, it shows up just fine.

This issue is driving me crazy and I would appreciate any hints. I will continue to search for the solution. It is probably a stupid mistake somewhere that I keep overlooking or maybe my understanding of render target API is too incomplete.

I have attached the sources and Visual Studio Express solution files. If someone needs more information, please let me know! And sorry that I did not strip down the project to the bare minimum.

Thank you for reading this!

Another day of investigating, still no luck.

I did find two strange behaviors. When switching on the render target, if I zoom in very close and orient the model that parts of it would exit my screen, the entire screen is colored with the texel that touches the screen. It behaves as if the only a very thin margin from the depth buffer is updated.

I also did another test. This is how the model normally looks:

[attachment=16286:Untitled01.png]

When rendering the full screen buffer, I disable the Z buffer:

Enjin::Device9->SetRenderState(D3DRS_ZENABLE, 0);
Enjin::Device9->SetRenderState(D3DRS_ZWRITEENABLE, 0); 

If I comment these lines out, and press 'b' to activate the render target the previous image of the model is preserved. But If I slowly move the mouse, parts of that preserved image are replaced with the buffer clear color:

[attachment=16287:Untitled02.png]

Yet still, the fricking cube renders just fine! My plan of porting in 2-3 weeks is not going that well .

I re-uploaded the sample with model and binary form.

I still didn't manage to find any hint at least related to this issue. There is clearly something wrong with the depth buffer, that is all I know. To make testing easier for me, I took the "BasicHLSL" sample from the DirectX SDK (C++\Direct3D\BasicHLSL) and made minimal modifications to it in order to enable the same off screen rendering. And the results are the same.

I uploaded the modified BasicHLS.cpp and BasicHLSL.fx files.

Even if I don't manage to fix this issue and I'm forced to remain with the XNA implementation, I need to find out at least why this is not working.

In OnCreateDevice I create a vertex declaration and initialize the full screen quad buffer:

V_RETURN(pd3dDevice->CreateVertexDeclaration(g_vertexElements, &vertexDecl));

screenQuadBuffer.Create(pd3dDevice, 6);

Then in OnResetDevice I create a render surface, a depth buffer, a texture, obtain the surface of that texture and initialize the full screen quad with resolution dependent coordinates:

        IDirect3DSurface9* origTarget = NULL;
	pd3dDevice->GetRenderTarget(0, &origTarget);
	D3DSURFACE_DESC desc;
	origTarget->GetDesc(&desc);
	SAFE_RELEASE(origTarget);

	// create our surface as render target
	V_RETURN(pd3dDevice->CreateRenderTarget(desc.Width, desc.Height, desc.Format, 
									D3DMULTISAMPLE_NONE, 0,
									false, &renderTarget, NULL));
	V_RETURN(pd3dDevice->CreateDepthStencilSurface(desc.Width, desc.Height, D3DFMT_D24S8, 
									D3DMULTISAMPLE_NONE, 0,
									true, &stencil, NULL));

	V_RETURN(pd3dDevice->CreateTexture(desc.Width,
                                 desc.Height,
                                 1,
                                 D3DUSAGE_RENDERTARGET,
                                 desc.Format,
                                 D3DPOOL_DEFAULT,
                                 &pRenderTexture,
                                 NULL));
	V_RETURN(pRenderTexture->GetSurfaceLevel(0, &pRenderSurface));

	screenQuadBuffer.FillScreenQuad(pBackBufferSurfaceDesc->Width, pBackBufferSurfaceDesc->Height);

Finally, in OnFrameRender first I get the original render target, then I set the new target and depth buffer and enable Z writes:

        IDirect3DSurface9* origTarget = NULL;
	pd3dDevice->GetRenderTarget(0, &origTarget);
	
	pd3dDevice->SetDepthStencilSurface(stencil);
	
	pd3dDevice->SetRenderTarget(0, renderTarget);

	V(pd3dDevice->Clear(0, NULL, D3DCLEAR_TARGET | D3DCLEAR_ZBUFFER, D3DCOLOR_ARGB(0, 45, 50, 170), 1.0f, 0));		

	pd3dDevice->SetRenderState(D3DRS_ZENABLE, 1);
	pd3dDevice->SetRenderState(D3DRS_ZWRITEENABLE, 1);

Then I StretchRect the render target of the offscreen surface to the surface descriptor of the texture and set back the original render target:

pd3dDevice->StretchRect(renderTarget, NULL, pRenderSurface, NULL, D3DTEXF_NONE);
pd3dDevice->SetRenderTarget(0, origTarget);

Finally, I render the texture to screen using the full screen quad and a shader:

       if(SUCCEEDED(pd3dDevice->BeginScene())) {
		
		/*pd3dDevice->SetRenderState(D3DRS_ZENABLE, 0);
		pd3dDevice->SetRenderState(D3DRS_ZWRITEENABLE, 0);*/

		pd3dDevice->SetSamplerState(0, D3DSAMP_MIPFILTER, D3DTEXF_NONE);
		pd3dDevice->SetSamplerState(0, D3DSAMP_MINFILTER, D3DTEXF_POINT);
		pd3dDevice->SetSamplerState(0, D3DSAMP_MAGFILTER, D3DTEXF_POINT);

		UINT totalPasses;
		D3DXHANDLE hTechnique;
		hTechnique = g_pEffect->GetTechniqueByName("RenderScene2");
		pd3dDevice->SetStreamSource(0, screenQuadBuffer.GetHandle(), 0, screenQuadBuffer.GetVertexSize());
		pd3dDevice->SetIndices(NULL);
		V(g_pEffect->SetTexture("g_MeshTexture", pRenderTexture));
		if (SUCCEEDED(g_pEffect->SetTechnique(hTechnique))) {
			if (SUCCEEDED(g_pEffect->Begin(&totalPasses, 0))) {
				for (UINT pass = 0; pass < totalPasses; ++pass) {
					if (SUCCEEDED(g_pEffect->BeginPass(pass)))	{
						pd3dDevice->DrawPrimitive(D3DPT_TRIANGLESTRIP, 0, 2);
						g_pEffect->EndPass();
					}
				}

				g_pEffect->End();
			}
		}

		pd3dDevice->SetStreamSource(0, 0, 0, 0);
		pd3dDevice->SetIndices(NULL);
	
		g_HUD.OnRender( fElapsedTime );
        g_SampleUI.OnRender( fElapsedTime );

        RenderText( fTime );

		V(pd3dDevice->EndScene());

	}

Normally, the two commented lines should be uncommented. If the are uncommented, just an empty blue background is rendered. If I comment the lines, I get that strange effect from the previous post, where I get an across the frames persistent outline of what I have rendered that only shrinks, never grows.

I am using a CModelViewerCamera from DXUT. While rendering normally, not to the offscreen surface, the camera behaves as expected, sio I don't think the camera is to blame.

In OnResetDevice:

float fAspectRatio = pBackBufferSurfaceDesc->Width / ( FLOAT )pBackBufferSurfaceDesc->Height;
g_Camera.SetProjParams( D3DX_PI / 4, fAspectRatio, 2.0f, 4000.0f );
g_Camera.SetWindow( pBackBufferSurfaceDesc->Width, pBackBufferSurfaceDesc->Height );

I was just checking your znear wasn't zero - a common gotcha.

I'm a noob, but not quite so .

Like in my full sample, I tried to render only a cube created with raw data. The results are interesting and it behaves the same way: rendering works fine. But rendering the mesh does not. What on Earth is wrong and why does calling DrawPrimitive make such a difference. Just uncommenting the DrawSubset line that should draw the mesh makes the cube and the arrows not render. And even when things get partially rendered, the entire depth buffer is wrong.

I added another version of the code with the cube.

I decided to try a new approach: I replaced ID3DXMesh with CDXUTSDKMesh, a full source implementation of a mesh object from DXUT. I am yet to discover how to obtain the correct vertex declaration from that CDXUTSDKMesh, so my rendering is messed up. The normals are not correctly interpreted. Still, the mesh and the arrows render, as seen in the attached image.

So I think that indeed DrawSubset is to blame for the issues I am experiencing.

Now the problem is finding a good replacement for ID3DXMesh.

Does anybody know of a good C++ library that loads meshes and can handle animations too? Preferably something lightweight that doesn't pull in a ton of dependencies. CDXUTSDKMesh uses the .sdkmesh format, which seems to be a custom format made for the demos, so not a widely supported format.

Lol, found the issue and it was very stupid. Problems with the FVF/vertex declaration for the full screen quad caused by a hidden call in DrawSubset. I'll fully investigate the issue and post why it failed in more detail tomorrow.