Skip to main content
GameDev.net gamedev.net
🔒 Locked

VMR-9 Renderless Mode Crash

Started by Evil Steve May 11, 2009 at 3:22 PM 8 replies 5.2k views
Original Post
Evil Steve
Evil Steve
Hi all, I'm trying to get some D3D+DirectShow video texture stuff working, but I don't seem to be getting anywhere. I'm trying to set up VMR-9 renderless rendering, by following the MSDN docs and platform SDK sample. So far I have the following code:

bool VideoTexture::Load(LPDIRECT3DDEVICE9 pDevice, const std::wstring& strFilename)
{
	// Init COM
	HRESULT hResult = CoInitialize(NULL);
	if(FAILED(hResult))
	{
		m_strError = L"Failed to initialize COM. Error: " + Util::Win32ErrorToString(hResult);
		return false;
	}

	// Create the filter graph
	IGraphBuilder* pGraphBuilder;
	hResult = CoCreateInstance(CLSID_FilterGraph, NULL, CLSCTX_INPROC,
		__uuidof(IGraphBuilder), (void**)&pGraphBuilder);
	if(FAILED(hResult))
		return false;

	// Create the VMR-9 filter
	IBaseFilter* pVmr = NULL;
	hResult = CoCreateInstance(CLSID_VideoMixingRenderer9, 0, CLSCTX_INPROC_SERVER,
		IID_IBaseFilter, (void**)&pVmr);
	if(FAILED(hResult))
		return false;

	// Get filter config interface
	IVMRFilterConfig9* pConfig = NULL;
	hResult = pVmr->QueryInterface(IID_IVMRFilterConfig9, (void**)&pConfig);
	if(FAILED(hResult))
		return false;

	// Set renderless mode
	pConfig->SetNumberOfStreams(1);
	hResult = pConfig->SetRenderingMode(VMR9Mode_Renderless);
	if(FAILED(hResult))
		return false;

	IVMRSurfaceAllocatorNotify9* pAllocNotify = NULL;
	hResult = pVmr->QueryInterface(IID_IVMRSurfaceAllocatorNotify9, (void**)&pAllocNotify);
	if(FAILED(hResult))
		return false;

	// Create the Texture Renderer object
	TextureRenderer* pTextureRenderer = new TextureRenderer(pDevice);
	if(FAILED(hResult) || !pTextureRenderer)
		return false;

	// Let the allocator and the notify know about each other
	hResult = pAllocNotify->AdviseSurfaceAllocator(0xcafef00d, pTextureRenderer);
	if(FAILED(hResult))
		return false;
	hResult = pTextureRenderer->AdviseNotify(pAllocNotify);
	if(FAILED(hResult))
		return false;

	hResult = pGraphBuilder->AddFilter(pVmr, L"VMR9");
	if(FAILED(hResult))
		return false;

	IMediaControl* pControl = NULL;
	hResult = pGraphBuilder->QueryInterface(IID_IMediaControl, (void**)&pControl);
	if(FAILED(hResult))
		return false;

	hResult = pGraphBuilder->RenderFile(strFilename.c_str(), NULL);
	if(FAILED(hResult))
		return false;

	hResult = pControl->Run();
	if(FAILED(hResult))
		return false;

	return true;
}

I know there's no cleanup code or error handling, this is just to test the concept. My TextureRenderer class is derived from IVMRSurfaceAllocator9 and IVMRImagePresenter9, and its IVMRSurfaceAllocator9 functions are being called just fine. My implementation of IVMRSurfaceAllocator9::AdviseNotify is as follows:

HRESULT STDMETHODCALLTYPE TextureRenderer::AdviseNotify(
	IVMRSurfaceAllocatorNotify9* pIVMRSurfAllocNotify)
{
	m_pSurfAllocNotify = pIVMRSurfAllocNotify;
	if(m_pSurfAllocNotify)
	{
		// Get D3D interface
		IDirect3D9* pD3D = NULL;
		HRESULT hResult = m_pDevice->GetDirect3D(&pD3D);
		if(FAILED(hResult))
			return hResult;

		// Get monitor for primary display adapter
		HMONITOR hMonitor = pD3D->GetAdapterMonitor(D3DADAPTER_DEFAULT);
		pD3D->Release();
		return m_pSurfAllocNotify->SetD3DDevice(m_pDevice, hMonitor);
	}

	return S_OK;
}

If I comment out the call to SetD3DDevice, then it doesn't crash. I get the video rendered in a seperate window. With the call to SetD3DDevice, I get "First-chance exception at 0xbaadf00d in VideoTexture.exe: 0xC0000005: Access violation", which is coming from another thread (Presumably a DirectShow spawned one) with a rather uninteresting call stack: baadf00d() ntdll.dll!__RtlUserThreadStart@8() + 0x1b bytes I'm deliberately not Release()ing any of the interfaces I have, and I'm not deleting my TextureRenderer class. I've zipped up my project, which you can download Here (VC2008 project). Does anyone know what could be causing this? I'm pulling my hair out trying to track it down [sad]
adeyblue
adeyblue
HRESULT STDMETHODCALLTYPE TextureRenderer::QueryInterface(REFIID riid, void** ppvObject){    if(riid == IID_IVMRSurfaceAllocator9)    {        AddRef();        *ppvObject = this;    }    else if(riid == IID_IVMRImagePresenter9)    {        AddRef();        *ppvObject = this;    }    else if(riid == IID_IUnknown)    {        AddRef();        *ppvObject = this;    }    else        return E_NOINTERFACE;    return S_OK;}


This is implemented wrong. VMR queries for the IVMRImagePresenter9 interface and it receives a pointer to a IVMRSurfaceAllocator9 interface/vtable instead as that's the first interface the object derives from. It then calls the wrong methods with the wrong parameters which eventually blows up.

The IVMRImagePresenter9 part of the check requires a static_cast on the this pointer to return the correct interface/vtable. There's a few other things in the code (assuming a failed new returns null etc) but I guess most of it is just for testing purposes.
Evil Steve
Evil Steve
Quote:
Original post by adeyblue
*** Source Snippet Removed ***

This is implemented wrong. VMR queries for the IVMRImagePresenter9 interface and it receives a pointer to a IVMRSurfaceAllocator9 interface/vtable instead as that's the first interface the object derives from. It then calls the wrong methods with the wrong parameters which eventually blows up.

The IVMRImagePresenter9 part of the check requires a static_cast on the this pointer to return the correct interface/vtable. There's a few other things in the code (assuming a failed new returns null etc) but I guess most of it is just for testing purposes.
Ah ha, that would explain it thanks!

It's been ages since I did multiple inheritance stuff, I completely forgot about that. And yeah, the code there is really just for testing, I know there's lots of things wrong with it (And the checking if new returns null is from me spending too much time doing DS dev [smile])
chiyuwang
chiyuwang
Hi
I ran your code on my own machine, it fails at IVMRSurfaceAllocatorNotify9::SetD3DDevice, do you have any idea?

I also have my own project it fails at same place.
http://slash-directx.blogspot.com/http://liushuo.mybrute.com
Evil Steve
Evil Steve
Quote:
Original post by chiyuwang
Hi
I ran your code on my own machine, it fails at IVMRSurfaceAllocatorNotify9::SetD3DDevice, do you have any idea?

I also have my own project it fails at same place.
Are you using the debug D3D runtimes on Vista? I found that it fails on that call if I'm using the debug runtimes, because they don't expose a particular interface that the release runtimes do.

EDIT: Or do you mean it crashes? If so, adeyblue posted the fix.
chiyuwang
chiyuwang
Thank you for reply.

I put static_cast and ran in xp.


I followed your advice and tried to compile the project in release mode, and it still failed.

Any other idea? This problem drives me crazy.

I tried to run the project on 3 machines. It fails on 2 of them, and one of them runs ok.

I also tried to run MPC Homecinema, in renderless mode, and it doesnt have that problem at all.
http://slash-directx.blogspot.com/http://liushuo.mybrute.com
Evil Steve
Evil Steve
Quote:
Original post by chiyuwang
I followed your advice and tried to compile the project in release mode, and it still failed.
I meant the D3D release runtimes, not the project in release mode.

And what do you mean it "failed"?
chiyuwang
chiyuwang
Ah, OK

Do you mean I need to install directx end user runtime or I need to do extra setting somewhere?


Failed means the function SetD3DDevice return E_NOINTERFACE.
http://slash-directx.blogspot.com/http://liushuo.mybrute.com
Evil Steve
Evil Steve
Quote:
Original post by chiyuwang
Do you mean I need to install directx end user runtime or I need to do extra setting somewhere?
Linky.

Quote:
Original post by chiyuwang
Failed means the function SetD3DDevice return E_NOINTERFACE.
Yup, that sounds exactly like my debug D3D runtimes problem.
chiyuwang
chiyuwang
It is working very well.


Thank you soooooooooo much.
http://slash-directx.blogspot.com/http://liushuo.mybrute.com

Topic Locked

This topic has been locked by a moderator. New replies are not allowed.

Sign in to reply to this topic.