Hello,
Background: I am currently migrating our game engine from Maya 2012 and OpenGL Viewport 1.0 renderer to Maya 2017 and Viewport 2.0 DirectX11. I am using ID3D11Device created by Maya so that our MPxGeometryOverrides used for selection and engine render loop (which is blitted to viewport by MUserRenderOperation) can share DX resources. Unfortunately it turns out that Maya creates the ID3D11Device with D3D11_CREATE_DEVICE_SINGLETHREADED flag, causing our asynchronous resource loading to crash:
ID3D11Device* device = (ID3D11Device*)MHWRender::MRenderer::theRenderer()->GPUDeviceHandle();
UINT flags = device->GetCreationFlags();
bool isDeviceMultithreaded = (flags & D3D11_CREATE_DEVICE_SINGLETHREADED) == 0;
assert( isDeviceMultithreaded ); // This fails
AFAIK there is no real reason for multithreaded device to be off (apart from minor single thread performance gains). I could rewrite our resource loading to use spinlocks on ID3D11Device creation methods, while deffering the actual calls to the time when I have the main thread (so that Maya itself won't interfere); but it seems like a hassle and I would like to avoid duplicating code or writing core engine's functionalities around Maya's quirks.
So does anyone know of any way to force D3D11_CREATE_DEVICE_SINGLETHREADED flag off? Can't really find anything in Maya's preferences and DirectX Control Panel doesn't provide this option. I have also tried to write a hook on D3D11CreateDeviceAndSwapChain using MinHook library, but I can't really figure out how to properly attach it with Maya's start so that I don't miss the device initialization.
Thanks in advance for all your help.
Update: I wrote a small exe that starts maya and injects dll with dllMain that injects my hooks (based on rederdoc code), but if I call CreateRemoteThread on Maya process, Maya shows empty "Command Window", halts, and immediately exits when I close this window. I am guessing that this maybe some kind of anti-piracy measure. Anyway, I am still stuck.