# [SlimDX] Device.VertexShader setter fails...

This topic is 3469 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

## Recommended Posts

Hi! I'm currently porting the 1.x branch of the Jad Engine (www.jadengine.com) from MDX to SLimDX and, first of all, let me say thank you for a really great wrapper, the code was compiling without errors in no time! But now I'm fighting with a strange bug for almost a week that I can't get rid of. When running any example that we have, the first frame cycle runs through without a problem. As soon as the first effect is processed in the second frame, when SetVertexShader of my EffectStateManager is invoked, setting the VertexShader fails with an access violation exception: As you can see, the Device and Function properties are not accessible in this context because they are throwing an access violation exception. I've checked the SlimDX method that invokes the SetVertexShader method with a breakpoint (IEffectStateManagerShim::SetVertexShader), there I can see that the InternalPointer property of the Device object that the vertex shader to be set is owning is giving a value that is different from the InternalPointer value of the device object that is currently "active" elsewhere. While browsing the SlimDX source (I'm currently working with the checkout from yesterday -> 2008/07/27), I've noticed that you have some vertex and pixel shader caching going on behind the scene, so as a test I've changed our effect builder system to recreate all effects every frame, and it works without any exceptions, but running slow as hell of course. I'm a bit clueless right now about how to go on, I did some test runs with MS PIX, but that didn't help either. I hope that some of you can maybe give me some hint on what I should check or how I should go on. Any help would be appreciated. Cheers Heiko

##### Share on other sites
Hmm. This could be a tricky one...can you provide us with a repro case? Smaller is better of course, but anything consistent will work. I really need to take a look at this for myself, I think. Also, what vertex and pixel shader caching are you talking about exactly? For the most part we avoid caching things because it's not safe, so I want to make sure we're not storing something we shouldn't be by mistake.

On a side note, it's been almost exactly a year since MDX was fully killed off and you guys decided to write up your own C++/CLI glue for handling DirectX. Are you looking at changing direction on that, or just generally messing around, or what? And what happened with Jad 2.0?

##### Share on other sites
Hi!

gonzo keeps doing work on the 1.X branch of Jade (the stable one) and he decided to move from MDX to SlimDX (great work Promit and the rest of the SlimDX crew) among a large quantity of other changes (fixes, new features,...), while the rest of the developers work on the 2.0 branch with our own C++/CLI wrapper.

But lately the 2.0 work has been pretty slow (nearly stopped) as we lack one developer with a good understanding of graphics programming/engine design with free time to keep building the engine. We have tried to search someone (not yet in gamedev) but we haven't had much luck.

Regards!

Vicente

##### Share on other sites
Quote:
 I'm currently porting the 1.x branch of the Jad Engine (www.jadengine.com) from MDX to SLimDX and, first of all, let me say thank you for a really great wrapper, the code was compiling without errors in no time!

That's cool. Way back when I first started contributing to SlimDX, one of the first things I tried to do was build Jad against it to see what we were missing. Had a ton of errors then.

##### Share on other sites
First of all, thanks for the fast replies!

Promit:
Regarding the "caching", I'm talking about the following code in direct3d9\effectstatemanager.cpp, in method IEffectStateManagerShim::SetVertexShader, at line 252:
m_WrappedInterface->SetVertexShader( VertexShader::FromPointer( pShader ) );

VertexShader::FromPointer looks like this (vertexshader.cpp, line 66):
VertexShader^ VertexShader::FromPointer( IDirect3DVertexShader9* pointer )	{		if( pointer == 0 )			return nullptr;		VertexShader^ tableEntry = safe_cast<VertexShader^>( ObjectTable::Find( static_cast<IntPtr>( pointer ) ) );		if( tableEntry != nullptr )		{			pointer->Release();			return tableEntry;		}		return gcnew VertexShader( pointer );	}

This getting the tableEntry from the ObjectTable seems to be some kind of caching mechanism (I haven't checked all the code that is involved, I must admit that my "C plus plus" reading ability got quite rusty over the years), or do I missunderstand this?

Thanks for your offer, but I'm afraid that doing a repro case will be almost impossible for me to do with all the EffectBuilder stuff going on in the background. But if you want to check it out, you could easily get the latest commit from http://www.assembla.com/wiki/show/jadengine and start the SimpleScene tutorial, it only creates a light, a camera and a textured box, should be simple enough.
You'll have to refresh the SlimDX references for the JadEngine and the Tutorials project so that they point to your local SlimDX build (to make stepping through while debugging possible) and you'll have to add the command line argument "-T:SimpleScene" (without the "") to the Tutorials project properties.

I'd be really thankful for any hint you could give me, lots of DX internals are still a mystery to me, but be warned, the code is still a mess! ;)

I'm doing this port mainly because of the reasons Vicente has already mentioned, plus I want to learn more about DX stuff so that I can be helping better with the 2.0 branch later.

jpetrie:
It only took me some 4 evenings to get from ~1200 errors down to 0, most of them were naming differences, I consider this quite fast. :)

Cheers

Heiko

##### Share on other sites
That "caching" is for maintaining sane reference counts to COM objects (all COM objects). SlimDX makes sure to give you the same managed reference for a COM object it already knows about, otherwise knowing when to correctly call Dispose() on a SlimDX object would be a nightmare.

More here and here on the background behind the decision. Promit's journal archives also discuss the problem.

##### Share on other sites
Is it intentional that in SetVertexShader you call

But in e.g. SerVertexShaderConstant you say

That is, without Adapter. Are those 'Device's the same instance?

Regards,
Andre

##### Share on other sites
Don't know what's going on there, but when I set Device.VertexShader I don't get any errors at all. It looks like your vertex shader function is set to NULL, which I don't think is right. It should have a ShaderByteCode object (at least mine does). So I'd guess something is resetting that?

##### Share on other sites
jpetrie: thanks for the links, I'll see if I can get a grasp of this with the additional information.

VizOne: Right now, Video.Device retuns _adapter.Device, so this shouldn't make a difference, I'll get rid of this soon so that there is only one property to get the device. Thanks for the hint though!

Tape_Worm: The strange thing is, if I'm setting a breakpoint at
m_WrappedInterface->SetVertexShader( VertexShader::FromPointer( pShader ) );

and stepping into VertexShader::FromPointer(), the tableEntry that is returned is still valid (the debugger is able to view the values of the Device and the Function properties) when the debugger is inside that function , but as soon as the code jumps to my SetVertexShader callback written in C#, both Properties cause access violation exceptions. I'm not sure if this could have to do with context switching or something like that...

I've just checked it again, when inside the VertexShader::FromPointer() function, as soon as pointer->Release() is executed (which happens the first time in the second frame) and if I then try to view tableEntry->Device with the debugger, the property causes an access violation exception, so it doesn't seem to be a problem of context switching... does one of you guys know what is going on there?

Cheers

Heiko

##### Share on other sites
So right now, trying to run SimpleScene, I get an E_FAIL on the first call to BeginPass that occurs in the second frame rendered (first frame apparently goes by fine). Haven't nailed down why that's happening yet.

##### Share on other sites
Yeah, that E_FAIL is returned by BeginPass because it's catching the exception that is thrown in Device::VertexShader::set (device.cpp, line 534). Thanks for looking into it! If you need any info or something else, please let me know.

Any clue why this pointer->Release() is causing problems?

Cheers

Heiko

##### Share on other sites
Ohh, I see. Wacky callbacks. Now let's see what's going on here...

##### Share on other sites
Okay, well I found the cause of that crash (bit of a misunderstanding between us and D3D), but there's another crash, also in EffectStateManager, that I don't understand yet. I'm still working on fixing that.

Here's the thing though -- you guys aren't using the state manager. Your implementation is just a stub. All you really have to do is comment out line 415 in JEffect.cs and Jad will run. In fact, I'm fairly sure this is the first time Jad's ever run on SlimDX

[EDIT] To be perfectly honest, this isn't going well. I have no idea why the EffectStateManager crashes the way it does, and even when I put in hacks to prevent the crash, it still doesn't render correctly.

[Edited by - Promit on July 29, 2008 4:04:14 PM]

##### Share on other sites
Oh my goodness! Promit, I'll build you a shrine!!! :D

The pic you've posted looks like something is wrong with the texture coordinates, could be that I did a mistake while porting it from pointer arithmetic to the DataStream thing, I'll check that again.

Thanks a ton for dealing with that so far!

##### Share on other sites
I saw a number of bugs or odd things just skimming through the code, you should definitely run through all the tutorials and make sure they run, and do it with D3D debugging turned on. For example, I noticed that you're creating your VBs wrong in some places. MDX took a vertex format and vertex count; SlimDX takes a byte count. That definitely failed at least one sample, which was trying to create a 4 byte VB. And in the shot above, it looks like it's using clamp for texture addressing mode instead of wrap.

##### Share on other sites
Hey there,

I finally found out what is causing this strange artifact in Promits screenshot. Indeed the reason why the texture looks strange is that its texture address sampler states are set to clamp.
But this is only from the second frame on, the texture address states are set to Wrap in the first frame actually. I discovered that the engine's state manager is saving the sampler states it has set in a cache and if the state to be set equals to the state that was set last, it will do nothing.
The question is why that worked with MDX, and why it doesn't with SlimDX. What is SlimDX doing differently? Do the states get reset after a call to Present or something like that?
I turned the state manager caching completely off, and it worked as expected (the texture was repeated in U and V direction), but then DirectX threw a ton of Ignoring already set sampler state warnings on me.
Is there a "right" way with SlimDX to know when I should set the state and when I can skip it?

Any help is very much appreciated.

##### Share on other sites
We aren't doing anything special with render states to cache them, or indeed even mess with them at all. If anything, it's MDX that was doing something under the hood. I'm afraid we're going to need more information to figure this one out.

As a side note, anything that happens in DirectX should happen in SlimDX as well. Therefore, if you want to test whether a certain issue is a bug in SlimDX, the most bullet-proof way is to implement a test case in DirectX using C++, and see if it exhibits the same problem.

##### Share on other sites
Well, I wouldn't entirely discount the object table caching from doing strange things in some circumstances. I'm not sure that's at all the case here, but I wouldn't bet my life on that code being bulletproof yet, either.

##### Share on other sites
I localized the problem to be an effect that sets the address modes to clamp, and then I found this in the engine code:
DXResource.Begin(FX.DoNotSaveState)

Ok, this was fixed easily and I just thought it slipped in there somehow, but what then hit me really hard was the fact that this line was already in the 1.1 release of Jade and that it worked nonetheless ... maybe MDX was doing some "I don't care what you want" state resetting behind the scenes... never mind.

Thanks for your time guys!

##### Share on other sites
I think maybe that the state manager you had was less clever than the D3DX one, so when combined with that flag, it worked okay. When you switched back to the D3DX default, it enabled the more complex logic which, combined with the DoNotSaveState flag, broke it.

Just a guess, though.

##### Share on other sites
Oh and here's what Promit's screeny should have looked like btw :P

##### Share on other sites
Well I've checked that, but the StateManager in the 1.1 release is doing the same thing as the current one, so no clue what was going 'wrong' back then... who cares, it works now. :)