The debug runtimes are debug DLLs that can be used instead of the normal D3D DLLs. They're turned on or off through a sperate utility rather than by linking to different libraries or anything.
To tun them on, go to the DirectX Control Panel, which is in Start Menu -> Programs -> DirectX SDK -> Utilities -> DirectX Control Panel, or if you're using an old SDK, it's in the Windows control panel.
You should see something like this:
You want to make sure that "Use Debug Version of Direct3D 9" is selected. If the option is greyed out, then you probably have an out of date SDK (So the runtimes are a differernt version to the SDK). Try downloading the latest DirectX SDK.
In debug builds (With the debug runtimes or not), you should also link to d3dx9d.lib instead of d3dx9.lib if you use D3DX. These libraries do more error checking than the standard version, so it's handy to always link to them in debug builds. My main D3D CPP file usually looks like:
# pragma comment(lib, "d3dx9d.lib")
# pragma comment(lib, "d3dx9.lib")
#pragma comment(lib, "d3d9.lib")
Under Managed Projects, you need to select "Enable unmanaged debugging" in the Debug section of the project properties. If you're running C# Express 2005, you cannot view the debug output using VS, and would need to use an external tool to catch debug output (like sysinternals' dbgview).
When the debug runtimes are on, you get useful information spewed out in the debug output window of your IDE. Visual Studio 2005 shows the following:
(Click to enlarge)
As you can see, you get the following in the debug output:
Quote:Which tells you the function that failed, the reason for failure, and the error code returned (Shown in the debug watch window in the screenshot).
Direct3D9: (ERROR) :Invalid iAdapter parameter. ValidateCreateDevice failed.
D3D9 Helper: IDirect3D9::CreateDevice failed: D3DERR_INVALIDCALL
That makes it far far easier to find problems than trudging through with the debugger, looking for the function you forgot to check the return value of.
There's a couple of other things the debug runtimes do. First, they'll do more parameter validation than the release runtimes, so some functions may work fine on the release runtimes, but fail on the debug runtimes. This is a problem with your code, not Direct3D. It just so happens that your drivers are being leniant. One example of this is DrawIndexedPrimitive(). On NVidia cards, the drivers used to ignore the NumVertices parameter (I'm not sure if they still do), which can be used as a performance optimisation (The card only needs to transform the vertices used in a call, not all of them). So if you enter a bogus number for this parameter, it'll work on the release runtimes if the driver doesn't mind, but it'll fail on the debug runtimes because they check the parameters. The bug is in your code (It wouldn't work on an ATI card for instance), the debug runtimes are just highlighting it.
Secondly, the debug runtimes will fill some resources with debug values. The most obvious one is the backbuffer if you use D3DSWAPEFFECT_DISCARD. When you use D3DSWAPEFFECT_DISCARD, you're telling D3D that after every Present(), the backbuffer can be thrown away, and you will not be relying on it containing anything useful, which allows the drivers to do some performance optimisations. A lot of drivers don't do anything special here and behave in a similar way to D3DSWAPEFFECT_FLIP, meaning that the backbuffer contains what it did on the last frame. However, since that's not guaranteed, D3D will fill the backbuffer with magenta or green (Alternating each frame). So if you try to use the backbuffer when it's not in a valid state, you'll get a green or magenta screen. You can see this if you try to render a scene without Clear()ing the backbuffer.
EDIT 16/Feb/2008: Removed gibberish about d3d9d.lib