Sign in to follow this  
jaafit

HLSL compiler crashed!

Recommended Posts

Hi, I'm an experienced programmer but a newb when it comes to shaders. This is the 2nd shader I've written and right from the beginning I ran into a compiler crash. I'd like someone who knows more about shaders to look at this and tell me what I'm doing wrong. What happens is, when I run my app, it get to call to D3DXCreateEffect and I get "Unhandled exception at 0x0118f8ef in RealFlightDeb.exe: 0xC0000005: Access violation reading location 0x00000060." Here is the call stack: RealFlightDeb.exe!D3DXShader::CPSLegacyProgram::AddPhaseLinkage() + 0x1e6 C++ RealFlightDeb.exe!D3DXShader::CPSLegacyProgram::PatternMatch() + 0x769 C++ RealFlightDeb.exe!D3DXShader::CShaderProgram::GenerateCode() + 0x529 C++ RealFlightDeb.exe!D3DXShader::CPSLegacyProgram::GenerateCode() + 0x55 C++ RealFlightDeb.exe!D3DXShader::CCompiler::CompileInternal() + 0x421 C++ RealFlightDeb.exe!D3DXShader::CCompiler::CompileFunction() + 0x36 C++ RealFlightDeb.exe!D3DXShader::CCompiler::CompileExpression() + 0x13a C++ RealFlightDeb.exe!D3DXEffects::Compiler::CEffectCompiler::CompileEffect() + 0x1e9 C++ RealFlightDeb.exe!_D3DXCreateEffect@36() + 0xd4 C++ RealFlightDeb.exe!DXEffect::CreateEffectLow() Line 833 + 0x38 C++ The good news is, I've narrowed it down to the exact HLSL line that's causing the crash. This function is called from my pixel shader. float4 Light_PointDiffuse(float4 LightDir, float3 Normal, float4 LightColor) { float AngleAttn = clamp(0, 1, dot(Normal, LightDir.xyz) ); return LightColor * LightDir.w * AngleAttn; // <<< THIS LINE } I can change the line to read return LightColor * LightDir.w; and the app runs fine. I need to be able to get the AngleAttn factor in there! btw I'm using vs_1_1, ps_1_4 if that helps. [Edited by - jaafit on March 3, 2006 7:04:08 PM]

Share this post


Link to post
Share on other sites
Which version of the DirectX SDK are you using?

Have you tried using fxc.exe to compile the shader outside of your program?

The shader profiles shouldn't make any difference as the compiler (if it functioned properly) would spit out well formed compile-time errors.

I'm not sure about the last couple of releases, but there have been plenty of HLSL compiler fixes as part of the bimonthly release schedule. It's rare to find this sort of error, but it's not unheard of [smile]

hth
Jack

Share this post


Link to post
Share on other sites
edit: here's another clue:

I edited a line before the faulty line above and once again got the compiler to not crash.

Old line:
Output.RGBColor = tex2D(BaseTextureSampler, inVert.TextureUV);

New line:
Output.RGBColor = float4(0,0,0,0);

But of course I need to sample that texture sooo, that's not going to work.

Quote:
Which version of the DirectX SDK are you using?

I have the Dec '05 DirectX SDK

Quote:
Have you tried using fxc.exe to compile the shader outside of your program?

fxc worked fine. Here is the output from fxc:

Microsoft (R) D3DX9 Shader Compiler 9.10.455.0000
Copyright (C) Microsoft Corporation 2002-2003. All rights reserved.

TerrainDecalAddSigned.fx(55): warning X4702: complement opportunity missed becau
se input result WAS clamped from 0 to 1

//
// Generated by Microsoft (R) D3DX9 Shader Compiler 9.10.455.0000
//
// fxc /Tps_1_4 /Eps_BasePlusDetailTimesDiffuse TerrainDecalAddSigned.fx
//
//
// Parameters:
//
// sampler2D BaseTextureSampler;
// sampler2D DetailTextureSampler;
// float4 sgv_light1Color;
//
//
// Registers:
//
// Name Reg Size
// -------------------- ----- ----
// sgv_light1Color c0 1
// DetailTextureSampler s0 1
// BaseTextureSampler s1 1
//

ps_1_4
def c1, 0, 0, 0, 1
texld r0, t1
mul r2.xyz, r0.w, r0_bias
phase
texcrd r0.xyz, t2
texld r1, t0
dp3 r0.w, r0, v0
add r3.w, -r0.w, c1.w
cmp r0.w, r3.w, r0.w, c1.w
mul r3, v0.w, c0
mul r0, r0.w, r3
add r1.xyz, r2_x2, r1
mul r0, r1, r0

// approximately 11 instruction slots used (3 texture, 8 arithmetic)

[Edited by - jaafit on March 3, 2006 5:48:44 PM]

Share this post


Link to post
Share on other sites
So, it compiles fine outside of your application (via fxc) but crashes when compiled inside. Correct?

Given your sampler line appears to be causing problems - what is the definition for your sampler?

What is the line of code (also the values of parameters) used to compile the shader?

Cheers,
Jack

Share this post


Link to post
Share on other sites
Quote:
Original post by jollyjeffers
So, it compiles fine outside of your application (via fxc) but crashes when compiled inside. Correct?

Given your sampler line appears to be causing problems - what is the definition for your sampler?

What is the line of code (also the values of parameters) used to compile the shader?

Cheers,
Jack


Yes, it compiles fine outside of the program, but crashes on D3DXCreateEffect inside the program.

The sampler is,

sampler BaseTextureSampler =
sampler_state
{
Texture = <sgv_baseTexture>;
MipFilter = LINEAR;
MinFilter = LINEAR;
MagFilter = LINEAR;
AddressU=Clamp;
AddressV=Clamp;
};

As I wrote in my first post, I can also fix the compiler error by reducing another needed line.



Here is the call to D3DXCreateEffect:


class DXEffectIncludeManager : public ID3DXInclude
{
public:
DXEffectIncludeManager( void ) { }
STDMETHOD(Open) ( D3DXINCLUDE_TYPE , LPCSTR , LPCVOID , LPCVOID *, UINT *);
STDMETHOD(Close)( LPCVOID pData );
};

DXEffectIncludeManager includeManager;
LPD3DXBUFFER pCompileErrorBuffer = NULL;
HRESULT hr;
hr = D3DXCreateEffect( gpDXDevice->m_pDeviceLow, pShaderFile-GetRawMemory(), pShaderFile->GetFileSize(), NULL, &includeManager, 0, NULL, &m_pEffectLow, &pCompileErrorBuffer );




At this point, my only solution is to use 2.0, but I don't think I'll be able to do that in the end due to compatibility issues. Does anyone know of a good workaround?

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this