Directx 12 hull shader error

Started by
3 comments, last by Sujal 7 years, 11 months ago

I have an application that uses DX12 with feature level 12.0. Recently hull shaders hlsl code compilation is crashing (at function D3DCompileFromFile) when I access multiple constant buffers from hull shader. Is there a limitation on how many constant buffers hull shader can access? I did not find anything mentioned here https://msdn.microsoft.com/en-us/library/windows/desktop/mt186615(v=vs.85).aspx.

If I use more than one constant buffer, it throws following error -

Exception thrown at 0x00007FFE9550D6C8 in AppWin.exe: Microsoft C++ exception: long at memory location 0x0000003A498F6C0C.
Exception thrown at 0x00007FFE9550D6C8 in AppWin.exe: Microsoft C++ exception: long at memory location 0x0000003A498F6D40.
error X8000: D3D11 Internal Compiler Error: Invalid Bytecode: Reference out of bounds, [0], on cb0[size==14] (if size is listed as 0, it means default size is used, which is 4096. Opcode #42, operand #2 (counts are 1-based).
error X8000: D3D11 Internal Compiler Error: Invalid Bytecode: Can't continue validation - aborting.

The error message says "D3D11 Internal Compiler Error...", even though I am using DX12.

I get error when I use -


HS_CONSTANT_DATA_OUTPUT mainConstantsHS_High(InputPatch<VS_OUTPUT, 3> inputPatch, uint patchID : SV_PrimitiveID)
{
	HS_CONSTANT_DATA_OUTPUT output = (HS_CONSTANT_DATA_OUTPUT)0;
	output.edges[0] = buffer0Data + buffer1Data;
	output.edges[1] = buffer0Data;
	output.edges[2] = buffer0Data;
	output.inside = buffer0Data;
	return output;
}

But no error when I use just one buffer -


HS_CONSTANT_DATA_OUTPUT mainConstantsHS_High(InputPatch<VS_OUTPUT, 3> inputPatch, uint patchID : SV_PrimitiveID)
{
	HS_CONSTANT_DATA_OUTPUT output = (HS_CONSTANT_DATA_OUTPUT)0;
	output.edges[0] = buffer0Data;
	output.edges[1] = buffer0Data;
	output.edges[2] = buffer0Data;
	output.inside = buffer0Data;
	return output;
}

I have GTX 970 video card with Windows 10 14925 and VS 2015 update 2.

Thanks

Advertisement

Can you provide a minimally compilable shader that includes the definitions for your constant buffers and HS_CONSTANT_DATA_OUTPUT? Have you tried compiling the shader offline using fxc?

Also, I don't think Windows 10 14925 is a thing (yet).

Adam Miles - Principal Software Development Engineer - Microsoft Xbox Advanced Technology Group

Thanks Adam for replying my post. I had never tried FXC command line before. I tried to too. Same error.

Here is the stripped down version of the code.


//make all the matrices row major
#pragma pack_matrix( row_major )

//structure for light
struct Light
{
	float4x4 viewProjMatrix; //view projection matrix of the light
	float3 position;	//position of light
	float nearPlane;	//near plane of light
	float3 direction;	//direction of light
	float farPlane;		//far plane of light
	float3 color;		//color of light
	float ambientAmount;	//amount of ambient light
	float lightInfluenceRadius;	//light influence radius
	float lightInfluenceMaxCutoff;	//max influence radius of light (it can be lightInfluenceRadius) If smaller creates spotlight
	float cosAngleInnerCone;	//cos range(inner cone)
	float cosAngleOuterCone;	//cos angle end (outer cone)
};

typedef float4 floatPlane;

//Constant buffer per frame
cbuffer ConstantBufferPerFrame : register(b0, space1)
{
	Light mainLight;	//light parameter
	float3 eyePosition;	//camera position
	float dt;	//delta time since last frame
	float3 eyeLookAt;	//camera lookat point
	float elapsedTime;	//total elapsed
	float3 eyeUp;	//camera up direction
	float depthDelta;	//for computing shadow
	float3 eyeSide;	//camera side
	bool enableShadow;	//is shadow enabled
	float4x4 viewMatrix;	//camera view matrix
	float4x4 projMatrix;	//camera projection matrix
	float4x4 viewProjMatrix;	//camera view and projection matrix combined	
	float4x4 viewMatrixInverse;	//inverse of camera view matrix
	float4x4 projMatrixInverse;	//inverse of camera projection matrix
	float4x4 viewProjMatrixInverse;	//inverse of camera view and projection matrix combined
	floatPlane clipPlane0;	//for clipping for reflection
	float3 fogColor;	//color of fog
	float fogDensity;	//fog density
	float3 gravity;		//gravity direction
	float waterLevel;	//water level
	float3 windDirection;	//direction of the wind
	float wetnessFactor;	//how wet is the scene [0-1]
	float coldnessFactor;	//how cold is the scene [0-1]
	float nearPlane;	//near plane of camera
	float farPlane;		//far plane of camera
	float maxTessellationDistanceAdjustment; //maximum tessellation distance
};

//Constant buffer per object
cbuffer ConstantBufferPerObject : register(b0)
{
	float4x4 worldMatrix;
	float4x4 worldMatrixInverseTranspose;
	float4 dataDiffuseAlpha;	//diffuse and alpha
	float4 dataNormalHeight;	//normal and height data
	float4 dataSpecularLevel;	//specular color and specular level (controls intensity)
	float4 dataIlluminationGlossiness;	//illumination color and glossiness (controls size of specular spot)
	float3 scatterColor;	//color that is scattered by sub-surface scattering, 0 will disable it
	float maxScatterDepthWorldSpace;	//maximum depth light can penetrate
	float R0; //computation based on nvidia's article
	float etaRatio; //Ratio of indices of refraction indexOfRefractionFirst/indexOfRefractionSecond
	float maxTessellationFactor;	//geometry tessellation
	float maxTessellationBumpHeight;	//maximum tessellation bump height
};

// Per-vertex data used as input to the vertex shader.
struct VS_INPUT
{
	float3 position : POSITION;
};

struct VS_OUTPUT
{
	float3 worldPosition : WORLDPOS;
};

struct HS_CONSTANT_DATA_OUTPUT
{
	float edges[3]	: SV_TessFactor;
	float inside : SV_InsideTessFactor;
};

struct HS_CONTROL_POINT_OUTPUT
{
	float3 worldPosition : WORLDPOS;
};

struct DS_OUTPUT
{
	float3 worldPosition : WORLDPOS;
};

// Per-pixel color data passed through the pixel shader.
struct GS_OUTPUT
{
	float4 screenPositionSV : SV_POSITION;
};

struct PS_OUTPUT
{
	float4 color0 : SV_Target0;
};
//////////////////////////////////////////////////////////////////////

// Simple shader to do vertex processing on the GPU.
VS_OUTPUT mainVS_High(VS_INPUT input)
{
	VS_OUTPUT output = (VS_OUTPUT)0;

	float4 position = float4(input.position.xyz, 1.0);
	position = mul(worldMatrix, position);
	output.worldPosition = (position.xyz / position.w);
	
	return output;
}

// Hull shader constant data
HS_CONSTANT_DATA_OUTPUT mainConstantsHS_High(InputPatch<VS_OUTPUT, 3> inputPatch, uint patchID : SV_PrimitiveID)
{
	HS_CONSTANT_DATA_OUTPUT output = (HS_CONSTANT_DATA_OUTPUT)0;

	////// Assign tessellation levels
	//output.edges[0] = maxTessellationFactor;
	//output.edges[1] = maxTessellationFactor;
	//output.edges[2] = maxTessellationFactor;
	//output.inside = maxTessellationFactor;

	//// Assign tessellation levels
	output.edges[0] = maxTessellationFactor + maxTessellationDistanceAdjustment;
	output.edges[1] = maxTessellationFactor + maxTessellationDistanceAdjustment;
	output.edges[2] = maxTessellationFactor + maxTessellationDistanceAdjustment;
	output.inside = maxTessellationFactor + maxTessellationDistanceAdjustment;

	return output;
}

//hull shader control pooint
[domain("tri")]
[partitioning("fractional_odd")]
[outputtopology("triangle_cw")]	//goes 1,2,3 and so on. Triangles are supplied in CCW order already so no need to change
[outputcontrolpoints(3)]
[patchconstantfunc("mainConstantsHS_High")]
[maxtessfactor(64.0)]
HS_CONTROL_POINT_OUTPUT mainHS_High(InputPatch<VS_OUTPUT, 3> inputPatch, uint uCPID : SV_OutputControlPointID)
{
	HS_CONTROL_POINT_OUTPUT    output = (HS_CONTROL_POINT_OUTPUT)0;

	// Copy inputs to outputs
	output.worldPosition = inputPatch[uCPID].worldPosition.xyz;

	return output;
}

//domain shader
[domain("tri")]
DS_OUTPUT mainDS_High(HS_CONSTANT_DATA_OUTPUT input, float3 barycentricCoordinates : SV_DomainLocation, const OutputPatch<HS_CONTROL_POINT_OUTPUT, 3> triPatch)
{
	DS_OUTPUT output = (DS_OUTPUT)0;

	//Interpolate world space position with barycentric coordinates
	float3 worldPosition =
		barycentricCoordinates.x * triPatch[0].worldPosition + 
		barycentricCoordinates.y * triPatch[1].worldPosition +
		barycentricCoordinates.z * triPatch[2].worldPosition;
	output.worldPosition = worldPosition;

	return output;
}


// Geometry shader
[maxvertexcount(3)]
void mainGS_High(triangle DS_OUTPUT input[3], inout TriangleStream<GS_OUTPUT> triStream)
{
	GS_OUTPUT output = (GS_OUTPUT)0;

	//perform view projection transformation
	for (int i = 0; i<3; i++)
	{

		float4 screenPosition = mul(viewProjMatrix, float4(input[i].worldPosition, 1.0));
		output.screenPositionSV = screenPosition;
		triStream.Append(output);
	}
	triStream.RestartStrip();
}


// Per-pixel color data passed through the pixel shader. A pass-through function for the (interpolated) color data.
PS_OUTPUT mainPS_High(GS_OUTPUT input)
{
	//set to zero
	PS_OUTPUT output = (PS_OUTPUT)0;
	output.color0 = float4(1.0f, 0.0f, 0.0f, 1.0f);

	return output;
}

I used FXC command line - "C:\Program Files (x86)\Windows Kits\10\bin\x64\fxc.exe" /Od /Zi ErrorTest.fx /Fo tmpHH.o /T hs_5_1 /E mainHS_High

I get the same error as before. But if I switch to the commented out code in the hull shader, it works fine;

I am using insider build of Windows. It should not matter but I just posted it there just in case.

Thanks

I'll send it through to the compiler team to take a look at.

The crash is actually already fixed in the latest internal builds of the compiler, so you'll be able to pick up this fix in the next version of the Windows SDK.

In the meantime, aside from placing one of your constant buffers in 'space1', you don't appear to be doing anything requiring Shader Model 5.1. The shader compiles fine if you use Shader Model 5.0 (hs_5_0). If you move your constant buffer from space1 into slot 'b1' of space0 and use hs_5_0, that'll unblock you in the meantime.

Thanks,

Adam

Adam Miles - Principal Software Development Engineer - Microsoft Xbox Advanced Technology Group

Thank you very much Adam. I will try the solution you recommended.

Earlier, I also posted it here too

https://social.msdn.microsoft.com/Forums/Windowsapps/en-US/b1bb849d-4908-48a9-b7a4-fa9f61d0a58c/directx-12-hull-shader-error?forum=wpdevelop

to find solution as? soon as possible :) I will try to remove it.

Thanks

This topic is closed to new replies.

Advertisement