• Create Account

## Deferred Rendering Problem

Old topic!

Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.

10 replies to this topic

### #1trevex  Members

125
Like
0Likes
Like

Posted 22 December 2012 - 07:12 AM

I am currently working on a small deferred rendering engine for this semester's coursework assignment. The code compiles fine and runs in Visual Studio without any problems except no output to the screen and that's where the problems start.

The deferred rendering system is nothing special it is basically the first draft of the algorithms presented in "Practical Rendering with DirectX 11" [2011 by Jason Zink, Matt Pettineo, Jack Hoxley]. No optimization like Attribute packing etc. are used. Only a simple Lighting system is used (no shadow).

The problems seems to be the fullscreen-quad(simply inefficient fullscreen-quads used for lighting passes) or a general problem. Since I get no errors in the debug log I tried to use PIX and PerfStudio to get some more information on the gpu side. Unfortunately PIX and PerfStudio are before the first frame with this error:

Invalid allocation size: 4294967295 bytes

So for whatever reason it seems to allocate some space with -1 bytes. Awkwardly everything is fine in VisualStudio debugging... and if I attach a debugger to the PIX process and break when the error happens I land in a debuggers header file.

I just started using DirectX with prior OpenGL experience, so I hope I did not something generally wrong. I used the executable that was output by the compiler in debug mode.

To avoid general logic mistakes, here is roughly what I currently do:

1. SetDepthStencilState (with DepthTest enabled)
2. clear all RenderTargetViews (was unsure about gBuffer but gBuffer is being cleared as well currently) and DepthStencilBuffer
3. bind gBuffer and DepthStencilBuffer
4. render Geometry
5. disable DepthTest
6. bind Backbuffer
7. render all lights with the associated shader (since I am using the effects framework I set the BlendState in the shader)
8. render CEGUI (works fine even if rest doesn't output anything)
9. present()

The lights are as already mentioned fullscreen quads. The lighting technique is simply passing the position through, so the quads vertices are in the range [-1, 1].

If you need any additional informations let me know.

Thanks, Nik

P.S. sorry for the bad english...

EDIT:

For further informations: The vertices and indices of the fullscreen quad

glm::vec3 vertices[] =    {		glm::vec3(-1.0f, -1.0f,  1.0f),		glm::vec3(-1.0f,  1.0f,  1.0f),		glm::vec3( 1.0f, -1.0f,  1.0f),		glm::vec3( 1.0f,  1.0f,  1.0f),    };	    UINT indices[] = { 0, 3, 2, 2, 0, 1 };

And a rough walkthough the code:

// before geometry pass
m_d3dImmediateContext->OMSetDepthStencilState(m_GeometryDepthStencilState, 1);
m_d3dImmediateContext->ClearRenderTargetView(m_RenderTargetView, reinterpret_cast<const float*>(&clearColor));
m_d3dImmediateContext->ClearRenderTargetView(m_gBuffer[0], reinterpret_cast<const float*>(&clearColor));
m_d3dImmediateContext->ClearRenderTargetView(m_gBuffer[1], reinterpret_cast<const float*>(&clearColor));
m_d3dImmediateContext->ClearRenderTargetView(m_gBuffer[2], reinterpret_cast<const float*>(&clearColor));
m_d3dImmediateContext->ClearRenderTargetView(m_gBuffer[3], reinterpret_cast<const float*>(&clearColor));
m_d3dImmediateContext->ClearDepthStencilView(m_DepthStencilView, D3D11_CLEAR_DEPTH|D3D11_CLEAR_STENCIL, 1.0f, 0);
m_d3dImmediateContext->OMSetRenderTargets(4, m_gBuffer, m_DepthStencilView);

// before lighting pass
m_d3dImmediateContext->OMSetDepthStencilState(m_LightingDepthStencilState, 1);
m_d3dImmediateContext->OMSetRenderTargets(1, &m_RenderTargetView, m_DepthStencilView);

// DXLightingShader::enable (the static cast is necessary because the engine supports opengl and directx this is my dirty way
static_cast<SDXRenderInfo*>(g_RenderInfo)->context->IASetInputLayout(m_InputLayout);
static_cast<SDXRenderInfo*>(g_RenderInfo)->context->IASetPrimitiveTopology(D3D11_PRIMITIVE_TOPOLOGY_TRIANGLELIST);
m_fxNormalMap->SetResource(m_NormalView);
m_fxDiffuseMap->SetResource(m_DiffuseView);
m_fxSpecularMap->SetResource(m_SpecularView);
m_fxPositionMap->SetResource(m_PositionView);
m_fxCameraPos->SetFloatVector(Camera->getPosition());

// depending on light-type this is how it is drawn
for(UINT p = 0; p < m_DirectionalLightDesc.Passes; ++p)
{
m_DirectionalLight->GetPassByIndex(p)->Apply(0, static_cast<SDXRenderInfo*>(g_RenderInfo)->context);
static_cast<SDXRenderInfo*>(g_RenderInfo)->context->DrawIndexed(6, 0, 0);
}

// present function called after light passes
CEGUI::System::getSingleton().renderGUI();
HR(m_SwapChain->Present(0, 0));

m_fxNormalMap->SetResource(NULL);
m_fxDiffuseMap->SetResource(NULL);
m_fxSpecularMap->SetResource(NULL);
m_fxPositionMap->SetResource(NULL);
m_DirectionalLight->GetPassByIndex(0)->Apply(0, static_cast<SDXRenderInfo*>(g_RenderInfo)->context);


Edited by trevex, 22 December 2012 - 07:28 AM.

### #2eppo  Members

4597
Like
1Likes
Like

Posted 22 December 2012 - 10:17 AM

Invalid allocation size: 4294967295 bytes

What makes you think this has anything to do with your deferred rendering setup? It could simply be an uninitialized variable somewhere else in your code. What header file do you break into?

### #3trevex  Members

125
Like
0Likes
Like

Posted 23 December 2012 - 06:45 AM

Thanks for the tip. I am currently in chrismas stress but I am trying to debug the application again now.

So I figured out what the problem was, it was quite simple actually... Visual Studio has a different runtime environment, so pix wasn't able to find some files...

I am investigating why it is not rendering now, I'll hopefully come back with more informations later...

### #4trevex  Members

125
Like
0Likes
Like

Posted 23 December 2012 - 07:30 AM

So I started debugging a frame with pix:

The 4 gBuffer textures are successfully being rendered.

The problem seems to be the fullscreen quad:

Since in the viewport output there is only a whiteline it seems to get discarded?

### #5unbird  Members

8297
Like
1Likes
Like

Posted 23 December 2012 - 07:47 AM

With z < 0 you're out of clipspace already. And a w of 0 will even send them to infinity. Can you show us that vertex shader code (and the transformation matrix values involved, if available).

### #6trevex  Members

125
Like
0Likes
Like

Posted 23 December 2012 - 05:36 PM

Sure, sorry for delayed reply chrismas time...


</p><div>float4 VSMain(in float3 Position : POSITION) : SV_Position </div>
<div>{ </div>
<div>     return float4(Position, 0.0f); </div>
<div>}</div>

for what ever reason the z value is always -1.0 also the preVS value...

but this is the vertex data in the associated buffer:


</p><div>glm::vec3 vertices[] =</div>
<div>    {</div>
<div>glm::vec3(-1.0f, -1.0f,  1.0f),</div>
<div>glm::vec3(-1.0f,  1.0f,  1.0f),</div>
<div>glm::vec3( 1.0f, -1.0f,  1.0f),</div>
<div>glm::vec3( 1.0f,  1.0f,  1.0f),</div>
<div>    };</div>

As already stated out previously I am new to directx, so is there anything that can influence how vertex data is interpreted since the preVS value is also -1?

### #7trevex  Members

125
Like
0Likes
Like

Posted 23 December 2012 - 05:46 PM

Ok just noticed I was setting last value to 0.0 and not 1.0...
The problem now is I changed the z value of the input vertices to 0.0f no change, I changed the shader to set the z value to 0.0f I get some output, still not the right shaded cube but it seems to be a problem with my lighting code...

But this would be kind of a dirty fix, so why is the Z value being set to -1...?

Edited by trevex, 23 December 2012 - 05:50 PM.

### #8unbird  Members

8297
Like
1Likes
Like

Posted 23 December 2012 - 06:42 PM

float4 VSMain(in float4 Position : POSITION) : SV_Position
{
return Position;
}

since w = 1 will happen automatically. This is the rare occasion where signature and layout may differ.

But your problem is elsewhere. PreVS means either your buffer init/update or your input layout is off - or you've bound the wrong buffer(s).

### #9trevex  Members

125
Like
0Likes
Like

Posted 24 December 2012 - 03:41 AM

Thanks alot indeed I forgot to set the VertexBuffer and since the vertices of my cube are quite similar, I didn't notice that. Can't believe I haven't noticed that...

The only thing left now is a bug in my lighting code, some surfaces stay black... The scene currently uses a single directional light for testing!
float3 CalcLighting(in float3 normal, in float3 position, in float3 diffuseAlbedo, in float3 specularAlbedo, in float specularPower, uniform int gLightingMode)
{
float3 L = 0;
float attenuation = 1.0f;
if (gLightingMode == POINTLIGHT || gLightingMode == SPOTLIGHT)
{
L = LightPos - position;
float dist = length(L);
attenuation = max(0, 1.0f - (dist / LightRange.x));
L /= dist;
}
else if (gLightingMode == DIRECTIONALLIGHT)
{
L = -LightDirection; }if (gLightingMode == SPOTLIGHT)
{
float3 L2 = LightDirection;
float rho = dot(-L, L2);
attenuation *= saturate((rho - SpotlightAngles.y) / (SpotlightAngles.x - SpotlightAngles.y));
}
float nDotL = saturate(dot(normal, L));
float3 diffuse = nDotL * LightColor * diffuseAlbedo;
float3 V = CameraPos - position;
float3 H = normalize(L + V);
float3 specular = pow(saturate(dot(normal, H)), specularPower) * LightColor * specularAlbedo.xyz * nDotL;
return (diffuse + specular) * attenuation;
}

The basic algorithm is basically out of a book so I assumed there would be nothing wrong the only thing I changed is using if and a uniform to embed it in an effects file.

The Buffers are filled with informations and there is nothing missing, these are values of a black pixel that is supposed to have some color:

EDIT:

Lighting Shader Code on pastebin http://pastebin.com/8gbYBLCX, because the code tags seem to be buggy currently either escaping html as well or completly breaking formating

Edited by trevex, 24 December 2012 - 03:49 AM.

### #10eppo  Members

4597
Like
1Likes
Like

Posted 24 December 2012 - 11:16 AM

What color do you expect the surface to be? The vector L / -LightDirection (unnormalized) hits the surface orthogonal to its normal, so nDotL equals zero. Since both the diffuse and the specular coefficient are multiplied by nDotL the surface reflects no light. If you don't want to fade specular hi-lights with the light's incidence, you should remove the last '* nDotL'.

### #11trevex  Members

125
Like
0Likes
Like

Posted 24 December 2012 - 11:43 AM

You are right, thanks alot for all the help, I am successfully rendering loads of pointlight, spotlight etc. now... Best christmas present so far

Old topic!

Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.