Jump to content

View more

Image of the Day

Trying out some of Pickle Jar compositions in @SketchUp and Unity, for a late #screenshotsaturday #gamedev https://t.co/HU0kZAnQtD
IOTD | Top Screenshots

The latest, straight to your Inbox.

Subscribe to GameDev.net's newsletters to receive the latest updates and exclusive content.

Sign up now

Deferred Rendering Problem

2: Adsense

Old topic!

Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.

  • You cannot reply to this topic
10 replies to this topic

#1 trevex   Members   


Posted 22 December 2012 - 07:12 AM

I am currently working on a small deferred rendering engine for this semester's coursework assignment. The code compiles fine and runs in Visual Studio without any problems except no output to the screen and that's where the problems start.

The deferred rendering system is nothing special it is basically the first draft of the algorithms presented in "Practical Rendering with DirectX 11" [2011 by Jason Zink, Matt Pettineo, Jack Hoxley]. No optimization like Attribute packing etc. are used. Only a simple Lighting system is used (no shadow).

The problems seems to be the fullscreen-quad(simply inefficient fullscreen-quads used for lighting passes) or a general problem. Since I get no errors in the debug log I tried to use PIX and PerfStudio to get some more information on the gpu side. Unfortunately PIX and PerfStudio are before the first frame with this error:

Invalid allocation size: 4294967295 bytes

So for whatever reason it seems to allocate some space with -1 bytes. Awkwardly everything is fine in VisualStudio debugging... and if I attach a debugger to the PIX process and break when the error happens I land in a debuggers header file.

I just started using DirectX with prior OpenGL experience, so I hope I did not something generally wrong. I used the executable that was output by the compiler in debug mode.

To avoid general logic mistakes, here is roughly what I currently do:

1. SetDepthStencilState (with DepthTest enabled)
2. clear all RenderTargetViews (was unsure about gBuffer but gBuffer is being cleared as well currently) and DepthStencilBuffer
3. bind gBuffer and DepthStencilBuffer
4. render Geometry
5. disable DepthTest
6. bind Backbuffer
7. render all lights with the associated shader (since I am using the effects framework I set the BlendState in the shader)
8. render CEGUI (works fine even if rest doesn't output anything)
9. present()

The lights are as already mentioned fullscreen quads. The lighting technique is simply passing the position through, so the quads vertices are in the range [-1, 1].

If you need any additional informations let me know.

Thanks, Nik

P.S. sorry for the bad english...


For further informations: The vertices and indices of the fullscreen quad

glm::vec3 vertices[] =    {		glm::vec3(-1.0f, -1.0f,  1.0f),		glm::vec3(-1.0f,  1.0f,  1.0f),		glm::vec3( 1.0f, -1.0f,  1.0f),		glm::vec3( 1.0f,  1.0f,  1.0f),    };	    UINT indices[] = { 0, 3, 2, 2, 0, 1 };

And a rough walkthough the code:

// before geometry pass
	m_d3dImmediateContext->OMSetDepthStencilState(m_GeometryDepthStencilState, 1);
	m_d3dImmediateContext->ClearRenderTargetView(m_RenderTargetView, reinterpret_cast<const float*>(&clearColor));
	m_d3dImmediateContext->ClearRenderTargetView(m_gBuffer[0], reinterpret_cast<const float*>(&clearColor));
	m_d3dImmediateContext->ClearRenderTargetView(m_gBuffer[1], reinterpret_cast<const float*>(&clearColor));
	m_d3dImmediateContext->ClearRenderTargetView(m_gBuffer[2], reinterpret_cast<const float*>(&clearColor));
	m_d3dImmediateContext->ClearRenderTargetView(m_gBuffer[3], reinterpret_cast<const float*>(&clearColor));
	m_d3dImmediateContext->ClearDepthStencilView(m_DepthStencilView, D3D11_CLEAR_DEPTH|D3D11_CLEAR_STENCIL, 1.0f, 0);
	m_d3dImmediateContext->OMSetRenderTargets(4, m_gBuffer, m_DepthStencilView);

// before lighting pass
        m_d3dImmediateContext->OMSetDepthStencilState(m_LightingDepthStencilState, 1);
	m_d3dImmediateContext->OMSetRenderTargets(1, &m_RenderTargetView, m_DepthStencilView);

// DXLightingShader::enable (the static cast is necessary because the engine supports opengl and directx this is my dirty way

// depending on light-type this is how it is drawn
        for(UINT p = 0; p < m_DirectionalLightDesc.Passes; ++p)
		m_DirectionalLight->GetPassByIndex(p)->Apply(0, static_cast<SDXRenderInfo*>(g_RenderInfo)->context);
		static_cast<SDXRenderInfo*>(g_RenderInfo)->context->DrawIndexed(6, 0, 0);

// present function called after light passes
	HR(m_SwapChain->Present(0, 0));

// DXLightingShader::disable
	m_DirectionalLight->GetPassByIndex(0)->Apply(0, static_cast<SDXRenderInfo*>(g_RenderInfo)->context);

If you need more information or some details of the shader implementation let me know

Edited by trevex, 22 December 2012 - 07:28 AM.

#2 eppo   Members   


Posted 22 December 2012 - 10:17 AM

Invalid allocation size: 4294967295 bytes



What makes you think this has anything to do with your deferred rendering setup? It could simply be an uninitialized variable somewhere else in your code. What header file do you break into?

#3 trevex   Members   


Posted 23 December 2012 - 06:45 AM

Thanks for the tip. I am currently in chrismas stress but I am trying to debug the application again now.


So I figured out what the problem was, it was quite simple actually... Visual Studio has a different runtime environment, so pix wasn't able to find some files...


I am investigating why it is not rendering now, I'll hopefully come back with more informations later...

#4 trevex   Members   


Posted 23 December 2012 - 07:30 AM

So I started debugging a frame with pix:


The 4 gBuffer textures are successfully being rendered.

The problem seems to be the fullscreen quad:





Since in the viewport output there is only a whiteline it seems to get discarded?

#5 unbird   Members   


Posted 23 December 2012 - 07:47 AM

With z < 0 you're out of clipspace already. And a w of 0 will even send them to infinity. Can you show us that vertex shader code (and the transformation matrix values involved, if available).

#6 trevex   Members   


Posted 23 December 2012 - 05:36 PM

Sure, sorry for delayed reply chrismas time...


</p><div>float4 VSMain(in float3 Position : POSITION) : SV_Position </div>
<div>{ </div>
<div>     return float4(Position, 0.0f); </div>


for what ever reason the z value is always -1.0 also the preVS value...

but this is the vertex data in the associated buffer:


</p><div>glm::vec3 vertices[] =</div>
<div>    {</div>
<div>glm::vec3(-1.0f, -1.0f,  1.0f),</div>
<div>glm::vec3(-1.0f,  1.0f,  1.0f),</div>
<div>glm::vec3( 1.0f, -1.0f,  1.0f),</div>
<div>glm::vec3( 1.0f,  1.0f,  1.0f),</div>
<div>    };</div>


As already stated out previously I am new to directx, so is there anything that can influence how vertex data is interpreted since the preVS value is also -1? 

#7 trevex   Members   


Posted 23 December 2012 - 05:46 PM

Ok just noticed I was setting last value to 0.0 and not 1.0...
The problem now is I changed the z value of the input vertices to 0.0f no change, I changed the shader to set the z value to 0.0f I get some output, still not the right shaded cube but it seems to be a problem with my lighting code...

But this would be kind of a dirty fix, so why is the Z value being set to -1...?

Edited by trevex, 23 December 2012 - 05:50 PM.

#8 unbird   Members   


Posted 23 December 2012 - 06:42 PM

Ah, already further, so yeah, your new vertex shader is probably fine. You can even do
float4 VSMain(in float4 Position : POSITION) : SV_Position
    return Position;
since w = 1 will happen automatically. This is the rare occasion where signature and layout may differ.

But your problem is elsewhere. PreVS means either your buffer init/update or your input layout is off - or you've bound the wrong buffer(s).

#9 trevex   Members   


Posted 24 December 2012 - 03:41 AM

Thanks alot indeed I forgot to set the VertexBuffer and since the vertices of my cube are quite similar, I didn't notice that. Can't believe I haven't noticed that...

The only thing left now is a bug in my lighting code, some surfaces stay black... The scene currently uses a single directional light for testing!
float3 CalcLighting(in float3 normal, in float3 position, in float3 diffuseAlbedo, in float3 specularAlbedo, in float specularPower, uniform int gLightingMode) 
float3 L = 0; 
float attenuation = 1.0f; 
if (gLightingMode == POINTLIGHT || gLightingMode == SPOTLIGHT)
L = LightPos - position; 
float dist = length(L); 
attenuation = max(0, 1.0f - (dist / LightRange.x)); 
L /= dist;
else if (gLightingMode == DIRECTIONALLIGHT)
L = -LightDirection; }if (gLightingMode == SPOTLIGHT)
float3 L2 = LightDirection; 
float rho = dot(-L, L2); 
attenuation *= saturate((rho - SpotlightAngles.y) / (SpotlightAngles.x - SpotlightAngles.y)); 
float nDotL = saturate(dot(normal, L)); 
float3 diffuse = nDotL * LightColor * diffuseAlbedo;
float3 V = CameraPos - position; 
float3 H = normalize(L + V);
float3 specular = pow(saturate(dot(normal, H)), specularPower) * LightColor * specularAlbedo.xyz * nDotL; 
return (diffuse + specular) * attenuation;
The basic algorithm is basically out of a book so I assumed there would be nothing wrong the only thing I changed is using if and a uniform to embed it in an effects file.

The Buffers are filled with informations and there is nothing missing, these are values of a black pixel that is supposed to have some color:


Lighting Shader Code on pastebin http://pastebin.com/8gbYBLCX, because the code tags seem to be buggy currently either escaping html as well or completly breaking formating

Edited by trevex, 24 December 2012 - 03:49 AM.

#10 eppo   Members   


Posted 24 December 2012 - 11:16 AM

What color do you expect the surface to be? The vector L / -LightDirection (unnormalized) hits the surface orthogonal to its normal, so nDotL equals zero. Since both the diffuse and the specular coefficient are multiplied by nDotL the surface reflects no light. If you don't want to fade specular hi-lights with the light's incidence, you should remove the last '* nDotL'.

#11 trevex   Members   


Posted 24 December 2012 - 11:43 AM

You are right, thanks alot for all the help, I am successfully rendering loads of pointlight, spotlight etc. now... Best christmas present so far :)

Old topic!

Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.