[SOLVED]Viewport Transform giving weird resuts. Need a fresh set of eyes.

Started by
10 comments, last by Postie 12 years, 1 month ago
Forgive the newbish plea for help but this is my first time using SlimDX and everything appears correct but I'm getting weird results (my quads are acting like they are being rendered straight to screenspace instead of being transformed by all the appropriate matrices). According to PIX, everything is as it should be until the viewport kicks in.

For reference, I'm outputting to a panel that is within a larger form (it's a simple level editor).

PIX Output:
http://i.imgur.com/v9uca.png

Code -

Device Setup/Viewport Creation
public DXRenderer(IntPtr outputHandle, System.Drawing.Size formSize)
{
SwapChainDescription swapDesc = new SwapChainDescription();
swapDesc.BufferCount = 1;
swapDesc.Usage = Usage.RenderTargetOutput;
swapDesc.OutputHandle = outputHandle;
swapDesc.IsWindowed = true;
swapDesc.ModeDescription = new ModeDescription(0, 0, new Rational(60, 1), Format.R8G8B8A8_UNorm);
swapDesc.SampleDescription = new SampleDescription(1, 0);
swapDesc.Flags = SwapChainFlags.AllowModeSwitch;
swapDesc.SwapEffect = SwapEffect.Discard;

if (SlimDX.Direct3D11.Device.CreateWithSwapChain(DriverType.Hardware, DeviceCreationFlags.None, swapDesc, out m_d3dDevice, out m_swapChain).IsFailure)
{
System.Windows.Forms.MessageBox.Show("Failed to create DX11 device.");
System.Environment.Exit(1);
}

var factory = m_swapChain.GetParent<Factory>();
factory.SetWindowAssociation(outputHandle, WindowAssociationFlags.IgnoreAll);

m_d3dContext = m_d3dDevice.ImmediateContext;
m_renderTarget = new RenderTargetView(m_d3dDevice, SlimDX.Direct3D11.Resource.FromSwapChain<Texture2D>(m_swapChain, 0));
m_d3dContext.OutputMerger.SetTargets(m_renderTarget);

SlimDX.Direct3D11.Viewport viewPort = new Viewport(0.0f, 0.0f, (float)formSize.Width, (float)formSize.Height, 0.0f, 1.0f);
m_d3dContext.Rasterizer.SetViewports(viewPort);

Camera frameCamera = new Camera();
float aspect = (float)formSize.Width / (float)formSize.Height;
float fov = (float)System.Math.PI * 0.5f;
frameCamera.ProjectionMatrix = SlimDX.Matrix.PerspectiveFovLH(fov, aspect, 1.0f, 1000.0f);
}


Shader Setup:
SlimDX.Direct3D11.Device device = DXRenderer.Instance.D3DDevice;
DataStream verticeStream;
// Setup Index/Vertex Buffers.
int vertexDeclSize = 24 * 4;
verticeStream = new DataStream(vertexDeclSize, false, true);
verticeStream.Write(new Vector4(0.0f, 0.0f, 0.0f, 0.0f));
verticeStream.Write(new Vector2(0.0f, 0.0f));
verticeStream.Write(new Vector4(0.0f, 1.0f, 0.0f, 0.0f));
verticeStream.Write(new Vector2(0.0f, 0.0f));
verticeStream.Write(new Vector4(1.0f, 1.0f, 0.0f, 0.0f));
verticeStream.Write(new Vector2(1.0f, 1.0f));
verticeStream.Write(new Vector4(1.0f, 0.0f, 0.0f, 0.0f));
verticeStream.Write(new Vector2(1.0f, 1.0f));
verticeStream.Position = 0;
m_vertexBuffer = new SlimDX.Direct3D11.Buffer(device, verticeStream, vertexDeclSize, SlimDX.Direct3D11.ResourceUsage.Dynamic, SlimDX.Direct3D11.BindFlags.VertexBuffer,
SlimDX.Direct3D11.CpuAccessFlags.Write, SlimDX.Direct3D11.ResourceOptionFlags.None, 0);
verticeStream.Close();
SlimDX.DataStream indexStream = new DataStream(sizeof(UInt16) * 6, false, true);
indexStream.Write((UInt16)0);
indexStream.Write((UInt16)1);
indexStream.Write((UInt16)2);
indexStream.Write((UInt16)0);
indexStream.Write((UInt16)2);
indexStream.Write((UInt16)3);
indexStream.Position = 0;
m_indexBuffer = new SlimDX.Direct3D11.Buffer(device, indexStream, sizeof(UInt16) * 6, SlimDX.Direct3D11.ResourceUsage.Default, SlimDX.Direct3D11.BindFlags.IndexBuffer,
SlimDX.Direct3D11.CpuAccessFlags.None, SlimDX.Direct3D11.ResourceOptionFlags.None, 0);
indexStream.Close();
// Compile shaders
SlimDX.D3DCompiler.ShaderBytecode vertexByteCode = SlimDX.D3DCompiler.ShaderBytecode.CompileFromFile("./Shaders/Sprite.fx", "VS", "vs_5_0", SlimDX.D3DCompiler.ShaderFlags.None, SlimDX.D3DCompiler.EffectFlags.None);
SlimDX.D3DCompiler.ShaderSignature inputSig = SlimDX.D3DCompiler.ShaderSignature.GetInputSignature(vertexByteCode);
m_vertexShader = new SlimDX.Direct3D11.VertexShader(device, vertexByteCode);
m_pixelShader = new SlimDX.Direct3D11.PixelShader(device, SlimDX.D3DCompiler.ShaderBytecode.CompileFromFile("./Shaders/Sprite.fx", "PS", "ps_5_0", SlimDX.D3DCompiler.ShaderFlags.None, SlimDX.D3DCompiler.EffectFlags.None));
m_inputLayout = new SlimDX.Direct3D11.InputLayout(device, inputSig, InputLayoutManager.TextureVertexLayout);

// Setup Constant Buffer and Texture Sampler State.
int matrixSize = (sizeof(float) * 4 * 4); // 4 rows, each with 4 values - x,y,z,w
DataStream constantStream;
constantStream = new DataStream(matrixSize * 3, true, true);
constantStream.Write(SlimDX.Matrix.Identity);
constantStream.Write(SlimDX.Matrix.Identity);
constantStream.Write(SlimDX.Matrix.Identity);
constantStream.Position = 0;
m_constantBuffer = new SlimDX.Direct3D11.Buffer(device, constantStream, matrixSize * 3, SlimDX.Direct3D11.ResourceUsage.Dynamic, SlimDX.Direct3D11.BindFlags.ConstantBuffer,
SlimDX.Direct3D11.CpuAccessFlags.Write, SlimDX.Direct3D11.ResourceOptionFlags.None, 0);
constantStream.Close();
SlimDX.Direct3D11.SamplerDescription samplerDesc = new SlimDX.Direct3D11.SamplerDescription();
samplerDesc.AddressU = SlimDX.Direct3D11.TextureAddressMode.Wrap;
samplerDesc.AddressV = SlimDX.Direct3D11.TextureAddressMode.Wrap;
samplerDesc.AddressW = SlimDX.Direct3D11.TextureAddressMode.Wrap;
samplerDesc.ComparisonFunction = SlimDX.Direct3D11.Comparison.Never;
samplerDesc.Filter = SlimDX.Direct3D11.Filter.MinMagMipLinear;
samplerDesc.MinimumLod = 0.0f;
samplerDesc.MaximumLod = float.MaxValue;
m_samplerState = SlimDX.Direct3D11.SamplerState.FromDescription(device, samplerDesc);


Actual Render call:
public void RenderSprite(Texture texture, Vector3 position, Vector2 size, Vector2 TopLeft, Vector2 BottomRight)
{
SlimDX.Direct3D11.DeviceContext context = DXRenderer.Instance.D3DContext;
context.InputAssembler.InputLayout = m_inputLayout;
context.InputAssembler.PrimitiveTopology = SlimDX.Direct3D11.PrimitiveTopology.TriangleList;
context.VertexShader.Set(m_vertexShader);
context.VertexShader.SetConstantBuffer(m_constantBuffer, 0);
context.PixelShader.Set(m_pixelShader);
context.PixelShader.SetSampler(m_samplerState, 0);
context.PixelShader.SetConstantBuffer(m_constantBuffer, 0);
DataBox vertices = context.MapSubresource(m_vertexBuffer, SlimDX.Direct3D11.MapMode.WriteDiscard, SlimDX.Direct3D11.MapFlags.None);
DataStream verticeStream = vertices.Data;
// 1 - 2
// | / |
// 0 - 3
verticeStream.Write(new Vector4(0.0f, 0.0f - size.Y, 0.0f, 0.0f));
verticeStream.Write(new Vector2(TopLeft.X, BottomRight.Y));
verticeStream.Write(new Vector4(0.0f, 0.0f, 0.0f, 0.0f));
verticeStream.Write(new Vector2(TopLeft.X, TopLeft.Y));
verticeStream.Write(new Vector4(0.0f + size.X, 0.0f, 0.0f, 0.0f));
verticeStream.Write(new Vector2(BottomRight.X, TopLeft.Y));
verticeStream.Write(new Vector4(0.0f + size.X, 0.0f - size.Y, 0.0f, 0.0f));
verticeStream.Write(new Vector2(BottomRight.X, BottomRight.Y));
context.UnmapSubresource(m_vertexBuffer, 0);
DataBox constant = context.MapSubresource(m_constantBuffer, SlimDX.Direct3D11.MapMode.WriteDiscard, SlimDX.Direct3D11.MapFlags.None);
DataStream constantStream = constant.Data;

Camera camera = Camera.Instance;
Matrix world = SlimDX.Matrix.Translation(position);

constantStream.Write(SlimDX.Matrix.Transpose(world));
constantStream.Write(SlimDX.Matrix.Transpose(camera.ViewMatrix));
constantStream.Write(SlimDX.Matrix.Transpose(camera.ProjectionMatrix));

context.UnmapSubresource(m_constantBuffer, 0);
context.InputAssembler.SetIndexBuffer(m_indexBuffer, SlimDX.DXGI.Format.R16_UInt, 0);
context.InputAssembler.SetVertexBuffers(0, new SlimDX.Direct3D11.VertexBufferBinding(m_vertexBuffer, 24, 0));
context.PixelShader.SetShaderResource(new SlimDX.Direct3D11.ShaderResourceView(DXRenderer.Instance.D3DDevice, texture.Resource), 0);
context.DrawIndexed(6, 0, 0);
}


Sorry for all the code (and some of it is poorly laid out, i.e. setting the pixel/vertex shader every time I render an object, but I was trying ANYTHING to get this thing working late last night. Hopefully someone can see what I'm obviously missing.
Advertisement
Are you sure your vertex shader is working properly? Your viewport seems to be set up fine.
Mike Popoloski | Journal | SlimDX

Are you sure your vertex shader is working properly? Your viewport seems to be set up fine.


That's what I thought as well. The vertex shader is basically straight from the DirectX samples.

//--------------------------------------------------------------------------------------
// File: Tutorial04.fx
//
// Copyright (c) Microsoft Corporation. All rights reserved.
//--------------------------------------------------------------------------------------
//--------------------------------------------------------------------------------------
// Constant Buffer Variables
//--------------------------------------------------------------------------------------
cbuffer ConstantBuffer : register( b0 )
{
matrix World;
matrix View;
matrix Projection;
}
Texture2D txDiffuse : register( t0 );
SamplerState samLinear : register( s0 );
//--------------------------------------------------------------------------------------
struct VS_INPUT
{
float4 Pos : POSITION;
float2 Tex : TEXCOORD0;
};
struct VS_OUTPUT
{
float4 Pos : SV_POSITION;
float2 Tex : TEXCOORD0;
};
//--------------------------------------------------------------------------------------
// Vertex Shader
//--------------------------------------------------------------------------------------
VS_OUTPUT VS(VS_INPUT input )
{
VS_OUTPUT output = (VS_OUTPUT)0;
output.Pos = mul( input.Pos, World );
output.Pos = mul( output.Pos, View );
output.Pos = mul( output.Pos, Projection );
output.Tex = input.Tex;

return output;
}

//--------------------------------------------------------------------------------------
// Pixel Shader
//--------------------------------------------------------------------------------------
float4 PS( VS_OUTPUT input ) : SV_Target
{
return txDiffuse.Sample( samLinear, input.Tex );
}
Are your matrix muls in the wrong order? I can never seem to remember which way it expects them to be by default, but you're transposing your matrices in there, so maybe you need to reverse them.
Mike Popoloski | Journal | SlimDX
Hmmm, still nothing. Looking at PIX the constant buffer appears to be getting set correctly. And the vertices appear to be transformed properly. Altering the Viewport dimension causes the rect to shrink/grow so obviously it's being applied... If I change the texture it's trying to load I get a different colored rect (I'm not setting blend states so it looks like a black/grey/red/whatever the texture is square - although it looks like the texture is always at the lowest MIP level...). So, obviously the pixel shader is getting executed as well.

I keep thinking it has to be something screwy with the order of how I'm setting the vertex / pixel shader and all their resources. I only create the constant buffer once at runtime (with all identity matrices) and then update it via the map/unmap methods (I've also tried just calling UpdateSubresource to directly update it without the Map commands).

I'm stumped. :/

Here's the Vertex Declaration (even though it looks fine to me):
static private SlimDX.Direct3D11.InputElement[] ms_texturedVertex;
static InputLayoutManager()
{
ms_texturedVertex = new[] { new SlimDX.Direct3D11.InputElement("POSITION", 0, SlimDX.DXGI.Format.R32G32B32A32_Float, 0, 0),
new SlimDX.Direct3D11.InputElement("TEXCOORD", 0, SlimDX.DXGI.Format.R32G32_Float, 16, 0) };
}
static public SlimDX.Direct3D11.InputElement[] TextureVertexLayout
{
get { return ms_texturedVertex; }
}


And the latest render call (with the changes I made in how I was populating the ConstantBuffer).
public void RenderSprite(Texture texture, Vector3 position, Vector2 size, Vector2 TopLeft, Vector2 BottomRight)
{
SlimDX.Direct3D11.DeviceContext context = DXRenderer.Instance.D3DContext;
context.VertexShader.Set(m_vertexShader);
context.PixelShader.Set(m_pixelShader);
context.PixelShader.SetSampler(m_samplerState, 0);
DataBox vertices = context.MapSubresource(m_vertexBuffer, SlimDX.Direct3D11.MapMode.WriteDiscard, SlimDX.Direct3D11.MapFlags.None);
DataStream verticeStream = vertices.Data;
// 1 - 2
// | / |
// 0 - 3
verticeStream.Write(new Vector4(0.0f, 0.0f - size.Y, 0.0f, 0.0f));
verticeStream.Write(new Vector2(TopLeft.X, BottomRight.Y));
verticeStream.Write(new Vector4(0.0f, 0.0f, 0.0f, 0.0f));
verticeStream.Write(new Vector2(TopLeft.X, TopLeft.Y));
verticeStream.Write(new Vector4(0.0f + size.X, 0.0f, 0.0f, 0.0f));
verticeStream.Write(new Vector2(BottomRight.X, TopLeft.Y));
verticeStream.Write(new Vector4(0.0f + size.X, 0.0f - size.Y, 0.0f, 0.0f));
verticeStream.Write(new Vector2(BottomRight.X, BottomRight.Y));
context.UnmapSubresource(m_vertexBuffer, 0);
DataStream constantStream = new DataStream(System.Runtime.InteropServices.Marshal.SizeOf(typeof(ConstantBuffer)), true, true);

Camera camera = Camera.Instance;
Matrix world = SlimDX.Matrix.Translation(position);
ConstantBuffer cBuffer;
cBuffer.WorldMatrix = SlimDX.Matrix.Transpose(world);
cBuffer.ViewMatrix = SlimDX.Matrix.Transpose(Camera.Instance.ViewMatrix);
cBuffer.ProjectionMatrix = SlimDX.Matrix.Transpose(Camera.Instance.ProjectionMatrix);
constantStream.Write(cBuffer);
constantStream.Position = 0;
context.UpdateSubresource(new DataBox(0, 0, constantStream), m_constantBuffer, 0);

context.VertexShader.SetConstantBuffer(m_constantBuffer, 0);
context.PixelShader.SetConstantBuffer(m_constantBuffer, 0);
context.InputAssembler.InputLayout = m_inputLayout;
context.InputAssembler.PrimitiveTopology = SlimDX.Direct3D11.PrimitiveTopology.TriangleList;
context.InputAssembler.SetIndexBuffer(m_indexBuffer, SlimDX.DXGI.Format.R16_UInt, 0);
context.InputAssembler.SetVertexBuffers(0, new SlimDX.Direct3D11.VertexBufferBinding(m_vertexBuffer, 24, 0));
context.PixelShader.SetShaderResource(new SlimDX.Direct3D11.ShaderResourceView(DXRenderer.Instance.D3DDevice, texture.Resource), 0);
context.DrawIndexed(6, 0, 0);
}
Also, it appears like the Viewport is being centered on the Form's size rather than the panel's size (the panel is only about 3/4th's the size of the entire editor(form) ). Is there anywhere in SlimDX where it assumes to use the application/forms size rather than the output panel you tell it about? Also, perhaps I need to update the X,Y of my viewport to account for that offset? Does the form need to inherit from SlimDX.Windows.RenderForm (it was previously, didn't seem to change things)?

Just throwing out suggestions here. :/
Are you actually setting up the Camera's ViewMatrix (aka LookAt)? I can see you're setting up the ProjectionMatrix, but don't see any mention of the ViewMatrix.
[size="2"]Currently working on an open world survival RPG - For info check out my Development blog:[size="2"] ByteWrangler
There are no assumptions made about the window. It should work fine with any native Win32 window handle.

Have you checked to see if there is any debug output that might shed some light? You can also try running using the reference rasterizer to see if it's a driver problem.
Mike Popoloski | Journal | SlimDX
I tried a reference model with no luck and found no info in the DXDebug log.

I've tossed the solution (it's only a few files and very simple) up here:

http://www.mediafire.com/?9vfgnms1segya66

If anyone could check it out and report back their findings, I'd greatly appreciate it. Otherwise I'll continue digging!
The w coords you're using for position are all 0 - they should be 1. Check your verticeStream.Write(new Vector4(...)) calls.

Direct3D has need of instancing, but we do not. We have plenty of glVertexAttrib calls.

This topic is closed to new replies.

Advertisement