Jason Z

Thanks for sharing your results so far.  From my past experience, what you have said is correct - mapping the resource on a deferred context will produce a temporary buffer that carries around your data with it, and the results are applied only when the command list is executed.  I also seem to recall that you can't read data from this pointer on a deferred context - only writing.

I'm not sure what the actual issue is now - are your changes not being applied to the mapped buffer after executing the command list?

Unbird is right - I meant the depth/stencil state and blend state combined when I referred to the output merger state.  Sorry for the confusion.

Regarding the default parameters, you can take a look at the description pages for each of the states, and they list the true defaults that should be sensible.  For example, here is the blend state description.  Once you get those initialized, it should be fairly easy to proceed with the debugging.

Can you do a PIX pixel history to see where the pixels generated from that geometry get dumped?  Typically you should be looking for something like a viewport not set (which appears not to be the case now) rasterizer state not correct (i.e. backface culling), scissor rect not set correctly (but the default state is neutral, so unless you set one it can't be the problem), and then depth test, stencil test, and blending.

Can you show us the rasterizer state and output merger state that are being used for your tessellation draw call?

The w-components of your post GS vertices is very large - that means that your geometry is very far away, and will appear small in your viewport.  Can you double check your view and projection matrices to make sure they are properly set?  Have you tried these matrices with a normal rendering setup (i.e. sans tessellation)?

interesting tip unbird. I thought that I couldn't have a geometry shader when the tessellation stages were active.

There isn't any restriction like that.  You have to have a vertex shader and a pixel shader, but outside of that there is no hard rule except that the last stage before the rasterizer has to output the SV_Position semantic.

About the control points - just output them as regular points, and ensure their positions are as expected.  Don't expend too much energy on the verification - you are just making sure it is set up the way you expect.  That should really only require the changes in the shaders, plus the primitive topology.

It looks like the declared position format in your vertex layout is too small.  You are using a 4 component float position, but only declaring a 3 component format.  Since your offset in the layout to the texture coordinates is 16 bytes, this is likely the source of your problems.  If you switch to having a DXGI_FORMAT_R32G32B32A32_FLOAT for your position, that should help.

Did you get any output in the debug window?  If your shader was expecting a float4 then you vertex layout would not have matched and it should have complained about that...  Is the debug layer enabled in the device you are using?

It looks to me like your reflection texture is wrapping because that same figure in your first image is the same shape as it is at your character's legs.  I'm not familiar with the rastertek tutorials, but is there any provisions for clamping the reflection texture coordinates to the [0,1] range?

You will not get many responses with a post like this.  In general, you should try to describe the problem more thoroughly (and not only in the subject line...).  What have you tried, and what symptoms do you see?  Where did your code come from - was it based on a known working program, or is it completely new code?

Information like this will help others help you.

When I started out with tessellation, I found it helpful to really simplify things and then build up the complexity step by step.  I would suggest making your pixel shader just output a solid white color - then you can eliminate any shading errors or missing constant buffers from the pixel shader right off the bat.

Next I would actually eliminate the hull and domain shaders, and just pass your geometry from vertex shader to pixel shader.  Verify that your vertices are appearing in the location that you expect (this will likely require moving some transformation code from the DS to the VS, but you can move it back fairly easily later).  Instead of doing instanced rendering, try with just standard rendering instead (that will eliminate issues with your instance buffers). 

Only once you have the geometry input to the tessellation stages verified, and then that the output stages from the tessellation stages verified, would I try to debug the hull and domain shaders.  Have you been able to verify these things yet?  If so, then we can move a little deeper into the investigation - if we know it is a tessellation issue or not!

How do pros compile their shaders as part of their build pipeline? 

 

Right now, I am using a custom build step in Visual Studio to call fxc.  I have to call fxc for each permutation (specifying the various #define), which is kind of annoying for shaders that have a lot of permutations.  On the other hand, the old "Effects" framework was not much better in that you had to define a technique for each permutation. 

Are you using VS2012?  If so, then you don't need to create a custom build step as there is integrated support for compiling HLSL files.  Unfortunately there is a limit of one build rule for each HLSL file, so you have to play games with adding an empty HLSL file and just include the original target file to be able to specify multiple build targets from a single file.

That is how I would handle it - use separate render targets for each of your sub-views, and then just render a textured quad that copies the data into your final render target.  That will allow you to optimize the most, by skipping updating any of the subportions until they really need to be.

If you absolutely must be able to keep everything within a single render target, you can render a quad instead of clearing the screen.  Then you just make sure that the depth stencil state is set such that the depth test always passes, and you update the depth to be the 'reset' value that you would normally have just cleared to.  That should work for render targets too if you need to clear their contents.

But like I mentioned before, I would prefer to use the separate render targets - it will make your life easier!

Especially if it keeps you from using techniques like instancing, this should be an alert sign!

The compiler will remove unused objects from the shader file.  However, you should consider that each object will be located at a different register if there are different numbers of textures and constant buffers being bound (this assumes you don't explicitly set the register by hand).  That really isn't too big of a deal, but you have to consider it anyways.

If you set your build chain up properly, then you should build your shader as soon as you save it to disk, and then just use the compiled shader at runtime.  This is the strategy imposed on you by the Windows Store style apps, and it actually makes sense to follow this path.  So in this case, it probably won't make to much of a difference if the compilation takes a bit longer to remove the unnecessary objects.

Are you certain that the width and height are being interpreted as floating point numbers before doing the division?  If not, then try casting the values to float individually prior to the division.  It could be that you are accidentally using integer division, which will give you step changes in situations like this.