Jump to content
  • Advertisement
Sign in to follow this  

OpenGL Can't get simple Tesselation Shaders to execute

This topic is 1152 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

Hello all!


I closely followed the following tutorial for basic OpenGL tesselation:




I am using it in a slightly different context with Deferred Rendering.


All I did was added a simple tesselation control shader to completely unalter the triangle patches:




Tesselation Control Shadee:

#version 420                                                                               
// define the number of CPs in the output patch                                                 
layout (vertices = 3) out;                                                                      
uniform vec3 gEyeWorldPos;                                                                      
// attributes of the input CPs                                                                  
  in vec3 WorldPos_CS_in[];                                                                       
  in vec2 TexCoord_CS_in[];                                                                       
  in vec3 Normal_CS_in  [];
  in vec3 Tangent_CS_in [];
// attributes of the output CPs                                                                 
 out vec3 WorldPos_ES_in[];                                                                      
 out vec2 TexCoord_ES_in[];                                                                      
 out vec3 Normal_ES_in[];                                                                        
 out vec3 Tangent_ES_in[];     
float GetTessLevel(float Distance0, float Distance1)                                            
    float AvgDistance = (Distance0 + Distance1) / 2.0;                                          
    if (AvgDistance <= 2.0) {                                                                   
        return 10.0;                                                                            
    else if (AvgDistance <= 5.0) {                                                              
        return 7.0;                                                                             
    else {                                                                                      
        return 3.0;                                                                             
void main()                                                                                     
    // Set the control points of the output patch                                               
    TexCoord_ES_in[gl_InvocationID] = TexCoord_CS_in[gl_InvocationID];                          
    Normal_ES_in[gl_InvocationID] = Normal_CS_in[gl_InvocationID];  
    Tangent_ES_in[gl_InvocationID] = Tangent_CS_in[gl_InvocationID];                          
    WorldPos_ES_in[gl_InvocationID] = WorldPos_CS_in[gl_InvocationID];                          
    // Calculate the distance from the camera to the three control points                       
    float EyeToVertexDistance0 = distance(gEyeWorldPos, WorldPos_ES_in[0]);                     
    float EyeToVertexDistance1 = distance(gEyeWorldPos, WorldPos_ES_in[1]);                     
    float EyeToVertexDistance2 = distance(gEyeWorldPos, WorldPos_ES_in[2]);                     
    // Calculate the tessellation levels                                                        
    gl_TessLevelOuter[0] = GetTessLevel(EyeToVertexDistance1, EyeToVertexDistance2);            
    gl_TessLevelOuter[1] = GetTessLevel(EyeToVertexDistance2, EyeToVertexDistance0);            
    gl_TessLevelOuter[2] = GetTessLevel(EyeToVertexDistance0, EyeToVertexDistance1);            
    gl_TessLevelInner[0] = gl_TessLevelOuter[2];                                                

And a simple Tesselation Evaluation Shader (this will ultimately sample from a Displacement Map, but at the moment I have the displacement commented out)

#version 420                                                                                 
layout(triangles, equal_spacing, ccw) in;                                                       
uniform mat4 gVP;                                                                               
uniform sampler2D gDisplacementMap;                                                             
uniform float gDispFactor;                                                                      
  in vec3 WorldPos_ES_in[];                                                                       
  in vec2 TexCoord_ES_in[];                                                                       
  in vec3 Normal_ES_in[]; 
  in vec3 Tangent_ES_in[];  
  out vec3 WorldPos_FS_in;                                                                        
  out vec2 TexCoord_FS_in;                                                                        
  out vec3 Normal_FS_in;                                                                          
  out vec3 Tangent_FS_in;
vec2 interpolate2D(vec2 v0, vec2 v1, vec2 v2)                                                   
    return vec2(gl_TessCoord.x) * v0 + vec2(gl_TessCoord.y) * v1 + vec2(gl_TessCoord.z) * v2;   
vec3 interpolate3D(vec3 v0, vec3 v1, vec3 v2)                                                   
    return vec3(gl_TessCoord.x) * v0 + vec3(gl_TessCoord.y) * v1 + vec3(gl_TessCoord.z) * v2;   
void main()                                                                                     
    // Interpolate the attributes of the output vertex using the barycentric coordinates        
    TexCoord_FS_in = interpolate2D(TexCoord_ES_in[0], TexCoord_ES_in[1], TexCoord_ES_in[2]);    
    Normal_FS_in = interpolate3D(Normal_ES_in[0], Normal_ES_in[1], Normal_ES_in[2]);            
    Normal_FS_in = normalize(Normal_FS_in);
	Tangent_FS_in = interpolate3D(Tangent_ES_in[0], Tangent_ES_in[1], Tangent_ES_in[2]);            
    Tangent_FS_in = normalize(Tangent_FS_in);
    WorldPos_FS_in = interpolate3D(WorldPos_ES_in[0], WorldPos_ES_in[1], WorldPos_ES_in[2]);    
    // Displace the vertex along the normal                                                     
    //float Displacement = texture(gDisplacementMap, TexCoord_FS_in.xy).x;                        
    //WorldPos_FS_in += Normal_FS_in * Displacement * gDispFactor;                                
    gl_Position = gVP * vec4(WorldPos_FS_in, 1.0);                                              

I am using NVidia NSight. If I do not attach the Control and Evaluation Shaders, the shader executes just fine. I can put breakpoints on my vertex and fragment shader in NSight, and the shaders are executing and I'm seeing what I'd expect on screen.


If I do attach the tesselation shaders, I see nothing on screen and putting breakpoints on the Vertex or Fragment shader doesn't work, indicating that they are not executing at all.


NSight requires that you are running OpenGL 4.x to run the shader debugger, so my OpenGL version should be fine. I remembered to call: glPatchParameteri(GL_PATCH_VERTICES, 3); and I checked my GL_MAX_PATCH_VERTICES and the value is 32.


I read that I may need to use GL_PATCHES as my primitive type, so I am using this in my draw call:

      glDrawElementsBaseVertex(GL_PATCHES, numIndices,
                                  (void*)(sizeof(unsigned int) * mesh->m_index_offset), 

 still the Shader refuses to execute.


Kind of at my wits end here. :\ does anyone have any ideas?




Share this post

Link to post
Share on other sites
You are not explicitly saying anything about it nor is it contained in the code you posted, so the obvious question first: Are you verifying that compiling all shaders works without errors or significant warnings? Do you check the same for linking? Note that you need to retrieve the compile/link status and info log with the appropriate calls, shader errors like this do not set the usual GL error state.

Share this post

Link to post
Share on other sites

Try updating your GPU drivers. I had a very similar issue with DX11, where the simplest of tessellation setup wouldn't render in my own application. After an update of my GPUs driver, and the required reboot, I finally got an output. So maybe this will help you too (though I don't fully understand why tessellation was working in other applications without the update, duh..)

Share this post

Link to post
Share on other sites
. If I do not attach the Control and Evaluation Shaders, the shader executes just fine. 

Just like this, are you multiplying your vertex position by the MVP (gVP in your case) matrix in your vertex shader ?

Because if it's working fine when you're not attaching them (without modification) then you are surely doing this multiplication in your vertex shader. And if that's the case, just multiply the position by the transfromation(like this in your tutorial WorldPos_CS_in = (gWorld * vec4(Position_VS_in, 1.0)).xyz; ) in the vertex shader, and then by the VP in your evaluation shader. 

Share this post

Link to post
Share on other sites

Thanks everyone.


It turned out the problem went away when I explicitly set the attribute layout locations in all the shaders (tesselation control, tesselation evaluation, and fragment shader). Initially I was only doing this in the vertex shader.


It seems odd to me. I had thought that if your vertex shader sets the position of each attribute, the subsequent shaders would assume the same convention provided they were listed in the same order and had the same type (which was the case).


Should this really have been necessary?

Share this post

Link to post
Share on other sites
Sign in to follow this  

  • Advertisement

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

Participate in the game development conversation and more when you create an account on GameDev.net!

Sign me up!