So a little background:

I've been working on this game engine for a few years and I recently committed a complete overhaul of some of the game systems, and editor, into the repo. You can check it out at https://github.com/gitbetter/Zenith​

I've come across an issue whereby a texture rendered onto a full-screen quad seems to warp down the diagonal. I've posted a gif demostrating the issue below. This seems related to how OpenGl (and most graphics APIs) break polygons down into triangles, but I've tried drawing the quad using GL_TRIANGLE_STRIP and GL_TRIANGLE_FAN and I still get the same effect.

At this point the engine is a bit too expansive and modularized to single out any part of the rendering code to post here for inspection, but suffice it to say I am drawing the scene, using several render passes, onto a separate target framebuffer, which gets pulled in by the editor and drawn onto a resized UI quad.

Initially I thought this might be due to some sort of buffer swap issue, but I figured that wouldn't matter (I think) since I am not drawing to the screen but a separate framebuffer instead, and buffer swaps are being done correctly after all draw calls have been issued. When blitting the scene buffer onto the default framebuffer (the current window's framebuffer) it looks fine. This only happens when drawing the scene texture onto a screen quad.

Any help here would be appreciated, if anyone's seen this issue before.

@fleabay Thanks for that, I'll definitely look into this for full screen framebuffer operations once I get my head around it. The strange thing is that this issue isn't present when blitting the framebuffer directly to the default framebuffer as mentioned, even though I'm also rendering into a quad (with GL_TRIANGLE_STRIP) there as usual.

And yeah haha put that license up when I started due to old plans but I keep forgetting to change it to an MIT license. Less restrictive.

ED: Done deal

@fleabay Yeah wouldn't make much sense turning off interpolation for the quad, otherwise how would the shader know what texture point to sample for the middle of a triangle for example?

Also tried switching to a full screen triangle instead of a quad for outputting the scene but this doesn't work. The problem seems to be in the UI element container for the scene (necessarily a quad since otherwise I would get a full blown triangle every time I want to render a rectangular UI element), although the full screen triangle vs. full screen quad is definitely a performance plus, after reading about it, so thanks for that bit of intel.

My next guess is that there's some sort of numerical impression causing the scene container to be a not-so-exact rectangle or some weird affine transformation somewhere that's causing the artifact to appear, but as I understand it OpenGL/GLSL should do perspective correct interpolation regardless.

So I took a look at my ui vertex shader

#version 450 core

layout (location = 0) in vec2 vertex;
layout (location = 1) in vec2 uv;
layout (location = 2) in mat4 instanceM;

out VS_OUT {
 vec4 FragPos;
 vec2 FragUV;
} vs_out;

uniform mat4 M;
uniform mat4 P;
uniform bool instanced = false;

void main()
{
   mat4 m = M;
   if (instanced) {
       m = instanceM;
   }
   vs_out.FragPos = m * vec4(vertex.xy, 0.0, 1.0);
   vs_out.FragUV = uv;
   gl_Position = normalize(P * vs_out.FragPos);
}

and noticed that I was normalizing the gl_Position output. After changing that last line to

gl_Position = P * vs_out.FragPos;

the artifact went away. I figured normalizing the output would somehow mess with the homogenous w coordinate and in turn mess with the perspective projection of the quad somehow.

@fleabay I appreciate the help. Wouldn't have thought to look if you didn't mention the whole vertex color interpolation thing.