3 hours ago, Zakwayda said:

if you submit the same input values to orthoRH() and orthoLH() and look at the output matrices, you'll see that a couple elements are negatives of each other.-

I have just tested this based on NDC coordinates (-1.0, 1.0, -1.0, 1.0, -1.0, 1.0).

A call to glm::ortho(left, right, bottom, top, near, far) produces the matrix:

[    1    0    0    0    ]

[    0    1    0    0    ]

[    0    0   -1    0    ]

[    0    0    0    1    ]

A call to glm::orthoLH(left, right, bottom, top, near, far) produces the matrix:

[    1    0    0    0    ]

[    0    1    0    0    ]

[    0    0    1    0    ]

[    0    0    0    1    ]

I would guess that the difference between the z ([2][2] index) in the two matrices would cause the effect that you were referring to when you said - "the RH version has the effect of negating coordinates along the z axis to get them into the left-handed space that OpenGL expects". 

3 hours ago, Zakwayda said:

if you look at the code for these two functions you should see this reflected in the code as well.

To be honest, the functions are kind of hard for me to interpret, I can see that there are differences, but I don't really understand what 'NO' and 'ZO' refer to and I'm not sure which one is actually being called after the calls to glm::ortho() and glm::orthoLH(). Here are the function definitions:

  template<typename T>
  GLM_FUNC_QUALIFIER mat<4, 4, T, defaultp> ortho(T left, T right, T bottom, T top)
  {
    mat<4, 4, T, defaultp> Result(static_cast<T>(1));
    Result[0][0] = static_cast<T>(2) / (right - left);
    Result[1][1] = static_cast<T>(2) / (top - bottom);
    Result[2][2] = - static_cast<T>(1);
    Result[3][0] = - (right + left) / (right - left);
    Result[3][1] = - (top + bottom) / (top - bottom);
    return Result;
  }

  template<typename T>
  GLM_FUNC_QUALIFIER mat<4, 4, T, defaultp> orthoLH_ZO(T left, T right, T bottom, T top, T zNear, T zFar)
  {
    mat<4, 4, T, defaultp> Result(1);
    Result[0][0] = static_cast<T>(2) / (right - left);
    Result[1][1] = static_cast<T>(2) / (top - bottom);
    Result[2][2] = static_cast<T>(1) / (zFar - zNear);
    Result[3][0] = - (right + left) / (right - left);
    Result[3][1] = - (top + bottom) / (top - bottom);
    Result[3][2] = - zNear / (zFar - zNear);
    return Result;
  }

  template<typename T>
  GLM_FUNC_QUALIFIER mat<4, 4, T, defaultp> orthoLH_NO(T left, T right, T bottom, T top, T zNear, T zFar)
  {
    mat<4, 4, T, defaultp> Result(1);
    Result[0][0] = static_cast<T>(2) / (right - left);
    Result[1][1] = static_cast<T>(2) / (top - bottom);
    Result[2][2] = static_cast<T>(2) / (zFar - zNear);
    Result[3][0] = - (right + left) / (right - left);
    Result[3][1] = - (top + bottom) / (top - bottom);
    Result[3][2] = - (zFar + zNear) / (zFar - zNear);
    return Result;
  }

  template<typename T>
  GLM_FUNC_QUALIFIER mat<4, 4, T, defaultp> orthoRH_ZO(T left, T right, T bottom, T top, T zNear, T zFar)
  {
    mat<4, 4, T, defaultp> Result(1);
    Result[0][0] = static_cast<T>(2) / (right - left);
    Result[1][1] = static_cast<T>(2) / (top - bottom);
    Result[2][2] = - static_cast<T>(1) / (zFar - zNear);
    Result[3][0] = - (right + left) / (right - left);
    Result[3][1] = - (top + bottom) / (top - bottom);
    Result[3][2] = - zNear / (zFar - zNear);
    return Result;
  }

  template<typename T>
  GLM_FUNC_QUALIFIER mat<4, 4, T, defaultp> orthoRH_NO(T left, T right, T bottom, T top, T zNear, T zFar)
  {
    mat<4, 4, T, defaultp> Result(1);
    Result[0][0] = static_cast<T>(2) / (right - left);
    Result[1][1] = static_cast<T>(2) / (top - bottom);
    Result[2][2] = - static_cast<T>(2) / (zFar - zNear);
    Result[3][0] = - (right + left) / (right - left);
    Result[3][1] = - (top + bottom) / (top - bottom);
    Result[3][2] = - (zFar + zNear) / (zFar - zNear);
    return Result;
  }

  template<typename T>
  GLM_FUNC_QUALIFIER mat<4, 4, T, defaultp> orthoZO(T left, T right, T bottom, T top, T zNear, T zFar)
  {
    if(GLM_CONFIG_CLIP_CONTROL & GLM_CLIP_CONTROL_LH_BIT)
    	return orthoLH_ZO(left, right, bottom, top, zNear, zFar);
    else
    	return orthoRH_ZO(left, right, bottom, top, zNear, zFar);
  }

  template<typename T>
  GLM_FUNC_QUALIFIER mat<4, 4, T, defaultp> orthoNO(T left, T right, T bottom, T top, T zNear, T zFar)
  {
    if(GLM_CONFIG_CLIP_CONTROL & GLM_CLIP_CONTROL_LH_BIT)
    	return orthoLH_NO(left, right, bottom, top, zNear, zFar);
    else
    	return orthoRH_NO(left, right, bottom, top, zNear, zFar);
  }

  template<typename T>
  GLM_FUNC_QUALIFIER mat<4, 4, T, defaultp> orthoLH(T left, T right, T bottom, T top, T zNear, T zFar)
  {
    if(GLM_CONFIG_CLIP_CONTROL & GLM_CLIP_CONTROL_ZO_BIT)
   	return orthoLH_ZO(left, right, bottom, top, zNear, zFar);
    else
    	return orthoLH_NO(left, right, bottom, top, zNear, zFar);
  }

  template<typename T>
  GLM_FUNC_QUALIFIER mat<4, 4, T, defaultp> orthoRH(T left, T right, T bottom, T top, T zNear, T zFar)
  {
    if(GLM_CONFIG_CLIP_CONTROL & GLM_CLIP_CONTROL_ZO_BIT)
    	return orthoRH_ZO(left, right, bottom, top, zNear, zFar);
    else
    	return orthoRH_NO(left, right, bottom, top, zNear, zFar);
  }

  template<typename T>
  GLM_FUNC_QUALIFIER mat<4, 4, T, defaultp> ortho(T left, T right, T bottom, T top, T zNear, T zFar)
  {
    if(GLM_CONFIG_CLIP_CONTROL == GLM_CLIP_CONTROL_LH_ZO)
    	return orthoLH_ZO(left, right, bottom, top, zNear, zFar);
    else if(GLM_CONFIG_CLIP_CONTROL == GLM_CLIP_CONTROL_LH_NO)
    	return orthoLH_NO(left, right, bottom, top, zNear, zFar);
    else if(GLM_CONFIG_CLIP_CONTROL == GLM_CLIP_CONTROL_RH_ZO)
    	return orthoRH_ZO(left, right, bottom, top, zNear, zFar);
    else if(GLM_CONFIG_CLIP_CONTROL == GLM_CLIP_CONTROL_RH_NO)
    	return orthoRH_NO(left, right, bottom, top, zNear, zFar);
  }

I'd want to look more closely before claiming certainty, but I think 'NO' and 'ZO' refer to NDC z range ([ -1 1 ] or [ 0 1 ]). I'm not sure what the initialisms 'NO' and 'ZO' stand for specifically though (maybe it's obvious and I'm just missing it).

As for the signs, you can see, for example, that the sign differs for element 22 between the LH and RH versions (which you may have already noticed).

4 hours ago, Zakwayda said:

I'd want to look more closely before claiming certainty, but I think 'NO' and 'ZO' refer to NDC z range ([ -1 1 ] or [ 0 1 ]). I'm not sure what the initialisms 'NO' and 'ZO' stand for specifically though (maybe it's obvious and I'm just missing it).

If that's the case then the acronyms could stand for 'zero-one' and 'negative-one'.

I will dig deeper into the matrices tomorrow but thanks a lot for all your help on this, I appreciate it.

On 5/1/2019 at 4:29 PM, Zakwayda said:

Also, although the claim is commonly made, I'd disagree with the claim made in the video that OpenGL is right-handed and DirectX is left-handed. Even if that was true at some point in the past, it hasn't been true for a very long time.

From https://stackoverflow.com/questions/4124041/is-opengl-coordinate-system-left-handed-or-right-handed:

Quote

“OpenGL is right handed in object space and world space. But in window space (aka screen space) we are suddenly left handed.

Because the z multiplier is (-2/(far-near)), the minus sign effectively scales z by -1. This means that [ if the near value passed to the projection matrix is less its counterpart far value ] "z" is turned left handed during the viewing transformation, unbeknownst to most people as they simply work in OpenGL as a "right handed" coordinate system.” *

The above quote seems to be compatible with the claim made in the video; that OpenGL is right handed until it comes to projection, where a left handed system is used for the purpose of the depth test. 

If I submit values which are consistent with a right handed coordinate system to the glm::ortho() function (+z as the near and -z as the far rather than vice versa):

float left   { -1.0F };
float right  {  1.0F };
float bottom { -1.0F };
float top    {  1.0F };
float near   {  1.0F };
float far    { -1.0F };

glm::ortho(left, right, bottom, top, near, far);

The result is an identity matrix, meaning that when the (projection * view * model) MVP matrix is calculated and multiplied by the following vertex positions: 

std::array<float, 12> RedSquareVertices
=	{
	 0.25F,-0.50F, 0.99F,
	 0.25F, 0.25F, 0.99F,
	-0.50F, 0.25F, 0.99F,
	-0.50F,-0.50F, 0.99F
	};

std::array<float, 12> BlueSquareVertices
=	{
	 0.50F,-0.25F,-0.99F,
	 0.50F, 0.50F,-0.99F,
	-0.25F, 0.50F,-0.99F,
	-0.25F,-0.25F,-0.99F
	};

The resulting x,y,z coordinates will be consistent with a right handed coordinate system. 

Furthermore, if I enable the depth test, the square made up of BlueSquareVertices renders in front of the square made up of RedSquareVertices which suggests that the quoted claim is at least partially correct, as despite the fact the projection matrix is based on a right handed coordinate system (the near value is greater than the far value), the square made up of lesser z values is on top of the square made up of higher z values, because the depth test is set to GL_LESS (and is therefore using a left handed coordinate system to determine the on screen position of the z coordinate).

Does this sound correct or am I missing something here?

EDIT: This is not to say that OpenGL's NDC coordinates are right handed, they are not. But the glm projection matrix expects a right handed coordinate system (unless you call the specific LH version), if you pass in near and far values based on left handed coordinates, the z coordinate will be scaled by -1, and the depth test will then work the opposite way around if the depth function is set to GL_LESS (z values are inverted, which will have the same effect as if the projection matrix was made up of right handed coordinates and the depth function was set to GL_GREATER). 

* Square bracket section inserted by me

Quote

The above quote seems to be compatible with the claim made in the video; that OpenGL is right handed until it comes to projection, where a left handed system is used for the purpose of the depth test.

The first thing I'd say is that the misconception that OpenGL is right handed is widespread, so finding that claim repeated online (such as in the thread you linked) is in no way surprising.

Also, if you read the entire thread, you'll see that several people correct the misconception in the thread itself, pointing out that OpenGL isn't inherently right handed and that the handedness of e.g. world and view space is up to you.

Regarding the statement you quoted:

Quote

"OpenGL is right handed in object space and world space."

Notice that the poster refers specifically to the legacy convenience functions glOrtho() and glFrustum(), and not to the programmable pipeline or to the legacy functions that allow you to specify matrices directly. Those legacy convenience functions [edit: meaning glOrtho() and glFrustum()] are in fact right handed, which may be why the poster claims OpenGL is right handed. However, even in 2012 when the answer was written this claim was incorrect, as the programmable pipeline was available then, and even with the fixed-function pipeline you've been able to specify transform matrices directly since very early in OpenGL's version history.

If you still feel OpenGL might be right handed, I'd simply ask: what makes it so? (I think if you dig into it, you'll find that the answer is 'nothing'.)

As for the issues with orthographic projection, where you're submitting near = 1 and far = -1 to effect a right-handed space, I think you may just be using the ortho function incorrectly there. Interestingly, this issue is actually addressed in the very post from the SO thread that you quoted, where the poster points out that the near and far values are specified as distances, and not as coordinates along the z axis. So unless I'm mistaken, even in right-handed space you'd submit near = -1 and far = 1, and not the reverse of that.

There's some further evidence here:

https://www.khronos.org/registry/OpenGL-Refpages/gl2.1/xhtml/glOrtho.xml

Where it says these values are distances and will be "negative if the plane is to be behind the viewer". If they were specified as z coordinates, they'd be positive if behind the viewer (given that glOrtho() is right handed). So I think you'd want to submit e.g. near = -1 and far = 1 irrespective of whether you're using the RH or LH version of the orthographic projection function.

This has gotten long, so I'll wrap it up. In closing though, this can definitely be confusing, and it's not helped by the significant amount of misinformation floating around online. There are many places where you can 'flip' things in the pipeline, including using RH or LH projections, deliberately inverting the near and far values (as you were doing), changing the depth range, or using different comparators for the depth test. Certain combinations of these can cancel each other out, leading to further confusion.

Lastly, I'll say that since you seem to want a left-handed system (based on earlier statements in the thread), I think you'll want to use LH versions of the projection functions, with far > near (the latter should be the case with RH as well). For the depth test you can switch things around if you want, but a depth range of [ 0 1 ] and < or <= for the depth test should give you the desired results (as I think you already know).

[Edit: Minor changes for clarity.]

28 minutes ago, Zakwayda said:

Interestingly, this issue is actually addressed in the very post from the SO thread that you quoted, where the poster points out that the near and far values are specified as distances, and not as coordinates along the z axis. So unless I'm mistaken, even in right-handed space you'd submit near = -1 and far = 1, and not the reverse of that.

The concept of 'near and far as distances' has just added another layer of confusion to the whole thing for me.

Distances to the near and far clipping planes from where? I thought that the projection matrix just represents the in screen space, if so how can one of the clipping planes be 'behind the viewer'? The screen is always in front of the viewers face so how can a clipping plane somehow be behind? 

I'm probably thinking of this in totally the wrong way but to be honest the whole thing is almost overwhelmingly confusing to me. 

I appreciate that this thread has gotten long, and you have already spent more than enough time trying to help me so feel free to just ignore this question if it has become exhausting.

Quote

I appreciate that this thread has gotten long, and you have already spent more than enough time trying to help me so feel free to just ignore this question if it has become exhausting.

No worries I don't think there's any need to hesitate when asking follow-up questions. Even if I'm not able to clear things up, someone else might jump in.

I don't know if this will be reassuring, but I agree that this can be confusing (as evidenced, perhaps, by the fact that even though I feel I understand it myself, I'm having some difficulty figuring out how to explain it concisely and clearly).

As I think you're probably already aware, the projection transform basically maps coordinates from an arbitrary axis-aligned box (in either a left- or right-handed coordinate system, based on your preference) to the 'NDC box' (which is left handed). The parameters left, right, bottom, top, near, and far describe the bounds of the 'projection box'.

That you specify 'distances' rather than absolute coordinates along the z axis is, I think, just a convention that's happened to prevail. If you wanted to, you could implement a function that built an orthographic projection matrix such that near and far (which might be more appropriately named something like back and front in this case) specified actual coordinates along the z axis.

I think what's meant by 'distance' in the conventional orthographic projection algorithm is distance from z = 0, assuming that positive (signed) distances are in the direction the viewer is looking, even if that's along the -z axis, as is the case in a right-handed system. It does seem a bit confusing (perhaps unnecessarily so, now that you've got me thinking about it).

In any case, if you want to use a left-handed system (as seems to be the case), you don't need to worry about the difference, as the signs of the distances and the absolute coordinates are the same.

I fear I may have made things more confusing, not less, but I'm just going to post this and hope for the best

Everything you have said has been really helpful, I just think I need to spend more time learning about perspective projection as this feels like a different ball game to orthographic projection.

One last question about perspective projection; is it possible for the camera/viewpoint to be in between the near and far planes or is this illogical? To me it seems illogical but in the playlist I am watching on this subject, there seem to be a lot of examples where this happens. 

Maybe I am getting confused between the all the different coordinate systems and the directional system used on the z axis. 

It is very frustrating as a beginner, things seem to be needlessly over complicated (and although I'm sure there are good reasons for this, it is still intimidating at first!).

Quote

One last question about perspective projection; is it possible for the camera/viewpoint to be in between the near and far planes or is this illogical? To me it seems illogical but in the playlist I am watching on this subject, there seem to be a lot of examples where this happens.

Regarding your seeing examples of the camera position being between the near and far planes of a perspective projection, we might need to see specific examples to understand to what you're referring. (Absent a specific example, I'm not sure I can comment on that effectively.)

Okay fair enough, here is where I have seen this:

After about ten views I'm still finding it really difficult to understand what is going on in this video. The specific bit I'm referring to is from about 10:20 but you would probably need to watch from that time stamp to understand his intentions (don't worry if you don't have the time, the section is just under six minutes long).

Also the bit at around 7:18 where he says that -1 is "behind us". 

Thanks again.