My matrices are row-major and OpenGL wants them as column-major, so that's a transpose call

HLSL matrices are column-major by default. You have to write row_major float4x4 blah; to get a row-major one. GLSL is the same -- there's keywords to force one array storage order or the other, but by default it will choose column major storage order.
Note that row/column major array storage order and row/column major mathematics are two completely different things.

A mathamatician might look at a matrix on paper that looks like:
Xx Xy Xz Xw
Yx Yy Yz Yw
Zx Zy Zz Zw
Wx Wy Wz Ww
And tell you that the basis vectors are stored in the rows of the matrix, therefore it's using the row-major convention...
But a computer scientist will say that this matrix is row-major if it's stored in memory as [Xx Xy Xz Xw Yx Yy Yz Yw Zx Zy Zz Zw Wx Wy Wz Ww] and column-major if it's stored in memory as [Xx Yx Zx Wx Xy Yy Zy Wy Xz Yz Zz Wz Xw Yw Zw Ww] (while the on-paper representation is unchanged).

The column_major / row_major keywords in HLSL (and layout(column_major) / layout(row_major) in GLSL) specify the computer science array ordering convention.
The mathematical convention is determined by whether you write mul( matrix, vector ) or mul( vector, matrix ).

In any case, there's no reason to do anything differently between GL and D3D here, both of them support both of the comp-sci-majorness conventions and both of the mathematical-majorness conventions, so you can/should use the same conventions everywhere.

The NDC z-axis goes from 0->1 in DX and -1->1 in OpenGL, so I have to take that into account, but that's just using a different projection matrix, right?

Correct - typically you'll construct your projection matrices differently for GL.

Alternatively you can use glClipControl to use D3D's conventions in GL.

Before I started I want to get a better picture of what "DX is left-handed, OpenGL is right-handed" means in the modern APIs.

In the context of the modern versions of the APIs it means absolutely nothing since nothing in either API enforces either handedness on you.

The matrix library you use may enforce a handedness on you, but any good matrix library should be capable of supporting either. So far as the API is concerned it's just a matrix * vector (or vector * matrix) operation and the mathematics are exactly the same irrespective.

However, by convention, OpenGL sample code may still assume a right-handed co-ordinate system whereas D3D sample code may still assume left-handed. But this is just a convention based on historical usage. So if you're using any sample code you will need to be aware of it's handedness, and unless explicitly stated otherwise you can use this convention as a rule of thumb to guess which handedness it uses.

GL_ARB_clip_control exists for NDC conventions. It doesn't affect viewports, scissor rects, etc, however, so that is something you need to watch out for. There is a rationale given for this in the extension spec, and I'm sure that the reasons were important to someone somewhere sometime, but IMO it's Not Good Enough. Either do it fully and consistently across the whole API, or don't do it at all, whichever, but don't half-ass it so that sometimes we have to use one convention and sometimes the other. Rant over.

For texture coords it's not actually that bad. Because you also load textures from bottom-left in GL, the two bottom-lefts cancel each other out and you can use the same texture coords and image data in GL as in D3D, unmodified. Where you may need to adjust is texture coords for render targets (or any other similar source).

OK, this might be complicated.

glTexImage2D (other texture specification calls work the same) specifies the following with respect to it's data pointer:

The first element corresponds to the lower left corner of the texture image. Subsequent elements progress left-to-right through the remaining texels in the lowest row of the texture image, and then in successively higher rows of the texture image. The final element corresponds to the upper right corner of the texture image.

So when you provide an array of data to glTexImage2D, data[0] is not actually the top-left of the texture image, it's the bottom-left of it.

So an image loaded through glTexImage2D will be upside-down by comparison to what you might expect.

However, OpenGL texture coords also work from bottom-left; i.e. {0,0} is the bottom-left corner of the texture. So this effectively undoes the upside-down load and everything comes out the same.

What this means in practical terms.

You need to change nothing in your engine when dealing with textures loaded from system memory pointers. You give the same data to glTexImage2D as you would give to your D3D11_SUBRESOURCE_DATA::pSysMem pointer, you use the same texture coordinates when drawing, and everything just works.

Thanks for the explanation :)

However, after reading your and Sean's post I have seen several posts on the internet about this particular issue and many suggestions on "how to solve it", eg. flip the image in memory before passing to OpenGL, flip it beforehand as part of the asset pipeline etc. If the texture coordinates are flipped too I don't see why people would be upset about it. Is there something I am missing here?

You'll find that the people flipping the image are also flipping the texture coords.

In other words, they're looking at the two differences in isolation from each other rather than putting them together and realising that they cancel out.