i'm trying to make a transformation gizmo for my scene editor to be able to easilly transform objects around but i've bumped into a little problem i need a little help with. I've started with the translation gizmo and i'd like that to be a series of three axis aligned billboards. I'd also like the gizmo to be a consistent size no matter distance.

To do this i've created a custom shader and i'm building a custom ortho matrix especially for rendering the gizmo. While there are some problems with positioning the gizmo at the correct position the issue i'd like help with is that i'd like the axis aligned billboards to use perspection so that they'll still show parallel to the selected object's axis. I'm not quite sure how to do this, currently i'm thinking i might need to render with a perspective matrix after all and do some custom scaling.

Here's the rendering code, it also includes the building of the ortho matrix and altered position:

//Create an orthographic projection matrix that can see everything the perspective camera can see. //Since the perspective camera's view is biggest near the far clipping plane we'll have to calculate //the orthographic matrix to include that area. float height = tan( mainCamera->GetFieldOfView() / 2.0f ) * mainCamera->GetFarClipDistance() * 2; float width = height * mainCamera->GetAspectRatio(); EEData::Matrix4x4 ortho = EEData::Matrix4x4::BuildOrthoMatrix( width, height, mainCamera->GetNearClipDistance(), mainCamera->GetFarClipDistance() ); renderer.ApplyMatrix( Renderer::PROJECTION, ortho ); //Since the object we're placing the gizmo on is rendered using a perspective view we'll //need to convert it's origin to a position that results in the same position on screen if //we're using an orthographic camera. This is needed so that the gizmo always stays at //the selection's origin. EEData::Matrix4x4 worldMatrix; //Find the position on screen. Vector3f clipPos = mainCamera->WorldSpaceToClipSpace( currentSelection->GetPosition() ); Logger::Instance().QuickPrint( "x: %f, y: %f, z: %f", clipPos.x, clipPos.y, clipPos.z ); //Fill in the new position using the screen position combined with the camera's coordinate system. Vector3f newPos; newPos += mainCamera->GetOwner()->GetMatrix().GetTangent() * (clipPos.x * width / 2.0f); newPos += mainCamera->GetOwner()->GetMatrix().GetNormal() * (clipPos.y * height / 2.0f); newPos += mainCamera->GetOwner()->GetMatrix().GetBiNormal() * mainCamera->GetOwner()->GetMatrix().GetBiNormal().DotProduct( currentSelection->GetPosition() - mainCamera->GetOwner()->GetPosition() ); worldMatrix.AsTranslation( newPos ); bool oldDepthTest = renderer.SetRenderState( Renderer::ECHORS_ZENABLE, false ); renderer.DrawMesh( transformMesh->GetId(), worldMatrix ); renderer.SetRenderState( Renderer::ECHORS_ZENABLE, oldDepthTest ); //Apply the camera's perspective matrix again for any futher rendering. renderer.ApplyMatrix( Renderer::PROJECTION, mainCamera->GetProjectionMatrix() );

Some extra references for the WorldSpaceToClipSpace function:

/** * Converts a world position to clip space position. * * @return: The clip space position. The left side of the screen is x = -1, the right * side of the screen is x = 1. The top side of the screen is y = 1, the bottom side of * the screen is y = -1; */ Vector3f ComponentCamera::WorldSpaceToClipSpace( const Vector3f& worldPosition ) const { EEData::Matrix4x4 viewProj = viewMatrix * projectionMatrix; Vector4f result = Vector4f( worldPosition.x, worldPosition.y, worldPosition.z, 1.0f ); viewProj.TransformPoint( result ); double rhw = 1.0 / result.w; result = Vector4f( (result.x * rhw), (result.y * rhw), (result.z * rhw), rhw ); return result.XYZ(); } void Matrix4x4::TransformPoint( Vector4f& point ) const { float x = point.x * values[ 0 ] + point.y * values[ 4 ] + point.z * values[ 8 ] + point.w * values[ 12 ]; float y = point.x * values[ 1 ] + point.y * values[ 5 ] + point.z * values[ 9 ] + point.w * values[ 13 ]; float z = point.x * values[ 2 ] + point.y * values[ 6 ] + point.z * values[ 10 ] + point.w * values[ 14 ]; float w = point.x * values[ 3 ] + point.y * values[ 7 ] + point.z * values[ 11 ] + point.w * values[ 15 ]; point.x = x; point.y = y; point.z = z; point.w = w; }

While there's probably still some errors in there i dont think they're affecting the gizmo's shape, are they? I think the error must be somewhere in my shader, so i'll post that aswell. The shader assumes a quite specific vertex format so a quick layout:

gl_Vertex.xy: The AABillboard's size where x indicates width and y indicates length in direction of the aligned axis.

gl_Vertex.z: Color index, used to color the quads. Colors are provided through a uniform array.

gl_Normal: The axis to which this billboard should be aligned. So for example 1.0f, 0.0f, 0.0f should result in a billboard that rotates around the x axis.

Texture coordinates are standard, i'll use these later to put an arrow texture or something on the billboards.

The shader responsible for rendering the gizmo.

Shader GizmoTransform { RenderQueue AlphaOff Properties { Matrix World Matrix ViewInverse Matrix Proj Matrix WorldViewProj Vec3 CameraPosition Vec3Array Colors } Pass p0 { VertexShader { #version 110 uniform mat4 World; uniform mat4 ViewInverse; uniform mat4 WorldViewProj; uniform vec3 CameraPosition; uniform vec3 Colors[ 3 ]; varying vec2 texCoord; varying vec3 color; void main() { vec3 origin; origin.x = World[ 3 ][ 0 ]; origin.y = World[ 3 ][ 1 ]; origin.z = World[ 3 ][ 2 ]; vec3 forward = normalize( origin - CameraPosition ); forward = normalize( vec3( ViewInverse[2][0], ViewInverse[2][1], ViewInverse[2][2] ) ); forward *= (vec3(1.0) - gl_Normal); forward = normalize( forward ); vec3 up = gl_Normal; vec3 right = normalize( cross( up, forward ) ); vec2 direction = gl_Vertex.xy; vec3 vRight = direction.xxx * right; vec3 vUp = direction.yyy * up; vec4 vPos = vec4( vRight + vUp, 1 ); gl_Position = WorldViewProj * vPos; //gl_Position.xyz *= (gl_Position.w); texCoord = vec2( gl_MultiTexCoord0.x, gl_MultiTexCoord0.y ); color = Colors[ int(gl_Vertex.z) ]; } } FragmentShader { #version 110 varying vec2 texCoord; varying vec3 color; void main() { //gl_FragColor = vec4( texCoord.x, texCoord.y, 0, 1 ); gl_FragColor = vec4( color, 1 ); } } } }

The glsl code can be found in the "VertexShader" and "FragmentShader" blocks, the rest is used by the engine and has no effect on the resulting shader whatsoever.

And finally a picture to show what i mean:

Please help me with any issues. Suggestions for a better approach to what i'm trying are welcome aswell.

*Edit: Oh right, i forgot to mention my matrices are row major and are not transposed when set as uniform. This allows me to write matrix * vector rather than vector * matrix