I'm trying to create a GUI system inspired by Apples UIView architecture. For those unfamiliar with it, a "UIView" is Apples version of a widget. Each widget can be transformed with a 4x4 matrix. Its transformation effects its children, its children's children, and so on down the hierarchy.

The problem I'm running into is a perspective problem: Each widget must transform with a vanishing point located at its center. The following two gifs show what I mean.
[table]
[tr]
[th]Incorrect[/th]
[th]Correct[/th]
[/tr]
[tr]
[td]The buttons in this gif use the center of the screen as their vanishing point.[/td]
[td]The buttons in this gif use their own center as the vanishing point (this is the expected behavior).[/td]
[/tr]
[tr]
[td][attachment=36013:buttons_bad.gif][/td]
[td][attachment=36015:buttons_good.gif][/td]
[/tr]
[/table]

Here is how I'm moving the vanishing point now:

• Start with an upright button whose vertices are centered around the origin.
• Apply a rotation matrix to them.
• Apply a translation matrix to move them forward so they are centered in front of the eye.
• Apply a perspective matrix to transform them to clipspace.
• Translate the X,Y clipspace coordinates moving the button to the lower-right corner of the dialog box.

The problem with this solution is it doesn't account for the parents transformation. In my gif examples the dialog box is the parent of the buttons.

[table]
[tr]
[td]The following gif shows the expected behavior of a rotation applied exclusively to the dialog box.[/td]
[td]Now imagine the dialog rotating, but this time with the buttons transformed and in the perspective depicted below. That's the behavior I need.[/td]
[/tr]
[tr]
[td][attachment=36014:dialog_good.gif][/td]
[td][attachment=36016:rotation.png][/td]
[/tr]
[/table]

5. Translate the X,Y clipspace coordinates moving the button to the lower-right corner of the dialog box.

The problem with this solution is it doesn't account for the parents transformation. In my gif examples the dialog box is the parent of the buttons.

Step 5 looks wrong to me.

Conceptually, you rotate and transform the button, and then apply the rotate and transform of whatever it is attached to (ie its parent).

The "Translate the X,Y clipspace coordinates" is only translation to the right position of a fixed parent, but you need to do all transformations, ie both translation and rotation of the parent, if the widget is to stay at the same point of the parent.

Step 5 looks wrong to me.

Conceptually, you rotate and transform the button, and then apply the rotate and transform of whatever it is attached to (ie its parent).

The "Translate the X,Y clipspace coordinates" is only translation to the right position of a fixed parent, but you need to do all transformations, ie both translation and rotation of the parent, if the widget is to stay at the same point of the parent.

Yup, your exactly right. The problem is...if I omit step #5 and apply the parents transform, then the button will not have the vanishing point that I need.

The gif below shows how the buttons would look if I omit step #5 and apply the parents transform. Notice how the buttons vanishing point is the dead center of the screen? That's not what I want. I want each buttons vanishing point to be their dead center for their local rotation. The 2nd gif shows how the buttons should look.
[attachment=36034:buttons_bad.gif]
I can get the correct vanishing point by performing step #5, but then I don't get the parents transform. That's what the following gif shows: The buttons with the correct perspective / vanishing point, but they do not account for the parents transform at all.
[attachment=36035:buttons_good.gif]
I need a solution that lets me achieve both: Account for the parents transform and keep the vanishing point at the buttons dead center. Right now I'm only able to achieve one or the other.

So, for clarification, the buttons need a vanishing point at their dead center for their local rotation. When applying the parent transform, the vanishing point should move to the parents dead center. And so on up the hierarchy. The problem is a standard perspective matrix applies the same vanishing point to all geometry. Its almost like the perspective somehow needs to be intermixed during the model matrix transforms.

It's a 3-step process.

0. Define button area in its own button space.

1. Rotate and translate (ie transform) button to anything you like. That gives the button you see without considering its surroundings.

2. Translate the button to the right position relative to the dialog origin.

Now, the dialog moves and rotates, so everything on it, including the button positions, move/rotate along with it. In other words, apply the dialog rotation and translation also to the transformed button.

The trick here is to think in layers, there is a button layer, a dialog layer, and a screen layer, each with its own translation and rotation relative to the next layer. The dialog layer is rotated and translated relative to the screen, and the button is translated and rotated relative to the dialog. To get the button where it belongs you work your way through all the layers ( button -> dialog -> screen ), and only then you have the position for display.

1. From the button model (a plain rectangle), you apply button rotation and translation to get a rotated button

2. To get the rotated button at the right spot in the dialog, you apply dialog rotation and translation to the rotated button.

Let's do this on 2 cubes stacked onto each other. Top cube rotates relative to the bottom cube (like your widget rotates relative to the dialog), and the bottom cube moves (like your dialog moves (actually also rotates, but I use a different word for clarity)).

You start with the unrotated top-cube, and apply rotation on it at current point in time. That gives you the rotated top cube, under the assumption that the bottom cube doesn't move. However, it does. To fix that, you apply the same movement to the rotated top-cube as well. That gives you a rotated, moved top-cube, that you can draw.

It's a 3-step process...

The first gif of my previous post is already doing exactly as you describe. I believe I've been incorrectly using the term "vanishing point" which is were the confusion may lie.

When a widget renders on screen it needs to appear as though the eye was looking at its dead center. When the parent transform is applied the perspective needs to be from an eye looking at said parents dead center. And so on up the hierarchy.

Take a look at the following two pictures. The red line shows the vanishing point (I'm using the correct usage of the term now). Notice how in the first picture the vanishing points of both buttons meet at the same point. That not what I want. I want the behavior in the 2nd screenshot. Each widget needs to render as if the eye was looking at its dead center.

[attachment=36036:vanishing_point.png]

So to be absolutely clear: When the cancel button has its local transform applied, its pixels should be rendered from the perspective of an eye directly over its center. The same is true for the OK button. When the dialog transform is applied to both buttons, its pixels must be rendered from the perspective of an eye at the dead center of the dialog. This behavior should continue up the hierarchy.

That post helps a lot.

As you can see in your own picture, each button is at the same position relative to its vanishing point. You have to translate each button to that position, transform, and translate back. This is needed, because scaling works on a straight line from origin to your button points. If the button is at a different position, the line from the origin changes, and thus the scaling result changes.

The above (translate, scale, translate back) is not is a linear transformation, so you cannot express that in a single 4x4 matrix.

The above (translate, scale, translate back) is not is a linear transformation, so you cannot express that in a single 4x4 matrix.

That's what I was afraid of :( It sounds like I may need a different solution since I cannot express what I need with 4x4 matrices.

My other idea involves using render targets for each widget. Basically I would do the following:

1. Start with the leaf widgets in the hierarchy and render them to a texture. So in my example I would render the Cancel and OK buttons to their own separate textures. When captured to a texture they must be positioned so the eye is looking directly at them.
2. The dialog, along with its buttons, must be rendered to a texture.
3. To render the dialog, first transform the buttons relative to it and render them using their render target captured in step #1.
4. If the dialog itself needs to be transformed then apply said transformations to it and its children.
5. This process needs to continue up the hierarchy until your done.

This would mean every widget would need its own render target. This would solve the problem, but I wonder if there is another way. My experience with render targets is that they are rather slow and there could be many widgets on screen.

Another problem is clipping: If the render target is not large enough, then it may not capture part of a really wide/tall widget. That opens up the possibility of the widget looking cut-off if its positioned somewhere to show a part of it that the render target did not capture.

Sorry, no idea, perhaps someone else can answer that.