• Advertisement

Recommended Posts

Since I've finished school and my previous game project, I've had a bit of down time before I start work (not game-dev related, unfortunately). I've decided to try my hand at writing something I've always been a little bit interested in: UI frameworks. Ideally I'd want something that offers a similar user experience to an immediate-mode framework, but with better support for things like styling and animation (and isn't horribly ugly). If completed, I'd want to be able to use it for tools, as well as in-game UI. Anyway, I've got a few loose ideas floating around in my head, so I thought I'd throw them out here to see if anyone had any insight. I have a decent amount of experience using a variety of UI frameworks in the past, but I've never actually implemented one before so if any of this sounds incorrect it's probably because I'm thinking about it from the wrong angle.

 

My first idea for this was to just use the same sorts of controls (and a similar layout algorithm) to that offered by WPF, but I'm a bit concerned that regenerating the entire UI every frame with that sort of system would be too inefficient. It requires two passes over every element, first to compute required width/height (bottom-up) and second to arrange children (top-down). This is the same design in use by Unreal for its editor. I began working on a simple implementation, but one snag I ran into was in the case of something like a WrapPanel. Say you have the following scenario:

  • For each column (assuming you're creating a vertical wrap panel), the amount of horizontal space allocated to each element in each column is determined by the amount of horizontal space required by the widest element in that column.
  • If the WrapPanel runs out of vertical space, a new column is created (regardless of remaining horizontal space).
  • Images compute required size based on the width and height of the image, but may scale up assuming they remain proportional. (So allocating more horizontal space also requires more vertical space).
  • In a given wrap panel you have some narrow elements, an image, and a wide element.
  • Based on individually computed height requirements, it is determined that all elements can fit in one column, with the wide element being the widest.
  • By allocating more horizontal space to the image, it now requires more vertical space and the wide element can no longer fit in the column, creating a new one.
    • Now the element that set the width of the column no longer exists in the column, which is weird.

I can think of a few solutions to this problem:

  • Only allocate height in a vertical wrap panel based on what the element computed. This would effectively disallow elements that must stretch proportionally from stretching at all. This seems like the most sensible solution, but I don't believe this is what WPF is doing (playing around with it, I'm really not sure at all what WPF is doing).
  • Place the wide element in a new column, and recompute layout for elements in the first column with the width of the next widest element (inefficient).
  • Place the wide element in a new column, and keep the elements in the first column as they are (might cause some elements to be clipped).
  • Keep all elements in the same column (will definitely cause some elements to be clipped).
  • Just force WrapPanels to have uniformly sized elements. I can't think of any use cases off the top of my head where this would be a significant issue.

 

On the complete opposite end of the complexity spectrum, one UI layout solution would be to simply disallow elements from computing width/height based on contents. Elements may stretch to fill their container, but UI is laid out in a single top-to-bottom pass where allocated width and height for each element are known up front. This may be acceptable for in-game UIs (which are typically very restrained in their complexity), but this breaks completely when faced with lists of non-uniformly sized elements. This could be loosened to a degree by requiring only one axis to be fixed (ie, allocated width is known but height may be computed, or allocated height is known but width may be computed), but this would face problems if you have say a horizontal list (where height allocated for each element is known, but not width) which contains a vertical list (where width allocated for each element is supposed to be known, but not height).

Even if that issue were resolved somehow, I imagine this would create quite a burden on the UI programmer, as they would have to perform a lot of guessing and checking on the requirements for certain elements, which in some cases may not even be possible.

 

One loosely formed idea I had was to use the layout to generate a linear set of instructions for computing coordinates/dimensions, ordered by the dependencies elements had on one another. So auto-sized elements would generate instructions to have the dimensions of their contained elements computed first, then compute their own coordinates/dimensions, then compute the coordinates of their contained elements. Fixed-size elements would simply compute their contained elements coordinates and dimensions directly. This is very similar to the first approach (and I imagine it could suffer from similar or even worse complexity issues), with the exception that the generated instructions wouldn't distinguish between any sort of bottom-up or top-down pass, which could potentially reduce the amount of redundant work. I haven't thought this idea through very thoroughly, however, so I could be wrong.

 

Anyway, I'd love to get a second opinion on these ideas, especially when it comes to performance. I wouldn't be opposed to introducing some burden on the application to specify which elements have become invalid in between UI iterations if necessary, but ideally those would be very coarse-grain at the least.

Edited by Salty Boyscouts

Share this post


Link to post
Share on other sites
Advertisement

When you say "immediate-mode" I immediately think of Unity's "OnGUI" approach.  It's simple for doing a few buttons, but gets massively slower the more you do - this is due to the OnGUI function hierarchy being called multiple times in different 'states' - layout, input handling, rendering, etc.

I *much* prefer OOP retained-mode UIs - WinForms, WPF, NGUI and UnityEngine.UI for example.  You construct an object hierarchy and separate methods on those objects can handle input, rendering, layout.  You can split things up by component like Unity does, keep objects monolithic like WinForms does, use visitor patterns, or however you feel like doing it.

As far as your layout concern goes, what I usually see is the layout is only recalculated if something changes - when a control is resized, it propagates the layout change up and down the hierarchy.  On frames where the layout is not changing, no layout code is executed at all and rendering simply uses the cached layout rects of each control from the last time layout was recalculated.

Edited by Nypyren

Share this post


Link to post
Share on other sites

I implemented a 2-pass widget layout system in OpenTTD, where the problem was that different languages differ in length for each string, so anything with fixed size will fail somewhere.

The basic idea is that all leafs compute their own minimal size (the size required to display its contents). This is propagated up from the leafs to the root. The intermediate nodes order child nodes in horizontal or vertical direction, and compute minimal size for the composition of their children. At the root you then know the minimal size of the window, which is then also the smallest size of the window. This size is then pushed down again, where all child nodes are adjusted to their smallest size (which may be larger than minimal size). When you reach the leafs, all nodes know their smallest size. This is the smallest size where all widgets are properly sized to cover the entire window.

The reality is a bit more complicated, because you also want to control where the extra space goes, and in how large quantities. Eg labels may get wider but not higher. When you resize the window you get similar problems, list elements must become larger in eg steps of 12 pixels (height of a text element, for example).

As it's all objects, widgets and windows have hooks to override calculations.

 

I think it does have a sort-of dynamic sized thing in a text-window, but it's not really dynamic like you discuss. It makes an estimate of the required size of the surface, and then applies the golden ratio to derive a width and, then computes required height by fitting lines of text until done. At other place there is really dynamic amount of text, which gets solved by rendering the content, and if the bottom is below the bottom of its widget, resize that window (if there is a simple window-size vs widget size relation), or resize the widget, and run the tree calculation again. It's expensive, but as the widget/window can only grow, it will terminate quickly. Finally, mark the window "needs a repaint" so it gets painted again in the next frame, and the user never sees it.

One of the nice things was that 0 width or height was a valid value, which means you can just hide parts of a window. If you hide some parts and show other parts, it appears to change the layout dynamically, which is nice :)

 

One error I made was that I didn't have a grid layout thing, so anything grid-ish was a nightmare where you had to do a lot of computing to do for several widgets to establish a common width or height. I fixed that in another project, with the added benefit that I didn't need horizontal or vertical containers any more, as grids of 1 row or column would do the trick too. I never implemented spanning of several rows or columns though, which would have been nifty, but likely not very needed :)

 

For really dynamic guis, you likely want to take another direction; assign a cost function for deviating from the desired position for each element, and then compute minimal total squared cost.

Edited by Alberth
smallest explanation was a bit too short

Share this post


Link to post
Share on other sites

For my own UIs, I generally prefer to avoid the "localization can change things size" idea - instead, I find the widest possible localized string and lay the UI out for that size by hand.  This guarantees that nothing surprising will happen due to automatic layout.

For user-entered data or things like large data sets, I prefer to build in WinForms/WPF-style vertical/horizontal splitters (draggable separators which let the user control the width/height of the two adjacent UI elements themselves).  I also prefer the WinForms style of using left/right/top/bottom Anchors to control child size when a parent size changes. This allows for the user to customize the UI for what they want to see, instead of being at the mercy of your automatic layout code and the UI design team.

In this approach, *nothing* changes size other than when the user explicitly changes it.

Edited by Nypyren

Share this post


Link to post
Share on other sites
9 minutes ago, Nypyren said:

For my own UIs, I generally prefer to avoid the "localization can change things size" idea - instead, I find the widest possible localized string and lay the UI out for that size by hand.

Haha, yes, now do that for 4000+ strings in 53 languages (about 110 windows) in the game, that constantly change, from countries all over the world (both left-to-right and right-to-left languages, which is another nice topic to consider :) )

In addition, each game extension comes with its own set of strings and translations thereof (a sub-set of those 53 languages), that is not under control of the game.

Edited by Alberth

Share this post


Link to post
Share on other sites

Yeah, 53 languages is a lot.  The games I work on only support 16.  I'm not sure how many strings-per-language we have, but it's definitely in the thousands.

We don't support third party mods, so that's a distinct advantage.

As far as finding the longest strings, we have a mode in our UI editor where it will will automatically look up all of the strings for a given widget, measure their size, and display the longest one.  Then the UI designer lays the dialog out so everything fits.  Then we have another mode where we rapidly cycle through all languages while rendering the dialog, and the UI designer interacts with the dialog while it's cycling through languages to make sure nothing breaks.

Edited by Nypyren

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now


  • Advertisement