At the end of yesterday, I figured out I truly needed a way more complete and less headache-involving approach to manage my simulated GUI. I couldn't just keep on hacking pixels together.
I had some experience with "real" GUIs in the past and while I am surely not a wizard like many others, I have a few scars. I recalled reading an article on Immediate Mode GUIs and I've decided to take a look at them.

The next part of the post is essentially a wrap-up of a day spent in understanding IMGUI I post here for other eventual readers in the hope they could save some time.

Just to aggregate the few resources I've considered:
probably first original discussion on the topic
original video by Casey Muratori on IMGUI paradigm
Game developer magazine, september 2005, The Inner Product, by Sean Barrett.
I also liked pretty much this SDL based tutorial.

At this point however, I was still not completely sold. Some of the so-called "cons" of retained mode GUIs didn't apply to the system I had in mind as I had additional machinery at my disposal that I could use to keep various things under control.
My goal was to figure out the shortest path to get the minimal GUI I need that is also "The Right Way". I figured out that I might have to consider fully fledget retained GUI in the future. It seems to be very reasonable and Unity seems to be following that same path, so why not?

For those not familiar with the paradigm, here are the starting point from which an IMGUI system works:

1. 
   
2. The most important thing is that the user will interact with a single control at time. Everyone of us probably knows that, kudos to Mr Muratori for formalizing that and proving a less involved GUI system is possible!
3. 
   
4. The second important thing is accepting a GUI is closely tied to the data it represents. A checkbox will generally represent a boolean property. Radio buttons give you a choice between the few designed. Callbacks and Listeners (as in SWING) could be fine if you need to manage non-trivial behaviour, but this could be embedded in the application on need, while leaving the paradigm simplier.
   As such, have the GUI controls work directly on application data (which is then GUI' state as well). The problem of connecting the app to the gui is now solved. Leave eventual non-trivial mappings to the application.
5. 
   

Now that those things are clear, IMGUI takes the form of a very nice, syncronized-looking call such as doButton(blah), it's almost going back to the old console-print and read function. This doButton function will return true if the state of the drawn button changed this tick. End of the story.
Now coming to the point...

As I would like to figure out a good way to do this which doesn't require me to rewrite everything... or even something for a while, I would like to have some brainstorming on what you wished your GUI could do (or is doing).
I know "more or less" what I need in the next future but I ask for help in drawing a bigger picture.

Quote:Original post by Krohm
I would like to have some brainstorming on what you wished your GUI could do (or is doing).

- Unicode support. My currently in-development GUI toolkit does this via cairo/pango, but I don't know of any others which can manage this, and it makes foreign localisation one hell of a lot easier.

- Automatic layout. IMGUI's are a bit easier to layout manually, but I really don't want to have to hand-configure (in code, no less) the GUI for each container at each resolution/aspect-ratio.

- Theming. There is no point adopting a 3rd party GUI toolkit if I can't theme it to completely match the rest of my application. This doesn't just mean custom images, but custom metrics, layout, etc.

Ease to implement.

GUI.Init();

Window = GUI.AddWindow();
Button = Window.AddControl(BUTTON);
Button.OnClick(OnButtonClickFunc);

GUI.Update();
GUI.Draw();

Full control over the rendering pipeline. I want to be able to completely integrate GUI element rendering into my own graphics engine, to do things like render GUI elements to textures rather than the framebuffer, optimize batching, and so on. This is tricky to get right but makes your GUI toolkit incredibly powerful, especially if you want to do nontrivial stuff with your UI.

Prototyping tools that allow me to edit and preview UI animations etc. outside the game are also invaluable. Ideally, a WYSIWYG editor should be available.

Thank you everybody for your suggestions. Reading them truly helped me focus. I'll reply them in the second part of the message, for this first part, I'd like to continue "the story" from what I've left in my original message.


In my first message, I wrote that "II was still not completely sold" from IMGUI paradigm. What truly scared me was the layout manager, for non-trivial widgets.
To solve this, I started looking at available libraries and yesterday I stumbled on nvidia-widgets. NVWidgets, as I'll call it now on, looked just right to me.

The first thing I've noticed is that the need to generate control ID is gone in nvWidgets. I long wondered why IMGUI controls needed a ID, as their existance lasts a moment, and they're almost completely identified by their evaluation. Taking out the unique ID helped me incredibly in closing the circle.

The second thing that nvWidgets does is taking it easy on layout. Hey, most controls are just rectangles! I was amazed to figure out I didn't notice this besides all my effort. Shame on you, complicated RMGUI paradigms, deforming my line of thinking to the point a control is necessarily a difficult, non-trivial thing. It's abstraction lost... this time for good. The layout algorithm in nvWidgets is driven by a couple of enums and bitmasks, somewhat ugly, if you ask me, but it works fine for the simple applications they need to build (have a look at NVSDK 10.5 for GL). I'll probably need something slightly more complicated, but at least I now know I was not keeping it simple enough.

The third thing is that nvWidgets tries to apply some separation between the logic and the rendering in a way that is a bit easy-going but nice in concept. More on that later.

The key thing is that it accumulates input in a buffer, not very differently from how a typical GUI manager would do, and then flush ASAP to the focused control.
I am still not 100% sure of how they manage their focus, it looks to me that in some odd cases (that is, a control taking place of the focused one, overlapping the focus point) would cause the focus to be transferred "transparently" without actually causing a loss of focus. It's probably far less worse than it sounds. Seems kinda rare and bad UI design since start.

A thing that made me somewhat skeptic was the way they manage their frames. Their frames are easy going, being basically just a border with no fill. This makes them order-independent. I wanted to support at least some blending, for which I should enforce a specific drawing order. This implied I had to truly cache some controls instead of drawing them to follow the IMGUI paradigm, and this is the first thing which I never figured out so far. Decoupling completely the rendering from the logic implies I can just work on the true hotspots (Model), and draw later (View), after all the controls' state are known. This still has the same layout issues as clicks would have already been resolved, but it's just so much better!
I am positive putting an IMGUI in a retained mode API must be done that way. No matter what, nobody can afford a VB Map per-control, tens of times per frame. The alternative, uniform fetching didn't look much better. Even if I could afford the GUI drawing itself, it would completely screw parallelism. It's just a no-go.
With decoupling, all of this is gone and the manager is left with a simple, comparisons of input vectors. The geometry is updated only on change. There are some issues with persistence, but I'll figure out something.


Now coming to your replies...

Quote:Original post by swiftcoder
Unicode support. My currently in-development GUI toolkit does this via cairo/pango, but I don't know of any others which can manage this, and it makes foreign localisation one hell of a lot easier.

My app is unicode-enabled ground up. I've even tested some hebrew, hindi and arabic. I still have some issues in correctly guessing the caret insertion positions. For the time being, I won't support arbitrary caret positions, because they interact with some additional machinery I plan to deploy (see below).

Quote:Original post by swiftcoder
- Automatic layout. IMGUI's are a bit easier to layout manually, but I really don't want to have to hand-configure (in code, no less) the GUI for each container at each resolution/aspect-ratio.

I'll be honest. I still have no idea on how to correctly resize windows/controls to manage aspect ratio.
Right now, I am thinking about assuming 4:3 ratio, by putting the coord system origin where it would be if the monitor would be 4:3.
That is, on a 4:3 monitor the coordinate system would have origin in the lower left corner (resolution independent). On a 16:9 and 16:10 screen the origin would be arranged so that normal 4:3 screen would be centered, meaning that I would get some "good pixels" for x<.0 and x>1.0, with windows popping "more or less on the center". This is just wrong for many cases, for which an alternative solution is definetly necessary, with x=0 at lower left corner depending on needs.
The method used to layout in the above mentioned nvWidgets is rather minimal but somewhat ok for very simple prototypes. I'm not sure I need so little.

Quote:Original post by swiftcoder
- Theming. There is no point adopting a 3rd party GUI toolkit if I can't theme it to completely match the rest of my application. This doesn't just mean custom images, but custom metrics, layout, etc.

It seems a far more advanced functionality than I would need for quite a while. Also makes little sense for the time being, as I don't plan to release it to the wild any time soon.
Thank you anyway for this, I'll take a memo about that.
I would appreciate if you could point out some "extreme" theming examples. I always considered Gnome themes to be rather nice, but they're also somewhat basic.

Quote:Original post by ApochPiQ
things like render GUI elements to textures rather than the framebuffer

This rationale has been there since day one. I have myself no idea how much time and effort I have spent on this up to now but I still think it's a primary requirement nowadays. There will also be special systems to remap input events such as drive mouse cursor on an arbitrary surface. Not quite arbitrary. Will probably be just quads and maybe cylindrical and spherical sections.

Quote:Original post by ApochPiQ
optimize batching

Of the GUI itself you mean? For complex GUIs that would be incredibly difficult given my current architecture. Interestingly enough, the roadmap would sooner or later take me to a future architecture optimizing this transparently but it's too soon to say.

Quote:Original post by ApochPiQ
Prototyping tools that allow me to edit and preview UI animations etc. outside the game are also invaluable. Ideally, a WYSIWYG editor should be available.

I'll be honest, I still haven't figured out how to be able to mix IMGUI with data driven design. Not in the details, at least.
My best bet is that if I provide a doButton(ButtonDescriptor&, bool&) call, I might eventually pool those descriptors as resources, maybe localized resources and then, in code do things like
This will happen in the future, hopefully soon, just not now.

Quote:Original post by Grafalgar
The games that do benefit from application-UI's are data-intensive ones - like MMO's or sports games (since they need to display statistics). For everything else, simplicity is key :)

This is the driving principle of IMGUI! Have a look at this paradigm!

I also considered going for a similar approach, only a few days ago this was my choice. It required alot, alot of extra machinery (not that IMGUI simply plugs in, but it requires way less).

The main concern about the data-driven model you also ended up with is linking callbacks. I suppose it's much easier in C# than it is in C++, but I simply had to jump thuru hoops to make an RMGUI work for generic data, on generic code polling arbitrary widgets. Too many variables for now bordering on the realm of deep magic.

The IMGUI examples I've seen are also very simple, but I don't agree on the need of tightly coupling logic and presentation, in fact, I just trashed a few hours of work to implement this separation. It is worth that, as I've noted in my previous message, logic and presentation must truly be temporally separated in my opinion, unless you're not scared of locking buffers/uploading uniforms potentially hundreds of times per frame.

Quote:Original post by Grafalgar
To me the approach seems tightly woven together, in that you specify a button to be rendered and then action if that button is to be interacted with.

And how is this functionally any better in a RMGUI? Sure, the artists can add a button to an RMGUI in the GUI designer, but until the programmer writes a function for the button to call, it doesn't do anything.

Quote:So, per my understanding, if an artist wants to add an additional button they need to create the resource and ask the programmer to add it in, before it can be visible on screen.

And that brings us to the meat of the issue: why in hell is your artist adding buttons willy-nilly? The artist doesn't know what functionality is required, that would be the programmer's responsibility.

You are thinking too much in an RMGUI workflow, where the artist designs the UI, and the programmer makes it work, typically requiring some cycles of artist<->programmer before it is correct.

With an IMGUI, the programmer figures out functionality, stubs out the GUI hooks *beforehand*, and then the artist modifies the layout and themes GUI based on the stub.

Quote:How does IMGUI handle complex animations? I admit that I am not intimately familiar with the approach aside from the handful of examples I've seen.

The exact same way an RMGUI does: you run a timer, which updates certain attributes of the GUI elements.

Simple IMGUI's (like the nvidia sample) obviously can't do this, because they don't persist any state behind the scenes, but an IMGUI of any complexity may retain just as much information as a RMGUI - the key element being that the user doesn't have to deal with it.