Designing a GUI system, hints appreciated.

Massimo Del Zotto · 2010-07-07T12:55:35

At the end of yesterday, I figured out I truly needed a way more complete and less headache-involving approach to manage my simulated GUI. I couldn't just keep on hacking pixels together. I had some experience with "real" GUIs in the past and while I am surely not a wizard like many others, I have a few scars. I recalled reading an article on Immediate Mode GUIs and I've decided to take a look at them. The next part of the post is essentially a wrap-up of a day spent in understanding IMGUI I post here for other eventual readers in the hope they could save some time. Just to aggregate the few resources I've considered:probably first original discussion on the topicoriginal video by Casey Muratori on IMGUI paradigm Game developer magazine, september 2005, The Inner Product, by Sean Barrett. I also liked pretty much this SDL based tutorial. At this point however, I was still not completely sold. Some of the so-called "cons" of retained mode GUIs didn't apply to the system I had in mind as I had additional machinery at my disposal that I could use to keep various things under control. My goal was to figure out the shortest path to get the minimal GUI I need that is also "The Right Way". I figured out that I might have to consider fully fledget retained GUI in the future. It seems to be very reasonable and Unity seems to be following that same path, so why not? For those not familiar with the paradigm, here are the starting point from which an IMGUI system works: The most important thing is that the user will interact with a single control at time. Everyone of us probably knows that, kudos to Mr Muratori for formalizing that and proving a less involved GUI system is possible! The second important thing is accepting a GUI is closely tied to the data it represents. A checkbox will generally represent a boolean property. Radio buttons give you a choice between the few designed. Callbacks and Listeners (as in SWING) could be fine if you need to manage non-trivial behaviour, but this could be embedded in the application on need, while leaving the paradigm simplier. As such, have the GUI controls work directly on application data (which is then GUI' state as well). The problem of connecting the app to the gui is now solved. Leave eventual non-trivial mappings to the application. Now that those things are clear, IMGUI takes the form of a very nice, syncronized-looking call such as doButton(blah), it's almost going back to the old console-print and read function. This doButton function will return true if the state of the drawn button changed this tick. End of the story. Now coming to the point... As I would like to figure out a good way to do this which doesn't require me to rewrite everything... or even something for a while, I would like to have some brainstorming on what you wished your GUI could do (or is doing). I know "more or less" what I need in the next future but I ask for help in drawing a bigger picture.

Shinkage

595

July 01, 2010 08:21 PM

Quote:Original post by swiftcoder
Why do you feel it is necessary to continually poll/update your IMGUI? Treat it exactly as you would any RMGUI, and only update when input is available, or when you need to redraw. Better yet, make your IMGUI multi-pass, so that you only invoke the necessary portion on input/drawing.

Okay, assume I do this, then the distinction between IM and RM has become vanishingly small and any advantage of putting an IM facade over the actual RM implementation... well, what was it again?

MoundS

162

July 01, 2010 09:34 PM

Quote:Original post by Shinkage
Okay, assume I do this, then the distinction between IM and RM has become vanishingly small and any advantage of putting an IM facade over the actual RM implementation... well, what was it again?

It's the only difference there ever was... the API. That's what IM and RM refer to.

In IMGUI, I can write something like this...

for (i = 0; i < n_shapes; ++i){    shape& s = shapes;    do_text(s.name);    if (s.loaded)    {        do_text(s.type);        do_check_box("visible", &s.visible);        if (s.visible)        {            do_color_control("color", &s.color);            do_drop_down("style", &s.style, style_choices);        }    }    else        do_text("loading...");    if (do_button("delete"))        // ...}

What's the RMGUI equivalent of that?

Shinkage

595

July 01, 2010 09:59 PM

Quote:Original post by MoundS
Quote:Original post by Shinkage
Okay, assume I do this, then the distinction between IM and RM has become vanishingly small and any advantage of putting an IM facade over the actual RM implementation... well, what was it again?

It's the only difference there ever was... the API. That's what IM and RM refer to.

In IMGUI, I can write something like this...

*** Source Snippet Removed ***

What's the RMGUI equivalent of that?

Well, keeping in mind that immediate mode and retained mode, generally speaking, refer to more than just the syntax of your API calls...

As for an RMGUI equivalent, that's something of a silly question, considering it'll depend entirely on the design of the API. As for my own GUI framework? It'd probably involve hooking a couple signals (shape_list.on_shape_loaded.connect(...), shape_list.on_shape_visible.connect(...) for example maybe), but it's kinda pointless to try to imagine an equivalent of a bunch of code that's been removed from any obvious context.

MoundS

162

July 01, 2010 10:10 PM

Quote:Original post by Shinkage
Well, keeping in mind that immediate mode and retained mode, generally speaking, refer to more than just the syntax of your API calls...

As for an RMGUI equivalent, that's something of a silly question, considering it'll depend entirely on the design of the API. As for my own GUI framework? It'd probably involve hooking a couple signals (shape_list.on_shape_loaded.connect(...), shape_list.on_shape_visible.connect(...) for example maybe), but it's kinda pointless to try to imagine an equivalent of a bunch of pseudocode that doesn't have any clear end purpose.

The point wasn't the syntax of the API calls. Sorry if you didn't understand the purpose of the code. Maybe it's hard at first to understand IMGUI code if you're not familiar with the ideas.

Do you understand the difference between IM and RM graphics libraries? I think that's a good stepping stone to understanding IMGUI and RMGUI. It's also easier to find information on.

Shinkage

595

July 01, 2010 10:46 PM

Quote:Original post by MoundS
The point wasn't the syntax of the API calls. Sorry if you didn't understand the purpose of the code. Maybe it's hard at first to understand IMGUI code if you're not familiar with the ideas.

Do you understand the difference between IM and RM graphics libraries? I think that's a good stepping stone to understanding IMGUI and RMGUI. It's also easier to find information on.

I really have no idea what you're getting at then if the point wasn't the syntax of the API calls. Yes, I understand quite well the difference between IM and RM graphics libraries: immediate mode sends commands to the graphics hardware immediately, while retained mode caches the data from commands and sends them in (ideally) optimized batches.

But that's somewhat beside the point, since every sophisticated IMGUI library is essentially a retained mode library with an immediate mode facade; they have to be in order to perform well. To be more specific by what I mean by that, IMGUI libraries generally will retain a complete model of the GUI "behind the scenes" as it were in order to not have to recalculate everything every frame. Hence, an IM facade over an RM library.

So... if the point wasn't the syntax of the API calls... then what was it?

apatriarca

2,365

July 02, 2010 06:24 AM

Quote:I really have no idea what you're getting at then if the point wasn't the syntax of the API calls. Yes, I understand quite well the difference between IM and RM graphics libraries: immediate mode sends commands to the graphics hardware immediately, while retained mode caches the data from commands and sends them in (ideally) optimized batches.

No, that isn't the difference between retained mode graphics and immediate mode graphics. A retained mode graphics library is a library where all the scene information is maintained by the library. It is usually based on some form of scenegraph. There isn't any draw call in these libraries, just functions to update the internal scene model. In an immediate mode graphics library it's the user who manage the scene and you have to decide each frame what you want to draw.

Caching is done in both modes. In OpenGL for example all the glVector, glNormal,... calls are cached and the data is then probably loaded in a stream VBO (this is at least how I would do it). Buffer objects are in some way just an optimization where the caching is done explicitly by in the application code, but it is still in some sense "immediate mode". It is in fact still the application who manage the scene and not the library.

IMGUI are similar. It's the application who explicitly manage the state of the GUI and not the GUI library itself. Caching is done to improve drawing performance (as in OpenGL IM), but the scene state isn't retained between frames by the library. It is the application who maintains those information.

Shinkage

595

July 02, 2010 09:29 AM

Quote:Original post by apatriarca
No, that isn't the difference between retained mode graphics and immediate mode graphics. A retained mode graphics library is a library where all the scene information is maintained by the library. It is usually based on some form of scenegraph. There isn't any draw call in these libraries, just functions to update the internal scene model. In an immediate mode graphics library it's the user who manage the scene and you have to decide each frame what you want to draw.

Now you're just arguing semantics; this is essentially exactly what I said. Caching = stored for later use. That's the difference between IM and RM--RM stores data for later use, IM requires that the data be fully specified whenever it's needed. Not sure why you feel the need to correct me on this? Maybe the language I chose just wasn't ideal.

EDIT: Just to be clear, to the extent that OpenGL or another API caches data behind the scenes, it is operating in a retained mode. There's no law saying you can't mix IM and RM practices, and that is in fact exactly what OpenGL--AND most sophisticated IMGUI libraries--actually do.

Quote:Original post by apatriarca
IMGUI are similar. It's the application who explicitly manage the state of the GUI and not the GUI library itself. Caching is done to improve drawing performance (as in OpenGL IM), but the scene state isn't retained between frames by the library. It is the application who maintains those information.

Have you read the past few posts I made before the one I responded to? Because what you're saying here is exactly my point. The thing about pure immediate mode rendering AND pure immediate mode GUIs is that the entire scene needs to be respecified every frame. That was 100% the problem I was getting at a few posts back; for complex scenes that can become very inefficient.

This is not a problem with me not understanding what an IMGUI is. It's nothing remarkable and certainly nothing new--I was seeing IMGUI implementations as a programmer 10-15 years ago and they're the same thing now that they were then.

apatriarca

2,365

July 02, 2010 10:14 AM

Quote:Original post by Shinkage
Quote:Original post by apatriarca
No, that isn't the difference between retained mode graphics and immediate mode graphics. A retained mode graphics library is a library where all the scene information is maintained by the library. It is usually based on some form of scenegraph. There isn't any draw call in these libraries, just functions to update the internal scene model. In an immediate mode graphics library it's the user who manage the scene and you have to decide each frame what you want to draw.

Now you're just arguing semantics; this is essentially exactly what I said. Caching = stored for later use. That's the difference between IM and RM--RM stores data for later use, IM requires that the data be fully specified whenever it's needed. Not sure why you feel the need to correct me on this? Maybe the language I chose just wasn't ideal.

EDIT: Just to be clear, to the extent that OpenGL or another API caches data behind the scenes, it is operating in a retained mode. There's no law saying you can't mix IM and RM practices, and that is in fact exactly what OpenGL--AND most sophisticated IMGUI libraries--actually do.

Maybe my post wasn't clear enough, but we were not saying the same thing. For me, retained mode isn't about caching or storing state at all. You have to store your state somewhere independently of the library used. The difference is in who update and manage that state. In retained mode you initialize your GUI or scene and then it's the library who update and render it, while in immediate mode is the application who manage the GUI or scene state, and then send it each frame to the library.

So, in my opinion, buffer objects or textures in OpenGL aren't part of a RM part of the library. You are just storing data in a buffer (just like using a malloc) and OpenGL does not know when and how to use it to render the scene.

Quote:Original post by Shinkage
Have you read the past few posts I made before the one I responded to? Because what you're saying here is exactly my point. The thing about pure immediate mode rendering AND pure immediate mode GUIs is that the entire scene needs to be respecified every frame. That was 100% the problem I was getting at a few posts back; for complex scenes that can become very inefficient.

This is not a problem with me not understanding what an IMGUI is. It's nothing remarkable and certainly nothing new--I was seeing IMGUI implementations as a programmer 10-15 years ago and they're the same thing now that they were then.

I read your past posts. But if done correctly, IMGUIs don't have to do a lot more calculations then RMGUIs. You don't recreate the GUI each frame, you are simply "iterating over your GUI" (whose state is maintained by the application) at each event (mouse click, render and so on). Your RMGUI have to do it anyway so the computation involved are actually similar. The only caching really required comes from rendering. But even in this case it doesn't store the state of previous frames.

MoundS

162

July 02, 2010 10:24 AM

Quote:Original post by Shinkage
Now you're just arguing semantics; this is essentially exactly what I said. Caching = stored for later use. That's the difference between IM and RM--RM stores data for later use, IM requires that the data be fully specified whenever it's needed. Not sure why you feel the need to correct me on this? Maybe the language I chose just wasn't ideal.

No, it's really not what you said. You were describing IM without caching vs IM with caching. Part of the problem may be that you're misusing the term "cache". Caching is when you store data that's available through other means but you want faster access to. If you remove a cache, things get slower, but you don't lose any data. An RM library doesn't cache the scene, because it's the sole authority on what's in the scene. If you delete the RM library's representation of the scene, it's lost. That's not true for IM.

Quote:EDIT: Just to be clear, to the extent that OpenGL or another API caches data behind the scenes, it is operating in a retained mode. There's no law saying you can't mix IM and RM practices, and that is in fact exactly what OpenGL--AND most sophisticated IMGUI libraries--actually do.

That's not "operating in retained mode." Again, it's the difference between caching and declaring yourself the sole authority on some data.

Quote:Have you read the past few posts I made before the one I responded to? Because what you're saying here is exactly my point. The thing about pure immediate mode rendering AND pure immediate mode GUIs is that the entire scene needs to be respecified every frame. That was 100% the problem I was getting at a few posts back; for complex scenes that can become very inefficient.

Well, it's a tradeoff between human effort and computer effort. With IMGUI, the computer does the work of figuring out what needs updating so that the programmer doesn't have to. And in a well designed IMGUI system, the computer doesn't do much more than just pass through the function and check if anything has changed. A lot of RMGUI programmers resort to that anyway to keep the update logic sane, but they have to write all the diffing and patching code manually.

In my experience, performance just isn't a problem. I have a full screen IMGUI app with a complex UI running on my old laptop right now, and it takes about 0.5 ms to do the full update/render cycle.

I can even imagine a more advanced system that would translate an IMGUI-style specification into a RMGUI-style one, or just keep track of what data the GUI references and monitor it for changes.

You can even do manual optimizations, like marking parts of UI as "static", kinda like telling the library to treat that part of your code as if it were RMGUI. RMGUI is like assembly language. You don't start out planning to write your app in it, or even a part of your app. You first write your app in a high-level way, and then if parts of it prove to be too slow, and none of your high-level optimizations have proved sufficient, only then do you resort to RMGUI/assembly. I haven't resorted to either in years. :-)

Krohm

5,051

Author

July 02, 2010 01:00 PM

Ok so, now that my first iteration was sort of complete - so I thought - I wanted to refine a bit ... and more problems arisen.
Today's quotes come from the [Sweng-gamedev] mailing list, more exactly from the "Why IMGUI arguments fall short" thread. I take something out of Tom Plunket's message which helped quite a lot:

Quote:The idea is that IMGUI holds /no/ state, so the application can be in
control and display exactly what needs to be displayed every frame.
...
IMHO, the place where the IMGUI camp really misses the mark is in
its complete and utter failure to notice the fact that all tools in
our toolboxes have places where they're appropriate, and all tools
have their places where they're inappropriate. IMGUI is not The
Silver Bullet.
...
point to Barrett's article itself; "[t]he most important thing to remember is
that immediate mode doesn't require us to create and destroy the
objects involved." Indeed, because there are no objects, or so the
implication is. But the fallacy of that argument comes clear the
second you take a look at what is actually happening under the hood;
draw instructions are in fact being created, from scratch, every
frame.

I grow increasingly convinced that IMGUI makes sense only for very simple, almost stateless systems. This happens because, as I noted before, allowing the IMGUI library to track state requires pretty strong suppositions - no data shuffling for example.

I also took a way too optimistically approach at this. As the quoted message reads, I had somehow got the idea that but recently it was "The Most Amazing Thing EVAR".
Reading MollyRocket's forum for example, not many posts says IMGUI sucks at this... or at that. This doesn't help. They say why you should go IMGUI instead of RMGUI (and not even very clearly) but they're not saying when you truly want to have RMGUI, with properly accessible, stateful widgets. Maybe they just took for granted everybody knew what a RMGUI would buy, as they've be around quite a while, I don't know, but it certainly would have helped. I have the feeling some of this approach carries on even there.

For example, to carry on the 3D API IM/RM parallel, nobody would ever want - say - to specify shaders per frame because that's expensive. GUIs are not the same case for me but they forget to note that having explicitly persistent state that can be referenced is nice by itself indeed. Even if shader compiling would be free, would you truly want to write SetVertexShader("blah")? I wouldn't.

It turns out that reading the above comments, I figured out I was doing some VERY WRONG things. I'm not going to say what as I find them so dumb to be just embarassing...
Yeah, objects.
Objects are my friends.
I like objects...
I think I will truly have to take a break and start with this "new fresh" point of view.

I post this in the hope someone in the future can find this and avoid falling in the "Most Amazing Thing EVAR" trap that ultimately lead me to wasting so much time and effort.
It turns out that writing an IMGUI library is not quite different from writing an RMGUI one - ok, I admit my previous try at a GUI library has been quite a while ago - with the added complexity of managing the on-the-fly creation.

I hope this helps.

As a side note, this thread helped me so much I really wanted to make a donation to gamedev.net, I was considering that since beginning, but swiftcoder' status made me jump on it.
I am rather sure the expenses generated by this topic are more than covered, yet considering the attention compared to other SW-ENG threads here, I think you should consider dropping a few cents.

Previously "Krohm"

Designing a GUI system, hints appreciated.

This topic is closed to new replies.

Popular Topics

Recommended Tutorials

Designing a GUI system, hints appreciated.

This topic is closed to new replies.

Popular Topics

Recommended Tutorials

Reticulating splines