In my immediate mode GUI I achieve some pretty nice font rendering, dynamic geometry for the whole UI is created each frame:
I create quad lists for the text and update a vertex and index buffer each frame. I originally looked at using distance field fonts but i found it hard to achieve the same quality of rendering. The bitmap font sheet was rendered with the Angel Code bitmap font tool. I've added in some GUI elements that use the different font effects such as italic and drop shadow.
The quickest solution would be IMGUI as suggested. I was never convinced it looked that great though and had limited skinning possibilities. I could be wrong though. If my understanding is correct and you wanted to ship the console with your game, it might be better to roll your own.
It's great because it's immediate, I don't have to convey the information about what properties I want in the property control using attributes (as in the style of C#) because I just do or don't give them in the GetProperties() method. I also don't have complicated macros because I just don't need a reflection system.
Are you actually making a game or a demo that requires these non-trivial entity hierarchies and multiple components of the same type? I integrated bullet into my entity-component system the most simple way which was one rigid body component per entity and one collision shape per component and i've yet to require anything else. Point being, just do what you need to do. There shouldn't really be any of these what-if scenarios until you actually need them and then the become scenarios and not what-ifs.
You might also consider using a pick buffer which is a render target where each pixel contains an ID that is the object that covers that pixel. Therefore sampling the pick buffer under the cursor gives you the object the mouse is over. This doesn't solve box selection though, but is a simple way of doing it that doesnt involve alot of projection/unprojection. Note also that its particularly good for picking terrain, where typically you don't want to be doing ray triangle intersection.
A reasonable compromise is to store a set of 1-bit flags on each entity, which tells you which components are attached to this entity. If you need to retrieve the component you'll still need to search for it in the relevant component array, but at least testing for it's presence is now cheap.
Sure that would work. I can't think of any occurrences where i've wanted to see if it has a specific component without actually wanting to do something with it though, not personally.
I decided to take the memory hit and set a MAX_ENTITIES that my engine can support and block out arrays for MAX_ENTITIES for each component. This way each entity is basically a column in the table and doing checks for "does my entity have X component" is just a case of checking the component at the same index as the entity. Ofcourse you will have unused slots in the array but you are also never left wanting as at any point its possible to turn on a new component for an entity by enabling a component at a given index without the cost of an allocation etc. I believe it is also popular to have a bitfield in the entity to discern which components are enabled for it.
Each subsystem iterates over the components checking components[x]->IsEnabled() before processing them.
There is a lot of crappy RMGUIs out there, so I understand peoples frustration and need for something new.
But RMGUIs doesn't have to be that crappy, and IMGUIs, while solving some problems, overlook others.
There is no reason an RMGUI would have to have complicated observers or invalidation logic.
In the RMGUI I'm currently writing, if you change something, vertex buffers will be invalidated and rebuilt before the next draw.
If not, well, then we reuse the vertex buffers from last frame. (normal case for 99% of the frames)
Its not hard to know if a change will mean the vertex buffer it belongs to need update.
I don't care at all about regions. That was relevant in software rendering. Not so much in HW.
You do decouple creation from update/reaction, but you do it for a reason. It might not even be the same person defining it.
But even if it is, I like to have my widgets initial position and layout defined in an xml. Nice and neat and does not pollute my code with layout, and I can even edit it in runtime and reload it for quick ui tweaks. (or complete ui changes as long as the same actions exists)
Since I use lambdas, there is no problem with extreme decoupling, all my callbacks are defined right then and there, without unnecessary code.
Code should be concerned with actions, and never care about exact sizes or positions.
I don't want to rewrite logic for when and how to draw my buttons for every game I do.
It makes sense for game engines where you have complex needs for culling and handling of transparency and effects.
Not that much for UIs.
To have higher class objects like views and popups make code brief and easy to read and modify the relevant parts of.
In my RMGUI, my UI is just another layer/pass in our graphics engine, I call "update" on it when it should be updated, and I call "draw" on it when it should be drawn. (so in a way IM, but on a higher level)
But I wouldn't want to call the equivalent of "draw" for each and every ui item each frame.
Not because I think it would be slow, but because I have no need to see and change that code, it can be completely generic for every UI.
Then of course, RMGUIs can be made overly complex... like most out there.
But it doesn't have to be like that, and I'm not sure IMGUIs really solve the problem, just moves the responsibilty
I don't want to discourage anyone from using an IMGUI though, I'm sure it can be perfect for a lot of projects.
I'll treat each paragraph as a bullet point, all are addressed:
1) Every RMGUI i've worked on (in games or otherwise) get overly complicated. It's inevitable. The most important thing in game development is iteration time, IMGUI tend to be quicker for this.
2) You can rebuild vertex buffers on invalidation sure, you should do that whether RM or IM. The topic is really about "what it takes to get my game gui done", both options have similar optimisations. I'm not sure what you mean by regions.
3) There isn't much practical need to decouple display and logic. The games ive worked on have had heavy XML driven RMGUI front ends and the artists and designers barely go near it. A well thought out IMGUI interface can produce code that is as easy to read as XML. If artists want to change the size of something they'll just look for the name and numbers wherever theyre declared.
4) Not sure about this but in my experience having callbacks registered to events etc gets very messy and it hard to track what is listening to what.
5) Surely when you call draw on your RMGUI it is fast enough that it doesn't hiccup the framerate much right? If so then just do it every frame. My IMGUI (fairly optimised) runs at around 1ms on reasonable hardware for my level editor. This is content that is much more complicated than any typical game GUI with health bars and so on. Also you don't change that code, it's completely generic for all UIs made with any given IMGUI.
6) They do move the problem to a certain degree but the fundamental benefits are that its easy to write new client UI content and you don't have application state stored in your UI widgets because they don't exist.