GUI rendering in software using threads

Graphics and GPU Programming Programming

Started by Kalith June 06, 2011 10:52 PM

12 comments, last by _the_phantom_ 12 years, 10 months ago

124

Author

June 06, 2011 10:52 PM

Hi,

I've developed (yet another) GUI library that I use for my own game projects. Most of the time, the GUI I make are quite complex : they uses many different textures and render targets.
Up until now, I've always relied on the render system of the graphics library I use to draw the game world (Ogre3D, SFML, ...) to draw my GUI on the screen. So the rendering is done in hardware.
Doing it in real time is a real performance hit, that I've minimized using cache render targets, that are only updated when their content is modified.

Yet I feel it's not enough. Plus, using screen wide render targets for caching has its own cost (both in memory and render time, for large resolutions).

So I had another idea. Seeing how powerful CPUs can get nowadays, and seeing how badly they are used in game programming, I was wondering if I could use that spare computing power to release the strain on the GPU and render the whole GUI in software, maybe in another thread.

Several other considerations tend to make this choice a credible alternative :

most of the time, GUI rendering consists of 1 to 1 texture drawing, which can be implemented as a simple copy (or blending) of pixel
proper 2D alpha blending is not feasible in one pass using todays hardware (afaik)
current GPUs are optimized for 3D rendering, and little effort is made to improve 2D performances
GUIs are most of the time made of plenty of small textures, which tends to create a large number of batches (even though this problem can partially be solved with texture packing algorithms, this system has its limitations)
hardware render targets can be tricky (poor support on old GPUs or with bad drivers, see Intel integrated chips)

I'm currently running some tests with my own software renderer (using SDL for texture loading and window management) and performances seem quite acceptable.
My test case, which consists of 9 sprites using 9 different 32x32 textures, with alpha blending, rendered into a 256x256 render target, which is then rendered into a 1024x768 render target takes approximately 1ms. Transferring the result to the screen then takes 4ms, but that depends highly on the underlying render system (here SDL), and is here a quasi constant cost (proportional to the final render area).
There is still room for optimization : I could use SSE instructions to blend colors for example.

My questions are :
Has anybody considered this alternative ?
Does it seem viable to you ?
What is the fastest, portable, software 2D renderer out there ?
Do I have a chance to do something better if I develop my own renderer, suited especially for my GUI system ?

Ashaman73

13,718

June 07, 2011 05:45 AM

most of the time, GUI rendering consists of 1 to 1 texture drawing, which can be implemented as a simple copy (or blending) of pixel

I for example use shaders for my GUI.

proper 2D alpha blending is not feasible in one pass using todays hardware (afaik)

It is not, why ?

current GPUs are optimized for 3D rendering, and little effort is made to improve 2D performances

That is the reason you render your GUI as 3D "objects".

GUIs are most of the time made of plenty of small textures, which tends to create a large number of batches (even though this problem can partially be solved with texture packing algorithms, this system has its limitations)

You use one or two atlases to get rid of the trouble. The use of texture arrays could even help to create one batch if you really need very large texture atlases.

hardware render targets can be tricky (poor support on old GPUs or with bad drivers, see Intel integrated chips)

You don't need render targets for GUI rendering, I use immediate OpenGL for rendering my GUI, if I would remove the shaders it could work on OpenGL1.1 compatible hardware.

My questions are :
Has anybody considered this alternative ?
Does it seem viable to you ?
What is the fastest, portable, software 2D renderer out there ?
Do I have a chance to do something better if I develop my own renderer, suited especially for my GUI system ?

I doubt that any serious GUI handling would be fast enough. Remeber that you compare the power of one CPU core vs hundreds of GPU "cores" doing some simple copy work. An other issue is the transfer from CPU memory to video memory, thought framebuffers etc. could speed up the process.

For comparision, I render 200-400 icons (48x48) + few hundreds of chars + few backgroud images (800x600) in immediate(!) OpenGL each frame in less than 5 ms.

Ashaman

Gnoblins: Website - Facebook - Twitter - Youtube - Steam Greenlit - IndieDB - Gamedev Log

Krypt0n

4,769

June 07, 2011 01:02 PM

some games do that, if you see what ppl do on consoles to save some ms (like doing a lot of transformation and culling on SPU to avoid doing that on the RSX, using the playstation edge library), then saving some ms by using a simple software gui rendering is a lot bang for a buck.

another nice thing bout software rendering is, that you can go oldschool and just update the regions that really change (it was common back then), as you anyway render to an offscreen buffer. doing so, you can have very complex gui rendering magnitude faster on cpu than using a gpu.

video game porting and optimization service + consulting

Hodgman

52,717

June 07, 2011 01:14 PM

As always, it depends on whether you're GPU-bound or CPU-bound on your target platform.

Lots of games use Flash for GUIs these days, which it turns out is horribly slow at drawing via the CPU, so GPU-accelerated flash renderers are worth a fair bit of cash to your average game studio ;)

Also, lots of games have HTML renderers in them these days -- many of which just use WebKit/Gecko/etc, which are CPU rendered. Though these usually aren't used for highly interactive or commonly visible items, like HUDs (probably because they are slow).

GPUs are very good at 2D copying though; even if you do perform it via triangles, they're highly optimized for the task and have specialized hardware to help. They're so good in fact, that you might not even need to bother with any cache targets...

High batch counts largely affects CPU performance, and unless something's going quite wrong, will of course take less time than the actual work done by the batch (e.g. the batch submission should be cheaper than the texture blitting in any case).

As above, I don't understand the "proper 2D alpha blending" complaint.
As for old hardware, you'd have to be aiming at ~pre 2006 hardware (which is pretty ancient by today's standards) in order to come across any kind of "poor support".

. 22 Racing Series .

Kalith

124

Author

June 07, 2011 05:55 PM

I for example use shaders for my GUI.

[quote name='Kalith' timestamp='1307400722' post='4820284']
proper 2D alpha blending is not feasible in one pass using todays hardware (afaik)

It is not, why ?[/quote]
You don't need "shaders" in software rendering, because you already have full control of the pipeline.
As for the alpha blending, maybe I'm mistaken, but with all graphics engine I've used (Ogre3D and HGE for example), you couldn't get "true" (as in Photoshop) alpha blending to work out of the box with render targets (I should have been more precise). In Ogre, as stated in the post I link, I had to use a two pass system. It could also be done in one pass with shaders I think, but I wonder how good this would perform...

That is the reason you render your GUI as 3D "objects".

Yes and that's my point. It could probably be much faster if things were optimized for 2D rendering.

You use one or two atlases to get rid of the trouble. The use of texture arrays could even help to create one batch if you really need very large texture atlases.

Atlases are the concepts I was referring to with my "texture packing algorithm". And as I say, they have their drawbacks : you cannot use texture tiling for example, except if you copy the same "sprite" multiple times, which sends N times more vertices to the GPU.

You don't need render targets for GUI rendering, I use immediate OpenGL for rendering my GUI, if I would remove the shaders it could work on OpenGL1.1 compatible hardware.

I do. Using render targets to cache your GUI can give very interesting performances, from 2 to 3 times faster with complex GUIs.
Of course, for a basic HUD with 2 textures and 3 lines of text, that's nonsense.

I doubt that any serious GUI handling would be fast enough. Remeber that you compare the power of one CPU core vs hundreds of GPU "cores" doing some simple copy work. An other issue is the transfer from CPU memory to video memory, thought framebuffers etc. could speed up the process.

For comparision, I render 200-400 icons (48x48) + few hundreds of chars + few backgroud images (800x600) in immediate(!) OpenGL each frame in less than 5 ms.

I know very little of how a GPU actually works, so I'll assume you're right on this.
Yet you cannot compare your timing performance with mine without saying what your GPU is. 5ms is sure a good timing though, and I doubt I'll be able to beat it with my old CPU.

Aren't Flash and HTML largely overkill for such simple tasks as rendering a GUI ? Apart from having an internet navigator ingame, I fail to see the point.
There is plenty of handy GUI libraries out there that I'm sure are more optimized, and even using my naive implementation of the software renderer it would probably be faster.

[quote name='Hodgman']GPUs are very good at 2D copying though; even if you do perform it via triangles, they're highly optimized for the task and have specialized hardware to help. They're so good in fact, that you might not even need to bother with any cache targets...

With texture atlases maybe. In my case it most certainly helps : my interface renders in 10ms without, and a little less than 3ms with cache targets. There is a little overhead when there is lots of constant changes in the GUI, but in my case that's unlikely.
By the way, I've put and example of what I consider a complex GUI below. It's a game editor here, but you can get the same level of complexity in games like WoW for example.

... or integrated Intel chips I have one on my laptop (I bought it this year), and SFML doesn't seem to like it.

[quote name='Krypt0n']another nice thing bout software rendering is, that you can go oldschool and just update the regions that really change (it was common back then), as you anyway render to an offscreen buffer. doing so, you can have very complex gui rendering magnitude faster on cpu than using a gpu.

That's more or less the kind of optimization I have in mind when I think of optimizing 2D drawing.
Plus, correct me if I'm wrong, but aren't all our OSes' UIs drawn in software ?

froop

642

June 07, 2011 08:39 PM

sorry to interrupt, but

you cannot use texture tiling for example

this is a myth which is burned into many peoples heads, but its not true. any sort of tiling can wonderfully be done with any arbitrary texture atlas and without creating extra vertices or something.

please proceed with this good topic now

Kalith

124

Author

June 07, 2011 08:50 PM

With shaders most likely yes. But it has a cost, compared to the "free" tilling you get by using classic textures.
Anyway, sorry for spreading a false rumor

Hodgman

52,717

June 08, 2011 12:35 AM

You don't need "shaders" in software rendering, because you already have full control of the pipeline.

A "shader" is just code... Software rendering uses code...
Shader's aren't some special feature -- it's just how you write software that runs on a GPU.
Even if you're not using shaders (i.e. you're using the fixed function pipeline), then that just means you're using the built-in shaders behind the scenes

As for the alpha blending, maybe I'm mistaken, but with all graphics engine I've used (Ogre3D and HGE for example), you couldn't get "true" (as in Photoshop) alpha blending to work out of the box with render targets ... It could also be done in one pass with shaders I think, but I wonder how good this would perform...[/quote]Yes, you're mistaken. As for the performance of shaders, your own shaders will probably perform better than the built-in general purpose (fixed function) ones.

Yes and that's my point. It could probably be much faster if things were optimized for 2D rendering.[/quote]Aside from overhead of setting up a triangle, the GPU launches hundreds of threads, each of which can take an unfiltered texture sample per cycle. How many texture samples can you read on the CPU per cycle?
In other words, its much more optimized for 2D rendering than a CPU is, even if it seems geared towards 3D usage.

Aren't Flash and HTML largely overkill for such simple tasks as rendering a GUI ? Apart from having an internet navigator ingame, I fail to see the point.
There is plenty of handy GUI libraries out there that I'm sure are more optimized, and even using my naive implementation of the software renderer it would probably be faster.[/quote]I wasn't making a point, I was just pointing out that those technologies currently are used in games, and often are rendered on the CPU, which answers your questions "Has anybody considered this alternative?" and "Does it seem viable to you?" with "yes", because it's been done.

As for why flash and HTML are used in games - it's because there's tools for authoring that kind of content. Building an engine/renderer is only half the battle. There's hundreds of engines out there with sub-standard tools to go with them. If you chose Flash as your GUI solution, then you can use industry-standard tools from Adobe to author your game, and hire graphic designers that don't need as much training.
With texture atlases maybe. In my case it most certainly helps : my interface renders in 10ms without, and a little less than 3ms with cache targets. There is a little overhead when there is lots of constant changes in the GUI, but in my case that's unlikely.[/quote]Is that 10ms of CPU time or GPU time? Either way, it just sounds like you're using the GPU inefficiently (sorry). If you understand CPUs better, then maybe that is a reason to make your optimised GUI renderer on that side though.
... or integrated Intel chips I have one on my laptop (I bought it this year), and SFML doesn't seem to like it.[/quote]Every integrated chip since 2006 is DX9.0c compliant (apparently), so render-to-texture should be possible. Likely that SFML needs some massaging if it's not letting you.
Plus, correct me if I'm wrong, but aren't all our OSes' UIs drawn in software?[/quote]As of Vista/Win7, no - they (can) use the GPU.