Sign in to follow this  

Windows GUI in Directx/OGL

This topic is 3660 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

Hello Guys, I have a very simple question but possibly with a very complex answer. In a full screen directx/ogl application (possibly a game), how can I show regular windows GUI components? For example how can I show just a blank windows form? Would it be possible to transfer the control to the new form (including with the mouse) so that user can interact, click buttons, write on a textbox while the game is still being shown in the background? I may come off as a bit clueless, but I have spent the past 3 months on this topic. I know it involves DLL injection/API hooking and I DO have a working program (hook in any dx8/dx9 game - dx10/ogl not yet implemented, draw on the screen such as fps, handle all the lost devices, so on so on). I can hook into all important methods such as createdevice, createadditionalswapchain, present, clear, so on. I have spent a long time for this and I finally got it to work a few days ago for all dx8/dx9. My initial purpose was to of course draw a winform on the screen (it'd also be amazing if I could do it in C#, maybe transfer the control over to C# from unmanaged dll file, pass the device pointer, draw, and pass it back as C++ is very hard). A good example to this is xfire. But of course once that's done I have to do a lot more complicated stuff, stuff that's never been done before, such as reflecting your entire desktop over the game and give user control to take actions such as go to start then firefox and check your e-mail, it'd be as if you are in windows without alt+tab. But for that to work, my plan was as follows: 1. Hook into game interface. DONE. 2. Draw on the game. DONE. 3. Handle exceptions/device losing/focus change/game issues. DONE. 4. Draw a simple winform. 5. Get the handle to the desktop, capture its view, reflect it on the screen, and do this in every Present() call. 6. Get the mouse actions, reflect it on the desktop. 7. Repeat step 5. I'd really appreciate any help. At this point, I am trying to find a source a document or something that can get me started. There is nothing in google about this and i am pulling my hair out. Thanks. PS If you are interested in this project, please PM me. At this point it became a huge project extending to unmanaged C++, kernel level device drivers (for CPU instructions), C#, server logic, benchmarking, hardware monitor, and a billion other things. Codebase is over 50k lines now and I am desparately looking for someone who'd like to join and help me out. [Edited by - TheQuestion on December 5, 2007 6:45:44 PM]

Share this post


Link to post
Share on other sites
I forgot to mention one thing. The way I want to do this is via win32 controls. I do NOT want to re-invent my own controls (I know there are dozens of articles on that), I don't want to write something from scratch. That's unacceptable. Look at xfire, if anyone used it, you'd know it uses straight win32 forms (set the skin to default and see in game it's a regular win32 form width default controls) and I highly doubt they sat down and spent months designing listboxes or checkboxes.

Another reason why I want win32 so bad is because it can be used in all dx8/dx9/ogl/dx10, windows xp/vista, 32bit/64bit. If I were to make my own or use another library, i'd increase the chances of having problems with all these different variations of systems.

Share this post


Link to post
Share on other sites
Its done already, see

This way you can later add in an OpenGL renderer, support Linux, MacOS, etc. The UI should not be OS dependent at all. Nor should your engine if its designed decently.

Edit: There are a ton of other UI libraries out there too, that was just the first one to come to mind.

Share this post


Link to post
Share on other sites
Quote:
Original post by Mike2343
Its done already, see

This way you can later add in an OpenGL renderer, support Linux, MacOS, etc. The UI should not be OS dependent at all. Nor should your engine if its designed decently.

Edit: There are a ton of other UI libraries out there too, that was just the first one to come to mind.


I think you misunderstood me. I am not looking for a UI library. What win32 has is enough for me. What I am looking for is a way to draw those UI components within the game. Please read my first post carefully.

I have pretty much control over the game, I can get the device, the main HWND (from CreateDevice() method argument), or any other major variables the game uses. I just need to figure out a way to draw my own screen on top of it without remaking what's already out there.

Thank you.

Share this post


Link to post
Share on other sites
Take a look at the DXUT GUI framework from the DirectX SDK. It uses standard windows controls and draws them through Direct3D by hooking the OwnerDraw message. You can use the same approach for OpenGL or rewire the DXUT GUI to work on your abstracted renderer interface.

Anyway, you would lose all portability, and that's something I wouldn't accept (I'm not even one of those everything-must-run-on-linux guys, it's just that the console market is very attractive lately ;)).

I don't see what's wrong with using CeGui or something similar. All common controls already written for me, works on both OpenGL and DirectX and if I need to, I can make it usable by gamepad.

-Markus-




Share this post


Link to post
Share on other sites
Quote:
Original post by Cygon
Take a look at the DXUT GUI framework from the DirectX SDK. It uses standard windows controls and draws them through Direct3D by hooking the OwnerDraw message. You can use the same approach for OpenGL or rewire the DXUT GUI to work on your abstracted renderer interface.

Anyway, you would lose all portability, and that's something I wouldn't accept (I'm not even one of those everything-must-run-on-linux guys, it's just that the console market is very attractive lately ;)).

I don't see what's wrong with using CeGui or something similar. All common controls already written for me, works on both OpenGL and DirectX and if I need to, I can make it usable by gamepad.

-Markus-


Oh, I should have said, this is for Windows O/S only. I am not writing this to be compatible with any other O/S.

I'll check out the DXUT GUI, thanks a lot :)

Share this post


Link to post
Share on other sites
I forgot to ask, would it be possible to do this with DDRAW overlays? I know it's old, but I kind a remember seeing an example that overlays a winform on top of a game and periodically checks whats typed in on the original textbox and updates the overlay.

So I could make my form in normal winform (like C#), hide it, overlay it in a game, track the mouse coordinates, and reflect it on the form.

The question is, is this method efficient? Is it widely used? and more importantly is it a good way to go? I also remember NVIDIA cards having some sort of hardware problem with alpha channels, and overlays not working.

So this should be something similar to the way xfire overlays work. With a mouse with a form, complete user interaction.

I just wanted to re-write what I wrote in the first post, because we kind of went off topic, I am not designing a game, I am trying to overlay a form on top of all existing games.

Share this post


Link to post
Share on other sites
I read and understood what you wrote, I was just trying to give you an easier idea. Getting Win32 to work on top of D3D/OpenGL window (which existing games use) will be difficult to do nicely in most cases. You'll spend time customizing the look of it too when you could just grab an existing UI and just go to it (unless you want default look). You will also have to tap into the game or driver somehow to stay on top of the game window.

Share this post


Link to post
Share on other sites
Quote:
Original post by Mike2343
I read and understood what you wrote, I was just trying to give you an easier idea. Getting Win32 to work on top of D3D/OpenGL window (which existing games use) will be difficult to do nicely in most cases. You'll spend time customizing the look of it too when you could just grab an existing UI and just go to it (unless you want default look). You will also have to tap into the game or driver somehow to stay on top of the game window.


The problem with that approach is as follows here:

1. My application will be using a skin which is defined by an xml file. Every single corner of it, every button color, the background image, on mouse hover colors, will be specified in an xml, parsed, and drawn. So there is no point of doing all those super complicated and advanced GUI libraries. a. I have no idea how to use them, b. I don't really care if they look good, after all my skin will cover it all:)

2. I need to do most of this stuff in .NET. My main hook library and everything is in C++. My question is, is it possible to do this by using both of them? I guess the point I am trying understand is "HOW" to do it, rather than "WHAT" to use. Because trust me, before I posted here, I already tried making a simple window, register it, and show it, but in game, it does not work. It either minimizes the game, or crashes the game. I need one simple example showing how I can overlay a simple form during game execution. From there, I can take over, but I need a sample or a source that shows you how to do it. I don't know whether to use ddraw or just pure win32 or directx. I don't know HOW to use them in this case.

I have access to device, present/clear/swapchainpresent/createdevice/createadditionalswapchain/reset methods, main HWND of the application. I can hook into more methods if I have to.

Once you create the window, how do you show it every time say Present() is called? Is that the right way to do? Is there anyway to a) Make a form (during game is running) in windows in the background, b) hide it, c) show it when a key is pressed?

I already have my nice fps counter like graphics working with this principle.

Edit: I also want to ask that why out of 173 views in this topic only 1 person responded? Should I have posted this in DirectX forums or gaming forums? Thanks.

Share this post


Link to post
Share on other sites
Is The Question going where no man has been before?

To me it seems so.

but I don't know, it just seems really difficult (working inside out... from graphics API to [hooking to] windows forms API etc).

(1)I would have started this as/with a windows form application.
Did you? Can you? It seems as if that was your original aim, what happened?
(2)Then custom skinned it(how do you do that? : research)
(3)Then Applied the graphics API (directx/opengl) to a Control ( e.g. Forms::Panel is a good one).
(4) Find out if 2 and 3 above compatible?

Bit like a sandbox or game/model/terrain editing tool?

Do you see that as possible?
Would you consider:
Can you/have you modularized you existing code enough to be able to create your app as a windows form application? Its just that I think if you could, then you would have a well known/used/understood base on which to continue to build your app with tons of typical ref material.

I don't think I have really answered your question but...its just that if you are not beyond the point of no return you could do (1) to (4) above and perhaps avoid what seems to be a difficult path.
Maybe you have already looked at (1) to (4) already deemed it not viable?

or maybe I do not quite understand your aim:
Is it to make you own version of an xfire type application?

Share this post


Link to post
Share on other sites
Quote:
Original post by steven katic
Is The Question going where no man has been before?

To me it seems so.

but I don't know, it just seems really difficult (working inside out... from graphics API to [hooking to] windows forms API etc).

(1)I would have started this as/with a windows form application.
Did you? Can you? It seems as if that was your original aim, what happened?
(2)Then custom skinned it(how do you do that? : research)
(3)Then Applied the graphics API (directx/opengl) to a Control ( e.g. Forms::Panel is a good one).
(4) Find out if 2 and 3 above compatible?

Bit like a sandbox or game/model/terrain editing tool?

Do you see that as possible?
Would you consider:
Can you/have you modularized you existing code enough to be able to create your app as a windows form application? Its just that I think if you could, then you would have a well known/used/understood base on which to continue to build your app with tons of typical ref material.

I don't think I have really answered your question but...its just that if you are not beyond the point of no return you could do (1) to (4) above and perhaps avoid what seems to be a difficult path.
Maybe you have already looked at (1) to (4) already deemed it not viable?

or maybe I do not quite understand your aim:
Is it to make you own version of an xfire type application?


You even confused me more lol:) I am not trying to do what xfire does. Why would I waste my time for something that is already out there:). But because what I am doing is so complex and hard to explain, the best example/the closest example to what I am doing is xfire. Like they do, I have to show my form, but instead of typing messages in it, I will show graphs, statistics (CPU temperature/GPU Temperature, etc).

You think this is hard? Try getting the RDMSR instruction to execute to get CPU temperature, then try getting PCI bus information for GPU temperature, and about 55 other things (information) about hardware:) (This is phase 1), then also try writing a hook that will never crash, but will hook into any game for dx8,dx9,ogl,dx10, both 32 bit and 64 bit, both XP and Vista compatible:) (This is phase 2). I did both 1 and 2 in about 4 months, I believe after what I saw in there, nothing is impossible:)

Now I'm in phase 3, show a window, very simple window, in a game:)

There has got to be a way, come on, if Columbus said "too hard", what would we do?:)

Share this post


Link to post
Share on other sites
yeah...I would love to hear from someone that has a robust example or solution too!
More curious would be the reason given for such a rare/unusual approach
The rare/unusual approach could be given as an explanation for the lack of responses that concerns you so far: Re:
Quote:

Edit: I also want to ask that why out of 173 views in this topic only 1 person responded?...
...The question is, is this method efficient? Is it widely used? and more importantly is it a good way to go?...




in the meantime...
using xfire as an example as you do...I must have misled myself.

I guess the impression xfire gives me is much more simple than the one it gives you: The evidence suggests it to be a standalone app [with typical custom skin(s)] that has no direct connection to any game or application on your system. It interogates the system info(a little like task manager) and the internet data packets sent to your computer for all the stuff it needs/shows. ?

Is there evidence to the contrary?

[Edited by - steven katic on December 7, 2007 9:37:17 PM]

Share this post


Link to post
Share on other sites
Quote:
Original post by steven katic
yeah...I would love to hear from someone that has a robust example or solution too!
More curious would be the reason given for such a rare/unusual approach
The rare/unusual approach could be given as an explanation for the lack of responses that concerns you so far: Re:
Quote:

Edit: I also want to ask that why out of 173 views in this topic only 1 person responded?...
...The question is, is this method efficient? Is it widely used? and more importantly is it a good way to go?...




in the meantime...
using xfire as an example as you do...I must have misled myself.

I guess the impression xfire gives me is much more simple than the one it gives you: The evidence suggests it to be a standalone app [with typical custom skin(s)] that has no direct connection to any game or application on your system. It interogates the system info(a little like task manager) and the internet data packets sent to your computer for all the stuff it needs/shows. ?

Is there evidence to the contrary?


Xfire uses a CBT global system wide hook to hook into every single process and injects in xfire_touchanXXXXX.dll file. You can do a simple listdlls on xfire.exe and see it yourself.

It then hooks into DirectX api and patches the Reset() function within DirectX for games to detect the event of a device lost/alt+tab to set/get focus.

I know about the Reset() function hook because I was hooking into the same function and everytime I start my program It'd crash if xfire or fraps was running. Doing a bit research showed that both Fraps and Xfire do the same thing.

However I don't know whether xfire uses DirectX API to draw on the screen or not, but I highly doubt that they do that. If you ever used it, you'd understand it's a simple skinned winform. In fact, if you set the skin to Default in xfire, you will end up (within the game itself) with a simple windows form.

I have no idea how they are doing it, in fact I don't want to know how they are doing it, I want to learn myself. But for me to learn, I need to have a starting point. The question is not which library to use or what to draw, the question is how to do it.

So if you were a DirectX game programmer, and your boss gave you a task that included creating a child MDI window within the game (like asking for password) which benefits from current windows controls such as listboxes or textboxes, you are telling me it's hard to do? It's really not making sense, I thought this stuff would be very trivial.

Share this post


Link to post
Share on other sites
Quote:

you are telling me it's hard to do? It's really not making sense, I thought this stuff would be very trivial.


sounds like frustration to me.
We always want thing to be easy, that's natural, and I agree, you would think that it would be trivial at first thought. But as time goes by...

(1)The problem reminds me of 'hacks', or more respectably 'solutions', that were invented to call/derive/have bits and pieces of MFC in a win32 app. The brittleness/interoperability problems seem to be similar.

Just because a solution may be possible and works doesn't mean that it will not work at some stage (or break very easily). We all want a robust/reliable solutions, otherwise....what?

(1) above would not be an acceptable long term solution in my software
(There was a time when I have been oblidged to do it commercially).

Commercially, (1) has been a solution, where timeliness takes priority over quality ( quality being still a monumental problem to be solved in the software engineering industry on the whole)

Anyhow, enough useless waffle, I have been researching for you (how 'bout that?)...no not really...I am just curious to see if the idea in (1) above can be confirmed in my mind (yes, my hidden agenda is not so...er... hidden).

Two things that may interest you (that I hope you can see as significant) are:

http://www.ddj.com/windows/184406078

and maybe:

http://www.microsoft.com/communities/newsgroups/en-us/default.aspx?dg=microsoft.public.dotnet.framework.windowsforms&tid=60c9aa39-c949-460d-80b5-feb59c35d758&p=1

[too lazy to link]

You could do more/up-to-date research with better results.

Quote:

...you are telling me it's hard to do? It's really not making sense, I thought this stuff would be very trivial.



Well, if it were trivial, the first reply would have contained a solution(unless people are willingly not helping you - I cannot imagine you have that many enemies ;))

But you seemed to imply that was not likely (?).. when you opened the thread with:
Quote:

I have a very simple question but possibly with a very complex answer....



Quote:

So if you were a DirectX game programmer, and your boss gave you a task that included creating a child MDI window within the game (like asking for password) which benefits from current windows controls such as listboxes or textboxes,


I think I've made my position quite clear without a response to that scenario; it would cause me to contribute more useless waffle to the thread.

What we want is a solution:
Come on someone... show us a good robust,... trivial solution!

I keep imagining it happening because then I do not have to think about your thread and saying to myself "quick get out of the hole you are digging before it gets too deep to climb out...and... grab that rope ladder (long enough) if you are game enough to go back in.

In the meantime hope...and hopefully not in vain.

Cheers

[Edited by - steven katic on December 9, 2007 3:47:54 AM]

Share this post


Link to post
Share on other sites
Just an update on this, the only way to do this is DDRAW overlays. There is no other "general" way to do it. It's just not possible. 3 days of non stop research also helps.

Basically it all goes down to this:

1. Make a general overlay library, overlay your winform and update it.
2. Make a separate GUI for every single interface - dx8,dx9,opengl32,dx10.

Obviously nobody wants to do #2.

I already saw several examples of overlays but unfortunately most of them don't work, the ones that do are not stable and the library they use (DDRAW) is very very very old. This'll be a very tough one, but we'll see.

Share this post


Link to post
Share on other sites

This topic is 3660 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this