Skip to main content
GameDev.net gamedev.net
🔒 Locked

[C++] Direct3D hooking sample

Started by Muhammad Haggag Nov 23, 2005 at 5:55 AM 31 replies 199.2k views
Original Post
Muhammad Haggag
Muhammad Haggag
Intro This sample demonstrates the second technique that can be used to render content on-top of other Direct3D applications. The first technique, overlays, is rather general purpose; it is not tied to Direct3D. You can use overlays to render on-top of everything. This technique, on the other hand, is closely tied to Direct3D. Due to its special-purpose nature, you lose some benefits and gain others. What you lose is that you can only draw on-top of Direct3D9 applications. What you gain is that you become Direct3D itself, so you're not limited to just drawing anymore. You can do things like count the number of batches submitted per frame, the number of triangles, replace textures, etc. In this sample, we use Windows hooks to inject our DLL code into a specific process - the target process - that uses Direct3D9. Our injected code replaces Direct3DCreate9 with our own function that calls the standard Direct3DCreate9, but returns our implementation of IDirect3D9, MyDirect3D9. MyDirect3D9 stores a pointer to the original IDirect3D9 interface, so you can do anything you like by forwarding the function calls you don’t want to change and rewriting the ones you want to. The actual hooking is done using APIHijack, with very minor modifications to make it build under VC++ 2003 and 2005. Namely, a couple of reinterpret_cast’s here and there. The sample is divided into 2 projects: A DLL containing interface implementations and APIHijack, and a launcher console application. Visual C++ 2003 and 2005 solutions are provided. The launcher is a simple Unicode application that sets up the single piece of information that the DLL needs: The target process name. In order to make things simple, the launcher is passed the target process name (i.e. exe name) as a command-line argument. It then creates or updates a registry key, HKEY_CURRENT_USER\Software\Direct3D-Hook, to store the name in its default value. The non-Unicode DLL, whose DllMain is entered whenever a process is loaded, compares the name of the loaded process to that in the registry and hooks Direct3D if they’re the same. It also gives an info beep on hooking. The DLL was not build with Unicode because APIHijack isn't, and I'm not interested in converting it. Feel free to do so if you want. I’ve only provided implementations for the interfaces IDirect3D9 and IDirect3DDevice9. You can implement any other interfaces if you like, provided that you modify the functions that create them to return a pointer to instances of your implementation. Keep in mind, though, that it’s a quite boring process. I nearly lost an arm and a leg writing the aforementioned implementations, even though I’ve copied the function declarators from the d3d9.h header file. You’ll understand when you see the code. [smile] The IDirect3D9 interface implementation is rather minimal, so it forwards all function calls except: 1. Release. We call IDirect3D9::Release and check its return value. If its zero, we delete the MyDirect3D9 instance. If we don’t do this, we’ll leak memory. 2. CreateDevice. We create a normal device but return our own implementation of IDirect3DDevice9, MyDirect3DDevice9. The device implementation, again, forwards all of the function calls except: a) Release, for the same reason as above. b) EndScene. This is where we can do all the rendering we want. The sample renders a colored quad to the top-left of the render-target. Limitations Applications that manually load D3D9.dll via LoadLibrary, retrieve a pointer to Direct3DCreate9 using GetProcAddress and call it will NOT be hooked. All Direct3D SDK samples do this. If you really want to hook them, you can probably do it in another way, by providing your D3D9.dll and putting it in the same folder. I might try that out later. Testing To test the sample, I used 2 applications. A build of Direct3D tutorial #2, “Vertices”, was used first. It worked fine, and it is included in the sample. To test it in a real-world scenario, though, I used the brilliant, addictive game: “Egyptian Addiction”. Yes, this is free advertising. Go give that game a try, it rocks. No, it’s not developed by Egyptians. The real-world test was useful in that it reminded me that I had to set the required render and texture stage states first. Don’t forget that there’s someone playing with the settings before you [smile]. The sample doesn’t restore the state after modifying it, assuming the hooked process sets its required states every time. This can break some applications where the application writers didn’t expect their states changing behind their back. You’ll need to cache all state changes on all the state changing calls, and revert to the cached ones after rendering if you’re dealing with such an application. License My code in this sample is public domain, do whatever you wish with it. I can’t say the same for the APIHijack library, however. The author does not mention any specific license in his code or article, so you’d probably want to contact him if you’re doing a commercial project. PICTARS!! Finally, screenshot time!

dave
dave
This is great and much needed since there have been so many "How do i's...".

PRAISE TO YAW!!!
jollyjeffers
jollyjeffers


I've never had the time to look into API hooking in much detail, but I've always been curious how it actually worked. I shall be sure to have a look through your code as/when I get some time [smile]

Cheers!
Jack
<hr align="left" width="25%" />
Jack Hoxley <small>[</small><small> Forum FAQ | Revised FAQ |
circlesoft
circlesoft
Niiiiiice, this is quality work. If I only had a dollar for each time I answered a "hooking" question, I would be a BAZILIONIONAIRE!!! Gotta love the tediousness of inheriting IDirect3DDevice9...there are only like 14 trillion member functions.
Muhammad Haggag
Muhammad Haggag
Quote:
Original post by circlesoft
Niiiiiice, this is quality work. If I only had a dollar for each time I answered a "hooking" question, I would be a BAZILIONIONAIRE!!!

No need to be modest, we all know you already are!

Quote:
Gotta love the tediousness of inheriting IDirect3DDevice9...there are only like 14 trillion member functions.

Yes, seriously. I almost broke my fingers on this one.

Leadorn
Leadorn
I did something similar with IDirectPlay8Client. Took a while getting it to work but now i have nice send/recv sniffer for combat flight simulator 3 and all other games using directplay8client.

Promit
Promit
Wow, this is awesome. I intend to have some fun with this. I enjoyed hooking OpenGL and writing wallhacks for HL1 and its mods*. Let's see what I can do with this. And, kudos for typing out all those functions. There sure are a lot...

* Of course I don't play publically with the hacks. I rather happen to enjoy writing them, is all.
SlimDX | Ventspace Blog | Twitter | Diverse teams make better games. I am currently hiring capable C++ engine developers in Baltimore, MD.
jollyjeffers
jollyjeffers
I'm wondering if there is some sort of image-analysis tool that can be built on top of this sort of technology.

If you could hook a commercial game and count the number of shaders used, number of passes, overdraw complexity and so on... would be an interesting thing to mess with if nothing else.

Make it work well and the hardware-nuts would love it - they spend enough time talking about clockspeeds and megahurtz that I'm sure they'll have good fun with a screenshot of the shader-complexity of a commercial game [lol]

Jack
<hr align="left" width="25%" />
Jack Hoxley <small>[</small><small> Forum FAQ | Revised FAQ |
circlesoft
circlesoft
Quote:
Original post by jollyjeffers
I'm wondering if there is some sort of image-analysis tool that can be built on top of this sort of technology.

If you could hook a commercial game and count the number of shaders used, number of passes, overdraw complexity and so on... would be an interesting thing to mess with if nothing else.

Make it work well and the hardware-nuts would love it - they spend enough time talking about clockspeeds and megahurtz that I'm sure they'll have good fun with a screenshot of the shader-complexity of a commercial game [lol]

Jack

That would be quite easy...just hook into IDirect3DDevice9::SetVertexShader() and IDirect3DDevice9::SetPixelShader(). Then count up how many changes you have per frame. You can do *all* kinds of performance analysis like that.

p.s. IIRC the effect framework still uses Set*Shader() in its underlying implementation
Promit
Promit
Maybe I'm missing something here, but isn't that exactly what PIX does?
SlimDX | Ventspace Blog | Twitter | Diverse teams make better games. I am currently hiring capable C++ engine developers in Baltimore, MD.
DirectXFreak
DirectXFreak
Wow! This is really cool!
You should make this an article =P

[Edited by - DirectXFreak on November 28, 2005 5:07:07 PM]
--m_nPostCount++
circlesoft
circlesoft
Quote:
Original post by Promit
Maybe I'm missing something here, but isn't that exactly what PIX does?

D'oh...

I just tried that using PIX on GTA: San Andreas. Some sample stats:

DrawPrim............32DrawIndexedPrim.....160VB Locks............45IB Locks............13SetRenderState......163SetTexture..........190SetVertexShader.....6SetPixelShader......0SetRenderTarget.....12


Considering that this was a really simple scene (just inside the main characters house, very low complexity) I think those numbers are high.
jollyjeffers
jollyjeffers
Quote:
Original post by Promit
Maybe I'm missing something here, but isn't that exactly what PIX does?

Yeah, essentially... I have to admit I haven't tried it, but you can't get PIX to generate an overdraw of the image, can you? deep pixel analysis yes, but not shader complexity and so on..?

The idea came from this thread. It'd be pretty awesome if you could somehow display the number of pixel shader arithmetic or texture instruction count per-pixel. Could make for a great performance debugging tool - the brighter the colours the more fill-rate related work you're doing etc...

Then again, hacking that in via hooking stands as much chance of breaking things as it does working [lol]

Oh, and the obvious advantage is that it'd be overlayed - giving immediate feedback rather than the capture full stream then analyse the data.

Jack
<hr align="left" width="25%" />
Jack Hoxley <small>[</small><small> Forum FAQ | Revised FAQ |
circlesoft
circlesoft
Quote:
Original post by DrGUI
Why might you need that many VB and IB locks per frame?
Was there a legendary particle system in the house or something?


Yea, and this is inside a really simple house. I did some more investigating, and profiled a complex scene outside:

DIP Calls:  1300 - 2300 (:O !!!)VB Locks: 120-145IB Locks: 50-75New VBs: 1-5New IBs: 1-7FPS: 10-20


Yea...so I wonder really what the frick is going on. Looks like someone at Rockstar needs to read the renown instancing slidedeck...
Person
Person
If I use that sample than AOE 3 hangs.
And nothing happened in COD 2.
Jason Z
Jason Z
Oh my. This could be used for some serious hacking of applications. Really, you could trace exactly how the api is used - frankly I think it is pretty cool but I am surprised that MS allows this type of thing to be done.

I think I am going to try a few experiments with my own engine and see if this can be used for some type of runtime analysis of my rendering calls. Great work, I appreciate your effort (and typing abilities!).
Jason Zink :: DirectX MVP   Direct3D 11 engine on CodePlex: Hieroglyph 3 Direct3D Books: 
circlesoft
circlesoft
Quote:
Original post by Jason Z
Oh my. This could be used for some serious hacking of applications. Really, you could trace exactly how the api is used - frankly I think it is pretty cool but I am surprised that MS allows this type of thing to be done.

I think so, since they full support the ultimate "hacking" application - PIX. PIX makes it easy to analyze exactly what an application is doing, D3D-wise. IMO, after doing some trials on commercial games, it is more helpful than the hooking approach for profiling.
Jason Z
Jason Z
Holy Crap!

I just ran a test run of PIX on the latest build of my game. I tried using PIX long ago when it first came out and wasn't very impressed, but it has really evolved quite a bit from what I remember.

The object window alone is worth using, not to mention the frame rendering stream. So this will work with any D3D9 commercial application? I guess once you ship something you are essentially allowing everyone to see what you are doing - it kind of opens up the doors to smaller developers (like me!) to see what the big boys are doing.

Thats pretty cool.
Jason Zink :: DirectX MVP   Direct3D 11 engine on CodePlex: Hieroglyph 3 Direct3D Books: 
circlesoft
circlesoft
Quote:
Original post by Jason Z
The object window alone is worth using, not to mention the frame rendering stream. So this will work with any D3D9 commercial application? I guess once you ship something you are essentially allowing everyone to see what you are doing - it kind of opens up the doors to smaller developers (like me!) to see what the big boys are doing.

Yea pretty much - they have worked on all commercial games I have tried it on (HL2, Farcry, GTA:SA). If you read a few posts above, I show some figures about GTA that are a little disturbing [crying]

Topic Locked

This topic has been locked by a moderator. New replies are not allowed.

Sign in to reply to this topic.