DirectX Hooking

Started by
11 comments, last by TheAdmiral 16 years, 10 months ago
Hello, New here :), I wanna try to hook to games runing DirectX to get their current front buffer (As I understand its the Image currently displayed on the screen by the engine) and save it, so it will be a screenshot of the game. I have no idea how to hook to other games thus I never tried it. And most of my programming expierience is network programming so I have no clue. Now where can I find information on that subject? And if its possible in VB6 as it the language im currently using for my almost finished project? And has anyone ever tried it here ? Thanks ahead.
Advertisement
Some games will be easier to hook than others. It ultimately depends on how they initiate the connection to the DirectX DLLs.

To get you started on the subject, there's a DX hooking sample, but it's written in C++. Still, it should be enough to get you started on understanding what things you need to be doing.

As for VB6, I'm not sure how possible that will be.
Sirob Yes.» - status: Work-O-Rama.
Are you talking about Direct3D or DirectDraw?

I've never had to hook a DirectX program, but have some more general experience. I can't think of a more direct manner than to inject a hook for IDirect3DDevice9::Present (or IDirectDrawSurface::Flip, but I'll assume the former), and grab the front-buffer.

Perhaps somebody could suggest a simpler method, but I'm sure that what I'm about to describe will work. It does feel a little like a nut-cracking sledgehammer though.

For a simple local hook like this, you should inject some code into the target process, locate the Present method and hook it with a wrapper of your own. For the sake of brevity I won't discuss it, but I suggest you put an equal amount of effort and care into unhooking gracefully.

There are many variations on the theme, but I've found DLL-injection with IAT-hooking to be the most flexible and effective. There is a reasonable amount of low-level dword-pokery involved, but it's well within the capabilities of VB6, provided you're prepared to copy-paste all those Win32 API declarations [rolleyes].

You'll need to write a separate DLL containing the code that needs to execute inside the target process. I suggest you do this in C++, as it's the best tool for the job. Feasibly, you could write the code straight into the main (VB6) application and remotely load the exe as a secondary module, but it seems a little perverse.

1. Find the target process. If it's a windowed application, I've found that a sequence of calls to FindWindow, GetWindowThreadProcessId, OpenProcess does the trick. Use Spy++ or something similar to find the window class's name, and open the process with PROCESS_ALL_ACCESS.

2. Inject your DLL. I use the second technique described in the article I linked to above, which is very clever. That is, VirtualAllocEx a buffer into the target process and WriteProcessMemory the path-name of the DLL to this buffer (which you can determine using LoadLibraryW and GetModuleFileNameW). Next, trick the target process into loading your DLL in by calling CreateRemoteThread with the address of LoadLibraryW (which will be the same as in your process) and a pointer to the injected DLL path-name as the parameters.

3. Execute the injected code. You could use the CreateRemoteThread method like before, to execute a function at any time, or you can just rely on DllMain to do everything for you.

The idea is to compile a proxy-Present function into your DLL. This is made a little bit more difficult by the fact that it is a class member-function (and hence a thiscall). You'll need to create a function that reroutes the execution to where it needs to go (so the back-buffer is actually displayed), but do all your own business at the same time. Something like this:

__cdeclHRESULT PresentHook(IDirect3DDevice9 *thisPtr, // Take all the arguments as they appear in the original prototype    CONST RECT * pSourceRect,                  // with an extra this pointer (according to the thiscall specification for virtual functions)    CONST RECT * pDestRect,     HWND hDestWindowOverride,     CONST RGNDATA * pDirtyRegion) {    // Take your screenshot here, using thisPtr as the device pointer    return thisPtr->Present(pSourceRect, pDestRect, hDestWindowOverride, pDirtyRegion);}


As it stands, if this function is called in place of IDirect3DDevice9::Present, it will perform exactly the same way. The point is that you are now free to do whatever you like with the parameters. But before this is of any use, you'll need to:

5. Install the hook. This should be done in the injected DllMain. The most reliable method is to patch directly over the beginning of the DLL function, but this is really messy. If you're into that sort of sadistic business, writing a 'JMP PresentHook' at the address of IDirect3DDevice9::Present will do. Otherwise, a less soul-corroding method is to use an IAT hook:

Provided you know that Present will be called from the exe, it suffices to reroute the executable's import address table (IAT). Call GetModuleHandle, passing NULL as the module name. This will return the base address of the target exe. A call to VirtualProtect will give you read-access to the PE header. Now you can use a little pointer arithmetic (or the IMAGE_NT_HEADERS32 struct) to find the location and size of the IAT. Treating this chunk of memory as an array of 32-bit unsigned longs, perform a linear search for the address of IDirect3DDevice9::Present (which you can get with GetProcAddress). Once you've found it, simply overwrite it with the address of your hook.

Now, whenever the exe attempts to call Present, it will unknowingly find its way through your hook.

6. Take the screenshot. This should be pretty straightforward. Before your hook passes control to the real Present method, it should call CreateOffscreenPlainSurface, followed by IDirect3DDevice9::GetFrontBufferData. D3DXSaveSurfaceToFile will help you get the screenshot to disk.

And that's it; you're done. The process felt a whole lot shorter in my head [razz].

Edit: Stumbled upon this old post and spotted an error in my __thiscall emulation.

[Edited by - TheAdmiral on November 24, 2007 1:37:16 PM]
Ring3 Circus - Diary of a programmer, journal of a hacker.
Well, it sucks that you don't know C++. In pure managed code its impossible to do what you want.

- HOWEVER -

You COULD write a short hooking DLL in C++, or someone else could, and make VB6 do the rest of the application. Someone recently tried to make a DWM clone for Windows XP. Though its kind of... well... useless in principal (It doesn't work right in most cases, and FastAero which does is not released yet.) it does in fact do the hooking correctly (Just the developer had the wrong idea when he made it) and funnily enough, he happened to use VB6 as a GUI and interface to a DLL (And that DLL connects to the DLL that does the hooking.)

Just try to understand the code. If you can't try ignoring what you don't understand and ask questions when you fail. Remember, DirectX and OpenGL work in C not VB so its not possible to even use VB6 code to swipe data (safely, atleast.)

http://www.aeroxp.org/board/index.php?showtopic=5556

Since you need to be a member, i uploaded it for you.
http://www.megaupload.com/?d=WNMSM9ZL
// NMN, the spam bot!class nmn : public n00bs{  bool exists;  float nmn;  bool intelligent;  nmn()  {    exists = false;    intelligent = exists ? 0:1;  }  char *spam()  {    if(intelligent) return "Linux rules";    float r = rand()%100;    if(r > 50)    {      return "MS Sux";    }    else    {      return "Apple Sux";    }  }}  }}
Quote:Original post by SSkillZ
First and Second step I think I can do.
Third step ill rely on DllMain
Fouth Step is missing...
Yeah. The fourth step was to have a beer [rolleyes].

Quote:And whats the diffrence between Third and Fifth step?
The third step loads your DLL into the target process. DllMain is called implicitly, and so step 5 is triggered. Conceptually, though, step 5 is a completely different operation. Injecting the DLL is done from your application, whereas installing the hook is done by the DLL in the target process's address-space.
Putting it another way, without (3), (5) can't happen. Without (5), your injected DLL never gets to do its thing.
Quote:Those two functions can be called in DLLmain no?
You mean the functions responsible for taking the screenshot? Unless you can think of a clever way to acquire a pointer to the graphics device, no.

Quote:The dll will have a DLLmain function which exectues on DLL load and takes the screenshot and maybe save it to a file.
This won't work for two reasons. First, you can't call GetFrontBufferData without a pointer to the device, and second, even if you could call it, you wouldn't necessarily get a screenshot, as the scene may not have been rendered. On this note, the hooking function should rather look like:

{    HRESULT return_value = thisPtr->Present(pSourceRect, pDestRect, hDestWindowOverride, pDirtyRegion);    // Take the screenshot here, once the scene has been rendered    return return_value;}


Quote:how its done with OpenGL?
It's the same idea, except you don't hook IDirect3DDevice9::Present, but glFlush/glFinish.

Admiral
Ring3 Circus - Diary of a programmer, journal of a hacker.
This generic DLL hook is something I've had to use more than a couple of times. With a little foresight, I would have written a reusable hooking class. I have an exam tomorrow, so I hope this (very briefly tested) botch-job serves as something of a substitute:

int InstallHookByName(char* existing_module_name, char* existing_function_name, char* hook_module_name, char* hook_function_name){	HMODULE existing_module = GetModuleHandleA(existing_module_name);	if (existing_module == 0) return 0;	HMODULE hook_module = GetModuleHandleA(hook_module_name);	if (hook_module == 0) return 0;	void* existing = reinterpret_cast <void*> (GetProcAddress(existing_module, existing_function_name));	if (existing == NULL) return 0;	void* hook = reinterpret_cast <void*> (GetProcAddress(hook_module, hook_function_name));	if (hook == NULL) return 0;	// Calculate offset	DWORD ex_int = reinterpret_cast<DWORD> (existing);	DWORD hook_int = reinterpret_cast<DWORD> (hook);	DWORD offset = hook_int - ex_int - 5;	// Install the patch	DWORD new_protect = PAGE_EXECUTE_READWRITE;	DWORD old_protect;	if (VirtualProtect(existing, 5, new_protect, &old_protect) == FALSE) return 0;	// Let's hope that the existing function is at least five bytes long,	// or this patch will overflow	unsigned char* fn_ptr = reinterpret_cast<unsigned char*> (existing);	*fn_ptr = 0xE9; // JMP Opcode	DWORD* offset_ptr = reinterpret_cast<DWORD*> (fn_ptr + 1);	*offset_ptr = offset; // Relative jump	// Restore the access setting	VirtualProtect(existing, 5, old_protect, &new_protect);	return -1;}


It should be sufficiently general for your needs. For example;
InstallHookByName("Kernel32.dll", "CreateFileA", "User32.dll", "BeginPaint");
would overwrite the beginning of Kernel32!CreateFileA with a 'goto User32.BeginPaint'. Of course, this would promptly cause a crash on attempted creation of the file, but you get the idea. If any of the modules or functions are not found, the function fails, returning 0.

To only tricky bit to follow is generation of the relative JMP instruction - you'd need to know a little bit about x86 to fully understand it. I haven't covered removal of the hook, but it should be similarly done by saving the five bytes being patched and putting them back in place. Ideally, you'd do this in the screenshot-taking function, for a one-shot hook.

I don't need to to tell you that this is (necessarily) very hacky and won't port to anything other than 32-bit Windows (XP or 2000).

Admiral
Ring3 Circus - Diary of a programmer, journal of a hacker.
Quote:Original post by SSkillZ
Still this is really appreciated but I Don't know C++ so I can't continue from here,
Please help.

Sorry, I just don't think VB6 has the tools to get the job done. Even if it did, I don't really think anyone here knows how to do it.
Sirob Yes.» - status: Work-O-Rama.
Quote:Original post by sirob
Quote:Original post by SSkillZ
Still this is really appreciated but I Don't know C++ so I can't continue from here,
Please help.

Sorry, I just don't think VB6 has the tools to get the job done. Even if it did, I don't really think anyone here knows how to do it.

It does - I've had the displeasure of doing it before. VB6 is pretty capable of doing this low-level stuff - more so than VB.NET in my experience, but it's a real pain to get things done. In particular, any API functions being used need to be translated to VB's typing and explicitly declared. In this situation, that's a whole lot of declarations. Perhaps more cripplingly, there is no support for unsigned 32-bit integers, so dealing with DWORDS is a real pain, as you have to explicitly poke around with two's complement checks and bit-shifting.

Quote:Still this is really appreciated but I Don't know C++ so I can't continue from here,
Please help.
Huh? That's not exactly meeting us half-way. If you can program, you can at least understand C++. I'm not prepared to rewrite the code in VB6, I'm afraid, and I strongly discourage you from doing so. If you know VB, you should be able to pick up C++ with relative ease. If you have any specific problems, we'll be more than happy to help, but nobody is going to do the work for you.

Admiral
Ring3 Circus - Diary of a programmer, journal of a hacker.
You could feasibly have some success with this rudimentary system:

Send some Windows messages to the game, to force focus, wait a little while (until it has painted), grab a desktop screenshot as described here (in VB6) and try to manually crop the image using GetClientRect. I'm not sure how reliable this would be. There may be a better way to query the OS for the window's DC, but I don't know of one.

Admiral
Ring3 Circus - Diary of a programmer, journal of a hacker.
In the same way that many counter-strike hacks work, couldn't you create fake DirectX and OpenGL DLLs and place them in the program's folder so that the program loads your DLLs instead? Your DLL would forward all the API calls to the real DLLs, except for Present which might do some extra work like spit out the contents of the back buffer. Maybe you should look into how these hacks work.
....[size="1"]Brent Gunning

This topic is closed to new replies.

Advertisement