Jump to content
  • Advertisement
Sign in to follow this  
XTAL256

Problem with loading DLL / SetWindowsHookEx

This topic is 2985 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

I am baffled as to why this doesn't work. I have created a DLL and am trying to load it using the code below.
/*** Called from my main executable "Window Detective.exe" ***/

HOOKPROC hookProc;
HINSTANCE hinstDLL;
HHOOK hhook;

hinstDLL = LoadLibrary(L"WD_Hook.dll");
hookProc = (HOOKPROC)GetProcAddress(hinstDLL, "hookProc");
hhook = SetWindowsHookEx(WH_GETMESSAGE, hookProc, hinstDLL,0);

// Send WM_NULL to all windows to force loading
// ...



/*** DLL code ***/

/// header file
#ifndef WD_HOOK_H
#define WD_HOOK_H

extern "C" {

#ifdef WD_HOOK_EXPORTS
  #define WD_HOOK_API __declspec(dllexport)
#else
  #define WD_HOOK_API __declspec(dllimport)
#endif


WD_HOOK_API LRESULT hookProc(int code, WPARAM wParam, LPARAM lParam);

}

#endif   // WD_HOOK_H

/// cpp file
#include "stdafx.h"
#include "Hook.h"

// Shared data
// Seen by both the instance of this DLL mapped into the remote
// process as well as the instance mapped into our exe
#pragma data_seg (".shared")
#pragma comment(linker,"/section:.shared,rws")
 HHOOK hook = NULL;
#pragma data_seg ()

BOOL APIENTRY DllMain(HMODULE module, DWORD reasonForCall, LPVOID reserved) {
    return TRUE;
}

// TODO: Catch messages here and do stuff with them
LRESULT hookProc(int code, WPARAM wParam, LPARAM lParam) {
    return CallNextHookEx(hook, code, wParam, lParam);
}


I have managed to get the DLL to load into my process (the one that calls this code), but it will not load into other processes. Now here is the thing i don't get. I use Smalltalk at work, so, as a test, i tried the above code in my Smalltalk IDE and it worked! It is essensially calling the same Windows functions, but for some reason it does not work when i do it in my project. I am using C++ with Qt (compiling with qmake and nmake) but i don't see how that could make a difference. My project is Unicode, whereas the Smalltalk VM is ASCII. But again, i don't think that would matter. I really can't give you any more information because i don't know what else i can try. Hopefully someone has an idea as to why this is not working...

Share this post


Link to post
Share on other sites
Advertisement
What operating system are you using (both at home and at work)? What permissions does your program have?

Share this post


Link to post
Share on other sites
Actually, both cases were tested at work, in an XP virtual machine.
I also thought about permissions and i was going to try running as admin but i forgot. I just tried that now (at home) but it still didn't work. I'm not sure what other permissions the program could have, but i doubt they would be running with different permissions.

Share this post


Link to post
Share on other sites
Hmmm ... It should work on XP, I was more concerned about the UAC in Vista/Win7 silently blocking your program from injecting.

I don't know if your program needs debug privileges to set a hook like that, but just in case here's a snippet (not mine, but I don't recall where I got it) I use before modifying other processes (it's required, at least, if you want to OpenProcess something with PROCESS_ALL_ACCESS in XP+):
void EnableDebugPriv() {
HANDLE hToken;
LUID luid;
TOKEN_PRIVILEGES tkp;

OpenProcessToken( GetCurrentProcess(), TOKEN_ADJUST_PRIVILEGES | TOKEN_QUERY, &hToken );

LookupPrivilegeValue( NULL, SE_DEBUG_NAME, &luid );

tkp.PrivilegeCount = 1;
tkp.Privileges[0].Luid = luid;
tkp.Privileges[0].Attributes = SE_PRIVILEGE_ENABLED;

AdjustTokenPrivileges( hToken, false, &tkp, sizeof( tkp ), NULL, NULL );

CloseHandle( hToken );
}

Share this post


Link to post
Share on other sites
Thanks for that code snippet Amrazek. I'm not sure where i should use it but it didn't work when i put it before the SetWindowsHook code or in the DLL itself.
I don't know what else to try.

Share this post


Link to post
Share on other sites
How do you know your DLL isn't loading into the other process? Do you ever get your DLL_PROCESS_ATTACH handler hit (You can shove in an OutputDebugString and pick it up with DebugView).

Do you get anything interesting showing up in Process Monitor?

Share this post


Link to post
Share on other sites

LRESULT hookProc(int code, WPARAM wParam, LPARAM lParam) {
return CallNextHookEx(hook, code, wParam, lParam);
}




You forget the WINAPI/CALLBACK part on the declaration of hookProc, it won't cause the dll not to load but it wont endear it to the target process when it unbalances the stack. Unless you're compiling with stdcall as the default convention that is, if so, ignore me.

This, along with Steve's suggestion might help you figure out what is going wrong. Since you're injecting into every app, open notepad and target that for the least debug spew.

Share this post


Link to post
Share on other sites
Thank you all for your help. I wouldn't know what to do without you [grin].

@Evil Steve: I am looking at the DLLs pane in Process Explorer to determine if it is loaded in other processes. When i tried in Smalltalk, i could see the DLL being injected into a windowed process as soon as it receives a window message (as the MSDN docs say).
I will also try the two things you suggested.

@adeyblue: Thanks for the link, I will have a look at it later. I will also use the WINAPI declaration.

Share this post


Link to post
Share on other sites
I put some OutputDebugString's in DllMain and it did not show it being attached to any process other than my own. Here is a portion of the output if anyone is interested (my debug strings are the dll attach flags):

...
...
'Window Detective.exe': Loaded 'C:\WINDOWS\system32\ieframe.dll'
'Window Detective.exe': Loaded 'C:\WINDOWS\system32\iertutil.dll'
'Window Detective.exe': Loaded 'C:\Documents and Settings\...\Window Detective\WD_Hook.dll', Symbols loaded.
DLL_PROCESS_ATTACH
'Window Detective.exe': Loaded 'C:\WINDOWS\system32\wdmaud.drv'
'Window Detective.exe': Loaded 'C:\WINDOWS\system32\wintrust.dll'
'Window Detective.exe': Loaded 'C:\WINDOWS\system32\crypt32.dll'
'Window Detective.exe': Loaded 'C:\WINDOWS\system32\msasn1.dll'
'Window Detective.exe': Loaded 'C:\WINDOWS\system32\imagehlp.dll'
'Window Detective.exe': Unloaded 'C:\WINDOWS\system32\wdmaud.drv'
'Window Detective.exe': Loaded 'C:\WINDOWS\system32\wdmaud.drv'
DLL_THREAD_ATTACH
'Window Detective.exe': Loaded 'C:\WINDOWS\system32\msacm32.drv'
'Window Detective.exe': Loaded 'C:\WINDOWS\system32\msacm32.dll'
'Window Detective.exe': Loaded 'C:\WINDOWS\system32\midimap.dll'
DLL_THREAD_ATTACH
DLL_THREAD_ATTACH
DLL_THREAD_DETACH
The thread 'Win32 Thread' (0xd08) has exited with code 0 (0x0).
DLL_THREAD_ATTACH
DLL_THREAD_DETACH
The thread 'Win32 Thread' (0xb98) has exited with code 0 (0x0).
...
...

Share this post


Link to post
Share on other sites
Obviously the first thing i checked when it didn't work was the error code returned by GetLastError. And i'm sure it did not give my any errors when i checked.

But i just checked again now that i am at home and i get error 127 (ERROR_PROC_NOT_FOUND). I don't know why i didn't get this the first time, and actually, this is after i added CALLBACK to the function's declaration, so maybe that has something to do with it.

I just looked at the function in Dependency Walker and it is defined as "_HookProc@12". Why is that? I am using extern "C" when declaring it, so it is not a C++ function. And when i checked Dependency Walker last time (i think it was before i added CALLBACK), the function was just called "HookProc"

Share this post


Link to post
Share on other sites
Sign in to follow this  

  • Advertisement
×

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

Participate in the game development conversation and more when you create an account on GameDev.net!

Sign me up!