Sign in to follow this  
wforl

Winapi debug

Recommended Posts

When debuging my code, or steping through it, i can only access the functions inside my wndprocess by leaving a breakpoint in each of the cases. When reviewing code, i like to not have any breakpoints and step through the code by pressing f11, (without pressing start debug), so that i see each line of code as it runs through the program. But this method doesn't to show any of the cases being used in wndprocess, once it enters this loop in WINMAIN, it just circles it over and over. I know that this is suppose to happen, and that wndprocess is being accessed, but how do i get to step into the wndprocess function, so that i can see the code going through the cases specified by the messages in the queue?
 while (GetMessage (&msg, NULL, 0, 0))
     {
          TranslateMessage (&msg) ;
          DispatchMessage (&msg) ;
     }
     return msg.wParam ;

Share this post


Link to post
Share on other sites
You'll have to set a breakpoint in the WndProc.

I think you'll quickly find that stepping through every line of code in a GUI program will quickly become tedious, especially with regard to message loops and WndProcs. For example, there are several messages sent during the creation of a window. You'll have to step through all of them even if you're only interested in stepping through WM_CREATE. It might take some time, but with a little practice you'll get the hang of which messages are important to check and which are not.

Share this post


Link to post
Share on other sites
hi lesbeard

i tried putting a breakpoint in one of the cases from wndprocess, but when debugging and it enters a case, say WM_PAINT, its runs through it fine, but then when it comes to the end of that case, (return 0; ), it brings up an error saying it cant return to the source code, and i have to stop debugging.

Share this post


Link to post
Share on other sites
Some debuggers won't let you set a breakpoint on a case statement. You might have to declare a "dummy" variable and initialize it just to set the break point on that statement.

eg.

int x = 0;

As for the error you're getting, just as you weren't able to step into the WndProc, you can't step out of it and back into the message loop. At that point try letting the program continue and see what happens.

Share this post


Link to post
Share on other sites
It is possible to step through an entire windows gui program, however, it requires using a debugger capable of transitioning from user mode into kernel mode and back into user mode and user mode debuggers don't do that. In addition, stepping through an entire program like that would have to be done entirely in assembly language and assembly language is not as compact as C or C++. It can be done in SoftIce, but getting SoftIce to run on Windows XP is difficult. It might be doable in Microsoft's kdebug or some other kernel debugger, but I haven't used those very much.

In case you weren't aware, user mode is the operating mode that Windows GUI programs run in. Kernel mode is the realm of the operating system, full of device drivers and other kinds of software that interface directly with the hardware. There is a lot more to these subjects than I'm relating so consider all this a broad overview.

Stepping through an entire GUI program is not possible in a user mode debugger because at various points, user mode gui programs "dip" into kernel mode. Console programs do too, but such calls are typically "stepped over" rather than through and that hides the transition. The difference between the two are the callback functions that are more prominently used in Windows GUI programs.

Anyway, let's unpack the message loop a little bit to flesh out what happens and why you can only access the wndproc with a user mode debugger.

The first reason is that you don't have access to the source code for various WinAPI functions. A debugger could disassemble API calls allowing you to step from C/C++ code into ASM and back again, but most of them don't do that opting instead to treat API calls like black boxes. You feed them inputs, they provide you with results (good or bad results depending on the correctness of the inputs per the "garbage in garbage out" principle). A user mode debugger "steps over" these calls.

Taking this into account, let's run through your simple message loop in a rough thumbnail sketch.


while (GetMessage (&msg, NULL, 0, 0))
{
TranslateMessage (&msg) ;
DispatchMessage (&msg) ;
}


It's easy to think that message queues belong to window instances, but they actually belong to threads. Note the first sentence from the MS documentation for GetMessage: The GetMessage function retrieves a message from the calling thread's message queue.

In the case of a simple program, the thread is the main thread of execution and the thread function is WinMain. You don't really need to know more about thread message queues at this point. What you need to know here is that this call to GetMessage fills out the MSG structure passed to it.


typedef struct tagMSG {
HWND hwnd;
UINT message;
WPARAM wParam;
LPARAM lParam;
DWORD time;
POINT pt;
} MSG,*LPMSG,*PMSG;


You don't manipulate the members of this structure. I just pasted it here to make it explicit. One of the things GetMessage does is supply the HWND associated with the message as well as the WPARAM and LPARAM. Note that along with the message, these are the arguments to the WndProc. If you think about what passing a message to a program involves, for example, clicking a button on the screen, you'll see that the operating system is involved at some point. The operating system does the work associating a mouse click on a particular spot on the screen with a particular program and not some other program hidden from view.

If you took apart GetMessage and traced through it's code, at some point you would see it "ask the operating system" to check the running thread's message queue to see if there are any messages available. I put "ask the operating system" in quotes because what really happens involves calling into kernel mode. A user mode debugger could "step over" such calls, but rather than disassembling GetMessage and step through every instruction, it's just more straightforward to step over the call to GetMessage. Considering that GetMessage is part of the condition for the main execution loop, if you stepped through it, you'd be spending a lot of time stepping through that function - again and again and again. Ugh! What a drag!

TranslateMessage isn't vital to this discussion so I won't remark on it other than to say it manipulates some messages putting them into a form more amenable to the WndProc.

DispatchMessage is where the fun really begins. This function takes the hwnd from the msg and uses it to obtain a function pointer to the WndProc, probably using an internal api similar to GetWindowLong (note the GWL_WNDPROC index). It takes that function pointer and calls it passing in the parameters (hwnd, message, wparam, lparam). When the WndProc finishes processing the parameters it returns to DispatchMessage which performs any clean up work it needs to do before itself returning to the message loop.

So if you were going to step through this, you'd need to step through the disassembly of DispatchMessage, into the WndProc and then back into DispatchMessage. And because most of that code is "fixed" (it's "black box" and isn't going to change), there's not much point to stepping through it with each pass through the loop.

Now, it would seem to make sense for the debugger, when stepping over DispatchMessage, to jump to the source code for the WndProc and from there jump back to the return in the message loop, but debuggers aren't typically set up to do that because of the complexity of the task. A debugger would have to examine the hwnd from the message at runtime, extract the address for the WndProc for that hwnd and then somehow know which snippet of source code in the IDE is associated with that address so that the source view could jump to the source code for that WndProc. And since a thread message queue could have messages for many different hwnds each with a different WndProc, that would be quite a task. In the end, it's just easier to require the programmer to set a breakpoint inside the WndProc he or she wants to step through.

Anyway, I hope that clarified rather than confused.

Share this post


Link to post
Share on other sites
Thanks for taking the time to write that Lessbeard,

pressing contiue in the case works, so i will continue to use that,


Share this post


Link to post
Share on other sites
I've been thinking about this some more and with some work it's doable. Check the documentation for your debugger, maybe there's a setting that needs to be enabled in order to step through an API into a callback function and back again.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this