Sign in to follow this  
Headkaze

Sending mouse messages to hidden IE ActiveX control

Recommended Posts

Headkaze    607
I'm trying to send mouse commands to a hidden IE ActiveX control that I'm grabbing frames from and displaying on a texture.
CoCreateInstance(CLSID_WebBrowser, NULL, CLSCTX_ALL, IID_IWebBrowser2, (LPVOID*)&m_browser);
The first thing I tried was to use CAxWindow and it's SendMessageToDescendants() method. But the problem is when clicking the mouse button on the DirectX window rendering the texture the focus is lost from the ActiveX window so the MouseUp message is never received. What I did with the Flash ActiveX control was to use the IOleInPlaceObjectWindowless and it's OnWindowMessage() method and that works great. There is no focus problem using that. Eg.
m_flash->QueryInterface(IID_IOleInPlaceObjectWindowless, (LPVOID*) &m_windowlessObject);

LRESULT aResult;
m_windowlessObject->OnWindowMessage(WM_MOUSEMOVE, 0, MAKELPARAM(x, y), &aResult);
The problem is the IOleInPlaceObjectWindowless interface doesn't exist in the WebBrowser control. Can anyone suggest a way I can send mouse messages to a hidden IE control?

Share this post


Link to post
Share on other sites
Headkaze    607
Would this question have made more sense in the DirectX forum? I guess the only people who would have experience doing this are DirectX/OpenGL developers.

Share this post


Link to post
Share on other sites
Spk    145
Here's some code snipets from my embedded IE browser code. The idea is to remap clicks on the "texture rectangle" to clicks on the offscreen IE window using regular window messaging.

void sendMouseClick( HWND hwnd, HWND hwndTop, int x, int y )
{
SendMessage( hwnd, WM_MOUSEACTIVATE, (WPARAM)hwndTop, MAKELPARAM( HTCLIENT, WM_LBUTTONDOWN ) );
PostMessage( hwnd, WM_MOUSEMOVE, 0, MAKELPARAM( x, y ) );
PostMessage( hwnd, WM_LBUTTONDOWN, MK_LBUTTON, MAKELPARAM( x, y ) );
PostMessage( hwnd, WM_LBUTTONUP, MK_LBUTTON, MAKELPARAM( x, y ) );
}

HWND IEBrowser::getDocumentHWND() const
{
if( !m_browser )
return NULL;

CComPtr<IDispatch> docDispatch = NULL;
if( FAILED( m_browser->get_Document( &docDispatch ) ) )
return NULL;

// get_Document can succeed and still return a NULL IDispatch pointer for some reasons...
if( !docDispatch )
return NULL;

CComPtr<IHTMLDocument2> doc = NULL;
if( FAILED( docDispatch->QueryInterface( IID_IHTMLDocument2, (void**)&doc ) ) )
return NULL;

CComPtr<IOleWindow> docWindow = NULL;
if( FAILED( doc->QueryInterface( IID_IOleWindow, (void**)&docWindow ) ) )
return NULL;

HWND hwnd = NULL;
docWindow->GetWindow( &hwnd );
return hwnd;
}

And then when you need to simulate mouse clicks:

sendMouseClick( getDocumentHWND(), getHwnd(), x, y );

getHwnd() should return the handle to the window that contains your WebBrowser document.

You also need to make sure you remap the mouse (x,y) properly. If I remember correctly, they're relative to the browser's top left corner, so all you need is a function that maps the texture rectangle to the equivalent browser client area rectangle.

Did this make any sense?

Share this post


Link to post
Share on other sites
Headkaze    607
Thanks for your reply Spk. I see how your method would work because you are sending a mouse click all in one go. I thought that breaking the mouse down and mouse up events would allow for more control (ie. Dragging scrollbars) but the problem is when I click on my DirectX window focus is lost from the ActiveX window so instead of recieving WM_LBUTTONDOWN then WM_LBUTTONUP it only recieves WM_LBUTTONDOWN.

The reason I thought there was a way to get this working is I successfully had it working with a Flash ActiveX component like this

void CFlash::MouseMove(int x, int y)
{
LRESULT aResult;
m_windowlessObject->OnWindowMessage(WM_MOUSEMOVE, 0, MAKELPARAM(x, y), &aResult);
}

void CFlash::MouseDown(int x, int y)
{
LRESULT aResult;
m_windowlessObject->OnWindowMessage(WM_LBUTTONDOWN, 0, MAKELPARAM(x, y), &aResult);
}

void CFlash::MouseUp(int x, int y)
{
LRESULT aResult;
m_windowlessObject->OnWindowMessage(WM_LBUTTONUP, 0, MAKELPARAM(x, y), &aResult);
}


The problem is there is no IOleInPlaceObjectWindowless interface in the WebBrowser control. So instead of using a windowless object I am sending the mouse messages directly to the CAxWindow.

void CIEBrowser::MouseMove(int x, int y)
{
m_cw.SendMessageToDescendants(WM_MOUSEMOVE, 0, MAKELPARAM(x, y));
}

void CIEBrowser::MouseDown(int x, int y)
{
m_cw.SendMessageToDescendants(WM_LBUTTONDOWN, 0, MAKELPARAM(x, y));
}

void CIEBrowser::MouseUp(int x, int y)
{
m_cw.SendMessageToDescendants(WM_LBUTTONUP, 0, MAKELPARAM(x, y));
}


After seeing your example I thought I would try adding a MouseClick function to see if that would work.

void CIEBrowser::MouseClick(int x, int y)
{
m_cw.SendMessageToDescendants(WM_LBUTTONDOWN, 0, MAKELPARAM(x, y));
m_cw.SendMessageToDescendants(WM_LBUTTONUP, 0, MAKELPARAM(x, y));
}


While that works everytime you click the mouse there is the windows "ding" sound. Not sure why. So I tried your method and it works. Great!

The only thing I don't like about sending a "mouse click" instead of WM_LBUTTONDOWN / WM_LBUTTONUP messages separately is that you can't hold down the mouse to drag the scrollbars you can only click to move them.

I really appreciate your help, but I'm still convinced there is a better way. Perhaps a similar way to how I have it working for the Flash object. Anyone have any ideas?

Share this post


Link to post
Share on other sites
Headkaze    607
Another problem I've noticed with using the IE control inside my engine is when a javascript error occurs a popup is displayed. Also if you click a link that opens a new window it will open a new window.

I found a post on here about the javascript thing, and a guy found out that you need to inject javascript into the document when the DocumentComplete event is triggered.

http://support.microsoft.com/default.aspx?scid=kb;EN-US;Q317024

Seems very hacky, but perhaps that method could be used to block popup windows also? Maybe I should start a new thread about that, but it seems that not many people have done this before.

Does anyone else use the IE ActiveX control successfully in a 3d environment?

P.S Can a moderator please move this thread to the DirectX forum? I think it will get a better response in there.

[Edited by - Headkaze on May 24, 2008 6:19:18 AM]

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this