[Windows C++] ShellExecuteEx + Wow64DisableWow64FsRedirection

Started by
11 comments, last by Kiristu 13 years, 4 months ago
Hi,

I'm trying to open folders with ShellExecuteEx.

It worked fine until I tried to make it compatible with Windows x64 (Win7).
The point with that system is the Wow64 redirection system that uses "bad" paths for some applications depending on the calling application is 32 or 64 bits.

Anyway, considering I always have the correct path, I use Wow64DisableWow64FsRedirection before each call to ShellExecute to launch the correct file.
It works fine for files, but for folders it crashes *sometimes* (even with the same test procedure ...). The SHELLEXECUTEINFO hInstApp member is then "5" (= ERROR_ACCESS_DENIED).

So, I tried to only enclose the ShellExecute for files between Wow64xxx functions and not the ones for folders.

But I still have the same problem.

The only thing I'm sure is that it is related to Wow64xxx because when I don't use them at all, folders can be opened correctly (but, of course, files that need redirections won't).

I imagine this could be related to something going wrong in COM/OLE because once it crashed, I have other problems with drag and drops in my application.
On the other hand, if I do any drag and drop before calling ShellExecute, the folders will open correctly and never crash again.

Any idea what I could do to progress with that ?

Thanks,
K.
Advertisement
Are you initializing COM properly? MSDN for ShellExecuteEx suggests

CoInitializeEx(NULL, COINIT_APARTMENTTHREADED | COINIT_DISABLE_OLE1DDE)

Regarding the error you're getting it says

ERROR_ACCESS_DENIED Access to the specified file is denied.

Perhaps you're trying to open folders that you don't have permission to open?

Do you get the same result using ShellExecute?

As for screwing with subsequent drag drop operations, are you calling Wow64RevertWow64FsRedirection afterward? See the example here: Wow64DisableWow64FsRedirection Function.
"I thought what I'd do was, I'd pretend I was one of those deaf-mutes." - the Laughing Man
I don't use CoInitialize directly because I need to call OleInitialize for drag and drop operations. MSDN says OleInitialize calls internally CoInitialize, so I guess it is done.

About permissions, this will do the same for any folders (I tested a lot of them).
Also, I don't see why I should not have permissions since I'm admin and only trying to access regular folders (nothing in Windows etc ...). I was able to access all of them before using wow64 redirections.

ShellExecute returns the same "5" error code.

And finally, yes I'm calling MyWow64RevertWow64FsRedirection as per the documentation.
Does the call to OleInitialize() succeed?

Is it always done before the ShellExecuteEx() call?

Do you always call both functions from the same thread?
I call only once OleInitialize at the beginning of my program (and once uninit at the end), is it a bad practice ?
Its call is successful.

The ShellExecute calls are not all in the same thread than the init, but the only one that causes issues is in the main thread, although it is called in a callback procedure (WndProc, via the GetMessage loop in main function).

Would it be better to call OleInitialize or even CoInitializeEx just before any ShellExecute call (with associated uninit after) ?

I thought it was the correct way to initialize that ... "stuff" at the beginning and here we go ... I have to admit I don't really understand what is all of this for ...

Edit: I also have some CoCreateInstance calls, would I need to put an init before and an uninit after ??
Well, I tried to call ShellExecute in a new thread, without calling any Ole/CoInit, and it seems to work ...

I don't really like this solution because it should'nt work (as the other version with init in the same thread should work).

I'll need to perform more tests with this, but for the moment it is the best I get.
Quote:Original post by Kiristu
I don't use CoInitialize directly because I need to call OleInitialize for drag and drop operations. MSDN says OleInitialize calls internally CoInitialize, so I guess it is done.


But is it done with the flags you need?

CoInitializeEx, says that OleInitialize calls it specifying COINIT_APARTMENTTHREADED but does not say whether COINIT_DISABLE_OLE1DDE is included or not. I presume not. That might be the problem.

"I thought what I'd do was, I'd pretend I was one of those deaf-mutes." - the Laughing Man
Unfortunately, there is no way to specify COINIT_DISABLE_OLE1DDE, because OleInit doesn't take any parameter.

Also, I need Ole because I use drag and drops (RegisterDragDrop), and I can't uninit it before the end of the program because it would "loose" the registration.

I think the good way is to call ShellExec in a separate thread. I just don't understand why it won't work with Co/OleInit and work perfectly without.
Have you tried calling both CoInitializeEx and OleInitialize?

If OleInitialize calls CoInitializeEx specifying COINIT_APARTMENTTHREADED and you call CoInitializeEx specifying COINIT_APARTMENTTHREADED | COINIT_DISABLE_OLE1DDE then COINIT_APARTMENTTHREADED remains specified. Try it once and see if it changes anything.

I agree that calling ShellExec in a separate thread is a good idea.
"I thought what I'd do was, I'd pretend I was one of those deaf-mutes." - the Laughing Man
I tried calling
CoInitializeEx(NULL, COINIT_DISABLE_OLE1DDE)
and
CoInitializeEx(NULL, COINIT_APARTMENTTHREADED | COINIT_DISABLE_OLE1DDE)

The first returns something about the fact I can't change the appartment once it is initialized, the second returns S_FALSE, meaning it is already initialized.

I then removed this and let only OleInit as it was before.

But, surprisingly, the calls to ShellExec now throw the error !
So, I put CoInitializeEx/uninit in the ShellExec thread, and it works as expected ... at least for the moment.

The only constant behavior is that:
- If I do a drag/drop operation, the ShellExec will always work until the app is stopped
- If the ShellExec fails once it will fail forever (only on folders, the files will always work !), and the drag/drop will not work anymore (the result of SHDoDragDrop is something like "calling procedure not specified").

So I think the drag/drop operation initializes something somewhere that is needed. And if this is not initialized, *sometimes* it is crashed by ShellExec and cannot be init again ...

This topic is closed to new replies.

Advertisement