And then it is possible that in our journey we encounter this:
This happens if you use PIX on an application that uses the D3DPERF_SetOptions(1) function to disable profiling/analysis tools.
An easy way to circumvent this problem is to edit the binary of the application.
Then the only thing we need to change in the binary is the argument of D3DPERF_SetOptions from 1 to 0.
As an example I will demonstrate it with the game Portal 2.
The tools I used:
[size="4"]Step 1: Locate where D3DPERF_SetOptions is called
First we need to figure out where D3DPERF_SetOptions is called in the application. For that we can use the API monitoring software WinAPIOverride32. The official website of WinAPIOverride32 contains very good tutorials.
First you need to create a monitoring file in order to let WinAPIOverride know what we want to monitor. Because D3DPERF_SetOptions is located in the Direct3D 9 DLL we want to create a description of the d3d9.dll. Thus, you can use DllExportFinder.exe on d3d9.dll in your Windows system directory or save the following in d3d9.txt at "winapioverride32_bin\monitoring files" in your WinApiOverride32 directory.
; Monitoring file generated for exports table of d3d9.dll v6.1.7601.17514 by MonitoringFileBuilder
!C:\Windows\SysWOW64\d3d9.dll|int D3DPERF_BeginEvent(D3DCOLOR col, LPCWSTR wszName)
!C:\Windows\SysWOW64\d3d9.dll|D3DPERF_SetMarker(D3DCOLOR col, LPCWSTR wszName)
!C:\Windows\SysWOW64\d3d9.dll|D3DPERF_SetRegion(D3DCOLOR col, LPCWSTR wszName)
!C:\Windows\SysWOW64\d3d9.dll|IDirect3D9 * Direct3DCreate9(UINT SDKVersion)
!C:\Windows\SysWOW64\d3d9.dll|HRESULT Direct3DCreate9Ex(UINT SDKVersion, IDirect3D9Ex **ppD3D)
Next, attach WinAPIOverride at application startup of the game you want to modify.
Select the API D3DPERF_SetOptions in the monitoring wizard and resume the execution of the attached application.
And WinAPIOverride shows us in which DLL and where D3DPERF_SetOptions is called (0x5D496D6F) (shaderapidx9.dll + 0x00026D6F). (I'm keeping Portal 2 running in windowed mode for the next step.)
[size="4"]Step 2: Modify the binary
Now you have located where D3DPERF_SetOptions is called in the application memory (0x5D496D6F). Lets browse the memory of the running application Portal2.exe and see with our own eyes where exactly the function is called in the memory. For this I like to use L. Spiro's Memory Hacking Software.
Open the running Process "Portal2.exe". Goto File => Properties and select the right chunk that contains the caller address. If you right click you can view it in a Disassembler or in a Hex Editor.
This is what the disassembler shows:
You can see where the first argument "1" is PUSHed onto the stack and where D3DPERF_SetOptions is CALLed. You can now choose to replace the "CALL" command by a "NOP"command (no operation) or change the argument that we pass to D3DPERF_SetOptions. I chose the latter option. So we just want to change the code "6A 01" to "6A 00" at 5D496D6B. This is how the application memory looks of the running portal2.exe in the hexeditor:
But off course we want to change the binary on the hard disk. So open the fileshaderapidx9.dll with the hex editor and go to the same location. (I just searched on the same sequence of hex bytes (55 8B EC 81 4C 01 00 56 etc.) of the line 0x5D496D60 with the find function).
Then modify 6A 01 to 6A 00.
Save the file and you are done!