Sign in to follow this  
  • entries
    557
  • comments
    1237
  • views
    421604

Untitled

Sign in to follow this  
Evil Steve

55 views

Yay, I got to play with virtual function tables today at lunch. Relevant thread.

What I found (VC8 specific probably, although it may work on other compilers):

  • The vtable pointer is held in the first 4 bytes of a class with virtual functions. That means you can cast the class to a size_t pointer, and dereference it to get the address of the vtable as a size_t.
    The vtable is an array of addresses, one entry per virtual function and is ordered in by the order the functions appear in the class. I'm not sure where a virtual destructor comes in, I didn't bother checking - but I expect it's the same as a normal func (Position defined by the location in the class header).

  • The vtable is write protected, so you need to use VirtualProtectEx to enable write access on it.

  • You need to use a naked stub function to call a "normal" C function to get around the calling convention crap.


Overall, nothing particularly mind blowing. Anyway, here's the code I ended up with (To hook the SetRenderState() call):

HRESULT MySetRenderState(D3DRENDERSTATETYPE State, DWORD Value)
{
// Do whatever you want here
}

__declspec(naked) HRESULT stub(D3DRENDERSTATETYPE State, DWORD Value)
{
__asm
{
push ebp;
mov ebp, esp;
sub esp, 0Ch;
}

MySetRenderState(State, Value);

__asm
{
mov esp, ebp;
pop ebp;
ret 0Ch;
}
}

// Grab the vtable and remove write protection on it
size_t nEntry = 57;
size_t* pVTable = (size_t*)*(size_t*)m_pDevice;
DWORD dwOldProt;
VirtualProtectEx(GetCurrentProcess(), &pVTable[nEntry], sizeof(size_t), PAGE_EXECUTE_READWRITE, &dwOldProt);

// Thunk the vtable so SetRenderState() calls "stub" instead
pVTable[nEntry] = (size_t)&stub;

// Test the new call
m_pDevice->SetRenderState(D3DRS_ZENABLE, 1);




Joy joy joy. Anyway, back to work for me...
Sign in to follow this  


0 Comments


Recommended Comments

There are no comments to display.

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now