Strange Memory Overwrite

Started by
15 comments, last by NightCreature83 10 years, 8 months ago

I have this snippet of code that should fill out a structure with my Vertex Layout Elements, however on the line where I allocate the array to hold the elements it overwrites memory. The actual memory location it overwrites is used in the code below it so the compiler should realise that it shouldn't do this. The first time the function is called it works fine but I am not passing it any textureCoordinateDimensions so if it overwrites that particular location it doesn't matter for that call, when I actually do pass it something the value in the vector is overwritten. This happens in a debug build and I will provide source and assembly code, I just don't understand why the compiler would generate this code in this case. The actual parameter comes from code that is two levels up in the call stack, code is compile in debug for a x64 platform with SSE2 optimization on, and no optimisations applied. I have already tried doing a full rebuild without any luck.


void VertexBuffer::createInputElementLayout( bool position, bool normal, const std::vector<unsigned int> &textureCoordinateDimensions )
{
    //Create the buffer layout elements
    unsigned int numberOfElements = 0;
    if (position)
    {
        ++numberOfElements;
    }
    if (normal)
    {
        ++numberOfElements;
    }
    numberOfElements += (unsigned int)textureCoordinateDimensions.size();
    m_vertexDataLayoutElements = new D3D11_INPUT_ELEMENT_DESC[numberOfElements];
    unsigned int currentElement = 0;
    m_vertexStride = 0;

    if (position)
    {
        m_vertexDataLayoutElements[currentElement].SemanticName = "POSITION";
        m_vertexDataLayoutElements[currentElement].SemanticIndex = 0;
        m_vertexDataLayoutElements[currentElement].Format = DXGI_FORMAT_R32G32B32_FLOAT;
        m_vertexDataLayoutElements[currentElement].InputSlot = 0;
        m_vertexDataLayoutElements[currentElement].AlignedByteOffset = m_vertexStride;
        m_vertexDataLayoutElements[currentElement].InputSlotClass = D3D11_INPUT_PER_VERTEX_DATA;
        m_vertexDataLayoutElements[currentElement].InstanceDataStepRate = 0;
        m_vertexStride += 12;
        ++currentElement;
    }
    if (normal)
    {
        m_vertexDataLayoutElements[currentElement].SemanticName = "NORMAL";
        m_vertexDataLayoutElements[currentElement].SemanticIndex = 0;
        m_vertexDataLayoutElements[currentElement].Format = DXGI_FORMAT_R32G32B32_FLOAT;
        m_vertexDataLayoutElements[currentElement].InputSlot = 0;
        m_vertexDataLayoutElements[currentElement].AlignedByteOffset = m_vertexStride;
        m_vertexDataLayoutElements[currentElement].InputSlotClass = D3D11_INPUT_PER_VERTEX_DATA;
        m_vertexDataLayoutElements[currentElement].InstanceDataStepRate = 0;
        m_vertexStride += 12;
        ++currentElement;
    }
    for (unsigned int counter = 0; counter < textureCoordinateDimensions.size(); ++counter)
    {
        m_vertexDataLayoutElements[currentElement].SemanticName = "TEXCOORD" ;
        m_vertexDataLayoutElements[currentElement].SemanticIndex = counter;
        m_vertexDataLayoutElements[currentElement].InputSlot = 0;
        m_vertexDataLayoutElements[currentElement].InputSlotClass = D3D11_INPUT_PER_VERTEX_DATA;
        m_vertexDataLayoutElements[currentElement].InstanceDataStepRate = 0;
        if (textureCoordinateDimensions[counter] == 2)
        {
            m_vertexDataLayoutElements[currentElement].AlignedByteOffset = m_vertexStride;
            m_vertexDataLayoutElements[currentElement].Format = DXGI_FORMAT_R32G32_FLOAT;
            m_vertexStride += 8;
        }
        else if (textureCoordinateDimensions[counter] == 3)
        {
            m_vertexDataLayoutElements[currentElement].AlignedByteOffset = m_vertexStride;
            m_vertexDataLayoutElements[currentElement].Format = DXGI_FORMAT_R32G32B32_FLOAT;
            m_vertexStride += 12;
        }
        else if (textureCoordinateDimensions[counter] == 4)
        {
            m_vertexDataLayoutElements[currentElement].AlignedByteOffset = m_vertexStride;
            m_vertexDataLayoutElements[currentElement].Format = DXGI_FORMAT_R32G32B32A32_FLOAT;
            m_vertexStride += 16;
        }
        else
        {
            //Assume 2D texcoords
            m_vertexDataLayoutElements[currentElement].AlignedByteOffset = m_vertexStride;
            m_vertexDataLayoutElements[currentElement].Format = DXGI_FORMAT_R32G32_FLOAT;
            m_vertexStride += 8;
        }
        ++currentElement;
    }
    m_numberOfVertexLayoutElements = currentElement;
}

ASM code:


//-----------------------------------------------------------------------------
//! @brief   TODO enter a description
//! @remark
//-----------------------------------------------------------------------------
void VertexBuffer::createInputElementLayout( bool position, bool normal, const std::vector<unsigned int> &textureCoordinateDimensions )
{
00000001400A6620  mov         qword ptr [rsp+20h],r9  
00000001400A6625  mov         byte ptr [rsp+18h],r8b  
00000001400A662A  mov         byte ptr [rsp+10h],dl  
00000001400A662E  mov         qword ptr [rsp+8],rcx  
00000001400A6633  push        rdi  
00000001400A6634  sub         rsp,50h  
00000001400A6638  mov         rdi,rsp  
00000001400A663B  mov         ecx,14h  
00000001400A6640  mov         eax,0CCCCCCCCh  
00000001400A6645  rep stos    dword ptr [rdi]  
00000001400A6647  mov         rcx,qword ptr [rsp+60h]  
    //Create the buffer layout elements
    unsigned int numberOfElements = 0;
00000001400A664C  mov         dword ptr [numberOfElements],0  
    if (position)
00000001400A6654  movzx       eax,byte ptr [position]  
00000001400A6659  test        eax,eax  
00000001400A665B  je          VertexBuffer::createInputElementLayout+47h (1400A6667h)  
    {
        ++numberOfElements;
00000001400A665D  mov         eax,dword ptr [numberOfElements]  
00000001400A6661  inc         eax  
00000001400A6663  mov         dword ptr [numberOfElements],eax  
    }
    if (normal)
00000001400A6667  movzx       eax,byte ptr [normal]  
00000001400A666C  test        eax,eax  
00000001400A666E  je          VertexBuffer::createInputElementLayout+5Ah (1400A667Ah)  
    {
        ++numberOfElements;
00000001400A6670  mov         eax,dword ptr [numberOfElements]  
00000001400A6674  inc         eax  
00000001400A6676  mov         dword ptr [numberOfElements],eax  
    }
    numberOfElements += (unsigned int)textureCoordinateDimensions.size();
00000001400A667A  mov         rcx,qword ptr [textureCoordinateDimensions]  
00000001400A667F  call        std::vector<unsigned int,std::allocator<unsigned int> >::size (140003CD0h)  
00000001400A6684  mov         ecx,dword ptr [numberOfElements]  
00000001400A6688  add         ecx,eax  
00000001400A668A  mov         eax,ecx  
00000001400A668C  mov         dword ptr [numberOfElements],eax  
    m_vertexDataLayoutElements = new D3D11_INPUT_ELEMENT_DESC[numberOfElements];//New overwrites data on the stack
00000001400A6690  mov         eax,dword ptr [numberOfElements]  
00000001400A6694  mov         qword ptr [rsp+38h],rax  
00000001400A6699  mov         eax,20h  
00000001400A669E  mov         rcx,qword ptr [rsp+38h]  
00000001400A66A3  mul         rax,rcx  
00000001400A66A6  mov         rcx,0FFFFFFFFFFFFFFFFh  
00000001400A66AD  cmovo       rax,rcx  
00000001400A66B1  mov         rcx,rax  
00000001400A66B4  call        operator new[] (1400AF640h)  
00000001400A66B9  mov         qword ptr [rsp+30h],rax  
00000001400A66BE  mov         rax,qword ptr [this]  
00000001400A66C3  mov         rcx,qword ptr [rsp+30h]  
00000001400A66C8  mov         qword ptr [rax],rcx  
    unsigned int currentElement = 0;
00000001400A66CB  mov         dword ptr [currentElement],0  
    m_vertexStride = 0;
00000001400A66D3  mov         rax,qword ptr [this]  
00000001400A66D8  mov         dword ptr [rax+18h],0  
    if (position)
00000001400A66DF  movzx       eax,byte ptr [position]  
00000001400A66E4  test        eax,eax  
00000001400A66E6  je          VertexBuffer::createInputElementLayout+198h (1400A67B8h)  
    {
        m_vertexDataLayoutElements[currentElement].SemanticName = "POSITION";
00000001400A66EC  mov         eax,dword ptr [currentElement]  
00000001400A66F0  imul        rax,rax,20h  
00000001400A66F4  mov         rcx,qword ptr [this]  
00000001400A66F9  mov         rcx,qword ptr [rcx]  
00000001400A66FC  lea         rdx,[tinyxml2::MemPoolT<88>::`vftable'+0D8h (1400F3B60h)]  
00000001400A6703  mov         qword ptr [rax+rcx],rdx  
        m_vertexDataLayoutElements[currentElement].SemanticIndex = 0;
00000001400A6707  mov         eax,dword ptr [currentElement]  
00000001400A670B  imul        rax,rax,20h  
00000001400A670F  mov         rcx,qword ptr [this]  
00000001400A6714  mov         rcx,qword ptr [rcx]  
00000001400A6717  mov         dword ptr [rcx+rax+8],0  
        m_vertexDataLayoutElements[currentElement].Format = DXGI_FORMAT_R32G32B32_FLOAT;
00000001400A671F  mov         eax,dword ptr [currentElement]  
00000001400A6723  imul        rax,rax,20h  
00000001400A6727  mov         rcx,qword ptr [this]  
00000001400A672C  mov         rcx,qword ptr [rcx]  
00000001400A672F  mov         dword ptr [rcx+rax+0Ch],6  
        m_vertexDataLayoutElements[currentElement].InputSlot = 0;
00000001400A6737  mov         eax,dword ptr [currentElement]  
00000001400A673B  imul        rax,rax,20h  
00000001400A673F  mov         rcx,qword ptr [this]  
00000001400A6744  mov         rcx,qword ptr [rcx]  
00000001400A6747  mov         dword ptr [rcx+rax+10h],0  
        m_vertexDataLayoutElements[currentElement].AlignedByteOffset = m_vertexStride;
00000001400A674F  mov         eax,dword ptr [currentElement]  
00000001400A6753  imul        rax,rax,20h  
00000001400A6757  mov         rcx,qword ptr [this]  
00000001400A675C  mov         rcx,qword ptr [rcx]  
00000001400A675F  mov         rdx,qword ptr [this]  
00000001400A6764  mov         edx,dword ptr [rdx+18h]  
00000001400A6767  mov         dword ptr [rcx+rax+14h],edx  
        m_vertexDataLayoutElements[currentElement].InputSlotClass = D3D11_INPUT_PER_VERTEX_DATA;
00000001400A676B  mov         eax,dword ptr [currentElement]  
00000001400A676F  imul        rax,rax,20h  
00000001400A6773  mov         rcx,qword ptr [this]  
00000001400A6778  mov         rcx,qword ptr [rcx]  
00000001400A677B  mov         dword ptr [rcx+rax+18h],0  
        m_vertexDataLayoutElements[currentElement].InstanceDataStepRate = 0;
00000001400A6783  mov         eax,dword ptr [currentElement]  
00000001400A6787  imul        rax,rax,20h  
00000001400A678B  mov         rcx,qword ptr [this]  
00000001400A6790  mov         rcx,qword ptr [rcx]  
00000001400A6793  mov         dword ptr [rcx+rax+1Ch],0  
        m_vertexStride += 12;
00000001400A679B  mov         rax,qword ptr [this]  
00000001400A67A0  mov         eax,dword ptr [rax+18h]  
00000001400A67A3  add         eax,0Ch  
00000001400A67A6  mov         rcx,qword ptr [this]  
00000001400A67AB  mov         dword ptr [rcx+18h],eax  
        ++currentElement;
00000001400A67AE  mov         eax,dword ptr [currentElement]  
00000001400A67B2  inc         eax  
00000001400A67B4  mov         dword ptr [currentElement],eax  
    }

    if (normal)
00000001400A67B8  movzx       eax,byte ptr [normal]  
00000001400A67BD  test        eax,eax  
00000001400A67BF  je          VertexBuffer::createInputElementLayout+271h (1400A6891h)  
    {
        m_vertexDataLayoutElements[currentElement].SemanticName = "NORMAL";
00000001400A67C5  mov         eax,dword ptr [currentElement]  
00000001400A67C9  imul        rax,rax,20h  
00000001400A67CD  mov         rcx,qword ptr [this]  
00000001400A67D2  mov         rcx,qword ptr [rcx]  
00000001400A67D5  lea         rdx,[tinyxml2::MemPoolT<88>::`vftable'+0E4h (1400F3B6Ch)]  
00000001400A67DC  mov         qword ptr [rax+rcx],rdx  
        m_vertexDataLayoutElements[currentElement].SemanticIndex = 0;
00000001400A67E0  mov         eax,dword ptr [currentElement]  
00000001400A67E4  imul        rax,rax,20h  
00000001400A67E8  mov         rcx,qword ptr [this]  
00000001400A67ED  mov         rcx,qword ptr [rcx]  
00000001400A67F0  mov         dword ptr [rcx+rax+8],0  
        m_vertexDataLayoutElements[currentElement].Format = DXGI_FORMAT_R32G32B32_FLOAT;
00000001400A67F8  mov         eax,dword ptr [currentElement]  
00000001400A67FC  imul        rax,rax,20h  
00000001400A6800  mov         rcx,qword ptr [this]  
00000001400A6805  mov         rcx,qword ptr [rcx]  
00000001400A6808  mov         dword ptr [rcx+rax+0Ch],6  
        m_vertexDataLayoutElements[currentElement].InputSlot = 0;
00000001400A6810  mov         eax,dword ptr [currentElement]  
00000001400A6814  imul        rax,rax,20h  
00000001400A6818  mov         rcx,qword ptr [this]  
00000001400A681D  mov         rcx,qword ptr [rcx]  
00000001400A6820  mov         dword ptr [rcx+rax+10h],0  
        m_vertexDataLayoutElements[currentElement].AlignedByteOffset = m_vertexStride;
00000001400A6828  mov         eax,dword ptr [currentElement]  
00000001400A682C  imul        rax,rax,20h  
00000001400A6830  mov         rcx,qword ptr [this]  
00000001400A6835  mov         rcx,qword ptr [rcx]  
00000001400A6838  mov         rdx,qword ptr [this]  
00000001400A683D  mov         edx,dword ptr [rdx+18h]  
00000001400A6840  mov         dword ptr [rcx+rax+14h],edx  
        m_vertexDataLayoutElements[currentElement].InputSlotClass = D3D11_INPUT_PER_VERTEX_DATA;
00000001400A6844  mov         eax,dword ptr [currentElement]  
00000001400A6848  imul        rax,rax,20h  
00000001400A684C  mov         rcx,qword ptr [this]  
00000001400A6851  mov         rcx,qword ptr [rcx]  
00000001400A6854  mov         dword ptr [rcx+rax+18h],0  
        m_vertexDataLayoutElements[currentElement].InstanceDataStepRate = 0;
00000001400A685C  mov         eax,dword ptr [currentElement]  
00000001400A6860  imul        rax,rax,20h  
00000001400A6864  mov         rcx,qword ptr [this]  
00000001400A6869  mov         rcx,qword ptr [rcx]  
00000001400A686C  mov         dword ptr [rcx+rax+1Ch],0  
        m_vertexStride += 12;
00000001400A6874  mov         rax,qword ptr [this]  
00000001400A6879  mov         eax,dword ptr [rax+18h]  
00000001400A687C  add         eax,0Ch  
00000001400A687F  mov         rcx,qword ptr [this]  
00000001400A6884  mov         dword ptr [rcx+18h],eax  
        ++currentElement;
00000001400A6887  mov         eax,dword ptr [currentElement]  
00000001400A688B  inc         eax  
00000001400A688D  mov         dword ptr [currentElement],eax  
    }

    for (unsigned int counter = 0; counter < textureCoordinateDimensions.size(); ++counter)
00000001400A6891  mov         dword ptr [counter],0  
00000001400A6899  jmp         VertexBuffer::createInputElementLayout+285h (1400A68A5h)  
00000001400A689B  mov         eax,dword ptr [counter]  
00000001400A689F  inc         eax  
00000001400A68A1  mov         dword ptr [counter],eax  
00000001400A68A5  mov         eax,dword ptr [counter]  
00000001400A68A9  mov         qword ptr [rsp+40h],rax  
00000001400A68AE  mov         rcx,qword ptr [textureCoordinateDimensions]  
00000001400A68B3  call        std::vector<unsigned int,std::allocator<unsigned int> >::size (140003CD0h)  
00000001400A68B8  mov         rcx,qword ptr [rsp+40h]  
00000001400A68BD  cmp         rcx,rax  
00000001400A68C0  jae         00000001400A6AB7  
    {
        m_vertexDataLayoutElements[currentElement].SemanticName = "TEXCOORD" ;
00000001400A68C6  mov         eax,dword ptr [currentElement]  
00000001400A68CA  imul        rax,rax,20h  
00000001400A68CE  mov         rcx,qword ptr [this]  
00000001400A68D3  mov         rcx,qword ptr [rcx]  
00000001400A68D6  lea         rdx,[tinyxml2::MemPoolT<88>::`vftable'+0F0h (1400F3B78h)]  
00000001400A68DD  mov         qword ptr [rax+rcx],rdx  
        m_vertexDataLayoutElements[currentElement].SemanticIndex = counter;
00000001400A68E1  mov         eax,dword ptr [currentElement]  
00000001400A68E5  imul        rax,rax,20h  
00000001400A68E9  mov         rcx,qword ptr [this]  
00000001400A68EE  mov         rcx,qword ptr [rcx]  
00000001400A68F1  mov         edx,dword ptr [counter]  
00000001400A68F5  mov         dword ptr [rcx+rax+8],edx  
        m_vertexDataLayoutElements[currentElement].InputSlot = 0;
00000001400A68F9  mov         eax,dword ptr [currentElement]  
00000001400A68FD  imul        rax,rax,20h  
00000001400A6901  mov         rcx,qword ptr [this]  
00000001400A6906  mov         rcx,qword ptr [rcx]  
00000001400A6909  mov         dword ptr [rcx+rax+10h],0  
        m_vertexDataLayoutElements[currentElement].InputSlotClass = D3D11_INPUT_PER_VERTEX_DATA;
00000001400A6911  mov         eax,dword ptr [currentElement]  
00000001400A6915  imul        rax,rax,20h  
00000001400A6919  mov         rcx,qword ptr [this]  
00000001400A691E  mov         rcx,qword ptr [rcx]  
00000001400A6921  mov         dword ptr [rcx+rax+18h],0  
        m_vertexDataLayoutElements[currentElement].InstanceDataStepRate = 0;
00000001400A6929  mov         eax,dword ptr [currentElement]  
00000001400A692D  imul        rax,rax,20h  
00000001400A6931  mov         rcx,qword ptr [this]  
00000001400A6936  mov         rcx,qword ptr [rcx]  
00000001400A6939  mov         dword ptr [rcx+rax+1Ch],0  
        if (textureCoordinateDimensions[counter] == 2)
00000001400A6941  mov         eax,dword ptr [counter]  
00000001400A6945  mov         edx,eax  
00000001400A6947  mov         rcx,qword ptr [textureCoordinateDimensions]  
00000001400A694C  call        std::vector<unsigned int,std::allocator<unsigned int> >::operator[] (14008F140h)  
00000001400A6951  cmp         dword ptr [rax],2  
00000001400A6954  jne         VertexBuffer::createInputElementLayout+382h (1400A69A2h)  
        {
            m_vertexDataLayoutElements[currentElement].AlignedByteOffset = m_vertexStride;
00000001400A6956  mov         eax,dword ptr [currentElement]  
00000001400A695A  imul        rax,rax,20h  
00000001400A695E  mov         rcx,qword ptr [this]  
00000001400A6963  mov         rcx,qword ptr [rcx]  
00000001400A6966  mov         rdx,qword ptr [this]  
00000001400A696B  mov         edx,dword ptr [rdx+18h]  
00000001400A696E  mov         dword ptr [rcx+rax+14h],edx  
            m_vertexDataLayoutElements[currentElement].Format = DXGI_FORMAT_R32G32_FLOAT;
00000001400A6972  mov         eax,dword ptr [currentElement]  
00000001400A6976  imul        rax,rax,20h  
00000001400A697A  mov         rcx,qword ptr [this]  
00000001400A697F  mov         rcx,qword ptr [rcx]  
00000001400A6982  mov         dword ptr [rcx+rax+0Ch],10h  
            m_vertexStride += 8;
00000001400A698A  mov         rax,qword ptr [this]  
00000001400A698F  mov         eax,dword ptr [rax+18h]  
00000001400A6992  add         eax,8  
00000001400A6995  mov         rcx,qword ptr [this]  
00000001400A699A  mov         dword ptr [rcx+18h],eax  
00000001400A699D  jmp         00000001400A6AA8  
        }
        else if (textureCoordinateDimensions[counter] == 3)
00000001400A69A2  mov         eax,dword ptr [counter]  
00000001400A69A6  mov         edx,eax  
00000001400A69A8  mov         rcx,qword ptr [textureCoordinateDimensions]  
00000001400A69AD  call        std::vector<unsigned int,std::allocator<unsigned int> >::operator[] (14008F140h)  
00000001400A69B2  cmp         dword ptr [rax],3  
00000001400A69B5  jne         VertexBuffer::createInputElementLayout+3E3h (1400A6A03h)  
        {
            m_vertexDataLayoutElements[currentElement].AlignedByteOffset = m_vertexStride;
00000001400A69B7  mov         eax,dword ptr [currentElement]  
00000001400A69BB  imul        rax,rax,20h  
00000001400A69BF  mov         rcx,qword ptr [this]  
00000001400A69C4  mov         rcx,qword ptr [rcx]  
00000001400A69C7  mov         rdx,qword ptr [this]  
00000001400A69CC  mov         edx,dword ptr [rdx+18h]  
00000001400A69CF  mov         dword ptr [rcx+rax+14h],edx  
            m_vertexDataLayoutElements[currentElement].Format = DXGI_FORMAT_R32G32B32_FLOAT;
00000001400A69D3  mov         eax,dword ptr [currentElement]  
00000001400A69D7  imul        rax,rax,20h  
00000001400A69DB  mov         rcx,qword ptr [this]  
00000001400A69E0  mov         rcx,qword ptr [rcx]  
00000001400A69E3  mov         dword ptr [rcx+rax+0Ch],6  
            m_vertexStride += 12;
00000001400A69EB  mov         rax,qword ptr [this]  
00000001400A69F0  mov         eax,dword ptr [rax+18h]  
00000001400A69F3  add         eax,0Ch  
00000001400A69F6  mov         rcx,qword ptr [this]  
00000001400A69FB  mov         dword ptr [rcx+18h],eax  
00000001400A69FE  jmp         00000001400A6AA8  
        }
        else if (textureCoordinateDimensions[counter] == 4)
00000001400A6A03  mov         eax,dword ptr [counter]  
00000001400A6A07  mov         edx,eax  
00000001400A6A09  mov         rcx,qword ptr [textureCoordinateDimensions]  
00000001400A6A0E  call        std::vector<unsigned int,std::allocator<unsigned int> >::operator[] (14008F140h)  
00000001400A6A13  cmp         dword ptr [rax],4  
00000001400A6A16  jne         00000001400A6A61  
        {
            m_vertexDataLayoutElements[currentElement].AlignedByteOffset = m_vertexStride;
00000001400A6A18  mov         eax,dword ptr [currentElement]  
00000001400A6A1C  imul        rax,rax,20h  
00000001400A6A20  mov         rcx,qword ptr [this]  
00000001400A6A25  mov         rcx,qword ptr [rcx]  
00000001400A6A28  mov         rdx,qword ptr [this]  
00000001400A6A2D  mov         edx,dword ptr [rdx+18h]  
00000001400A6A30  mov         dword ptr [rcx+rax+14h],edx  
            m_vertexDataLayoutElements[currentElement].Format = DXGI_FORMAT_R32G32B32A32_FLOAT;
00000001400A6A34  mov         eax,dword ptr [currentElement]  
00000001400A6A38  imul        rax,rax,20h  
00000001400A6A3C  mov         rcx,qword ptr [this]  
00000001400A6A41  mov         rcx,qword ptr [rcx]  
00000001400A6A44  mov         dword ptr [rcx+rax+0Ch],2  
            m_vertexStride += 16;
00000001400A6A4C  mov         rax,qword ptr [this]  
00000001400A6A51  mov         eax,dword ptr [rax+18h]  
00000001400A6A54  add         eax,10h  
00000001400A6A57  mov         rcx,qword ptr [this]  
00000001400A6A5C  mov         dword ptr [rcx+18h],eax  
        }
        else
00000001400A6A5F  jmp         00000001400A6AA8  
        {
            //Assume 2D texcoords
            m_vertexDataLayoutElements[currentElement].AlignedByteOffset = m_vertexStride;
00000001400A6A61  mov         eax,dword ptr [currentElement]  
00000001400A6A65  imul        rax,rax,20h  
00000001400A6A69  mov         rcx,qword ptr [this]  
00000001400A6A6E  mov         rcx,qword ptr [rcx]  
00000001400A6A71  mov         rdx,qword ptr [this]  
00000001400A6A76  mov         edx,dword ptr [rdx+18h]  
00000001400A6A79  mov         dword ptr [rcx+rax+14h],edx  
            m_vertexDataLayoutElements[currentElement].Format = DXGI_FORMAT_R32G32_FLOAT;
00000001400A6A7D  mov         eax,dword ptr [currentElement]  
00000001400A6A81  imul        rax,rax,20h  
00000001400A6A85  mov         rcx,qword ptr [this]  
00000001400A6A8A  mov         rcx,qword ptr [rcx]  
00000001400A6A8D  mov         dword ptr [rcx+rax+0Ch],10h  
            m_vertexStride += 8;
00000001400A6A95  mov         rax,qword ptr [this]  
00000001400A6A9A  mov         eax,dword ptr [rax+18h]  
00000001400A6A9D  add         eax,8  
00000001400A6AA0  mov         rcx,qword ptr [this]  
00000001400A6AA5  mov         dword ptr [rcx+18h],eax  
        }
        ++currentElement;
00000001400A6AA8  mov         eax,dword ptr [currentElement]  
00000001400A6AAC  inc         eax  
00000001400A6AAE  mov         dword ptr [currentElement],eax  
    }
00000001400A6AB2  jmp         VertexBuffer::createInputElementLayout+27Bh (1400A689Bh)  

    m_numberOfVertexLayoutElements = currentElement;
00000001400A6AB7  mov         rax,qword ptr [this]  
00000001400A6ABC  mov         ecx,dword ptr [currentElement]  
00000001400A6AC0  mov         dword ptr [rax+1Ch],ecx  
}
00000001400A6AC3  add         rsp,50h  
00000001400A6AC7  pop         rdi  
00000001400A6AC8  ret

Worked on titles: CMR:DiRT2, DiRT 3, DiRT: Showdown, GRID 2, theHunter, theHunter: Primal, Mad Max, Watch Dogs: Legion

Advertisement

This isn't exactly a solution, but why not to use vector<D3D11_INPUT_ELEMENT_DESC> for m_vertexDataLayoutElements? That would solve all problems with memory handling and indexes. You'd just push_back() new elements and that's it, there's 0 chance of overwrite.

Yeah as I posted the code here I thought wouldn't a vector cause some cleaner code as well so I tried that with no luck, instead of the new I do a reserve and when that is done in the same call as before it messes the vector up dramatically, the capacity and size members go to large numbers sadly, with the size being bigger then the capacity so something is seriously going wrong here.

The memory handling here isn't the issue, the issues is that the code changes a value that is on the stack from code that shouldn't touch that variable at all. The code that is generated for this call is some how just wrong.

Worked on titles: CMR:DiRT2, DiRT 3, DiRT: Showdown, GRID 2, theHunter, theHunter: Primal, Mad Max, Watch Dogs: Legion

Are you sure you're calling the function on a valid object? Are you sure that all pointers and references in the surrounding/calling code are valid? Are you sure nothing is getting prematurely destroyed, or unexpectedly copied or assigned? Do the classes involved follow the rule of three?

Are you sure you're calling the function on a valid object? Are you sure that all pointers and references in the surrounding/calling code are valid? Are you sure nothing is getting prematurely destroyed, or unexpectedly copied or assigned? Do the classes involved follow the rule of three?

This is all called on a VertexBuffer object directly that gets constructed within the same scope, prior to the new call in the function I provided all values are correct as soon as the new is done the value in the passed parameter is wrong. I found this out with a databreak point on the location that gets changed.

The vertex buffer class itself has no virtual methods and relies on compiler generated assignment operator and copy constructor. IN the calling code we don't assign the vertexbuffer to a new object we only pass the pointer to a container and manager class. So copy construction and assignment are out in this case. Besides it would be weird if in unoptimised code a copy or assignment would happen during a call to a private method from a public one, you wouldn't be able to trust your application flow anymore, ow and before you ask this code is single threaded.

I should have mentioned this earlier but the function is a private method of the class that gets called from a public function that passes it it's data which comes from the calling function and the overwritten param is on the stack there as well(actually on the line above the public function call of the vb object) and directly passed into this function.

Worked on titles: CMR:DiRT2, DiRT 3, DiRT: Showdown, GRID 2, theHunter, theHunter: Primal, Mad Max, Watch Dogs: Legion

Does it work when compiled as 32-bit code?

I don't suppose you have a bit field as the last member in any of the classes involved? If so you've probably run into http://connect.microsoft.com/VisualStudio/feedback/details/777184/c-compiler-bug-vtable-pointer-put-at-wrong-offset-in-64-bit-mode

If that doesn't help, a minimal and compilable bit of code that reproduces the bug would be handy.

As a rule I delete any pointer inside a "createXX" before new'ing another one. Also see that you're matching delete [] with new [] (instead of using the non-array version of delete)

I have an issue understanding what is being corrupted. How do you notice the stack is corrupted? You posted the assembly, but the comment that the stack gets corrupted is on C++ code, but for what you're telling it would be after the "call operator new[]" instruction; by stepping in it should be quite obvious if inside new something is getting mangled or not.

Data breakpoints are your friend here.

Another reason could be missmatching CRT (debug vs release, dll vs static) or you (or one of the libraries you use) overloaded the new/delete operators and stuff is getting mixed (or the overload is wrong, eg. overloading new & delete but forgot the array variants, or the nothrow variants. There is a total of 4 news & 4 deletes to overload).

You have to show more information on how this gets called. Also, what is the data that is being written onto your object. Knowing the data would help know the writing.

Chances are that whatever you are passing in has been freed. That's why new would write to it.

What happens is that the value inside the vector becomes 30000ish from its original value. The passed in vector is not newed it is stack allocated and passed as a const reference to both functions. This means that when the code comes to figuring out which texture coordinate definition to choose it messes up.

After all your posts and seeing that it is to acces trying certain things through the tinyXml mempool in the asm, gives me a starting point.

@Pink Horror: the memory that is written to is on the stack not the heap and isnt explicitly being freed nor is that vector being cleared in between these calls.

The offending asm line is:
Mov qword pointer [rax], rcx
At the end of the new block.

Worked on titles: CMR:DiRT2, DiRT 3, DiRT: Showdown, GRID 2, theHunter, theHunter: Primal, Mad Max, Watch Dogs: Legion


If that doesn't help, a minimal and compilable bit of code that reproduces the bug would be handy.

Also, building such an example might help you isolate the cause.

This topic is closed to new replies.

Advertisement