Strange Memory Overwrite

Started by
15 comments, last by NightCreature83 10 years, 8 months ago

Well I did do some restructuring of that particular piece of code and have gotten ride of the memory allocation in vertex buffer and also gotten ride of having to actually store the elements in a member variable anyway. This has moved the problem into the destructor on the passed in vector, which is also a nightmare to debug because of the STL syntax and unlikely it is due to the use of STL.

Also build a win32 executable and can say it is nothing to do with the bit fields as the win32 executable crashes at the same location sad.png.

If I knew what was going on I could actually build a minimal case, as it is crashing in code that shouldn't it gets weirder than it should be as I have no starting point for this crash. The odd bit is that we are only every accessing members of the vector and never actually writing to it, at least not from the C++ code as it looks sad.png.

BTW here is the reworked code for vertex buffer.

//-----------------------------------------------------------------------------
//! @brief   TODO enter a description
//! @remark
//-----------------------------------------------------------------------------
bool VertexBuffer::createBufferAndLayoutElements( const DeviceManager& deviceManager, unsigned int bufferSize, void* data, bool dynamic, bool position, bool normal, const std::vector<unsigned int>& textureCoordinateDimensions, ID3DBlob* vertexShaderCodeBlob )
{
    D3D11_BUFFER_DESC bufferDescriptor;
    ZeroMemory(&bufferDescriptor, sizeof(D3D11_BUFFER_DESC));
    bufferDescriptor.Usage = D3D11_USAGE_IMMUTABLE;
    bufferDescriptor.ByteWidth = bufferSize;
    bufferDescriptor.BindFlags = D3D11_BIND_VERTEX_BUFFER;
    bufferDescriptor.CPUAccessFlags = 0;
    bufferDescriptor.MiscFlags = 0;

    D3D11_SUBRESOURCE_DATA initData;
    ZeroMemory(&initData, sizeof(D3D11_SUBRESOURCE_DATA));
    initData.pSysMem = data;
    initData.SysMemPitch = 0;
    initData.SysMemSlicePitch = 0;

    HRESULT hr = deviceManager.getDevice()->CreateBuffer( &bufferDescriptor, &initData, &m_buffer );
    if (FAILED(hr))
    {
        MSG_TRACE_CHANNEL("VERTEXBUFFER", "Failed to create a D3D11 Buffer object with code: 0x%x", hr );
        return false;
    }

    createVertexInputLayout(deviceManager, vertexShaderCodeBlob, createInputElementLayout(position, normal, textureCoordinateDimensions));
    
    m_vertexCount = bufferSize / m_vertexStride;
    dynamic = false;

    return true;
}
//-----------------------------------------------------------------------------
//! @brief   TODO enter a description
//! @remark
//-----------------------------------------------------------------------------
bool VertexBuffer::createVertexInputLayout( const DeviceManager& deviceManager, ID3DBlob* vertexShaderCodeBlob, const std::vector<D3D11_INPUT_ELEMENT_DESC>& inputElements )
{
    HRESULT hr = deviceManager.getDevice()->CreateInputLayout(&inputElements[0], (unsigned int)inputElements.size(), vertexShaderCodeBlob->GetBufferPointer(), vertexShaderCodeBlob->GetBufferSize(), &m_inputLayout );

    if (FAILED( hr ) )
    {        
        MSG_TRACE_CHANNEL("VERTEXBUFFER", "Failed to create the input layout: 0x%x", hr )
            return false;
    }

    return true;
}
//-----------------------------------------------------------------------------
//! @brief   TODO enter a description
//! @remark
//-----------------------------------------------------------------------------
std::vector<D3D11_INPUT_ELEMENT_DESC> VertexBuffer::createInputElementLayout( bool position, bool normal, const std::vector<unsigned int> &textureCoordinateDimensions )
{
    //Create the buffer layout elements
    unsigned int numberOfElements = 0;
    if (position)
    {
        ++numberOfElements;
    }
    if (normal)
    {
        ++numberOfElements;
    }
    numberOfElements += (unsigned int)textureCoordinateDimensions.size();
    std::vector<D3D11_INPUT_ELEMENT_DESC> vertexDataLayoutElements;
    vertexDataLayoutElements.reserve(numberOfElements);//New overwrites data on the stack
    m_vertexStride = 0;
    if (position)
    {
        D3D11_INPUT_ELEMENT_DESC layout;
        layout.SemanticName = "POSITION";
        layout.SemanticIndex = 0; 
        layout.Format = DXGI_FORMAT_R32G32B32_FLOAT;
        layout.InputSlot = 0;
        layout.AlignedByteOffset = m_vertexStride;
        layout.InputSlotClass = D3D11_INPUT_PER_VERTEX_DATA;
        layout.InstanceDataStepRate = 0;
        vertexDataLayoutElements.push_back(layout);
        m_vertexStride += 12;
    }

    if (normal)
    {
        D3D11_INPUT_ELEMENT_DESC layout;
        layout.SemanticName = "NORMAL";
        layout.SemanticIndex = 0; 
        layout.Format = DXGI_FORMAT_R32G32B32_FLOAT;
        layout.InputSlot = 0;
        layout.AlignedByteOffset = m_vertexStride;
        layout.InputSlotClass = D3D11_INPUT_PER_VERTEX_DATA;
        layout.InstanceDataStepRate = 0;
        vertexDataLayoutElements.push_back(layout);
        m_vertexStride += 12;
    }

    for (unsigned int counter = 0; counter < textureCoordinateDimensions.size(); ++counter)
    {
        D3D11_INPUT_ELEMENT_DESC layout;
        layout.SemanticName = "TEXCOORD";
        layout.SemanticIndex = counter; 
        layout.InputSlot = 0;
        layout.InputSlotClass = D3D11_INPUT_PER_VERTEX_DATA;
        layout.InstanceDataStepRate = 0;
        if (textureCoordinateDimensions[counter] == 2)
        {
            layout.AlignedByteOffset = m_vertexStride;
            layout.Format = DXGI_FORMAT_R32G32_FLOAT;
            m_vertexStride += 8;
        }
        else if (textureCoordinateDimensions[counter] == 3)
        {
            layout.AlignedByteOffset = m_vertexStride;
            layout.Format = DXGI_FORMAT_R32G32B32_FLOAT;
            m_vertexStride += 12;
        }
        else if (textureCoordinateDimensions[counter] == 4)
        {
            layout.AlignedByteOffset = m_vertexStride;
            layout.Format = DXGI_FORMAT_R32G32B32A32_FLOAT;
            m_vertexStride += 16;
        }
        else
        {
            //Assume 2D texcoords
            layout.AlignedByteOffset = m_vertexStride;
            layout.Format = DXGI_FORMAT_R32G32_FLOAT;
            m_vertexStride += 8;
        }
        vertexDataLayoutElements.push_back(layout);
    }

    return vertexDataLayoutElements;
}

Worked on titles: CMR:DiRT2, DiRT 3, DiRT: Showdown, GRID 2, theHunter, theHunter: Primal, Mad Max, Watch Dogs: Legion

Advertisement

I really need to see how the function is being called to have any idea how to help.

void Mesh::initialise(const ShaderInstance& shaderInstance)
{
    if (m_modelData.empty())
    {
        VertexBuffer* vb = new VertexBuffer();
        IndexBuffer* ib = new IndexBuffer();
        m_modelData.push_back(new MeshGroup(GeometryInstance(vb, ib), shaderInstance));
        if (m_modelData[0]->getShaderInstance().getMaterial().getEffect().getVertexShaderBlob() == 0 )
        {
            m_modelData[0]->getShaderInstance().getMaterial().getEffect().loadEffect(getGameResource().getDeviceManager(), "Shaders\\SimpleEffect.fx", Effect::e4_0);
        }

        m_nummultitexcoords = 1; //HACK FIX THIS
        unsigned int bufferSize = 0;
        bufferSize += sizeof(float) * 3 * (unsigned int)m_vertices.size();
        bufferSize += sizeof(float) * 3 * (unsigned int)m_normals.size();
        bufferSize += sizeof(float) * 2 * m_nummultitexcoords * (unsigned int)m_vertices.size();//(unsigned int)m_texcoords[0].size();
        byte* vertexData = new byte[bufferSize];
        for (unsigned int counter = 0; counter < m_vertices.size(); ++counter)
        {
            *(float*)vertexData = m_vertices[counter].x(); vertexData += sizeof(float);
            *(float*)vertexData = m_vertices[counter].y(); vertexData += sizeof(float);
            *(float*)vertexData = m_vertices[counter].z(); vertexData += sizeof(float);
            *(float*)vertexData = m_normals[counter].x(); vertexData += sizeof(float);
            *(float*)vertexData = m_normals[counter].y(); vertexData += sizeof(float);
            *(float*)vertexData = m_normals[counter].z(); vertexData += sizeof(float);
            for (unsigned int texCoordCounter = 0; texCoordCounter < (unsigned int)m_nummultitexcoords; ++texCoordCounter)
            {
                TexCoords texCoords = m_texcoords[texCoordCounter];
                if (texCoords.size() > 0)
                {
                    *(float*)vertexData = texCoords[counter].x(); vertexData += sizeof(float);
                    *(float*)vertexData = texCoords[counter].y(); vertexData += sizeof(float);
                }
                else
                {

                    *(float*)vertexData = 0.0f; vertexData += sizeof(float);
                    *(float*)vertexData = 0.0f; vertexData += sizeof(float);
                }
            }

            m_boundingBox.enclose(m_vertices[counter]);
        }
        std::vector<unsigned int> texCoordDimensions;
        for (unsigned int texCoordCounter = 0; texCoordCounter < (unsigned int)m_nummultitexcoords; ++texCoordCounter)
        {
            texCoordDimensions.push_back(2);
        }
        vertexData = vertexData - bufferSize;
        vb->createBufferAndLayoutElements(getGameResource().getDeviceManager(), bufferSize, vertexData, false, true, true, texCoordDimensions, shaderInstance.getMaterial().getEffect().getVertexShaderBlob());
        //vb->createVertexInputLayout(getGameResource().getDeviceManager(), m_modelData[0]->getShaderInstance().getMaterial().getEffect().getVertexShaderBlob());
        delete [] vertexData;

        ib->setNumberOfIndecis( (unsigned int)m_indices.size() );
        ib->createBuffer(getGameResource().getDeviceManager(), (unsigned int)m_indices.size() * sizeof(unsigned int), (void*)&m_indices[0], false, D3D11_BIND_INDEX_BUFFER);

        std::vector<Vector3>         emptyVertices;
        std::vector<Vector3>         emptyNormals;
        std::vector<unsigned int>     emptyIndices;
        MultiTexCoords                 emptyTexcoords;
        std::swap(m_vertices, emptyVertices);
        std::swap(m_normals, emptyNormals);
        std::swap(m_indices, emptyIndices);
        std::swap(m_texcoords, emptyTexcoords);
    }
    else
    {
        m_modelData[0]->setShaderInstance( shaderInstance );
        if (m_modelData[0]->getShaderInstance().getMaterial().getEffect().getVertexShaderBlob() == 0 )
        {
            m_modelData[0]->getShaderInstance().getMaterial().getEffect().loadEffect(getGameResource().getDeviceManager(), "Shaders\\SimpleEffect.fx", Effect::e4_0);
        }
    }  
}

Worked on titles: CMR:DiRT2, DiRT 3, DiRT: Showdown, GRID 2, theHunter, theHunter: Primal, Mad Max, Watch Dogs: Legion

I'm not familiar with your code, but it looks that this snippet:

            for (unsigned int texCoordCounter = 0; texCoordCounter < (unsigned int)m_nummultitexcoords; ++texCoordCounter)
            {
                TexCoords texCoords = m_texcoords[texCoordCounter];
                if (texCoords.size() > 0)
                {
                    *(float*)vertexData = texCoords[counter].x(); vertexData += sizeof(float);
                    *(float*)vertexData = texCoords[counter].y(); vertexData += sizeof(float);
                }
Should be like this (swapped counter with texCoordCounter as indexes):

            for (unsigned int texCoordCounter = 0; texCoordCounter < (unsigned int)m_nummultitexcoords; ++texCoordCounter)
            {
                TexCoords texCoords = m_texcoords[counter];
                if (texCoords.size() > 0)
                {
                    *(float*)vertexData = texCoords[texCoordCounter].x(); vertexData += sizeof(float);
                    *(float*)vertexData = texCoords[texCoordCounter].y(); vertexData += sizeof(float);
                }
This is further strengthen by the fact that you commented out "m_texcoords[0].size();", which suggests that your layout is m_texcoords[vertexIdx][uvIdx] and not m_texcoords[uvIdx][vertexIdx]

The way you use vertexData (allocate it with new[], then add stuff to it, then subtract what you added to it before calling delete[] vertexData) is just asking for trouble, just make a copy of the pointer if you are going to manipulate it.

I'd do something like this:

byte* vertexData = new byte[bufferSize];

float* floatArray = reinterpret_cast<float*>(vertexData);

// do stuff with float array, no need to keep adding sizeof(float) either when you work with floatArray, just use ++floatArray. Don't ever change the value of vertexData

delete[] vertexData;

EDIT: Whoops ;)

"Most people think, great God will come from the sky, take away everything, and make everybody feel high" - Bob Marley

I'm not familiar with your code, but it looks that this snippet:


            for (unsigned int texCoordCounter = 0; texCoordCounter < (unsigned int)m_nummultitexcoords; ++texCoordCounter)
            {
                TexCoords texCoords = m_texcoords[texCoordCounter];
                if (texCoords.size() > 0)
                {
                    *(float*)vertexData = texCoords[counter].x(); vertexData += sizeof(float);
                    *(float*)vertexData = texCoords[counter].y(); vertexData += sizeof(float);
                }
Should be like this (swapped counter with texCoordCounter as indexes):

            for (unsigned int texCoordCounter = 0; texCoordCounter < (unsigned int)m_nummultitexcoords; ++texCoordCounter)
            {
                TexCoords texCoords = m_texcoords[counter];
                if (texCoords.size() > 0)
                {
                    *(float*)vertexData = texCoords[texCoordCounter].x(); vertexData += sizeof(float);
                    *(float*)vertexData = texCoords[texCoordCounter].y(); vertexData += sizeof(float);
                }
This is further strengthen by the fact that you commented out "m_texcoords[0].size();", which suggests that your layout is m_texcoords[vertexIdx][uvIdx] and not m_texcoords[uvIdx][vertexIdx]

Actually it shouldn't be like you suggest the naming in that case might be off but the texCoordCounter indexes the vector< vector<Vector2> which holds all of the loaded texture coordinates. There is no layout in the storage buffers in this case all that is there is the raw data, this function is interleaving the data into the one that gets sent to the GPU. The storage buffers all hold as many coordinates or normals as there are vertices in the mesh.

Worked on titles: CMR:DiRT2, DiRT 3, DiRT: Showdown, GRID 2, theHunter, theHunter: Primal, Mad Max, Watch Dogs: Legion

The problem is cuased by the GeometryInstance not having a copy constructor and assignment operator but having a destructor that is non trivial. The code just happens to run on fine for a while but as we are using that memory the heap catches a corruption.

And the problem is caused by the inline creation of the GeometryInstance in the constructor call to MeshGroup.

Worked on titles: CMR:DiRT2, DiRT 3, DiRT: Showdown, GRID 2, theHunter, theHunter: Primal, Mad Max, Watch Dogs: Legion

This topic is closed to new replies.

Advertisement