I am working on a method to pick in the 3d world and get the coordinates of where the mouse clicked on an object. Using the mouse x and y in viewport space along with sampling the depth buffer I should be able to figure this out, but I am a bit lost here. I can sample the my depth buffer(R32F format) by locking the surface and reading the surface bytes, but I am not sure how to understand that data.

            if(_d3dDevice->GetRenderTargetData(_views[view].gBuffer._depthBufferSurf, _views[view].gBuffer._offscreenR32FSurface) != D3D_OK)

                ErrorMessenger::ReportMessage("Failed to GetRenderTargetData!", __FILE__, __LINE__);



            //Lock surface data

            D3DLOCKED_RECT surfaceData;

            _views[view].gBuffer._offscreenR32FSurface->LockRect(&surfaceData, 0, D3DLOCK_READONLY);

   

            BYTE* bytePointer = (BYTE*)surfaceData.pBits;



            //Get the index where the mouse is located

            DWORD index = (point.x * 4 + (point.y  * (surfaceData.Pitch)));

 
            //Dont know what the values I am retrieving represent. Depth buffer is from 0.0f to 1.0f but I am getting values of 107, 114 ect...
           //Read surface data

            BYTE depth = bytePointer[index];

OK, that seems to be giving me the correct value. So the next question is turning my x, y, and now depth value into a 3d position. My lacking math skills are holding me back and the only resources I have found online say to perform an inverse transformation on my x,y and z. But the inverse of which matrices I am unsure and my experimentation's have thus far failed.

Thank you for any help you can provide

Im sorry to keep bugging on this subject, but I am getting some wonky results. Here is the code

POINT point = _inputReader->GetCursorAbs();

    //Adjust mouse position to client space
    point.x -= _views[view].location.left;
    point.y -= _views[view].location.top;

    if(point.x < 0 || point.y < 0)
        return nullptr;

    //Normalize mouse coordinates to [-1,1]
    D3DXVECTOR3 v;
    v.x =  ( ( ( 2.0f * point.x ) / _appWidth  ) - 1 );
    v.y = -( ( ( 2.0f * point.y ) / _appHeight ) - 1 );
    v.z =  1.0f;
    
    //Get depth data into offscreen surface for reading
    if(_d3dDevice->GetRenderTargetData(_views[view].gBuffer._depthBufferSurf, _views[view].gBuffer._offscreenR32FSurface) != D3D_OK)
        ErrorMessenger::ReportMessage("Failed to GetRenderTargetData!", __FILE__, __LINE__);

    //Lock surface data
    D3DLOCKED_RECT surfaceData;
    _views[view].gBuffer._offscreenR32FSurface->LockRect(&surfaceData, 0, D3DLOCK_READONLY);
    
    //Get the index where the mouse is located
    DWORD index = (point.x * 4 + (point.y  * (surfaceData.Pitch)));

    //Find depth
    static float* depth = new float;
    BYTE* bytePointer = (BYTE*)surfaceData.pBits;
    depth = reinterpret_cast<float*>(bytePointer + index);

    //Unlock
    _views[view].gBuffer._offscreenR32FSurface->UnlockRect();

    if(depth)
    {
        //Solve mouse world position
        D3DXMATRIX invViewProj;
        D3DXMatrixMultiply(&invViewProj, &_camera->GetViewViewMatrix(0), &_camera->GetViewProjectionMatrix(0));
        D3DXMatrixInverse(&invViewProj, 0, &invViewProj);

        D3DXVECTOR4 out;
        D3DXVec3Transform(&out, &D3DXVECTOR3(v.x, v.y, *depth), &invViewProj);
                
        *pos = LWXVector3(out.x / out.w, out.y / out.w, out.z / out.w);
    }
 

This is really bugging me. If I can get this down it will open up tons of functionality that I need. Thank you again

Ok guys. Thank you for all your help. I have realized what is wrong and have fixed the problem.

I might need something similar soon, so maybe you can post what the problem was and the solution to it ?

Thanks