You're taking a value of type "BYTE*" and casting it to type "void**". In other words you're casting from a pointer to a double pointer. You need to take the address of your pointer and cast that to (void**).
I even tried to create a box using D3DXCreateBox() and tried to retrieve the values, but can't get it to work: LPD3DXMESH mesh;
D3DXCreateBox(d3ddev, 100.0f, 200.0f, 300.0f, &mesh, NULL);
BYTE *pData = NULL;
mesh->LockVertexBuffer(D3DLOCK_READONLY, (void**)&pData);
I notice you aren't telling the compute routine which part of the vertex buffer to get it's data from. A vertex buffer holds multiple items such as the vertices, UV coordinates, normals etc...
Instead of using a BYTE for the pData, use a structure matching your vertex layout such as the following...
I also want to mention that when I create a box in 3Ds Max with (Length, Width, Height) of 800 and export it to .x file then load it into DirectX 9 program, then I create btBoxShape by using 400 which is half 800 as the following:
btVector3 boxHalfExtents(400, 400, 400);
btBoxShape* boxShape = new btBoxShape(boxHalfExtents);
The physical box appears much larger than the rendered one, I am trying to get the physical box to be exactly the same size as the box that I create in 3Ds Max so I can have a box behaving like in physical world.
Well, all I can say is that it seems your physical world scale and renderer scale are out of sync. Are you sure you export with the correct units in 3DS Max? If I remember correctly Bullet uses the metric system.