You know that was so straight forward... I forgot the nature of the pipeline. Thanks! I'll try to implement this tonight!
Yes and no.
I don't know the particulars of DirectX but I think there are a few steps you're missing. When I'm saying you "do the math in the GPU" I'm saying that you do the transforms in a shader. As far as I'm seeing you're not using any shaders.
Shaders are programs that are executed in the GPU. They're coded in a specific language, DirectX shader language is HLSL.
The vertex shader stage is dedicated for this purpose. When you issue your draw call (after setting up the appropriate vertex buffer and matrix constant buffer) the GPU will grab the vertices (in model space, like when you loaded them) in the order they're presented (or by indexing like you're doing) and it will multiply each one by the matrix you're passing to the vertex shader.
In your vertex shader you'll have something like:
incoming vertexPosition; // This one will be changed each time the program gets executed for each vertex you send to the GPU.
incoming constant matrixTransform; // This one is constant for the whole model. Changed before you draw the next model.
finalPosition = vertexPosition * matrixTransform; // Row vertex * 4x4 transform matrix (which is modelMat * viewMat * projMat, precomputed in CPU ).
( I could put "correct" GLSL but it wouldn't help you at all
After that you have a pixel shader stage that will compute each pixel produced by the geometry that was processed by the vertex shader.
Now you might be using fixed function pipeline on DirectX (9 I think?), things change quite a bit in that case, and there is where my knowledge ends. No idea how to use OpenGL fixed function pipeline, much less DX's.