for some time now I have been exporting meshes from Maya using my own plugin into my own binary format. The format is simply list of vertices and a index buffer.
Now I am thinking of upgrading and I ran into a few obstacles.
The first is that internally Maya meshes are stored in a format similar to OBJ files format. Until now, when exporting, I have iterated over faces' vertices and their indices into arrays of vertex positions, UV coordinates and normals. I would save each unique combination of these indices and gave it an unique integer for creation of index buffer. If my exporter encountered the same unique combination, it would simply add that combination's index into my index buffer and not add the vertex to the vertex buffer, therefore saving space as it is a shared vertex.
The problem I ran into there is that unless the two faces sharing a vertex are coplanar, the same vertex has multiple normals based on the normal of the face it belong to. The best example would be a cube, where each corner vertex has three different normals.
As you can imagine, in some cases this could result in vertex buffer larger that original number of vertices in a model and index buffer that never references a vertex twice.
I though of calculating a single normal for each vertex by averaging the normals of faces that share this vertex, however I can imagine this could result in wrong normals especially where hard edges are required.
Now I am thinking of using multiple buffers, ie. one containing Positions and UVs, and second containing normals. However, I am not sure how this would work, since I would need two index buffers, one indication unique positions and one for indication unique normal.
Maybe I am over thinking this, but surely there must be a better way than what I am using right now.
Yeah if you use the raw positions/UVs/colors/normals/etc. from Maya then you will need to have a step where you combine indentical verts and then build a new vertex buffer. You might want to look into using the MGeometry class instead, which gives you Maya's cached vertex and index data that they use for rendering.
Awesome, since MGeometry (as I just found out thx to you) is used in their Viewport 2.0, which is very simmilar to game engine rendering . This should make my exported 100 times simpler, faster and hopefully the meshes will be more optimized. Thx !!!