Im curious what most people do in terms of storing and reading triangle face indices.
Before thinking about compression I used to store files with non-unified indices (like .obj). This would reduce (uncompressed) file size as there were no duplicates (in terms of whole vertices/normals/texcoords) being stored. However when building unified vectors for OpenGL VBO's, I would not check for duplicates and end up with a bloated and inefficient buffer sizes.
Checking for these duplicates at run time seems wasteful to me so then when exporting to my run-time format I now check for duplicates and store everything in unified indices. This bloated the file a little but reading them in was fast, trivial and more efficient in terms of VBO size.
So my question is: before I send my model data into some sort of compression algorithm should I unify my indices and check for duplicates or leave them as is and let the decompression deal with duplicate values/unifying the indices.