Why do new engines change their file extensions and formats all the time?

Started by
8 comments, last by WoopsASword 7 years, 6 months ago

Guys, these days I spend most of my time making 3D models, rigging and then animating them. But all the models are unbelievably ugly, and my walking and running animations are just AWFUL. :unsure: That's why I decided to stop torturing the modelling software and gave up.

I decided to steal some models from Dota 2 just to use them for fun, not for sale, of course. But guess what, Valve recently changed their engine to Source Engine 2 and instead of .mdl, the meshes are vmdl_c, vmesh_c and other crazy extensions for textures and materials and nobody on the Internet knows how to deal with this.

And not just different file names, the formats are structured differently on purpose. And I was wondering - what is the reason for this? :huh:

It's not like I can publish a game using their models, this is not allowed, of course. Why would they waste time to change everything every time? And not just Valve, I just gave it as an example, other guys do it too.

There must be something very logical, of course, I'm not criticizing ( maybe just a little bit :angry: ), but I decided to ask anyways.

( If anyone has some kind of home-made converter, PM me, pls. :rolleyes: )

Advertisement

Probably to add features existing formats dont have, compression, mipmaps precalculated, HDR, could be a million things other formats dont support or dont do well enought for them.

When you use a piece of technology for a multi-year game development project, you learn a lot about how it works in practice versus how it works when you initially designed and wrote it. When you get the chance to fix all of the issues you identify with the technology at the start of the project, you usually take the opportunity to do so. Maybe you can reorganize the format to be more efficient for your actual use-cases, maybe you can add support for new bits of data in the storage format to allow for newer technologies that have come around or that an updated minimum specification lets you take advantage of... there's lots of reasons, and most of them are to serve practical needs of the project.

We often change file formats multiple times within the development of one single game, let alone multiple games!

Data structures change as requirements change / are discovered.

For us, changing a structure is not a big deal as all the files are automatically generated from source files (e.g. FBX/DAE/etc). If you change the model format, you just run a batch file and it automatically regenerates all of the game's model files in this new format.

Another reason may be changes in the production pipeline. What you tell us seems as if they preprocess/compress textures and models, compile shaders as I do too in my engine into some more valuable bitcode format and whatever things need to be done. Maybe the support for some old/unused file formats are removed from the engine to reduce maintain costs or outsourced to a more central way so that the engine dosent need to cary file converters for every file type but has one single converter tool.

To solve this you should take a look into Source Engine 2 documentation and its toolchain utilities to find out what happens there

Guys, thanks to all for clearing this up for me. ^_^

It seems that backward compatibility is not possible when changing file formats. -_-

To solve this you should take a look into Source Engine 2 documentation and its toolchain utilities to find out what happens there

You are right, they have documented the new file formats, everything is fine. I just need to find a way to decompile these files.

EDIT: I think I solved it.

For anyone who wants to use Valve Models, the easiest ways are:

1st way: download the previous Dota 2 using the old source engine, then use GCFScape to extract the .pak files that can be found in the steam/steamapps/common/your_game, then use the cannonfodder's MDL decompiler to get the .SMD files from .MDL. Then download this for Blender:http://steamreview.org/BlenderSourceTools/, this allows you to import .smd models and animations. Now you have access to all valve models and animations. You can import everything into your game.

For the textures use VTFedit, you can convert a whole folder of .vtf textures to .tga using that program, instead of doing it one by one.

( if you are using DevIL library, you need to use the .png because the .tga doesn't load the textures properly all the time ).

And if someone wants to use Source Engine 2 files, I found a source 2 decompiler here: https://github.com/Penguinwizzard/Source2ResourceDecompiler. ( haven't tried it yet)

Have fun. :wink:

It seems that backward compatibility is not possible when changing file formats.

Of course it is, and it's done all the time. Just because it wasn't done in the few instances you've observed doesn't mean it's impossible. It simply means it wasn't necessary or desirable for the developers, who could probably simply re-export their old source art into the new format more easily.

It seems that backward compatibility is not possible when changing file formats. -_-

It's not that it's impossible, but rather that we don't want to or we don't care.

Games are loading proprietary file formats and they do not owe anyone the obligation to maintain any kind of backwards or forward compatibility (as it does happen with MS Office, Maya, Blender, 3DS Max, GIMP, Photoshop, etc). These files are for internal use only.

Furthermore backwards compatibility limits flexibility & performance. A newer format may store the same data but in a different layout. For example the new format may have decided to reverse the order in which LODs are stored so they get parsed first, streamed & displayed while the higher quality LODs are still loading in the background.

Why they weren't stored in reverse order in the first place? Maybe an oversight, a mistake, or simply because years ago the engine didn't support background streaming; or mesh loading was not a performance concern at all.

Thanks. ^_^

Guys, these days I spend most of my time making 3D models, rigging and then animating them. But all the models are unbelievably ugly, and my walking and running animations are just AWFUL. :unsure: That's why I decided to stop torturing the modelling software and gave up.

I decided to steal some models from Dota 2 just to use them for fun, not for sale, of course. But guess what, Valve recently changed their engine to Source Engine 2 and instead of .mdl, the meshes are vmdl_c, vmesh_c and other crazy extensions for textures and materials and nobody on the Internet knows how to deal with this.

And not just different file names, the formats are structured differently on purpose. And I was wondering - what is the reason for this? :huh:

It's not like I can publish a game using their models, this is not allowed, of course. Why would they waste time to change everything every time? And not just Valve, I just gave it as an example, other guys do it too.

There must be something very logical, of course, I'm not criticizing ( maybe just a little bit :angry: ), but I decided to ask anyways.

( If anyone has some kind of home-made converter, PM me, pls. :rolleyes: )

Couple of reasons to change the format:
A- Performance : Maybe the previous format did not represent a utilized model properly, and instead of doing rectangles they are using a triangle fan.

B- Extensibility : Some formats are no longer extensible and hard to add new features (HD textures, proper mesh groups, etc..)

C- They lack: They could use .obj files if they wanted to export everything, but .obj files does not represent everything that's needed in their model system. So the .obj file lack needed functionality.

There are many reasons to change a format and i've just stated 3 of them. (Which I think the most common)

This topic is closed to new replies.

Advertisement