Back to General and Gameplay Programming

Why do game companies use custom texture formats instead of the DDS format?

General and Gameplay Programming Programming

Started by EmptyVoid August 26, 2008 03:10 AM

84 comments, last by Zipster 15 years, 7 months ago

46,221

August 28, 2008 06:23 AM

Quote:Original post by Kylotan
Quote:Original post by frob
Quote:Original post by ndatxcod
The point of protecting data is not making it impossible to access but to deter most users wanting to take a peek at it, or even to steal it for their own projects, also that data comes usually from third party vendors (sounds and music) who require you to protect it before shipping the product.

I'm going to repeat this line, only because it is true: Do not try to obscure, encrypt, or otherwise hide your data. There is no point to it.

Frob, did you read his post? I've selectively quoted the important part. It is required. It may be practically pointless, but it doesn't make it any less necessary.

No, I don't need to reread it. I saw that and hoped it was addressed clearly enough.

I've worked with several companies that had the requirement. Their goal has been to simply remove it from common container formats so people don't take it.

Every one of them have been satisfied by simply packing the data in something other than .wav, .mp3, .tga, .mb, or whatever other 'original' format the stuff took.

Since that is exactly what we are doing, repackaging it for our memory needs rather than their portable formats, it tends to fill the requirement.

Kest

547

August 28, 2008 08:59 AM

On the protection front, has anyone considered that obscuring/hiding data may be beneficial to players? Letting them easily browse through your assets is essentially like allowing theater play-goers to see your actors dressing up for their parts.

Everyone is curious, but not everyone really wants to see.

As for the OP question, I don't use the DDS format because it's not as widely supported by the various software I use for development. The PNG format is lossless, compressed, supports transparency, and is supported by every application I've come across. It's been ideal. My game stores all resource data, including textures, into a single game data file. There's no encryption on it, but one would need to write specialized software to open it.

EmptyVoid

102

Author

August 28, 2008 09:25 AM

So just wondering, does the png format support volume textures and cubemaps?

This is your life, and it's ending one minute at a time. - Fight club

swiftcoder

18,997

August 28, 2008 09:44 AM

Quote:Original post by EmptyVoid
So just wondering, does the png format support volume textures and cubemaps?

Volume textures no. Cubemaps can be trivially stored in any image format by storing by layting out the sides next to each other (typically in a cross shape), ala SOIL, Horde, etc.

Tristam MacDonald. Ex-BigTech Software Engineer. Future farmer. [https://trist.am]

frob

46,221

August 28, 2008 01:54 PM

Quote:Original post by Kest
On the protection front, has anyone considered that obscuring/hiding data may be beneficial to players? Letting them easily browse through your assets is essentially like allowing theater play-goers to see your actors dressing up for their parts.

Everyone is curious, but not everyone really wants to see.

Quite a few games do provide them in easy formats.

Many games, even from megacorps and major publishers, have audio files directly in mp3, wma, ogg, or other formats, ready for your listening pleasure. If you look on console game disks, you can sometimes find these.

Occasionally you can find cut scenes in wma, swf, or mpg formats. Some have .bik and .smk files, which are less obviously movies for the uninitiated.

In homebrew and F/OSS games you see games that include a ton of image files, but they are very rare in larger commercial games, for reasons discussed above.

_the_phantom_

11,263

August 28, 2008 01:59 PM

Quote:Original post by Kest
As for the OP question, I don't use the DDS format because it's not as widely supported by the various software I use for development. The PNG format is lossless, compressed, supports transparency, and is supported by every application I've come across. It's been ideal. My game stores all resource data, including textures, into a single game data file. There's no encryption on it, but one would need to write specialized software to open it.

What applications don't support DDS then?

Also, while PNG might well be compressed it is only compressed on disk; DXT compressed images remain compressed in texture ram, which is basically 'free' AND has the added benifit of cutting down bandwidth requirements for texture sampling as well as making better use of the texture cache on the chips (again, removing bandwidth pressure). For many images, even if the compression is lossy, this doesn't matter as you simply don't notice.

Granted, not everything needs to be compressed (much like you shouldn't turn on mipmaping and AF filtering by default for all textures) but it can be a good thing to take advantage of.

Yann L

1,806

August 28, 2008 02:25 PM

Quote:Original post by phantom
Also, while PNG might well be compressed it is only compressed on disk; DXT compressed images remain compressed in texture ram, which is basically 'free' AND has the added benifit of cutting down bandwidth requirements for texture sampling as well as making better use of the texture cache on the chips (again, removing bandwidth pressure). For many images, even if the compression is lossy, this doesn't matter as you simply don't notice.

As I said somewhere above, it doesn't make sense to compare on-disk storage formats and in-VRAM formats, since they can be converted on the fly. And it's not slower either. We have a threaded system that will load and decode a (container-wrapped) PNG on one core, while at the same time encoding and uploading the previous DXT texture on the second core. The current bottleneck of the system is the harddisk. In fact, it is even faster than the loading of precompressed DDS, since PNGs will often require less harddisk IO to load. And we have the added benefit of letting the user choose a quality compression level in the preferences (eg. when he switches from a 512MB to a 1GB VRAM graphics card...)

Oh, and btw, of course PNG can store volume textures, texture arrays, cubemaps and mipmaps.

swiftcoder

18,997

August 28, 2008 02:51 PM

Quote:Original post by phantom
What applications don't support DDS then?

Pretty much anything on a Mac [smile]
Seriously, there is exactly one external tool (Aorta) which produces DDS on the Mac.

Quote:Original post by Yann L
Oh, and btw, of course PNG can store volume textures, texture arrays, cubemaps and mipmaps.

For some definition of store. Yes, you can store pretty much any aribtrary data in a PNG wrapper, but it doesn't provide anything much in the way of support for volume textures or mipmaps - i.e. no metadata (depth for volume textures, mipmap count for mipmaps, etc.).

Tristam MacDonald. Ex-BigTech Software Engineer. Future farmer. [https://trist.am]

Numsgil

501

August 28, 2008 03:05 PM

Quote:Original post by Yann L
Quote:Original post by phantom
Also, while PNG might well be compressed it is only compressed on disk; DXT compressed images remain compressed in texture ram, which is basically 'free' AND has the added benifit of cutting down bandwidth requirements for texture sampling as well as making better use of the texture cache on the chips (again, removing bandwidth pressure). For many images, even if the compression is lossy, this doesn't matter as you simply don't notice.

As I said somewhere above, it doesn't make sense to compare on-disk storage formats and in-VRAM formats, since they can be converted on the fly. And it's not slower either. We have a threaded system that will load and decode a (container-wrapped) PNG on one core, while at the same time encoding and uploading the previous DXT texture on the second core. The current bottleneck of the system is the harddisk. In fact, it is even faster than the loading of precompressed DDS, since PNGs will often require less harddisk IO to load. And we have the added benefit of letting the user choose a quality compression level in the preferences (eg. when he switches from a 512MB to a 1GB VRAM graphics card...)

Oh, and btw, of course PNG can store volume textures, texture arrays, cubemaps and mipmaps.

You should try compressing the DDSs on disk. So you take your texture, store it as a DDS using a lossy compression, and then compress that image using some lossless compression algorithm. It sounds weird but it should improve load times.

[size=2]Darwinbots - [size=2]Artificial life simulation

Yann L

1,806

August 28, 2008 03:57 PM

Quote:Original post by swiftcoder
Quote:Original post by Yann L
Oh, and btw, of course PNG can store volume textures, texture arrays, cubemaps and mipmaps.
For some definition of store. Yes, you can store pretty much any aribtrary data in a PNG wrapper, but it doesn't provide anything much in the way of support for volume textures or mipmaps - i.e. no metadata (depth for volume textures, mipmap count for mipmaps, etc.).

PNG supports meta data chunks. We use them very successfully to store everything I listed above, and more. The only thing PNG can't be used for is HDR/fp textures (at least not without major modifications of the format). We designed special Photoshop and Maya plugins that transparently handle the volume/cubemap/mipmap data directly without user intervention.

Quote:Original post by Numsgil
You should try compressing the DDSs on disk. So you take your texture, store it as a DDS using a lossy compression, and then compress that image using some lossless compression algorithm. It sounds weird but it should improve load times.

We don't want our on-disk assets to be lossy compressed. Our engine is used for highend architectural visualization, and customers often complain about DXTC artifacts that are visible on smooth and subtile colour gradients, as they are used in modern interior design. So we need the option to turn off DXT compression without changing the assets.

Besides that, I'm not sure how DXTC would actually compress. DXTC is a pretty crude compression system design for speed rather than quality. It generates a lot of quantization errors that could limit the way a lossless compressor may process the data. But I never tried it. Could be interesting to do.

Why do game companies use custom texture formats instead of the DDS format?

This topic is closed to new replies.

Popular Topics

Recommended Tutorials

Why do game companies use custom texture formats instead of the DDS format?

This topic is closed to new replies.

Popular Topics

Recommended Tutorials

Reticulating splines