# Why do game companies use custom texture formats instead of the DDS format?

## Recommended Posts

Numsgil    501
Quote:
Original post by Yann L
Quote:
 I would argue a different way: DXT let's you use higher resolution textures with the same file size as a smaller resolution model. That is, if I use DXT5, I can use a 1024x1024 image where a 512x512 image was before, without any additional costs in terms of performance or VRAM.

Not entirely correct. Texture fetch latency and caching behaviour will be (slightly) affected, and due to more mipmap levels you'll inevitably increase memory usage (assuming DXT5).

How so? If the compressed, but larger resolution texture is the same size in memory as an uncompressed, smaller version. Also, I don't think it'll increase memory usage even counting the mip map levels (compared with an uncompressed but smaller texture + its mip maps). Or if it does, it's like 4 bytes for the one extra level :)

Quote:
 However, this was not my point. When you design your game and all artwork assets, you define a common lowest denominator for it to work on. Say you target 512MB VRAM class hardware. You will choose your resolutions and compression settings so to avoid (too much) swapping on such a card. Now, imagine some user buying a new 1GB VRAM card. Essentially, since your game was designed with 512MB in mind, half of his memory will be unused and the textures will look the same as on a 512MB card. Mr.NewCardOwner will not be happy.

In your scenario you're doing exactly what I'm saying. You're essentially designing your game with imaginary future tech cards in mind, with large texture sizes, and allowing current lowest denominator cards to run by compressing the textures.

Quote:

Quote:
 Probably the optimal version would let the artists specify an optional compression format in an options file for every texture.

I wouldn't trust an average artist to handle that [wink]

Really? Is it that you're afraid the artists won't ever choose the compression because they don't want their art to have artifacts?

Quote:
 If you think like that, then why not consider JPEG instead ? It's going to be smaller, after all. Of course there's the JPEG->DXTC transcoding, which would suck. But seriously, while the texture loading is HDD limited, it only makes a small percentage of the overall loading time of a scene.

What else are you doing in your scene? The time it takes to dump stuff (textures, models, etc.) from disk to GPU should be the bottleneck. If other stuff is slowing your loading times down, I can't help but feel you're doing something wrong. I can't imagine anything CPU intensive that should be going on during level load (except things like uncompressing from disk to make up for poor hard drive speeds). And probably random properties I guess. If you have 1000 monsters that start with random positions, that might take a while. But most games aren't like that.

Quote:
 I just can't understand how people can store their assets in a format that was never meant to be a lossy image storage format. It was designed as a compression with a GPU-friendly decoding process. It is good at doing this for now, but why use it for something where other, better alternatives exist ?

If you're storing textures on disk, you probably want to avoid JPG because it's the worst of both worlds: artifacts and uncompressed in VRAM (or worse, you're DXT compressing the JPG, adding artifacts to artifacts). That leaves lossless, where PNG is king. Or hardware accelerated lossy, where DDS makes the most sense, since it natively supports it. Or roll your own, of course, that could support either.

If/when they invent hardware accelerated JPG and PNG compression, I will totally jump on that bandwagon. But I would still probably be using a DDS format: Microsoft would probably invent some new DDS mode that lets you store as JPG or PNG compression to keep DDS current. Compression aside, DDS is the best general purpose texture format, because it's the only one specifically developed with game hardware in mind.

##### Share on other sites
Yann L    1802
Quote:
 Original post by NumsgilHow so? If the compressed, but larger resolution texture is the same size in memory as an uncompressed, smaller version. Also, I don't think it'll increase memory usage even counting the mip map levels (compared with an uncompressed but smaller texture + its mip maps). Or if it does, it's like 4 bytes for the one extra level :)

Sure, the difference is small - but saying that it is the same would be incorrect. Nitpicking, I admit ;) However, the increase of memory footprint on the cache is very much noticeable, since texture cache usually operates on decoded values.

Quote:
 Original post by NumsgilIn your scenario you're doing exactly what I'm saying. You're essentially designing your game with imaginary future tech cards in mind, with large texture sizes, and allowing current lowest denominator cards to run by compressing the textures.

So ? Set aside that such cards aren't imaginary (but haven't yet reached the required market penetration for the target audience), my approach can seamlessly improve texture quality on such cards. Yours cannot without shipping a new texture pack.

Quote:
 Really? Is it that you're afraid the artists won't ever choose the compression because they don't want their art to have artifacts?

Amongst others, that is a factor. Another one is that artists may not understand all technical implications of such a choice. We usually try to limit the options of our artists to 'art thingies' only.

Quote:
 What else are you doing in your scene? The time it takes to dump stuff (textures, models, etc.) from disk to GPU should be the bottleneck. If other stuff is slowing your loading times down, I can't help but feel you're doing something wrong.

Lol, no, I don't think so ;)

Quote:
 I can't imagine anything CPU intensive that should be going on during level load (except things like uncompressing from disk to make up for poor hard drive speeds).

Let's see - transcoding PRT coefficient maps for partially-dynamic global illumination, generating and compiling metashaders, tracing horizon maps, setting up dynamic partitioning systems, uncompressing progressive LOD data, preparing energy transfer volumes for dynamic soft shadows, generating seed and permutation tables for fractal vegetation and clouds, and so on. All this is dependent on the hardware the engine is being run on, in order to smoothly adjust the delicate performance-quality balance to match the players/users hardware and preferences. There's a lot more to a modern 3D engine than 'dumping stuff from disk to GPU'. Now granted, what we do is very high end stuff, and it's not for a game. But most of these techniques also apply to advanced game 3D engines.

Quote:
 If/when they invent hardware accelerated JPG and PNG compression, I will totally jump on that bandwagon. But I would still probably be using a DDS format: Microsoft would probably invent some new DDS mode that lets you store as JPG or PNG compression to keep DDS current. Compression aside, DDS is the best general purpose texture format, because it's the only one specifically developed with game hardware in mind.

Let's not slip into religious fanboi-isms, and stay with the facts, shall we ? DDS is definitely not the best general purpose format. DDS isn't even a general purpose format. It is a quite specific format, that suits a certain type of situation, for certain people. Take console development, for example. Using precompressed DDS on XBox makes perfect sense, because your target hardware will never change.

There's nothing wrong with using DDS - but make sure you understand the implications of doing so, that you understand the available alternatives, and that after careful consideration, you have opted that using DDS is in fact the best solution in your situation.

But advocating it as a generalized 'best format', just because you happen to be familiar with it, without taking into account the specific context of usage, is very naive and misleading.

##### Share on other sites
Kest    547
Quote:
Original post by Hodgman
Quote:
Original post by Kest
Quote:
 Also, while PNG might well be compressed it is only compressed on disk; DXT compressed images remain compressed in texture ram

Any format can be compressed once its in system ram.
IIRC, DirectX actually has the ability to load a PNG and save it as DDS, or vice versa. Once the texture is in ram, all format bets are off.
^^fixed to show apples and oranges ;)

^^fixed to show apples and oranges ;)

You didn't fix my statement, you modified it. I was referring to texture ram. Once the texture is in video ram, it no longer matters how it was stored on the disk. When, where, and how it gets processed from disk to v-ram doesn't change any of the facts.

##### Share on other sites
frob    44919
Quote:
Original post by Numsgil
Quote:
 Really? Is it that you're afraid the artists won't ever choose the compression because they don't want their art to have artifacts?

Amongst others, that is a factor. Another one is that artists may not understand all technical implications of such a choice. We usually try to limit the options of our artists to 'art thingies' only.
Ditto on that.

Sure, artists are able to pick the best format, it isn't that they will lack the skill if you give them the proper training. Some of them will understand the importance of picking the most appropriate format for the technical rather than artistic reasons, even if it reduces the artistic quality or image fidelity from the originals.

But the real problem is consistency.

They're dealing with many hundreds, even potentially several tens of thousands of graphics files over the course of the project. They're working with these files all day, every work day.

Do you honestly expect them to evaluate each file and determine the best format whenever they save their work? Every time they modify a file?

I don't. I expect a tool to do this when it builds and packages all the art assets. Automatic this task saves between hundreds and thousands of hours over the course of the project, depending on the number of artists. It simplifies their workflow, and it also gives better results for the game. Everybody wins.

In fact, we do have tools for this. The tool converts it into all the possible formats, evaluates them based on the resulting image and the data size, and picks the best one based on those results. The art programmers, technical artists, and other qualified people determine the heuristics for evaluating what is the best, allowing us to trade off size, speed, and most importantly image fidelity without sacrificing technical quality.

##### Share on other sites
Numsgil    501
Quote:
Original post by Yann L
Quote:
 If/when they invent hardware accelerated JPG and PNG compression, I will totally jump on that bandwagon. But I would still probably be using a DDS format: Microsoft would probably invent some new DDS mode that lets you store as JPG or PNG compression to keep DDS current. Compression aside, DDS is the best general purpose texture format, because it's the only one specifically developed with game hardware in mind.

Let's not slip into religious fanboi-isms, and stay with the facts, shall we ? DDS is definitely not the best general purpose format. DDS isn't even a general purpose format. It is a quite specific format, that suits a certain type of situation, for certain people. Take console development, for example. Using precompressed DDS on XBox makes perfect sense, because your target hardware will never change.

There's nothing wrong with using DDS - but make sure you understand the implications of doing so, that you understand the available alternatives, and that after careful consideration, you have opted that using DDS is in fact the best solution in your situation.

But advocating it as a generalized 'best format', just because you happen to be familiar with it, without taking into account the specific context of usage, is very naive and misleading.

Best for game development I meant :) And not counting whatever you make yourself, of course. And compression aside (yes, that's a rather large aside). Other than compression there's nothing that PNG or TGA or JPG get you above what DDS gets you, while DDS gets you some things beyond PNG et al. Like native support for cubemaps. Sure you could just do a cross in a PNG, but why would you when DDS can natively support it? You know, besides that whole compression thing.

Unless I'm missing something?

##### Share on other sites
Numsgil    501
Quote:
Original post by frob
Quote:
Original post by Numsgil
Quote:
 Really? Is it that you're afraid the artists won't ever choose the compression because they don't want their art to have artifacts?

Amongst others, that is a factor. Another one is that artists may not understand all technical implications of such a choice. We usually try to limit the options of our artists to 'art thingies' only.
Ditto on that.

Sure, artists are able to pick the best format, it isn't that they will lack the skill if you give them the proper training. Some of them will understand the importance of picking the most appropriate format for the technical rather than artistic reasons, even if it reduces the artistic quality or image fidelity from the originals.

But the real problem is consistency.

They're dealing with many hundreds, even potentially several tens of thousands of graphics files over the course of the project. They're working with these files all day, every work day.

Do you honestly expect them to evaluate each file and determine the best format whenever they save their work? Every time they modify a file?

I don't. I expect a tool to do this when it builds and packages all the art assets. Automatic this task saves between hundreds and thousands of hours over the course of the project, depending on the number of artists. It simplifies their workflow, and it also gives better results for the game. Everybody wins.

In fact, we do have tools for this. The tool converts it into all the possible formats, evaluates them based on the resulting image and the data size, and picks the best one based on those results. The art programmers, technical artists, and other qualified people determine the heuristics for evaluating what is the best, allowing us to trade off size, speed, and most importantly image fidelity without sacrificing technical quality.

I know this is something of a pipe dream, but seriously, artists should understand the technical implications of their work! This hand holding and coddling for artists that seems to be prevalent makes my stomach turn. At the very least an artists should have a basic grounding in calculus and linear algebra. They should understand the lighting model. They should be able to write their own shaders. They should understand and make decisions about how to use the hardware of video cards. I'm guessing this isn't the way things are simply because there aren't enough artists that can do that, and simple demand and lack of supply is driving the standards.

In a perfect world, I as a programmer would give a max budget to my art team. Do not exceed 160 MB. They would build their level with a mind towards that limit. If there's a far away billboard, they can save MBs by saving it as a DXT1. If there's some grass off in the distance, they might need an alpha channel and opt for DXT3 or 5. If there's a baddy that will be right in front of the player, they can go for a full 32 bits. Something in game counts the total MBs of all assets and gives it as a readback when the artists run the game. The art lead cracks down when some junior artist uses a 1024x1024 32 bit texture for a postage stamp. If render times are too high, they'd load up PerfHud and take a look at the draw commands.

I know that's not reality. I don't know if it's even achievable. Perhaps we should just be thankful when an artist makes something look good, and leave all the technical implications to the programmers.

##### Share on other sites
Zipster    2359
Quote:
 Original post by NumsgilI know this is something of a pipe dream, but seriously, artists should understand the technical implications of their work! This hand holding and coddling for artists that seems to be prevalent makes my stomach turn. At the very least an artists should have a basic grounding in calculus and linear algebra. They should understand the lighting model. They should be able to write their own shaders. They should understand and make decisions about how to use the hardware of video cards. I'm guessing this isn't the way things are simply because there aren't enough artists that can do that, and simple demand and lack of supply is driving the standards.

Calculus and linear algebra? Pipe dream indeed ;) But seriously, what you're talking about sounds a lot like a technical artist, and there are plenty of those out there if you know where to look. We just hired one a few weeks ago. They're basically programmers who also have a good understanding of the art side of development as and can use Max, Photoshop, write scripts, shaders, etc. I don't know if I'd trust a full-blown artist to write shaders though. Shaders can truly be time-critical code if you're GPU-bound, and in my experience artists will use every single feature they possibly can to achieve a better visual effect unless you pull back on the reins a bit. We have a running joke in the graphics department whenever we add a new feature, "don't tell the artists!" :P

##### Share on other sites
Kest    547
Logic versus ingenuity. You can favor both equally, be naturally pulled toward one and annoyed with the other, or somewhere in between. To get the best of both worlds, you need at least two people. Or a split personality.

##### Share on other sites
Numsgil    501
Quote:
Original post by Zipster
Quote:
 Original post by NumsgilI know this is something of a pipe dream, but seriously, artists should understand the technical implications of their work! This hand holding and coddling for artists that seems to be prevalent makes my stomach turn. At the very least an artists should have a basic grounding in calculus and linear algebra. They should understand the lighting model. They should be able to write their own shaders. They should understand and make decisions about how to use the hardware of video cards. I'm guessing this isn't the way things are simply because there aren't enough artists that can do that, and simple demand and lack of supply is driving the standards.

Calculus and linear algebra? Pipe dream indeed ;) But seriously, what you're talking about sounds a lot like a technical artist, and there are plenty of those out there if you know where to look. We just hired one a few weeks ago. They're basically programmers who also have a good understanding of the art side of development as and can use Max, Photoshop, write scripts, shaders, etc. I don't know if I'd trust a full-blown artist to write shaders though. Shaders can truly be time-critical code if you're GPU-bound, and in my experience artists will use every single feature they possibly can to achieve a better visual effect unless you pull back on the reins a bit. We have a running joke in the graphics department whenever we add a new feature, "don't tell the artists!" :P

Ha, thanks, that was a good buzz word to search for. Yes I think that's what I'm talking about. Let's just train up a small army of technical artists and leave the regular variety to concept art :D

##### Share on other sites
emeyex    382
Quote:
 Original post by Numsgil...At the very least an artist should have a basic grounding in calculus and linear algebra...

That's the best line I've heard all year! The funniest part is that you said "at the very least". But seriously, if you do hear that Leonardo da Vinci is looking for a job, let me know, I'd love to hire him :)

##### Share on other sites
Numsgil    501
Quote:
Original post by emeyex
Quote:
 Original post by Numsgil...At the very least an artist should have a basic grounding in calculus and linear algebra...

That's the best line I've heard all year! The funniest part is that you said "at the very least". But seriously, if you do hear that Leonardo da Vinci is looking for a job, let me know, I'd love to hire him :)

Yeah, I guess I have high expectations :) I guess that amounts to saying that every game artist should have minored in math.

##### Share on other sites
emeyex    382
Quote:
 Original post by NumsgilYeah, I guess I have high expectations :) I guess that amounts to saying that every game artist should have minored in math.

Nothing wrong with high expectations, I agree that it would be nice! I just wouldn't go designing your pipeline around those expectations :)

##### Share on other sites
Numsgil    501
Quote:
Original post by emeyex
Quote:
 Original post by NumsgilYeah, I guess I have high expectations :) I guess that amounts to saying that every game artist should have minored in math.

Nothing wrong with high expectations, I agree that it would be nice! I just wouldn't go designing your pipeline around those expectations :)

Probably not. I'm still learning to lower my expectations. I still think choosing a DXT compression level is something an artist can handle, though. There really aren't that many choices: none, DXT1, DXT3, DXT5. That and a texture budget should mean they make the right choice.

##### Share on other sites
frob    44919
Quote:
Original post by Numsgil
Quote:
Original post by emeyex
Quote:
 Original post by NumsgilYeah, I guess I have high expectations :) I guess that amounts to saying that every game artist should have minored in math.

Nothing wrong with high expectations, I agree that it would be nice! I just wouldn't go designing your pipeline around those expectations :)

Probably not. I'm still learning to lower my expectations. I still think choosing a DXT compression level is something an artist can handle, though. There really aren't that many choices: none, DXT1, DXT3, DXT5. That and a texture budget should mean they make the right choice.

Something they *can* choose? Sure. Any artist could be taught that fairly easily.

Something the *should* choose, every single time the export the art? That's a job for a tool.

##### Share on other sites
Numsgil    501
Quote:
Original post by frob
Quote:
Original post by Numsgil
Quote:
Original post by emeyex
Quote:
 Original post by NumsgilYeah, I guess I have high expectations :) I guess that amounts to saying that every game artist should have minored in math.

Nothing wrong with high expectations, I agree that it would be nice! I just wouldn't go designing your pipeline around those expectations :)

Probably not. I'm still learning to lower my expectations. I still think choosing a DXT compression level is something an artist can handle, though. There really aren't that many choices: none, DXT1, DXT3, DXT5. That and a texture budget should mean they make the right choice.

Something they *can* choose? Sure. Any artist could be taught that fairly easily.

Something the *should* choose, every single time the export the art? That's a job for a tool.

But how will the tool know what to compress and what to leave in full 32 bits?

##### Share on other sites
swiftcoder    18432
Quote:
Original post by Numsgil
Quote:
 Original post by frobSomething the *should* choose, every single time the export the art? That's a job for a tool.

But how will the tool know what to compress and what to leave in full 32 bits?

Heuristics, based on the compression savings, texture budgets, and quality loss (an estimation of which can be calculated by analysing the image).

##### Share on other sites
Numsgil    501
Quote:
Original post by swiftcoder
Quote:
Original post by Numsgil
Quote:
 Original post by frobSomething the *should* choose, every single time the export the art? That's a job for a tool.

But how will the tool know what to compress and what to leave in full 32 bits?

Heuristics, based on the compression savings, texture budgets, and quality loss (an estimation of which can be calculated by analysing the image).

Well, it's a fixed compression ratio, so I guess that means the heuristic would just ignore textures under some fixed size. How do you develop a quality loss estimation algorithm? The naive implementation I can think of, basically just comparing pixels and recording the difference, isn't all that effective.

##### Share on other sites
Daaark    3553
Quote:
 Original post by NumsgilThe naive implementation I can think of, basically just comparing pixels and recording the difference, isn't all that effective.
Color, brightness, saturation, contrast. There is also edge erosion which is even visible with JPEGs saved at 100%.

##### Share on other sites
Yann L    1802
An FFT can help a lot when comparing image quality by analysing the frequency domain. This can help to estimate the amount of quantization error, bluriness, and high frequency artifacts. Converting to alternative colourspaces, especially those separating chromaticity from brightness (YUV or Yxy, for example) can help at detecting subtile local colour shifts such as those common with DXTC. When combining this with psychovisual profiles, then such a quality estimation can be much more accurate than the subjective guess from a human.

##### Share on other sites
Numsgil    501
Hmm, interesting. Are there any utilities floating around the intraweb that do this (that is, plug in a texture in PNG or something and a desired compression, and get back some sort of score for quality)? Or is this the sort of thing that everyone rolls their own for?

##### Share on other sites
swiftcoder    18432
Quote:
 Original post by NumsgilHmm, interesting. Are there any utilities floating around the intraweb that do this (that is, plug in a texture in PNG or something and a desired compression, and get back some sort of score for quality)? Or is this the sort of thing that everyone rolls their own for?
There is an astounding body of research on the subject, but actual tools seem in short supply. The only one I am aware of is for video.

##### Share on other sites
frob    44919
Quote:
Original post by swiftcoder
Quote:
 Original post by NumsgilHmm, interesting. Are there any utilities floating around the intraweb that do this (that is, plug in a texture in PNG or something and a desired compression, and get back some sort of score for quality)? Or is this the sort of thing that everyone rolls their own for?
There is an astounding body of research on the subject, but actual tools seem in short supply.
A few high-end numeric algorithms libraries, many computer vision libraries, and most commercial image processing toolkits all include image fidelity computations.

I don't know of any that are free, but I know of an in-house version that works well for us. A quick Google search shows quite a few commercial libraries, and looking at a few shows them around $500 per license, or "call for details" pricing. If you are interested in doing this, finding a professional computer vision or image processing library shouldn't be too hard. Even if you do use some of the computer vision or image processing libraries, image fidelity metrics are not a single number, nor are they generic "plug in your image files engine and get a magical number" systems. They require actual work on your own tool using your own knowledge of what heuristics to use and how to intelligently compare the fidelity metrics. #### Share this post ##### Link to post ##### Share on other sites Numsgil 501 Quote:  Original post by frobI don't know of any that are free, but I know of an in-house version that works well for us. A quick Google search shows quite a few commercial libraries, and looking at a few shows them around$500 per license, or "call for details" pricing. If you are interested in doing this, finding a professional computer vision or image processing library shouldn't be too hard.Even if you do use some of the computer vision or image processing libraries, image fidelity metrics are not a single number, nor are they generic "plug in your image files engine and get a magical number" systems. They require actual work on your own tool using your own knowledge of what heuristics to use and how to intelligently compare the fidelity metrics.

Right now our artists export images from photoshop using a free DDS plugin from NVidia, choosing what format to use (DXT1, 3, 5 or uncompressed mostly). If there was a way to have a tool decide intelligently when to compress and when not to, that'd free up some work for our artists. Which would be a good thing. Not something I think we'd want to spend a lot of money on, though.

But, back to this whole coddling artist issue, at some point it would need to come down to a magic number. And they'd still have to be able to add information like "this is far away, so quality doesn't matter" and "this is close up so quality matters" and "this is a cutout where the alpha is very important", etc. So if there is a pre-existing tool that aided the artists in choosing a format (or even was smart enough to really decide entirely by itself), I'd give it a spin. Not something I'm going to want to make myself though.

Well, you just invest the 500 and the man hours to get it up and running. This is a one time cost. The time you'll lose with artists deciding (and correcting potential errors later due to incorrect decisions) are recurring costs. And if we're talking about a commercial game title here, then $500 for a license is pocket change. #### Share this post ##### Link to post ##### Share on other sites Numsgil 501 Quote: Original post by Yann L Quote:  Original post by NumsgilI'd also like to point out that when people say things like "don't let your artists decide, make the tools decide automatically", well, if you're tools just make everything a DXT5 because more complex analysis costs$500 plus a month of programmer labor, how's that better?

Well, you just invest the 500 and the man hours to get it up and running. This is a one time cost. The time you'll lose with artists deciding (and correcting potential errors later due to incorrect decisions) are recurring costs. And if we're talking about a commercial game title here, then \$500 for a license is pocket change.

I was thinking more the one programmer man month spent tweaking the heuristic. That's closer to several thousand dollars in time. For an artist, choosing which compression to use is as simple as choosing an item from a drop down box. That's an extra 5 seconds. The item chosen will be remembered for the next round of saving, so let's say something on the order of 20 seconds per texture over the project lifetime. Assuming something on the order of 2000 textures, you're talking maybe 12 hours of work over the lifetime of the project. Even adding in extra time for things like an artist figuring out that normal maps can't be DXT, it just doesn't add up as cost effective.

If there was an existing tool, I'd use it. But I can't see anyone justifying the expense of building such a tool compared with just having an artist save the file as a DDS and load it up in something like DX Texture Tool when they have problems.