Quote:Original post by Yann L

Quote:
I would argue a different way: DXT let's you use higher resolution textures with the same file size as a smaller resolution model. That is, if I use DXT5, I can use a 1024x1024 image where a 512x512 image was before, without any additional costs in terms of performance or VRAM.

Not entirely correct. Texture fetch latency and caching behaviour will be (slightly) affected, and due to more mipmap levels you'll inevitably increase memory usage (assuming DXT5).

How so? If the compressed, but larger resolution texture is the same size in memory as an uncompressed, smaller version. Also, I don't think it'll increase memory usage even counting the mip map levels (compared with an uncompressed but smaller texture + its mip maps). Or if it does, it's like 4 bytes for the one extra level :)

Quote:
However, this was not my point. When you design your game and all artwork assets, you define a common lowest denominator for it to work on. Say you target 512MB VRAM class hardware. You will choose your resolutions and compression settings so to avoid (too much) swapping on such a card. Now, imagine some user buying a new 1GB VRAM card. Essentially, since your game was designed with 512MB in mind, half of his memory will be unused and the textures will look the same as on a 512MB card. Mr.NewCardOwner will not be happy.

In your scenario you're doing exactly what I'm saying. You're essentially designing your game with imaginary future tech cards in mind, with large texture sizes, and allowing current lowest denominator cards to run by compressing the textures.

Quote:

Quote:
Probably the optimal version would let the artists specify an optional compression format in an options file for every texture.

I wouldn't trust an average artist to handle that [wink]

Really? Is it that you're afraid the artists won't ever choose the compression because they don't want their art to have artifacts?

Quote:
If you think like that, then why not consider JPEG instead ? It's going to be smaller, after all. Of course there's the JPEG->DXTC transcoding, which would suck. But seriously, while the texture loading is HDD limited, it only makes a small percentage of the overall loading time of a scene.

What else are you doing in your scene? The time it takes to dump stuff (textures, models, etc.) from disk to GPU should be the bottleneck. If other stuff is slowing your loading times down, I can't help but feel you're doing something wrong. I can't imagine anything CPU intensive that should be going on during level load (except things like uncompressing from disk to make up for poor hard drive speeds). And probably random properties I guess. If you have 1000 monsters that start with random positions, that might take a while. But most games aren't like that.

Quote:
I just can't understand how people can store their assets in a format that was never meant to be a lossy image storage format. It was designed as a compression with a GPU-friendly decoding process. It is good at doing this for now, but why use it for something where other, better alternatives exist ?

If you're storing textures on disk, you probably want to avoid JPG because it's the worst of both worlds: artifacts and uncompressed in VRAM (or worse, you're DXT compressing the JPG, adding artifacts to artifacts). That leaves lossless, where PNG is king. Or hardware accelerated lossy, where DDS makes the most sense, since it natively supports it. Or roll your own, of course, that could support either.

If/when they invent hardware accelerated JPG and PNG compression, I will totally jump on that bandwagon. But I would still probably be using a DDS format: Microsoft would probably invent some new DDS mode that lets you store as JPG or PNG compression to keep DDS current. Compression aside, DDS is the best general purpose texture format, because it's the only one specifically developed with game hardware in mind.

Quote:Original post by Numsgil
How so? If the compressed, but larger resolution texture is the same size in memory as an uncompressed, smaller version. Also, I don't think it'll increase memory usage even counting the mip map levels (compared with an uncompressed but smaller texture + its mip maps). Or if it does, it's like 4 bytes for the one extra level :)

Sure, the difference is small - but saying that it is the same would be incorrect. Nitpicking, I admit ;) However, the increase of memory footprint on the cache is very much noticeable, since texture cache usually operates on decoded values.

Quote:Original post by Numsgil
In your scenario you're doing exactly what I'm saying. You're essentially designing your game with imaginary future tech cards in mind, with large texture sizes, and allowing current lowest denominator cards to run by compressing the textures.

So ? Set aside that such cards aren't imaginary (but haven't yet reached the required market penetration for the target audience), my approach can seamlessly improve texture quality on such cards. Yours cannot without shipping a new texture pack.

Quote:
Really? Is it that you're afraid the artists won't ever choose the compression because they don't want their art to have artifacts?

Amongst others, that is a factor. Another one is that artists may not understand all technical implications of such a choice. We usually try to limit the options of our artists to 'art thingies' only.

Quote:
What else are you doing in your scene? The time it takes to dump stuff (textures, models, etc.) from disk to GPU should be the bottleneck. If other stuff is slowing your loading times down, I can't help but feel you're doing something wrong.

Lol, no, I don't think so ;)

Quote:
I can't imagine anything CPU intensive that should be going on during level load (except things like uncompressing from disk to make up for poor hard drive speeds).

Let's see - transcoding PRT coefficient maps for partially-dynamic global illumination, generating and compiling metashaders, tracing horizon maps, setting up dynamic partitioning systems, uncompressing progressive LOD data, preparing energy transfer volumes for dynamic soft shadows, generating seed and permutation tables for fractal vegetation and clouds, and so on. All this is dependent on the hardware the engine is being run on, in order to smoothly adjust the delicate performance-quality balance to match the players/users hardware and preferences. There's a lot more to a modern 3D engine than 'dumping stuff from disk to GPU'. Now granted, what we do is very high end stuff, and it's not for a game. But most of these techniques also apply to advanced game 3D engines.

Quote:
If/when they invent hardware accelerated JPG and PNG compression, I will totally jump on that bandwagon. But I would still probably be using a DDS format: Microsoft would probably invent some new DDS mode that lets you store as JPG or PNG compression to keep DDS current. Compression aside, DDS is the best general purpose texture format, because it's the only one specifically developed with game hardware in mind.

Let's not slip into religious fanboi-isms, and stay with the facts, shall we ? DDS is definitely not the best general purpose format. DDS isn't even a general purpose format. It is a quite specific format, that suits a certain type of situation, for certain people. Take console development, for example. Using precompressed DDS on XBox makes perfect sense, because your target hardware will never change.

There's nothing wrong with using DDS - but make sure you understand the implications of doing so, that you understand the available alternatives, and that after careful consideration, you have opted that using DDS is in fact the best solution in your situation.

But advocating it as a generalized 'best format', just because you happen to be familiar with it, without taking into account the specific context of usage, is very naive and misleading.

Quote:Original post by Yann L

Quote:
If/when they invent hardware accelerated JPG and PNG compression, I will totally jump on that bandwagon. But I would still probably be using a DDS format: Microsoft would probably invent some new DDS mode that lets you store as JPG or PNG compression to keep DDS current. Compression aside, DDS is the best general purpose texture format, because it's the only one specifically developed with game hardware in mind.

Let's not slip into religious fanboi-isms, and stay with the facts, shall we ? DDS is definitely not the best general purpose format. DDS isn't even a general purpose format. It is a quite specific format, that suits a certain type of situation, for certain people. Take console development, for example. Using precompressed DDS on XBox makes perfect sense, because your target hardware will never change.

There's nothing wrong with using DDS - but make sure you understand the implications of doing so, that you understand the available alternatives, and that after careful consideration, you have opted that using DDS is in fact the best solution in your situation.

But advocating it as a generalized 'best format', just because you happen to be familiar with it, without taking into account the specific context of usage, is very naive and misleading.

Best for game development I meant :) And not counting whatever you make yourself, of course. And compression aside (yes, that's a rather large aside). Other than compression there's nothing that PNG or TGA or JPG get you above what DDS gets you, while DDS gets you some things beyond PNG et al. Like native support for cubemaps. Sure you could just do a cross in a PNG, but why would you when DDS can natively support it? You know, besides that whole compression thing.

Unless I'm missing something?

Quote:Original post by frob

Quote:Original post by Numsgil

Quote:
Really? Is it that you're afraid the artists won't ever choose the compression because they don't want their art to have artifacts?

Amongst others, that is a factor. Another one is that artists may not understand all technical implications of such a choice. We usually try to limit the options of our artists to 'art thingies' only.

Ditto on that.

Sure, artists are able to pick the best format, it isn't that they will lack the skill if you give them the proper training. Some of them will understand the importance of picking the most appropriate format for the technical rather than artistic reasons, even if it reduces the artistic quality or image fidelity from the originals.

But the real problem is consistency.

They're dealing with many hundreds, even potentially several tens of thousands of graphics files over the course of the project. They're working with these files all day, every work day.

Do you honestly expect them to evaluate each file and determine the best format whenever they save their work? Every time they modify a file?

I don't. I expect a tool to do this when it builds and packages all the art assets. Automatic this task saves between hundreds and thousands of hours over the course of the project, depending on the number of artists. It simplifies their workflow, and it also gives better results for the game. Everybody wins.

In fact, we do have tools for this. The tool converts it into all the possible formats, evaluates them based on the resulting image and the data size, and picks the best one based on those results. The art programmers, technical artists, and other qualified people determine the heuristics for evaluating what is the best, allowing us to trade off size, speed, and most importantly image fidelity without sacrificing technical quality.

I know this is something of a pipe dream, but seriously, artists should understand the technical implications of their work! This hand holding and coddling for artists that seems to be prevalent makes my stomach turn. At the very least an artists should have a basic grounding in calculus and linear algebra. They should understand the lighting model. They should be able to write their own shaders. They should understand and make decisions about how to use the hardware of video cards. I'm guessing this isn't the way things are simply because there aren't enough artists that can do that, and simple demand and lack of supply is driving the standards.

In a perfect world, I as a programmer would give a max budget to my art team. Do not exceed 160 MB. They would build their level with a mind towards that limit. If there's a far away billboard, they can save MBs by saving it as a DXT1. If there's some grass off in the distance, they might need an alpha channel and opt for DXT3 or 5. If there's a baddy that will be right in front of the player, they can go for a full 32 bits. Something in game counts the total MBs of all assets and gives it as a readback when the artists run the game. The art lead cracks down when some junior artist uses a 1024x1024 32 bit texture for a postage stamp. If render times are too high, they'd load up PerfHud and take a look at the draw commands.

I know that's not reality. I don't know if it's even achievable. Perhaps we should just be thankful when an artist makes something look good, and leave all the technical implications to the programmers.

Quote:Original post by Zipster

Quote:Original post by Numsgil
I know this is something of a pipe dream, but seriously, artists should understand the technical implications of their work! This hand holding and coddling for artists that seems to be prevalent makes my stomach turn. At the very least an artists should have a basic grounding in calculus and linear algebra. They should understand the lighting model. They should be able to write their own shaders. They should understand and make decisions about how to use the hardware of video cards. I'm guessing this isn't the way things are simply because there aren't enough artists that can do that, and simple demand and lack of supply is driving the standards.

Calculus and linear algebra? Pipe dream indeed ;) But seriously, what you're talking about sounds a lot like a technical artist, and there are plenty of those out there if you know where to look. We just hired one a few weeks ago. They're basically programmers who also have a good understanding of the art side of development as and can use Max, Photoshop, write scripts, shaders, etc. I don't know if I'd trust a full-blown artist to write shaders though. Shaders can truly be time-critical code if you're GPU-bound, and in my experience artists will use every single feature they possibly can to achieve a better visual effect unless you pull back on the reins a bit. We have a running joke in the graphics department whenever we add a new feature, "don't tell the artists!" :P

Ha, thanks, that was a good buzz word to search for. Yes I think that's what I'm talking about. Let's just train up a small army of technical artists and leave the regular variety to concept art :D