• Advertisement
Sign in to follow this  
  • entries
  • comments
  • views

Texture memory virtualization

Sign in to follow this  


So, I've decided to try out full texture memory virtualization of a scene, as opposed to just the terrain (which was ending up being a lot like clipmaps, really). I figure I'll talk about it a bit, and then discuss how I'm going to go about it. Note that I will not be referring to D3D10 in this discussion, even though I know it has virtual memory. This is from a purely D3D9 perspective. Why bother with older tech, though? Well, D3D9 is going to be around for 4 or 5 years at least, so a lot of research is still really applicable for it, in my opinion. Kind of like how, in some cases, some people are still looking at 1.x shaders, even though DX8 is now more than 5 years old.

First, a look at what we actually use in graphics. As an example, let's have a frame rendered at 1024x768, with mipmapping and aniso filtering on (make it a bad case and say a 2:1 texel to pixel ratio), and 3 textures for each object onscreen (diffuse, normal data, and some specular stuff. Let's say that that data is all 3Bpt). It doesn't matter how hi res any of the source data is. As long as it satisfies those requirements, you only need 13.5MB of data onscreen, and that's a pretty bad case. Oh, and that's without even thinking about data compression.

Let that number sink in a bit:
That's 10.5% of the total texture memory on an average video card with 128MB of memory.
At 60fps, and needing a COMPLETELY new set of data every frame, that's 810MB/s of data transfer, less than 40% of the full AGP8x transfer rate.

And yet, here we are, with cards on the market that have 4 times that data; and PCIe 16x now in existence with 2 times the transfer rate.

Where is this absolutely ridiculous need coming from? Why do we think that regardless of an object's position, orientation, and visibility that we need to send to the video card 1024*1024*4/3*4Bpp = 5.3MB (the 4/3 is due to mipmapping) for a hires texture and just forget about it? It could be 4 pixels wide but we'd still kindly ask API to use the full 5.3MB of data even though you're using less than a kilobyte of it. And what about future games (e.g. UT2007 and other UE3 titles), where we're going even HIGHER than that, with 2048*2048*4/3*4Bpp = 20MB of data for a SINGLE texture, WHEN WE CANNOT EVEN SEE THAT FULL TEXTURE, EVER?!

This is just a huge waste, and the answer to that is memory virtualization. Obviously, we don't have the hardware to do that, so the programmers have to do it manually. Basically this boils down to sending to the video card only the data that we need in a frame, all the way down to 64x64 or 32x32 chunks of a given mipmap.

Which leads into my solution. It's still a high level design, but I see no real problems with this:
  • Draw the scene using a simple software rasterizer (kind of a slightly more functional version of Yann L's occ query softrast system, I'll explain the motivation later)

  • Use that data to figure out what tiles of mipmap levels will be needed, and what the arrangement of the primary texture will be (this is probably going to be the hardest part, I'll talk about it later)

  • Copy the data from system memory to a texture in the SYSTEMMEM pool

  • Add the necessary dirty rects to the target texture in the DEFAULT pool (which will be 2048x2048 for lower resolutions. One dimension might have to be increased to 4096 for higher res's though)

  • Call UpdateTexture so that the data is passed to the GPU, and ask the API to generate a few mipmaps from that data (OR, call UpdateSurf to fill in the mipmaps as needed)

  • Setup a stream of texcoords for the objects so that the texture coordinates go at the right spot on the main texture

  • Wait for a tumbleweed to pass by, reach for my revolver and call out "DRAW!"

  • So, motivation for a software rasterizer: Well, like how Yann used one to determine occlusion of whole characters, a softrast could be used to determine occlusion of sub-triangle data. For example, let's take a look at this picture I found on a GIS:

    (click for bigger image)
    With a softrast, you can say that parts of the books behind the kids head are not visible, and that the hidden texture data is not needed. Also, consider the stacks of floppies and CDs to his right. Only a sliver of the data on some of those disks or cases is visible, so a softrast could easily tell you that exactly what parts are not needed. Even without those considerations though, it can help out in determining what mipmaps of textures are needed based on aniso filtering. Heck, an image like that is almost the poster child (no pun intended) for why you'd want a softrast to help with the paging of only necessary data.

    The arrangement of the tiles is going to be a bit hairier though, and as I said will be the hardest part of this. The reasons are two fold:
  • Continuity will be required between tiles. I can't just slap the tiles around randomly.

  • Retain data in the same location when possible. Since the continuity might require a texture to end up going into an area that some other texture already occupies, the second or first texture might have to be moved. There are not any practical ways of moving this data, so the texture being moved would have to be totally reloaded. I know I said before that AGP transfer speeds are insane, but that doesn't mean I can just keep wailing on it like crazy.

  • Lastly, what're the benefits that all of this will bring?
  • Ability to have hi-res textures everywhere (including ones that may be larger than the hardware could handle, e.g. for terrain)

  • There's only one texture bound for all of the drawing (in most cases; it could result in two for cases of high resolution plus lack of 4kx4k texture support) resulting in some nice batching. This means that 100% unique textures can be used practically.

  • A LOT of textures can be used without the video card keeling over in pain, e.g. with the stacks of discs in the above photo.

  • Relatively light memory usage, so even non hardware enthusiasts with GeForce 6200s or X300s can still see some kickass texturing.

  • So, that's what I'm working on right now, and I really hope that it turns out nicely.
    Sign in to follow this  


    Recommended Comments

    There are no comments to display.

    Create an account or sign in to comment

    You need to be a member in order to leave a comment

    Create an account

    Sign up for a new account in our community. It's easy!

    Register a new account

    Sign in

    Already have an account? Sign in here.

    Sign In Now

    • Advertisement