In my MacBook Pro (using Bootcamp) on a GeForce GT 750M, the NVidia system information says:

Total available graphics mem: 4096MB

Dedicated video memory: 2048MB GDDR5

System video memory: 2048MB

It's advertised as a 2Gb card but I appear to be running out of video memory at just over 1Gb. I'm handling several 8192x8192 textures for terrain editing (I don't stream whilst using the editor):

2 x D3DFMT_R16F 8192x8192 for flip-flopping the vertex texture (for terrain heights)

1 x D3DFMT_L8 8192x8192 for material type

1 x X8R8G8B8 8192x8192 for terrain normal

That's 4 8192x8192 textures I'm creating - Pix says 6 with an extra D3DFMT_L8 created by Direct3D (not the application) and another X8R8G8B8 - not sure why these are being created, so I've calculated all that (including the ones created without my consent) at 939,524,096 bytes, or 939Mb. The rest of my textures/indexbuffers/vertex buffers according to Pix come to around 70Mb.

That makes it feel to me like I only have 1Gb of space to play in.

Should I be able to use that full 2Gb or not?

Thanks for the replies guys. I'm not using mipmaps for these textures, they're not for display. The 8bit one is accessed with a point filter and is used to determine which material is used at that vertex point and serves as a way for me to blend layers. The R12F ones are both vertex textures used for the heights - I have two because I'm using a shader to raise/lower/smooth the terrain and I need to pass in the vertex texture in order to average across pixels for smoothing. The normal map covers the entire terrain and I need this to happen (even in game mode) because I need that much distant detail. In game mode, I stream in the vertex textures so I have more memory then.

This gets weirder though, I got the out of memory issue when I tried to write the normal map out to disk to see what it looked like. If I don't do that, it all works fine without running out of memory - I even changed my R12F floating point vertex textures back to R32F and it still works fine. I've been flying around the 8192x8192 terrain with full normal map and umpteen detail layers with no memory issues.

I wonder what happens internally when you save texture to disk - it's only that which runs out of memory.

The fragmentation comment was interesting - as these textures are (should be) always on the GPU, are they susceptible to fragmentation or is that only when textures are loaded/unloaded/loaded, etc?

It would depend on how you're saving to disk, really. Are you using D3DXSaveSurfaceToFile or rolling your own?

Yes (albeit D3DXSaveTextureToFile)

The fragmentation comment was interesting - as these textures are (should be) always on the GPU, are they susceptible to fragmentation or is that only when textures are loaded/unloaded/loaded, etc?

I actually don't know, but I assumed. Maybe someone who knows more can answer. I don't know enough about the details of how

a D3D9 texture is managed on the GPU - but if anyone external to the driver (or whatever) holds a pointer to that memory, then it can't be moved around in the GPU's (presumably 32-bit?) address space, so I assume you'd be subject to fragmentation.

When exactly do you get the out of memory error? What API call are you making? D3DXSaveTextureToFile? So maybe it's just related to that function. Maybe it writes to a memory buffer first and it can't allocate a big enough chunk? Is this a 32-bit process?

What does the DirectX debug runtime say? Anything interesting?

It's a Win 32 bit process. The runtime just says out of memory and the HRESULT for D3DXSaveTextureToFile also returns the code for out of memory. It's quite odd, I've created a couple more 8192 x 8192 textures on top since the error and it's still working fine and they are being used. I guess I just won't call that method and roll my own.

Thanks all