I get an out of video memory D3D exception when calling
vertexBuffer = new VertexBuffer(Device, VertexPositionNormalTextureColorTangent.SizeInBytes * vertexCount, Usage.WriteOnly, SlimDX.Direct3D9.VertexFormat.None, Pool.Default);

Can I delete/clear the vertexbuffer before each call? I have about 2GB of video memory according to DXdiag. I have DXdiag debugging turned on and break on memory leaks but I dont see any messages in the output window.

Also in the code below, I am trying to avoid creating arrays on the large object heap so as to avoid fragmentation. But I am having second thoughts on my approach. I calculate the maxloh array size based on the LOH limit of 85k, im using 80 to be safe.

I calculate the capacity of each array and then create new ones when the max size is reached.

int sizeNmTxTn = maxloh / VertexPositionNormalTextureColorTangent.SizeInBytes;

I check if size of the list is greater than or equal to the maximum size allowed to avoid arrays being sent to LOH and add new ones as necessary.
if (verticesNmTxTn.Count >= sizeNmTxTn)
{
verticesNmTxTn = new List<VertexPositionNormalTextureColorTangent>(sizeNmTxTn);
verticesNmTxTxlst.Add(verticesNmTxTn);
}

What do you guys think of this approach? For managed d3d apps (slimdx), It is a pain to have to break down arrays into smaller ones to avoid them being created on LOH.
Are there better approaches to creating large vertexbuffers and not create arrays on LOH?


public void LoadVerticesNmTxTn()
{
Geometry geometry = mesh.Geometry;
Matrix meshRootTransform = getModelTransformationMatrix();

List<Vertex> vertexlst = geometry.Vertices;

vertexCount = vertexlst.Count;
int maxloh = 80000;
int sizeNmTxTn = maxloh / VertexPositionNormalTextureColorTangent.SizeInBytes;
List<List<VertexPositionNormalTextureColorTangent>> verticesNmTxTxlst = new List<List<VertexPositionNormalTextureColorTangent>>();
verticesNmTxTn = new List<VertexPositionNormalTextureColorTangent>(sizeNmTxTn);
verticesNmTxTxlst.Add(verticesNmTxTn);

foreach (Vertex vertexData in vertexlst)
{
float x = vertexData.Position.X;
float y = vertexData.Position.Y;
float z = vertexData.Position.Z;
Vector3 position = new Vector3(x, y, z);

position = Vector3.TransformCoordinate(position, meshRootTransform);

float nX = vertexData.Normal.X;
float nY = vertexData.Normal.Y;
float nZ = vertexData.Normal.Z;
Vector3 normalPosition = new Vector3(nX, nY, nZ);

//normalPosition = Vector3.TransformCoordinate(normalPosition, meshRootTransform);

float u = vertexData.TextureCoordinate.U;
float v = -(vertexData.TextureCoordinate.V - 1);
float w = vertexData.TextureCoordinate.W;

Vector2 textureCoordinate = new Vector2(u, v);

float tX = vertexData.Tangent.X;
float tY = vertexData.Tangent.Y;
float tZ = vertexData.Tangent.Z;

Vector3 tangentPosition = new Vector3(tX, tY, tZ);

float R = diffuseColor.X;
float G = diffuseColor.Y;
float B = diffuseColor.Z;
float A = diffuseColor.W;
Color4 diffuse = new Color4(A, R, G, B);

if (verticesNmTxTn.Count >= sizeNmTxTn)
{
verticesNmTxTn = new List<VertexPositionNormalTextureColorTangent>(sizeNmTxTn);
verticesNmTxTxlst.Add(verticesNmTxTn);
}
verticesNmTxTn.Add(new VertexPositionNormalTextureColorTangent(position, normalPosition, textureCoordinate, diffuse, tangentPosition));

}
if (VertexPositionNormalTextureColorTangent.SizeInBytes * vertexCount > Device.AvailableTextureMemory)
{
MessageBox.Show("Not enough texture memory available!");
}
vertexBuffer = new VertexBuffer(Device, VertexPositionNormalTextureColorTangent.SizeInBytes * vertexCount, Usage.WriteOnly, SlimDX.Direct3D9.VertexFormat.None, Pool.Default);

DataStream dataStream = vertexBuffer.Lock(0, 0, LockFlags.None);
foreach (List<VertexPositionNormalTextureColorTangent> vlist in verticesNmTxTxlst)
{
foreach (VertexPositionNormalTextureColorTangent vertex in vlist)
{
dataStream.Write<VertexPositionNormalTextureColorTangent>(vertex);
}
}

vertexBuffer.Unlock();
}


guys any input on this?

Can I delete/clear the vertexbuffer before each call?[/quote]
Yes. And I think that's actually your leak problem. Explicitly call Dispose() on your vertexBuffer. DirectX resources won't get freed by the garbage collector automatically, you have to do it manually (in the current SlimDX version always with a Dispose() call).

To further reduce payload you could consider using different types for your vertex elements (e.g. do you really need a full Color4 for the diffuse ?).

I agree with mhagain: I wonder if your LOH approach is helping here, especially since the resources are finally bound either by the DX runtime or by the driver/graphics card memory. Never played with LOH problems, but this is CLR stuff as far as I understand. Still: Would be interesting to know if you get any benefit from that approach.

Can I delete/clear the vertexbuffer before each call?

Yes. And I think that's actually your leak problem. Explicitly call Dispose() on your vertexBuffer. DirectX resources won't get freed by the garbage collector automatically, you have to do it manually (in the current SlimDX version always with a Dispose() call).

To further reduce payload you could consider using different types for your vertex elements (e.g. do you really need a full Color4 for the diffuse ?).

I agree with mhagain: I wonder if your LOH approach is helping here, especially since the resources are finally bound either by the DX runtime or by the driver/graphics card memory. Never played with LOH problems, but this is CLR stuff as far as I understand. Still: Would be interesting to know if you get any benefit from that approach.
[/quote]

well one reason for the LOH approach is because I was getting out of memory exceptions while iterating through the large VB. Managed apps have a memory cap of ~1.3gb after which they crash. but the concern is the video memory leak. I am calling dispose after each model load. like this


public void ResetScene()
{
Device.VertexDeclaration = null;

if (sceneModels != null)
{
foreach (ColladaModel cm in sceneModels)
{
if (cm.DiffuseTexture != null)
cm.DiffuseTexture.Dispose();
if (cm.NormalTexture != null)
cm.NormalTexture.Dispose();
if (cm.SpecularTexture != null)
cm.SpecularTexture.Dispose();
if (cm.ReflectionTexture != null)
cm.ReflectionTexture.Dispose();
if (!cm.VertexBuffer.Disposed)
cm.VertexBuffer.Dispose();
if (!cm.VertexDeclaration.Disposed)
cm.VertexDeclaration.Dispose();
}
sceneModels.Clear();

Device.SetStreamSource(0, null, 0, 0);
}
}


The vertexbuffer should be cleared I would assume. but somehow its keeps using up video memory until it runs out of it... I can't get rid of Color4 since I want transparency and reflection.

Really sounds like you're still leaking, even though it looks like you're disposing everything.

Enable SlimDX.Configuration.EnableObjectTracking, render once (or not at all, just upload, to narrow it further down) and investigate SlimDX.ObjectTable right before your app ends. You even got a stacktrace there for DX objects still lying around. Never used that Break on Memory Leak from the DX debug runtime (I actually really wonder how one can detect a leak before the app closes). But the leaks should show up in the log, too.

I can't get rid of Color4 since I want transparency and reflection.[/quote]
Me might have a misunderstanding here. I did not mean to get rid of elements/components but to reduce their precision. A Color4 is 4 32-bit floats. I might be wrong but for a diffuse et.al this sounds a bit much IMO. A half-float or even a byte per color channel might be enough. Same applies to the normals and tangents. Unfortunately, there's no 3-component half-float vector. But you could pack both into a two- and four-component vector and unpack them in the vertex shader. Anyway: The Geometry Compression chapter from the ShaderX 2: Tips and Tricks is worth a read.

I found out the cause of the out of video memory exception. It is because I am creating my device using CreateFlags.Multithreaded.
My viewport is actually a control that is initilized inside a wpf app. If I remove the Multithreaded flag, I start getting

Direct3D9: (WARN) :Device that was created without D3DCREATE_MULTITHREADED is being used by a thread other than the creation thread.

I recall reading somewhere that the device must be created by parent window. going to try creating the d3ddevice from my wpf app next...

sure enough the solution was to simply create my HWnD handle from the MainWindow class. Now thats what I call two birds with one stone! no more outofvideomemory exception and no more multithreaded warnings.