DX11 GPUList<T>: A C# list class with a DX11 Buffer backend

This topic is 2471 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

Recommended Posts

Hey everyone,

So earlier today, I was playing with some compute shader stuff, and I found myself getting very frustrated dealing with buffers, copying the data back and forth, ect...
I thought, it would be really nice if there was some kind of pretty C# interface to a list of data stored on the GPU...

Well, I never found one, so I wrote my own! I would like to share it in hopes that it will cure at least one more headache.

Its essentially an implementation of IList<T> which is backed using a SlimDX Buffer.
Any operation you do directly affects this underlying buffer.
Therefor, its very easy to make a list of data to give / receive to / from your shaders.

Its not perfect, but it gets the job done. Its pretty, easy to use, and (somewhat) tested.

Source code for the List and a small test program are below.
Please let me know if you have comments, suggestions, fixes, or find any bugs.

~Chris

[ps I tried attaching the .cs files directly but it doesn't like them?]

 using System; using System.Collections.Generic; using System.Linq; using SlimDX; using SlimDX.Direct3D11; using SlimDX.DXGI; using SlimDX.D3DCompiler; using Device = SlimDX.Direct3D11.Device; using Buffer = SlimDX.Direct3D11.Buffer; namespace GPUTools { class Program { /// <summary> /// Simple structure representing some kind of particle /// </summary> struct Particle { public Vector3 Position; public Vector3 Velocity; public override string ToString() { return "Position: " + Position.ToString() + ", Velocity: " + Velocity.ToString(); } } static void Main(string[] args) { // Source code for the simple integration compute shader const string integrateSource = @" struct Particle { float3 position; float3 velocity; }; RWStructuredBuffer<Particle> particles; [numthreads(1, 1, 1)] void Integrate(uint3 threadID : SV_DispatchThreadID) { particles[threadID.x].position += particles[threadID.x].velocity; } "; // Make device Device device = new Device(DriverType.Hardware, DeviceCreationFlags.None, FeatureLevel.Level_10_0); // Compile compute shader ShaderBytecode integrateBytecode = ShaderBytecode.Compile(integrateSource, "Integrate", "cs_4_0", ShaderFlags.None, EffectFlags.None); ComputeShader integrate = new ComputeShader(device, integrateBytecode); // Create GPUList of particles using the immediate context GPUList<Particle> particles = new GPUList<Particle>(device.ImmediateContext); // Add some particles for (int i = 0; i < 10; i++) particles.Add(new Particle() { Position = new Vector3(i, 0, 0), Velocity = new Vector3(0, i, -i) }); // Print the particles Console.WriteLine("Before Integration:"); foreach (Particle p in particles) Console.WriteLine(p.ToString()); // Run the compute shader device.ImmediateContext.ComputeShader.Set(integrate); device.ImmediateContext.ComputeShader.SetUnorderedAccessView(particles.UnorderedAccess, 0); device.ImmediateContext.Dispatch(particles.Count, 1, 1); // Print the particles again Console.WriteLine("\n\nAfter Integration:"); foreach (Particle p in particles) Console.WriteLine(p.ToString()); Console.Read(); } } } 

 /* Who: Chris Giles * What: A GPU-based implementation of List<T> * When: 5/16/2011 @ ~4 AM * Where: My house * Why: I was tired of dealing with low level graphics resources * This is almost completely interchangable with built in C# List<T> * Only, the backend is a DirectX11 Buffer * You can use the underlying buffer in any kind of shader * But with a pretty C# interface for messing with its data */ using System; using System.Collections.Generic; using System.Linq; using System.Runtime.InteropServices; using SlimDX; using SlimDX.Direct3D11; using Device = SlimDX.Direct3D11.Device; using Buffer = SlimDX.Direct3D11.Buffer; namespace GPUTools { public class GPUList<T> : IList<T>, IDisposable where T : struct { #region GPUList Members private DeviceContext context; private Buffer buffer; private ShaderResourceView shaderResourceView; private UnorderedAccessView unorderedAccessView; private int count; private int capacity; /// <summary> /// Makes a new list given initial data /// </summary> /// <param name="context">Device context on which to perform all operations</param> public GPUList(DeviceContext context) { this.context = context; } /// <summary> /// Makes a new empty list with an initial capacity /// </summary> /// <param name="context">Device context on which to perform all operations</param> /// <param name="initialCapacity">The initial capacity</param> public GPUList(DeviceContext context, int initialCapacity) { this.context = context; this.setCapacity(initialCapacity); } /// <summary> /// Makes a new list given initial data /// </summary> /// <param name="context">Device context on which to perform all operations</param> /// <param name="data">The initial data</param> public GPUList(DeviceContext context, IEnumerable<T> data) { this.context = context; this.AddRange(data); } /// <summary> /// Gets the DeviceContext on which to perform all operations /// </summary> public DeviceContext Context { get { return context; } } /// <summary> /// Gets the underlying Buffer stored on the GPU /// </summary> public Buffer GPUBuffer { get { return buffer; } } /// <summary> /// Gets a ShaderResourceView for use in a shader /// </summary> public ShaderResourceView ShaderResource { get { return shaderResourceView; } } /// <summary> /// Gets a UnorderedAccessView for use in a shader /// </summary> public UnorderedAccessView UnorderedAccess { get { return unorderedAccessView; } } /// <summary> /// Gets or Sets the internal GPU buffer capacity in number of elements /// </summary> public int Capacity { get { return capacity; } set { setCapacity(value); } } /// <summary> /// Adds a range of items to the end of the list /// </summary> /// <param name="range"></param> public void AddRange(IEnumerable<T> range) { if (range == null) throw new ArgumentNullException(); int dataSize = Marshal.SizeOf(typeof(T)); if (range is GPUList<T> && range != this) { GPUList<T> list = range as GPUList<T>; if (list.Count == 0) return; if (Count + list.Count > Capacity) setCapacity(2 * Count + list.Count); count += list.Count; list.CopyRangeTo(0, list.Count, this, Count - list.Count); } else { T[] data = null; if (range is T[]) data = (T[])range; else if (range is List<T>) data = (range as List<T>).ToArray(); else { List<T> dataList = new List<T>(range); data = dataList.ToArray(); } if (data.Length == 0) return; if (Count + data.Length > Capacity) setCapacity(2 * Count + data.Length); count += data.Length; SetRange(Count - data.Length, data, 0, data.Length); } } /// <summary> /// Inserts a range of items into the list /// </summary> /// <param name="index">Position in the list to insert items</param> /// <param name="range">The range of items to insert</param> public void InsertRange(int index, IEnumerable<T> range) { if (range == null) throw new ArgumentNullException(); if (index < 0 || index > Count) throw new IndexOutOfRangeException(); if (index == Count) { AddRange(range); } else { GPUList<T> newList = null; if (range is GPUList<T>) { GPUList<T> list = range as GPUList<T>; if (list.Count == 0) return; newList = new GPUList<T>(Context, list.Count + Count); newList.count = list.Count + Count; this.CopyRangeTo(0, index, newList, 0); list.CopyTo(newList, index); this.CopyRangeTo(index, Count - index, newList, index + list.Count); } else { T[] data = null; if (range is T[]) data = (T[])range; else if (range is List<T>) data = (range as List<T>).ToArray(); else { List<T> dataList = new List<T>(range); data = dataList.ToArray(); } if (data.Length == 0) return; newList = new GPUList<T>(Context, data.Length + Count); newList.count = data.Length + Count; this.CopyRangeTo(0, index, newList, 0); newList.SetRange(index, data, 0, data.Length); this.CopyRangeTo(index, Count - index, newList, index + data.Length); } setCapacity(0); count = newList.count; capacity = newList.capacity; buffer = newList.buffer; shaderResourceView = newList.shaderResourceView; unorderedAccessView = newList.unorderedAccessView; } } /// <summary> /// Removes a range of items from the list /// </summary> /// <param name="index">Index to start removing</param> /// <param name="count">Number of items to remove</param> public void RemoveRange(int index, int count) { if (count == 0) return; if (count < 0) throw new IndexOutOfRangeException(); if (index < 0 || index >= Count) throw new IndexOutOfRangeException(); if (index + count < 0 || index + count > Count) throw new IndexOutOfRangeException(); if (index + count == Count) { this.count -= count; return; } GPUList<T> newList = new GPUList<T>(Context, Capacity); newList.count = this.count - count; this.CopyRangeTo(0, index, newList, 0); this.CopyRangeTo(index + count, this.count - (index + count), newList, index); setCapacity(0); this.count = newList.count; this.capacity = newList.capacity; this.buffer = newList.buffer; this.shaderResourceView = newList.shaderResourceView; this.unorderedAccessView = newList.unorderedAccessView; } /// <summary> /// Sets a range of data in the list based on an array /// </summary> /// <param name="index">Index in the GPUList to start setting data</param> /// <param name="array">The array of data to use</param> /// <param name="arrayIndex">The start index in the array</param> /// <param name="count">The number of elements to set</param> public void SetRange(int index, T[] array, int arrayIndex, int count) { if (count == 0) return; if (count < 0) throw new IndexOutOfRangeException(); if (index < 0 || index >= Count) throw new IndexOutOfRangeException(); if (index + count < 0 || index + count > Count) throw new IndexOutOfRangeException(); if (array == null) throw new ArgumentNullException(); if (arrayIndex < 0 || arrayIndex >= array.Length) throw new IndexOutOfRangeException(); if (arrayIndex + array.Length < 0 || arrayIndex + count > array.Length) throw new IndexOutOfRangeException(); int dataSize = Marshal.SizeOf(typeof(T)); DataStream stream = new DataStream(array, true, false); stream.Seek(arrayIndex, System.IO.SeekOrigin.Begin); context.UpdateSubresource(new DataBox(0, 0, stream), buffer, 0, new ResourceRegion(dataSize * index, 0, 0, dataSize * (index + count), 1, 1)); } /// <summary> /// Copies a range of data to an array /// </summary> /// <param name="index">Index from which to start copying</param> /// <param name="count">Number of elements to copy</param> /// <param name="array">Destination array to copy to</param> /// <param name="arrayIndex">Destination array index to copy to</param> public void CopyRangeTo(int index, int count, T[] array, int arrayIndex) { if (count == 0) return; if (count < 0) throw new IndexOutOfRangeException(); if (index < 0 || index >= Count) throw new IndexOutOfRangeException(); if (index + count < 0 || index + count > Count) throw new IndexOutOfRangeException(); if (array == null) throw new ArgumentNullException(); if (arrayIndex < 0 || arrayIndex >= array.Length) throw new IndexOutOfRangeException(); if (arrayIndex + array.Length < 0 || arrayIndex + count > array.Length) throw new IndexOutOfRangeException(); int dataSize = Marshal.SizeOf(typeof(T)); Buffer stagingBuffer = new Buffer(context.Device, dataSize * count, ResourceUsage.Staging, BindFlags.None, CpuAccessFlags.Read, ResourceOptionFlags.None, 0); context.CopySubresourceRegion(buffer, 0, new ResourceRegion(dataSize * index, 0, 0, dataSize * (index + count), 1, 1), stagingBuffer, 0, 0, 0, 0); DataBox box = context.MapSubresource(stagingBuffer, 0, dataSize * count, MapMode.Read, SlimDX.Direct3D11.MapFlags.None); box.Data.ReadRange<T>(array, arrayIndex, count); context.UnmapSubresource(stagingBuffer, 0); stagingBuffer.Dispose(); } /// <summary> /// Copies a range of data to another GPUList /// </summary> /// <param name="index">Index from which to start copying</param> /// <param name="count">Number of elements to copy</param> /// <param name="list">Destination list to copy to</param> /// <param name="listIndex">Destination list index to copy to</param> public void CopyRangeTo(int index, int count, GPUList<T> list, int listIndex) { if (count == 0) return; if (list == this) throw new Exception("Cannot copy range into self...yet"); if (count < 0) throw new IndexOutOfRangeException(); if (index < 0 || index >= Count) throw new IndexOutOfRangeException(); if (index + count < 0 || index + count > Count) throw new IndexOutOfRangeException(); if (list == null) throw new ArgumentNullException(); if (listIndex < 0 || listIndex >= list.Count) throw new IndexOutOfRangeException(); if (listIndex + list.Count < 0 || listIndex + count > list.Count) throw new IndexOutOfRangeException(); int dataSize = Marshal.SizeOf(typeof(T)); context.CopySubresourceRegion(buffer, 0, new ResourceRegion(dataSize * index, 0, 0, dataSize * (index + count), 1, 1), list.buffer, 0, dataSize * listIndex, 0, 0); } /// <summary> /// Copys the list to another GPUList /// </summary> /// <param name="array">The list to copy data to</param> /// <param name="arrayIndex">The starting index in the array</param> public void CopyTo(GPUList<T> list, int listIndex) { CopyRangeTo(0, count, list, listIndex); } /// <summary> /// Trims excess capacity on the internal GPU buffer to match the count /// </summary> public void TrimExcess() { setCapacity(Count); } /// <summary> /// Returns a copy of the list into an array /// </summary> /// <returns></returns> public T[] ToArray() { T[] data = new T[Count]; CopyTo(data, 0); return data; } /// <summary> /// Sets the internal capacity of the list /// </summary> /// <param name="newCapacity"></param> private void setCapacity(int newCapacity) { if (newCapacity < 0) throw new ArgumentOutOfRangeException(); if (newCapacity == capacity) return; if (newCapacity == 0) { if (unorderedAccessView != null) unorderedAccessView.Dispose(); if (shaderResourceView != null) shaderResourceView.Dispose(); if (buffer != null) buffer.Dispose(); unorderedAccessView = null; shaderResourceView = null; buffer = null; capacity = count = 0; return; } int dataSize = Marshal.SizeOf(typeof(T)); Buffer newBuffer = new Buffer(context.Device, dataSize * newCapacity, ResourceUsage.Default, BindFlags.ShaderResource | BindFlags.UnorderedAccess, CpuAccessFlags.None, ResourceOptionFlags.StructuredBuffer, dataSize); int sizeToCopy = Math.Min(newCapacity, Count); if (sizeToCopy > 0) context.CopySubresourceRegion(buffer, 0, new ResourceRegion(0, 0, 0, dataSize * sizeToCopy, 1, 1), newBuffer, 0, 0, 0, 0); count = sizeToCopy; capacity = newCapacity; if (unorderedAccessView != null) unorderedAccessView.Dispose(); if (shaderResourceView != null) shaderResourceView.Dispose(); if (buffer != null) buffer.Dispose(); buffer = newBuffer; unorderedAccessView = new UnorderedAccessView(context.Device, buffer); shaderResourceView = new ShaderResourceView(context.Device, buffer); } #endregion #region IDisposable Members /// <summary> /// Disposes the list by destroying all GPU resources used /// </summary> public void Dispose() { setCapacity(0); } #endregion #region IList<T> Members /// <summary> /// Adds one item to the end of the list /// </summary> /// <param name="item">The item to add</param> public void Add(T item) { T[] array = new T[] { item }; AddRange(array); } /// <summary> /// Gets the index in the list of the first item found /// </summary> /// <param name="item">The item whose index is searched for</param> /// <returns>Index of the item found, or -1 if not found</returns> public int IndexOf(T item) { T[] data = new T[count]; CopyTo(data, 0); for (int i = 0; i < data.Length; i++) if (data.Equals(item)) return i; return -1; } /// <summary> /// Inserts an item at a specific index /// </summary> /// <param name="index">The index in the list to insert the item</param> /// <param name="item">The item to insert</param> public void Insert(int index, T item) { T[] array = new T[] { item }; InsertRange(index, array); } /// <summary> /// Removes an item at a specific index /// </summary> /// <param name="index"></param> public void RemoveAt(int index) { RemoveRange(index, 1); } /// <summary> /// Gets or Sets an element at some index in the list /// </summary> /// <param name="index">Index in the list to get or set</param> /// <returns></returns> public T this[int index] { get { if (index < 0 || index >= count) throw new IndexOutOfRangeException(); T[] array = new T[1]; CopyRangeTo(index, 1, array, 0); return array[0]; } set { if (index < 0 || index >= count) throw new IndexOutOfRangeException(); T[] array = new T[] { value }; SetRange(index, array, 0, 1); } } #endregion #region ICollection<T> Members /// <summary> /// Clears the list of all elements /// </summary> public void Clear() { count = 0; } /// <summary> /// Returns true if the item is contained in the list, or false if not /// </summary> /// <param name="item">The item to check containment for</param> /// <returns></returns> public bool Contains(T item) { foreach (T i in this) if (i.Equals(item)) return true; return false; } /// <summary> /// Copys the list to an array /// </summary> /// <param name="array">The array to copy data to</param> /// <param name="arrayIndex">The starting index in the array</param> public void CopyTo(T[] array, int arrayIndex) { CopyRangeTo(0, count, array, arrayIndex); } /// <summary> /// Gets the number of items in the list /// </summary> public int Count { get { return count; } } /// <summary> /// Returns whether or not the list is read only /// </summary> public bool IsReadOnly { get { return false; } } /// <summary> /// Removes an item from the list. Only the first item found is removed /// </summary> /// <param name="item">The item to remove</param> /// <returns>True if the item is removed, false otherwise</returns> public bool Remove(T item) { int index = IndexOf(item); if (index >= 0) RemoveAt(index); return index >= 0; } #endregion #region IEnumerable<T> Members /// <summary> /// Gets the enumerator for this list. Any changes made while enumerating will not be reflected /// </summary> /// <returns></returns> public IEnumerator<T> GetEnumerator() { T[] array = new T[count]; CopyTo(array, 0); return array.AsEnumerable<T>().GetEnumerator(); } #endregion #region IEnumerable Members /// <summary> /// Gets the enumerator for this list. Any changes made while enumerating will not be reflected /// </summary> /// <returns></returns> System.Collections.IEnumerator System.Collections.IEnumerable.GetEnumerator() { return GetEnumerator(); } #endregion } } 

• 10
• 12
• 10
• 10
• 11
• Similar Content

• Hi there, this is my first post in what looks to be a very interesting forum.
I am using DirectXTK to put together my 2D game engine but would like to use the GPU depth buffer in order to avoid sorting back-to-front on the CPU and I think I also want to use GPU instancing, so am I looking at implementing my own sprite rendering?

• I am trying to draw a screen-aligned quad with arbitrary sizes.

currently I just send 4 vertices to the vertex shader like so:
pDevCon->IASetPrimitiveTopology(D3D_PRIMITIVE_TOPOLOGY_TRIANGLESTRIP);
pDevCon->Draw(4, 0);

then in the vertex shader I am doing this:
float4 main(uint vI : SV_VERTEXID) : SV_POSITION
{
float2 texcoord = float2(vI & 1, vI >> 1);
return float4((texcoord.x - 0.5f) * 2, -(texcoord.y - 0.5f) * 2, 0, 1);
}
that gets me a screen-sized quad...ok .. what's the correct way to get arbitrary sizes?...I have messed around with various numbers, but I think I don't quite get something in these relationships.
one thing I tried is:

float4 quad = float4((texcoord.x - (xpos/screensizex)) * (width/screensizex), -(texcoord.y - (ypos/screensizey)) * (height/screensizey), 0, 1);

.. where xpos and ypos is number of pixels from upper right corner..width and height is the desired size of the quad in pixels
this gets me somewhat close, but not right.. a bit too small..so I'm missing something ..any ideas?

.
• By Stewie.G
Hi,
I've been trying to implement a gaussian blur recently, it would seem the best way to achieve this is by running a bur on one axis, then another blur on the other axis.
I think I have successfully implemented the blur part per axis, but now I have to blend both calls with a proper BlendState, at least I think this is where my problem is.
Here are my passes:
D3DX11_TECHNIQUE_DESC techDesc; mBlockEffect->mTech->GetDesc( &techDesc ); for(UINT p = 0; p < techDesc.Passes; ++p) { deviceContext->IASetVertexBuffers(0, 2, bufferPointers, stride, offset); deviceContext->IASetIndexBuffer(mIB, DXGI_FORMAT_R32_UINT, 0); mBlockEffect->mTech->GetPassByIndex(p)->Apply(0, deviceContext); deviceContext->DrawIndexedInstanced(36, mNumberOfActiveCubes, 0, 0, 0); } No blur

PS_BlurV

PS_BlurH

P0 + P1

As you can see, it does not work at all.
I think the issue is in my BlendState, but I am not sure.
I've seen many articles going with the render to texture approach, but I've also seen articles where both shaders were called in succession, and it worked just fine, I'd like to go with that second approach. Unfortunately, the code was in OpenGL where the syntax for running multiple passes is quite different (http://rastergrid.com/blog/2010/09/efficient-gaussian-blur-with-linear-sampling/). So I need some help doing the same in HLSL :-)

Thanks!

• Back around 2006 I spent a good year or two reading books, articles on this site, and gobbling up everything game dev related I could. I started an engine in DX10 and got through basics. I eventually gave up, because I couldn't do the harder things.
Now, my C++ is 12 years stronger, my mind is trained better, and I am thinking of giving it another go.
Alot has changed. There is no more SDK, there is evidently a DX Toolkit, XNA died, all the sweet sites I used to go to are 404, and google searches all point to Unity and Unreal.
I plainly don't like Unity or Unreal, but might learn them for reference.
So, what is the current path? Does everyone pretty much use the DX Toolkit? Should I start there? I also read that DX12 is just expert level DX11, so I guess I am going DX 11.
Is there a current and up to date list of learning resources anywhere?  I am about tired of 404s..

• By Stewie.G
Hi,

I've been trying to implement a basic gaussian blur using the gaussian formula, and here is what it looks like so far:
float gaussian(float x, float sigma)
{
float pi = 3.14159;
float sigma_square = sigma * sigma;
float a = 1 / sqrt(2 * pi*sigma_square);
float b = exp(-((x*x) / (2 * sigma_square)));
return a * b;
}
My problem is that I don't quite know what sigma should be.
It seems that if I provide a random value for sigma, weights in my kernel won't add up to 1.
So I ended up calling my gaussian function with sigma == 1, which gives me weights adding up to 1, but also a very subtle blur.
Here is what my kernel looks like with sigma == 1
[0]    0.0033238872995488885
[1]    0.023804742479357766
[2]    0.09713820127276819
[3]    0.22585307043511713
[4]    0.29920669915475656
[5]    0.22585307043511713
[6]    0.09713820127276819
[7]    0.023804742479357766
[8]    0.0033238872995488885

I would have liked it to be more "rounded" at the top, or a better spread instead of wasting [0], [1], [2] with values bellow 0.1.
Based on my experiments, the key to this is to provide a different sigma, but if I do, my kernel values no longer adds up to 1, which results to a darker blur.
I've found this post
... which helped me a bit, but I am really confused with this the part where he divide sigma by 3.
Can someone please explain how sigma works? How is it related to my kernel size, how can I balance my weights with different sigmas, ect...

Thanks :-)