Hello,

I'm currently trying to mipmap 3D textures in D3D12. So the basic idea was to build 2 miplevels at once as I'm running 4x4x4 threads in group. Also using groupshared memory to do some optimizations. What is sad (on mipmap generation) - that for 1st mip calculation just 8 threads will do some work and just single thread for the 2nd mip calculation. All of them are occupied only when filling data into groupshared memory. If you're curious, the compute shader is here: https://pastebin.com/4DYWbsDZ (it was too long to dump it here)

To generate complete mipchain, I use a loop like:

        // Initial setup
	int todo = mMiplevels - 1;
	int base = 0;
	int dimension = mDimensions;

        // As long as there are some mip-levels to generate
	while (todo != 0)
	{
                // Always generate 2 miplevels at once (as long as we can)
		int mipLevels = 2;
		if (todo == 1)
		{
			todo++;
			base--;
			dimension *= 2;
		}

                // Record what we need to do into cmd list
		context->SetConstants(0, Engine::DWParam(base), Engine::DWParam(mipLevels), Engine::DWParam(1.0f / (float)dimension));
		context->SetDescriptorTable(1, mColorTexture->GetSRV());
		context->SetDescriptorTable(2, mColorTexture->GetUAV(base + 1));
		context->SetDescriptorTable(3, mColorTexture->GetUAV(base + 2));
		context->Dispatch(dimension / 4, dimension / 4, dimension / 4);

		base += mipLevels;
		dimension /= pow(2, mipLevels);
	} 

After this for loop there is a 'Finish' call which executes command lists, and of course a barrier as the volume is going to be used. Note that the volume is generated and used within single frame.

The question here is (to others who most likely already tried similar thing), can this be done in better way?

Thanks for the responses. I'm setting up timer queries around all the dispatches now to measure changes I'm about to try (probably no 'easier' way around - then to try & fail modifications to this approach), I hope to have results shortly - so I'll post them here.

The ALU work is nothing so it does not hurt when threads become idle.

This was my experience too. I had another case where I had to mipmap a 64bpp gbuffer for a deferred impostor rendering system. I used the same algorithm to generate three mip-levels at a time, which results in a lot of idle threads.
I managed to speed it up by completely re-packing the gbuffer into 32bpp, which involved adding A LOT of ALU operations for shifting/masking/OR'ing bits around the place, not to mention that I had to implement the 2x2 box filter with ALU instead of using a texture filter... So even in that very ALU-heavy mipmapping filter, the only real cost was memory bandwidth.

Alright, so I've set up timer queries wherever I needed and profiled the hell out of several variations of code. So, first some notes about the setup:

Application generates dynamic voxel representation of Crytek's Sponza at 512x512x512 resolution, which is then mip-mapped. For proof-testing (that my mip maps are correct), I've calculated ambient occlusion using voxel cone tracing. The timestamps are obtained before and after each Dispatch during mipmap generation, and also right before and after the loop for generating mipmaps (so I can calculate overhead).

And my test scenarios.

1. Naive generation

Using 2x2x2 workgroup I generate lower miplevel from each higher miplevel. The generation stores 2x2x2 voxels in groupshared memory and calculates lower miplevel out of it only using 1st thread in workgroup. The results are (I've picked 3 samples of profiling output):

Dispatch[0] (256 256 256): 31.613440ms

Dispatch[1] (128 128 128): 4.366080ms

Dispatch[2] (64 64 64): 0.542720ms

Dispatch[3] (32 32 32): 0.068000ms

Dispatch[4] (16 16 16): 0.008640ms

Dispatch[5] (8 8 8): 0.002080ms

Dispatch[6] (4 4 4): 0.000480ms

Dispatch[7] (2 2 2): 0.000320ms

Total Time: 36.616640ms

Call overhead: 0.014880ms

Dispatch[0] (256 256 256): 29.836800ms

Dispatch[1] (128 128 128): 3.298880ms

Dispatch[2] (64 64 64): 0.412480ms

Dispatch[3] (32 32 32): 0.052000ms

Dispatch[4] (16 16 16): 0.007040ms

Dispatch[5] (8 8 8): 0.001280ms

Dispatch[6] (4 4 4): 0.000800ms

Dispatch[7] (2 2 2): 0.003520ms

Total Time: 33.616160ms

Call overhead: 0.003360ms

Dispatch[0] (256 256 256): 31.044640ms

Dispatch[1] (128 128 128): 3.807680ms

Dispatch[2] (64 64 64): 0.485600ms

Dispatch[3] (32 32 32): 0.066720ms

Dispatch[4] (16 16 16): 0.006560ms

Dispatch[5] (8 8 8): 0.001280ms

Dispatch[6] (4 4 4): 0.000800ms

Dispatch[7] (2 2 2): 0.004000ms

Total Time: 35.420960ms

Call overhead: 0.003680ms

2. Generate 2 levels at once

Using 4x4x4 workgroup always 2 miplevels at once are generated. If there is one last level (odd number of mipmaps), two smallest are re-generated. Uses groupshared memory - for first always "1st out of 2x2x2 sub-group" or 1st out of whole workgroup does the actual mipmapping:

Dispatch[0] (128 128 128): 8.049120ms

Dispatch[1] (32 32 32): 0.125120ms

Dispatch[2] (8 8 8): 0.004320ms

Dispatch[3] (2 2 2): 0.000320ms

Total Time: 8.180960ms

Call overhead: 0.002080ms

Dispatch[0] (128 128 128): 8.042560ms

Dispatch[1] (32 32 32): 0.125600ms

Dispatch[2] (8 8 8): 0.004160ms

Dispatch[3] (2 2 2): 0.000320ms

Total Time: 8.175040ms

Call overhead: 0.002400ms

Dispatch[0] (128 128 128): 7.860160ms

Dispatch[1] (32 32 32): 0.123840ms

Dispatch[2] (8 8 8): 0.004000ms

Dispatch[3] (2 2 2): 0.000320ms

Total Time: 7.990560ms

Call overhead: 0.002240ms

3. Generate 3 levels at once

As there is improvement, the logical step would be to go further and attempt to generate 3 levels at once, right? So I dispatch 8x8x8 workgroup and generate 3 miplevels at once (magic constants for masking are starting to be a bit of black magic here). Total number of Dispatch functions is down to 3:

Dispatch[0] (64 64 64): 18.375040ms

Dispatch[1] (8 8 8): 0.035040ms

Dispatch[2] (2 2 2): 0.000320ms

Total Time: 18.414240ms

Call overhead: 0.003840ms

Dispatch[0] (64 64 64): 16.648960ms

Dispatch[1] (8 8 8): 0.035360ms

Dispatch[2] (2 2 2): 0.000320ms

Total Time: 16.686240ms

Call overhead: 0.001600ms

Dispatch[0] (64 64 64): 20.219360ms

Dispatch[1] (8 8 8): 0.048960ms

Dispatch[2] (2 2 2): 0.000320ms

Total Time: 20.270560ms

Call overhead: 0.001920ms

Summary

As you can notice the timing got worse, I already think I have explanation for that (I'm not absolutely sure this is the case). In case where I generate 2 miplevels I use workgroup of 4x4x4 - each thread first stores data in groupshared memory, and then it is retrieved from apropriate locations and stored again (and thread_id location). Now as cache line for shared memory has 64-bytes, it is efficient.

For 8x8x8 when generating 3 miplevels at once this groupshared array has to be 512-bytes in size, therefore reading from it and writing from it at some locations will cause slow down.

The current code for 512^3 volume generates mipmaps in 8ms on AMD Rx 480 GPU. I still think that this is way too much, but maybe it is fast enough!

What next?

Thanks for advices (any further hint is of course welcome!). I might have also noted that my mip generation function is not just a sum (but counts voxels that contain something and uses that value as a divisor) - yet it is still just few operations, I might try to do the sum/counting in parallel reduction approach, but that will introduce even more memory barriers, which I'd suspect makes performance drop (but it is worth trying).

EDIT: Side not, parallel reduction won't work, adding more memory barriers will further decrease performance. As it is obviously memory bound (those 7 additions and 1 division are have literally no impact).

Lower resolutions?

Note, just out of curiosity, I took the one that generates 2 levels at once and put it in for 256^3 volume. The results are:

Dispatch[0] (64 64 64): 0.988000ms

Dispatch[1] (16 16 16): 0.016480ms

Dispatch[2] (4 4 4): 0.000480ms

Dispatch[3] (2 2 2): 0.000320ms

Total Time: 1.007360ms

Call overhead: 0.002080ms

Dispatch[0] (64 64 64): 0.974880ms

Dispatch[1] (16 16 16): 0.016480ms

Dispatch[2] (4 4 4): 0.000480ms

Dispatch[3] (2 2 2): 0.000320ms

Total Time: 0.994240ms

Call overhead: 0.002080ms

Dispatch[0] (64 64 64): 0.985120ms

Dispatch[1] (16 16 16): 0.016480ms

Dispatch[2] (4 4 4): 0.000480ms

Dispatch[3] (2 2 2): 0.000320ms

Total Time: 1.004640ms

Call overhead: 0.002240ms

Which is about 1ms for whole mip chain (and 8 times faster than doing the 512^3 volume).

Thanks for explanation. Here is the code:

cbuffer InputDimensions : register(b0)
{
	uint3 dimensions;
}

cbuffer InputMiplevels : register(b1)
{
	uint srcMiplevel;
	uint miplevels;
	float texelSize;
}

SamplerState srcSampler : register(s0);
Texture3D<float4> srcLevel : register(t0);
RWTexture3D<float4> mipLevel1 : register(u0);
RWTexture3D<float4> mipLevel2 : register(u1);
RWTexture3D<float4> mipLevel3 : register(u2);

groupshared float tmpR[512];
groupshared float tmpG[512];
groupshared float tmpB[512];
groupshared float tmpA[512];

void StoreColor(uint idx, float4 color)
{
	tmpR[idx] = color.r;
	tmpG[idx] = color.g;
	tmpB[idx] = color.b;
	tmpA[idx] = color.a;
}

float4 LoadColor(uint idx)
{
	return float4(tmpR[idx], tmpG[idx], tmpB[idx], tmpA[idx]);
}

float HasVoxel(float4 color)
{
	return color.a > 0.0f ? 1.0f : 0.0f;
}

[numthreads(8, 8, 8)]
void GenerateMipmap3(uint GI : SV_GroupIndex, uint3 DTid : SV_DispatchThreadID)
{
	float4 src[8];
	float3 uvw = (DTid.xyz + 0.5f) * texelSize;
	src[0] = srcLevel.SampleLevel(srcSampler, uvw, (float)srcMiplevel);
	StoreColor(GI, src[0]);
	GroupMemoryBarrierWithGroupSync();

	if ((GI & 0x49) == 0)
	{
		src[1] = LoadColor(GI + 0x01);
		src[2] = LoadColor(GI + 0x08);
		src[3] = LoadColor(GI + 0x09);
		src[4] = LoadColor(GI + 0x40);
		src[5] = LoadColor(GI + 0x41);
		src[6] = LoadColor(GI + 0x48);
		src[7] = LoadColor(GI + 0x49);

		float div = 0.0f;
		for (int i = 0; i < 8; i++)
		{
			div += HasVoxel(src[i]);
		}

		if (div == 0.0f)
		{
			src[0] = 0.0f;
		}
		else
		{
			src[0] = (src[0] + src[1] + src[2] + src[3] + src[4] + src[5] + src[6] + src[7]) / div;
		}

		// Store value + write into shared memory
		mipLevel1[DTid / 2] = src[0];
		StoreColor(GI, src[0]);
	}

	GroupMemoryBarrierWithGroupSync();

	if ((GI & 0xDB) == 0)
	{
		src[1] = LoadColor(GI + 0x02);
		src[2] = LoadColor(GI + 0x10);
		src[3] = LoadColor(GI + 0x12);
		src[4] = LoadColor(GI + 0x80);
		src[5] = LoadColor(GI + 0x82);
		src[6] = LoadColor(GI + 0x90);
		src[7] = LoadColor(GI + 0x92);

		float div = 0.0f;
		for (int i = 0; i < 8; i++)
		{
			div += HasVoxel(src[i]);
		}

		if (div == 0.0f)
		{
			src[0] = 0.0f;
		}
		else
		{
			src[0] = (src[0] + src[1] + src[2] + src[3] + src[4] + src[5] + src[6] + src[7]) / div;
		}

		mipLevel2[DTid / 4] = src[0];
		StoreColor(GI, src[0]);
	}

	GroupMemoryBarrierWithGroupSync();

	if (GI == 0)
	{
		src[1] = LoadColor(GI + 0x04);
		src[2] = LoadColor(GI + 0x20);
		src[3] = LoadColor(GI + 0x24);
		src[4] = LoadColor(GI + 0x100);
		src[5] = LoadColor(GI + 0x104);
		src[6] = LoadColor(GI + 0x120);
		src[7] = LoadColor(GI + 0x124);

		float div = 0.0f;
		for (int i = 0; i < 8; i++)
		{
			div += HasVoxel(src[i]);
		}

		if (div == 0.0f)
		{
			src[0] = 0.0f;
		}
		else
		{
			src[0] = (src[0] + src[1] + src[2] + src[3] + src[4] + src[5] + src[6] + src[7]) / div;
		}

		mipLevel3[DTid / 8] = src[0];
	}
}

EDIT: As per your calculation I could technically switch to smaller type than float4 per voxel - which I might do in the end. Along with lowering resolution of 3D texture. While this won't solve the problem, it will move all the times into acceptable frame time which I'd like to achieve.

I'm also looking forward to try to put all the voxels into octree instead and perform cone tracing with that. Although I suspect that octree generation will use less bandwidth than mipmapping of 3D texture.