• Create Account

## Convert pixel formats in Compute Shader

Old topic!

Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.

14 replies to this topic

### #1Naruto-kun  Members

411
Like
0Likes
Like

Posted 11 October 2013 - 06:29 AM

Hi guys

I have a 64bit R16G16B16A16 image which i need to convert to a 32bit R8G8B8A8 image using Compute Shaders. Does anyone have any examples of how to do this?

Thanks

### #2Styves  Members

1629
Like
0Likes
Like

Posted 11 October 2013 - 11:00 AM

That depends. What's the use case? Do you want to compress it or just straight convert? Do you want to rescale or do you want to clamp? Etc.

### #3Naruto-kun  Members

411
Like
0Likes
Like

Posted 11 October 2013 - 11:44 AM

Im not too familiar with the terminology but ill try to describe it....rescale sounds the closest. Basically so that half the max 16bit value would line up with half the 8 bit value and so on.

### #4MJP  Moderators

18215
Like
0Likes
Like

Posted 12 October 2013 - 12:50 PM

The hardware can do the format conversion for you. Just sample the source texture as a Texture2D in your compute shader, and it will be converted to 32-bit floating point. Then write to a RWTexture2D, and it will be converted to the destination format.

### #5Naruto-kun  Members

411
Like
0Likes
Like

Posted 12 October 2013 - 06:36 PM

Thanks. I am still getting garbage out though. And for some reason, I am getting a pitch of 3584 when i lock the 64bit surface which is odd since i should only be getting a pitch of 3520 (440*8). Any ideas?

Edited by Naruto-kun, 13 October 2013 - 07:07 AM.

### #6MJP  Moderators

18215
Like
1Likes
Like

Posted 13 October 2013 - 01:13 PM

That's typical. Usually the pitch includes some padding for hardware-specific reasons. You just need to copy/read the first 3520 bytes, and then make sure you advance your pointer by 3584 to get to the next row of pixels:

const uint8_t* mappedData = reinterpret_cast<const uint8_t*>(mapped.pData);
std::vector<uint8_t> texelData(width * height * texelSize);
for(y = 0; y < height; ++y)
{
memcpy(&texelData[y * width * texelSize], mappedData, width * texelSize);
mappedData += mapped.RowPitch;
}


How are you interpreting the data once you map it? Are you using a floating point format, or an integer format for your texture?

### #7Naruto-kun  Members

411
Like
0Likes
Like

Posted 13 October 2013 - 01:37 PM

That's typical. Usually the pitch includes some padding for hardware-specific reasons. You just need to copy/read the first 3520 bytes, and then make sure you advance your pointer by 3584 to get to the next row of pixels:

const uint8_t* mappedData = reinterpret_cast<const uint8_t*>(mapped.pData);
std::vector<uint8_t> texelData(width * height * texelSize);
for(y = 0; y < height; ++y)
{
memcpy(&texelData[y * width * texelSize], mappedData, width * texelSize);
mappedData += mapped.RowPitch;
}


How are you interpreting the data once you map it? Are you using a floating point format, or an integer format for your texture?

I have tried both but i think am messing up badly somewhere in my setting of the resource and access views. When loading the Texture2D i was using the D3D10_SUBRESOURCE_DATA struct in the initial data parameter for the CreateTexture2D method. When i copy from the RWTexture2D after mapping it I am using a Assembly/SSE blit function of my own making as i find it faster than memcpy. Works quite well provided it contains proper values...

But yeah i would guess the 2 key points where i am messing up are the loading of the Texture2D from the 64bit surface, and the way i am setting up the Texture2D and RWTexture2D objects inside the compute shader itself.

Anyone got an example of how i should be copying the data into the source?

Edited by Naruto-kun, 14 October 2013 - 04:26 AM.

### #8Naruto-kun  Members

411
Like
0Likes
Like

Posted 14 October 2013 - 12:14 PM

Update: I switched from trying to use a ComputeShader to my old faithful fullscreen quad and pixel shader. However i have 2 questions as I am still getting mostly garbage but a bit more legible than previous efforts.

Some source and a screenie:

The render target in the popup square (I hooked it from the game and when saving to file it comes out right) is a R16G16B16A16_FLOAT format. In accordance with this my input layout for the vertex shader is:

D3D10_INPUT_ELEMENT_DESC ied[] =
{
{"POSITION", 0, DXGI_FORMAT_R16G16B16A16_FLOAT, 0, 0, D3D10_INPUT_PER_VERTEX_DATA, 0},
{"NORMAL", 0, DXGI_FORMAT_R16G16B16A16_FLOAT, 0, 8, D3D10_INPUT_PER_VERTEX_DATA, 0},
{"TEXCOORD", 0, DXGI_FORMAT_R16G16_FLOAT, 0, 16, D3D10_INPUT_PER_VERTEX_DATA, 0},
};


The shader resource that I copy the original render target to is the same size and format.

Here is how it is loaded:

void CamToTex::CreateInput(void* data, UINT pitch, UINT stride)
{
if(src_view)
src_view->Release();
src_view = NULL;//Gets released and recreated at each redraw

D3D10_SUBRESOURCE_DATA subsrc;
subsrc.pSysMem = data;//Raw data from mapped render target
subsrc.SysMemPitch = pitch;//Set to 3584 from mapped render target
subsrc.SysMemSlicePitch = 0;
D3D10_TEXTURE2D_DESC tdsc;
ZeroMemory(&tdsc, sizeof(tdsc));
tdsc.Height = 440;
tdsc.Width = 440;
tdsc.MipLevels = 1;
tdsc.ArraySize = 1;
tdsc.Format = DXGI_FORMAT_R16G16B16A16_FLOAT;
tdsc.SampleDesc.Count = 1;
tdsc.Usage = D3D10_USAGE_DEFAULT;
ID3D10Texture2D* temp = NULL;
HRESULT hr = dev->CreateTexture2D(&tdsc, &subsrc, &temp);
if(temp)
{
temp->Release();
}
};


I have verified that the raw data is correct by saving it to a bitmap which showed correctly.

The render target that i create however to reprocess to 32bit is a 440x440 R8G8B8A8_UINT surface. No extra processing is done in the shader. The shader resource view is sampled and applied to the full screen quad as is.

After the rendering is complete the render target is copied to a mappable surface, mapped, the bytes copied into the image which is displayed on that small screen on the 3D parts in the image below.(excuse lack of anti aliasing and low res textures...I run on lowest settings during these kind of experiments)

Edited by Naruto-kun, 14 October 2013 - 12:46 PM.

### #9Nik02  Members

4074
Like
1Likes
Like

Posted 15 October 2013 - 03:48 AM

That is very likely an issue with the pitch. You mentioned using your own blit function; are you sure that it takes into account that pitch may not equal to width * numBytesPerPixel? Also, the pitch of two resources with exact same width can be different. In fact, during every Map(), the pitch can potentially change.

In the compute shader, you don't have to care about the pitch of the resource as the addressing hardware on the GPU will take care of that for you (though you still need to stay within the addressable dimensions of the resource).

As a side note, the vertex format is completely unrelated to the render target format; you don't have to use 16-bit floats for the vertices even though the pixel format of the target happens to use them.

Edited by Nik02, 15 October 2013 - 03:59 AM.

Niko Suni

### #10Naruto-kun  Members

411
Like
0Likes
Like

Posted 15 October 2013 - 04:40 AM

That is very likely an issue with the pitch. You mentioned using your own blit function; are you sure that it takes into account that pitch may not equal to width * numBytesPerPixel? Also, the pitch of two resources with exact same width can be different. In fact, during every Map(), the pitch can potentially change.

In the compute shader, you don't have to care about the pitch of the resource as the addressing hardware on the GPU will take care of that for you (though you still need to stay within the addressable dimensions of the resource).

As a side note, the vertex format is completely unrelated to the render target format; you don't have to use 16-bit floats for the vertices even though the pixel format of the target happens to use them.

Good point....ill try changing the order of things a bit.

### #11Naruto-kun  Members

411
Like
0Likes
Like

Posted 15 October 2013 - 07:02 AM

Ok i changed things around a bit. The Shader resource view Texture2D obejct is now a class member that isnt released until shutdown. It is created with Dynamic usage and CPU Write access.

void CamToTex:CreateInput(void* data, UINT pitch, UINT stride)
{
if(src_view)
src_view->Release();
src_view = NULL;

D3D10_MAPPED_TEXTURE2D img;
HRESULT hr = src->Map(0, D3D10_MAP_WRITE_DISCARD, 0, &img);

if(hr == S_OK)
{
char* s = (char*)data;
char* d = (char*)img.pData;
for(int i = 0; i < 440; i ++)
{
SSEmemcpy(d, s, 3520);
s += 3584;
d += 3584;
}
src->Unmap(0);
hr = D3DX10SaveTextureToFile(src, D3DX10_IFF_BMP, L"DX10Tests\\test.bmp");
}
};


As you can see i am only copying the 440*8bytes resulting in a pitch of 3520 but advancing the pointers by 3584 at the end since thats the pitch i get from img.RowPitch for both the source and destination surfaces.

I save out the texture to a 64bit format BMP file and when i view it I get a nice clean clear image. However I still get garbage coming out of the shader.

struct VS_Output
{
float4 position : SV_POSITION;
float2 tex : TEXCOORD0;
};

VS_Output VS(uint id : SV_VertexID)
{
VS_Output Output;
Output.tex = float2((id << 1) & 2, id & 2);
Output.position = float4(Output.tex * float2(2,-2) + float2(-1,1), 0, 1);
return Output;
}


Texture2D shaderTextures;
SamplerState SampleType;

struct VS_Output
{
float4 position : SV_POSITION;
float2 tex : TEXCOORD0;
};

float4 PS(VS_Output input):SV_TARGET
{
float4 color1;

return color1;
}


Any ideas on where it is going wrong here?

### #12Nik02  Members

4074
Like
0Likes
Like

Posted 15 October 2013 - 07:33 AM

As you can see i am only copying the 440*8bytes resulting in a pitch of 3520 but advancing the pointers by 3584 at the end since thats the pitch i get from img.RowPitch for both the source and destination surfaces.

Do not assume that this is the case everytime. Every Map call can potentially return a different pitch for even the same resource. The only guarantee is that it is greater than or equal to the width * number of bytes per pixel.

Edited by Nik02, 15 October 2013 - 07:34 AM.

Niko Suni

### #13Naruto-kun  Members

411
Like
0Likes
Like

Posted 15 October 2013 - 07:39 AM

As you can see i am only copying the 440*8bytes resulting in a pitch of 3520 but advancing the pointers by 3584 at the end since thats the pitch i get from img.RowPitch for both the source and destination surfaces.

Do not assume that this is the case everytime. Every Map call can potentially return a different pitch for even the same resource. The only guarantee is that it is greater than or equal to the width * number of bytes per pixel.

Gotcha will try incrementing by the actual returned pitch values.

### #14Naruto-kun  Members

411
Like
0Likes
Like

Posted 15 October 2013 - 08:44 AM

Nope still no luck

### #15Naruto-kun  Members

411
Like
0Likes
Like

Posted 15 October 2013 - 11:00 AM

:Facepalm: Disregard....My absolute idiocy. I failed to notice that the 32bit format render target was also having this slight difference in pitch and when i adjusted for it, I got a nice clean image as a result.

Thanks for the help though...really appreciate it.

Old topic!

Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.