HLSL (MonoGame) Trying to set pixels to avg color of a non-square block

Started by
1 comment, last by Thraka Andy 7 years, 11 months ago

I'm trying to convert some C# code into a shader so that I can get better performance. What my code does is look at the final scene rendered and average out the colors every (for example) 16x8 block of pixels. I've tried for 3 days to get this code working in a shader but I cannot figure it out. Shaders are really beyond me and there doesn't seem to be any up-to-date examples that I can hack together (which is all I could really do)

I did try to use two rendertargets, render the full scene then render it to a smaller target to force a resize. But I ended up losing too much clarity and it doesn't look as good as hand calculating the averages. I think this is because my average code doesn't assume the block area is square, which a resize would.

Can someone help me? Here is my C# code:


private void TranslateImageToTextSurface2(Texture2D image, TextSurface surface)
{
    surface.Clear();
    image.GetData<Color>(pixels);

    // surface.Font.Size represents the size of blocks in pixels.

    for (int h = 0; h < image.Height / surface.Font.Size.Y; h++)
    {
        int startY = (h * surface.Font.Size.Y);
        
        for (int w = 0; w < image.Width / surface.Font.Size.X; w++)
        {
            int startX = (w * surface.Font.Size.X);

            float allR = 0;
            float allG = 0;
            float allB = 0;
            for (int y = 0; y < surface.Font.Size.Y; y++)
            {
                for (int x = 0; x < surface.Font.Size.X; x++)
                {
                    int cY = y + startY;
                    int cX = x + startX;

                    Color color = pixels[cY * image.Width + cX];

                    allR += color.R;
                    allG += color.G;
                    allB += color.B;
                    //allBri += color.GetBrightness();
                }
            }

            // This only works for square fonts... ack??
            byte sr = (byte)(allR / (surface.Font.Size.X * surface.Font.Size.Y));
            byte sg = (byte)(allG / (surface.Font.Size.X * surface.Font.Size.Y));
            byte sb = (byte)(allB / (surface.Font.Size.X * surface.Font.Size.Y));

            var newColor = new Color(sr, sg, sb);

            // HERE IS WHERE I WOULD ADD PROCESS THAT AVG COLOR IN MY GAME ENGINE
        }
    }
}

Here is an example of what I end up out putting just using my code above versus forced resized through two render targets.

Ada457I.png

Advertisement

You're asking for help for a shader algorithm, but you haven't posted any shader code.

You showed the C# code to do this on the CPU, but it's unfinished (you don't do anything with "newColor") - and yet somehow you have a screenshot of the results of this for use as a reference image of the working version (presumably this is what "Me doing the average calculation" is - though it certainly doesn't look like it's averaging out colors - it looks like some dither pattern).

I'm just confused. And probably other people are too, hence the lack of replies.

You're trying to create a (width / 16) X (height / 8) size image where each pixel is the average of a 16x8 block of pixels of the original image - is that correct?

- You could accomplish this by rendering smaller versions of the original scene, but you'll need to go in steps of 2, since a bilinear filtering operation is only sampling from neighboring pixels (i.e. if you sample exactly from the corner of 4 pixels, you'll get a result that is the average of that 2x2 square) . So you'd need to go from say, 16x8 to 8x4 to 4x2 to 2x1 to 1x1.

- You could also have your render target automatically generate mipmaps, but those will be half sizes, so you'll still need to do at least one "resizing" pass to get your rectangular sample area.

Hi Phil! I really appreciate the reply.

The only shader code I have is basic "render the exact pixel" everything I try never turns out near to what I'm looking for. When I get home I'll post what I have but it's a giant mess of commented code and trying different things.

I don't post my code of what I'm doing with those colors because it's not really important. What I do is a bunch of custom code that writes the color (and character based on brightness of that color) to my ascii game engine (http://github.com/thraka/sadconsole) which is rendered to the screen.

You are correct, the only output I need from the shader is the same output from my C# code. A single pixel per block.

The 16 and 8 values could really be anything. 16x16, or 8x16, even 4x2...

This topic is closed to new replies.

Advertisement