Skeletal animated character and transparency

Graphics and GPU Programming Programming 2D Character Advice

Started by SasQ November 26, 2017 08:39 PM

8 comments, last by CrazyCdn 6 years, 4 months ago

122

Author

November 26, 2017 08:39 PM

I have a character in a game, made of multiple sprites in a hierarchy for skeletal animation (also called cutout animation by some).

I want to make that character semi-transparent for a while, like a ghost.

The problem is, when I naïvely set the transparency on the top-most sprite and all of its children, it doesn't look as I expected, because the parts of the sprites that were supposed to be occluded by other sprites can now be seen through their occluders Like, for example, you can see the body through the clothes, or parts of the limbs inside the body, etc. (#3 in the picture below).

I'd rather like it to be made transparent as a whole, as if it were a single image (#2 in the picture below).

So my question is, how is this effect usually achieved in video games?
(Preferably in OpenGL, but I'd like to know the general approach, so that I could apply it to whatever tool I'll use.)

My first thought of how to fix it, was to render the sprite chierarchy to texture first, and then map that texture onto a simple rectangular mesh, and apply opacity to that.
But this approach would have some drawbacks. One of them I guess being the additional overhead required for rendering to texture, especially if there will be more "ghosts" on the screen at the same time, all with different transparencies. Another one is of course the overhead in code – it would require a framework of managing these hierarchies and their associated render targets – in other words: a MESS!

Is there any other, better way to do that than rendering the entire character to texture?

If there isn't one, then maybe you could give me some advices about how to organize that mess in the code?

CrazyCdn

1,442

November 26, 2017 09:48 PM

Another option, since I'm not exactly sure what your issue is (images would be a good thing to post), would be to render to texture but instead of multiple render to textures for multiple ghosts... Figure out how many you have to render and enlarge the texture and turn it into a texture atlas of sorts. It would reduce resource and switching costs at least.

"Those who would give up essential liberty to purchase a little temporary safety deserve neither liberty nor safety." --Benjamin Franklin

Hodgman

52,717

November 26, 2017 10:04 PM

Yeah rendering all the ghosts to a texture atlas wouldn't be out of the ordinary.

You could also use the stencil buffer. Clear it to zero at the start of the frame, then for the first ghost, configure the stencil test to enabled=true, reference=1, pass if stencil is not reference, on pass: set to reference, on fail: keep previous. Then for the second ghost, set reference=2, etc.

Then draw your character's cut-out parts in front to back order. The stencil test will ensure that each ghost can only write to a pixel once (the first triangle to cover that pixel).

If you've got over 255 ghosts, then after the 255th, clear the stencil to 0 again and set the stencil ref back to 1 (and count up again).

. 22 Racing Series .

SasQ

122

Author

November 27, 2017 01:08 AM

4 hours ago, Mike2343 said:

images would be a good thing to post

OK I added a picture that shows the essence of the problem in the original post.

#1 is the fully opaque version.
#2 is what I want to get: the image of the entire hierarchy being semi-transparent, and whatever has been occluded in the original (opaque) image, stays occluded in the transparent one.
#3 is what I actually get when I naively set the transparency on each of the constituent parts - they become transparent, but then they no longer occlude each other in the way they were supposed to do, and I can see their internals (imagine one of these balls being the clothes, and the other one being the body of a character; or one of them being the body, and the other being a limb that sticks out of it, partially occluded by the body).

I may composite the image first by rendering it to a texture and then rendering a quad with that texture applied, along with transparency on that quad, and it works the way I want, but I'm worried that this might not scale well if there will be more such "ghosts" on the screen. (BTW how many render targets is considered too much? :q )

4 hours ago, Mike2343 said:

but instead of multiple render to textures for multiple ghosts... Figure out how many you have to render and enlarge the texture and turn it into a texture atlas of sorts. It would reduce resource and switching costs at least.

Interesting idea. So you're saying that I should render all those "ghosts" to one texture and then use it to render the quads as single sprites? Hmm... would it still be possible for each of them having a different opacity? Or will it only work if all these ghosts are at the same opacity level?

Nevertheless, I'm afraid this approach will be a mess to code in order to manage all those render targets, unless there's some organized way to do it. (Any tutorials perchance? It's not that this is something extraordinary, right? I've seen such effects being used in video games for the last 10 years or so :q ) But I guess I would have to go with that approach anyway if I wanted to add some more effects to the hierarchy, such as glow, blur, waving or some other shader effects... Am I right? :q

4 hours ago, Hodgman said:

You could also use the stencil buffer.

Hmm.... Sounds interesting. At least it wouldn't require keeping track of all those render targets kicking around with each sprite hierarchy. Switching rendering parameters seems easier. But I've never used stencil buffers before, so I guess I have some learning to do, because I don't quite understand all the details of how your solution works. Is it supposed to prevent the pixels of the transparent parts of the same character from overwriting each other or something? Then maybe it would suffice with just one index? I could reset it to zero after drawing each hierarchy.

Wouldn't it mess up the antialiased pixels on the edges of the sprites though? :q Once they've been drawn to the stencil from the top sprite, it would protect these pixels from mixing with the pixels of the next sprite underneath, right?

Hodgman

52,717

November 27, 2017 02:32 AM

1 hour ago, SasQ said:

Hmm.... Sounds interesting. At least it wouldn't require keeping track of all those render targets kicking around with each sprite hierarchy. Switching rendering parameters seems easier. But I've never used stencil buffers before, so I guess I have some learning to do, because I don't quite understand all the details of how your solution works. Is it supposed to prevent the pixels of the transparent parts of the same character from overwriting each other or something? Then maybe it would suffice with just one index? I could reset it to zero after drawing each hierarchy.

Yeah.

The stencil test occurs before pixel shading and looks something like


if stencilTestEnabled then
  if stencilTest(stencilBuffer[x,y], stencilReference) then
    stencilBuffer[x,y] = onPass(stencilBuffer[x,y], stencilReference)
    frameBuffer[x,y] = pixelShader()
  else
    stencilBuffer[x,y] = onFail(stencilBuffer[x,y], stencilReference)
  end
else
  frameBuffer[x,y] = pixelShader()
end

You can use OpenGL's API to configure

Set "stencilTestEnabled=true":
glEnable(GL_STENCIL_TEST);

Set "stencilTest(buffer, reference)" to "return buffer != reference", and set "reference=0":
glStencilFunc(GL_NOTEQUAL, 0, 0xFF)

Set "onPass(buffer, reference)" to "return reference", and "onFail(buffer, reference)" to "return buffer":
glStencilOp(GL_KEEP, GL_REPLACE, GL_KEEP)

You then end up with a stencil test that looks something like:


if stencilBuffer[x,y] != stencilReference then
  stencilBuffer[x,y] = stencilReference
  frameBuffer[x,y] = pixelShader()
end

Clearing the stencil buffer to zero is kind of expensive (at 1080p, there's two million pixels in a fullscreen buffer...), hence my suggestion to simply use a different "stencilReference" value for each ghost.

Having this kind of stencil test as a condition on running the pixel shader, means that for each pixel in the ghost, it first checks to see if the "ghost ID" (stencil reference value) is already present at that pixel location. If it's not, then the ghost draws itself there and also saves the "ghost ID" into the stencil buffer. If another polygon with the same "ghost ID" then comes along afterwards (e.g. the body underneath the clothes), it will realize that the current "ghost ID" has already been written at this pixel location, so will skip running the pixel shader / will early exit.

. 22 Racing Series .

Styves

1,811

November 27, 2017 03:51 AM

7 hours ago, SasQ said:

The problem is, when I naïvely set the transparency on the top-most sprite and all of its children, it doesn't look as I expected, because the parts of the sprites that were supposed to be occluded by other sprites can now be seen through their occluders

If you want "occlusion" from top-most sprites, then couldn't you just have the transparent sprites write to the depth mask and let depth-testing take care of it? The second sphere would then automatically depth-fail against the previous sprites, giving the result in #2.

SasQ

122

Author

November 27, 2017 04:10 AM

Not if the sprites have some areas that are supposed to be semi-transparent over each other.

Suppose that these spheres have some glowing auras around them. This aura should blend with the sprites underneath, but the balls should occlude each other, even if I make the entire hierarchy semi-transparent.

Now when I think of it, this may also render the stencil buffer solution inapplicable. Too bad, I guess the only way to go for me is with the "composite first, transparency next" approach and rendering the entire hierarchy to a texture cannot be avoided.

(Correct me if I'm wrong, though. I'm pretty new to those additional buffers. I'm here to learn how those effects are usually done in video games.)

Styves

1,811

November 27, 2017 05:08 AM

48 minutes ago, SasQ said:

This aura should blend with the sprites underneath, but the balls should occlude each other, even if I make the entire hierarchy semi-transparent.

You could probably get away with this with some depth output tweaks, but I see what you mean. I don't think it'd be worth the hassle.

One option is to do the same thing Hodgman suggested but manually using shaders and MRT rather than the stencil buffer. Done this way you have full control of where this "occlusion" should be.

50 minutes ago, SasQ said:

Too bad, I guess the only way to go for me is with the "composite first, transparency next" approach and rendering the entire hierarchy to a texture cannot be avoided.

(Correct me if I'm wrong, though. I'm pretty new to those additional buffers. I'm here to learn how those effects are usually done in video games.)

8 hours ago, SasQ said:

One of them I guess being the additional overhead required for rendering to texture, especially if there will be more "ghosts" on the screen at the same time, all with different transparencies. Another one is of course the overhead in code – it would require a framework of managing these hierarchies and their associated render targets – in other words: a MESS!

Well, you don't need a complex framework for several render targets to do this unless you want to cache those results somewhere. I wouldn't particularly suggest it since you'll have to fight resolution mis-match with the main image.

A better approach might be to simply have a single full-screen temporary render target in which you render each object to. Use scissors + viewport + clear between each object draw to clean up the region the next object is going to be rendered to, and composite that region back onto the main render target after the sprites for that object are drawn.

CrazyCdn

1,442

November 27, 2017 07:39 PM

18 hours ago, SasQ said:

Interesting idea. So you're saying that I should render all those "ghosts" to one texture and then use it to render the quads as single sprites? Hmm... would it still be possible for each of them having a different opacity? Or will it only work if all these ghosts are at the same opacity level?

Just to answer this question, if you create the FBO in RGBA format then yes each ghost could have different values for alpha.

Tutorials/Information on Frame Buffer Objects (or Render to Texture):

http://www.songho.ca/opengl/gl_fbo.html

http://www.opengl-tutorial.org/intermediate-tutorials/tutorial-14-render-to-texture/

This one is in Java but the OpenGL commands are the same:

http://www.swiftless.com/tutorials/opengl/framebuffer.html