Jump to content

  • Log In with Google      Sign In   
  • Create Account


2D blob shadow trimming based on another image


Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.

  • You cannot reply to this topic
3 replies to this topic

#1 TooLz   Members   -  Reputation: 176

Like
0Likes
Like

Posted 16 February 2014 - 03:24 PM

Hey guys, 

 

So I've been talking to people in the chat about this and was suggested to make a topic to possibly get the fully correct answer. 

 

I'm trying to "trim" a blob shadow according to another texture in a purely 2D game.

 

http://imgur.com/a/6m312

 

The images illustrate what I need to happen (for only the background in that case but the same applies for things you could stand on in the level as well (picture 3)

 

I originally was trying to accomplish this with a shader that would check the thing you are standing on (if not standing on something, then just the background) and trimming according to the alpha channel of the image.  I think my shader isn't working correctly because of having to get the correct texCoord of the Trimming image according to where the shadow is. Not sure what I would have to transform it by to make sure that i get the same pixel that is under the shadow's pixel. 

 

But anyways, it was also suggested that I use a StencilBuffer as well but I've never set this up and I'm kind of looking for some help...Most of the things I find are for just geometry/shadow rendering. 

 

I've gotten some info from 
http://blogs.msdn.com/b/shawnhar/archive/2007/05/17/transitions-part-three-stencil-swipes.aspx

for what stencil setup I should use but not exactly sure how to use it now though. I know I need to pass it the alpha texture(trimming texture) and then use the compare to check the shadows pixel as well. 

 

One problem I for see with this stencil setup is that as in picture 3, if the alpha value of the image under the top part of the shadow is white (because the bg image has white there), it won't get trimmed even though it needs to be (as it shouldn't just be floating in the air on top of the object you're standing on...)   I could be wrong in this but I have no idea. Unless I use 2 stencils, one to check against the bg and then the other to check against the objects in the scene?

 

Any help would be super appreciated...I've never really thought trying to this till it came up and yeah, I'm just not sure which would be the most effective way to trimming anywhere from 1-8+ shadows (not sure the max atm) 

 

Thanks for reading biggrin.png

 

PS:

 

One other thing, when I add an alpha channel to my image and save as TGA and import into xna, it is auto trimming any of the black part of the image without me asking it to (even if I set the begin BlendState to Opaque it still throws out the part that is black), any idea on how to fix that?


Edited by TooLz, 16 February 2014 - 03:27 PM.


Sponsor:

#2 TooLz   Members   -  Reputation: 176

Like
0Likes
Like

Posted 17 February 2014 - 09:22 AM

I got half the problem working. 

 

The part where the background now seems to stencil correctly (the shadow doesn't overlap the Black area (although I found out I had to flip that, so white had to be black etc.))

 

But now I can't seem to get the same thing to work for props etc that would have to be on top of that BG. I have figured out as well that if I want to make it clip on a prop, the image will have to be bigger than the prop actually (to give a place where the shadow can be clipped against) but still to no avail, the prop doesn't clip the sides or the Top (going outside of the object itself)... Do I have to do a separate set of draw calls for the props? Shouldn't I be able to put the props and BG in one set of calls to accomplish the same clipping?

 

Stencils and AlphaTestEffect Creations

//THE STENCILS

//This is a stencil. It keeps track of pixels. But only the ones we tell it to.
        //In this case, every pixel we pass to this stencil will be written to the buffer because we are using CompareFunction.Always
        public static DepthStencilState AlwaysStencilState = new DepthStencilState()
        {
            StencilEnable = true,
            StencilFunction = CompareFunction.Always,
            StencilPass = StencilOperation.Replace,
            ReferenceStencil = 1,
            DepthBufferEnable = false,
        };

        //This one will check what stuff was written to the stencil buffer and will help us to know what pixels to mask.
        public static DepthStencilState EqualStencilState = new DepthStencilState()
        {
            StencilEnable = true,
            StencilFunction = CompareFunction.GreaterEqual,
            StencilPass = StencilOperation.Keep,
            ReferenceStencil = 1,
            DepthBufferEnable = false,
        };


//THE ALPHA TEST EFFECT CREATIONS
//Create Alpha Test Effect
            _alphaEffect = new AlphaTestEffect(_spriteBatch.GraphicsDevice);
            _alphaEffect.AlphaFunction = CompareFunction.GreaterEqual;
            //Clip if opacity mask says it's Black = 255
            _alphaEffect.ReferenceAlpha = 255;

            //Create next Alpha Test Effect for clipping area
            _alphaEffect2 = new AlphaTestEffect(_spriteBatch.GraphicsDevice);
            _alphaEffect2.AlphaFunction = CompareFunction.GreaterEqual; //Use the "greaterEqual than" comparison, the alpha value of the pixel must be greater than the ReferenceAlpha
            _alphaEffect2.ReferenceAlpha = 0;    

Here's my DrawCall

protected override void Draw(GameTime gameTime)
{
            GraphicsDevice.Clear(Color.Transparent);
            
            //Clear the stencil's buffer
            GraphicsDevice.Clear(ClearOptions.Stencil, Color.Black, 0, 0);
            

            //Paint the color layer to the stencil buffer
            _spriteBatch.Begin(SpriteSortMode.Immediate, null, SamplerState.LinearClamp, AlwaysStencilState, null, _alphaEffect);
            //Draw the tile
            
            _spriteBatch.Draw(_tile, new Vector2(0, 0), null, Color.White, 0f, Vector2.Zero, Multiplier, SpriteEffects.None, 0);
            _spriteBatch.Draw(_boxTex, new Vector2(250, 250), null, Color.White, 0f, Vector2.Zero, Multiplier, SpriteEffects.None, 0);
            
            //End
            _spriteBatch.End();


            _spriteBatch.Begin(SpriteSortMode.Immediate, null, SamplerState.LinearClamp, null, null, _alphaEffect2);
            //Draw the shadow
            _spriteBatch.Draw(_shadowTex, _shadPos, null, Color.White, 0f, Vector2.Zero, Multiplier, SpriteEffects.None, 0);
            //Done with grass!
            _spriteBatch.End();

            //Now we paint the mask texture
            _spriteBatch.Begin(SpriteSortMode.Immediate, null, SamplerState.LinearClamp, EqualStencilState, null, null);

            //Draw the mask
            //_spriteBatch.Draw(_maskTex, new Vector2(200,200), Color.White);
            _spriteBatch.Draw(_maskTex, new Vector2(0, 0), null, Color.White, 0f, Vector2.Zero, Multiplier, SpriteEffects.None, 0);
            _spriteBatch.Draw(_boxAlpha, new Vector2(250, 250), null, Color.White, 0f, Vector2.Zero, Multiplier, SpriteEffects.None, 0);
            
            //End
            _spriteBatch.End();

            base.Draw(gameTime);
}

Edited by TooLz, 17 February 2014 - 09:55 AM.


#3 unbird   Crossbones+   -  Reputation: 4755

Like
0Likes
Like

Posted 17 February 2014 - 01:34 PM

But now I can't seem to get the same thing to work for props etc that would have to be on top of that BG. I have figured out as well that if I want to make it clip on a prop, the image will have to be bigger than the prop actually (to give a place where the shadow can be clipped against) but still to no avail, the prop doesn't clip the sides or the Top (going outside of the object itself)... Do I have to do a separate set of draw calls for the props? Shouldn't I be able to put the props and BG in one set of calls to accomplish the same clipping?

 
Not sure I follow exactly here, screenshots might help (again). Anyway, I consider stenciling is a bit an advanced topic, so congrats on your progress so far wink.png

When setting a state (stencil, blend, whatever), it persists for all subsequent draw calls, so if you want different states you have to set/reset it, i.e. here start another sprite Begin()/End() pair. In particular, I don't know if XNA's sprite End() resets those states automatically: check the documentation or reset them manually. But yeah, if you want the same masking effect, you could batch them.

Maybe you're just making your life too complicated. How about stenciling only the shadows and use painters algorithm for the rest (drawing farthest objects first). So:

- Draw background (and prepare stencil mask, too)
- Draw (all) stenciled shadows
- Draw props on top (no stenciling anymore)

Also make sure your props have proper alpha. In this regard: Your last screenshot looks suspicious.

Again: Sorry if I misunderstood and my hints aren't helpful.

#4 TooLz   Members   -  Reputation: 176

Like
0Likes
Like

Posted 20 February 2014 - 06:35 AM


Maybe you're just making your life too complicated. How about stenciling only the shadows and use painters algorithm for the rest (drawing farthest objects first). So:

- Draw background (and prepare stencil mask, too)
- Draw (all) stenciled shadows
- Draw props on top (no stenciling anymore)

 

That's what I'm doing already (more or less) and that part works. 

What I got is like this

-Draw Bg (and prepare stencil of the BG)

-Draw Shadow

-Draw BG Stencil to trim shadow

 

 

now what I also need is to do the same but 

-Draw Bg (and prepare stencil of the BG)

-Draw Prop (prepare stencil of Prop)

-Draw Shadow

-Draw BG Stencil to trim shadow

-Draw Prop Stencil

 

 

As the props also need to have the trimming if the shadow interacts with it.  I can't seem to use the same set up of shader/stencil for the props though and just haven't really been able to figure out why, other than XNA likes to auto trim alpha values.. (we have gotten it to work with just a black and white image now but the props still have to have an alpha around the see through areas...






Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.



PARTNERS