hi

1st thing i would try is this:

turn off the depth buffer and then render the shadow. After that turn it on again. (i did not try this)

2nd

1. render the shadow only into stencil buffer and set the values in the stencil to value 1, like this (and don't forget to turn off the depth buffer):

glStencilFunc (GL_ALWAYS, 1, 0xFFFFFFFF);
glStencilOp (GL_REPLACE, GL_REPLACE, GL_REPLACE);

2. now render the shadow to screen with depthbuffer off and use the stencil buffer like this:

glStencilFunc (GL_EQUAL, 1, 0xFFFFFFFF);
glStencilOp(GL_KEEP, GL_ZERO, GL_ZERO);

this means that the shadow will be rendered only where stencil is 1 and after that it will be set to 0. So next time when som other polygon is rendered, that causes us trouble, there will be a zero in the stencil buffer. So it should not render and do the trouble. (but i did not try this).

3rd

i am working right now on shadows too. I need it, so if i won't have any problems i would like to finish it in one or two weeks and then i will put in public on my homepage http://xormultiplexus.googlepages.com/home. Maybe this could help you.

4th - trick
if the background on the picture is only a big polygon, then render the shadow to the stencil buffer and then render the shadow on the screen not with the mesh of the shadow but with the big polygon using stencil buffer.

Hope i helped somehow.

XorMultiPleXus

ok

here's the code:

//we enable stencil test
glEnable(GL_STENCIL_TEST);

//now we turn off the depth and color buffer, so we can render only to stencil buffer
glDepthMask(GL_FALSE);
glColorMask(GL_FALSE, GL_FALSE, GL_FALSE, GL_FALSE);

//we set this to render the mesh into the stencil buffer
glStencilFunc (GL_ALWAYS, 1, 0xFFFFFFFF);
glStencilOp (GL_KEEP, GL_KEEP, GL_REPLACE);

//RENDER THE SHADOW

//now we turn on the depth and color buffer
glColorMask(GL_TRUE, GL_TRUE, GL_TRUE, GL_TRUE);
glDepthMask(GL_TRUE);


//here I render only the shadow
glStencilFunc (GL_EQUAL, 1, 0xFFFFFFFF);
glStencilOp(GL_KEEP, GL_KEEP, GL_KEEP);
glColor4f(.5,.5,.5,0.5);
glBegin(GL_TRIANGLES);
glVertex3f(-300, -100, 0);
glVertex3f(300, -100, 0);
glVertex3f(0, 300, 0);
glEnd();

//and here the polygon
glStencilFunc (GL_NOTEQUAL, 1, 0xFFFFFFFF);
glStencilOp(GL_ZERO, GL_ZERO, GL_ZERO);
glColor4f(.5,.5,.5,1);
glBegin(GL_TRIANGLES);
glVertex3f(-300, -100, 0);
glVertex3f(300, -100, 0);
glVertex3f(0, 300, 0);
glEnd();

glDisable(GL_STENCIL_TEST);

Here's the explanation:
1. render the shadow to the stencil buffer == easy
2. render the polygon to render the shadow with help of stecil buffer
3. render the polygon itself

My advice for you:
0. render background
1. place the shadow mesh in front of the background polygon
1. render the shadow to the stencil buffer == easy
2. render the the shadow with help of stecil buffer

That should work for you.

XorMultiPleXus

As I understand it you actually render the model 3 times.

One for the stencil buffer one for the shadow and one for the character?

The characters are bone animated and some of them have up to 20k polygons. I think this would be a little too expensive in computational power?

Hope you will bear with me if my questions seem extremely stupid :-)

well, not.

You have to render the character and it's shadow. The shadow must be rendered into the stencil buffer. Then you render only the background polygon. Now you must render the shadow. Not with the character mesh, but with only one polygon. Take the background polygon. Translate it in front of the background (which is already rendered) and render it with stencil on. That will render the shadow.

Yes it is a little bit computational expensive. But today we have fast computers and even if you play today's games you can see, that they don't bother with effiecienci ;-D.

XorMultiPleXus

Ok, So I do e.g. the following

Render the Shadow to stencil buffer
Render a "square" from stencil buffer on top of background
Render the Character.

yes, that's it.

Thanks. Your the greatest!

Now we are on the subject. Any easy way to create soft shadows this way :-)