Im wanting to display hidden units like so the blue units in this screenshot Im wondering what the best method would be (I dont want to use occlusion test or stencil) What would be the simpliest method? cheers zed

The simplest method:

Render the unit once as normal. Then render the unit again as a solid color, but only allow pixels that fail the depth test.

Note that it isn't necessarily perfect, since you can't control what objects cause this kind of occlusion (including self-occlusion). For instance, you might not want a tight cluster of units to occlude one another, but rather only "big" terrain features like trees or buildings. The smarter method is indeed to add a stencil test to control what can occlude.

cheers zipster, though that not gonna work
eg in the above shot the horses will have the legs drawn in blue.

yes thinking about it some more I think the only way to do it reasonably is with stencil

Quote:Original post by zedz
cheers zipster, though that not gonna work
eg in the above shot the horses will have the legs drawn in blue.

Well, yes, that was my point, that only using the depth buffer is going to be rather simple, but not leave you with a lot of decent options or results [smile] However if you throw in the stencil buffer, you still have a fairly simple approach that produces much better visuals:

1) Render all objects normally.
2) Render all occluding objects to the depth-stencil buffer, clearing it first.
3) Render all "occludeable" objects as a solid color, and only pass pixels that pass the stencil test and fail the depth test.

The only restriction here is that occluders can't also be occludeable or else you'll get self-occlusion, but I've used this technique before in a few RTS games and rarely did I ever run into that case.

1) Render all objects normally.
2) Render all occluding objects to the depth-stencil buffer, clearing it first.
3) Render all "occludeable" objects as a solid color, and only pass pixels that pass the stencil test and fail the depth test.

This method will result in a lot of re-transformed data depending on how many objects "occlude". Obviously this might not be a concern for you but RTS games do tend to get unit heavy and thus i would expect it is going to be a worry eventually.

the faster method is to setup the stencil and enable it for ALL objects such that:

Those objects that occlude mask in the top bit (128)
Those objects that need to be occluded simply add 1

then do a full screen pass using stencil test for > 128.

Provided your overdraw for this pass does not hit 128 (and i'd be shocked at >15) then this works fine.

Note - this advice is based on having control over the stencil test/masking.

for the above 2 posts
I had a think about it in bed, I think I can get by without step 3.
Ill try that first up today.
I had a play of age of mythology last night, it seems they only do the blue units when the center of the unit is obscured.

ray cast (ray->center of unit) if ray goes first through 'occluder geometry' then mark as to be drawn occluded.

btw the game started really to chug bad like ~10fps, which is a worrying thing, since Ive got a new pc + the game is from ~5years ago, yes RTS are very cpu intensive but still hopefully I can avoid it, I notice their path tracing is not always perfect, eg create a long maze like wall, or blocking geometry(houses etc) send a unit to the other side, + the thing goes in the wrong direction :)
if (havent found a path in 10000ticks)
just follow whatever you do have

Quote:Original post by AndyFirth
This method will result in a lot of re-transformed data depending on how many objects "occlude". Obviously this might not be a concern for you but RTS games do tend to get unit heavy and thus i would expect it is going to be a worry eventually.

The approach I mentioned was for an RTS. The whole reason you'd use this technique in the first place is to help show hidden units. Thus the idea is that only big objects that can actually hide whole units are occluders, which disqualifies most units in a typical RTS from being occluders since they're not big enough. Plus they'd likely need to be occluded as well, which means nasty artifacts when clustered in groups. So it's mostly buildings and large geographic features that need to occlude.

Besides, your method won't work for a number of reason:

• Masking itself isn't a supported stencil buffer operation. The only way to tell the difference between an occluder and a unit with the stencil buffer (and essentially perform some type of "mask") would be to render all units before all occluders, and use a stencil test on the occluder pass to check where units had written. However performing a stencil test means you need a second pass that's separate from the main rendering pass. There's no way you can do both at once.
  
  
• Your method doesn't distinguish between a unit in front of an occluder versus one behind an occluder. You'd get a stencil value of >128 regardless of depth order. I can tell you that trying to fix that first problem will only lead you down a rabbit hole of more problems, with fixes that only lead to more problems and edge cases that result in false-positives. However since you can't mask in the first place it's a moot point.
  

So yes, a separate pass is required for units and occluders if you want correct behavior with the least amount of extra hoops and effort. However zedz just reminded me that we also used ray-casts to optimize the process, so that you only needed to perform a second rendering pass on units that couldn't "see" the camera from their center. All things considered, I believe this is about the best middle-ground between performance and effort that you're going to get:

1. Render the scene (terrain + objects) normally. Keep the depth buffer, clear the stencil buffer. Save the renderstate, disable depth and framebuffer writes.
   
2. Perform a ray-cast on all occlude-able objects in the scene, render those that can't "see" the camera using a GREATER depth test. Write a sentinel value into the stencil buffer when the depth test passes.
   
3. Enable framebuffer writes.
   
4. Render all occluders using a LESSEQUALS depth test, and enable a stencil test that checks for the sentinel value. If both tests pass, write the occlusion color/texture.
   
5. Restore the renderstate saved in step 1 (just cleanup).

Just to clarify something that might not have been clear from the beginning, both occluders and occlude-able objects are assumed to be opaque. Taking into account transparency is another ballgame.

>>Masking itself isn't a supported stencil buffer operation

on ps3 / xbox360 you simply setup the stencil pass write mask to the mask you want, then set the reference value to 0xff using a write operation of "replace", all bits outside the mask will be preserved, however this post was in regards to PC which is why i put the caveat at the end as while 360/ps3 use Dx9+ Parts I've not been a DirectX programmer for a long time (since Dx5) and thus my experience is primarily with X360/PS3.

The method does require that you render all units first yes, however the cost of that "should" be relatively low compared to the bulk of the environment.

also fail and pass perform different stencil operations and can massively improve performance with operations like this when used thoughtfully.

Quote:Original post by AndyFirth
on ps3 / xbox360 you simply setup the stencil pass write mask to the mask you want, then set the reference value to 0xff using a write operation of "replace", all bits outside the mask will be preserved, however this post was in regards to PC which is why i put the caveat at the end as while 360/ps3 use Dx9+ Parts I've not been a DirectX programmer for a long time (since Dx5) and thus my experience is primarily with X360/PS3.

That's true, there is a write mask I forgot about. But since it's a renderstate you're also potentially adding a lot of draw calls. Plus you also have to make sure all your occlude-able meshes get sorted to the front of the draw order, and then all your occluders, and finally everything else. And then there's the full-screen pass, which certainly raises red flags.

So I suppose that method could also work, but I'm not too crazy about the unknown number of extra draw calls, the sorting, and the full-screen pass. Whether or not it's faster is debatable and only a real implementation of both approaches will tell.

i do stuff like this now in our game... it isn't RTS so nowhere near the unit counts BUT the objects are in the 30,000 - 40,000 vertex range so it is important that the vertex unit processes the data as few times as possible.

as to whether it will be faster... given small units, large occluders and a decent sorting algorithm i don't see how it won't be faster over an approach that renders elements more than once.

that said - its true, i haven't implemented this so not an easy thing to say yay or nay on.