Disappearing Doors

posted in Milkshake's Dev Diary

Published July 15, 2006

This week I've been thinking about doors. The problem is that when a door opens, it needs to go somewhere ... but most of the time, there's nowhere for it to go.

Closed doors are no problem. Here are some smugly closed doors in their natural environment. They're rough prototypes I knocked up on the train to play with, but they're good enough to experiment with.

Open them up however and they really ruin the whole cutaway-wall look on the doorframes - particularly in the foreground. In fact, because the wall tiles I'm using here are recessed, you can even see them poking through the rear walls too. This really ruins the "believable, solid world" feeling I'm going for.

Open Doors

Unfortunately, I don't think this is going to be something I'm going to solve today. The options seem to be:

1) Animate the door vertices, collapsing them as they enter the doorframe. In order to avoid any texture warping as you deform the geometry, this would need a truck-load of vertices across the face of the doors - and would be really frustrating to model given the curved doorway I'm using. Worse, this time consuming animation job would need to be re-done once for every new door/doorway shape combination. This feels like a hack. Another similar alternative is to animate the vertex colour to transparent as it slides through the doorway - but this has all the same drawbacks - so I've pretty much given up on any vertex animation based solution.

2) Make one side of the door texture transparent, and "slide" the door image across a static quad to make it look like the door is opening. I think this is what the moonpod guys did for Mr Robot, so it can obviously look very good (and can I just say those guys are AWESOME - I'm more excited by Mr Robot than anything commercial that's coming out - plus they have a kick-ass development diary). But it does have a few downsides, namely that the door is as flat as a pancake (which could look odd in a game that lets you rotate the camera), the door can't cast shadows (as it's flat) and the door needs a totally planar UV mapping (meaning it can't re-use texture space). This also stops me doing any cool detail on the door itself (like interlocking sections that slide out from each other, animated textures for display panels, etc)

3) Use the stencil buffer to mask out the region around the door. This obviously turns door rendering into a multi-pass activity, plus (because it's done in screen space with no depth component) it only works well when the door is mostly square to the screen. As soon as you look across an oblique angle, it's going to be hard to exclude the right region. I've pretty much given up on this one.

4) Use clip planes to define a rough model of the doorframe. My initial hesitation with these is getting them into both the DirectX and OpenGL code paths I have, plus planning ahead for programmable shaders. I did a little research on this today, and it looks like it is available everywhere - even if there's some debate on how expensive it can be in a pixel heavy shader. Still, this is my current favourite, as it allows me to make any kind of cool doors/locks that can be real-3D, without interfering with any other rendering trick (e.g. skinning, vertex, or texture animation) I might want to use on the door somewhere. The downside of this approach is that, because clip planes are global render state - this will need me to flush my render queues on both sides (breaking material/depth sorting goodness) to ensure that the clip planes only effect the door itself. As a final bonus - once I have these setup, I can re-use them for line-of-sight portal rendering if I ever want to go down that path.

5) (this last one was suggested by a guy at work today) Implement real-time Booleans operators to clip the door in software. This is an impressive idea, but interpolating all the various types of per-vertex data (and possibly re-indexing the geometry to remove primitives that are entirely culled) seems a little bit like overkill - not to mention the fact that it will require every door to make a unique copy of the door geometry to modify.

As hinted above - I'm leaning towards the 4th option - as it seems to be the cleanest solution. I just need to work out how to manage the interactions between the scene graph traversal and the renderer. In the mean time though, I throw down the gauntlet to any reader to suggest other clever solutions!

Cheers,
G

Previous Entry The Very First One

Next Entry Matrix Animation

0 likes 14 comments

Comments

Drilian

6. Draw a flat black polygon underneath the world (to emulate the blackness of the background) and make the doors slide DOWN instead of out :)

July 15, 2006 12:31 PM

Milkshake

Possibly I'm missing the magic here - but wouldn't you still need enough internal vertices across the face of the door to ensure it only starts to go "down" as it passes into the door frame? If not, you'd see the faces sloping "down" even while they're still visible inside the doorway would you not? Won't this have all the same issues of simply collapsing the geometry as it enters the doorframe? (e.g. texture warping, shading artifacts as the deform the faces, etc)?

Again, apologies if I'm missing the cunning. Either way - thanks for the suggestion! Anything to keep me from having to add clip planes would make me a very happy camper.

July 15, 2006 12:52 PM

Milkshake

Wait there - just re-read you post. You're saying "have doors which open downwards instead of sliding open" right?

Yup - these are definitely much easier to do, and I do plan to have some vertical doors. But side-slidy star trek swooshing doors are just something I have to have =)

Oh - and also, even though the test scenes I'm using here are flat, I'm really keen to have much more "vertical" levels that have you climbing/descending a lot more - so in these cases, it could get tricky to even draw a limit black polygon under each door without having it show up in the scene "below" the door (i.e. when the door is not the lowest point in the scene).

July 15, 2006 12:54 PM

Wyzfen

Two suggestions - both in design, not code...

Could you redesign it so the doors are physically possible (ie, they have somewhere to go, and dont magically go through the frame). I think any solution you come up with otherwise will look a bit strange. Even in trek, the doors slid into the walls.

OR,

in your model editor, use boolean operations to clip the door models to inside the door-frame for each frame or animation (probably only 5 positions or so) - then instead of moving your door, just change which mesh you use (turn off all the ones not relevant). It'll take more data in the file, but no longer to draw. Of course, that option wont look smooth unless you do enough seperate frames.

Wyzfen

July 15, 2006 05:20 PM

Milkshake

For the doorways at the rear, it would be entirely possible to re-design the walls to give the doors somewhere to go (indeed, it's just because I was too lazy to create a un-recessed wall panel that you can see any problem with the rear doors in the screenshots). It's the doorways in the front of the room which are the problem.

The issue is that the walls aren't drawn (because they'd obscure the player's view of the room). The front walls exist - the room just decides to hide them to give you a better view of what's happening in the room (if you spin the camera around, it would draw/hide different walls based on the orientation of the room vs the camera). So even if the walls were modelled to realistically give the doors somewhere to slide - when the walls are hidden, the door will look silly. And I don't want to hide the door, because I want it to be clear to the player that there is a closed door blocking their progress that they'll need to open/unlock to proceed through a given doorway. Does that explain why I'd still need to hide the bit of the door that's outside the doorway?

And I did think of generating a few distinct versions of the door and playing between them - the issue then is obviously that the animation will look a little jerky when you play it back at an arbitrary frame rate. For small/fast doors, it's probably not too bad and a good fall-back solution. If I start using some bigger doors (large portculis on a castle, or a honking airlock) it might not look so good.

Incidentally Drilian, I did experiment more with the "black polygon" suggestion. I'm already using a similar trick for the recessed walls. There is a black "baffle" down the outsides of the recessed wall sections, so that when you look from side on, you don't see the back-side of the walls. I rigged up something similar for the doors (sort of a black box for them to slide into), but as soon as you have a wall that's elevated off the floor, the black envelope shows up on top of the floor =(

Thanks again for helping me to mull this over.

I'm slowly starting to see some synergy between this problem, another unresolved issue I had supporting arbitrarily shaped portals (e.g. triangular, circular, etc), and possibly adding some portal-based rendering code that would let you see "through" doors and windows. This might be a late night =)

July 15, 2006 07:19 PM

Wyzfen

Are you using shaders ? If so ...

If you have a 1bit mask texture that represents the area where the doors should be drawn, you could;
- in the vertex shader, offset the texture so only the bit of door that should be drawn has an 'on' value in the texture. ie, texture moves across door model, at same speed as door moves, in opposite direction. (The offset would be passed as an input)

- in the pixel shader, use the texkill instruction to remove any pixel that that isnt 'on' in the mask texture.

The other idea is to use clip planes (uses a texture slot per 2 planes) or apparently 'see the "TexKillClipPlanes" example delivered in the NVIDIA Effects Browser' for a way of doing your own clip-planes in a shader.

Wyzfen

July 15, 2006 08:13 PM

Drilian

Texkill and user clip planes are generally not advisable - performance implications and the like. (Texkill totally disables some of the z buffer optimizations on newer hardware, and user clip planes are also supposedly slow, though I cannot recall exactly how).

Could be the simplest (and perhaps easiest) way would be to just use the stencil buffer for each door.

A two-pass method, that would go something like this:


// On Render begin
Clear stencil buffer to 0.


// During render:
n = 1; // Yes, 1.
for each door that is not fully closed (or, possibly, fully open as well)
  Set stencil buffer to write the value of n to the stencil buffer on render when the depth pass succeeds.  
    (Disable the stencil test - we're writing a stencil value, not reading it)
  Draw a set of "dummy" geometry inside the door frame (i.e. a polygon that covers the doorway)
  Now set the stencil buffer to not write, but to fail whenever the current stencil value is not equal to the value of n.
  Draw the actual door
  n++
end for
disable the stencil test.

Basically, you'd render a unique value to the stencil buffer for each door frame that needs it, and only inside of the actual door frame. Then, you render the door, and fail rendering whenever the stencil buffer is not whatever number this is.

The advantage of this is that you don't have to clear the stencil buffer between each render, and it should even handle doors that overlap in space. You also don't have to do the stencil test for doors which are fully closed - there's nothing sticking out, so you don't have to worry about it.

Hope that helps, and don't hesitate to ask if I have been unclear about anything.

EDIT: A quick edit to note the placement for what should be rendered into the stencil plane - I was thinking if you have a polygon on the near side of the door frame that completely fills the doorframe, you'd render that (to the stencil buffer only, and not to the color/depth buffer, but you would probably want the depth test enabled in case of occlusion). i.e. if the doorframe looks like:


  ______
 /      |
|       |
|_______|

(note that backslash was giving me trouble, so the upper-right corner is not as "rounded" as I would have liked, and it's late and I'm tired), then you'd want a polygon that fits it perfectly. and it'd be facing the camera on the side of the door that is nearest to the camera.

I think it should work...again, let me know if you're unclear - maybe I'll draw a better diagram or something.

EDIT2: Shortened the line length in the code block, it wasn't word-wrapping.

July 16, 2006 02:29 AM

Milkshake

Great discussion gents - really helpful for me to kick around alternatives like this.

So I'm not using shaders ... yet. I'm trying to keep myself focused on gettings something I can play first. But at the same time, I will definitely be adding shaders - so I'm trying to make sure anything I do will work with shaders as well.

The 1-bit mask would be cool - but I think this would need to be a multi-pass solution similar to Drillian's suggestion below because the doors aren't flat - so the mask would change based on the orientation of the door. And I'm guessing as soon as you're generating this mask dynamically, the stencil buffer is probably as easy as anything.

I was kicking around some ideas for the stencil buffer myself - but not as well thought through as yours. I would unfortunately need to clear the stencil buffer every frame, because I've got stencil buffer shadows in as an option right now (and yes, I know how slow they are =)

Another option I was thinking about is a variation on the black polygon. If I render all the doors last (and I'm guessing this will interfere with my transparent object pass), I could setup an enclosure on each side of the doorway for the doors to slide into - but instead of rendering these as black polys, I could just render them to the depth buffer with colour disabled. This would stop them rendering anything on top of the other things in the scene. Plus it will be fast. I just need to work out what this will mean for transparency (particularly because I'll be using a lot of transparency in the doors themselves to put in windows so doors don't block the view so much ... and I think it will look cool). The more I think about the issues with transparency though, the more I think I should probably use the stencil buffer ... which is a shame as I was hoping to keep this free for other rendering tricks I might need it for down the road.

It's similarly unfortunate that clip planes perform so poorly with hardware shaders. There will never be that many doors on screen, and most of the other options I'm considering will setup multi-pass anyway, so I'm not sure how much I should worry about it.

Finally, the other option I've been thinking some more about is doing booleans in the modeller. But instead of exporting a fixed number of keyframes, bake the effect of the boolean into the vertex animation curves. This also has some performance implications as I'll now be doing mesh deformation on the CPU anytime a door is moving - but my biggest concern with it right now is still the texture warping as soon as there is any non-uniform UV projection.

July 16, 2006 07:29 AM

Milkshake

Oh, and Wyzfen, thanks for pointing out the texture slot impact of clip planes. I hadn't realised that.

July 16, 2006 07:43 AM

Drilian

The stencil buffer is not as slow as you think - it's roughly on par with the depth buffer, and since it's even in the same buffer as the depth buffer, if you're already clearing depth (Which I assume that you are), then you basically get the stencil clear for free, as well. The real performance impact of stencil buffers is that they require drawing geometry into them. It's fill-rate. But if you're writing just to the stencil buffer, you can turn off color and depth writes, and you'll get alot of that back (and, on some hardware, there are special optimizations for stencil-only rendering, where it can draw more quickly to just the stenicl buffer...I think it's nVidia cards that have this).

Point is, don't disregard the stencil buffer due to performance issues - The Doom 3 engine couldn't exist if they were as slow as a lot of people think :) The trick is in knowing how to use it effectively - then it's an amazingly useful tool.

July 16, 2006 04:40 PM

Milkshake

But I'd need to clear the stencil buffer twice if I'm using it for both shadows and this doorway trick right? And while the first stencil buffer clear can be done in the same operation as the colour/depth buffer clears (which is what I'm currently doing), any additional clears would be separate operations which would flush/stall the render pipe in the middle of the scene render wouldn't they?

July 16, 2006 05:35 PM

Drilian

Well, if you use a specific stencil reference value (you have 8 bits, so maybe start that reference ('n', in my earlier description) at 100 or somewhere around there), then you shouldn't have to do a seperate clear. Unless you plan to have a situation where the stencil buffer, during shadow rendering, can end up at 100 (which is pretty unlikely, since your performance would be so amazingly bad in that case).

The idea is you're telling the little invisible door-frame polygon to render EXACTLY a reference value to the stencil buffer, and if that exact value isn't in the stencil buffer while rendering a door, it won't render. So you don't have to rely on stencil increments/decrements to get it done, you can just set an arbitrarily large (but not too large) value, and run with that. No extra clearing required.

July 16, 2006 08:32 PM

Milkshake

I can see that would work if the shadows were cast before rendering the doors. But if you want the doors to be able to cast/receive shadows, the shadow pass has to happen after the doors are rendered. Which I think either means you need to partition the bits somehow (so that shadows only use the low 4 bits, and portals use the high 4 bits), in which case, you're down to a resolution of 15 portals, and 8 shadow overlaps (as the shadows can be + or -) ... and even then, I'd have to check how clever the stencil test can be to see whether the shadows could mask bits out AND do a >= 0 test. Or it will require the extra clear inbetween passes. ... Or I'm just getting tired and being silly =)

The other thing that still concerns me with stencil masking is viewing the door/doorframe at "grazing"/oblique angles, as, because you loose all the depth information going through the stencil buffer mask, I think it's possiblle to design doors that can't be un-ambiguously masked out (i.e. by creating a masking volume large enough to include all the detail inside the doorway, you will accidentally include some of the stuff at the front/back too). This would benefit enormously from a diagram, but imagine a door that has a big honking door-handle that sticks out the front, and then view it almost from the side. Any volume that is "loose" enough to include the sticking-out door-handle, will also permit pixels hanging out the back of the doorframe when you look at it from nearly side on.

Do you see what I mean? I might be smoking the curtains here - so feel free to tell me if that doesn't make sense.

And that's where a solution which does the occlusion in 3d (like clip planes, or booleans) will still work, while a projected solution (like stencil buffer) will fail.

I know there's a certain point where you just turn those kind of limitations into parameters that door models need to satisfy (e.g. "Don't make doors with sticky-outey door handles"), I've only been kicking this around for a day or three in my spare time, so I've not given up hope of an "unbreakable" solution yet.

Oh - and I also realised that any solution that uses a depth buffer trick (e.g. render the black polygon only into the depth buffer) is no good either, as these will break any other rendering effects which rely on the depth buffer values corresponding to colour pixels that have been rendered - like shadows or depth or field. It would be impossible to cast a shadow on the ground behind a door that had been clipped.

Perhaps I focus my efforts on ensuring that door masking/clipping is only ever used as the door is in the process or opening or closing (i.e. ensure doors that are just sitting there open or shut don't need any fancy rendering tricks) - as it would be very rare to have more than 1 door opening/closing at any given time, and most of the time, there won't be any doors moving. And then just pay the price of an expensive solution like clip planes that guarantee me accurate results which won't interfere with any other rendering effects.

July 16, 2006 08:55 PM

Drilian

Ah, yes. I can see the limitations that you're talking about. I didn't think of that :) It might be worth, if you have the time and/or inclination when this becomes a true issue and not merely an annoyance, to stop into the afternet.org IRC channel #graphicsdev. You can get there via a web applet on http://graphicsdev.net/irc.php. There are a bunch of super-knowledgable people there that might have other ideas.

July 17, 2006 12:09 AM

You must log in to join the conversation.

Don't have a GameDev.net account? Sign up!

Milkshake

Author

Disappearing Doors

Comments

Milkshake

Latest Entries

Moose Sighting

Constraints

Moose

Melon Golf

Toon

Spaceships

Rendering PtIII: Lighting and Shadows

Rendering Pt2

Rendering Pt1: The Boring Bit

Hardware Shaders

Disappearing Doors

Comments

Milkshake

Latest Entries

Moose Sighting

Constraints

Moose

Melon Golf

Toon

Spaceships

Rendering PtIII: Lighting and Shadows

Rendering Pt2

Rendering Pt1: The Boring Bit

Hardware Shaders

Reticulating splines