http://people.ucsc.edu/~asue/terrainportal.jpg Heres a simple picture to illustrate the problem Suppose we want to put a building into a terrain engine. If the building is completely above ground this is easy. But in this situation, we have a building that has a basement; it extends below ground level. A bunker built into a hillside. Im doing portal engine for the building architecture. referring to diagram: In Black a sector used to build the walls and external parts of the bunker. this sector would have transparent sky and outer walls, to allow the player to leave into the terrain, and have a few solid walls to build the bunker itself. Note that polygons in this sector, should be covered by the terrain -where the bunker walls meet the ground for example. In Green, this sector is a problem, it is inside the doorway of the bunker, but the terrain crosses it. In Brown, a sector completely inside the bunker from here it might be possible to look out the doorway and see the terrain, but for the most part isnt an issue so long as the terrain is flat enough that no terrain polygons face inwards. anyhow, the issue is that Green sector in the doorway and the fact that the terrain would cover it, when really you need to be able to see down it. so, in terms of visiblity ordering: Terrain covers black sector walls, Black sector covers green sector walls, Green sector covers terrain polygons from each of these three regions need to overwrite one another, but in an order that is impossible Descent 3's solution to this issue i believe, was to allow you to delete specific polygons from the terrain heighmap whenever you wanted to tunnel below ground. but this sounds like a bad solution to me, since you must then align all your doorways and windows with terrain grid cells. Does anyone have any suggestions for how to handle this transition between terrain and portal engines nicely? I suspect that a stencil buffer trick is going to be needed, but would rather avoid doing that, since some cards lack hardware stencil, and it takes extra passes....

It's been a while, but in Unreal Editor I think what you were looking to do was called "subtracting." It would find the places where the terrain intersected with the object, then clip the terrain's polygons (creating new ones where necessary) to form around the new void.

a large problem with this is that my terrain is procedurally generated at runtime

so any special changes i make to it in the vicinity of a building would need to be a special check to load overriding data

this also introduces the problem of, assuming i allow for overiding or deleting specific polygons in the terrain,(and the associated overide data loading/caching scheme) rendering the heightmap becomes messy, currently its all in a vertex array
breaking holes means no more continous tri-strip, and multiple array draw calls



well... its just looking to me like theres no clean way of joining the two engines without putting a lot of special case hacked stuff into each one
it'd be nice if there was a way for them to somehow not need to care about one another

well, what about stencil approaches? i havent used stencil buffer for much other than simple shadows...anyone have suggestions on how it might help here?

i am starting think something like....

first render terrain and outdoor sectors, with depth testing on
so the outside walls clip agaisnt the ground as they should

then i have special portals that lead indoors - to sectors where the terrain loses out
so i render these transition portals as just flat polygons across their entrances, and use that to mark the stencil buffer

then i clear the depth buffer
and render indoor sectors, but only write to screen in areas that also were marked in stencil buffer

does that sound workable?
but that has to clear the depth buffer in the middle...and the stencil beforehand
and the depth buffer again between frames
which kinda sucks....

Or you make sure to design your building so that once you get far enough into it, there's no way to see any terrain and it's then switched off for rendering suitably in time before going into the basement. This is one of those things where you can consider if it's worth the extra trouble to work on the tech. I guess it would be if your game has many small buildings that lead underground, without a good way to hide the terrain switching off, then try to find a solution. Otherwise you might just be designing overhead. Sorry if not very helpful.

You said your using a portal engine. You should be able to tell whether you are inside or outside a building (assuming all buildings connect to the outside world with portals in the windows / doors etc). Can't you just ignore the terrain when you're inside? It only exists on the outside? Then your buildings can be any shape you like. The only issue would be collision detection on the boundaries when the physical body is half way through a portal.