• Advertisement
Sign in to follow this  

Terrain streaming

This topic is 3324 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

I want to use multiple heightmap textures for rendering large terrain. I'm using vertex texture for terrain rendering, meaning my terrain size is limited by the maximum texture size supported by the hardware.(and also it is memory consuming to use one big texture). So if I want very large terrain, I need to implement some kind of terrain streaming. Questions: - The size of the heightmap tile textures? (large draw distance is still important) - What about terrain texturing. Each heightmap tile has it's own base texture and detail textures etc? - How is streaming and unloading data performed. Using a only distance criterion? - What are the pros and contras of streaming. - Links for papers about streaming. Thanks in advance. [Edited by - Hiyar on January 15, 2009 6:45:07 AM]

Share this post


Link to post
Share on other sites
Advertisement
What your looking for is something called mega texturing or in some instances geometry clipmapping (it's not technically the same thing but the idea is the same), its a method in where you use several large repeating textures where each one covers a larger area than the next.
the idea is that as you move across these textures you replace small areas of it just beyond your visibility limit, thus you always see a fresh set of textures.

1.size could be anything, rage from id software uses textures up to 20GB in size. However the size used in the GPU memory is always constant and whatever you say it is
2.texturing uses the same method, in fact it was originally made for this.
3.yes, distance and projected heading (if you can you should get ahead of yourself), unloading data is not needed, just replacing.
4. pros: it's seamless, has high resolution and you can have a unique texture all over the place.
Cons: it uses a lot of memory and if you move to fast across the map you will have problems updating it.
5. try google, but there isn't that much work done on this

Another method is something called texture paging or terrain paging or something like that, it is basically using several smaller textures and then rendering smaller patches of the mesh for each one.
It works pretty well, but you can often see the texture borders between them and if it's combined with LOD small gaps can occur.

Share this post


Link to post
Share on other sites
No, not mega texturing. Maybe you misunderstood my post. Thread is about geometry streaming, not only texturing. The heightmap texture is used for vertex texture fetching. XZ position values of the terrain are computed in the shader using scale bias etc, heightmap vertex textures is used in the shader for the terrain elevations.(Like displacement mapping) This approach saves memory, but like I said terrain size is limited by the maximum texture size. Because heightvalues are stored in the texture. I wanted to know how to use multiple heightmap (vertex)textures. And load them when need.

I'dont really want to implement streaming, but I want to know how it works for vertex texture based terrain.

Share this post


Link to post
Share on other sites
Well, as far as i know, there isn't that much of a difference between textures used for vertex texture fetch and "regular" textures. At least from the point of few of the application (i.e. the only real difference are in the shaders). So any method that allowes your application to stream/page color textures should be directly applicable to vertex textures as well.

Share this post


Link to post
Share on other sites
Yes, right. Generally it's the same. I don't know much about mega texturing, I didn't know that this technique is applicable for heightmap based terrain geometry.
If I use texture paging/terrain paging I think the size of the heightmap texture is important. I need more details please.

Share this post


Link to post
Share on other sites
I haven't done anything significant in this direction either but i don't think that there is one best size to use for your textures here. Bigger textures need more time to load, smaller textures meens you have to reload more often.
Why not read up a little bit on the techniques lc_overlord mentioned and build a little demo app? Shouldn't take that long and would allow you to try out yourself what sizes work best and try some methods to determin what terrain tiles/patches to load when.

Share this post


Link to post
Share on other sites
I think what Sebbit says is valid. You want to look into Clipmap Texturing, MegaTexturing (not a lot of technical information has been published afaik), virtual texture (e.g. sparse virtual textures). Maybe you can use this (http://www.gamedev.net/community/forums/topic.asp?topic_id=504549)?

Share this post


Link to post
Share on other sites
Quote:
Original post by Hiyar
This approach saves memory, but like I said terrain size is limited by the maximum texture size.


Terrain size is never limited by max texture size, it only limits it's resolution.
Far away you really want less resolution, it's only up close you need more, and mega texturing does this.
As far as i understand this might save memory under the right circumstances, at least make the memory it takes up constant, and if it takes up to much you could always drop a layer in resolution.

Quote:
Original post by Hiyar
If I use texture paging/terrain paging I think the size of the heightmap texture is important. I need more details please.

Oonly for the individual pieces, there are two main issues to it.
1. your dealing with individual textures here meaning that your doing a lot of texture binding and that they have to match up with the individual mesh pieces you render.
2. it really doesn't take LOD in consideration

Geanerally speaking the streaming part of these two methods are almost the same, it's what you do with it later that differs.

Share this post


Link to post
Share on other sites
My terrain is tiled/chunked. If the file for the tile doesnt exist, it generates one from an array of heightmaps (one tile would be only a part of a height map). This is for initial generation of terrain, usually the terrain is already packaged as a file. I use several LOD's, which are just different index buffers at the moment. The terrain objects are a linked list in 2 dimensions, and they add and release as you move around. The data is stored as byte[] because .Net streaming is too slow. I go straight to/from the graphics card, so I dont have to worry about marshalling. I dont know if this is the best way of doing it, but I am happy with performance so far.

I should say, terrain, not including water, trees etc renders pretty fast regardless in my experience. Its all the other add ons that really test your engine.

Share this post


Link to post
Share on other sites
Quote:
Original post by lc_overlord
Terrain size is never limited by max texture size, it only limits it's resolution.

If using vertex texture without any kind of streaming, max texture resolution supported by the hardware limits the terrain size. That's what I mean in the above post. (Generally, using vertex texture reduces memory cost, because I can use a single small vertex buffer for the entire terrain, rendering (and LOD) is done by using scale and bias in the shader)

I will look into geo clipmapping, but initially I wanted to know how to use multiple heightmap textures.Take a look at this paper:Click (Read terrain part)

They are using multiple vertex textures and a single vertex buffer for terrain rendering.(if I understand that correctly). I wanted to how is the streaming performed, if using several vertex textures.

Edit: Typo

[Edited by - Hiyar on January 17, 2009 12:43:21 AM]

Share this post


Link to post
Share on other sites
I think you're still missing the point about the fundamental tradeoff of size vs. resolution. But ignoring that... most streaming systems are VERY simple. You make a big uniform grid of all your stuff (in this case, one heightmap texture per cell in the grid). Then you try to ensure that you always have some local region around the camera loaded. Either the closest four cells, or the current cell and the 7 neighbors, whatever. As the camera moves around, you use spare cycles (or another thread) to load new cells on the horizon and throw out old ones.

Clipmapping and megatexure are just elaborate extensions of that idea, so that you have more resolution in the local neighborhood of the camera, and less resolution as you get farther away. But again, understanding that requires that you realize that texture size does not limit terrain size! My 256x256 heightmap could represent all of North America ... except that each polygon would cover something like 15 miles.

Share this post


Link to post
Share on other sites
Quote:
Original post by Hiyar
They are using multiple vertex textures and a single vertex buffer for terrain rendering.(if I understand that correctly). I wanted to how is the streaming performed, if using several vertex textures.


What exactly do you want to know here? How the load new vertex textures in the background? The look for papers/tutorials that describe things like texture streaming/mega texturing as it has already been said. Google should know quite a few.
Or do you want to know how to use more than one heightmap texture when finally rendering your terrain? That has nothing to do with "streaming".
Just seems like we're always talking about different things here.

Share this post


Link to post
Share on other sites
Quote:
Original post by osmanb
I think you're still missing the point about the fundamental tradeoff of size vs. resolution. [...] Clipmapping and megatexure are just elaborate extensions of that idea, so that you have more resolution in the local neighborhood of the camera, and less resolution as you get farther away. But again, understanding that requires that you realize that texture size does not limit terrain size!

Great, I don't know much about mega texturing and I need to ask: How is this done without streaming?
I said:
Quote:
Original post by Hiyar
If using vertex texture without any kind of streaming, max texture resolution supported by the hardware limits the terrain size.

I might be wrong. As far as I understand, mega texturing uses different resolutions for the LOD levels, how is data managed in this case?

PS: For my terrain I'm using 4096x4096 texture and the distance between the vertices is 2 meters. So the total terrain size is ca. 8 km². The point is I can't load the whole texture at creation time, if I want much larger terrain, hence hardware limitation. (And of course it wouldn't make sense, because of the high memory cost.)

Share this post


Link to post
Share on other sites
Quote:
Original post by Hiyar
Quote:
Original post by Hiyar
If using vertex texture without any kind of streaming, max texture resolution supported by the hardware limits the terrain size.

I might be wrong. As far as I understand, mega texturing uses different resolutions for the LOD levels, how is data managed in this case?

Basically megatexturing is a few layers with the same size, each one stretched further and further until the last one which is stretched across the whole terrain, then they are blended together to provide the best resolution possible.

Quote:
Original post by Hiyar
PS: For my terrain I'm using 4096x4096 texture and the distance between the vertices is 2 meters. So the total terrain size is ca. 8 km². The point is I can't load the whole texture at creation time, if I want much larger terrain, hence hardware limitation. (And of course it wouldn't make sense, because of the high memory cost.)


If you use bilinear sampling(something you have to implement yourself) you don't have to limit yourself to one vertex per textel.

Share this post


Link to post
Share on other sites
Mega texturing is a streaming technique. You have one large texture such as 32kx32k on harddrive and required parts of the texture are streamed at run time.



Bilinear sampling performs bilinear interpolation between four texels, so bilinear sampling can not refine my terrain. It is not possible to define or modify the values between texels.



I will use multiple smaller vertex textures and a simple distance based criterion for streaming and just look how it works.

Share this post


Link to post
Share on other sites
Quote:
Original post by Hiyar
Bilinear sampling performs bilinear interpolation between four texels, so bilinear sampling can not refine my terrain. It is not possible to define or modify the values between texels.

True, but at least you don't get the step pyramid / terrased look on your terrain if your vertics does not line up exactly to the textels.

Share this post


Link to post
Share on other sites
Quote:
Original post by Hiyar
For my terrain I'm using 4096x4096 texture and the distance between the vertices is 2 meters. So the total terrain size is ca. 8 km². The point is I can't load the whole texture at creation time, if I want much larger terrain, hence hardware limitation. (And of course it wouldn't make sense, because of the high memory cost.)

If you want bigger terrain, you have to stream it (slowly over the course of multiple frames with sufficient periods of no CPU/GPU load in between them to allow for smooth user interaction).
Chunk after chunk, once it reaches the given threshold distance.
You really have to think of LOD right away or suffer the consequences of rewriting it several times. There are just too many chunks to load them all in full detail - see My Screenshot of Streaming Terrain
So you have to cheat.

You need to load the data asynchronously and make sure the renderer works just fine even if the desired data isn`t still loaded, so you need to have some backup plan for this case.
Obviously, there are lots of other issues that you discover only after the basic implementation works, but there`s no point in discussing them until you see them. Only you can decide if the issues are worth fixing or not.

Share this post


Link to post
Share on other sites
Sign in to follow this  

  • Advertisement