'Texture streaming'?

Started by
3 comments, last by DanielKruyt 10 years, 4 months ago

Hi there,

I'm looking to learn the correct terminology for what I can best describe as 'texture streaming' to aid me in my Googling. It's the problem of memory constraint, when you need to load a texture dynamically because the camera has moved from a city to a forest and you need a whole stash of new textures.

The texturing system I currently use in my engine ( which I am writing for educational purposes as well as a hobby ) is based on a texture array. I use each layer as an atlas except in the case of textures that need to be repeated, where I resize the textures to the maximum resolution and fit them into their own layers.
The problem with this is that I do not know if I even *can* 'stream' new textures into the same texture array, or any algorithms to do so in a timely fashion, hence I need to know what this is called. Any papers/tutorials/etc on the subject are appreciated as well.

Thanks for your time,

Daniel Kruyt

Advertisement

You can map the texture array elements as subresources.

Your approach sounds similar to ID's "megatexturing", where parts of the physical texture are loaded on demand based on an analysis frame which determines what logical textures are needed for the drawn frame, and in what relative size the logical textures should be within the physical texture.

Also search for "clipmapping". The public literature on the subject has been written before hardware-supported texture arrays became common. But, if anything, the newer available hardware techniques make the implementation easier.

Niko Suni

I've a feeling that what you're really looking for here is not actually streaming, but rather just a simple map loader. When the player moves from a city to a forest you pause the game, put up a "Loading" screen, delete all the city textures and load all the forest textures.

This doesn't only apply to textures - when you leave the city you're not going to need all those building models any more, but you will need tree models, so you'll be wanting to do the same with vertex buffers.

Direct3D has need of instancing, but we do not. We have plenty of glVertexAttrib calls.

The borders are fuzzy at some point. Prefetching resources on demand can be done on in-game triggers if you want to avoid the loading screen. You just need to detect which resources will be needed next and which resources can be dropped now. The former one can be done on proximity or on explicit trigger volumes, while for the latter aspect also a decent resource cache system can be used. A more elaborate prefetch system may start with low detail level resources, and prefetch successively higher levels when needed.

A Clip-map (and its derived work MegaTexture and Virtual Texture) actually load LODs on demand, IIRC, but are intended for handling very large textures instead of dealing with totally exchanging the appearance. However, technically the difference is small to a texture management with prefetch and caching considering LOD.

The texturing system I currently use in my engine ( which I am writing for educational purposes as well as a hobby ) is based on a texture array. I use each layer as an atlas except in the case of textures that need to be repeated, where I resize the textures to the maximum resolution and fit them into their own layers.

Texture atlases are usually somewhat big, say 2k by 2k. Upscaling repeatable textures just for the purpose of fitting them into the one texture array seems me a bit strange. When the texture is relatively small originally, this would waste much texture space.

The problem with this is that I do not know if I even *can* 'stream' new textures into the same texture array

Yes, it is possible to replace parts of textures. However, the question is whether this would not block the entire texture from use by the rendering process or at least would introduce another temporary copy. I assume that in case that a GPU command is still queued that access the texture for rendering, an update on the texture will cause some inefficiency. This would at least not occur when using different textures where the targeted texture is known to be obsolete (its similar to double buffering then).

@haegarr: Yes, I was aware that resizing the texture was a bit, well.. stupid, but I was not sure how costly glBindTexture was, to be honest, and my run-time atlas and model compilation took me quite a while to think up, so I wasn't sure how I could modify the code to suit binding different textures.

I'm guessing I can categorise models and terrain into world-space cubes for pre-emptive loading, and build a megatexture for all the static geometry. I'll probably do it at run-time, I believe I saw an article from a Leadwerkz developer about that. Time to rewrite everything for the umpteenth time, lol.

Thanks for all the advice, guys. :)

This topic is closed to new replies.

Advertisement