Hi all, I am trying to make a Pirate game and I want to make a huge world and load only portions of the world at a given time. What are the ways of doing it? I am quite confortable with FPSs scene graph and collision detection tecniques like BSP, Octrees and OBBTress... going right to the point I am migrating from indoor to outdoor scenes and would like some help to identify the crucial diferences in the scene management.... I have already a terrain engine that uses a height map and procedural shading to create the world.

Quote:What are the ways of doing it?

You say you're familiar with using various scene-graphs/heirachical methods - so you'll want to stick with them (they're a good tool). Ideally you'd want to tailor your source data to fit in with this scheme... right?

The following would be my personal choice, I'm sure others will suggest variations... I've done a fair bit of work with outdoor terrain engines, but nothing substantial with indoor engines - so I can't give you the full comparison/picture.

I'd allocate a given amount of storage space to the world geometry, be this in SYS-RAM or V-RAM (or both), all current data should be stored in here as it is the working set. The working set should scale depending on the hardware, this way on high end machines you'll get limited loading/changes and on low-end machines it'll still work out.

I'd then use a LRU algorithm to purge tiles/sectors from the working set based on how long since they've been used. Either purge tiles after a given time, or when a new tile is loaded it purges the oldest in storage.

When it comes to loading tile data I found a "time sliced" approach to give the absolute best results in performance. Basically, if you can identify the direction your character/camera/whatever is moving then you can predict what tile(s) will need to be in the working set soon. Sure, it won't be perfect, but in my experience it works for the majority of the time.

When you've identified a tile as being needed, you can start loading it from disk/memory at a few kilobytes/frame. I measured the frame rate before hand, decided that it couldn't go below 30fps, took the difference - and this time slot (say 10ms/frame) is used to load as much as possible from disk. After the time elapsed it stopped loading (but kept the data) and did the next frame...

With this set working reasonably well (the ratio of time used, data loaded and total data size is difficult to balance at first) for most cases the new tile will be resident in memory by the time the player actually interacts with it. If it isn't quite ready, you can just load any remaining data on the last frame - which would hopefully only delay proceedings slightly.

One idea, that I've never tried, but seen in GTA:VC was using a LOD sector before the "real" one is displayed. If you jump on a bike in Vice-City and belt it from one end to another you can sometimes see the LOD algorithm and paging system play "catch up". For a few seconds all of the buildings are nothing more than low-res textured cubes. At least you know roughly what is there and you don't end up driving off the end of the world [wink]

------------

Anyway, thats about all I can think of from my trial-and-error. If it's a bit too vague I can probably explain bits in more detail if they're of any use.

hth
Jack

Quote:Original post by jollyjeffers
When it comes to loading tile data I found a "time sliced" approach to give the absolute best results in performance. Basically, if you can identify the direction your character/camera/whatever is moving then you can predict what tile(s) will need to be in the working set soon. Sure, it won't be perfect, but in my experience it works for the majority of the time.

When you've identified a tile as being needed, you can start loading it from disk/memory at a few kilobytes/frame. I measured the frame rate before hand, decided that it couldn't go below 30fps, took the difference - and this time slot (say 10ms/frame) is used to load as much as possible from disk. After the time elapsed it stopped loading (but kept the data) and did the next frame...

I'd second that system, with the only exception being to the actual loading of the data. Rather than allocate a time-slice of your game loop I'd place resource loading into a separate thread. Reading from disk will spend most of it's time waiting for the transfer between the disk and memory to happen, so having the I/O in your game loop will be wasting time as it waits. By keeping this in a separate thread you main game loop can still execute while the I/O thread is waiting for the transfer.

Think I get it but...


How exactly I will define this working set?

How am I going to limit the scene graph? Is there any technique or something?

Quote:How exactly I will define this working set?

I just had an array of indicies effectively.

Say you have your working area of memory (say 32mb for example), you allow one container to manage this - and when it loads something into memory it assigns it an ID that you can use as an index. My rendering code then maintains a list of indicies that are currently on screen (this can be cross references to add and remove segments) for rendering.

Quote:How am I going to limit the scene graph?

I'm not sure I get your question here. The scene graph itself, at least in my case, is a relatively trivial heirachichal structure that doesn't have a heavy memory footprint. I keep it in memory all the time.

When traversing the graph and determining what I need to display on screen I refer to the working-set/data-pool for the actual data (back to the whole paging in/out process).

hth
Jack

My exact problem is that the world is huge... I do not know if my question make any sense maybe even the largest MMO world fit in 32 MB of RAM...

If it is big enough and I must keep only a portion of the world in memory?

Quote:Original post by tulio
My exact problem is that the world is huge... I do not know if my question make any sense maybe even the largest MMO world fit in 32 MB of RAM...

If it is big enough and I must keep only a portion of the world in memory?

The discussion thus far has been about the world being "huge" - in the sense that it is TOO BIG to fit in memory all in one go. This is often the case with your situation (from what I understand). If you could fit it all into memory then much of this conversation would be demoted to a useful optimization and not a requirement.

I originally stated that you should scale according to the capabilities of the target system. At some point (maybe when you first start loading the data) you will know how much memory it will take. Maybe it'll be 32mb, maybe it'll be 1000mb.

When you know how big it is, and when you know the capabilities of the system you can determine what sort of scaling you need. For example:

presume we have 175mb of pure geometry to deal with, that is, the landscape that you are wanting to load occupies ~5.7m verticies. At a crude guess, this could be upto 2000x2000 square kilometers of landscape - enough for most uses I'd hope [grin]

On a low end system, 128mb SYS-RAM, 32mb V-RAM, you can't fit any more than about 15-20mb at any given time. Thus you're going to have to aggressively page data in/out of V-RAM and to some extent also SYS-RAM.

On a medium system, 512mb SYS-RAM, 128mb V-RAM, you can fit the entire data set into SYS-RAM, which is relatively fast. But you can't fit it all into V-RAM, so whilst you can store about 1/2 of the data in V-RAM at a given time, you're still going to have to page data to/from SYS-RAM every now and then.

On a high end system, 1024mb SYS-RAM, 256mb V-RAM, you can fit an entire copy of the dataset in SYS-RAM and probably an entire copy in V-RAM. No paging required, best case scenario. However, you're likely to want a whole load of textures as well, which might well mean that you can't burn all 175mb of V-RAM on geometry, maybe only 100 or so... which means you still want to use paging...


I'm reckoning you can develop a system that is adaptive to its host environment. Using a clever set of variables to decide how much data can be stored at a given point in time, and how much data is required etc... you could initialize the variables at startup based on the capabilities of the host, and on the low end system it'll be aggressive, and on the high end system it'll be fairly smooth with no swapping.

Is that a bit clearer?
Jack

Yeah...

Thanks for the help...

Now I get it