Jump to content
  • Advertisement
Sign in to follow this  
spek

Loading array with dynamic sub-array's from a file

This topic is 3417 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

I was wondering how to properly structurize a set of nodes. In my game, sectors are loaded in the background while the players moves around. From what I know, it's best to keep the chunks as large as possible. While the HD is reading, the other threads can continue with the game. If that is true, it would be best to allocate the whole thing in one step, then read the data into the record-array. This time I need to find a way to store a large set of (pathfinding) nodes. Something like
<Delphi code>
TRelation = record
  targetNode : pointer; // pointer to another TNode
  data  : integer; // Some additional data stuff
  ...
end;

TNode  = record
  relations     : array of TRelation; // dynamic array
  relationCount : integer;  // count of related nodes
end;

nodes : array of TNode; // The array I'd like to write/read from a file
So, there is an array of nodes with a known side. But the relation count could differ for each node. Also loading and storing pointers doesn't work I guess, so I'll have to replace the "targetNode : pointer" with "targetNodeIndex : integer", which is an index in the "nodes" array. I could read node by node, and allocate the "relations" array for each node
   stream.read( nodeCount, 4 );   // read the count of nodes
   setLength( nodes, nodeCount ); // Allocate node array
   // Now start reading node by node
   for i:=0 to nodeCount-1 do begin
      stream.Read( nodes.relationCount, 4 );  // Read relation count into node
      setLength( nodes.relations, nodes.relationCount );
      // Read relations
      stream.read( nodes.relations, sizeOf(TRelation) * nodes.relationCount );
   end;
Works, but something tells me it can be done better. And when continously loading/unloading sectors, won't this fragmentate the memory? Greetings, Rick

Share this post


Link to post
Share on other sites
Advertisement
Quote:
Original post by spek
I was wondering how to properly structurize a set of nodes. In my game, sectors are loaded in the background while the players moves around. From what I know, it's best to keep the chunks as large as possible. While the HD is reading, the other threads can continue with the game. If that is true, it would be best to allocate the whole thing in one step, then read the data into the record-array.


Windows provides asynchronous IO via Overlapped IO structures. That would be the "best" way to do it.

Quote:
This time I need to find a way to store a large set of (pathfinding) nodes.
More than 20 megabytes? If not, then just load it once, it's not worth the complications.

Quote:
Works, but something tells me it can be done better.
Not really. It's just serialization.

Quote:
And when continously loading/unloading sectors, won't this fragmentate the memory?

No.

Share this post


Link to post
Share on other sites
20 MB...I whish:) All the nodes together might eat 1 gig. There are millions of them. However, I only load the sectors nearby the player, so probably I only need a few MB at a time. But due lots of loading/unloading, I was afraid of not doing it right.

The loading happens in the background in another thread. The loading doesn't have to be superfast. More important is not to block the game threads too much. From what I understand, other threads can just continue while reading. But in the code I gave, the reading is continuously interrupted with reading small bits and setting array lenghts. Maybe its not so bad, but if there is a better way...

But if I get your reply right, there's not really a better way to load? I'm not familiar with "Overlapping IO structures". I did a quick read on MSDN, but I don't think its really helping in this particular situation. But I could be wrong of course.

Share this post


Link to post
Share on other sites
Look into hierarchical pathfinding. Doing any pathing on "millions" of nodes is going to be insanely slow, so you get two bonuses: you don't have to load the lower, finer-grained portions of the graph unless you absolutely need them for pathing; and pathing calculations are automatically faster because you reduce the size of the working set of nodes.

Share this post


Link to post
Share on other sites
Well, there actually are two levels above this one for "global pathfinding". The millions of nodes will never be loaded or used at the same time. But when summing everything together, this is the estamated node count. One of the ideas was just to load all nodes initially, in order to save headaches with background loading, allocating memory, etc. But in this case the world is too big to load everything at once.

Share this post


Link to post
Share on other sites
Sign in to follow this  

  • Advertisement
×

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

We are the game development community.

Whether you are an indie, hobbyist, AAA developer, or just trying to learn, GameDev.net is the place for you to learn, share, and connect with the games industry. Learn more About Us or sign up!

Sign me up!