Multilevel Grid Spacial Partition

Published September 26, 2014 by Agustin Daniel Perez Paladini, posted by agudpp
Do you see issues with this article? Let us know.
Advertisement

Space partition data structures are used in different fields such as collision detections, ray casting, and to perform efficient spatial queries. There are different data structures to partition the space, each one with cons and pros (QuadTrees, BSP, k-d trees, Bounding Volume hierarchy, grid, etc). In this case we propose a new type of grid space partition, where we can recursively divide the cells into new grids. The main objective of this data structure is to optimize the performance and memory usage in comparison with the normal grid space partitions.

Flat grids

Flat grids data structure are a very easy to implement and very fast when performing different queries. The main problem with this data structure is that it's good only when the elements are uniformly distributed in space. Suppose the next example:

flat_grid.png

Where each box represent an object and the grid (cells numbered from 0 to 15) represents the game world (where the entities can move for example). In this particular case a flat grid will be fine since all the entities can move around all the world space. But, there are some cases where the level is not fully available for the entities, there are some parts of the world where the entities cannot move around (a big river, buildings, forests, etc). We represent these things with red-cells in the next image:

problems_flat.png

In such cases we are wasting memory since those cells will be never used.

Multilevel grid

To improve these situations we propose a multi-level grid space partition. The concept is simple, and we will show it with images. For the example shown above, we can first divide the matrix into four cells:

multileve1.png

We can see that for cell 0 nothing will be never inside of it, so basically this cell is discarded for further subdivisions. Also, entities will be able to move on cell 3 only around the "white" space, so the real "valid space" of that cell is 1/4 of cell 3 size (assuming we discretize the space in Bounding Boxes). We can check also that cell 1 and 2 are the most used by the entities. For those cases we want to subdivide into more cells again. We can do it in different ways, the next image is one example of all those possibilities:

multilevel2.png

In this case what we did was: Subdivide the cells 1 and 2 (of first divisions image) into 4 cells and leave cells 0 and 3 as they were. For this case we have a total of 12 cells (4 cells in the first division and 8 cells in the second subdivision). Note that we are free to subdivide each cell as we want, it is not necessary to divide each cell with the same amount of rows and columns. We can think of each cell as a new "world" and then work on them recursively. In this example we save a total of 4 cells, having the same performance in Big O notation (we can access a cell in constant time with a factor of 2 instead of 1). Note that the factor will be associated with the number of subdivisions (deep in the hierarchy). In almost all the cases the number of subdivisions will be small (if we subdivide cells into 4 - 2 rows x 2 columns - recursively we will end up with performance similar to a Quadtree). Someone can think that this is waste of time and we are complexifying a problem too much for a very poor memory gain, for sure that for this particular example it isn't worth it, but for big scenarios these multi-level subdivisions can guide us to improve a lot in memory and time space.

Conclusion

In this short article we present an idea to optimize particular cases of the normal flat grid space partition, where using the same amount of memory we can increase substantially the performance of the "spatial queries". One interesting point about this data structure is that it could be automatically generated without big complications. From a list of entities sizes (bounding boxes) and static objects / unavailable parts of the level, we can calculate a good space division. Obviously for an optimum one we need to provide more information like a simulation of entities movement around the world. But this is another topic and something that should be analyzed in another article :). In https://github.com/agudpp/MGSP you can find the source code for this data structure (under development!).

Cancel Save
0 Likes 11 Comments

Comments

alnite

I don't think it's helpful at all to have an article this short without any data to back it up. How fast is this compared to having fixed-depth quadtree? How much memory does this save? How do you efficiently flatten and expand each node when objects are added/removed/moved?

April 24, 2014 05:35 PM
ferrous

I too would like to see a more detailed comparison to a quadtree, as at first glance, I can't really tell the difference between what you have here, and a quadtree.

April 24, 2014 06:05 PM
agudpp

Hi all, thanks for the comments.

I think I will need to add more data about this, I assumed things that I can see that are not obvious at all, my apologies for that.

This weekend I will try to add a more detailed explanation about it.

Answering your questions:

  • Difference between quadtrees and this structure in performance is almost analogy to the performance difference between a tree and a M-level hashtable (M << lg(N), where lg(N) is the levels of a tree).
  • Memory efficient is comparable to a Flat grid as I said in the introduction, the idea of this structure is to have better performance than a quadtree (fixed) having the same performance than a flat grid, but using a lot less memory than a flat tree (with the same number of divisions).
  • Note that this structure is not intended to be modified in runtime, like Grid's Space Partition, are calculated / built before running. Is a memory fixed data structure.
  • A quad-tree is only divisible by 4 recursively in each node/cell, this grid can allow you to divide by MxN the cells you want, depending your needs. If you divide by 4 each cell, this is much more memory expensive and probably with worst performance than a quadtree.
  • Another comparable (more efficient) operations against quadtrees and flat grids are: pick objects in a certain point, pick all the objects intersecting a rectangle / shape, perform "segment" raycasts. I will try to add comparable information about that too.

Note that I'm developing the data structure. My intention wasn't create a big article with implementation details, but for sure I should add more information about this kind of topics.

Thanks for the comments, I will update the article soon!

Regards

April 24, 2014 10:17 PM
slayemin

As it stands now, this article is not ready for publishing. It doesn't meet what I think are the standards for quality and depth.

I'm also very curious to see exactly how this is better than an optimized quad tree. What are the costs and gains (in memory cost, CPU performance, and programmer implementation time)?
Quad trees are easy to understand, work well with a tree-like data structure, can be implemented within a day, and have a sexy Log base 4 runtime. They also work very well with moving objects, where you can either rebuild the tree every frame, or with a little more work, do an in-place update of a branch.

You're right to note that this might be complexifying a problem, and I do level that claim against you. I suspect that this MGSP approach is just a pre-mature optimization on the quad tree, where the performance gains are so slight that they're inconsequential, and the true burden of cost is in programmer time. You really need to give hard data and performance metrics to show that the cost vs. benefit is favorable.

The limitation for supporting only stationary objects is also quite a big limitation to impose! Games usually have moving objects in them which need to be tested for collision. If I imagine a quake3 game with rocket launchers and flying rockets and moving players, how would using a static grid data structure be advantageous over other choices?

April 24, 2014 10:37 PM
Akhaumereallen
Its cool anyway!
April 24, 2014 11:47 PM
agudpp

Hi Slayemin, thanks for your comment.

Here are my comments (quoting yours in bold):

As it stands now, this article is not ready for publishing. It doesn't meet what I think are the standards for quality and depth.

It could be. Check my comments above.

I'm also very curious to see exactly how this is better than an optimized quad tree. What are the costs and gains (in memory cost, CPU performance, and programmer implementation time)?

Sure, I need to implement it before doing some benchs. I already implement grid spatial structure and I could get a very good performance for more than 8k units moving at the same time and getting the collisions for each one, higher performance than using quadtrees (obviously depends of a lot of factors). The only problem I had was the structure was a little memory expensive.

The nature of the quadtrees as well as all the trees have problems with memory locality (cache), not able to vectorize code, not random acces (log(n)), etc. Here the idea is have locality for cells, able to access to a cell in "amortized constant" time, not uses too much memory as grids.

Also you have to take into account different kind of queries and the performance of those (not only moving objects).

About programmer implementation time, after I finish my implementation (using my current idea) it should be very easy.

Quad trees are easy to understand, work well with a tree-like data structure, can be implemented within a day, and have a sexy Log base 4 runtime.

I agree with you. This is not as difficult as you may think, but probably a little more difficult than trees. Note that this is also a recursive data structure. Anyway as I said in the introduction: "The main objective of this data structure is to optimize the performance and memory usage in comparison with the normal grid space partitions."

They also work very well with moving objects, where you can either rebuild the tree every frame, or with a little more work, do an in-place update of a branch.

I'm assuming the reader knows how is implemented a grid, is fairly trivial too.

You're right to note that this might be complexifying a problem, and I do level that claim against you. I suspect that this MGSP approach is just a pre-mature optimization on the quad tree, where the performance gains are so slight that they're inconsequential, and the true burden of cost is in programmer time. You really need to give hard data and performance metrics to show that the cost vs. benefit is favorable.

I'm not agree with this. Is a optimization of the grid data structure. Quad-trees are quad trees, where each cell is divided into 4... This is a grid (MxN cells, you choose M and N), and each of those cells could be divided into M¹xN¹, M²xN²... etc or not divided at all (recursively).

I agree, as the comment above, I should put numbers, tables and performance metrics :). But the objective of this article is only to show an idea (you can consider the github just like the implementation of an idea), not an implementation nor a revolutionary data structure. As I said in the article, a trivial idea just to take into account when you use grid space partition data structure. Probably the title should be: "An idea: Multilevel Grid Spacial Partition."

Obviously the implementation of this data structure is more difficult than a quad-tree, but the idea is to have an implementation of it (that's why the github) so you can use it and not reinvent the wheel :). About the performance vs complexity trade of, I will tell you after I do the benchmarks :). You have to take into account, that this is intended for particular problems and not a general data structure that will work fine in almost all the projects (like probably quad-trees).

The limitation for supporting only stationary objects is also quite a big limitation to impose!

Games usually have moving objects in them which need to be tested for collision. If I imagine a quake3 game with rocket launchers and flying rockets and moving players, how would using a static grid data structure be advantageous over other choices?

I don't understand this. This data structure support moving and static objects...

Also, note that this (as I said) is intended for particular problems, and when you are thinking in use a flat grid space partition data structure.

April 25, 2014 12:25 AM
serumas

"The main objective of this data structure is to optimize the performance and memory usage in comparison with the normal grid space partitions."

I think the main objective of article is wrong. Lats say we have 30x30 cells , so we must to visit 900 cells and as author says half of tham are empty, so its perhaps max 0.010 ms job for pc. So we are trying to save this tiny time?

"The main problem with this data structure is that it's good only when the elements are uniformly distributed in space."

Its a problem for maybe all broadphase algorithms.

After lots of months spent on broadphase algorithms, mainly quadtree and grid, I would say that most memory and performance penalty goes to objects data structure. For example in my grid 10k circle objects (very tiny data structure) broadphase tooks ~0.6 ms, in more complicated data structure it tooks twice time.

So you cant compare your results with oter peaple implementation results, until you recreate exact physical scene. So I can compare just my implementations, my optimized grid 5 times faster than my not fully optimized quadtree, but I dont believe that even optimized quadtree can beat grid becouse of multilayering, reqursion, harder insertion, harder iteration... smile.png

April 25, 2014 05:59 AM
agudpp

Hi serumas, thanks for the comments

I think the main objective of article is wrong. Lats say we have 30x30 cells , so we must to visit 900 cells and as author says half of tham are empty, so its perhaps max 0.010 ms job for pc. So we are trying to save this tiny time?

Yes, you can visit all cells, or not, I will not recommend that at all, is very inefficient. Something like this will be better:

for each object

mgsp.getCollisions(object, collidingNeighbors);

// do what you want with this collisions

Note that this way we will only check the cells where the object belongs (very fast) and if some cell is empty it will be never analyzed :).

There are a lot of other sophisticated methods to perform collisions and avoid checking it twice, but this article is not intended for broadphase algorithms.

Its a problem for maybe all broadphase algorithms.

After lots of months spent on broadphase algorithms, mainly quadtree and grid, I would say that most memory and performance penalty goes to objects data structure. For example in my grid 10k circle objects (very tiny data structure) broadphase tooks ~0.6 ms, in more complicated data structure it tooks twice time.

So you cant compare your results with oter peaple implementation results, until you recreate exact physical scene. So I can compare just my implementations, my optimized grid 5 times faster than my not fully optimized quadtree, but I dont believe that even optimized quadtree can beat grid becouse of multilayering, reqursion, harder insertion, harder iteration... smile.png

What I mean with "is a problem":

1) We are wasting unused cells => poor performance. And almost all the levels in games will contain static objects (not overlapping) that can be known before the game starts.

2) To get better performance (more cell divs) we will need a LOT of more memory, increasing also the unused grid cells (those cells where are static and not overlapping objects).

The Multilevel Grid is an idea/approach to have the same number of cells than in a normal grid, but trying to use it all of them in spaces where objects can really move, not in places (like I said, rivers, buildings, etc) where units cannot move.

The images are not very convince because I use little subdivisions, this for sure was played against me since it seems that the problem was not obvious at all.

I agree that grids are faster than quadtrees for high volume of objects, but are also much more memory expensive and use a "constant" amount of memory during all its life.

I hope this clarifies you the problem of flat grids, and the intention of this data structure.

April 25, 2014 11:35 AM
serumas

for each object

mgsp.getCollisions(object, collidingNeighbors);

// do what you want with this collisions

Note that this way we will only check the cells where the object belongs (very fast)

I dont agreee with this. Yeah its the worst scenario to find collisions. Ill suggest to change a grid concept a little bit and you can get much better performance becouse such implementation is doomed for poor performance. Good luck

April 25, 2014 07:19 PM
agudpp

There are other options to perform better, like having a list of dirty cell ids and perform over those only. Perform only over the objects that had moved, or depending what you need.

If you need to get all the collisions that's one possibility.

I don't think there are one correct and better way, is problem-dependent.

Regards

April 25, 2014 07:27 PM
serumas

I mean

(slower)

for each object

mgsp.getCollisions(object, collidingNeighbors);

vs

(faster)

broadphase.getAllPossibleColisionPairs(pairs);

for each pair

doCollision(pair.a, pair.b);

April 25, 2014 08:33 PM
You must log in to join the conversation.
Don't have a GameDev.net account? Sign up!

This data structure is a generalization of the grid space partition data structure. The main idea is: improve memory usage having the same or greater performance than when we use a normal (flat) grid space partition approach.

Advertisement

Other Tutorials by agudpp

agudpp has not posted any other tutorials. Encourage them to write more!
Advertisement