Hello,

I just read the following article which I believe has been mentioned here before as well.

http://realtimecollisiondetection.net/blog/?p=86

As I have a fair understanding of the DirectX 11 API for most of my needs and a somewhat fair understanding of OOP I want to start

designing a render loop (short of calling it game loop since there's no AI or physx yet) around a more scale-able architecture that could ultimately grow into a game... That means separating responsibilities in different classes and decoupling what can be, and ideally have a render class rather than have objects draw themselves.

Mind you I'm still at rastertek tutorial level of understanding LOL and it's mostly how to connect all the dots that's difficult more so than each separate topic such as shadows, skyboxes, reflections, render to texture or what-have-you.

It's how games elegantly add creatures, remove them, iterate through them, draw them in the most optimal order etc that seems to be the crux of videogame development imho. I believe this article pretty much hits the nail on the head and draw call sorting might make everything fall into place...

Now initially that sounded weird... sorting "function calls" ?!? lolwut? but of course it's the data that has to be rendered that's sorted by renderstate. Fine. Now in the article they mention "using standard sorting algorithms" and "key/value" pairs. That to me screams std::map but is this what is usually used by professionals? I tried a bit of dummy example code with std::map to see how fast it is and it appeared that even iterating through a couple of entries on my i5 laptop with 256 kb of cache takes a couple of milliseconds, an eternity when you've got only 16 ms per frame and you are supposed to iterate 2000 draw calls. I admit I haven't tried to actually sort 2000 entries as I had to come up with 2000 things to fill it with but could it be that it won't take much longer as some of this time was overhead? Or is std::map just not the right solution? What I liked about it is that it's actually automatically sorted but maybe some unordered map followed by a sort is better? (then again my measurement was done for iterating only, after I had inserted values so the timing would even be worse otherwise)

Maybe my measuring tool wasn't appropriate? I used "time.h" clock() like so:

t=clock()

//iterate std::map with barely 6 key/value pairs in it

t=clock()-t;

cout << t <<endl;

I know there are higher resolution timers out there but since we're talking milliseconds here I guess I'd measure the same. Now if it's simply an inaccurate tool I'll reconsider...

Don't use std::map, use std::vector, and store the key in the render instance you submit. Then you can write a custom sort function:

std::stable_sort(m_vInstances.begin(), m_vInstances.end(), QueueSorter());

		struct QueueSorter
		{
			inline bool operator()(const RenderInstance* pLeft, const RenderInstance* pRight)
			{
				return pLeft->sortKey.bits > pRight->sortKey.bits; 
			}
		};

Make sure you put the sorting function in the header and as a functor struct, that way its likely going to be inlined and thus way faster.

I'm not sure I understand this answer. Functors? Is that some C++11 construct? I guess I have some more reading to do...

Vectors don't have the notion of key/value and I also don't get what you mean by "and store the key in the render instance you submit"

Functor is simply that what my "QueueSorter" is. A struct with a ()-operator, that is called instead of a "QueueSort"-function, since the compiler most likely won't be able to inline the latter.

Well, just look at what the () operator does. It takes two RenderInstances, and compare their keys. As I said, store them inside whatever structure you would store in the map. Like this:

		/*******************************************
		* Render instance
		********************************************/
		struct RenderInstance
		{
			Key64 sortKey;

			const DrawCall* pCall;
			const StateGroup** pStateGroups;
		};

The realmtimecollisiondetection-article doesn't explain that bit in too great detail, I suggest you have a close read at this topic: http://www.gamedev.net/topic/604899-frostbite-rendering-architecture-question/

OK thanks. I think I got the functor now after some googling. Weird they didn't mention this in learncpp's chapter on operator() overloading.

I'll try to see where this can take me.

Sorting data is a huge way to get gains in performance. This is due to a number of effects, the chief ones being caching and driver or state switching overhead. When you sort data, you're more likely to hit the instruction cache since you're probably going to be doing similar operations for a large group of data at once. You may also get data cache hit rate improvements as well.

With respect to graphics in particular, the GPU is very intolerant of state switching. Its entire performance potential is contingent on not having special cases! Whether this means branching or switching off entire buffers/textures, it doesn't really matter, they all have pretty significant costs on execution speed. The reasons for this are due to GPU architectures which I'm not very well versed in, but if you have a basic understanding of CPU architectures and pipelines, and extrapolate that to hundreds of simple execution units, then you can get an idea of why the GPU is so intolerant of handling "uncommon cases" (branching, state switching, etc).

The article you refer to touches upon many different ideas used by people to improve the performance of game engines; more specifically, rendering speed. Suppose you weren't sorting your draw calls and instead you just drew every item in your game individually and kicked off a draw call for each item. Since your items may not be sorted, you may have to set up state just to draw the item. When you finish with that, you end up going to the next item which could be totally different and requires a different set of state to be enabled. This is very costly.

Remember, the GPU wants to be fed HUGE batches of data at once for it to munch on. It's completely against what the GPU wants for you to process data and feed it in this manner. When you sort the data/state, you can give it larger batches for it to draw. Because you're switching state much less often and you're giving it larger amounts of data to process on each draw, it should lead to higher performance during rendering.

Additionally, it may be required for you to sort to even get a correct result. Transparency requires you to draw back to front. If you violate this, you get an image that does not look correct. There are many ways people solve this problem, but they pretty much all revolve around doing some kind of sort to get the primitives fed to the GPU in the right order for alpha blending.

Now, to your main question:

Do people use std::map for sorting? Almost never. Not if you care about performance. STL map is incredibly slow. Its slowness can be attributed to the nature of the data structure: pointer based tree. Almost certainly, the STL is allocating a new node on the heap for every single element. You have no real guarantees on where that allocation is in the memory space. This leads to poor cache behavior. Additionally, each node contains a lot of extra linkage information just to make the tree work, so it's pretty bad in memory use as well.

If you're only interested the data and its sorted version, there is no reason for you to use the map. Just throw it into an array and then sort the array. This will be significantly faster. Your example of 2000 items is just far too small to be able to see any difference given the resolution of your timer.

Here's an example piece of code I wrote to illustrate (requires C++11 support, I use g++ 4.8.0 to compile):

#include <vector>
#include <map>
#include <algorithm>
#include <chrono>
#include <random>
#include <utility>
#include <cstddef>
#include <cstdint>
#include <cstdio>

using namespace std;

class Clock
{
public:
    Clock()
        : m_constructTime(GetTime())
    {
    }

    static double GetTime()
    {
        TPSeconds now(HighResClock::now());

        return now.time_since_epoch().count();
    }

    double GetTimeSinceConstruction() const
    {
        return GetTime() - m_constructTime;
    }

private:
    typedef std::chrono::high_resolution_clock HighResClock;
    typedef std::chrono::duration<double, std::ratio<1>> Seconds;
    typedef std::chrono::time_point<HighResClock, Seconds> TPSeconds;

    double m_constructTime;
};

int main()
{
    const size_t N = 5000000;
    mt19937 rng(0);
    vector<pair<uint32_t, size_t>> randomData;
    randomData.reserve(N);

    // Generate random data.
    {
        Clock clock;

        for (size_t i = 0; i < N; ++i)
        {
            randomData.emplace_back(rng(), i);
        }

        printf("Random data generation took %f seconds!\n", clock.GetTimeSinceConstruction());
    }

    // Insert random data into a map.
    {
        multimap<uint32_t, size_t> data;
        Clock clock;

        for (auto iter = randomData.cbegin(); iter != randomData.cend(); ++iter)
        {
            data.insert(*iter);
        }

        printf("Map sort time: %f seconds.\n", clock.GetTimeSinceConstruction());

        double start = Clock::GetTime();
        int sum = 0;

        for (auto iter = data.cbegin(); iter != data.cend(); ++iter)
        {
            sum += iter->second;
        }

        double end = Clock::GetTime();
        printf("Map iteration time: %f seconds, sum %d.\n", end - start, sum);
    }

    // Sort our random data.
    {
        Clock clock;

        sort(randomData.begin(), randomData.end());
        printf("Vector sort time: %f seconds.\n", clock.GetTimeSinceConstruction());

        double start = Clock::GetTime();
        int sum = 0;

        for (auto iter = randomData.cbegin(); iter != randomData.cend(); ++iter)
        {
            sum += iter->second;
        }

        double end = Clock::GetTime();
        printf("Vector iteration time: %f seconds, sum %d.\n", end - start, sum);
    }

    return 0;
}

Compiled with:

g++ --std=c++11 sortspeed.cpp -O3 -o sortspeed

This code generates 5 million random pieces of data (just some unsigned ints) and then inserts them into a map and sorts an array version of the data. Time is measured for each version and printed out. The sum performed in the code is bogus. It's just there to give the iteration something to look at and forces the compiler to actually generate code that touches each element in the containers.

On my Intel Q9550 @ 2.83 ghz, I get the following output:

Random data generation took 0.096909 seconds!
Map sort time: 5.418803 seconds.
Map iteration time: 0.550815 seconds, sum 1642668640.
Vector sort time: 0.563086 seconds.
Vector iteration time: 0.013663 seconds, sum 1642668640.

As you can see, the map is significantly slower to the vector, both in terms of sorting speed and iteration speed, by a factor of 10 or more. You need to carefully determine your requirements for the algorithm you're trying to execute and use the correct data structures to maximize performance. Although a toy piece of code, this illustrates that certain structures may provide what you need, but they aren't always the minimum requirement. The map might be more suitable for a problem if it were necessary for you to insert/delete from the list many times. In games, especially for rendering, insertion and deletion is often not required. Many engines simply keep a memory block for queuing up draw requests and once the frame is finished, it resets a pointer back to the beginning of the list to effectively erase it in constant time. Then, the engine just requeues up all elements that need to be drawn again for the next frame. Because of the fact that the engine doesn't always know what needs to be drawn at any given time, it's often simpler and faster to just rebuild the list of drawables instead of trying to apply deltas to it (inserting/removing the elements that are viewable or not).

Edit: I decided to run some perf tools on the code to provide a little more context.

Map only:

Random data generation took 0.097099 seconds!
Map sort time: 5.419488 seconds.
Map iteration time: 0.553151 seconds, sum 1642668640.

 Performance counter stats for './sortspeed':

       6846.409743 task-clock                #    1.000 CPUs utilized          
       263,232,170 cache-references          #   38.448 M/sec                   [33.36%]
        81,311,169 cache-misses              #   30.890 % of all cache refs     [33.40%]
    19,219,819,177 cycles                    #    2.807 GHz                     [33.38%]
     3,390,728,460 instructions              #    0.18  insns per cycle         [50.03%]
       762,888,168 branches                  #  111.429 M/sec                   [49.96%]
        80,913,764 branch-misses             #   10.61% of all branches         [49.97%]

       6.849046381 seconds time elapsed

Vector only:

Random data generation took 0.097330 seconds!
Vector sort time: 0.595129 seconds.
Vector iteration time: 0.013621 seconds, sum 1642668640.

 Performance counter stats for './sortspeed':

        716.372024 task-clock                #    0.997 CPUs utilized          
        22,809,109 cache-references          #   31.840 M/sec                   [33.01%]
           849,197 cache-misses              #    3.723 % of all cache refs     [33.01%]
     1,845,627,135 cycles                    #    2.576 GHz                     [33.88%]
     1,508,337,419 instructions              #    0.82  insns per cycle         [50.53%]
       352,990,250 branches                  #  492.747 M/sec                   [50.24%]
        48,613,614 branch-misses             #   13.77% of all branches         [50.26%]

       0.718218847 seconds time elapsed

Notice the cache miss rate on each: 30.89 % (map) vs 3.723 % (vector).

Edit 2: I decided to run my code with 2000 elements.

Random data generation took 0.000035 seconds!
Map sort time: 0.000456 seconds.
Map iteration time: 0.000042 seconds, sum 1999000.
Vector sort time: 0.000149 seconds.
Vector iteration time: 0.000004 seconds, sum 1999000.

Your timer does not have enough resolution to be able to detect the difference. Again, 2000 items on a computer is very small. You need to be thinking about hundreds of thousands or millions for it to really matter to computers (unless your algorithm is very slow...).

The article you refer to touches upon many different ideas used by people to improve the performance of game engines; more specifically, rendering speed.

Just one comment on that. While it is generally true that state monitoring can improve performance, the methond mentioned in the article does not serve the main puprose of improving performance. The author himself notes that he is sacrificing (some) performance in advance of flexibility. Also, this method has nothing to do with state monitoring per se. In a primitive approach, or if you are too lazy like me to come up with a good solution, you will still set state after state after state without caring what the last state was. So at all, you will loose a litte performance - there is no way sorting and pre-processing draw calls is going to be faster than directly submitting them. With state monitoring, you pull workload from the gpu to the cpu. There, it might eventually pay out. Also you can easier use multithreading with such a queue (multiple cores submit render states to it, one core draws them afterwards).

So while it might actually lead to that, gaining performance is, once again, NOT the main purpose of this rendering architecture, just to make this clear. One of the main purposes is, from what I understand, to further decouple high and low level rendering, and adding some flexibility - it doesn't care now when you submit a render command. The key will tell when it needs to be drawn - be it the first insertion or the last. Without such a render queue, precautions have to be made that the opaque geometry gets drawn before the transparent, and the transparent needs to be sorted back to front. Now the queue takes care of this. That is also why it doesn't make sense to add a time delta to the submitted command. We don't sort by call time, we sort by predefined settings, determined in the sort key.

OK thanks all I think I've understood it now. And to Snowman thanks for the code and detailed explanation. This will prove very useful.