So, maxgpgpu, let me get this straight. You have the most powerful hardware on the planet available to you - your GPU - but yet you're flat-out refusing to use it for the kind of processing it's best at. You have well-known, tried-and-tested solutions for collision and bboxes that have been proven to work for over a decade, but yet you're also flat-out refusing to use them. You have a VBO solution that involves needing to re-up verts to arbitrary positions in the buffer in a non-predictable manner - congragulations, you've just re-invented the worst case scenario for VBO usage - you really should profile for pipeline stalls sometime real soon.

None of this is theoretical fairyland stuff. This is all Real, this is all used in Real programs that Real people use every hour of every day of every week. You, on the other hand, have a whole bunch of theoretical fairyland stuff. It's not the case that you're a visionary who's ideas are too radical for the conservative majority to accept. It is the case that your ideas are insane.

Am I missing anything here?

Look, I don't know everything. I might be making mistakes, maybe serious ones. However, I do have reasons for every choice I make. Good reasons? Hopefully most of the time. But maybe not, and I'm totally open to that, and my reason for posting this thread was to scare up ideas that help me brainstorm. I can say one thing, in usual cases, the engine is plenty fast in normal cases. As I explained in this thread, for those cases, my approach is inherently GPU limited (or more exactly, not CPU or GPU limited).

And hey, I love having the GPU do instancing. I never criticized instancing. I know, because I never would. I also love having the GPU perform bumpmapping and parallax-mapping via relaxed cone-step mapping with self-shadowing to generate tons of fine details without need to have zillions of micro-triangles.

You talk about pipeline stalls like I never thought about that issue (one of many issues that interact, unfortunately). Yet my approach draws many objects with unlimited vertices with a single glDrawElements() call. And *****I***** don't care about pipeline stalls? While the conventional way only draws one object per draw call because it needs to change transformation matrices (and sometimes other state) between objects. To be honest, if I have one worry about my approach, it is that I pay too much attention to avoiding pipeline stalls.

[quote name='maxgpgpu' timestamp='1339696164' post='4949215']
See, this is the problem. And frankly, I can't blame you, because --- as I'm sure you're aware --- once any of us adopts a significantly non-standard approach, that radically changes the nature of many other interactions and tradeoffs. Since you and virtually everyone else has so thoroughly internalized the tradeoffs and connections that exist in more conventional approaches, you don't (because you can't, without extraordinary time and effort), see how the change I propose impacts everything else.

No, it doesn't take that much effort to evaluate other approaches -- there is no "conventional" apprach; we're always looking for alternatives. I've used methods like this on the Wii, where optimised CPU transform routines were better than it's GPU transforms (and also on the PS3, where it's got way more CPU power than GPU power), but without knowing the details of the rest of those specific projects, we can't judge here the cost/value of those decisions.[/quote]
Wow! First of all, thanks for your fabulous post. And let me correct myself and say that some people responding, but not you, appear to not be taking the time to consider the consequences of the approach I propose.

The problem is that your alternative isn't as great or original as you think it is.[/quote]
If I thought it was, I'd probably decide to keep it proprietary for as long as I could. In fact, I always new it adopted some "retrograde" elements. I think that's what sets some people off before they think everything through. I don't say that because "I'm necessarily right" about anything. I say that because of the rude and thoughtless way some people respond.

However, if we consider two extremes -- complete CPU vertex processing vs entirely GPU driven animation/instancing of vertices -- then you just have to consider that there is a continuous and interesting middle ground between the two extremes at all times. Depending on your target platforms, you find different kinds of scenes/objects are faster one way or the other. Different games also have different needs for the CPU/GPU of their own -- some need all the CPU time they can get, and some leave the CPU idle as the GPU bottle-necks. Our job should be to always consider this entire spectrum, depending on the current situation/requirements.[/quote]
I agree with you. I certainly hope nobody was thinking I want to return to the days of a frame buffer in CPU memory, and everything done bo the CPU! Ouch! When I started this thread, I tried to limit the context to next-generation CPUs and next-generation GPUs on a PC, with the minimum bar set at an 8-core CPU (say, FX8150) and GTX-580 ~ GTX-680 GPU. Of course this context tends to work against my proposed approach given the spectrum of possibilites you mention. I also tried to limit the context to a general-purpose game/simulation engine, which means it is stuck trying to efficiently handle a huge spectrum of games and simulations with a huge spectrum of requirements.

One reason for the approach I mentioned is my observation that most games and simulations contain 99% to 99.99% fixed objects (meaning, they do not rotate or move during a given frame). Here I'm not counting instanced objects like leaves and blades-of-grass blowing in the wind, which clearly are best handled with GPU instancing support. On these non-moving objects, I believe my approach is faster. This forced me to include separate support for "large moving objects", which are handled in the more conventional way (local-coordinates in GPU). I've mentioned all these things, but it doesn't seem to matter. It seems I offended the mainstream religion.

I understand why this seems counterintuitive. Once you adopt the "conventional way" (meaning, you store local-coordinates in GPU memory), you are necessarily stuck changing state between every object. You cannot avoid it. At the very least you must change the transformation matrix to transform that object from its local coordinates to screen coordinates.[/quote]
No, this is utter bullshit.
You haven't even understood "conventional" methods yet, and you're justifying your clever avoidance of them for these straw-man reasons? Decent multi-key sorting plus the simple state-cache on page#1 removes the majority of state changes, and is easy to implement. I use a different approach, but it still just boils down to sorting and filtering, which can be optimised greatly if you care.[/quote]
Okay, maybe I don't correctly understand what is conventional to you or others here. I don't entirely understand what you're saying here, but does what-you-say eliminate the need to supply a separate transformation matrix to the GPU for every object, or are you putting a whole slew of transformation matrices in a uniform buffer object (or texture, or something)... and somehow arranging things so the shader programs know which matrix to access for each vertex? If so, that's one way to address the problem my approach also addresses.

Moreover, there's no reason that you have to have "the transform matrix" -- it's not the fixed function pipeline any more. Many static objects don't have one, and many dynamic objects have 100 of them. Objects that use a shared storage tehcnique between them can be batched together. There's no reason I couldn't take a similar approach to your all-verts-together technique, but instead store all-transforms-together.[/quote]
That seems wise. Is that conventional practice now? Of course that still leaves some of us with the difference of opinion about where collision-detection and collision-response/physics, where many/most/all people here insist physics must be done without vertices (much less in a consistent inertial coordinate-system like world-coordinates).

However, your argument more-or-less presumes lots of state-changes, and correctly states that with modern GPUs and modern drivers, it is a fools game to attempt to predict much about how to optimize for state changes.

However, it is still true that a scheme that HAS [almost] NO STATE CHANGES is still significantly ahead of those that do. Unfortunately, it is a fools game to attempt to predict how much far ahead for every combination of GPU and driver!

I don't agree with how you've characterised that. I meant to imply that state-change costs are unintuitive, but that they are still something that can be measured. If it can be measured, it can be optimised. If it can't be measured, then your optimisations are just voodoo.

Also, just because you've got 0 state changes, that in no way grants automatically better performance. Perhaps the 1000 state-changes that you avoided wouldn't have been a bottle-neck anyway; they might have been free in the GPU pipeline and only taken 0.1ms of CPU time to submit? Maybe the optimisation you've made to avoid them is 1000x slower? Have you measured both techniques with a set of different scenes?[/quote]
Yeah, I did bencharms to test some alternatives, though I'm sure you understand there are many cases and "the answer" never applies to all of them. And furthermore, I hadn't implemented or handled every feature at the start, so these benchmarks could only be guidelines. One benchmark I performed was to figure out at what point to consider objects "large moving objects" (how large and how-often moving [recently]), which makes them get handled differently (local-coordinate in GPU). Will that point change over time? Most certainly. Will it vanish? That's more an architecture question, which is what I was trying to stimulate.

Certain facts simply exist. A modern GPU has about 512~1024 cores. If the engine is rendering a bunch of small objects, or far-away objects (meaning "few vertices" rendered per object), and a new transformation matrix gets uploaded between each object, then each object might not even keep all 512 cores busy for even an instant. That's gotta put a crimp in bandwidth. The GPU finishes the object in "no time", then all 512 cores are stalled while the CPU uploads a new transformation matrix [and maybe other state]. That's precisely what doesn't happen in the alternate approach, where dozens to hundreds of objects get rendered by a single glDrawElements(). However, if you have alternate ways to assure the GPU can keep on trucking without [many] breaks, that's great. Maybe I haven't given enough attention to figuring out those alternative ways, but I don't see what that should stimulate so much vitriol.

I really don't know what you mean when you say BattleField 3 instances everything. The only thing I can figure is... there are ZERO objects that only exist once. If you have a [Soldier0] object, then you have several instances of [Soldier0]... only with the kind of variations from one displayed-instance to another that instancing provides. If that's what you mean, then I understand what you mean. But that sounds like a mighty strange game! Having said that, I can certainly imagine lots of games where a large majority of displayed objects are instanced... blades of grass, trees, leaves, bricks, stepping stones, and so forth. Maybe you don't mean "everything" literally.

Maybe you don't want to waste VBO memory on storing a unique mesh for every "Soldier" in the game, so you only store a single (local space) "base mesh" that's kind of an average soldier. Your art team still models 8 unique soldiers that the game requires, as usual, but they are remapped as deltas from the base mesh. You then store your 1 base-mesh and 8 delta/morph-meshes together, and then instance 100 characters using the 1 base-mesh, each of which will be drawn as one of the 8 variations as required/indexed.[/quote]

I never said "no instancing". Where does anyone get that idea? I love instancing! I just never heard of a game or simulation where there were literally no unique objects. But hey, that's just little old me!

In your system, we not only need the original 9 data sets, but also enough memory for the 100 transformed data-sets. We then need enough GPU memory for another ~200 (n-hundred) transformed meshes, because it's dynamic data and the driver is going to want to ~double buffer (n-buffer) it.

I really don't know why I bothered with this, as you're being very dismissive of the real criticism that you solicited, but again, look at your memory trade-offs... The "conventional" requirements look like:
[attachment=9490:1.png]
whereas yours looks like:
[attachment=9491:2.png]

i.e. conventional memory usage
= Models.Verts * sizeof(Vert) //local data
+ Instances * Model.Bones * sizeof(float4x4) //transforms

your memory usage
= Models.Verts * sizeof(vert) //local data
+ Instances * Model.Bones * sizeof(float4x4) //transforms
+ Instances * Models.Verts * sizeof(vert) //world data
+ Instances * Models.Verts * sizeof(vert) * N // Driver dynamic VBO buffering.[/quote]
I'm sorry you read someone else's false claim that my approach prohibits instancing. If you read my posts, you shouldn't get that impression. You would also find out that I render "large moving objects" by putting local-coordinates in the VBOs, because benchmarks (and common sense) show doing so is more efficient. Nonetheless, I'm glad you ran the numbers, because they are useful to stare at and consider.

If you're going next-gen, you're looking at huge object counts and huge vertex counts. In the first equation above, there is no relation between the two -- increasing instances does not impact vertex storage, and increasing vertices does not increase the per-instance storage cost. However, in the latter, they are connected -- increasing the amount of detail in a model also increases your per-instance memory costs. Seeing as modern games are getting more and more dynamic (where more things are moving every frame) and more and more detailed (where both vertex and instance counts are increasing greatly), I'd give your users the option to keep their costs independent of each other.[/quote]
This is indeed the kind of consideration that I think about, especially how a general-purpose engine can (automatically if possible) do the efficient thing with every object. To be sure, objects that exist many times are more efficient with GPU instancing. Hell, even two instances is justification if they contain lots of vertices.

e.g. the example I gave earlier -- of 100 animated characters, with 100 bones each, using a 1M vertex model -- clearly is much more efficient in the "conventional" method than in yours. So if you wanted to support games based around animated characters, you might want to support the option of a different rendering mode for them.
[/quote]
Yes, I need to keep my eyes on these situations. If they're "large moving objects" or "obviously instancable objects", then I already have special cases for them. What I don't have, but you made me think more about, is in-between cases like "making all (or most of) your soldiers (or other object) from a single template, and performing [potentially-tricky] instancing on them. If the environment won't fit in GPU memory, then clearly some forms of instancing approaches must be dreamed up to squeeze more into less GPU memory space. I haven't run into that yet, but that doesn't mean I won't.

One useful thing I learned in this thread is how much of an outlier I am when it comes to what I perform collision-detection and collision-response AKA physics upon. In most cases, I perform both processes on the full set of vertices with unique positions. Some people don't perform collision-detection and/or collision-response at all (because their game doesn't require it). Others perform collision-detection and/or collision-response on an AABB or OOBB, because (I guess), that's good enough for their game/application - or because they need their game/application to run on slower hardware and have no choice. Others define a separate vertex cloud strictly for collision-detection and/or collision-reponse, which they process on the CPU only "as needed" (when their AABB [approximations] overlap in broad-phase tests). Others (not here, but in my reading) have the GPU feedback world-coordinates of all objects back to CPU memory (or specify a feedback shader for only those objects that moved).

That's quite a wide range of approaches. Clearly my approach is an outlier compared to the people who posted in this thread, in that I perform collision-detection and collision-response on all the unique vertices in every object --- when called for. I mean, my broad-phase looks for overlap of object AABBs and doesn't look at individual vertices (though 3 lines of assembly-language update the AABB of the object it is part of, every time the object is transformed (because it rotated or moved)). But the highly optimized GJK algorithm that powers my narrow-phase collision-detection routine does walk the full set of unique vertices in both objects to detect intersections (or find closest features when no intersection is detected). Making this decision then became an important part of me deciding to transform rotated/moved objects on the CPU... so I'd have world-coordinates in CPU memory to perform collision-detection and also collision-response. Clearly I agree that IF I was satisfied with CPU transformed AABB test, or CPU transformed OOBB tests, or CPU transformed simplfied object tests... I might have made different choices.

Maybe I'm looking too far ahead. Or maybe the criticism of my choice to perform more detailed/accurate collision-detection [and physics] is correct. I mean, hey! I certainly understand that. I created two commercial video game engines on contract several years ago, and I was forced by the realities of throughput to adopt crude collision-detection strategies. In one the artists had to define a few extremity points that were most likely to intersect other objects. The purist in me hated that... but I could find no way out consistent with the performance I had available. The other engine was a bit better, but nowhere near what I'm planning now.

Who knows, maybe I overshot the mark. I put a lot of effort into writing the fastest possible broad-phase, GJK narrow-phase, and even a final phase for arbitrarily irregular "concave" objects (that only executes if the GJK hulls overlap, and only on object known to be concave (or not known to be convex)). Maybe all that effort was wasted, because that won't be practical on top-end CPU/GPU combinations for another 10 years.

I hope that's not the case. So far, it doesn't appear to be the case, even in tests with many simultaneously rotating and moving moderately high-vertex-count objects. I don't have physics done yet, so I just paint the individual intersecting triangles green as they pass through each other. The frame doesn't slow down until my test gets "rather absurd". Though a room chock full of bouncing socker balls might slow things down eventually when I'm forcing it to always run the concave-object phase too (whether the object is concave or not). Even so, I have more room to improve the concave algorithm based on "closest features" hints from the GJK routine on the same object pair the previous frame. All I need is more time!

[quote name='maxgpgpu' timestamp='1339828421' post='4949721']
with unlimited vertices

ah.. now I know who you really are
[/quote]
Right. I sell RAM to GPU makers! :-)

Okay, unlimited except by available GPU memory and 32-bit indices... though I admit to implementing all batches with 16-bit indices except when a single object has more than 65536 vertices.

[quote name='maxgpgpu' timestamp='1339836940' post='4949737']
Okay, unlimited except by available GPU memory and 32-bit indices... though I admit to implementing all batches with 16-bit indices except when a single object has more than 65536 vertices.

whatever mate.
My suggestion to you would be to stop writing walls of text on the forums and use the energy to actually profile your code, that'll show you how wrong you are... or, if you really feel you're right, I reiterate the request for a video that showcase what this silly engine of yours is capable of doing... until then, it's all air.
[/quote]
Look, I'm not here to impress anyone. That's not my purpose. But if you want to propose a specific simple situation, I'll try to generate it and see what happens. The engine is optimized for "procedurally generated content", which means objects assembled from full or partial primitive shapes like [partial] disks, [tapered] tubes, [truncated] cones, etc. So if you specify something that's made up of objects that can be built out of such primitives, that's fine. I can import 3DS too, though I haven't tested it thoroughly so I don't know what percentage of 3DS objects it will puke on. As I also said, physics is not done, but collision-detection is (broad-phase, convex-narrow-phase (GJK) and concave-narrow-phase). If you want this to provide honest opinions, I'm fine. If you just want another reason to trash someone, go find someone else.

Certain facts simply exist. A modern GPU has about 512~1024 cores. If the engine is rendering a bunch of small objects, or far-away objects (meaning "few vertices" rendered per object), and a new transformation matrix gets uploaded between each object, then each object might not even keep all 512 cores busy for even an instant. That's gotta put a crimp in bandwidth. The GPU finishes the object in "no time", then all 512 cores are stalled while the CPU uploads a new transformation matrix [and maybe other state]. That's precisely what doesn't happen in the alternate approach, where dozens to hundreds of objects get rendered by a single glDrawElements(). However, if you have alternate ways to assure the GPU can keep on trucking without [many] breaks, that's great. Maybe I haven't given enough attention to figuring out those alternative ways, but I don't see what that should stimulate so much vitriol.

The problem is your 'facts' are wrong.
Utterly.

As we are talking about next generation GPUs lets take one I know about; the AMD Southern Islands aka Radeon HD7000 Series.
A 7970, the card I have, has 2048 'cores' however when you issue a draw call the GPU doesn't go 'all 2048 threads, do this!' instead it operates on wavefronts (NV calls them 'warps', the general term is 'work groups').

AMD breaks the SI core up into 'compute units'.
Each compute unit has for SIMD units in it.
Each SIMD unit can deal with upto 10 wave fronts
Wave fronts are made up of 64 work items.

So each compute unit can have upto 2560 work items in flight. (And if I'm reading my clinfo output correctly the HD7970 has 32 such units)

But, each of these don't have to come from the same draw call nor do they have to be the same shader as each wave can have its own instruction buffer and data.

Now add in what Hodgman said about the async nature of the GPU and CPU (by default I believe the drivers can buffer up to 3 frames ahead) and you can see that your outline above simply isn't true - you don't issue a call which takes up the whole device and then waits, you build a command stream which can take up sections of the device with threads being swapped in and out as resources demand it.