Gamedevs presenting language problems to C++ standards working group

Started by
25 comments, last by l0calh05t 9 years ago

Figured people here might be interested in this: Some game developers met up at cppcon 2014 and decided we wanted to try to address issues in C++ for use in game development. We're planning to present our findings to the C++ standards committee in Lenexa in the month of May.

This is a work in progress rough draft, so it would be great to get some feedback from game developers before we go and present this!
http://open-std.org/JTC1/SC22/WG21/docs/papers/2015/n4456.pdf

We are mostly building on the work set out by the EASTL paper, so this paper is also of interest:
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2007/n2271.html

Thanks!

Advertisement
Can you summarize the proposals/complaints for the lazy among us (cough cough, me) who don't want to read the whole PDF?

Wielder of the Sacred Wands
[Work - ArenaNet] [Epoch Language] [Scribblings]

From reading it, looks like a complaint with no solutions.

* "don't like global new/delete"

* Lack of alignment and debug allocation tools.

* C++11 thread pool latency

* Standard library does some frustrating things, see eastl, 13 bullet point restatement of that.

* Some new features may use RTTI but RTTI is usually disabled in games due to paid penalties

* vague complaint about disabling exceptions

* request for standard method to get vtable handle

* request for "flat_map" class (from boost?) to be standard

* Complaint about how implementations add safety features to standard library, especially in debug builds, want those off by default

* standard SIMD types

* Desire for static reflection (despite the fact that this is against the compilation model)

* Want an SoA/AoS translator that is layout-agnostic and includes SIMD types

Think that is it.

Unlike most other proposals, this is only problems but no solutions are offered.

Seems reasonable.

It's kinda hard to propose a fixes for a language that has to be compiled down to assembly.

So far... all the complaints seem perfectly valid... except for the SoA/AoS translator. To me that just seems like laziness.

Plus... isn't SIMD controlled by hardware companies? Usually you need a library for that. Or at least a compiler.

Thanks to other posters for helping summarize. smile.png

There are actually currently ongoing proposals for SIMD types and static reflection, so it's not as sci-fi as you may think. Mainly we're interested in making sure that those features are implemented in a way that make it possible to solve common game development problems. For example, great care has to be taken with alignment because C types only need alignment up to 8. If the language was aware of 16-aligned SIMD values, then that could become a default minimum alignment for things like new/malloc, which fixes many problems related to new and allocators and function calling and etc.

The SoA/AoS is just an idea for a use case of static reflection, and it happens to mesh well with SIMD. Some other languages make this possible (eg: ISPC), and the result is that programmers can easily experiment with different layouts to find optimal configurations.

Do note that N4456 is not a proposal, it's just a paper where we tried to summarize some ideas. Actual proposals or proofs of concept might come out of discusssions with this. flat_map is a candidate for making an actual proposal out of.

Didn't want to comment TOO early after going through that list, but here's my take on them:

Very few large applications (such as games) use the global new and delete, so I don't think it is a realistic complaint. Often they use OS-specialized allocators coupled with logging or debug-tacking systems. There are so many good debug memory libraries out there, each with their own specific set of solutions. Even boost has a large number of allocation libraries. Whatever general-purpose solution the standards committee were to implement, you are certain to find groups who are unhappy with it.

Those issues are not new. The EASTL paper addressed most of those issues already. The problem is one of specialization. The serious difficulty is that whenever you build a general-purpose library there will always be solutions that are a bad fit.

Boost does a lot of stuff with memory, but even all those variants are not a good fit universally. Sure it can handle alignment needs, but they offer allocation systems for interprocess allocators, pool allocators, small object allocators, adaptive pool allocators, synchronized allocators, and a few more; yet people still need more for custom needs. eastl did some minor modifications to various libraries but in order to implement many of them they ended up needing to customize their allocators as well. And eastl needed to work in a world without exceptions, meaning some small but high-impact changes to how the c++ standard library handles things.

The desire for a few classes to be implemented in the standard library, or for some utility functions to be in the standard library, I don't see those as a good fit. They are too special purpose. Better to throw them into a more mainstream -- but not universal -- library. A SoA/AoS translator, SIMD types, and the rest seem like they could be better vetted through boost rather than jumping directly to the language core.

And as for the vtable thing, I'm not particularly certain what you would do with the vtable handle since calling functions or specializations with the vtable is implementation defined.

The complaint about RTTI is real but very minor, since there is a small but measurable cost to both RTTI and C++ Exceptions just by their very existence. Unfortunately there are some edge-cases within C++ that require those solutions. There are not a lot of these problems, but even so, they exist. So asking the C++ Standards Committee to turn off the features won't happen since it breaks those corner-cases of the language. If the feature needs RTTI and you need a compiler option to disable RTTI, those features just won't be accessible.

The desire for a reflection of enumerations isn't new either, and a few suggestions have been made over the years. But the compilation model that dates back from the early 1960s is contrary to that mode of compilation. Enumerations are named constant values that vanish during compilation. Wanting source names to survive compilation is a common feature of JIT compilation, but not the C++ object file model where ultra-compact reductions are the goal. Microsoft managed to implement a good chunk of that in their C++/CLR compiler, but they spent a fortune doing it, it required a massive systems rewrite, they took a lot of flack for it and even with all that it still does not work universally. In a few places it is at directly cross purposes with the compilation model, so the result would be contrary to existing C++, making it a language fork.

As they say, the first step is recognizing that there's a problem.

Michael Wong (representative of the Canadian National Body on the committee) was taking input from various game developers back at CppCon to let us air our complaints. I then put together an unofficial mailing list for a handful of us to discuss out plans for a proper paper. The original efforts centered around analyzing the old EASTL paper, which we got through most of, but still need public benchmarks. Then Work(tm) happened and I got insanely busy and didn't follow through, but Michael, Nicholas, and others thankfully took up the mantle and got this initial paper put together.

The primary focus of the paper is mostly to raise awareness in the committee and to possibly help spark the formation of a proper sub-group. Such an SG would investigate these and related issues and try to get more involvement from game companies in the committee. We are some of the biggest and most important users of the language yet we have near no representation on the committee!

There's talk of possibly combining with the real-time and embedded development sub-group in the works, since our needs are pretty similar. If all goes well, we might see a new said group formed to help shepherd related papers and foster better community interaction.

In the meantime, though, yes we need to start writing up papers for real solutions. I've got a small stack of papers that are either half-written or in early draft forms that I didn't have the time or energy to finish off for the Lenexa committee meeting, but there's always next time. Hopefully others - maybe some of you fine forum goers - step up to bat and push even more proposals and fixes!

You can get involved with the committee at https://isocpp.org/. The site includes public forums frequented by a number of committee members and has information on how to submit a paper for the committee to review.

Sean Middleditch – Game Systems Engineer – Join my team!

I'm pretty new to this sort of thing (so I might be totally wrong), but it really does seem like most of those "problems without solutions", have largely the solution of "implement it"; i.e. not very complex solutions. It seems more like philosophy or opinion gets in the way more than any lack of "proposed solutions".

Many of the complaints in the paper are out of date( as the follow up comment often shows), why include that stuff?

Wanting the language to remove global new is abit silly

Regarding customizing vtables, I think typeclasses would be an improvement over the inheritance crap we have today for runtime polymorphism, so a proposal for that would be nice

I think the biggest improvement for games in c++17 would be modules, most of the rest of this stuff like flat_map we can already do ourselves(although it would be nice to standardize it)

Maybe we need to switch to Rust, it has almost none of these problems..


And as for the vtable thing, I'm not particularly certain what you would do with the vtable handle since calling functions or specializations with the vtable is implementation defined.


The discussion is here: https://groups.google.com/d/msg/unofficial-real-time-cxx/8oPFd52Rk10/kofd0p5FWwQJ

This topic is closed to new replies.

Advertisement