STL on PS3/Wii/360
Author
Couple questions...
-Does each of the new consoles have a robust and complete STL implementation?
-Are they architected to standard or follow some custom design considerations?
-Can I port between them without refactoring (excepting perhaps platform-specific allocators or something)?
Cheers
Generally, yes. It depends on the compiler you use though. The 360 will be using MSVC, which definitely has a full STL implementation. The Wii will be using CodeWarrior, which has a pretty good STL implementation.
You shouldn't even need custom allocators, since the default allocators will probably be using malloc(), which is handled by the SDK.
You shouldn't even need custom allocators, since the default allocators will probably be using malloc(), which is handled by the SDK.
It isn't just a matter of having STL support.
Probably the biggest thing to consider with console development is memory quotas. You don't get the luxury of swap space on these things. Even though using the STL is a great programming practice for PC apps, there are really good reasons to avoid the types of dynamic allocation that the STL does on consoles. There are even good reasons to avoid dynamic allocation completely on them, although that discussion (pros and cons) are covered in depth at other boards, like ps2dev and others. Not knowing in advance how much memory each of the infinitely many code paths will take is general a Really Bad Idea.
Also, any program being developed for these systems is going to have thousands of legacy files. They cover all the low level operations from controller input, disc streaming, to memory card access and the rendering pipeline. In order to work well with the existing libraries, you need to generally be using c-style everything.
Although the STL and other libraries *can* play well with these older libraries, see the earlier note about quotas. The other libraries generally have their own quotas that take up all or nearly all of the device's memory. The legacy system doesn't give much room for the STL to work with.
Although they exist and we use them on our tools, I don't know of console projects that use the STL container classes or STL streams inside their console projects.
Probably the biggest thing to consider with console development is memory quotas. You don't get the luxury of swap space on these things. Even though using the STL is a great programming practice for PC apps, there are really good reasons to avoid the types of dynamic allocation that the STL does on consoles. There are even good reasons to avoid dynamic allocation completely on them, although that discussion (pros and cons) are covered in depth at other boards, like ps2dev and others. Not knowing in advance how much memory each of the infinitely many code paths will take is general a Really Bad Idea.
Also, any program being developed for these systems is going to have thousands of legacy files. They cover all the low level operations from controller input, disc streaming, to memory card access and the rendering pipeline. In order to work well with the existing libraries, you need to generally be using c-style everything.
Although the STL and other libraries *can* play well with these older libraries, see the earlier note about quotas. The other libraries generally have their own quotas that take up all or nearly all of the device's memory. The legacy system doesn't give much room for the STL to work with.
Although they exist and we use them on our tools, I don't know of console projects that use the STL container classes or STL streams inside their console projects.
Quote:Original post by frob
Even though using the STL is a great programming practice for PC apps, there are really good reasons to avoid the types of dynamic allocation that the STL does on consoles.
Yes, there are certainly pitfalls. No, there is no reason to avoid the STL on console development if you know what you're doing. It boils down to a religious war. You can create custom allocators and whatnot so that the STL fits into whatever memory management bucket you could possibly design. Historically, some engineers have been burned in the past with the STL because they or someone on their team didn't really know how to use it. Going forward the people in their studio are allergic to the idea of STL in console apps because they spent many long nights fixing problems that could have easily been avoided.
Quote:Original post by frob
In order to work well with the existing libraries, you need to generally be using c-style everything.
Not sure what titles you're working on, but I've been doing console development for the last 4 years and haven't seen a single bit of main game code c-style anything. Everything in the AI and gameplay world in which i work is C++; our whole graphics pipe was also C++.
Quote:Original post by frob
Although they exist and we use them on our tools, I don't know of console projects that use the STL container classes or STL streams inside their console projects.
Heh. Again it breaks down along "religious" lines. I haven't seen a single console title that _doesn't_ use the STL. In the 3 titles i've shipped and in the one i'm about to ship we use STL with custom allocators.
-me
Actually, the previous 2 DS titles, and the current DS title I'm working on have all used the STL without the need for custom allocators (Although there's a custom malloc() allocator), and the DS only has 4MB of RAM. So long as you're careful what you do with the containers, there's no need to worry.
Quote:Not sure what titles you're working on, but I've been doing console development for the last 4 years and haven't seen a single bit of main game code c-style anything. Everything in the AI and gameplay world in which i work is C++; our whole graphics pipe was also C++.
I've been doing console developement for 3 years, all but 1 project I worked on used straight C (PC/XBOX/PS2/NGC/PSP titles), only 1 used C++ and that was only because we were porting code from another company who had originally created the game.
It really comes down to the company you work for, the programmer that origianlly created the engine here were total zellots of C over C++. Personally I would quite like to see that change when we create our next-gen engine but we'll see.
I haven't personally had trouble with STL algorithms, though the containers caused some issues. Even with custom allocators, we had some long term fragmentation problems - not bad enough to prevent shipping, but enough to cause a great deal of concern.
Allocation frequency was also an issue - worst case scenario, std::strings were getting created each frame. This caused a large number of critical section hits (we didn't have thread-local heaps).
The core problem is that stl containers made it a bit simpler to write careless code. Storing entities in a std::list/vector makes it very easy to get in the habit of thinking there is no upper bound. There are at least two; when you exceed the expected budget, and when malloc returns NULL/throws an exception. Unfortunately, only one of these is easy to detect without additional instrumentation.
There seem to be two different extreme paths - 'Fixed numbers/sizes of everything, no allocations after startup' vs 'Do all the allocations you want, trap the problem at a lower level (the allocator)'. In one case, people working on content have to deal with boundaries that can be changed. In the other case, its on them to find the balance themselves.
Personally, I prefer the version where I know fragmentation/out of memory conditions aren't possible. I'd rather make the road towards gold a bit smoother at the cost of slightly complicating development over the lifetime of the project.
Allocation frequency was also an issue - worst case scenario, std::strings were getting created each frame. This caused a large number of critical section hits (we didn't have thread-local heaps).
The core problem is that stl containers made it a bit simpler to write careless code. Storing entities in a std::list/vector makes it very easy to get in the habit of thinking there is no upper bound. There are at least two; when you exceed the expected budget, and when malloc returns NULL/throws an exception. Unfortunately, only one of these is easy to detect without additional instrumentation.
There seem to be two different extreme paths - 'Fixed numbers/sizes of everything, no allocations after startup' vs 'Do all the allocations you want, trap the problem at a lower level (the allocator)'. In one case, people working on content have to deal with boundaries that can be changed. In the other case, its on them to find the balance themselves.
Personally, I prefer the version where I know fragmentation/out of memory conditions aren't possible. I'd rather make the road towards gold a bit smoother at the cost of slightly complicating development over the lifetime of the project.
Unfortunatley, the STL allocator interface is totally inadequate - it's considered too limited even amongst the C++ standards committee, who are planning to replace them with a new policy based approach.
Some examples:
Q: When push_back() is called on a vector where size()==capacity(), how much memory is allocated?
A: Implementation defined. Can't be controlled by the allocator at all, even though it's one of the most important aspect of memory management with vectors. reserve() doesn't help either - it guarentees that the vector's capacity increases to be greater then or equal to the size requested (or has no effect).
Q: How big are the chunk used by std::deque?
A: Implementation defined. iso14882 doesn't even mention chunks. Again, this can't be controlled by allocators.
Q: How long can a std::string be before it uses dynamic memory allocation?
A: Implementation defined. Most implementations use this optimization, but it isn't required and the magic size isn't programmatically accessible
These three issues could all be fixed by using std like containers with policy class template parameters
Q: How much memory does a std::stringstream allocate when used to convert an int to a string?
A: Implementation defined.
This one can be fixed by using boost::lexical_cast and specializing it for int->string so that no memory is allocated
Some examples:
Q: When push_back() is called on a vector where size()==capacity(), how much memory is allocated?
A: Implementation defined. Can't be controlled by the allocator at all, even though it's one of the most important aspect of memory management with vectors. reserve() doesn't help either - it guarentees that the vector's capacity increases to be greater then or equal to the size requested (or has no effect).
Q: How big are the chunk used by std::deque?
A: Implementation defined. iso14882 doesn't even mention chunks. Again, this can't be controlled by allocators.
Q: How long can a std::string be before it uses dynamic memory allocation?
A: Implementation defined. Most implementations use this optimization, but it isn't required and the magic size isn't programmatically accessible
These three issues could all be fixed by using std like containers with policy class template parameters
Q: How much memory does a std::stringstream allocate when used to convert an int to a string?
A: Implementation defined.
This one can be fixed by using boost::lexical_cast and specializing it for int->string so that no memory is allocated
Quote:Original post by PalidineIn our games:Quote:Original post by frob
Although they exist and we use them on our tools, I don't know of console projects that use the STL container classes or STL streams inside their console projects.
Heh. Again it breaks down along "religious" lines. I haven't seen a single console title that _doesn't_ use the STL. In the 3 titles i've shipped and in the one i'm about to ship we use STL with custom allocators.
- we need to be 100% in control of memory
- we want code that is cache efficient (beyond just being algorithmically efficient)
- we want code that is straightforward and not overengineered
- we want code that you can read and step through
- we want code that gives clear error messages
- etc.
Some people may not care about one or more of these issues or have other needs, which is fine; that's their choice after all. However, none of the titles I've ever worked on (all console games, except one) has used STL for the aforementioned reasons. (This refers to STL in the game; we've used STL in the tools.) For us, our decision is commonsense, not religion, but to each his own.
Finally, I take objection to your claim that "No, there is no reason to avoid the STL on console development if you know what you're doing." That's you claiming that the team of programmers that I work with don't know what we're doing, and it's total flamebait. I won't rise to it, but I think the games we've done speak for themselves and I'll let others judge your statement in that light.
Just some general questions to shoot out there... It seems that many game developers (especially those that started out programming in the early 90s or earlier) are of the same mind - performance is #1, memory fragmentation is a huge problem, and don't touch exceptions because they're too slow, and we don't have the time to educate ourselves on how to use them effectively.
Why are game developers so worried about memory, but totally lack modern software engineering practices like exception-safety?
Why is there so much worry that someone might write a line of code that isn't absolutely as efficient as possible? Why aren't you instead concentrating on developer productivity and on writing flexible, maintainable code? Why are you prematurely optimizing everything, instead of using robust standard solutions that are guaranteed to work, and then properly profiling the code to find what actually IS slow?
What about using a compacting garbage collector ala .NET? All possible worries of memory fragmentation then go out the door, and objects allocated close in time never get farther apart in memory, but may get closer...
What about using the boost memory pool library? Or any of the boost libraries? Are you familiar with boost::bind and boost::function, and how much they can help to decouple code? Are you familiar with how C++0x will change the language, and what is being added to the standard library? How are you planning on taking advantage of the new features? Do you know what template 'concepts' are and how they can greatly improve core libraries and other templated functions? Many professional game developers have been using C++ for several years, and it has been standardized for 8 years now. How many people are still using character arrays and strcat(), instead of std::string and std::stringstream's?
Do you employ usage of smart pointers at your studio? Did you implement your own or did you use something conventional like boost::shared_ptr or boost::intrusive_ptr (for you cache nuts)? Do your smart pointers support custom deleters such that they can become a more general 'resource handle'?
How many times does someone static_cast a base class pointer to a derived class pointer based on some 'enumerated type' value stored in the base class? Why are you losing the type information, and not educating developers about type safety and good use of templates to write less-voodoo code?
Why are game developers so worried about memory, but totally lack modern software engineering practices like exception-safety?
Why is there so much worry that someone might write a line of code that isn't absolutely as efficient as possible? Why aren't you instead concentrating on developer productivity and on writing flexible, maintainable code? Why are you prematurely optimizing everything, instead of using robust standard solutions that are guaranteed to work, and then properly profiling the code to find what actually IS slow?
What about using a compacting garbage collector ala .NET? All possible worries of memory fragmentation then go out the door, and objects allocated close in time never get farther apart in memory, but may get closer...
What about using the boost memory pool library? Or any of the boost libraries? Are you familiar with boost::bind and boost::function, and how much they can help to decouple code? Are you familiar with how C++0x will change the language, and what is being added to the standard library? How are you planning on taking advantage of the new features? Do you know what template 'concepts' are and how they can greatly improve core libraries and other templated functions? Many professional game developers have been using C++ for several years, and it has been standardized for 8 years now. How many people are still using character arrays and strcat(), instead of std::string and std::stringstream's?
Do you employ usage of smart pointers at your studio? Did you implement your own or did you use something conventional like boost::shared_ptr or boost::intrusive_ptr (for you cache nuts)? Do your smart pointers support custom deleters such that they can become a more general 'resource handle'?
How many times does someone static_cast a base class pointer to a derived class pointer based on some 'enumerated type' value stored in the base class? Why are you losing the type information, and not educating developers about type safety and good use of templates to write less-voodoo code?
This topic is closed to new replies.
Advertisement
Popular Topics
Advertisement
