Sign in to follow this  
Metus

STL why's and why not's in game programming

Recommended Posts

Hello. I'm pretty new in the "professional" game developer scene and found out that old demo-programmers seems to avoid STL and all the sweet libs that boost presents. The only reason they're writing their own string-, vector and mathclasses is that "stl is generating to big of a overhead" and "i want total control of my application" I found that really irritating: no matter how good your string-class is, it wont be as heavily tested as std::string and your matlib will probably never be as fast and optimized as D3DX******** So... are you an old demo-programmer or a cool dude? I'm using stl and boost whereever I can - AND if it is approperiate

Share this post


Link to post
Share on other sites
Quote:
Original post by Metus
The only reason they're writing their own string-, vector and mathclasses is that "stl is generating to big of a overhead" and "i want total control of my application"


I would say the real reason is lack of knowledge & old compilers give you misconceptions [wink]

Quote:
Original post by Metus
So... are you an old demo-programmer or a cool dude?


Nothing about being cool, just not being ignorant & arrogant [grin] and wasting your's and compiler implementor's time & money.

Share this post


Link to post
Share on other sites
Maybe 5-10 years ago or so when computers were slow enough to warrent such control/optimizations, then ya, STL would seem like a huge overhead. Today however, there is no reason to go recreate a String class or a dynamic array (vector) and especially Sets, Maps and Hashmaps (red-black trees....*shudder*).

We are in the programming age of using highly optimized libraries created so we can spend more time on our application itself. The tools are there, the overhead is negligable, use them.

Share this post


Link to post
Share on other sites
I agree that there is no real reason to reinvent the basic parts of the STL, like the string and vector classes... But, some things are badly implemented in STL, like the auto_ptr. As for math libraries, its very possible to come up with better optimizations than the d3dx math. Also, the d3d math classes are not portable... Plus, its a little annoying that a 3D API tries to impose you its own 3d math stuff.

My 3D engine supports both OpenGL and D3D at the low level. We have an abstract DLL interface low level enough to render vertex buffers in both... And uh, its sort of, part of the point of this system not to include the neither the d3d or the OpenGL libraries in the engine. There isn't that much math stuff to code, and for the flexibility it can bring, its worth it to implement it yourself.

Share this post


Link to post
Share on other sites
Guest Anonymous Poster
Everyone should write their own string, hashmap, and math vector classes for learning purposes, but then use the standard libraries in real code.

Share this post


Link to post
Share on other sites
Speaking of performance and STL, everyone should check this out.

Go to:

http://www.gdconf.com/archives/2004/index.htm

Check out the powerpoint: Common C++ Performance Mistakes in Games

Many of the performance problems are due to mis-use of STL. With a better understanding of what some of the functions do, and properly using them you can eliminate alot of the performance overhead. Some of the examples in the article make me think "wtf were they thinking", but I learned some new stuff from it.

Share this post


Link to post
Share on other sites
Quote:
Original post by Anonymous Poster
Everyone should write their own string, hashmap, and math vector classes for learning purposes, but then use the standard libraries in real code.


Exactly. As a beginner myself (3 months so far), I don't want to blindly use the STL without at least understanding how the classes work. For me, the best way to understand how they work is to come up with my own implementation. I've done a string class, math vector class, and a pseudo-hash table. I'm currently learning about exception handling and the STL for use in my next project [smile].

Share this post


Link to post
Share on other sites
Quote:
Original post by ontheheap
For me, the best way to understand how they work is to come up with my own implementation.


By all means if you haven't taken or currently taking a course on data structures & algorithms then it is worth doing your imps just to learn.

Actually i think if you open up your compiler's implementation of STL you'll learn even more, aswell as learning the structure you'll learn common techniques & idioms to overcome language flaws, i really recommend everyone has glance at them at least once, you'll see things such as:

Empty Base optimization (EBO)
resource acquisition is initialization (RAII)
type traits & compile-time dispatch on types (e.g if its a POD type use memcpy instead, this is at compile time not run-time).

being employed.

Share this post


Link to post
Share on other sites
Guest Anonymous Poster
Using STL and boost is likely to make your game _much faster_!

Reasoning:

The three man-months you avoid mucking with your own container implementations etc. at the _start_ of the project can instead be spent when the game actually works. Imagine having three man-months to profile and optimize the **** out of the game where performance _actually matters_.

I know where I'd rather spend months.

Most optimizations before the program is completed are random speculations and make the code _worse_ and harder to optimize.

Make you game _work_. Use all the ready-to-use stuff you can find, even including stuff you know you can't use in shipping game. Make it work!

_Then_ optimize it where it matters. Then replace that copyrighted temp stuff with your own code.

How many percent of hobby game projects are completed? Not many I would guess. Everyone is busy writing string classes until they get bored with the project :-)


/Marcus

Share this post


Link to post
Share on other sites
Quote:
Original post by snk_kid
Quote:
Original post by ontheheap
For me, the best way to understand how they work is to come up with my own implementation.


By all means if you haven't taken or currently taking a course on data structures & algorithms then it is worth doing your imps just to learn.

Actually i think if you open up your compiler's implementation of STL you'll learn even more, aswell as learning the structure you'll learn common techniques & idioms to overcome language flaws, i really recommend everyone has glance at them at least once, you'll see things such as:

Empty Base optimization (EBO)
resource acquisition is initialization (RAII)
type traits & compile-time dispatch on types (e.g if its a POD type use memcpy instead, this is at compile time not run-time).

being employed.


It's part of my major, but I haven't taken it yet. I definitely plan on checking out the STL implementation for my compiler. Plus since I've written the classes myself, I can see how they went about solving the same problem and learn more in the process.

Share this post


Link to post
Share on other sites
Quote:
Original post by Max_Payne
But, some things are badly implemented in STL, like the auto_ptr.


It's not so much an implementation issue as a design issue. std::auto_ptr's usability is restricted to a very specific domain: ensuring a resource gets released even in presence of exceptions. It is not intended to be a generic lifetime management tool. For that, you will want something like boost::shared_ptr, which have been accepted into the C++ standard library (q.v. C++ library TR) - so I guess that really make them std::tr1::shared_ptr.

Share this post


Link to post
Share on other sites
Agreed. auto_ptr isn't bad - the problem is that shared_ptr is missing, or an ownpointer=true flag in STL containers.

Share this post


Link to post
Share on other sites
Quote:
Original post by Fruny
Quote:
Original post by Max_Payne
But, some things are badly implemented in STL, like the auto_ptr.


It's not so much an implementation issue as a design issue. std::auto_ptr's usability is restricted to a very specific domain: ensuring a resource gets released even in presence of exceptions. It is not intended to be a generic lifetime management tool. For that, you will want something like boost::shared_ptr, which have been accepted into the C++ standard library (q.v. C++ library TR) - so I guess that really make them std::tr1::shared_ptr.


This is funny. Not even a minute ago did I start reading about auto_ptr and RAII! And how if you throw an exception from within a constructor, the destructor won't be called.

Share this post


Link to post
Share on other sites
Guest Anonymous Poster
To answer the OP's question, STL is not heavily used in professional game development for the following reasons:

1. Fear and ignorance (the biggest reason)

C++ and STL are not used because professional game developers lack knowledge and skill. In general, professional game developers are many years behind the times when it comes to software development -- not everyone, of course, but a big percentage. One reason is that professional game developers spend so much time trying to finish their games, they don't have the time to learn new things. Sad, but true. Another reason is that incorporating a new technique in a game means a higher risk of failure. The last thing a game developer wants is more risk in an already risky situation.

2. Code size and performance problems (Fear and ignorance, part II)

Problems with code size and performance (real and imagined)caused by templates discourage programmers from considering using STL.

3. STL containers rely heavily on dynamic memory allocation.

On consoles, memory is very precious and a lot of effort goes into controlling how and when memory is allocated. STL containers don't support that very well, so the costs of using STL are higher than normal (especially considering reasons 1 and 2).

Share this post


Link to post
Share on other sites
Writing your own containers does not take 3 man months. Unless you suck.

STL and std::string are "forbidden" at my place of work. There are a couple of reasons... their allocation practices being foremost. These can be avoided and/or worked around, but most people are used to using STL without thinking. In fact STL encourages less thinking... that's the whole point. I have mixed feelings on the issue. Sometimes STL would be very handy but for the most part I haven't missed it.

Share this post


Link to post
Share on other sites
From the time I typed printVal(sizeof(std :: string)) and got 42 on VC++ NET I know one day I must write my own string implementation. However, other containers suffice me at the moment.

Quote:
There are a couple of reasons... their allocation practices being foremost. These can be avoided and/or worked around, but most people are used to using STL without thinking.


Their default allocation behaviour may be bad for you, but in that case you may supply your own allocators (though that can be tricky in some cases).

Quote:
Problems with code size and performance (real and imagined)caused by templates discourage programmers from considering using STL.


No, God, no... templates are dicouraging them? They're normal classes created at compile time and in many cases (ie. when you don't use template specialization) compiler may optimize them internally so that there won't be X classes for T1, T2, T3 etc. but one class for void* which will result in less memory overhead. It may look unsafe but it it's up to compiler to do it right, and if it would be dangerous, I think it wouldn't be eventually implemented.
Ok, so if they don't use templates for it... what else? Macros? Manual void*? or inheriting? :-/


BTW: very interesting topic, maybe someone who is programming profesionally may give as some hints?

Share this post


Link to post
Share on other sites
Quote:
Original post by Koshmaar
Their default allocation behaviour may be bad for you, but in that case you may supply your own allocators (though that can be tricky in some cases).


Right, but then you still have people running into the "gotchas" of using STL, and as a bonus you've invalidated every argument about how you should use STL because it's well-tested. Also as you mentioned, it can be tricky.

Quote:
BTW: very interesting topic, maybe someone who is programming profesionally may give as some hints?


Don't I count? :(

Share this post


Link to post
Share on other sites
I have used the vector class to store data for vertices for a mesh , i can say that the code ran S L O W E R than my dynamic array version , when i put the result in this site i got flamed as hell being disinformed about stl and being not able to write working code , and also being a stupid person after all.

Share this post


Link to post
Share on other sites
Quote:
Right, but then you still have people running into the "gotchas" of using STL


There aren't as many of those, and with time you will become more acquainted with existing (the same goes with your home grown). Also, after reading Herb Sutters "Exceptional C++" I know much more why certain things in STL are like they are, and why more intuitive way may bring potential security and holes and performance bottlenecks into code.
Not to mention that now I'm sure STL was designed by very experienced programmers that have almost whole C++ in small finger (in fact, they're the creators of C++). Well, chances are they come up with better library design than you may ;-) also, peaople who actually implement STL have more time for subtle language stuff and optimization.

To be clear, I'm not arguing that STL for a profesionall is a must, but there are places when it's very handy and simply ignoring it sometimes can lower your productivity and app performance. And for amateurs it's a godsend.


Quote:
Don't I count? :(


Sorry, I meant: we want MORE profesionalls ;-)

Share this post


Link to post
Share on other sites
Quote:
Original post by ketek
I have used the vector class to store data for vertices for a mesh , i can say that the code ran S L O W E R than my dynamic array version , when i put the result in this site i got flamed as hell being disinformed about stl and being not able to write working code , and also being a stupid person after all.


There's a boost class that works better if you just want a nice class wrapper for an array, rather than a resizable object.

Share this post


Link to post
Share on other sites
Quote:
Original post by Koshmaar
Sorry, I meant: we want MORE profesionalls ;-)


OK, I'm a professional so I guess I'll chime in :)

First, I'll give you some background. Most of the projects I've worked on were console titles. On these projects using STL in game code was forbidden and the use of templates was kept to a minimum. However, the last project I worked on was a PC title and the STL was used a lot and templates in custom code were used quite a bit more, and in more exotic ways.

Before I get into talking about the details, I think it's important to note that when people say they do/don't use the STL they're mostly referring to the STL's containers. I don't think most people have a problem using the STL algorithms, for instance. So, effectively, when you choose to use the STL containers(or not) you're deciding how you're going to hold and access most of the data used in your game. Which is a pretty big deal.

Now, one of the big differences between these projects is that on the console titles we used multiple compilers(VC6, SN Systems, Code Warrior, etc) on a single project for the different platforms. This, by itself, is one of the big reasons that the console guys tried to avoid using the STL. Since the implementation of the STL is different with each compiler it means that the way you store and access your games data on each platform is going to be different. This makes development much harder since you can't do something on one platform and expect the same memory usage, performance, etc on all of the other platforms. Also, we did much of our development on the PC for quicker iteration times. On the PC we used VC6, which had horrible template support and a it's implementation of the STL wasn't very good. And that pretty much put the nail in that coffin.

In stark contrast, on the PC title we only had one platform to support and it was the same platform we worked on every day. This meant we could use the STL containers and see their effects on speed and memory use easily. Furthermore, we started with VC7 and upgraded to VC7.1 early in the project, which has really good support for templates and a pretty nice STL implementation.

In the end, like the use of any tool, I think it pays to be pragmatic. There's no clear-cut answer for anything. And I think the choice to use the STL or not on each of the projects I've worked on was probably the right choice.

On a final note, one thing to think about is that when it comes right down to it, 99% of the general container use in any game will consist of arrays and linked-lists. Sure, you might need the occasional map/hash-table, set, and maybe a queue here and there. But, the bread and butter are your arrays and lists. And, as luck would have it, if you find that the STL won't work well for most of your needs it's not very hard to make your own array or list class :)

-John

Share this post


Link to post
Share on other sites
I agree that you should stay away from STL when multiple compilers/platforms are involved, due to implementation issues. However, to not use things like std::string, std::map, etc just because you think you can do a better one and you think its too much overhead is just silly. I wonder how many java programmers refuse to use java.lang.String because they can do better?

Share this post


Link to post
Share on other sites
Although I use std::string, I've implemented my own linked list and trees. I can't stand the syntax of most of the STL stuff. It's absolutely vulgar.

Share this post


Link to post
Share on other sites
Quote:
Original post by GroZZleR
Although I use std::string, I've implemented my own linked list and trees. I can't stand the syntax of most of the STL stuff. It's absolutely vulgar.


There must be a very fine line between vulgar and elegant...

Share this post


Link to post
Share on other sites
Quote:
Original post by GroZZleR
Although I use std::string, I've implemented my own linked list and trees. I can't stand the syntax of most of the STL stuff. It's absolutely vulgar.


You have to get over that. Are you going to rewrite OpenGL or DirectX because you think the "syntax" is vulgar? What happens if you are work on a team and another person writes vulgar code? Are you going to rewrite all his/her code? What happens if that other person thinks your code is vulgar? A war would break out and the project would never finish because it would be endlessly rewritten.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this