Jump to content
  • Advertisement
Sign in to follow this  
hoffman_jwr

How Can I Reduce My Compile Times?

This topic is 2719 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

Hi there,
I am a graduate games programmer with around 6 months industry experience, and this dilemma is something that I often ponder. Up to this point, I have worked on a few projects which have grown to become substantial in size (approx. twenty - thirty thousand lines). Although I have strived for software orthogonality (i.e. modularity), and worked towards decoupling the various aspects of the systems I worked on, inevitably the projects bloat, interdependencies pop up, and compile times become long. I work in predominantly in C; I always ensure forward declarations over header includes, I use pointers wherever possible. Is it possible to modularise to the point where a module is completely free-standing? Would it be possible to place the various components of an engine into dll files (something I have never done)? On large scale projects, is this practice? - or does the solution reside in the one project file (somehow I doubt that)? dll's are windows specific; are there any platform unspecific ways to build linkable object files?

Share this post


Link to post
Share on other sites
Advertisement
Grouping modules by what they do is a good practice. A lot of engines do that. The base dll defines only interfaces and plugins implement sub-systems. It will definitely reduce dependencies to only 1 dll (the base module). Having said that, a good way to reduce compile times is to use pre-compiled header files. The idea may seem trivial but having a good set of base headers that seldom change in a single header file and compiling that file once will reduce build times significantly. I am writing this from my personal experience.

Share this post


Link to post
Share on other sites
It sounds like you already have a pretty good handle on the basics. Really it's a matter of keeping things as decoupled as possible, and minimizing the scope of dependencies. The real killers are things that crop up in, say, 75% of your code base and require vast recompiles every time something subtle changes.

One thing a lot of people overlook is dividing a single module into multiple header/code files. For instance, you may have a chunk of game logic that affects a lot of the engine; split it up into smaller groups based on whatever criteria makes the most sense, and go from there. This can radically reduce compile times when you only make localized, contained changes. Be careful, though, as on some compilers having too many files can actually end up costing you time due to the spin-up cost of starting compilation separately on every file.

Short of that... throw more hardware at it wink.gif


Honestly, though, 20-30 KLOC is nothing. Wait until you're in the millions of lines of code range and a full rebuild takes an hour or two. That's when it's worth doing major devilry to make your compile times faster.

Share this post


Link to post
Share on other sites
For starters, it seems like you've been on the right track. But my question is going to be: what compile time is getting too large?

Look into a build solution like Cruise Control to take care of the full build+test cycle for you. That way, the longest build times are hopefully dealt with on a dedicated machine.

Make sure your build system is setup properly. For instance "make" by default requires the "-j" option to be added to get a multi-threaded build. Make sure your dependencies are properly being maintained if you don't have an IDE that generates them for you. Or better yet, spend the time to make sure it is automated.

Make sure your interfaces are being respected. Systems can get too tangled together when someone on the team decides to grab what was an internal header and use it somewhere because it had a feature they needed. In theory, you should be able to seperate collections of subsystems into static libraries and only rebuild them as needed (ie, compile Bullet Physics as a static lib, and add it to the project). Only go with the dll/so runtime loading if you gain something from it.


One thing a lot of people overlook is dividing a single module into multiple header/code files.


Seconded. If you have a "mathutil.h" floating around, consider breaking it up. Maybe it contains a lot of unrelated systems. You could end up with "interpolate.h", "springs.h", and "range_limits.h" as well, because people just tossed "math" stuff in there when they needed to wrap, clamp, mirror, spring, blend, accelerate_smooth... etc. Go further? have a spring1d.h spring2d.h and spring3d.h if there are enough functions to warrant it.

Share this post


Link to post
Share on other sites
Seconded (thirded?) about precompiled headers. A lot of programmers seem oddly sour on PCHes, possibly because their very first C++ experience was MSVC complaining that it couldn't find the #include <stdafx.h>. But I've had it reduce total project compile times by a factor of eighty.

Share this post


Link to post
Share on other sites
Like others said, use precompiled Headers. They exist for MSVC and for GCC as far as I know.
They already reduce your compile time by at least 60%.
Just make sure you only add "static" Headers to your include-file which the PCH is built from.
If you include headers there which are changed very often, you save nothing.
But adding there all the boost-, stl-, library-headers you never gonna change, reduces your compile time dramatically.
HTH
regards

Share this post


Link to post
Share on other sites
All very good advice, thanks to everyone! A common suggestion seems to be the use of precompiled headers, something that I will look into. When you say precompiled headers, do you mean header files that will not have to be rebuilt after a full clean and rebuild? For the record, we are using gcc, and the full rebuild time on our current project is around 6 minutes. Essentially, what I am working towards is a way to add (unchanging) modules to new projects without having to transfer all of the implementation files; I think this is a static library?

Thanks for not flaming me wink.gif

Share this post


Link to post
Share on other sites

All very good advice, thanks to everyone! A common suggestion seems to be the use of precompiled headers, something that I will look into. When you say precompiled headers, do you mean header files that will not have to be rebuilt after a full clean and rebuild? For the record, we are using gcc, and the full rebuild time on our current project is around 6 minutes. Essentially, what I am working towards is a way to add (unchanging) modules to new projects without having to transfer all of the implementation files; I think this is a static library?

Thanks for not flaming me wink.gif


Well, yes ofc, if you want to reuse self programmed modules, use static libraries.
Precompiled Headers is a newer technology which allows the Compiler to "precompile" Headerfiles you gonna use.
Basically it makes a static library out of all your headers. While this is technically not entirely true it explains best whats happening.
That reduces compile time too since the Headers do not need to be parsed every time you build if the headers are not changed.

See:
http://msdn.microsoft.com/en-us/library/z0atkd6c%28v=vs.80%29.aspx
http://gcc.gnu.org/onlinedocs/gcc/Precompiled-Headers.html
http://en.wikipedia.org/wiki/Precompiled_header

I would adive you to make a static or dynamic library out of your modules and to use PCH.
A combination is best.

Share this post


Link to post
Share on other sites
Sign in to follow this  

  • Advertisement
×

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

We are the game development community.

Whether you are an indie, hobbyist, AAA developer, or just trying to learn, GameDev.net is the place for you to learn, share, and connect with the games industry. Learn more About Us or sign up!

Sign me up!