Jump to content
  • Advertisement
Sign in to follow this  
GuyWithBeard

How long does the C/C++ preprocessor keep going?

This topic is 896 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

I have a question about how the C/C++ preprocessor works. Lately I have been experimenting with some quite neat preprocessor macros to enable reflection-like functionality in a project, and everything I have done seems to be working nicely. However, it got me wondering, how does the preprocessor know when to stop processing a piece of code?

 

For example, let's say I have a macro that accepts two parameters. The macro puts those two parameters together using the "##" operator, which in turn produces another macro, which is then expanded. It seems to me the preprocessor has to do several iterations over the compilation unit, expanding things that are expandable, to produce the final output. Is this how it works, or have I misunderstood it all? Also, if it does do these kinds of iterations, does it keep going until it cannot find anything more to expand or should I be worried about hitting some kind of max depth?

Share this post


Link to post
Share on other sites
Advertisement

You basically got it right, it just keeps expanding things until it doesn't find a match any more.

It works line by line from top to bottom, rather than the entire compilation unit.

 

"Finished" lines are then fed to the 'real' compiler, for further processing/compiling.

 

 

Theoretically, there is no limit to the expansion. However, since your computer has finite memory etc, you'll run into some limit at some point.

I once #include-ed a file into itself, and g++ ran into the stack limit, giving me a very long error "stack limit exceeded in file foo, included in file foo line X, included in file foo line X, ..." a thousand times :)

 

I never tried

#define X X
X

but I expect that to give similar results.

 

Other compilers may have other safe guards like a max iteration count, it depends on the pre-processor that you use (and if you do hit a limit, it's likely, you can configure something to increase it).

Share this post


Link to post
Share on other sites

The algorithm followed by the preprocessor is actually quite elaborate and involves stacked contexts to correctly handle nested and/or recursive macros (for instance if a token is expanded using some macro, then that macro will not be considered in expanding the resulting token). Probably the most direct way to fully understand the system, if you want to, is to read the docs and maybe even the internal docs.

 

But in most cases the preprocessor can be thought of as a process that does a single pass over the input stream, keeps track of all preprocessor macros encountered so far, and tries to expand each token according to these macros. If a token is encountered before the macro that's supposed to handle it is defined, the token will be left untouched.

 

Be aware that despite its apparent simplicity, advanced use of the preprocessor is typically not portable; things will work on a particular compiler and will break horribly on another because it doesn't expand your tokens correctly or somesuch. If your preprocessor definitions are growing out of control, consider reviewing your design to see if all these macros are needed, and then think about using a more specialized preprocessing system like m4 or similar.

Share this post


Link to post
Share on other sites

#define X X
X

Actually that is never a problem because one of the rules the preprocessor follows is not to expand an expression again while already expanding it.

You can however have something like that happen if you ping-pong between two different defines.

Share this post


Link to post
Share on other sites

It seems to me the preprocessor has to do several iterations over the compilation unit, expanding things that are expandable, to produce the final output. Is this how it works, or have I misunderstood it all?

Close, but the compiler actually only needs over the compilation unit once for the preprocessor pass. After it expands a macro, it simply restarts parsing at the start of where the replacement occurred. It does not restart at the beginning.

Share this post


Link to post
Share on other sites
Sign in to follow this  

  • Advertisement
×

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

Participate in the game development conversation and more when you create an account on GameDev.net!

Sign me up!