Sign in to follow this  

Fed up with Make. Is there a better way?

This topic is 2842 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

As the title says I've had it with Make. As my homebrew projects have grown over the years I've found myself stringing together dozens of little custom converters, utilities and assemblers into these fragile monolithic Makefiles, typically with data and code being baked in fifteen different ways with dependencies going left and right before finally producing some output image. Not to mention all of the little peculiarities of the language, or at least of GNU Make on Windows. Why can't I have a hash mark within a quoted variable definition? Why does a parallel build freak over rules with multiple targets? And who's bright idea was it to use tabs to identify commands anyway? I'm not expecting any other tool to remove my dependencies for me, but surely there must be a better language out there for expressing this sort of thing? Note that I have little interest in trying to manage builds for a hundred and three different environments. Furthermore I'm looking for a small standalone executable that I can painlessly package together with my code and other tools to send to artists and such, not something that needs a gazillion other utilities or some "run-time" installed.

Share this post


Link to post
Share on other sites
Quote:
Original post by cache_hit
Hopefully I'm not asking the obvious, but you already said you're on windows, so why on earth can't you use Visual Studio Express edition?
I do use Visual Studio for C++ development. However I mostly use Make for cross-compiling homebrew projects (e.g. my current project is a C64 game.)

Still... Wouldn't I have to resort to some external Make-like utility when baking data in intricate ways for any platform?

Quote:
Original post by bubu LV
Waf
Quote:
Original post by Hyrcan
I am quite happy with SCons.

Thank you. I'll check those out.

Share this post


Link to post
Share on other sites
Quote:
Original post by implicit
Quote:
Original post by cache_hit
Hopefully I'm not asking the obvious, but you already said you're on windows, so why on earth can't you use Visual Studio Express edition?
I do use Visual Studio for C++ development. However I mostly use Make for cross-compiling homebrew projects (e.g. current project is a C64 game.)

Still... Wouldn't I have to resort to some external Make-like utility when baking data in intricate ways for any platform?


Ahh gotcha, didn't realize there was cross-compiling going on.

Have you considered CMake? It's a script system built on top of make (and visual studio, for that matter) that sits at a higher level and generates makefiles, visual studio projects, etc automatically.

Share this post


Link to post
Share on other sites
Quote:
Original post by cache_hit
Have you considered CMake? It's a script system built on top of make (and visual studio, for that matter) that sits at a higher level and generates makefiles, visual studio projects, etc automatically.
++ for CMake. I use it to compile on Mac/Windows/Linux, and I have had considerably less trouble than with automake/scons/etc. For the most part, CMake's ability to generate Visual Studio/XCode projects allows much better integration with native tools.

Share this post


Link to post
Share on other sites
Quote:
Original post by cache_hit
Have you considered CMake? It's a script system built on top of make (and visual studio, for that matter) that sits at a higher level and generates makefiles, visual studio projects, etc automatically.
Not really. It seemed geared towards solving a different problem from the one I am having. That is smoothly compiling C projects on multiple platforms as opposed to assembling code for a very specific machine and massaging data in byzantine ways (not to mention that the ten-megabyte installer kind of put me off.)
But perhaps it's worth taking a second look?

Share this post


Link to post
Share on other sites
Quote:
Original post by Antheus
Hudson is often used for CI infrastructure.

The fact that it's Java shouldn't be used to dismiss it early.
I'll check it out. I don't have a problem with Java as long as it works out-of-the-box on whatever run-time comes bundled with Windows.

Share this post


Link to post
Share on other sites
Quote:
Original post by implicit

I'll check it out. I don't have a problem with Java as long as it works out-of-the-box on whatever run-time comes bundled with Windows.


You don't install it, you put in on a server, then it monitors whatever data stores you have, and acts upon them.

If you're dealing with non-technical people, teach them either how to use versioning software, or better yet, dropbox. Do the heavy lifting yourself.

In practice, deploying build systems to Joe User isn't viable, since there are too many variables to account for, just Windows is a nightmare, *nux even moreso.

Share this post


Link to post
Share on other sites
Quote:
Original post by Antheus
You don't install it, you put in on a server, then it monitors whatever data stores you have, and acts upon them.

If you're dealing with non-technical people, teach them either how to use versioning software, or better yet, dropbox. Do the heavy lifting yourself.

In practice, deploying build systems to Joe User isn't viable, since there are too many variables to account for, just Windows is a nightmare, *nux even moreso.
Oh... Then I'm afraid that's not going to work. My first priority is to get artists to actually *work* on my game projects, and I'm having enough trouble as it is getting people to run simple command-line tools.

On a related subject, I see that Waf and SCons are Python-based. Can I extract the interpreter and whatever libraries are necessary from the Python distribution, such that they can be packaged up with everything else needed to compile the project in a tidy little archive? Bouncing ZIP-files around by email is about the highest level of complexity I'm willing to risk here.

Share this post


Link to post
Share on other sites
Quote:
Original post by implicit

Can I extract the interpreter and whatever libraries are necessary from the Python distribution, such that they can be packaged up with everything else needed to compile the project in a tidy little archive?


Think about this for a second. Build system is just a script of sorts. It relies on everything else, obscure things you consider, um, ... implicit. Such as configuration for ld so it works with that distribution's elf. GNU make does this first time it runs, and this is what those hundreds of "checking for..., has a ...., ...." messages are. They are combined 30 years of hard work and real world experience on what all goes on just on common distributions.

The reason makefile is such a mess is because it needs to install and configure actual tools. You cannot package everything unless you make a bootable Linux distro on a CD or USB. Then there's licensing restrictions, you cannot distribute MVS and DX or even debug runtimes.

It's not the make tool, it's the execution environment which includes the OS itself.

On Linux, things are fairly easy, use apt. On Windows, you'll need to write a list on how to download, install and patch all the actual tools you use (you cannot distribute most of them, nor automate them) + a zip file that does your own stuff.

Share this post


Link to post
Share on other sites
Quote:
Original post by Antheus
Quote:
Original post by implicit

Can I extract the interpreter and whatever libraries are necessary from the Python distribution, such that they can be packaged up with everything else needed to compile the project in a tidy little archive?


Think about this for a second. Build system is just a script of sorts. It relies on everything else, obscure things you consider, um, ... implicit. Such as configuration for ld so it works with that distribution's elf. GNU make does this first time it runs, and this is what those hundreds of "checking for..., has a ...., ...." messages are. They are combined 30 years of hard work and real world experience on what all goes on just on common distributions.

The reason makefile is such a mess is because it needs to install and configure actual tools. You cannot package everything unless you make a bootable Linux distro on a CD or USB. Then there's licensing restrictions, you cannot distribute MVS and DX or even debug runtimes.

It's not the make tool, it's the execution environment which includes the OS itself.

On Linux, things are fairly easy, use apt. On Windows, you'll need to write a list on how to download, install and patch all the actual tools you use (you cannot distribute most of them, nor automate them) + a zip file that does your own stuff.
All of that is certainly true for regular C development but this is where the beauty of hacking on small C64 projects in assembly language comes into play. There's only one *very* specific target here, certainly no build options to discover. And the entire project, including code, data and every tool needed fits easily in a self-contained ZIP file on the order of a megabyte or so, without the need for any installers or run-times.
Granted, that assumes the user is running Windows, but that's a safe enough bet.

Seriously. I have it down to the prospective artist unpacking a ZIP file and running a "build" batch file to rebuild the game. Unfortunately that also seems to be about maximum level of complexity I dare have without scaring people off.

Share this post


Link to post
Share on other sites
As far as I can tell CMake is the only proposed utility which is conveniently distributable as a single executable. Well.. Technically there's also the CRT DLLs but how hard can it be to reconfigure a cross-platform build tool to compile statically ;)

So I'll give it a try then, especially as it seems to be quite popular. Hopefully the scripting language is reasonably clean and expressive. Although given that it still produces Makefiles I'm a little worried that it won't insulate me completely from and/or be as powerful as Make (it will be interesting to see how it handles that problem with multiple targets during parallel gmake builds for one.)

Share this post


Link to post
Share on other sites
Quote:
Original post by theOcelot
Though I've never tried it, I always thought the_edd's build system looked nice.

slam build system.


It's fine for some things (static libs and executables), but I wouldn't recommend it here. I plan to retire it at some point in future (perhaps with a replacement, if I ever get around to it...)

Share this post


Link to post
Share on other sites
IIRC there is a wiki page on this subject, if the wiki was working of course.
Personally I would add Premake to the list of suggestions. Also there have been a couple of threads on sweng gamedev on a related topic.

Share this post


Link to post
Share on other sites
Quote:
Original post by frob
ant / nant
msbuild
Visual Build
ClearMake
Rake
BuildIT
Maven


Please strike Maven off any such list of recommendations. It is practically inextensible, only suitable for Java, designed to conform projects to how it wants to do things rather than to adapt to user needs, and fundamentally incompatible with sound configuration control practices.

Chains of lots of custom tools are a very good fit for Ant, which has good facilities to manage lists of files and directories, a well refined "exec" task to invoke arbitrary command line tools, an explicit and readable dependency graph and parallel task invocations.
Ant needs a Java runtime environment and the large but trivial to install Ant tool itself; the former can be usually expected, the latter might be perhaps bundled with your sources.

I have significant experience using Ant with involved Java projects, and I've never had trouble getting it to do what I want without quirks; if assimilating command line tools isn't enough, there is the possibility of defining tasks as macros of other tasks, as embedded scripts in various languages or as Java classes based on a simple API, so adopting Ant isn't going to be a 90% solution.

Share this post


Link to post
Share on other sites
Quote:
Original post by dmail
IIRC there is a wiki page on this subject, if the wiki was working of course.
Personally I would add Premake to the list of suggestions. Also there have been a couple of threads on sweng gamedev on a related topic.
That one looks pretty sweet at first glance. Just a single executable with powerful Lua-based scripts.

Unfortunately a deeper look reveals that it doesn't support custom build tools. I might have been willing to give a beta version the benefit of the doubt, but a supposedly powerful build tool at version four point two which cannot handle generated code is a joke as far as I'm concerned.

Quote:
Original post by LorenzoGatti
Chains of lots of custom tools are a very good fit for Ant, which has good facilities to manage lists of files and directories, a well refined "exec" task to invoke arbitrary command line tools, an explicit and readable dependency graph and parallel task invocations.
Ant needs a Java runtime environment and the large but trivial to install Ant tool itself; the former can be usually expected, the latter might be perhaps bundled with your sources.
Thanks! I'll give Ant a second look then. To be honest the less-than-elegant XML build scripts and the need to drop out into external Java classes to define custom tools kind of put me off, but beggars can't be choosers.

To be honest I have a nagging suspicion that all of this venting of my frustration on every build system in sight is just my subconscious' way of working me up to writing a Make replacement of my own.

Share this post


Link to post
Share on other sites
Quote:
Unfortunately a deeper look reveals that it doesn't support custom build tools. I might have been willing to give a beta version the benefit of the doubt, but a supposedly powerful build tool at version four point two which cannot handle generated code is a joke as far as I'm concerned

How's that? We haven't yet upgraded to Premake4, but the predecessor supported Visual Studio "custom build step" (which we use for assembling .asm and generating self-tests). Or did you mean something else?

Share this post


Link to post
Share on other sites
Quote:
Original post by Jan Wassenberg
Quote:
Unfortunately a deeper look reveals that it doesn't support custom build tools. I might have been willing to give a beta version the benefit of the doubt, but a supposedly powerful build tool at version four point two which cannot handle generated code is a joke as far as I'm concerned

How's that? We haven't yet upgraded to Premake4, but the predecessor supported Visual Studio "custom build step" (which we use for assembling .asm and generating self-tests). Or did you mean something else?
The impression I got from reading the documentation and browsing the forum was that premake supported dumb pre/post-build steps, but not custom rules with non-trivial interdependencies. So, e.g., adding an EXE-packer as a post-build step would be fine, but not plugging in custom C preprocessor.

Of course it's entirely possible that I've just gotten the wrong impression here, I haven't actually tried to use it after all.

Share this post


Link to post
Share on other sites
Quote:
The impression I got from reading the documentation and browsing the forum was that premake supported dumb pre/post-build steps, but not custom rules with non-trivial interdependencies. So, e.g., adding an EXE-packer as a post-build step would be fine, but not plugging in custom C preprocessor.

It should indeed be possible to plug in a C preprocessor. What we do for the self-tests is add a package/project that generates cpp files from special headers via cxxtestgen (which is the "custom build step" according to which the headers are 'built'), then add a second package/project depending on the first that compiles these newly created cpp files.
(We patched in support for "custom build step" into Premake3, and I understand the same has been added to the official Premake4. This affords much more possibilities than a mere pre/post build step.)

Share this post


Link to post
Share on other sites

This topic is 2842 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this