A Postmortem of Game Programming with Digital Mars’ D Programming Language  GameDev.net

A Postmortem of Game Programming with Digital Mars’ D Programming Language
by Jonathan Breuer


ADVERTISEMENT

Prologue

One thing I find very interesting is different computer languages, their development and their differences. I spend most of my time in C++, but the little exposure I’ve had to LISP or to other languages always leaves me wishing that C’s macros were cleaner, or that C++ had a way to implement a nice multi-variable for-each.

This past month I’ve been on vacation and since my wife would be bringing her laptop, since I didn’t feel like bringing the MSVC++ CDs or backing up my projects directory, and since I didn’t really have any current project I wanted to work on, I decided to break out Digital Mars’ D Programming Language and try to learn it.

I had read through some of the first pages of Digital Mars’ D’s documentation and propaganda, but I had never taken the time to actually work with the language. After a few weeks of fiddling with it when not at the pool, I have to say I really like the language.

Two major design philosophies that I feel when working in D are: 1) Things that you do a lot should be simple and 2) A language should be clean, simple and consistent.

This diary is about the specific joys and difficulties of working with D.

D is billed as a ground up rewrite of C – a new version of a good language, or everything that C++ wishes it could be but never will be.

Complaint #0: “D” is horrible to search for. “C++” is a distinct string. I ended up having to use “D Programming language” or “Digital Mars D” every time I wanted results. Searching from within DigitalMars.com and then forcing Google to use the within domain and the “-archive” flag helped when I wanted language specific answers.

Installing

Google is everyone’s best friend. “D Programming Language Download” drops you to a page with all the downloads. I was running Windows XP Pro in the hotel room, so I grabbed DMC.ZIP and DMD.ZIP. (The Linux people will want to grab the DMD.ZIP for Linux and must have GCC lying around. Presumably you’ll know where to find that. ) You might also want to find CandyDoc because it’s pretty.

It’s a quick download, which is nice because I’m on dialup at home.

I initially wanted to run the programs from the desktop, so that I could easily clean everything off when I left the hotel. That didn’t work out in the brief time I spent fighting, but the following files seem to be important if you want to bash D into running from a non-root location: DMD\bin\sc.ini and DMD\source\d\all.sh. There’s probably more to it, but I gave up. At least some of the tools don’t handle file paths with spaces in them.

I still didn’t want several folders littering my C: root, so I extracted the DMD.zip into c:\DMD and the DMC.ZIP into c:\DMD\DMC. This required modifying the LINKCMD in DMD\BIN\SC.INI to reflect the moved DMC folder. (On a second attempt at home, I extracted everything into c:\dmd\... \bin, \src, \include. You have to update sc.ini again to reflect D's settings, but it worked.)

I created a projects folder under c:\dmd\projects.

Getting work done

I like using Textpad as my editor with a modified D.syn file. The standard Textpad download D.SYN is missing some keywords. I also opened two cmd windows with the following batch file

@ECHO OFF
SET PATH=%PATH%;"C:\DMD\bin";"C:\DMD\dmc\bin"
CMD ; Run a command line and don't return
(At home, I don't need the c:\dmd\dmc\bin, because I successfully flattened DM-C and DM-D into the same set of directories.)

The batch file was configured to run in c:\dmd\projects.

My first cmd window was used for compiles and my second was used for running my programs. This resulted in some mouse clicking, but left everything open. Both BUD and dmd also have the "execute on successful compile" option, so having just one cmd window works as well. I also opened folder windows to the Phobos standard library source and to the documentation and had Textpad taking up ¾ the screen.

A good test of whether your install worked is to run your command window, then navigate to c:\dmd\samples\d\ and run:

shell all.sh
If it works, D is configured and installed correctly. If it doesn’t, you failed somewhere. It took me a few tries at home and 1-2 tries at the hotel to get things up and going.

My experiences

The online documentation, a lot of which is installed with the compiler, is very readable. I’m a bit odd, but I found the documentation to be enjoyable reading. (I find Microsoft help to be useless and Microsoft online documentation to be needlessly obfuscated. (Unix’s man help pages are alright. They are uniformly terse, but are still complete and tend to actually answer the questions I have .) After I started working with the D compiler, I found that Digital Mars D Documentation suffers from two inexcusable lacks. First, there is no comprehensive index or table of contents. All month, I’ve been struggling with 2-3 recurrent problems with features which I want to use, but don’t remember how they worked. I’ve been searching the documentation for “const” and “function parameters" numerous times, but since Digital Mars uses Google as their index front end, the results are all original discussions of the feature in the forum archives (filled with people much smarter than I am) and buried somewhere is the actual documentation page. Continually searching for “site:digitalmars.com –archive class object” improves the results.

Request #1: The D Programming Language Documentation needs to be comprehensively indexed – both by subject and by concordance keyword.

The second lack in the documentation is its terseness. (This may be aggravated by the difficulty I found in finding answers). The documentation seems to be written by extremely smart people to explain to other extremely smart people the differences between C++ and D. (Or written to explain to a writer of another D compiler exactly how to implement it.) I would prefer that some things that should be obvious or are explained under another topic be spelled out in their own page. Something more on the lines of Programming in Lua by those guys, or The C++ Programming Language by Stroustrup. There are answers to many questions in the documentation, but you must randomly search for it. Only a little of the documentation appears to be available from the D Programming Language home page.

Request #2: The documentation needs to be doubled: preferably by explaining more things, but perhaps by merely storing related topics on separate pages with footer linking.

Remember, I said I found the documentation to be an enjoyable read. Since D has built in support for code auto-documentation, the library documentation tended to be complete. It also had a comprehensive table of contents as a sidebar. Again, repeatedly listing requirements and error cases would be nice for each function, instead of writing the doc once per module, but again this is a language written by very smart people for very smart people.

Beginning work : Dir2

I started by grabbing the “hello world” or “word count” app, I forget which. I copied it from \samples to \projects, changed the documentation to reflect that I was now its author and started work.

I’ve wanted a nice directory diff tool for a while now. I realize that there are many of these available, but I’ve always been busy when I needed one, I’m too cheap to spend the $20 for a commercial one and too lazy to search for a good open source one. Besides, not finding a good tool has left me this opportunity to write one. When I got to the hotel, I immediately copied my 1000 crappy 80’s games CD on to the computer and Windows copy barfed after 975 of them. This left me with no good way of knowing which of the 100,000 files the problem was, but I tried to manage. This is a perfect illustration of one application of my directory diff tool. I want to be able to repair large directory copies, not merely delete and retry.

The first job is to write a tool which can list all files recursively across a folder and report the findings. D has a function with this functionality; listdir. I immediately encountered the unacknowledged recursive NTFS listdir bug. In the D archives, there’s a complaint that listdir doesn’t always recurse. The response is the classic “It works on my machine." On the hotel computer, I had this same problem. I don’t know why, but I had to write a listdir which could recurse. (I later encountered a copy-file problem: std.file.write( out_filename, std.file.read(in_filename)); worked fine, but copy_file(out_filename, in_filename); didn’t. Ah, turns out that the parameters are SRCPATH, DESTPATH – no wonder the file didn’t exist.)

Complaint #3: The D compile errors are obscure, occasionally incorrect, and happen in a multi-pass manner.

It’s probably a good thing that the D compiler is a multi-pass thing. There are no header files and no predeclerations. However, because the compiler is multi-pass, it tends to barf on an early pass before moving on to a next pass. I think there are 3ish compile passes and 1 link pass. If your code is significantly messed up, you will bring your number of errors down past 1 several times only to see new sets of errors crop up. (It could also be that I just caused more errors by fixing one) The errors usually refer to the line that failed, but not always. Factoring out code and block-commenting code seemed to be required. I had a problem with using an anonymous enum which reported the error on the declaration line, not on the usage line.

Complaint #4: The D run time errors are obscure.

When debugging in MSVC++, you get the line, call stack and all other information you could want about the failing code. When the program crashes, it would be nice to get a line number, a function name, or the option to debug the problem. D just dies to the command line with an error message and no context. The same problem applies to the asserts. MSVC++ asserts are continuable, possibly an implementation failure, since asserts should be fixed. It would be nice to be able to continue past asserts to get a comprehensive list of failures, but I recognize the point.

Being able to toss strings and arrays of objects around as parameters and return values without worrying about the memory leaks is really nice. In C++ you must always be aware of whose responsibility an object is. D is really nice about allowing you to pass references (low overhead on function calls) around without worrying about later releasing the objects or performing expensive copies.

DERELICT Game Library

After finishing my directory diff tool and deciding that the HTML source code reformatter was too boring to work on; I returned to trying to get some kind of a game working in Digital Mars’ D programming language.

I downloaded Derelict, the everything-related-to-games-or-graphics linkage library for D. It takes good advantage of D’s ability to statically and dynamically link to C without hassle. I liked Derelict. Add one additional initialization function call and the C code for your OpenGL or SDL program is very nearly ready to run.

Derelict needed Build / BUD in order to run. (BUD is the name of the tool formerly known as Build. BUD parses your source files as needed and acts as an intelligent MAKE. You’ve already specified in your source which imports you need. BUD uses this information to compile and link your executable.). (Download this, then rename the junk file that downloaded as “bud.exe”. The download link was broken for me.) After using BUD to get Derelict compiling, I discovered I like it better then dmd for my compiles. No make files. The naming scheme of your files appears to be sensitive, but you can build a single file and all dependencies will be dragged in.

One of the tutorials I read recommended breaking Derelict’s zip file layout by searching for every subdirectory named “derelict” and then dropping those files into DMD’s SRC directory. This worked very well.

I grabbed one of Nehe’s articles (Try 6 and 15, I forget which I used) as a base, started with the OpenGL “flying a little plane around the screen with the keyboard” and built upwards to a Asteroids / Bosconian type game.

I took that basic file and started adding layers of functionality. I mildly kept in mind how I would be using my code in the future, but mostly I just kept a list beside my keyboard of the features I wanted to add, and then refactored the code as I went.

I haven’t been a fan of C# and Java’s one file layout because it seemed cluttered to me. A header gives you a nice concise list of what functions and structures are available, leaving the source file to do actual work in. Using BUD made splitting D source files very easy, since once the files marked what imports they needed, the compile just worked. D has nice class / object oriented support, but doesn’t mandate it religiously. I find it easier to refactor stray functions out into new files than to break an existing class in half to make navigating the files easier.

If I had worked much longer with my little toy game, I would have broken the render and keyboard process functions in half. I had code for both gameplay and the main menu in the same functions and that looked ugly. I would probably add a function pointer “process mode” system to handle which render and process was currently executing. After adding many enemies and weapons, I was starting to get irritated at the duplicated code in the “behaviors” file. Most of that code could be converted to data and placed in an external script or in an .INI type definition file.

Various Notes on D

writefln

writef is a cool function. If you are looking for control, it works just like printf.

writef(“%d + %d is %d\n”, 1, 1, 1 + 1);
However, if you are in a hurry and want a bunch of debug stuff dumped out:
writef(filename, “ “, line_number, “\n”);
Even better: since every printf I’ve ever written in C has had \n tacked to the end of it:
writefln(“%d + %d is %d”, 1, 1, 1+1);
writef is type safe. It auto-detects sign, so no more %u / %d nonsense. You can also use %s for every insertion and D will lexical cast everything into a string before displaying.

Strings

In D, strings are a built-in part of the language. But it’s better than that. In D, arrays are a first-class part of the language. Strings (being arrays of characters) benefit from all the cool things you’ve ever wanted arrays to do.

return array[4..length]
This is possibly one of the coolest single lines of code I’ve ever seen. It’s free! Because I’m making no modifications, but am just returning a slice of an array, the line of code above uses 8ish bytes of memory and no processing time. Also note that length is automatically scoped inside array.

Unit Testing

On these first two projects, I didn’t use D’s built-in unittest keyword . Later, on another project, I decided that I should toss in some tests. unittest{} is very well written. Anywhere in your files, you can drop a unittest{} block (treat it like a self-calling pseudo-function (if you’re from C++)) and those tests will get executed when you run a –unittest build. It is a simple written system that works well. I don’t normally unit-test my code, but with as easy as D makes it, I can see the benefits. After filling out one of these blocks, I know my code is not broken in several ways. I also know that it cannot become accidentally broken, because my tests will demonstrate that. I found this pattern throughout my experience with D. D makes it easier for me to be the kind of programmer I would like to be.

Unfortunately, the simplest version assert is about the only built-in testing function. More about that next.

Code generation

In C, macros are seriously powerful tools. I don’t accept the arguments of people that macros are ugly, type-unsafe, code-unfriendly, and un-debuggable. That is all perfectly true, but in C/C++, there are jobs which can be done either only by macros, or can be done best by macros. Inline templated functions help, but macros still fill a niche. D doesn’t have a macro preprocessor. There’s static if, static assert, templates, delegates and mixins. The big point here is that there isn’t a separate macro language you have to learn. The entire D language is available to you AT COMPILE TIME! (There are also nested functions, which help alleviate the macro problem, but that’s a different topic.) Unfortunately, D ends up being short a little functionality by removing the full preprocessor. After working with the language a little, I wanted to add a more comprehensive assert_equal(param1, param2) to my programs. This construct is very useful for unit testing, but ends up being very ugly or unhelpful without full support. In C, this function is implemented as:

#define assert_equal(param1, param2) \
  do {	 \
    if((param1) != (param2)) { \
      printf(“Test %s == %s failed.\n”, #param1 #param2); \
      printf(“%s != %s\n”, as_string(param1), as_string(param2) ); \
      assert(0); \
    } else { \
      printf(“Test %s == %s passed.\n”, #param1 #param2); \
    } \
  } while (0) 
This has the problem of evaluating the params too many times, but gets close. The important things that I want here are that the text of the test (Test obj.name == foobar passed) and the reason for the failure (bazdoo != foobar) are both printed. Also, when this code is run in a debugger or the assert(0) line fires, I want the file / line number reported to be that of the client code, not the file location of the macro. I have made several attempts at this and failed one of my three requirements. Using mixins (evaluate a string as code at compile time) is ugly. Instead of assert_equal(obj.name, “foobar”), I end up with mixing assert_equal(“obj.name”, “\”foobar\””). This code also doesn’t report the failure line correctly, and I couldn’t get a __LOCATION__ (defined as __FILE__, __LINE__) compile time variable to work. Using a normal function prevents me from reporting both the text of the test and the correct failure line. For example broken code, see the appendix.

Documenting

Built-in source documentation. I realize Java and C# and other development platforms offer this. D is nice in that this is part of the basic compiler (add one flag and you get a nice HTML file showing your documentation) and that it was designed in from the start. Further, the documentation thing isn’t some separate source file based xml parser: It’s part of the same language. (CONSISTENCY!) Add an extra trailing comment character to your comment:

/// I’m a documentation comment
/** so am I */
/++ and so am I! +/
… and the compiler does the rest. Because this is built-in to the language and is so easy to use, you’ll use it too. There isn’t a separate mark-up language you have to learn (though there are some features if you care) it’s all part of D, and D is a clean language. Using CandyDoc as part of your project gives a nice sidebar and some syntax coloring.

Automatic variable typing

autos are really, really nice. In C++, if you want to iterate across a list, you have to do some nonsense like:

for(list >::const_iterator iter = my_list.begin(); iter != my_list.end(); iter++) …
D recognizes that this is complete nonsense. The compiler already knows what type my_list is of.
for(auto iter = my_list.begin(); iter != mylist.end(); iter++) …
And because D has a for-each:
foreach(auto iter; my_list) …
Call me crazy, but that looks like something C++ should have had all along.

autos are also nice for weird functions where you just need the return value of a function.

auto object_of_some_oddly_named_return_structure = unstd.odd.count_fibwotzes();

Problems with object pointers

I had noticed earlier that D doesn’t have the -> or * operators. The language seemed to really downplay the differences between variables, references, pointers and smart pointers. I thought that it was a nice touch, since the compiler knows what each type of variable is and doesn’t need a syntactic hint. However, I got several problems trying to build my objects and later in building a vector class for my objects. D objects are all dynamically allocated – always.

my_class my_object; // This is a pointer thingie initialized to null
my_struct my_struct_obj; // this is a struct object – actual variable
my_struct *my_pointer; // this is a pointer initialized to null
my_class my_real_object = new my_class; // this is a class object ready to use

my_object.do_stuff(); // crashes
my_struct_object.do_stuff(); // fine
my_pointer.do_stuff(); // crashes, but you knew that
my_real_object.do_stuff(); // fine
The next thing that bit me was that since every class object MUST be dynamically allocated, there is no way to define that every object has an internal position vector (using class) defining its position. I had to change the vector type to struct and it was fine. I seem to remember Java works this way, and it’s an important difference I overlooked when starting.

I also had trouble checking object pointers against null. I hated this one. Because the difference between pointers, references and object variables is mildly hidden, checking an object for null is different than in C.

if(my_obj == null) { // Crashes!
That line of code calls the virtual opEqual function defined for my_obj (which might be null) Trying to grab addresses off of my_obj is just wrong.
if(&my_obj == null) { // JUST PLAIN WRONG
This won’t crash, and will never return false - & is checking the address of the reference variable my_obj, not its contents.

Apparently, the right way is to use a special is operator for this.

if(my_obj is null) { // yeah!
I also got some mileage out of ! and direct eval but I like is better.
if(myobj) {
if(!my_obj) {

Object member functions

As I build my classes, in this project particularly the vector class, I generally don’t add member functions or functionality until I’m actually using it in the client code. D’s operator overload is a bit different than C++’s because the operators are all named as functions (opMul, opAdd), instead of as operators (operator*, operator+). This adds clarity as to what the functionality being added is, instead of drawing attention to the specific syntactic sugar the language will let you use in calling. I think this naming scheme (while requiring a function name look-up to add operators) also reminds users that these functions can be used in callbacks.

Nested functions

Nested functions are cool. You can use them both as parameters for delegates and as static nested functions they are a nice way to handle simple topical tasks. In C++ I would have either done a huge block of logical statements or a separate static function placed earlier in the file. This moves the control logic away from where it is actually used. In D, you can define a little utility function (with or without access to all the variables of the parent function) and use it multiple times thereafter. One way to think of non-static nested functions is like class member functions, except instead of a hidden data pointer to an object of that class being passed to the function, there is a hidden data pointer to all the local variables of the parent function.

with keyword

Have you ever programmed in BASIC or Pascal? Did you then move to C/C++ and wonder how those programmers ever got any work done? What is it with those programmers and their dot-target fetishes? ( “.” or ”->” ) That’s right. C/C++ is completely missing the with/using keyword. Member functions alleviate some of that problem, but in the same way that C doesn’t have nested functions to let you simply and directly factor out coupled but duplicated code, C doesn’t have a way to get rid of those nasty pointers.

Here’s how it works for everyone who hasn’t seen this yet. In C/C++, you are continually typing

my_object->member_pointer-> silly_array[my_object.iterator].member_1 = …;
my_object->member_pointer-> silly_array[my_object.iterator].member_2 = …;
my_object->member_pointer-> silly_array[my_object.iterator].member_3 = …;
	… ad nauseum …
Of course, you get tired of that and use a cached pointer.
my_pointer = &my_object->member_pointer-> silly_array[my_object.iterator];
my_pointer->member_1 = …;
my_pointer->member_2 = …;
my_pointer->member_3 = …;
But honestly, that’s not any better. If you have a lot of work to do down in the elements of silly_array, that pointer is going to be thrown everywhere.

Enter with. This was an essential keyword in Pascal, and I don’t know how I’ve lived without it in C++.

with( my_object->member_pointer-> silly_array[my_object.iterator] ) {
  member_1 = …;
  member_2 = …;
  member_3 = …;
}
Now isn’t that a lot better?

Libraries

D is really cool because it is link compatible with all C libraries ever written (if they use the gcc compiler). There’s a bit of work to get the externing and DLL linking (such as Derelict does) working, but all those libraries written in the past for C will work for D as soon as someone writes a new header file (essentially) for D.

On another project, I started trying to write win32 apps using D. I found that the Phobos library std.windows was missing a lot of the win32 library. It was very simple to use the MSDN documentation to get C function prototypes for each function I needed and then add them to my project. D automatically linked to the windows libraries.

Wrapup

I really liked Digital Mars’ D Programming Language. There are some rough edges, but these are getting ground off as more people use the language. It felt easier to do what I want to do. I’m just learning it, but even with looking things up in the help, working in D felt good. When I wanted to write a function or move a block of code around, there was less boiler plate and fewer interruptions to getting the work done. The Computer Language Benchmarks Game puts D very close to C/C++ in performance, while most of the irritations which would drive people to Java/C# are removed. D is a simple clean language with all the features you’ve been dying to have. D is what C has always wanted to become and what C++ should have been from the start.

References

Installing

  • DMD.ZIP - The D Programming Language
  • DMC.ZIP - Digital Mars C (compiler backend)
  • BUD - A nice auto builder (Required by Derelict)
  • Derelict - OpenGL / SDL / * linkage wrapper library
  • SDL - Popular gaming library
  • CandyDoc - Documentation prettifier for D

Useful Sites

Other Tools Used

  • Textpad - My favorite text editor
  • Inkscape - A pretty cool vector graphics program
  • Windows XP Pro / Home
  • Wordpad / Word
  • Perforce - Free for the hobbyist. Use source control.
  • ExamDiff - Good for reviewing your changes.
  • Beyond Compare - Another diff tool. I use this at work and really like it.

My Source Files

  • Der - My game (in D)
  • Dir2 - My directory diff / sync tool (in D)
  • Upgraded D.SYN - Syntax file for Textpad, revised
  • Broken D macros - In-depth discussion of what macro should do.
Download the source file here.

About the Author

Jonathan has been programming since 1992. He saw a friend programming Hangman in GWBASIC, became addicted to the idea of global domination through control of computers, and hasn't stopped programming games yet. He has a BS-Computer Science from the University of Missouri - Rolla. Jonathan currently works for Volition writing video games for the Xbox 360. His interests include computers, computer programming, programming languages, game programming, computer games, space elevators and nursery rhymes. He is married and has 4 children who have a guinea pig.

Discuss this article in the forums


Date this article was posted to GameDev.net: 1/18/2008
(Note that this date does not necessarily correspond to the date the article was written)

See Also:
Programming

© 1999-2009 Gamedev.net. All rights reserved. Terms of Use Privacy Policy
Comments? Questions? Feedback? Click here!