  
Welcome to Ventspace! Most posts here are delayed copies of posts from the real Ventspace.
  I Believe in Static Checking II: Tests and Bugs | 
Posted - 7/26/2007 9:01:00 PM | Ok, I slipped schedule slightly. Just think of me like VG Cats -- the update days are just a formality.
I Believe in Static Checking II: Tests and Bugs
First, I'll address a comment on the last entry:Quote:| It's not that simple. You need a static type system or some other form of syntactic analysis if you want to verify your code is free of deadlock, for example. Dynamic typing can't do that. |
The trick is that static analysis can be run dynamically any time you want (barring limitations like needing source code). And a lot of these static analysis tools don't and can't make any assertions about the code, anyway. They merely attempt to detect obvious or common mistakes in code. They can't make an assertion as strong as "this will not deadlock" except for extremely trivial situations. They're still very useful tools, of course.
When it comes to application bugs, there are three basic points in time that a mistake can be detected:- Build time
- Test time
- Runtime
Not all languages/environments incorporate a proper build time, but generally speaking this is anything that involves tools that look at and process the source code (or the output binary) without doing any actual execution. Test time is where special test code is run in order to check the correctness of the code, and includes both unit testing and regression testing. (Lots of people run this step as part of their build, but it is basically a separate phase.) Lastly, we've got runtime, which is when somebody is actually using your application, whether that someone is you, QA, or a customer/client.
When it comes to bugs, we don't want to find them at runtime if we can help it. If a bug pops up at runtime, it's far too late, and in many cases, may be quite difficult or tedious to reproduce and debug. Additionally, these bugs need to be triggered by a user; they will not make themselves readily known (except for severe mistakes). Worst of all, these are the bugs that are likely to slip out to customers. Finding bugs during the build or test stages is far safer and quicker. There's far less danger of an old bug silently reappearing unnoticed, and the problems are more readily reproduced. And when they're connected to the build system, you can pinpoint with considerable accuracy what it was that caused a test to break.
Ok, so now it's fairly obvious that post-build testing is a great idea. Why is it relevant though? As I mentioned in my last entry, testing is necessary for reliably findings places where constraints break, regardless of whether you have some static checking. Tests aren't perfect, though. They only check the code that they actually hit, and they only check that code for the values they're programmed to. Getting 100% coverage from your tests is near impossible, let alone running through all of the possible states that can occur for the code. For complex situations (for example, checking that your DAG based scene graph is correctly collecting and sorting all objects to be rendered, where correctly sorted is loosely defined), simply writing the code carefully and then doing QA and debugging is likely to be cheaper and easier than any testing method. (Unfortunately, this seems to come up in games a lot.)
Despite all those problems, testing gives us a lot of opportunities to sanity check code and prevent old bugs from reappearing. The truth is, tests are for all practical purposes nothing more than customized static checks. Whether or not our code is being run (in the case of tests) or simply read and processed (in the case of static analysis tools) is just an implementation detail. Suppose Valgrind could sandbox compile and run pieces of your code in order to check thread safety. Would that really change anything? Of course not.
So with this altered definition of static checking, it'd be pretty hard not to believe in it; not doing static checking would amount to developing software by accident. Don't worry, though -- I'm not about to leave you on such a pathetic cop out. I do believe in proper static checking and static typing, regardless of what kind of test framework you've got running post-build. In other words, I prefer to catch constraint mistakes at build time rather than test time. The reasons for that preference will be the subject of the next entry. 
| |
| Monday II | |
Posted - 7/23/2007 10:56:08 PM | I had a busy day today, so today's going to be a lame cop out entry. I'll post the SlimDX status update to the thread at some vague point in the future.
Monday II
| |
| I Believe in Static Checking I: Constraints | |
Posted - 7/20/2007 7:45:18 PM | Developing a Graphics Driver III is delayed for a while until I decide exactly what I want in that entry. It won't come until at least Wednesday. Monday should be a SlimDX status update.
I Believe in Static Checking I: Constraints
This entry is actually related to the last one. I mentioned that I have some issues recommending a dynamically typed language to newbies. The truth is that it runs much deeper than that; I have issues with dynamic typing, period. I'm pretty much going to outline my way of thinking here, and explain why I take the views I do. (The title has already spoiled the ending, of course.) I am not trying to change any minds or start a language war here. What I do want is to clarify some of the basic arguments at a theoretical level about static and dynamic typing and my thoughts on them. And lastly, I should mention that this post is by no means grounded in formal computer science theory, so I apologize if I offend any of you heavy CS theory types.
First, forget types. They're not particularly important to the discussion at hand. Hopefully some of you are squirming a little already, because I started off talking about static and dynamic typing, and type systems are frequently the defining aspect of a language. As it turns out, they're not nearly as important as most people think. Second, realize that this discussion is largely independent of specific languages or even paradigms. As such, I'm going to avoid using the term "variable" in favor of the more paradigm neutral "value".
That's the preliminaries out of the way. Let's dig in.
Every program performs various operations on values. In most cases, there are constraints on these values. Constraints limit the flexibility of values and allow us to make concrete assertions about the inputs and outputs of our algorithms. Some constraints are permanent, and some are temporary. Some can be broken for short periods of time, and some are always present. For example, consider a value which represents a month. There are three permanent constraints:- This value must be an integer.
- This value must be greater or equal to 0.
This value must be less than 12.
The first in this list is what we previously referred to as the "type"; I will call it the primary constraint. It's the primary because the other constraints are meaningless without it. However, all three constraints are still of equal importance. Violating the integer constraint is no less significant than violating the less than 12 constraint. Here's an alternate method of representing a month:- This value must be a member of the set {Jan, Feb, Mar, Apr, May, Jun, Jul, Aug, Sep, Oct, Nov, Dec}
Most of you will recognize this as an enum. (Be careful though -- there is no integer conversion here.) In this definition, the primary constraint is the only constraint, and attempting to apply #2 and #3 from the other representation would be nonsensical.
Temporary constraints, on the other hand, are constraints that are applied at some point and then removed later. For example, a function might take a reference with a not-null constraint as a parameter. The calling code may well have a reference which does not have this constraint. When the function is entered, the constraint is applied to the value. That constraint is valid for the life of the function, which is also the life of that value (which is of course a copy). You'll notice that temporary constraints will always reduce the valid domain (I use the term roughly based on its mathematical meaning) for a value; if it doesn't, the constraint was redundant and can be ignored.
Constraints are a simple and intuitive concept, so I won't waste any more time on them. (Although there is plenty more to talk about if you want to flesh out the system.) Here is the key to it all, though:All* constraints can be dynamically checked. Some constraints cannot be statically checked. In particular, temporary constraints can never** be statically checked. All this talk of static and dynamic typing is just blind dancing around this simple statement of a rather obvious fact.
* Within some limitations. Constraints are themselves code that runs, and we're subject to restrictions about what we can do there.
** I'm lying. Temporary constraints can be statically checked in certain situations, but we're not going to go there. Yet.
The real advantage of a dynamic type system is that all of the constraints on a value can be unified into a single place, both in space (where in the code they are) and time (when you're told about constraint violation). You can simply write a function which can be invoked at any point to check if a value satisfies certain constraints. (And one could imagine that a language could provide all sorts of syntactic support for this.) And having all of the constraints listed in one place is certainly useful, since the conditions for a value to be valid are immediately visible. Additionally, the distinction between temporary and permanent constraints becomes irrelevant at the entry point to a function; the parameters must satisfy certain constraints, and whether or not those were permanently applied is irrelevant. Again, all constraints are equally important; if a reference's not-null constraint is broken, it doesn't matter that the constraint was temporary.
Take note of the connection between duck typing and constraints. Whether an value supports a specific operation or message simply becomes a temporary constraint that is applied. It is checked at the same time as all other constraints, too. The key to verifying the code is test driven development; as long as your tests are fairly comprehensive, you can catch obvious bugs.
A static system, on the other hand, splits the primary constraint away from the rest, because a primary constraint can usually be checked statically. (For an example of when a primary constraint can't be checked statically, consider the last time you used RTTI of any sort, especially when dealing with base and child classes in an inheritance hierarchy.) Classical static type systems attempt to enforce just the primary constraint at compile/parse time, and leave the remaining constraints to the programmer. You can see this behavior often enough, in the form of various checks at the top of a function (whether that function is logic or merely some kind of setter). Most static systems are clumsy and crude, and we're forced to hack around them fairly often. And because we can't check temporary constraints statically anyway, unit testing is still necessary to verify that constraints are being met.
So far, static constraint checking doesn't sound that great. Dynamic checking has apparently everything going for it (except some performance concerns, but that's not something I intend to look at in this series). Static checking doesn't save you the trouble of writing unit tests, and it makes code more complex. But you saw the title. Stay tuned.
| |
| Python, C#, and Beginners | |
Posted - 7/18/2007 8:14:45 PM | Python, C#, and Beginners
When it comes to recommending a language for a complete beginner to start with, the field is usually narrowed to C# and Python. Forget the details of why other languages are rejected; that's not relevant to this post. This journal entry is about something that's been bothering me for a very long time. A beginner should start with C# or Python. But which one?
The fact of the matter is that I have deep seated reservations about both languages and I don't have a strong recommendation for either. I tend to side with C#, but I'm reluctant to do it and I won't actively disagree or object to Python as a suggestion. So I decided to voice some of my concerns with the whole matter. I'm fairly interested in how all of you feel about this, so please comment if you've got anything to say in favor of either language or just in general.
C#'s an absolutely beautiful language. It's not perfect, but it's as close as I've ever seen. It really takes a lot of lessons from Java and Delphi, and working in it is a pleasant experience. Massive support from Microsoft, combined with the fact that a lot of people know and love C#, means that any time you run into problems, you've got a lot of places to go for help. And the .NET Framework supports almost any type of development, from applications to web development to Windows services and everything in between. For the most part it's blazing fast, especially if you know what you're doing. The biggest advantage of all, though, is that anyone can get the Express Edition IDE for free, and work in what amounts to the foremost professional grade development environment in the world. Combined with XNA, it's a great way to make games.
Python's a great language in a lot of ways. It's developed a fairly active community that has produced a lot of good information and libraries. Since it's been used heavily in the commercial sector, it's a mature language that is known to be usable for real life work. Learning the basics is a rather simple task, though it includes plenty of advanced functionality. Many people have learned from Python, and many people use it to great effect. There's even a completely free book online called Dive Into Python that you can use to learn. And the presence of an REPL makes experimenting with Python coding very easy. When you're reading to do games, you can move smoothly into PyGame and hit the ground running.
Both languages seem great, because they are great. So the question is, why do I have such serious misgivings about letting beginners loose in either?
C# throws a lot of information at a beginner. Consider what a beginner will typically deal with in C#:
using System;
namespace App
{
class Program
{
public static void Main(string[] args)
{
Console.WriteLine( "Woot!" );
}
}
}
That is a lot of things going on for someone who's never seen coding. That snip has 7 reserved keywords in it, and a number of them are somewhat difficult to explain. And asking a newbie to simply ignore bits of the code as magic really bothers me. They did it to me when I learned C++, and I used things like using namespace std; blindly for months. And once you start getting involved with WinForms, or really any part of the Framework, things get much more noisy and complex. Everything is great once you get over that hill and are basically familiar with the principles at play (classes mainly), but in the meantime, C# code is a confusing mix of simple statements and total nonsense.
So maybe you're thinking at this point, Python doesn't suffer that problem. The equivalent Python program is a single line:
print "Woot!"
Nothing more. The use of "print" is historic and maybe a little strange to someone new to coding, but it's a minor detail and other than that, this code is dead simple. There are a few different problems with Python though. First, although having an REPL is nice, you don't write regular code in that thing, and the actual Python IDEs suck very badly. The end result is that developing an application purely in Python can be kind of a pain, unless you're particularly fond of the Emacs or Vim environments. Second, the Python syntax is completely foreign from anything else. It won't help you understand Java, C#, or C++, which means that learning those languages eventually will involve a bit more work.
However, my deepest reservation with Python is the type system, which is dynamic typed and duck typed. I feel that the biggest pain in programming is learning that computers deal with very strict rules, and I tend to think it's better to start from more rules and then eventually relax some of them. That way, since you're more used to the restrictive rules, you tend not to outright abuse the relaxed rules. For example, arbitrarily changing the type of a variable and using it for some other purpose than when you started really bothers me. Python and most dynamic languages take the line that variable types are locked, and you're simply redirecting the name assigned to a variable. I don't buy that. While it's technically and theoretically accurate, it's a completely worthless perspective to take in terms of actually coding software. All it does in reality is confuse people about what a given variable is being used for. In general, Python is very, very lax about rules, and that tends to result in some rather grievous code being turned out by people who mainly learned to code in Python.
So at the end of it all, I don't know what to say to newbies about which one to select. I personally started in VB6, which involved a lot of behind the scenes magic, but turned out fairly simple code. And VB6 really had very relaxed rules indeed, and is a lot like Python in that regard. I had an intuition early on that this kind of behavior was "bad", and so my VB code never involved variants being thrown around or any of that stuff. (I did use duck typing, but inheritance was seriously broken so there weren't many options.) A lot of people learning VB6 turned out really horrendous code, and I expect the same exact thing to happen with people who learn with Python. That's why I usually side with C# at the end of it all, but I can't say I'm happy with it.
| |
| Why I don't like XNA | |
Posted - 7/16/2007 8:50:27 PM | Well, I think it's time to get back on track here. I'm going to try an MWF schedule this week -- daily felt far too dense to me. And if I get a lot of comments, I might even throw in a Saturday entry.
Upcoming topics are:
* Python, C#, and Beginners
* Developing a Graphics Driver III
* SlimDX Status Update
Why I don't like XNA
First, let's make sure there aren't any misconceptions. I think there's a lot of great things about XNA. It's really easy to get up and running, whether you're a beginner taking first steps, or an experienced developer doing rapid prototyping. If you are just trying to learn, or if you're just playing around and testing things, XNA is a beautiful, fun, and comfortable environment to work in. If any of that describes you, then what I'm about to talk about probably has absolutely nothing to do with you. Keep that in mind as you read.
On the other hand, if you're the sort of person who intends for other people to play your games, XNA has some rather serious problems. You can develop games in XNA and release them easily enough -- but I don't think it's worth bothering. There's too much of a disconnect between XNA and reality. If this is your intent, then what I'm about to talk about has everything to do with you.
XNA is first and foremost an API designed for the Xbox 360. There's no sense trying to avoid this fact. The reason it's this way is simple. The basic goal of XNA is to allow you to write basically identical code for Windows and Xbox, and have it run fine on both. The only way to do this is to force a least common denominator, and while the abstraction is leaky, the basic principle remains. And as a practical matter, the Xbox is the source of most of the limitations of XNA. It is a console, after all, and doesn't need to support the vast array of things that Windows does. Now of course the Xbox is capable of a few things that the PC isn't, but those aren't really exposed either. And then there's a few things that fall through the cracks just because the implementation is rather different between PC and Xbox -- occlusion queries are missing, for example.
So when it comes to making XNA based games, there are two possible alternatives. You can be developing for Windows, or you can be developing for Xbox. The two are not exclusive, of course. But no matter which one you're doing, there are some serious problems once it comes around to actually giving your game up for other people to play. Now up to this point, I've been rather vague and general. Let's get down to details.
First, the Xbox, because this is by far the more stupid platform to screw up. In order to develop for the Xbox, you need to join the XNA Creator's Club, which runs a hundred bucks a year. Fine, whatever. Paying such a small sum to develop is not a big deal at all, considering it's a tiny, tiny fraction of what a real devkit costs. (Plus unlike a real devkit, you can actually get one.) There are two basic problems though. First, everyone who is interested in playing your game must also be an XNA CC member, because that is the only way to get hold of the XNA Framework on your Xbox. So we've already excluded very nearly all casual and hardcore gamers, because neither of them are interested in paying that money just to play the indie games -- most of which aren't any good to begin with -- that might pop up. You're left with just other XNA developers, and the truth is that most of them are too busy making their own bad games to play yours. The second problem is that the XNA Framework on the Xbox is so closely linked to the dev tools that in order for anyone to play, they need your full source code and source assets. Then they need to build the game, connect their Xbox to their PC and transfer the files, and only then can they play. Update: I've been informed this is no longer true.
So ask yourself. Who is really going to bother? Have you bothered to play anyone else's XNA games on Xbox? I haven't even signed up for Creator's Club, because I'm not interested in paying to make games that no one can play anyway. Might as well stick to Windows.
XNA on Windows has its own problems, though. The only reason that these problems are not lethal to XNA games running on Windows is because, unlike on Xbox, we can actually use libraries other than XNA when running on Windows. The real irony is that the main library you end up using is, in fact, Managed DirectX. (By the way, your users now need both the DX and XNA redist packages!) And why do people end up with MDX code in their XNA games? Well, there are two reasons, and they both start with X: XInput and XACT.
Unless you're really out of things, you know what's wrong with XInput already. It's a great API, as long as you're using an Xbox controller. Not everybody has an Xbox controller. There's hundreds of non-XI compatible peripherals out there which people like to game with. I have a Thrustmaster controller I rather like. A lot of people use Logitech's Dual Shock knockoffs, or just connect an actual Dual Shock via a PS2->USB adapter. And let's not forget all the people with flight sticks, steering wheels and pedals, and all the other sorts of game controllers out there. Heck, some people might even like force feedback support once in a while. The end result is that you're forced to pull in DirectInput in order to give people the chance to play with anything other than keyboard and mouse. (This of course sucks even worse because DInput is such a miserably painful and unwieldy API. SlimDX doesn't support it yet either.)
And then there's XACT. It's not a great API by any stretch of the imagination, and the XACT Authoring Tool can be a really, really frustrating tool to work with. Put all that aside though. The real insult and killer blow is the format support. See, XACT will take your source files and shove them into these special files it has. It does a format conversion to something it can handle, and then lets you load them nicely at runtime without having to deal with decompression and decoding. XACT, like XInput, came from the Xbox. On the Xbox, it supports two compression formats. One is ADPCM, which is a simplistic lossy compression algorithm that can hit ratios up to 4:1. The tests I ran showed that a typical ADPCM file was about 4-5 times larger than a similar MP3 at 128 kbps, which is clearly far too large for music. The other format XACT supports is XMA, which is some sort of variation of WMA which can be decoded in hardware by the Xbox. Here's the catch: XMA does not work on PC, and there's no replacement. So your compression options are ADPCM, and...no, just ADPCM. Using XACT on PC, 5 minutes of music will probably cost you around 20-25 MB. Better hope your players have high bandwidth connections. Your best alternative is DirectSound coupled with either Ogg Vorbis or DirectShow.
It's also the case that I have some technical objections to design decisions made for XNA. I'm not going to elaborate on those points, though, because they pale in comparison to what I've discussed. (Although the occlusion queries thing really bothers me. They're generally considered fundamental to modern graphics engine design.) Right now, XNA simply isn't viable for anything except learning or prototyping.
Will things be fixed? I don't know. In principle, it shouldn't be too big of a problem to patch XACT up to be more useful, but there's rumblings of a new API, XAudio 2, that will be used for XP, Vista, and Xbox. If that's the case, then XACT may be nothing more than a stopgap. XInput won't ever gain support for non-Xbox controllers, and I don't think it should. As for the Xbox, I have no idea which of those restrictions are technical and which are intentional business decisions. All I know is that I won't be writing games in XNA anytime soon.
| |
|
| S | M | T | W | T | F | S | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | | 17 | | 19 | | 21 | 22 | | 24 | 25 | | 27 | 28 | 29 | 30 | 31 | | | | |
OPTIONS
Track this Journal
ARCHIVES
October, 2009
September, 2009
August, 2009
July, 2009
June, 2009
October, 2008
June, 2008
May, 2008
April, 2008
March, 2008
February, 2008
January, 2008
December, 2007
November, 2007
October, 2007
September, 2007
August, 2007
July, 2007
June, 2007
May, 2007
February, 2007
|
