Jump to content

  • Log In with Google      Sign In   
  • Create Account

MJP's Last Stand

DXSAS Controls Library

Posted by , 12 November 2008 - - - - - - · 479 views

Already posted this in the XNA CC forums, but I figured "why not post it here too?"

Hey everyone. While working on a ModelViewer tool for my current project, I developed a handful of WinForms controls for editing the shader parameters of my material effects. I ended up making them use the DXSAS 1.0 UI Annotation (as documented here) so that I wouldn't have to change the existing annotations I'd added for XSI ModTool, and also so that they'd be somewhat useful outside of this one project.

Anyway I'd planned on making some sort of sample or article on this topic using the code I developed, but at the moment I've just got so much to do and I don't think I'll have time to get around to it. I decided I'd share the code here, in case anyone might find it useful to just plug into their project or to serve as a base for something else. I figure that if any of you guys are like me, you'd rather be optimizing shader code than spending time messing around with TextBox's :P. So if anyone's interested, I've uploaded the code here. It's just the classes themeselves, no library project or sample project or anything like that. You should be able to just add them to any project that has the XNA Framework assemblies referenced, it doesn't depend upon anything aside from that (and WinForms, obviously).

If you want to see what they look like, there's a picture of my ModelViewer below. That's them in the panel on the right-hand side, right below the combo boxes for picking the MeshPart and the Effect. From the top to the bottom you've got a Slider, ColorPicker, Slider, Numeric, FilePicker, and ListPicker. As you can see they're nothing fancy at all, but they get the job done.

If you have any questions, feel free to ask here or email me at the address listed in the ReadMe.txt inside the zip.

HDR Rendering Sample

Posted by , 29 September 2008 - - - - - - · 554 views

Well I finished up my HDR Rendering Sample, and Rim posted it on xnainfo.com. He's still working on getting all the content stuff worked out for the site, so you'll have to make do with reading the write-up in .doc format. :P

If anyone catches any mistakes/bad practices/misleading comments please let me know, as I'd be very interested in correcting such things.

A Preview of My New Sample

Posted by , 24 September 2008 - - - - - - · 642 views

Over the weekend I finally got around to coding up one of the samples I've been meaning to do: a full HDR pipeline in XNA that uses LogLuv encoding. It's pretty neat: it lets you switch between fp16/LogLuv, and also lets you switch on and off multisampling and linear filtering fordownscaling so you can see the results of linear filtering when LogLuv is used. Plus I don't think there's any samples out there that show you the basics of HDR with XNA, so I think it's something people would find useful.

I haven't finished with my write-up yet, but I did take a sweet screenshot:

Look at the bloom on that teapot! And yes I know...teapots and Uffizi Cross have been done in just about every other Direct3D sample. What can I say, I'm not that creative.

LogLuv Encoding for HDR

Posted by , 16 September 2008 - - - - - - · 846 views

I originally posted this over on the xnainfo.com blog, but I've decided I like the entry so much that I'm going to shamelessly rip myself off. Enjoy!

Designing an effective and performant HDR implementation for my game's engine was a step that was complicated a bit by a few of the quirks of running XNA on the Xbox 360. As a quick refresher for those who aren't experts on the subject, HDR is most commonly implemented by rendering the scene to a floating-point buffer and then performing a tone-mapping pass to bring the colors back into he visible range. Floating-point formats (like A16B16G16R16F, AKA HalfVector4) are used because their added precision and floating-point nature allows them to comfortbly store linear RGB values in ranges beyond the [0,1] typically used for shader output to the backbuffer, which is crucial as HDR requires having data with a wide dynamic range. They're also convenient, as this it allows values to be stored in the same format they're manipulated in the shaders. Newer GPU's also support full texture filtering and alpha-blending with fp surfaces, which prevents the need for special-case handling of things like non-opaque geometry. However as with most things, what's convient is not always the best option. During planning, I came up with the following list of pro's and con's for various types of HDR implementations:

* Standard HDR, fp16 buffer
+Very easy to integrate (no special work needed for the shaders)
+Good precision
+Support for blending on SM3.0+ PC GPU's
+Allows for HDR bloom effects
-Double the bandwidth and storage requirements of R8G8B8A8
-Weak support for multi-sampling on SM3.0 GPU's (Nvidia NV40 and G70/G71 can't do it)
-Hardware filtering not available on ATI SM2.0 and SM3.0 GPU's
-No blending on the Xbox 360
-Requires double space in framebuffer on the 360, which increases the number of tiles needed
* HDR with tone-mapping applied directly in the pixel shader (Valve-style)
+Doesn't require output to an HDR format, no floating-point or encoding required
+Multi-sampling and blending is supported, even on old hardware
-Can't do HDR bloom, since only an LDR image is available for post-processing
-Luminance can't be calculated directly, need to use fancy techniques to estimate it
-Increases shader complexity and combinations
* HDR using an encoded format
+Allows for a standard tone-mapping chain
+Allows for HDR bloom effects
+Most formats offer a very wide dynamic range
+Same bandwidth and storage as LDR rendering
+Certain formats allow for multi-sampling and/or linear filtering with reasonable quality
-Alpha-blending usually isn't an option, since the alpha-channel is used by most formats
-Linear filtering and multisampling usually isn't mathmatically correct, although often the results are "good enough"
-Additional shader math needed for format conversions
-Adds complexity to shaders

My early prototyping used a standard tone-mapping chain and I didn't want to ditch that, nor did I want to move away from what I was comfortable with. This pretty much eliminated the second option for me off the bat...although I was unlikely to choose it anyway due its other drawbacks (having nice HDR bloom was something I felt was an important part of the look I wanted for my game, and in my opinion Valve's method doesn't do a great job of determining average luminance). When I tried out the first method I found that it worked as well as it always did on the PC (I've used it before), but on the 360 it was another story. I'm not sure why exactly, but for some reason it simply does not like the HalfVector4 format. Performance was terrible, I couldn't blend, I got all kinds of strange rendering artifacts (entire lines of pixels missing), and I'd get bizarre exceptions if I enabled multisampling. Loads of fun, let me tell you.

This left me with option #3. I wasn't a fan of this approach initially, as my original design plan called for things to be simple and straightforward whenever possible. I didn't really want to have two versions of my material shaders to support encoding, nor did I want to integrate decoding into the other parts of the pipeline that needed it. But unfortunately, I wasn't really left with any other options after I found there were no plans to bring the support for the 360's special fp10 backbuffer format to XNA (which would have conveniently solved my problems on the 360). So, I started doing my research. Naturally the first place I looked was to actual released commercial game. Why? Because usually when a technique is used in a shipped game, it means it's gone trhough the paces and has been determined to actually be feasible and practical in game environment. Which of course naturally led me to consider NAO32.

NAO32 is a format that gained some fame in the dev community when ex-Ninja Theory programmer Marco Salvi shared some details on the technique over on the beyond3D forums. Used in the game Heavenly Sword, it allowed for multisampling to be used in conjuction with HDR on a platform (PS3) whose GPU didn't support multisampling of floating-point surfaces (The RSX is heavily based on Nvidia G70). In this technique, color is stored in the LogLuv format using a standard R8G8B8A8 surface. Two components are used to store X and Y at 8-bit precision, and the other two are used to store the log of luminance at 16-bit precision. Having 16 bits for luminance allows for a wide dynamic range to be stored in this format, and storing the log of the luminance allows for linear filtering in multisampling or texture sampling. Since he first explained it other games have also used it, such as Naughty Dog's Uncharted. It's likely that it's been used in many other PS3 games, as well.

My actual shader implementation was helped along quite a bit by Christer Ericson's blog post, which described how to derive optimized shader code for encoding RGB into the LogLuv format. Using his code as a starting point, I came up with the following HLSL code for encoding and decoding:

// M matrix, for encoding
const static float3x3 M = float3x3(
0.2209, 0.3390, 0.4184,
0.1138, 0.6780, 0.7319,
0.0102, 0.1130, 0.2969);

// Inverse M matrix, for decoding
const static float3x3 InverseM = float3x3(
6.0013, -2.700, -1.7995,
-1.332, 3.1029, -5.7720,
.3007, -1.088, 5.6268);

float4 LogLuvEncode(in float3 vRGB)
float4 vResult;
float3 Xp_Y_XYZp = mul(vRGB, M);
Xp_Y_XYZp = max(Xp_Y_XYZp, float3(1e-6, 1e-6, 1e-6));
vResult.xy = Xp_Y_XYZp.xy / Xp_Y_XYZp.z;
float Le = 2 * log2(Xp_Y_XYZp.y) + 127;
vResult.w = frac(Le);
vResult.z = (Le - (floor(vResult.w*255.0f))/255.0f)/255.0f;
return vResult;

float3 LogLuvDecode(in float4 vLogLuv)
float Le = vLogLuv.z * 255 + vLogLuv.w;
float3 Xp_Y_XYZp;
Xp_Y_XYZp.y = exp2((Le - 127) / 2);
Xp_Y_XYZp.z = Xp_Y_XYZp.y / vLogLuv.y;
Xp_Y_XYZp.x = vLogLuv.x * Xp_Y_XYZp.z;
float3 vRGB = mul(Xp_Y_XYZp, InverseM);
return max(vRGB, 0);

Once I had this implemented and worked through a few small glitches, results were much improved in the 360 version of my game. Performance was much much better, I could multi-sample again, and the results looked great. So while things didn't exactly work out in an ideal way, I'm pleased enough with the results.

Don't Cast Function Pointers (Unless You Really Know What You're Doing)

Posted by , 15 August 2008 - - - - - - · 480 views

Yet another Evil Steve-esque journal entry that I can instantly whip out when needed, instead of typing out a detailed explanation

*Note: Any information here only strictly applies to Microsoft Visual C++, I'm nowhere near experienced enough in any other compilers to comment on them. And as always, if anything is wrong please let me know so I can correct it. I'm much more interested in correctness than in pride. [smile]

**Extra Note: I compiled and ran the sample code on Visual C++ 2005, your results may differ for other versions. However that should just serve to show you how unpredictable and nasty this stuff can be.

In my last real journal entry I mentioned how the For Beginners forum typically has all kinds of Bad Win32 Code floating around. Well it doesn't just stop there...it's also brimming with Really Bad C++ Code, and even Completely Horrifying C++ Code. For this entry I'll be tackling something so scary it keeps me lying awake at night: function pointer casts in C++. Anybody who's used C++ for more than a month knows how dangerous casting can be, yet we still see it commonly used as a tool to "simply make the compiler shut up". Casting function pointers is even more dangerous, and we're going to talk about why.

At the lowest low-level, a function pointer is exactly that: a pointer to a function. Functions have an address in memory where they're located, and a function pointer contains that address. When you want to call the function pointed to by the function pointer, an x86 "call" command is used an execution starts at the address contained in the pointer. However there's more to calling a function then simply running the code: there's also the function parameters, the return value, and other information that needs to be stuck on the stack (like the return address, so execution can return to the calling function). How this exactly happens is determined by the calling convention of a function. The calling convention specifies how parameters are passed to the function (usually on the stack), how the return value is passed back, how the "this" pointer is passed (in the case of C++ member functions), and how the stack is eventually cleaned up when the function is finished. This entry from Raymond Chen's blog has a good summary of the calling conventions used in 32-bit Windows. As Raymond puts it on his blog, a calling convention is a contract that defines exactly what happens when that function is called. Both sides (the caller and the callee) must agree to the contract and hold up their respective ends of the bargain in order for things to continue smoothly.

So it should be obvious by now that function pointers are more than just an address: they also specify the parameters, the return value, and the calling convention. When you create a function pointer, all of this information is contained in the pointer type. This is a Good Thing, because it means that the compiler can catch errors for you when you try to assign values to incompatible types. Take this code for instance, which generates a compilation error:

#include <conio.h>
#include <iostream>

using std::cout;

int DoSomething(int num)
return num + 1;

int DoSomethingElse(int num, int* numPtr)
int result = num + *numPtr;
return result;

typedef int (*DoSomethingPtr)(int);
typedef int (*DoSomethingElsePtr)(int, int*);

int main()
DoSomethingPtr fnPtr = DoSomethingElse;
int result = fnPtr(5);
cout << result;


return 0;

Look at that, the compiler saved our butt. We were trying to do something very bad! But of course since this is C++ we're talking about, the compiler does not have the final say in what happens. If we want, we can say "shut up compiler, and do what I tell you" and it will happily oblige. So go ahead and change the first line of main to this:

DoSomethingPtr fnPtr = (DoSomethingPtr)DoSomethingElse;

and watch the compiler error magically vanish. But now try running the code, in debug mode first. And look at that, an Access violation. Why did we get an access violation? Well that's easy: we called a function that expected two parameters on the stack. However we were using a function pointer that only specified one parameter. In other words, we violated the contract on our end. The callee however dutifully followed the contracted, and popped two parameters off the stack. The second value on the stack happened to be NULL, which caused an exception to be thrown when we tried to dereference NULL.

This is actually a pretty "nice" error. The exception happens right when we call the function, so naturally the first thing we'd do is go back to where we called the function and see what went wrong. So in the event of an accidentally erroneous cast, we'd figure it out pretty quickly. But of course, that's not always the case. Try compiling and running in release mode. And look at that: no crash! However that return value of "-1559444344" sure does look funky...clearly we weren't so lucky this time. Now instead of a nice informative crash, we have a function that just produces a completely bogus value. Maybe that value could be used for something immediately after and we'll notice it's bogus, maybe we won't notice until we've made 8 calculations based on it. Either way something down the line will get screwed up, and the chance that you'll trace it back to a bogus function pointer get slimmer and slimmer every step of the way.

But wait...the fun doesn't end there. Casting problems can be more subtle than that...as well as more catastrophic. Let's try this nearly-identical program instead:

#include <conio.h>
#include <iostream>

using std::cout;

int __stdcall DoSomething(int num)
return num + 1;

int __stdcall DoSomethingElse(int num, int* numPtr)
int result = num + *numPtr;
return result;

typedef int (*DoSomethingPtr)(int);
typedef int (*DoSomethingElsePtr)(int, int*);

int main()
DoSomethingPtr fnPtr = (DoSomethingPtr)DoSomething;
int result = fnPtr(5);
cout << result;


return 0;

Look at that, we're actually pointing our function pointer to the right function this time! This should work perfectly, right? Right? Go ahead and run it. And what to do you know, it spits out the anticipated result! But no go ahead and press a key to let the program close up and....crash. A strange one too...access violation? At address 0x00000001? No source code available? What the heck code are we even executing? A look at the call stack shows that we're somehow executing in the middle of nowhere!

So how did this happen? Once again, we're crooks who violated the contract. The functions were declared with the calling convention __stdcall, which specifies that the function being called cleans up the stack. However our function pointers were never given an explicit calling convention, which means they got the default (which is __cdecl). This meant we put our parameter and other stuff on the stack, we called the function, the function cleaned up the junk on the stack by popping it off, and then when we returned the main function once again cleaned junk off the stack. Except that since the junk had already been cleaned up already, we instead completely bungled up our stack and wound up with an instruction pointer pointing to no-man's land. Beautiful. For those wondering, the correct way to declare the function pointers would be like this:
typedef int (__stdcall *DoSomethingPtr)(int);
typedef int (__stdcall *DoSomethingElsePtr)(int, int*);

And of course, the even smarter thing to do would have been to have no cast at all, since then the compiler would have caught our mistake and whacked us over the head for it.

By now I hope I've gotten my point across. If I haven't, my point is this: don't cast function pointers unless you're extremely careful about it, and you absolutely have no choice. Type safety exists for a reason: to save us from ourselves. Make use of it whenever you can.

EXTRA: On a somewhat related note, sometimes what you think is a function pointer isn't really a function pointer at all. For instance...what you get back when you pass GWLP_WNDPROC to GetWindowLongPtr. Yet another reason to be careful with function pointers.

OpenGL 3.0

Posted by , 14 August 2008 - - - - - - · 291 views

After taking a look at the newly-released spec and the cries of rage and disappointment that ensued...all I can say is "thank god I don't target Mac and Linux".

Posting WM_DESTROY is *not* how you destroy a window

Posted by , 12 August 2008 - - - - - - · 663 views

In general, the Ask Beginners forum is positively rife with all kinds of Bad Win32 Code. I don't really fault the noobies for writing it...especially since chances are extremely good that they copied it from some crappy tutorial or used some outdated boilerplate code. Heck, for a long time the default Visual C++ Win32 code contained a function-pointer cast! Yikes.

The target of my ranting for this entry is code that posts the WM_DESTROY message as a mechanism for terminating their message loop and destroying their window. The problem is that WM_DESTROY is a notification, not a message you're supposed to be passing around. Notifications are messages that tell your program when something important has occurred...in this particular case, this "something important" is the fact that your window is being destroyed. In other words it's like the Windows way of saying "hey your window's about to be gonzo, so do whatever you have to do before we pull the plug!". This is made clear by the documentation, which explains the exact circumstances which cause your message handler to receive the message.

Now the important distinction here is that WM_DESTROY is *not* what's destroying your window, it's simply the natural reaction that occurs when a window is destroyed. It's the thunder to the lighting, if you will. Therefore it really doesn't make any sense to use it to destroy a window, since it doesn't destroy anything! It's just a notification, not a mechanism for destroying windows. Using the previously mentioned analogy, posting a WM_DESTROY message is a bit like making a thunderclap and excepting lightning to strike. The *actual* way of destroying a window is to call DestroyWindow and pass along the handle of the window.

The real problem here is that from the point of view of the noobie, doing this does work as excepted....or at least it seems to. Typically their message handler will have something like this in it:


Which means that WM_QUIT will get sent, and the user will break out of their message loop. And of course the main window is destroyed along with the program. The catch here is that it didn't go away because of anything the programmer explicitly requested, it happened as an automatic part of cleaning up the thread and the process (since, as many lazy programmers already know, Windows will clean up after you).

So you might be thinking now, "so what's the big deal then if it gets destroyed anyway?". Well it's really just about enforcing good programming practices. Leaking windows and handles is just plain bad practice, especially for programs that typically run for long periods of time and consume resources. Doing things in ways other than what the Windows API Documentation specifies is also a great way to make sure your app becomes one of the many Poorly Behaved Windows Programs that become the scourge of a user's PC once they upgrade to a new version of Windows.

So the moral is: before you use something, read the documentation. In most cases, it should be pretty clear whether or not you're using it right.

Working With Unicode in the Windows API

Posted by , 16 March 2008 - - - - - - · 658 views

So the issue of Unicode and character sets is one that seems to come up quite a bit in the For Beginners forum (and elsewhere). Usually someone who is new to Windows programming will make a thread saying that the compiler barfs when it gets to their "MessageBox" call, and has no idea how to deal with it. Therefore I spent a lot of time explaining what their problem is and how to fix it, and usually this involves me explaining how the Windows API deals with strings. This happened again today and it resulted in me writing up an explanation that I rather like, so I've decided to post it here so that I can simply link people to it when necessary. I hope that those who read it find it useful, and get on their way with Windows programming. If there's anything hideously wrong or that you think could be added, please let me know.

Also before or after you read this, you may want to consult the official documentation at MSDN regarding Unicode and character sets. All the information I've explained is in there, you may just have to go through more reading to get to it. You can also find some more good information on the topic in this blog entry by Joel Spolsky.


The Windows API supports two kinds of strings, each using two types of characters. The first type is multi-byte strings, which are arrays of char's. With these strings each glyph can either be a single byte (char) or multiple bytes, and how the data is interpreted into glyphs depends on the ANSI code page being used. The "standard" code page for Windows in the US is windows-1252, known as "ANSI Latin 1; Western European". These strings are generally referred to as "ANSI" strings throughout the Windows documentation. The Windows headers typedef the type "char" to "CHAR", and also typedef pointers to strings as "LPSTR" and "LPCSTR" (the second being a constant pointer to a string). String literals for this type simply use quotations, like in this example:

const char* ansiString = "This is an ANSI string!";

The second type of string is what is referred to as Unicode strings. There are several types of Unicode, but in the Windows API "Unicode" generally refers to UTF-16 encoding. UTF-16 uses (at least) two bytes per glyph, and therefore in C and C++ the strings are represented as arrays of the type wchar_t (which is two bytes in size, and therefore referred to as a "wide" character). Unicode is a worldwide standard, and supports glyphs from many languages with one standard code page (with multi-byte strings you'd have to use a different code page if you wanted something like kanji). This is obviously a big improvement, which is why Microsoft encourages that all newly-written apps use Unicode exclusively (this is also why a new Visual C++ project defaults to Unicode). The Windows headers typedef the type "wchar_t" to "WCHAR", and also typedef pointers to Unicode strings as "LPWSTR" and "LPCWSTR". String literals for this type use quotations prefixed with an "L", like in this example:

const wchar_t* unicodeString = L"This is a Unicode string!";

Okay, so I said that the Windows API supports both the old ANSI strings as well as Unicode strings. It does this through polymorphic types and by using macros for functions that take strings as parameters. Allow me to elaborate on the first part...

The Windows API defines a third character type, and consequently a third string type. This type is "TCHAR", and its definition looks something like this:

#ifdef UNICODE
typedef WCHAR TCHAR;
typedef CHAR TCHAR;

typedef TCHAR* LPTSTR;
typedef const TCHAR* LPCTSTR;

So as you can see here, how the TCHAR type is defined depends on whether the "UNICODE" macro is defined. In this way, the "UNICODE" macro becomes a sort of switch that lets you say "I'm going to be using Unicode strings, so make my TCHAR a wide character." And this is exactly what Visual C++ does when you set the project's "character set" to Unicode: it defines UNICODE for you. So what you get out of this is the ability to write code that can compile to use either ANSI strings or Unicode strings depending on a macro definition or a compiler setting. This ability is further aided by the TEXT() macro, which will produce either an ANSI or Unicode string literal:

LPCTSTR tString = TEXT("This could be either an ANSI or Unicode string!");

Now that you know about TCHAR's, things might make a bit more sense if you look at the documentation for any Windows API function that accepts a string. For example, let's look at the documentation for MessageBox. The prototype shown on MSDN looks like this:

int MessageBox( HWND hWnd,
LPCTSTR lpCaption,
UINT uType

As you can see, it asks for a string of TCHAR's. This makes sense, since your app could be using either character type and the API doesn't want to force either type on you. However there's a big problem with this: the functions that make up the Windows API are implemented as precompiled DLL's. Since TCHAR is resolved at compile-time, the function had to be compiled as either ANSI or Unicode. So how did MS get around this? They compiled both!

See, the function prototype you see in the documentation isn't actually a prototype of any existing function. It's just a bunch of sugar to make things look nice for you when you're learning how a function works, and tells you how you should be using it. In actuality, every function that accepts strings has two versions: one with an "A" suffix that takes ANSI strings, and one with a "W" suffix that takes Unicode strings. When you call a function like MessageBox, you're actually calling a macro that's defined to one of its two versions depending on whether the UNICODE macro is defined. This means that the Windows headers has something that looks like this:

__in_opt HWND hWnd,
__in_opt LPCSTR lpText,
__in_opt LPCSTR lpCaption,
__in UINT uType);
__in_opt HWND hWnd,
__in_opt LPCWSTR lpText,
__in_opt LPCWSTR lpCaption,
__in UINT uType);
#ifdef UNICODE
#define MessageBox MessageBoxW
#define MessageBox MessageBoxA

Pretty tricky, eh? With these macros, the ugliness of having two functions is kept reasonably transparent for the programmer (with the disadvantage of causing some confusion among Windows newbies). Of course these macros can be bypassed completely if you want, by simply calling one of the typed versions directly. This is important for programs that dynamically load functions from Windows DLL's at runtime, using LoadLibrary and GetProcAddress. Since macros like "MessageBox" don't actually exist in the DLL, you have to specify the name of one of the "real" functions.

Anyway, that's basically a summarized guide of how the Windows API handles Unicode. With this, you should be able to get started with using Windows API functions, or at least know what kinds of questions to ask when you need something cleared up on the issue.


The above refers specifically to how the Windows API handles strings. The Visual C++ C Run-Time library also supports its own _TCHAR type which is defined in a manner similar to TCHAR, except that it uses the _UNICODE macro. It also defines a _T() macro for string literals that functions in the same manner as TEXT(). String functions in the CRT also use the _UNICODE macro, so if you're using these you must remember to define _UNICODE in addition to UNICODE (Visual C++ will define both if you set the character set as Unicode).

If you use Standard C++ Library classes that work with strings such as std::string and std::ifstream and you want to use Unicode, you can use the wide-char versions. These classes have a w prefix, such as std::wstring and std::wifstream. There are no classes that use the TCHAR type, however if you'd like you can simply define a tstring or tifstream class yourself using the _UNICODE macro.

October 2016 »

161718192021 22