  
Welcome to Ventspace! Most posts here are delayed copies of posts from the real Ventspace.
  Finalizers and DirectX 11 | 
Posted - 6/29/2009 8:45:52 PM | This story was actually posted a week ago on Ventspace!
Long story short: I'm not so sure about them.
The reason SlimDX does not, by and large, include finalizers is because of the threading issues involved. You can enable multithreaded devices in D3D 9 and 10, but it's not really a good idea in most cases and we don't see any reason to handle it differently. Direct3D 11, however, is making a big push for multithreaded rendering, and that means we could credibly enable finalizers just for it. There is a single threaded flag, but we can detect that and avoid finalizing those objects. But there's more to it than that.
Let's recap. Finalizers generally only make sense as part of the IDisposable pattern. The idea is that you expose Dispose to allow deterministic destruction of unmanaged objects, but in the case that you don't call Dispose -- deliberately or accidentally -- the finalizer comes around and does it for you. This is mostly designed for Windows handles and the like, which I think is why it fails to really take threading into account. Because most DirectX objects are not free threaded, that goes to hell in SlimDX. We chose to eschew the finalization part of the pattern completely, and nobody really seems to mind.
From a strictly technical standpoint, enabling finalizers for 11 would be easy. It would take less time than writing this, actually. Josh pointed out that it's a break in consistency, but I believe we can at least make an argument for it being part of the multithreaded support. The Windows API Code Pack provides finalizers, which tends to support the idea we should do it. However, there are a number of issues I'm concerned about, and for June we almost certainly won't have them. I'm not sure we ever will.
At some level, finalizers exist to patch up programmer error. While this is not necessarily a bad thing, I have some misgivings about it. SlimDX doesn't clean up after you, but it does provide some very powerful facilities for finding out what didn't get cleaned up, and we even have some options for expanding that functionality. Finalizers will tend to interfere with that tracking in unpredictable (but innocuous) ways. If you have repeating leaks, maybe some will get finalized and maybe some won't. The ObjectTable won't be able to provide you with reliable results. We could provide a configuration option for it, like we do for exceptions -- but that's not a good idea. Basically if you're exposing a choice to the user, it means you didn't have the balls to actually make the choice and you're passing the buck. I think Raymond Chen said something similar at one point, in not quite the same language.
There's also a question of interop objects. We can track the single threaded flag in SlimDX code, but not in externally created objects. Sure it might be possible to get the device for an object and check its caps (I haven't checked), but even if that works sometimes, it's extra overhead and I'm guessing that the number of special cases involved is prohibitive. We could assume all external objects are multithreaded, and then someone will come complaining about a decision that we can no longer reverse, and which is impacting their performance negatively. Supporting interop smoothly is a critical use case for us, not least because it's a major differentiating point against XNA. It has worked very well so far and I'm not thrilled about potentially breaking it now.
I guess I was lying at the beginning, actually. I am fairly sure about them. In fact, I'm fairly sure we won't have them. They are looking to cause more problems than they solve, and nobody seems to care about the problem they're supposedly solving in the first place.
| |
| The SlimDX Architecture, Part 2 | |
Posted - 6/26/2009 1:45:35 PM | Why did this post show up so much sooner at Ventspace? Because Wordpress is a lot better than GameDev at this sort of thing. Also their PageRanks are much higher for some reason.
Now we get into the real substance of things. In Part 1, I discussed some of the historical elements that went into the design of ComObject and ObjectTable, but stopped short of explaining the current system. In this segment, I'll cover the essentials of how things work today. You should also read COM and SlimDX, Part I, as it does a better job than me of covering some of the details. (Remember, these are test drafts for new documentation.)
As SlimDX and its userbase grew, a number of cracks began to show up in the design. At the heart of things was that a lot of new objects were created every time you tried to do something, which meant a lot of things to dispose. Sure we had no problems with events chaining endlessly, but even with the leak tracker it was a real pain in the ass to make sure everything was handled properly. As a result, we also had to be very, very clear about where objects were created. With an object such as Texture that allows you to get its parent Device, a Device property was out of the question. It had to be GetDevice(), or the amount of stealthy object creations could get out of hand very quickly -- and for no apparent reason.
Consider what a function like GetDevice() has to do. The native API gives us an IDirect3DDevice9*, which we have to convert into a managed reference to Device. The obvious way to do this -- the way MDX worked -- is to create a new Device object, passing it the pointer to be used internally. The device has already had the COM AddRef() function called to adjust the reference count, and the reference count is restored when Dispose() is called. It's a simple and efficient strategy to implement, and creates a huge amount of allocations. We referred to this as the GetDevice() != GetDevice() problem at the time, because we didn't implement equality operators and so the new objects didn't even compare to the same thing.
Josh had the initial idea to address this. I never did research this in depth, but when you ask for Visual Studio to add a reference to a COM object, it creates something called a COM callable wrapper. Apparently the wrapper uses some kind of table of objects; I don't really know the details. The basic idea, though, was to construct a table of mappings from native pointers to managed instances. By switching over to factory construction for most objects internally, we were able to make the table work.
ObjectTracker became ObjectTable, and maintains a Dictionary of IntPtr -> ComObject. Internally, when we call a method that always creates a new object (eg D3DXCreateTextureFromFile), the new object is added to the table. For methods that might return an object that is already in the table, we check the table to see if it contains the pointer we just got. If it does, the table calls Release on the object and then returns the original managed instance. When an object is disposed, we call Release and then remove it from the table. It's set up this way in ordeto maintain a very important invariant:
SlimDX is always responsible for at most one reference on a COM object.
By maintaining this invariant, lifetime management is much easier to control. Instead of having to match up a large list of AddRef and Release calls to make sure everything comes together cleanly, all you have to do is Dispose the object once, usually the first reference to it that you saw. Stuff like GetDevice() became properties, which return the original instance without any caveats and without creating memory management headaches. Actually, that's not quite true -- there is one caveat if you're doing interop. COM and SlimDX, Part II has a proper explanation.
There is one more issue outstanding with the ObjectTable, which Josh will write up for me soon. For part 3 of my series, I'll step away from this stuff and take a look at DataStream.
| |
| Binaural Audio Follow-up | |
Posted - 6/25/2009 12:34:40 PM | When I last posted, the demo video was the Barbershop recording. Although it's an excellent video, it's not ours and the only media I had to show that was actually by us was an eight second clip that was not terribly impressive. That is no longer the case!
Grab headphones and listen!
The game is previewed, with the real in game sound, near the end. But listen to the whole thing, and remember that it works in real time on pretty much any current platform. With any luck, we'll have an SDK ready for Q1 2010, maybe even earlier.
Oh, and that's still part 1. If you're interested in animation and if you think that what NaturalMotion did was pretty cool, you'll be in for a big treat later on.
| |
| Highlight: AI War | |
Posted - 6/23/2009 11:28:27 AM | This post appeared last week on Ventspace. In case you haven't noticed, most posts are there first.
Arcen Games recently released their first game, AI War. It's available directly from them, along with a demo version You can also download it via Impulse, which is Stardock's equivalent to Steam. ("Steam Powered", "Impulse Driven". I see what they did there!)
I am highlighting (not reviewing) this game mainly because it's the first selling game, indie or otherwise, that is actually built on top of SlimDX. It's been out for some time, and we've been aware of it, but for some reason we only just noticed that it's an actual title rather than just another hobbyist pseudo-game. Visually it's not terribly sophisticated, but that's simply not the point. The UK Gamer blog has a review, for some more discussion of the actual gameplay.
What is interesting about AI War is, well, the AI. Since I haven't actually played the game yet, I can't actually tell you how effective it is. But as a strictly technical piece of work, it's really quite impressive. The guy behind the game is Chris Park, who has his own blog. He's written two entries about the AI so far (One, two), and more are coming. There's lots more tech information over there though (and also discussion of game design), so I'd recommend skimming the blog. It's similar to a post-mortem, except he's only talking about the positives so far. That's a hint, Chris!
I'm hoping to see more games rolling out on top of SlimDX. XNA's done a lot to further the case for managed games in general, but with support for the new 10 and 11 APIs, 64 bit, and all sorts of other things, I'm eager to see what people can do with our tools as well.
Chris wrote a response to this post that I like to read. Possibly because it's very flattering about my project. I just wanted to clarify that his is the first selling game built on SlimDX. We're a lot more popular for non-games work, actually.
| |
| Revolution, Part 1: Audio | |
Posted - 6/19/2009 12:15:59 PM | A friend of mine has been working very hard on a project which involves two revolutionary technical innovations. I am not usually the kind to care for revolutions, but I honestly believe that both of these are really, really significant. I'm actually hoping to join up and help make these things into a proper product, in fact. These are things you have not seen in games, and he has it working in prototype form on a mobile platform. That's unusual in this industry.
Here, listen to this Youtube video while reading. You need headphones for this, which is why the tech is launching on the iPhone first. The video is not ours, but it is a very good demonstration. Just listen and read. Remember, it doesn't work without headphones -- if you don't have any, it might actually be worth reading this only after you've found some.
The underlying principal is binaural recording. (This is not the revolutionary bit, and has been around for quite a long time.) Games have had 3D audio for ages, so that is in itself nothing to get excited about. Current 3D audio basically works by modifying channel volumes for playback of a mono sound in order to simulate a 3D space. It works alright if you have a 5.1 setup, but it's not terribly effective in stereo and in general the effect is a bit weak. Binaural recording, however, is a method of recording sounds with a pair of microphones and an actual head model that attempts to produce a stereo sound that simulates what our ears hear. You need headphones because of the recording methodology, and if you're listening to the video I linked, you're probably spazzing out right now.
There is a catch to all this, which is that nobody can synthesize it. The sound is recorded by physically placing it relative to the head, so you can't go back later and place it at an arbitrary location. (Some people have pointed out that there are processors and algorithms that try, but they are expensive and don't really work well.) That's essentially why it's never showed up in games --although headphones-only isn't a thrilling restriction, either. Still enjoying the barber?
Here is his binaural recording. (It's 8 seconds, just pause the barber.) There's one key difference, though. That's not a binaural recording of a sound being moved in front of a recording head. It is done in real-time. (This is the revolutionary bit.) This friend of mine has figured out how to do it. The original implementation worked very well but required a lot of memory and processing power. But the current system is efficient enough to fit on the iPhone. I've seen and heard the demo, working in real time off an iPod Touch. It works well enough to make your skin crawl, like those scissors are probably doing right now if you're still listening to the barber.
I can't really say too much about how he's pulled it off, because we think it's kind of a big deal. You'll see iPhone game releases with the technology later this year, and hopefully by early to mid next year we'll be licensing an actual SDK for whatever platform you might care to use. We're fairly confident this is technology people will want, and hey, it wouldn't hurt to forward this post around the office.
| |
| The SlimDX Architecture, Part 1 | |
Posted - 6/19/2009 8:47:13 AM | This post appeared on Ventspace on Wednesday.
As I mentioned earlier, there's very little documentation on how the internals of SlimDX work, and why they work that way. This is important documentation, both for the team and for anyone looking to make changes or add things. I do intend to add full documentation; however, how to lay it out properly isn't quite clear to me. I'm writing this series of blog posts as a dry run to get a sense for what topics are significant and what ordering would make the most sense. I'll try a chronological approach first, and discuss the history of how things used to work.
The original SlimDX code was not intended to be the all-encompassing library it is now. It was just a simplified copy of Managed DirectX, and so it used the same design. The only real difference was that I took out the event stuff. I wasn't using them anyway, due to Tom Miller's post about how they were basically kind of dangerous. I wrote SlimDX by tearing MDX out of a game I'd written the previous semester, and retrofitting it with SlimDX until it worked. That game was simple and used a simple resource management model, so there simply was no need to examine the behavior farther. Everything got disposed in its proper way, and that was that.
Needless to say, this isn't a very nice approach to things in the managed world. There was an early attempt to re-enable finalizers, but of course most of DX isn't thread safe and this approach was abandoned quickly. I came up with a stopgap, which was to incorporate registration of every object via the common base class into an ObjectTracker.
The common base class was one of the design cues taken directly from MDX, although the idea is relatively straightforward. The original class was even called the same as MDX, DirectXObject. It's not in the code quite from the beginning, but at r6 it's damned close. It eventually gets renamed to BaseObject and then finally ComObject in r320. It has been a consistently useful class to have, by incorporating a lot of the basic boilerplate of handling COM objects in a way that makes sense in the managed world. It's a relatively small role for a long time, but later in the SlimDX lifecycle this class becomes one of the most complex and subtle ones we have. The current version is truly terrifying. More on that later.
ObjectTracker was kind of a cool tool, because it meant that at any point in time you could see exactly what objects were outstanding and where they had come from. We even hooked the process exit to report the entire list to the debug spew, so you'd get a nice list of what you leaked. It had a public toggle, so you could turn it off and on as desired. It was added in July of 2007 (r110) and the functionality is still there today.
However, although the functionality is still included in SlimDX, the ObjectTracker is long gone. It was replaced in late February 2008 (r389) with the ObjectTable. ObjectTable is nearly identical to the old ObjectTracker, but with one fundamental behavior that changed everything. The current SlimDX design is essentially defined by the interactions between the ObjectTable and ComObject classes, which took a very long time to get right. Every little detail is critical, and if you don't understand or pay attentiont to those details -- and we've noticed at least a few people don't -- you're liable to break the library.
I'll go more into depth on those two and why they work the way they do in the next entry.
| |
| It's Dee-Ex Damnit | |
Posted - 6/17/2009 2:01:32 PM | P.S. We’re dropping VS 05 support for the samples, since a number of them didn’t work properly anyway. If you really needed those samples and Express 08 isn’t an option, let us know ASAP.
| |
| SlimDX's Hobbyist Curse, and Open Source Blessing | |
Posted - 6/14/2009 1:20:27 PM | This post appeared on Ventspace yesterday.
There are two attributes that fundamentally differentiate SlimDX from MDX and XNA:
- It's a "hobbyist" project.
- It's open source.
The Windows API Code Pack mucks this up a bit, but more on that another time. I've already opened my mouth about it once without knowing what I was talking about, so I'll actually do my due diligence this time.
Our status of being "hobbyists" is a problem for some people. This is a trend I usually observe on the MS newsgroups and Stack Overflow. Usually the thinking is that XNA and MDX, by virtue of being written by Microsoft, are better supported, less likely to die out, and generally safer to use. The snarky thing to do at this point is to ask how MDX 2.0 worked out for them, but generally speaking if you're trying to pull someone to your side, making fun of them is not terribly effective.
Of course the people who are open minded, who know us, etc don't have this problem. It's well known in our circles (GameDev, mostly, and XNA to a lesser extent) that we're professional developers who know what we're doing. There's a fair argument to be made that you don't want the other kind of user, but it's difficult not to bristle a bit when you read a post by someone who doesn't want to use the library because it's not official.
Our open source status is in many ways the saving grace, though. When I started the project around January of 2006, it was a very easy decision that the source code had to be readily accessible and usable. I settled on the MIT license because it essentially sets no restrictions on what you can do with the library. Not only are there no restrictions on usage, but you can basically do whatever you want with any of the source code. The idea was, of course, that people could add the functionality they needed (since the original had almost none) and everybody would be happy with their own variations on SlimDX.
SlimDX's scope grew from a little personal wrapper to a full replacement project, but that element of customization has been quite powerful. A number of our users, especially the corporate ones like Ventuz, use customized builds that work better for them, for whatever reason. It also allows them to fix bugs as soon as they're identified, instead of waiting for us to get around to it. (Sadly this usually means we don't find out about those bugs.)
And, as I've always told some of the doubters, the source code is hosted on Google servers with every revision ever, and requires just a few very common bits to build. Even if we "hobbyists" give it up, so what? It's not like MDX 2.0, where the binaries time bomb and vanish, and the samples dissolve into thin air. You can even implement DirectX 12 or whatever it turns out to be, and you'll still benefit from all of our architecture and basically stable codebase. At least, you could in theory. In practice, there's one problem -- nearly all of the internal documentation is in our heads. I'll be working to change that over the coming weeks, right here. And yes, new documentation will show up as well.
| |
| SlimDX Compile Times | |
Posted - 6/11/2009 1:40:21 PM | This post was taken from Ventspace.
When I was doing the November 08 release of SlimDX, I had to do a lot of builds to get it right. As a result, I was keenly aware of compile times, which sucked to be quite frank. On my main development workstation, a full rebuild of Release x86 was around ten minutes. Debug was a bit quicker, but still around eight minutes. I don't have a very powerful machine at home, but when I tested the fairly powerful system at work, it still didn't go terribly fast. It turns out that you really don't have to accept the long builds that naive project layout creates.
I've just now done a rebuild of Release x86 in under five minutes. A few relatively simple steps have cut it in half; Debug builds are almost down to a third. The changes, although tedious, were relatively simple and mechanical, and didn't really involve any increased risks for bugs. That's a very good thing with large changes, especially when trying to attack problems that are arguably not THAT important.
The first change was to roll out precompiled headers across the library. Our PCH file is extremely simple; it just includes all of the system and DirectX headers, and a handful of library headers. These headers are fairly massive though, and we saw benefits from the PCH basically immediately. I'd avoided PCH at the beginning because I've always found it to be a pain in the ass, but the trade off was worth it. It helps that the headers in there don't really change.
The other trick was a bit more subtle. See, Visual C++ compiles all of its input .cpp files to .obj files in the project intermediate directory. However, it does not preserve any input directory structure into that folder. So in SlimDX, we had direct3d9/Device.cpp, direct3d10/Device.cpp, directinput/Device.cpp, and so on. In the intermediate directory, these became Device1.obj, Device2.obj, Device3.obj, etc. That's not compiler-automatic, either -- it has to be configured in the project build settings per file. The IDE is capable of doing that step automatically, but when it comes to actually doing the build, the compiler process has to be spun up separately for each individual file. Compared to the amount of compilation required for any given C++ file, this is a fairly substantial cost, especially when multiplied by a hundred or so.
Once we renamed the files so they no longer conflicted in a flat directory structure, we got the current build timings. In fact, it takes almost as much time to link as it does to compile. (I'm sad that linking is still a single threaded operation.) Not bad, all in all.
| |
| C# and DirectX 11 -- Yes you can! | |
Posted - 6/9/2009 7:22:37 PM | This post was taken from Ventspace.
Along with Windows 7, several beta APIs to go along with the new OS have been available. One of the biggest is Direct3D 11, and although we were dabbling with it before, what was previously available was strictly experimental. Although it is by no means stable, SlimDX now enjoys full support for Direct3D 11! Thanks to Mike Popoloski for doing an insane amount of work, again.
Right now, the proper support is only in the Subversion repository. However, our June release should be out shortly and that will have a Windows 7 companion DLL with the D3D 11 support. Direct2D and DirectWrite are in there as well. Now keep in mind that all of these APIs are beta, and our wrappers are unstable as a result too. You'll also have to translate the C++ samples in the DX SDK for the time being, as we don't have our own samples yet. We're working on that, and hopefully we'll have a few samples out later in the year.
There's a lot of cool new stuff in 11. First off, it supports feature levels, just like 10.1. Although you'll have to have Vista SP2+ to run 11 based apps, they will run on any WDDM-compatible hardware, which should include the entire gamut of Direct3D 9 and 10 hardware from all major manufacturers (and I don't just mean the big three). It also has full multithreaded rendering support -- not the faked granular locking that 9 and 10's multithreaded devices have, but proper synchronization. This is supported even on the lower feature levels, so if you can stand to give up XP and pre-SP2 Vista as targets, you might be in line for some big performance benefits. I found a really nice blog post that goes into a lot more detail.
There are a number of new shader types, but basically the entire API has been divided into separate draw and dispatch pipelines. The heart of the dispatch pipeline is the compute shader, which serves the same purpose as OpenCL, CUDA, and so on. In other words, it provides support for general purpose GPU (GPGPU) computing. This is a big deal for high performance computing on the Windows platform, since CUDA is NVIDIA-specific and OpenCL support is mythical at best. I'm eager to see some high performance applications which are powered by managed code and then go straight to the GPU for real heavy lifting. (This will support DX 10 class hardware by final release, but support is not available yet.)
SlimDX has got you covered on all of this. These APIs are expected to be final with the launch of Windows 7, which means that our fourth quarter release will have full, non-beta support for 11. (No more separate DLL at that point.) We'd like our implementation to be as good as it can be, and for that, we need users to see how it works out and to let us know about anything we're missing! Sorry Microsoft, but when it comes to DirectX on the .NET platform, we have every intention of being the de facto standard.
| |
|
| S | M | T | W | T | F | S | | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | | 10 | | 12 | 13 | | 15 | 16 | | 18 | | 20 | 21 | 22 | | 24 | | | 27 | 28 | | 30 | | | | |
OPTIONS
Track this Journal
ARCHIVES
October, 2009
September, 2009
August, 2009
July, 2009
June, 2009
October, 2008
June, 2008
May, 2008
April, 2008
March, 2008
February, 2008
January, 2008
December, 2007
November, 2007
October, 2007
September, 2007
August, 2007
July, 2007
June, 2007
May, 2007
February, 2007
|
