  
Welcome to Ventspace! Most posts here are delayed copies of posts from the real Ventspace.
  Developing a Graphics Driver II | 
Posted - 6/21/2007 9:13:35 PM | Ok, I missed a lot of days. It's been a bit of a rough week. XNA entry comes tomorrow. By the way, I have a GameDev article coming up soonish. I'll probably make a lot of noise once it actually shows up. This particular article is about some incredibly awesome work I did late last year.
Developing a Graphics Driver II
In part 1, I briefly covered how things are structured. This entry is all about debugging the driver. Like I said, the driver lives partly in usermode and partly in kernel mode. As a practical matter, it's convenient to be able to access either part at will. In other words, a usermode debugger isn't enough -- we need a full kernel debugger. And because kernel debugging freezes the entire machine completely when you hit a breakpoint, it's absolutely necessary to have two machines. One runs all the programs, and the other debugs it. You can connect the two in any of several ways; Firewire is the most convenient. Serial works, if you want to suffer that, and USB might work, with catches that I am not familiar with. The computers had firewire ports, so I used them.
Sadly, VS does not support kernel debugging. I have no idea why not. The tool of choice for most people who need to do Windows kernel debugging is WinDbg. This program is a psychotically powerful debugger, with a terribly akward and irritating UI frontend attached. It's also kind of slow. Still, it does the job, and setting it up isn't too bad. Once you have the two machines connected, you need to set up the slave for kernel debugging. Once that's done, you can fire up WinDbg on the master, and boot the slave. WinDbg will automatically establish a connection when the machine comes up. (Note that things don't need to happen in this order. WinDbg can connect to an already running machine.) After that, it functions like a normal debugger, except that everything on the machine is being debugged. Any process or module that invokes an int 3, the breakpoint interrupt, will be caught by WinDbg. There is one catch, though; the OS can't recover from a driver invoking a breakpoint. So if you accidentally run a driver build with debugging stuff enabled but without a debugger attached, there's a good chance you'll hard lock the machine.
I'll set breakpoints in the driver as necessary to inspect what I'm trying to debug. When a breakpoint is hit, it's pretty much like debugging in VS. All the same information is available, albeit in a much worse interface. It's particularly important to make sure that WinDbg knows where to find debug symbols. Sources of symbols include the PDB generated by the build, and Microsoft's public symbol server. With those correctly set up, I can see correct call stacks through the NT kernel and the driver. WinDbg also knows how to find the driver's code files from those symbols, so when a breakpoint is hit, it can open the relevant code and point at what's going on. (Again, in a much worse way than VS. We're more at the WinDiff level of "prettiness" here.) What I don't usually get is symbols for the actual application being debugged. That's not surprising; we have access to a lot here, but debug builds of games plus all symbols, let alone code, are not really part of that. So I can see what the application is doing externally, but not what is going on inside its head. It's an interesting role reversal, actually. It's also shown me that a lot of games -- even major commercial AAA+ titles -- behave rather badly with respect to the driver.
Most of the hard work is really in isolating the conditions that cause a bug to occur, and closing in on the source of the problem. Once you know why something has gone wrong, it's usually a fairly trivial change. Not always, of course. Occasionally, it's a real pain, especially when dealing with a badly behaved or cruel application that hits a soft spot, or expects certain behavior where no such behavior was guaranteed. (And of course, it worked on the small set of hardware the game developer has, which may not even be NVIDIA based.)
I wrote this up fairly quickly. Feel free to ask questions, but be aware that there are very constricting limits on how much I can say. Don't let that stop you from asking; just don't be disappointed if I don't provide an answer.
| |
| Friday I | |
Posted - 6/15/2007 9:39:53 PM | Fridays are Youtube days.
Today's entry is The Boy Who Cried Ninja. It's awesome.
| |
| Slick New Feature in SlimDX | |
Posted - 6/14/2007 12:26:39 PM | Yesterday's entry went up late, so check it out if you haven't already. Today's is going up early to make up for it.
Slick New Feature in SlimDX
The idea for this came up last night during a conversation with CodeImp. It's not implemented yet, but it should be fairly simple to write, and quite useful. I'll have to see exactly when it goes in; hopefully I can add it into the SVN this weekend.
So the basic problem is to handle a missing DX runtime. When .NET apps can't find a DLL that they want, they generate a FileNotFoundException. That kind of makes sense. The problem is that the error doesn't have any useful information, and to the user it just looks like the developer screwed up the packaging. And since DLLs are delay loaded, this exception can come up at a rather akward point, especially in the case of tools and other non-games who don't immediately start rendering things. Normally this shouldn't happen because your installer should have set up the runtimes correctly, but in the case of a damaged install or just an app that doesn't have an installer, it can be nice to do a check in the application.
SlimDX is adding functionality to do an explicit check for DX. I haven't nailed down the function names for this yet, but there will probably be one for each major component of DX. Essentially what these functions will do is to probe the relevant component of DX, forcing the DLL to load when called. This is done for each specific DLL and the exception is caught separately, and then wrapped as an InnerException of a subclass of DirectXException that gives some more detail about the problem. So for example, a call to CheckDirect3D could raise D3DNotFoundException in case DX9 is missing, or D3DXNotFoundException in case the specific version of D3DX is missing. You can catch those exceptions and give your user a meaningful description of the problem and solution. You can also define a precise failure point for your app in the event that something with the machine's setup is broken, rather than failing at some arbitrary point in the future when the runtime goes looking for a missing DLL.
It's this sort of thing that really makes me feel good about the SlimDX project. First of all, these new features serve as a useful differentiating and advertising point. I can point to it when someone asks what makes SlimDX different from MDX and it doesn't require long winded explanation. More importantly, though, it reflects a strong desire to adapt to developers' needs. That's inherent in everything from the design changes we've mdae to the license of the code itself. Hopefully, as the library matures a bit further, that will pay off for us and everyone else who's interested in the project.
| |
| I Bought a Zune | |
Posted - 6/13/2007 9:32:04 PM | [EDIT] I would like to point out that this is my 11000th post on GameDev. Yes, eleven thousand.
I Bought a Zune
Yep. Been looking to get a new player for a while, and I've sworn off iPods. I was fond of the Zune already, and didn't find anything that seemed better for a reasonable price. To make a long story short, I love this thing and would absolutely buy it again.
The plus points first. The Zune itself is absolutely magnificent. It's light and reasonably sized. The screen is great, so much nicer than on the iPod. I don't know about colors so much, but it's larger and better utilized. The navigation works great too. I was always ambivalent about the scroll wheel. The Zune has a D-pad that follows your classic "start slow then get faster" pattern. The speeds are very well done, and sort of ramp up rather than snapping from slow to fast. It takes a little bit to get used to the speeds and timings, but once you do, it works great. It has the added touch that while you're scrolling at high speed, the screen shows the letter you're at. So you can watch that rather than trying to read one of the entries as they whoosh past, or stopping. There's all sorts of other little touches in navigation too -- the controls on Zune are 2D rather than 1D, which opens up some interesting tricks. When I'm viewing all the albums under a specific artist, I can hit left or right to switch to the previous or next artist. I don't have to back up to the previous menu level. That's really handy. Oh, and it has an FM tuner. Maybe I can listen to Car Talk for once. (No recording, sadly. Blame the RIAA and MS' lack of balls.)
The Zune software is great too. It's got a few weirdnesses where I preferred WMP 11. (I don't like iTunes, WinAmp, etc.) There's one key thing though that places the Zune software way the hell ahead of everything else on the market. You know the progressive search behavior that WMP 11 has? It searches as you type in the current list, and if it doesn't find it there, it opens up the search to the entire library. Well, Zune does that with the Zune store. It's a ridiculously fast way of finding music to download from the store. I can pull up songs, artists, albums, whatever on demand. iTunes is completely eclipsed by how easy it is to work with.
I decided to go with the subscription service for music. Basically, the deal is all you can eat for $15/mo, as long as you keep paying. I think the device wants to refresh its licenses against the internet every 30 days, but I'm not sure. So I picked up a 3 month subscription for just under $45. I've downloaded about 4.7 GB of music from the Zune store so far. I think that probably puts me somewhere in the thousand song range. Off iTunes, that would be about a thousand dollars. That sum of money would keep my subscription going for five and a half years. And I can pull stuff down at whim. I'd normally only buy songs I already like. Here, I'm literally just dragging entire albums at random onto my Zune from artists who I've been meaning to listen to for a while (U2, for example). And with 30 GB of space and no interest in music, I can keep going for a rather long time. I've slowed down drastically in downloading stuff, since I've got the things I really wanted -- and I've still got 23 GB left.
Of course, things are not perfect. Well, one thing is perfect. The Zune itself is pure brilliance, and I have no complaints. None at all. Coming from someone who likes to identify the faults with anything and everything, that's serious praise. I guess the battery life is not what some people would like it to be, but I don't need it to survive more than 7 or 8 hours. I bought a USB wall charger, and both trains and planes have power sockets nowadays. As for the wireless, screw it. I haven't tried to use it and I may not try for a long time. We all know it's basically worthless. Accessories are rather thin as well, but I prefer generic accessories anyway; ones that connect to the headphone out or the USB rather than the proprietary slot.
The out-of-box experience for the Zune software sucks. You know why? Because it doesn't work out of the box. I did as it asked, and everything worked fine right up until I tried to download music. Not so much. I had to follow these instructions to fix it. I should probably be a lot more mad about this than I am, but being a technical person, this kind of speed bump doesn't bother me that badly. There's one really glaring omission here, though, which I am not too happy about. As far as I can tell, the Zune software does not have the ability to stream directly off the Zune. So if you want to play music while it's hooked up to a computer, that computer needs to pull the files locally and play them. I'm hoping I've missed some option here, but either way, this is not good. (On the bright side, I can stream directly from the store online without downloading the files, so that's something.) Lastly, there's no Linux or Mac support worth mentioning, but of course you probably knew that.
The store isn't quite perfect, either. First of all, the subscription service does not give you free access to everything, just nearly everything. The glaring omissions? I have no Metallica or RHCP right now, and I can only get them legally via MS points. And Beatles isn't available at all. This upsets me, and I will probably go ahead and P2P the files. (Relax, I have the damn CDs, just didn't bring them.) I guess if the recording companies won't play ball, there's nothing to be done. Also, while it doesn't bother me much, there appears to be no video or podcast stuff in the store, at all. None. It's just music. While I understand that Apple's relationship with Disney* gives them a lot more leverage than Microsoft has, you'd think one of the most significant software companies on the planet could negotiate something. Or maybe they're just dragging their feet on the software side. I don't know.
I like the Zune. From a purely hardware perspective, I don't understand how anyone could prefer an iPod. It's absolutely no contest. (Unless you like crappy expensive speaker systems which can only talk to iPods, I guess.) I like subscription music. I like the Zune software. I just wish they hadn't done some of the really boneheaded stuff -- let me stream my damn songs!. It's so easy to fix, too. No new hardware releases, or anything. They can push firmware updates automatically, and they can push software updates automatically. So they need to DO it, and the sucker in me is hopeful that they will. Lucky for me, I enjoy massive hard drives and a relatively small collection of songs, so it's not something that is a huge problem for me. It's a stupid problem though, and it really mars the experience.
* As a result of the Pixar acquisition, Steve Jobs is now the single largest shareholder in Disney Corporation.
| |
| Blast From Our Graphics Past I | |
Posted - 6/12/2007 3:52:06 PM | Blast From Our Graphics Past I
Most of us graphics people spend our time living on the bleeding edge of technology. New papers, new techniques, new hardware, and new effects are our obsession. Personally, I've always found the past fascinating. What was the bleeding edge 15 years ago? Or, a slightly different question: What did people think the bleeding edge was 15 years ago? What did they predict would be the state of the industry 5 years ago?
I was skimming some presentations today, and there's some really rather interesting stuff from way back. The history relating to Microsoft and DirectX after the Windows era began is particularly colorful. It's no secret that I'm a huge fan of Direct3D and the Windows platform today, as well as the efforts MS is undertaking now and in the future. Looking back, though, there were some painfully hilarious missteps. Today's flashback is all about Microsoft Talisman.
Reading the wiki page is enough to get all the facts on the matter (all the ones I know about, anyway). Still, the magnitude of how misguided the whole thing was might not sink in immediately. Talisman was built around one basic assumption, which was that memory bandwidth would quickly become the ultimate bottleneck in consumer computer graphics. The goal was to drastically reduce the required memory bandwidth, apparently by using some kind of psychotic recomposition model. Basically, if something hasn't changed since the last time you rendered it, don't render it again; simply blit it over to where it needs to be. Parallax was done via affine transforms, until the error was great enough that a 32x32 block of pixels needed to be updated again. (Note that these blocks exist in 3D, for various objects, so occlusion is going on.) There was also some extra cleverness for patching together the blocks without problems with seams. No framebuffer, either; all this was generated as the monitor drew. Naturally, the application had to be structured in a very specific way to accomodate this sort of rendering pipeline.
I don't know how this sounded in 1996, but right now it sounds totally insane. I think it's not a bad guess that it sounded insane back then too, because by the time Talisman was getting ready to go, modern 3D accelerators appeared (the Riva 128 presumably), solved the bandwidth problem by simply throwing large amounts of on-card and on-chip memory at the problem, and tore Talisman to shreds for performance.
So, total failure?
It's funny how useful things can come out of even the most misguided project. In this case, we owe a couple things to Talisman:
* Texture compression, and more recently, Z buffer compression. These are both fairly vital for modern graphics applications. Disabling Z compression by accident really sucks, because your app will suddenly get much slower for no apparent reason. (Especially under heavy multipass situations.) And texture compression is a no-brainer.
* Tiled rendering. The Xbox 360 uses predicated tiled rendering, because the framebuffer does not generally fit on the 10 MB DRAM chip which has been set aside for it. So the screen needs to be split into pieces and rendered one piece at a time.
* Anisotropic texture filtering was brought to the forefront by Talisman.
* Not useful per se, but BeginScene and EndScene were added into DX here.
The next entry in this series is yet another Microsoft failure, and SGI is in the mix too. Some of you already know what I'm talking about.
| |
| Monday I | |
Posted - 6/11/2007 10:44:25 AM | It is clear to me that Monday mornings are kind of bleh. I much prefer waking up at 10am instead of 6am. So every Monday morning, I will post a picture here. Could be anything.
| |
| Developing a Graphics Driver I | |
Posted - 6/9/2007 3:03:52 PM | Well, I've finished my first week. I want to write about what it's like to work on developing a graphics driver. First, we need to cover some basic architecture. Then I'll talk about the actual process. And remember, I'm on the DirectX driver team, so I'm going to focus on that. (I looked at OpenGL too, but I'm less clear on the architecture.) Not surprisingly, most development is currently focused on Windows Vista. We need to bring the Vista driver up to par with the XP driver, both in terms of correctness and performance.
If you take a D3D application, it's actually sitting on top of several layers. The first layer is the D3D runtime itself. What is underneath that changes somewhat between XP and Vista. In XP, D3D is on top of a kernel mode driver (KMD), which interacts with the HAL and a miniport driver (also kernel). There's a spliting of responsibilities between the various pieces. D3D essentially accumulates and validates information, and then forwards it onto the kernel driver at appropriate times. (Draw calls, present calls, etc.) The miniport driver handles low level communication details. Everything else is handled by the main kernel driver. The driver is very much on its own here, and has a relatively direct connection to the application.
In Vista, things shifted around and the splitting changed. The main graphics driver no longer lives in the kernel, but rather in user mode as a normal DLL which D3D loads for you. D3D comunicates with this layer through a series of calls and callbacks. There's an equivalent, separate layer for OpenGL. Both the OGL and D3D user mode drivers communicate with a common kernel mode layer. The KMD interacts with Vista's backend graphics scaffolding. In particular, Vista takes over on two fronts: resource management and GPU scheduling. In XP and earlier, the driver was solely responsible for deciding how allocations and deallocations were placed and managed, as well as how DMA was handled and when the GPU did what. All of those responsibilities have been taken over by the OS* now. Vista provides the scheduling, virtual memory, etc this way. The driver negotiates with Vista over that, and it's worth noting that direct pointers are no longer manipulated, because physical addresses are not always available to the driver. (Quite rarely, actually.) Kernel mode switches in this setup only happen when information needs to be communicated down to the kernel layer; this happens only when the user mode command buffer is full, when a present happens, or a couple other situations which force the KMD to become involved.
It should be clear by this point that the interactions between the various components are more complex under Vista. It shouldn't be a surprise, of course -- virtual memory, scheduling, etc weren't going to come for free. For the most part, the existing NVIDIA driver architecture seems to have fit in quite well; I assume that the designs of the ATI and NVIDIA drivers were both major factors in the design of the new Windows Display Driver Model (WDDM, formerly LDDM for Longhorn). Admittedly I don't know what things looked like pre-Vista, but right now it all looks fairly sane. The behavior of the driver isn't that different between XP and Vista; the main difference is that under Vista, some of the pieces aren't written by us anymore.
One particularly uncomfortable question is whether or not Vista is inherently slower than XP when it comes to graphics. That's a sticky discussion, and like all performance related discussions, it's hideously complex and intertwined. All I can really tell you is that Vista is for the most part slower than XP right now -- benchmarks by hardware sites have established that quite soundly. There are many issues to be worked through, both in our drivers and in Vista itself. We have plenty of tuning to do, and MS has some mistakes to fix. I suspect that things will get much, much better after Vista SP1 is released, thanks to work on the parts of both companies. (Also, I can't share the expected timeline for that. Sorry.) Things are getting better every day and will continue to improve.
That's enough for now. In the next part, I'll discuss the actual process of working on the driver.
* This is described in internal documentation under the sub-heading: "All Your Bufs are Belong to OS".
| |
| Stop! Internship time. | |
Posted - 6/7/2007 3:52:54 PM | So as some of you may know, I'm currently working for NVIDIA. I was hired as an intern on the DirectX driver team, and started on Monday. Just getting settled in now, and I'll mainly be doing performance tuning work for the Vista driver.
Warning: the rest of this post will read like an HR/marketing recruiting presentation.
This place is amazing. First of all, it's Silicon Valley, so the weather is pretty much permanently nice. The NVIDIA campus is fairly compact, consisting of 6 buildings. There's a cafeteria, which is quite good and seems to have a solid variety. The food isn't free, but everybody gets a $5/meal credit (excluding breakfast, I think). I eat lunch for about $0.50 a day. I may start having dinner more often here too. The food services are run by Aramark, who I happen to like. (They were running the food services at school as well.) There's actually a bus that goes direct from my place to NVIDIA and back, but the latest bus in the morning leaves at 7:30 and arrives at 7:45, and in the evening leaves at 5:30 and arrives at 5:45. These are, of course, not natural times of day for an engineer, and nobody is around when I come in. I will be obtaining a car shortly, though that has been a huge headache. More on that some other time.
The culture is very much a relaxed, open environment, much like Microsoft was. Of particular interest is that there are no offices here. (Everybody at MS has an office except for some of the HR people.) Everybody has a cubicle. I'm told the CEO merely has a wall and a table. I don't mind it, except that with an office you can use speakers for music, which doesn't really work in a cube farm. I'm sharing my cube with another intern, but I believe all the full time guys get their own. I have two machines, an XP machine where I work and a Vista machine where stuff actually runs. I remote debug the Vista machine from the XP machine. There's a phone too, not that I ever use it.
Then of course there's the engineering. There's a particular dedication to care and quality here. Everyone strives to do their job and do it well. That's been emphasized over and over, and it's clearly quite fundamental to everything that happens here. Apart from that, I've had the privilege of seeing the details of how the hardware and drivers work, which has been extremely enlightening, even though I've just scratched the surface. Naturally I can't really talk about it. It's really quite stunning though, how beautifully well done the unified driver is. It's not simple, and it's not hack free, and it's not small. Still, the immense challenge it handles is just incredible. It's a single codebase that works across at least nine operating systems, with every GeForce NVIDIA has ever made (which is probably a couple dozen), and with all the versions of OpenGL and Direct3D over the years.
The interns are treated like any new hire here. That means I'm doing real work -- in particular, fixing the performance of a rather high profile game on Vista. Of course, it also means no hand holding. There are no tutorials here people. I have several million lines of code, and a bunch of architectural documentation. Doesn't bother me, but I and the other interns are all expected to take our bugs and figure out what's going on with relatively little guidance. There's a vast number of in house tools for doing all sorts of different things (like, a dozen or more), and there's internal docs to get you up to speed. These tools are seriously awesome. And after hours, plenty of employees like to stay and game and eat dinner here rather than going home.
I could get used to this.
[EDIT] I almost forgot. These are the expensive cars I've seen so far:
* Porsches (Boxster, Carrera, 911 Turbo)
* Corvettes (Stingray, C3, C4, C5, C6)
* Lots of high end Mercedes
* Ferraris (F430)
* Aston Marton V12 Vanquish
* Bentley Continental GT
* Dodge Viper
* Jaguars (don't know models)
* Lamborghini Gallardo
| |
|
| S | M | T | W | T | F | S | | | | | | 1 | 2 | 3 | 4 | 5 | 6 | | 8 | | 10 | | | | | | 16 | 17 | 18 | 19 | 20 | | 22 | 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 | | | | | | | |
OPTIONS
Track this Journal
ARCHIVES
October, 2009
September, 2009
August, 2009
July, 2009
June, 2009
October, 2008
June, 2008
May, 2008
April, 2008
March, 2008
February, 2008
January, 2008
December, 2007
November, 2007
October, 2007
September, 2007
August, 2007
July, 2007
June, 2007
May, 2007
February, 2007
|
