Archived

This topic is now archived and is closed to further replies.

_goat

Lets make DirectX

Recommended Posts

Okay, I really don''t mind if you flame me for this. I think it''s a silly idea as well. Useful, but silly. My question is, how did they make DirectX/OpenGL, and how do you make your "own" version of them. Interfacing with the hardware, using drivers, yeah. But how? I''m not fazed by ASM, nor using drivers, but there is very little information on this topic out there. And by "out there", I mean Google. :] Thus, I turn to you good people, to tell me why I should just use one of the existing ones. Oh, if you''d help me with my question, well, I''d appreciate that too. - Cheers I''''m not easily impressed. Ooh! A blue car!

Share this post


Link to post
Share on other sites
Oh yeah. I put this under Graphics Programming rather than DirectX because we wouldn''t be working with DirectX. We''d be starting from scratch. Writing all the code ourselves, which means 3D graphical mathematics. Which means this thread.

Just so you don''t get angry. :]

I''''m not easily impressed.
Ooh! A blue car!

Share this post


Link to post
Share on other sites
You can design your own 3D API and implement it in SOFTWARE (ie, on the CPU) using C++/ASM or whatever.

Getting IHVs to support your new API is not feasible. (Hey nVidia, wanna write a driver for my l33t3DAPIversion 0.01a ?)

Writing a software renderer for the purposes of learning/experimental however, is a worthy goal.

Share this post


Link to post
Share on other sites
If you want to do programming where you interface directly with the hardware you should either get into console programming or try and get a job with one of the PC 3D hardware vendors writing drivers. You won''t get the information you need to do it yourself (even if it was feasible in principle, which it''s not) on the PC. The hardware vendors are pretty protective of the details of how their hardware works.

Share this post


Link to post
Share on other sites
Guest Anonymous Poster
Dreamcast is a nice console to try, as the hardware interface is pretty well documented in KOS (Kallisti OS), and it''s easy to run homebrew software on it

Share this post


Link to post
Share on other sites
in this thread I asked a similar question... I was told that DirectX, OpenGL, and the GDI all interface directly with the device drivers and that in theory you could duplicate what they do by reading the Microsoft DDK (which I guess is a guide for manufactures to make Windows device drivers) and learning how to interface with the drivers... this obviously would only work on Windows

let me show you how it would go down with post-its attached to the top of pens.

Share this post


Link to post
Share on other sites
Tz tz tz,what shame!The more the years pass the more people forget about software rendering and it''s something SO FREQUIN COOL!Good to see some people are still intressted in that.

"You losers better learn...NOONE CONTROLS OUR GOD DAMN LIFE!!!" - MANOWAR

Share this post


Link to post
Share on other sites
you should check out the Andre Lamothe''s latest book: Tricks of the 3D Game Programming Gurus. He writes a 3D software rasterizer from scratch (well you start with the directdraw7 framebuffer so you dont have to write video card drivers ...) At the end you have a full API that handles textures( 8 and 16 bit ), lighting, shadows and animations. He shows you all the steps to build the 3D pipeline and its very good book so far ( im near page 600 out of 1700!)

So well if your interested in that kind of stuff, its a book that you''ll definitly like

Share this post


Link to post
Share on other sites
quote:
Original post by _goat
Interfacing with the hardware, using drivers, yeah. But how? I''m not fazed by ASM, nor using drivers, but there is very little information on this topic out there. And by "out there", I mean Google. :]


You''d need the precise mapping of all registers and ports used by the target 3D chipset. And that information is classified, as it is considered trade secret by the respective manufacturers. Your only possibility to get it, is to reverse engineer the original drivers. At least on nvidia, ATI is a little more open with their lowlevel specs, I think. Still, you''d probably have to sign a heavy NDA (and pay lots of k$).

Share this post


Link to post
Share on other sites
First of all, I wonder why I put a question mark next to the the topic. It''s not a question is it?

Second of all, let me make sure I''ve got this right. The hardware has drivers, written by the manufacturers. By using the appropriate code, you can interface with the drivers ("using" them), to control the hardware. Now, if I''m wrong, then please correct me.

quote:

you should check out the Andre Lamothe''s latest book: Tricks of the 3D Game Programming Gurus. He writes a 3D software rasterizer from scratch (well you start with the directdraw7 framebuffer so you dont have to write video card drivers ...)



Yeah, but that''s the point. You''d still be reliant on DirectX. To be completely seperate, you''d have to interface directly with the drivers, without using DirectX at all.

Yann L - reverse engineering? NDA? huh? could someone please fill me in on these legal aspects. I don''t really know that much about the legal side of things.



I''''m not easily impressed.
Ooh! A blue car!

Share this post


Link to post
Share on other sites
Reverse Engineering is, I assume, deconstructing something to learn how it works.

NDA = Non-Disclosure Agreement.


I to am very interested in the lowest level aspects of Graphics Programming. However, I''ve come to the conclusion that I''m probably not going to get anywhere by myself. Information is just far too scarce to piece anything together. My hope is that maybe I can get an internship of some sort at a company that handles this sort of thing (and that has access to the materials needed). I''m also thinking seriously about taking Electrical Engineering(I''m almost positive I''m going to).

Share this post


Link to post
Share on other sites
Yeah, well, I''m still in Senior School. (high school for all you Americans, or... others.).

And yes, I know what Reverse Engineering is, I just don''t know if it''s legal, what you can do with the knowledge. Not sell it, obviously, but... use it? Knew I should have taken law... damnit.

Anyway, cheers to TempusElf for the idea using the DDK. Also, I''d like to congratulate him for watching http://www.homestarrunner.com - using some of the best Flash I''ve ever seen. Strongbad''s emails are the speciality.

And if anyone could help me with the driver thing (like, DOES DirectX interface directly with the hardware, or does it use the drivers), I''d be much appreciated.

I have a feeling that this is going to be near the end of the thread, mainly due to the lack of information out there, so, just thankyou to all the people who gave me some clues.




I''''m not easily impressed.
Ooh! A blue car!

Share this post


Link to post
Share on other sites


quote:
I just don''t know if it''s legal

and who gives a damn about if its legal. they cant catch u

i suppose there is some common interface (Hardware Abstraction Layer ?) provided by the drivers with which to operate. i''ll have to look at the ddk for that.
there must be some functions provided by the drivers to acces VRAM and to operate the framebuffer.



/*ilici*/

Share this post


Link to post
Share on other sites
Hmm, why are people so interested in lowlevel 3D chip access ? I mean, it''s just an ugly combination of port and mapped memory accesses. If you want a pretty lowlevel access, use OpenGL, which is very near to the original hardware. And if you want to go lowlevel, keep in mind that your knowledge aquired by hard reverse engineering work will be meaningless on the next chipset revision: other than D3D/OGL, lowlevel specs can (will) change from one chipset revision to the next. Eg, if your lowlevel access works on a GF4 Ti4600, it will most probably fail on a GF4 Ti4800. And you can restart from scratch, if you want to support the GF FX.

What a great way of wasting time...

If you really want to learn about the interna, the write a DX9-compatible software renderer. You''ll get a lot of valuable experience from that. You will learn nothing graphics-related from writing a lowlevel driver though.

quote:
Original post by Xanth
I to am very interested in the lowest level aspects of Graphics Programming. However, I''ve come to the conclusion that I''m probably not going to get anywhere by myself. Information is just far too scarce to piece anything together. My hope is that maybe I can get an internship of some sort at a company that handles this sort of thing (and that has access to the materials needed).


You won''t get access to this kind of material, unless you are fully employed by the company, and again, you''ll have to sign confidentiality agreements. If you break them (ie. use the information and release them/derived products), they are going to sue you.

quote:

And yes, I know what Reverse Engineering is, I just don''t know if it''s legal, what you can do with the knowledge. Not sell it, obviously, but... use it? Knew I should have taken law... damnit.


It''s legal pretty much anywhere, except in the US. But if you do it for private use only, I don''t see a problem. Just be sure to not release anything to the public, and you should be fine.

Share this post


Link to post
Share on other sites
ok after looking at the DDK i found out some things:

all display drivers export functions which you can use to ur own will but to do this i think the app has to get special access rights.

as Yann said u wont do anything by knowing the specs of a certain chipset coz these things change. in the old days there were standards for these but now the only standard is that the driver exports certain functions( thats why every chipset has its own opengl driver). directX i think uses some drivers functions(in the win98 sdk u''ll find more info)

if u want to make ur own DX u should try to use the driver functions windows uses coz no company will make u special drivers.

this is an interesting thing to program, but u''ll need a lot of background on the misterious inner part of the windwos kernel and on the way it operates with drivers.



/*ilici*/

Share this post


Link to post
Share on other sites
Solution: make your own operating system.

Well, after extensive searching, I''ve found that it''s entirely possible to derive the DirectDraw object, thus giving you access to the memory, by Lock() and Unlock(). Still, I''d prefer to know how it worked in bits. And yes, the DDK didn''t help me very much either.

It''s a pity that there''s no standards in the computer world. It''d make things so much easier, and there''d be so much more experimentation and competition and, oh. That''s probably the reason. Competition. Well, I think I''ll scrap that idea until I''ve got more resources. Like Uni proffesors or something (they have to be useful for something right?).

Still, no one has cleared this up for me:

Hardware -> Drivers -> OS -> Software

Now, I know that''s right. But is it possible for the Software to bypass the OS and access the drivers itself? Is that right? Can it be done?

Because if you think about it, OSes allow the installation of new drivers to run the hardware. And they don''t need to write a new version of WindowsXP for that. Therefore, it should be possible to bypass the OS and use the drivers (which I assume would have a standard set of functions or interfaces or what-have-you), and use the drivers directly.

If anybody has any ideas, please let me know. Much obliged.





I''''m not easily impressed.
Ooh! A blue car!

Share this post


Link to post
Share on other sites
quote:
Original post by _goat
Hardware -> Drivers -> OS -> Software


Not always. This is more or less the case with DirectX. But it''s not for OpenGL - it''s more direct:

Hardware -> Drivers -> (OS wrapper ->) Software.

I''ll take the example of an nVidia card. Hardware is your 3D card (obviously). Drivers is the nvidia OpenGL ICD: nvgl.dll (or similar). It will directly access the hardware (or better: through an additional, very thin OS layer, but that can be ignored right now). On top of that, you have opengl32.dll. That''s the ''OS-wrapper'' part. While pretty much everybody uses it (for compatibility with non-nVidia cards, in this case), it''s theoretically optional. All it does, is forwarding OpenGL calls to the appropriate vendor specific ICD (nvgl in this case). You could also directly link to nvgl, exporting the GL functions yourself. That will give you a slight speedup (a little less function call overhead), but will make your code more vendor specific.

Up to this point, everything is well documented. But if you really want lowlevel access to the 3D card, you''ll need to rewrite the driver - and that''s only possible with either insider knowledge, or reverse engineering. And quite frankly, it would be a huge waste of time.

Share this post


Link to post
Share on other sites
What do you mean it''s a pity there''s no standards in the computer world? What do you think DirectX and OpenGL are? The whole point of these APIs is so that there is a standard way of accessing the hardware leaving the hardware manufacturers free to experiment and innovate without worrying about application compatibility. In the bad old days of DOS and pre-DirectX Windows both hardware and software were limited by the need to interface with hardware at a very low level. Hardware manufacturers had a hard time introducing new features because there was no standard way of exposing them for software to use and because if they changed the low level interface old software would not work with new hardware. Software developers could not take full advantage of the hardware because they had to write a lot of low level code for each new piece of hardware they wanted to support.

It''s fine to be curious about the details of the hardware but if you think it would somehow be better if we went back to having direct low level access to the hardware you''re very much mistaken. The existence of high level standard APIs is in large part responsible for the tremendous progress of the 3D hardware industry over the last 10 years. It encouraged and facilitated the competition between nVidia and 3DFX in the old days and between nVidia and ATI now which has driven the hardware forward at such a pace. The fact that we no longer interface with the hardware directly is a very good thing for software developers, hardware manufacturers, consumers and the industry in general.

Share this post


Link to post
Share on other sites
quote:
Original post by Xanth
I''m also thinking seriously about taking Electrical Engineering(I''m almost positive I''m going to).



If it''s computer architecture and related digital electronics that you are interested in, you need to do an Electronic not Electrical degree. There''s a difference, though they do overlap in places.



Death of one is a tragedy, death of a million is just a statistic.

Share this post


Link to post
Share on other sites
Yann L: Thankyou for clearing that up for me.

mattnewport: What I mean is that it''s a pity there is no standard amongst hardware for introducing features. The OS wrapper could be ignored if every graphics hardware exposed it''s features in a standard way, thus allowing you to call the same function for any piece of hardware, and not worry about whether or not it will work, etc.

I admit that you''d probably have to re-wire EVERYTHING for that to happen. But still, it would have benefits. It''ll never happen though. And if it does, well, I won''t complain.

Share this post


Link to post
Share on other sites
All graphics hardware does expose it''s features in a standard way - through OpenGL extensions or through new DirectX versions. Having standardization at the API / driver level rather than at the hardware interface level is absolutely the right way to do things. If some standards body had to hammer out a standard for the hardware interface every time a manufacturer wanted to introduce a new feature it would greatly slow the pace of innovation in the industry. It would take a lot longer for new features to be introduced and it would be limiting for both software developers and IHVs.

As an example look at anti-aliasing - if developers acessed hardware directly without a driver mediating then the only way to take advantage of it in old software would be for the developers to release a patch. With an API and driver level in between it is easy to have an option in the drivers to turn anti-aliasing on for all existing software without any change to the software. As another example look at nVidia''s special stereoscopic display drivers which let you play many existing games in stereoscopic 3D with LCD shutter glasses or VR headsets without modifying the games. That would be impossible if the games were accessing hardware directly. Or how about the driver nVidia released to allow developers to emulate a GeForce FX before the actual hardware was available: how would that be possible with a direct hardware interface?

There might be some slight performance advantages to direct hardware access (though they would almost certainly be a lot less than you think) but they would be vastly outweighed by the disadvantages of having too low level standards. Higher level APIs exist because the way things used to be done (direct hardware access) proved to be a bad way of doing things.

Share this post


Link to post
Share on other sites
Fine, point conceeded.

Well then, is it possible to use OpenGL (of which I have never used) to allow me to access the graphics card''s memory directly? It is possible with DirectX (LPDIRECT3DSURFACE8), but is it with OpenGL?

I only say this because what I REALLY want to achieve is a cross-platform set of functions that I can use to make them nice gfx. And I want to do it by accessing the memory diretly, to set the pixels to the correct colour directly in the VRAM.

Please inform me if I''m gravely wrong. The lack of information about these things at school is tremendous. Although they DID teach us how to write memos. Useless school. :]

Share this post


Link to post
Share on other sites
"I only say this because what I REALLY want to achieve is a cross-platform set of functions that I can use to make them nice gfx. And I want to do it by accessing the memory diretly, to set the pixels to the correct colour directly in the VRAM."

TinyPTC. Or SDL.

j

Share this post


Link to post
Share on other sites
I don''t really know OpenGL so I can''t help you there but I''m sure there''s a way of doing what you want.

"I only say this because what I REALLY want to achieve is a cross-platform set of functions that I can use to make them nice gfx. And I want to do it by accessing the memory diretly, to set the pixels to the correct colour directly in the VRAM."

You''ve got some conflicting goals here. If you want cross-platform functions then you really want a higher level API than OpenGL / DX not a lower level way of accessing the hardware - like TinyPTC or SDL as suggested by jdi. Why do you think you want to do this by accessing the memory directly in VRAM? Do you think it will be faster? That''s not necessarily going to be the case, particularly if you access the pixels in a non-sequential fashion or need to read as well as write pixels. You''ll usually be better off building your image in system memory and then transferring it to VRAM in one go when it''s complete. Do you have some specific effect in mind that you want to achieve or are you just looking to play around with various ideas? If you just want to experiment go for the easiest set up that lets you try things out and don''t worry about performance until it becomes an issue. It''s extremely unlikely on a modern PC that any effect you want to do will be limited in performance by the transfer of your image to the frame buffer.

Share this post


Link to post
Share on other sites