It involves mainly vector and matrix math. However, the whole subject is so large it easily fills a book - so I recommend you get a book on the subject. I personally recommend "3D Game Engine Design" by Dante Treglia - that easily gives you enough information for writing a software engine.
Note that the book involves some pretty complicated math - just so you know what you''re in for. In the end this is the best way to learn about 3D graphics though, even if you don''t write a software engine eventually.

- JQ
Full Speed Games. Coming soon.

Flipcode''s got an article series on how to build your own software engine. The first couple of articles discuss the basics of 3d math and converting the 3d coordinates into screen coordinates (basically what you want). Heres the address:

http://flipcode.com/portal/

Getting a good and subject-complete book is always good, especially as reference. But, in lack of that, you should be able to gather most information about 3D-programming, and I mean the basics, not how to use a 3D API like Direct3D, here at GameDev''s reference section. It''s great! You should look primarily on these:

Programming->Graphics - in general
Programming->Graphics->Denthor''s Asphyxia Tutorials - specifically for the basic stuff
Programming->Math and Physics
Programming->Math and Physics->Matrices
Programming->Math and Physics->Vectors