# Things every graphics programmer should know?

This topic is 2137 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

## Recommended Posts

Greetings,

I am an aspiring professional graphics programmer. Interestingly enough, despite being extremely close to finishing my undergraduate degree in Computer Science (with a concentration in graphics, mind you), most of the material I know has been self taught. Recently I bought [i]Real-Time Rendering[/i] and [i]Physically Based Rendering[/i] and my mind was blown on how much I still don't know. Even after nearly 10 years of hobby experience, I feel completely ignorant. I can definitely see why people need to get PhD's in the subject to actually get anywhere in the research realm.

As fascinated as I am in the subject, I feel like I have some serious gaps in my knowledge; things that I don't even realize I need to know. I guess I'd like to hear your opinion on what every graphics programmer should know. Here are some things that I can think of:

Linear Algebra - Unfortunately, I was never able to take linear algebra (long story--I transferred schools and planning classes has been a nightmare), but it's the most obvious math subject in graphics. You won't get far without a great understanding of vectors and matrices! I'm finding that the more heady parts of linear algebra are also a basic necessity as well, such as understanding eigenvalues (I still don't...). I just recently bought Gilbert Strang's book on Linear Algebra, so hopefully I will soon be a master of the subject. [img]http://public.gamedev.net//public/style_emoticons/default/smile.png[/img]

Calculus - The deeper I delve into topics like radiosity and spherical harmonics, the more I realize I should have been a math major. I've had up through Calc II, and what I really need is Calc III/Differential Equations! I'm looking into fitting these classes into my schedule.

Computational Geometry - Things like voronoi diagrams and delaunay triangulations, convex hulls, line configurations, etc. I feel like I don't have an excuse not to know these.

Graph Theory - This is a new realization for me. I'm finding that I need to know my way around graphs...[b]really[/b] well.

Rasterization/Ray Tracing Experience - I feel like every graphics programmer needs to have written at least one ray tracer and one software rasterizer. I've done the latter, but not the former (yes, I'm lame). That's one reason I'm super excited to read [i]Physically Based Rendering.[/i]

The reason I ask is I want to graduate from the "I've messed around with DirectX and little math and stuff" stage to the "I'm a [b]real [/b]graphics programmer" stage. It seems like I basically should have been a math double major. Thankfully, it's never too late, but I'm having to scramble to learn all the math now.

What do you guys think, are there other things?

##### Share on other sites
you got to strive to know every thing about graphics.

thats what i do, and ive been at it for about six years now and i learnd everything myself

##### Share on other sites
Unfortunately, I could get a PhD in the subject and still wouldn't know everything. I guess I'm looking for more specific examples of what I should focus on, as well as what foundational things might be really helpful but not immediately obvious.

##### Share on other sites
At a minimum, everyone should write one each of these:

- efficient 2D software rasterer: Bressenham lines, circles, filling polygons, drawing sprites with masks or alpha
- ray tracer
- software texture mapping

When playing with software 2d on any new device, platform or whatever:

1. can i draw a dot?
2. can i draw a line of dots?
3. can i draw a polygon of scanlines?
4. can i copy colors from a texturemap as i draw those lines?
5. can i blend the new colors on top of whats already there?
6. can i transform 3d coordinates to 2d screen coordinates?

At step 2, you can make simple games like old-school asteroids. When you get to step 6, you can draw 'anything' You'll also have appreciation for what the hardware does.

##### Share on other sites
[quote name='ZBethel' timestamp='1330529554' post='4917783']Linear Algebra, Calculus, Computational Geometry, Graph Theory.[/quote]Linear Algebra is probably the most important thing -- even if your theory isn't completely solid, you need to have a solid practical understanding - to be able to intuit transforming values between different spaces/basis, intuiting dot and cross products etc...
Being familiar with the others is of course good and opens more opportunities, but aren't quite as key as linear algebra.
I also missed out on doing much math in my tertiary education and it slows me down sometimes, but doesn't hold me back completely.[quote]Rasterization/Ray Tracing Experience - I feel like every graphics programmer needs to have written at least one ray tracer and one software rasterizer. I've done the latter, but not the former (yes, I'm lame). That's one reason I'm super excited to read [i]Physically Based Rendering.[/i][/quote]FWIW, I've not done my pilgrimage to the home-made software rendering gods either, but I [i]am[/i] also currently reading PBRT ;)[quote]What do you guys think, are there other things?[/quote]If I was interviewing a graphics programmer, I'd probably be most interested in their practical knowledge of rendering techniques. Off the top of my head, I'd ask things like -
* What are some ways that you can filter shadow maps? What other techniques can you use to get better results from shadow-mapping?
* What are some techniques for rendering reflective materials?
* How would you implement DOF on SM3? What about SM5?
* Give an overview of how skinned meshes are drawn.
* What's the difference between forward and deferred shading schemes?
* What are some different lighting models, and what's special about them?
* What are the pros/cons of performing lighting in tangent space vs view or world space?
etc...

##### Share on other sites
Just for funsies:

[quote name='Hodgman' timestamp='1330564564' post='4918040']
[/quote]
'Shadow volume algorithms' is a bit ambiguous. In terms of shading calculation, I've looked into the bog-standard Crow 1977 paper and Forest's Penumbra Wedges idea, but it's not something I've really investigated much. (see later) In terms of masking you have the standard Z-pass and Z-fail/Carmack's Reverse.

Second part: In practice, never. Shadow maps are superior in pretty much every conceivable way these days. Filtering's better, you get free support for alpha-tested surfaces, you can amortize a lot of the cost over several frames, etc. Translucent surfaces can also receive shadow maps, too, which is something shadow volumes are flat-out incapable of. Point lights are a little more difficult to do, but there exist methods like dual-paraboloid and cube projection that can work pretty effectively.

Lastly, I figured out how F.E.A.R./F.E.A.R. 2 got such nice shadow edge transitions not that long ago. They actually apply a depth bias to the shadow boundary geometry so you don't get that ugly faceting artifact near silhouette edges. It's so unbelievably stupid, it's brilliant. If you must use shadow volumes, give that a whirl.

[quote name='Hodgman' timestamp='1330564564' post='4918040']
* What are some ways that you can filter shadow maps? What other techniques can you use to get better results from shadow-mapping?
[/quote]
This could probably be rephrased as 'how *can't* you filter shadow maps, and the answer is 'there aren't any ways' [img]http://public.gamedev.net//public/style_emoticons/default/smile.png[/img]
Percentage-closer filtering is the oldest (I think) and works pretty well for the general case. Variance shadow maps and friends are rather popular these days, though I think exponential variance shadow maps in particular are the flavor of choice. Shadow maps for area lights seem to be pretty in vogue for CG research, and most of them seem to involve microfacet back-projection, though there are some more interesting VSM-like approaches I know of (the rather underutilized interpolation-friendly soft shadow maps seems promising, but I haven't yet got around to implementing it so I can't speak with total authority) Crytek's also experimented with using some sampling noise to get softer shadows for marginal performance cost, and this looks really good if you render the contribution into a God of War-style white buffer and blur it a bit. I won't tell if you won't ;)

Better's also a vague term. If you want to avoid acne, generating the shadow map by way of using backfaces for map generation works fairly well, as does using midpoint shadow maps (though the latter is a lot more expensive and is NOT compatible with all filtering methods, at least not to the extent that backface maps usually are)

[quote name='Hodgman' timestamp='1330564564' post='4918040']
* What are some techniques for rendering reflective materials?
[/quote]
Reflective in the sense of 'analytical lights' would mean your standard rogue's gallery of BRDF models. Image-based reflection is a little less broad in scope and is really constrained by the capabilities of your target hardware more than anything. You have your cubemaps, spheremaps, esoteric-projection maps and even some raytracing done in screen space if you want to get really fancy.

[quote name='Hodgman' timestamp='1330564564' post='4918040']
* How would you implement DOF on SM3? What about SM5?
[/quote]
Rendering budget would play a larger role in determining the algorithm(s) used rather than shader model, IMHO. Bokeh DOF by way of scads of billboards has been receiving some attention by Epic and friends, inelegant as it may be. While they make use of D3D10+ hardware features, I think you could probably cook up something pretty similar using vertex texture fetch/render to vertex buffer and point sprites on D3D9-equivalent feature levels. Probably wouldn't be as fast due to the rather inexpressive API, though.

If you want cheap and are doing bloom, you can also do things the Unreal Way™ and repurpose your bloom buffer. You don't get out-of-focus bleed, but if your game involves a lot of fast motion the artifacts probably won't be too noticeable. Infinity Ward also details some tricks to work around that problem that mostly involves blurring your circle of confusion, but that's not quite perfect either.

Also, if you want bokeh DOF but don't want to commit performance seppuku DICE details how to do a separable blur with a hexagonal shape. Whoever thought that up needs to get a pay hike; it's brilliant and involves decomposing the shape down into a series of skewed box blurs.

[quote name='Hodgman' timestamp='1330564564' post='4918040']
* Give an overview of how skinned meshes are drawn.
[/quote]
Animate bones, collect/weight influences, project to screen(?) This seems a little too easy.

Note that I don't include things like tangent transformation since there are some more modern bump mapping approaches (thinking primarily of Mikkelsen's stuff here) that don't actually need it. You could also use quaternions or even dual quaternions if you want to save some memory/interpolators and preserve volume across deformations, respectively.

[quote name='Hodgman' timestamp='1330564564' post='4918040']
* What's the difference between forward and deferred shading schemes?
[/quote]
Forward shading considers lighting as an object-space query, deferred shading opts for a more screen-space approach. There's so much futzing around with G-buffer design I don't consider going into detail about the various data designs useful due to their fluidity. I'm actually working on expanding some of the ideas behind light-indexed deferred rendering to try and get some of the best from both worlds.

[quote name='Hodgman' timestamp='1330564564' post='4918040']
* What are some different lighting models, and what's special about them?
[/quote]
Isotropic microfacet BRDFs like Phong/Blinn-Phong consider the surface to have lots of bits pointing every-which-way, and are a good fit for most surfaces.

Anisotropic BRDFs, like the Ashikmin-Shirley and Kajiya-Kay models, are designed to simulate the effects of surfaces where the little bits are instead usually facing along a single direction, and are practical for things like hair or brushed metal.

BSSRDFs are probably the most expressive/expensive, so you don't see them much in games, though there's some new research by d'Eon, Jimenez and Mikkelsen about efficiently implementing them on a GPU. (Blur kernels say hello) Eric Penner's got some really cool stuff that calculates the scattering ahead of time and examines the curvature of the surface to try and fit what you've got to what you precomputed. Hacky, but extremely fast and surprisingly effective.

I consider things like energy conservation to be subtopics of the above three items, and if you aren't paying attention to this, then your graphics programmer license is revoked until you do [img]http://public.gamedev.net//public/style_emoticons/default/smile.png[/img] In short, you'll need to adjust the approximate models like Blinn-Phong so that they reflect the amount of light they receive based on the properties of the surface, (tl; dr BRDF normalization) and for extra credit adjusting the diffuse reflection such that it only accounts for light that was NOT reflected based on the specular component of your shading model.

[quote name='Hodgman' timestamp='1330564564' post='4918040']
* What are the pros/cons of performing lighting in tangent space vs view or world space?
[/quote]
Precision reasons and being able to reuse tangent-space normal maps for different objects, though the latter is sort of a non-feature these days. World-space shading can also make handling scads of lights easier as there's less transforming to be done/interpolators that need to be used. Object-space shading (conspicuously absent, I note [img]http://public.gamedev.net//public/style_emoticons/default/smile.png[/img]) can end up being the best of both worlds, actually. Contrary to popular belief, it can also play nice with skinned meshes.

##### Share on other sites
[quote name='InvalidPointer' timestamp='1330622551' post='4918253']
Isotropic microfacet BRDFs like Phong/Blinn-Phong consider the surface to have lots of bits pointing every-which-way, and are a good fit for most surfaces.
[/quote]

This might sound pedantic, but I don't think I'd call Blinn-Phong a "microfacet BRDF". Nevermind Phong, which isn't even close.

##### Share on other sites
[quote name='MJP' timestamp='1330642546' post='4918403']
[quote name='InvalidPointer' timestamp='1330622551' post='4918253']
Isotropic microfacet BRDFs like Phong/Blinn-Phong consider the surface to have lots of bits pointing every-which-way, and are a good fit for most surfaces.
[/quote]

This might sound pedantic, but I don't think I'd call Blinn-Phong a "microfacet BRDF". Nevermind Phong, which isn't even close.
[/quote]

Sure it is! Blinn-Phong in particular is an (very good, at least in the normalized incarnation) approximation of a Gaussian distribution centered on the normal. You're evaluating what fraction of the facets face the halfway vector and thus reflect light into the camera.

##### Share on other sites
[quote name='InvalidPointer' timestamp='1330650679' post='4918448']

Sure it is! Blinn-Phong in particular is an (very good, at least in the normalized incarnation) approximation of a Gaussian distribution centered on the normal. You're evaluating what fraction of the facets face the halfway vector and thus reflect light into the camera.
[/quote]

A distribution is not a microfacet BRDF, it's one part of a microfacet BRDF.

##### Share on other sites
Hmm, tricky. Very tricky.
* Math. Lots of math. Immense amounts of math. Linear algebra and basic computational geometry used to be pretty satisfactory. Not anymore! Need extensive calculus, signal processing, statistics, all kinds of stuff.
* Ground level knowledge. Basic rasterization and raytracing algorithms, the basic structure of a 3D render pipeline, shading concepts independent of hardware, etc.
* Currently standard rendering approaches. Forward and deferred renderers, lighting, shadows, post processing effects, spatial subdivision/acceleration, animation systems, etc.
* Debugging and performance analysis. Be able to make it work, fix it when it breaks, and hit that golden 30/60hz number. This implies a lot of familiarity with hardware and the tools for dissecting what is going on in that hardware.
* Generalized massively parallelized compute outside of the standard pipeline. GPGPU. We've reached the point where non-graphics GPU code is critical for graphics.
* High end/future/idealized rendering models. Ray tracing, radiosity/radiance/global illumination methods, physically motivated shading (BRDFs), basically a long term view of where we're headed.
* Art packages. You need to be proficient with Photoshop, Max/Maya/ZBrush, HDR tools, all that stuff. Plan on writing plugins for at least some of those tools.
* [b]Photography and cinematography.[/b] Maybe how to draw and paint. This is one I only understood fairly recently. The vast majority of rendering is motivated by cinema approaches and some photography, which in turns draws a lot from old school hand-created art. There's no point developing visual effects if you don't understand why and how they're used.

Hodgman's list might make good interview questions, but I feel that they're too narrow and specific to be useful goals. They're things you should have picked up along the way. I dislike Draco's list of low level functions, because knowing that stuff doesn't make you even a slightly competent graphics engineer. A competent engineer should probably know those things, but he should also know linear algebra and neither of those makes you good.

##### Share on other sites
-Fundamentals of Computer Graphics by Peter Shirley, Michael Ashikhmin, Steve Marschner. (Know about the Ashikhmin-Shirley BRDF ? It's them.)
-Game Engine Architecture by Jason Gregory. (AAA game developer.)
-Mathematics for 3D Game Programming and Computer Graphics from [color=#000000][font=verdana, arial, helvetica, sans-serif][size=1]Eric Lengyel. (He was one of the first to push for the correct "bitangent" name over "binormal".)[/size][/font][/color]

I assume you already know aglorithms, CPU architecture, how memory works, GPU architecture, C++ and at least one graphics API.

If it's not the case I can recommend a few more books on those topics.

##### Share on other sites
Hidden
-Fundamentals of Computer Graphics by Peter Shirley, Michael Ashikhmin, Steve Marschner. (Know about the Ashikhmin-Shirley BRDF ? It's them.)
-Game Engine Architecture by Jason Gregory. (AAA game developer.)
-Mathematics for 3D Game Programming and Computer Graphics from [color=#000000][font=verdana, arial, helvetica, sans-serif][size=1]Eric Lengyel. (He was one of the first to push for the correct "bitangent" name over "binormal".)[/size][/font][/color]

I assume you already know aglorithms, CPU architecture, how memory works, GPU architecture, C++ and at least one graphics API.

If it's not the case I can recommend a few more books on those topics.

You know, just some light reading when I have the time. [img]http://public.gamedev.net//public/style_emoticons/default/smile.png[/img]

[quote name='Promit' timestamp='1330669098' post='4918504']
Need extensive calculus, signal processing, statistics, all kinds of stuff.
[/quote]

Indeed, I'm beginning to realize this now that I'm studying topics like radiosity, spherical harmonics, and monte carlo integration. Even basic mip-mapping has signal processing theory behind it.

When I first came to GameDev (like 10 years ago), I was just learning DirectX and basic programming. I've spent a lot of time trying to learn APIs and rendering techniques, but until recently I've neglected more academic aspects of graphics like fully understanding the math behind everything and really learning the data structures and algorithms. I'm realizing that even after 10 years of programming with DirectX, I'm still really not a graphics programmer; I feel a bit like a poser with a lot of upper level knowledge without the fundamentals to back it up. I want to change that. At the same time though, it seems like to really grow, you need to do it full-time in a production environment (i.e. get an entry level graphics job or something).

##### Share on other sites
If your maths isnt as strong as you may think, pre calculus books are an excellent way to 'catch up', it makes those calculus books easier to work with, the big book on linear algebra is a bit worrying though, I wouldnt be surprised if its all wordy and 'boring', especially considering the size similarity to Calculus which I have worked with and it contains linear algebra and pre calculus anyway.

One thing I would recommend is you download Microsoft Math, its free and its graphing features and well everything else are excellent, much better than a graphical calculator plus it has onenote integration which is great.

Oh and for the sake of procedural programming, have a look at "Texturing & Modeling - A Procedural Approach" and GPU Gems, Real time rendering does reference GPU Gems a lot and just adds more words, I found GPU Gems to be easier to read than Real time rendering but it really depends on your style of learning, some people like lots of theory some people dont.

##### Share on other sites
[quote name='Dynamo_Maestro' timestamp='1330821490' post='4919025']
[/quote]

Another thing about Real-time Rendering is that, aside from a view chapters that discuss theory (in very good detail), it's more of an overview of what is out there than a "how-to" book. But it's great for helping one decide what direction to take, and it is extremely well-referenced, and so it also serves as a directory for finding the implementation details that you're looking for. Don't take my word for it, though; I'm somewhat of a newbie.

##### Share on other sites
[quote name='CDProp' timestamp='1330826070' post='4919053']
[/quote]

Doesnt really surprise me, MS are bad when it comes to informing / advertising, sign up to all their spam mail and you'll likely only get generic mail on core products, things like Math, Robotics, Visio, project etc only seem to be discovered by error these days

##### Share on other sites
[quote name='Dynamo_Maestro' timestamp='1330821490' post='4919025']
If your maths isnt as strong as you may think, pre calculus books are an excellent way to 'catch up', it makes those calculus books easier to work with, the big book on linear algebra is a bit worrying though, I wouldnt be surprised if its all wordy and 'boring', especially considering the size similarity to Calculus which I have worked with and it contains linear algebra and pre calculus anyway.
[/quote]

Ironically, the Linear Algebra book is the one I'm most enjoying at the moment. Strang is an incredible professor (he teaches the OpenCourseware Linear Algebra course at MIT). Thank you for the Microsoft Math suggestion, I have been looking for something like that for a while now! I haven't started Real-Time Rendering yet, but it looks very heady, and it will probably be a slow read.

##### Share on other sites
Thanks guys ...your suggestions will definitely help me a long way...i have just started my graphics programming ,starting off with the API DirectX, then now i have moved on to shaders, and the books you have suggested , vil start reading them, though i knew some of them, but ,this post will be really helpful....

##### Share on other sites
[quote][color=#282828][font=helvetica, arial, verdana, tahoma, sans-serif][size=3][left]* [/left][/size][/font][/color][b]Photography and cinematography.[/b][left][font="helvetica, arial, verdana, tahoma, sans-serif"][size="2"][color="#282828"] Maybe how to draw and paint. This is one I only understood fairly recently. The vast majority of rendering is motivated by cinema approaches and some photography, which in turns draws a lot from old school hand-created art. There's no point developing visual effects if you don't understand why and how they're used.[/quote][/color][/size][/font]
[font="helvetica, arial, verdana, tahoma, sans-serif"][size="2"][color="#282828"]^^ this, this, and this again ^^ [/color][/size][/font]

Photography and cinematography are absolutely vital for realism. Art, design, and sculpture come into their own when you start dealing with NPR techniques. Take as many art/sculpture/design/photography/animation classes as you can. It doesn't matter if you suck, you will get better, and I can't emphasise enough how much that knowledge will help you in the long run. The two biggest benefits I've personally found is that you develop a very critical eye for flaws in your own work, and it allows you to communicate with the art/animation departments using their terminology (thus reducing the amount of wasted work and general confusion). It's also possibly telling that around half of the graphics programmers I work with, have a formal education in art, myself included (I originally entered the games industry as an 3D artist before making the switch to the dark side). [/left]

##### Share on other sites
The Linear Algebra book by Strang is pretty good, in my opinion. Much of that space is taken up by the HUGE amount of problems he has at the end of each chapter - the actual material presented is light, but there's plenty of self-education in those problems. Careful though, many of them are written for people with extensive math backgrounds already - I took a year of calculus but found he was assuming some knowledge I didn't have.

Those books look very theoretical overall, which isn't a problem if that's what you like. Many of them are more applicable for computer graphics people - for anyone who wants to be a game programmer and isn't as sold on graphics theory, they're less essential.