But how's your knowledge of Lagrangian mechanics? Poisson distributions? Standard deviation? The rules of integration and differentiation? Pipelined architectures? Data structures? Computational efficiency? BN notation? Colour theory? The physics behind lenses? Mutexs? Locks? Lock-free code? SIMD? Solutions to sparse matrices? Can you dervie the equations to convert a quaternion to a matrix from scratch?
Yeah, it's probably not necessary to be an expert in every single mathematical and engineering subject to be a good game programmer. Most of the stuff you cited can be looked up on the internet or in a book without needing to remember it. Sure, knowing calculus is very useful, having a good grasp of geometric optics and how pipelined processors work helps a lot, but if someone asks me to spit out the computational complexity of [insert obscure datastructure here] or to evaluate [insert complex integral here], I'm not going to waste my time trying to recall that information or spend fifteen minutes integrating by parts and starting over because I made a sign error, I'm just going to look it up on google/give it to mathematica respectively. It's not being lazy, it's being pragmatic.
Never memorize something you can look up. Oh, and another one I like: the important stuff is never on exams. So learn in your own time, if possible before you cover it in class, and use high school/university to guide you to your next topic and patch up any holes you might have missed during your self-learning (that last point is very important! if you don't regularly recalibrate your own knowledge with some reference curriculum you will crash and burn very quickly! and remember to practice what you learned as well, just "knowing" is useless). That's always worked for me, so I recommend it though people vary in how they learn so YMMV.