The math you'll most likely need is not so "deepish" to be fair, it's mostly only parts of courses that students learn in their first year for a CS bachelor degree, and when I say parts it's really a small portion compared to the whole material students learn (that's in case you don't want to do something really advanced of course). For most parts of 3d graphics you'll need analytical geometry and linear algebra - pretty basic stuff like operations with vectors and matrices. If you delve into physics you'll need some calculus, differential equations, here things might get hairy (or let's say if you want to try something with spherical harmonics).