Modern compilers are much better at optimizing code than they used to be. Without a deep understanding of processor architecture, you're much better off writing clean C++ code, and leaving the assembly-level optimization to the compiler.
Yeah, and not only clean but also "sane" and with performance in mind. I remember seeing some code recently which copied a buffer into an STL list of bytes. It looked clean but it was simply *wrong* (or stupid?).
Anyway if you want to start learning assembly one way to do it is to start writing some small routines in C and disassembling them (with debuggers like gdb you are even able to have a view of disassembly with interleaved corresponding C code (I mean /m switch here)).
But with learning assembler you need to know a lot of things around like ABIs of your target platform, details on ISA, details on syntax of your assembler of choice. It would be also nice to have some idea about linking and C++ ABI if you want to use assembly with C++ code. And if you want to write in asm to get better performance you'll want to know timings of exact mnemonics on your target processor and details on how your target CPU works. And even if you make your code super-optimal for certain core it probably won't have the same performance on different one (e.g. intel vs amd, or even various intel chips).
It's all very interesting knowledge, but it's really lots of it and I seriously doubt you could write anything more optimal in terms of performance than what modern compilers can generate for you. + if you want it for making games then probably you mean intel ISA, and this brings even more details and problems.
I don't want to discourage you, I think it's still worth knowing, but not exactly for making computer games.