I'v been planning on writing a long reply to your previous post on the subject, but never got the time, so I'll just write some short observational stuff here...
I'v done Pair programming and XP reasonably seriously, and I ran into the same things as you pretty much when starting out (although this was not in game development).
1. Development methodologies shouldn't be Laws. Use what works, dont use the rest. Just as you say, if something is trivial there is no point in pair-programming it. *however* defiantly stick with it now while you are learning, it takes longer than you think at first before you really make the mental switch, and its better to just go 100% by the book at first.
2. you will sometimes run into a pair-mate you don't click with at all. Many places rotate people between pairs, but personally I prefer people picking their own groups. Another way which works really well for breaking in new recruits and getting them up to speed is to pair them with a more experienced programmer thereby teaching/spreading the knowledge. Of course, this can be frustrating for the senior since it will feel like he is being slowed down, but I think it defiantly makes the overall team stronger in the long run.
3. 100% pair programming unfortunately requires that you work at the same times as your partner :(
4. you talk about scheduling problems. what if you kept on pair programming? bring your partner and solve your bugs, then solve his. you will both gain knowledge of each others code areas, which can only be a good thing.
5. a tip: if you notice you both aren't speaking, there is probably something wrong going on.
6. speedwise, it is usually quite a bit faster than normal speed when pair programming, so you shouldnt really write it off as half of the time wasted (although this is certainly true in the beginning).
7. I like two machines placed next to eachother instead of just one. This way the partner can google solutions, looks up API stuff etc etc instead of those idle moments you speak of. also if you have a bunch of really simple tasks that require no pair programming it is less of a shift to do them sitting next to eachother.