Let me put it this way as a game programmer who recently returned to University to get my degree. You DO NEED the degree to do anything remotely useful in game development.
Topics that you will need to know that only Univeristy can teach you are:
- 3D graphics (not how to use OpenGL or DX but the actual algorithms used)
- Multivariable calculus and differential equations (required for graphics and animation)
- Linear Algebra (required for graphics and animation)
- Data Structures (required in all programming)
I know the saying that you can learn anything on your own but in reality, it's not true unless you're Isaac Newton. The programming language the school uses to teach you is irrelevant as your concern should be the algorithms and how to implement them. So stay in school, study hard and apply what you learned a little at a time.
I voted you down here. Not because what you wrote was really wrong. But to claim that ONLY a university can teach you the knowledge you point out to be important for a game developer, which is utterly wrong. All the mention classes above can be learned by one self by reading the relevant books on the topic. Do a google search or a search on Amazon.com and the knowledge will be there for you to grab.
I DO agree that all the classes you mention are necessary for a game developer, but do not claim that these topics only exist at the university. We live in the age of the free internet(for now at least).
To the OP I would say if possible then by all means hang on there and endure to you graduate. If that is not possible then follow your heart