Oh, I wouldn't say so. I learn absolutely best when I need the technology (ie. a language) for something else (ie. a game). I know a lot of people that feel likewise.

Besides, switching to a new language is only hard if it has new concepts. Ie. switching from procedural to object oriented to functional to logical is hard, while merely learning a new syntax or a slightly different module system is not so hard. Since at least Delphi (I wouldn't know about VB) is object oriented, it shouldn't be too much trouble; in fact, the designer behind both Delphi and C# is Anders Hejlsberg.