It's tough to give a good suggestion without a better idea of the game as a whole, target audience, etc. Is everyone in the game a mage learning spells, or is this just one possible class? Is this an MMO or a single-player RPG? Is it action-focused or more turn-based combat?
If you want the most control over when and how a player gains new spells, I'd definitely add some unique resource (Research Points or whatnot) that they need. Then you can give players Research Points naturally as they kill enemies, add some mini-game where they can earn them, or just let them buy them (so different player types can unlock according to their play-style).
If this is an action-RPG like Magicka, the self-exploration would probably best compliment the game. When casting is quick and fun it's cool to experiment. But if this is a turn-based thing it could grow tedious trying out a bunch of combinations, and then accidentally killing the low-level enemy I'm practicing against and having to go find another one.