The towns have more people, and the people are a bit more varied and interesting. The ruins are much more varied and interesting [and better placed; that is a high level dungeon isn't right next to a low level dungeon]. The quests are generally a bit more interesting; and at least it seems like there's far less kill/fetch quests.
The combat is interesting too. It doesn't seem like it should be given you're only given two choices [block/attack] instead of one, but it's well implimented. Running is feasible; just blocking isn't really; just attacking isn't really... All in all it's much improved over Morrowind. I just tried out some archery for my second, more thiefy/evil character and it seems overpowered [sneak multiplier for ranged attacks!].
I actually think some of the little things really help here too. The over-exagerated deaths [shooting up into the air when fireballed, shooting backwards when arrow'd, slumping lifelessly when frozen...] all quite satisfying. The fact that arrows stick in enemies and that they drop their weapons [or rather throw their weapons to the wind] when killed really adds to the effect. 'thunk's when arrows hit [and stick] to chests, and the clangs of swords dropped on stone are great.
Really, nobody gets tired of seeing ragdoll skeletons exploding into firey bits, or dead ragdoll goblins slump over after you give their corpses an extra kick for good measure.