On the 'create entities/components' thing; it wouldn't just be a case 'create a new class', you'd have to create it, register it, expose it and allow for it to interact in some manner. Most of this work would have to be done by hand, much of it every time you create a new entity/component to be used. If however your entities/components can be expressed by script then it's a simple matter of creating a script and having a system in place to load those scripts and run a register function to expose it. You'd write the initial load/register functionality once and you are done for all future iterations.
C#: Put a custom attribute on your class or implement an interface. A write-once class uses reflection to find and register everything automatically.
In general, scripting allows modification of certain things in the game without having the whole development environment...
Notepad vs. Installing C# Express and XNA isn't that big of a burden, in my opinion. Compared to the typical tools you need for professional games with MASSIVE content pipeline tools, installing Visual Studio is cake.
If you have a script, you can pause your game, edit the script, flush the cache and try it with the new script. No need to re-start your game and recreate whatever scenario you're testing (as would be required in a compiled language).
Visual C#: edit-and-continue (as long as you aren't using lambdas or iterators in the code you want to edit).
Scripting languages tend to be more focused to your problem domain, thus nicer to work with.
This I agree with! You're allowed to focus the scripting language on the problem(s) you want solved.
It's generally not easy enough (or possible) to customize a general purpose language to the same extent as writing a scripting language. However, I think that using a general purpose scripting language such as Lua (without customizing it) has absolutely no benefit.