A couple of options to look at:
Unity. The homepage is currently showcasing the engine's 2d capabilities, but that's only because they were just added in the latest release. It's a mature and very capable engine that should have no problem with the level of visuals you want to achieve. You'll need a pro licence (US$1500) for high quality writing, but you could do almost all of your development with the free version before making the purchase. You can program in C#, UnityScript (very similar to JavaScript), or Boo (very similar to Python), or if you purchase the Playmaker add-on ($65) you can use a visual editor instead of programming at all.
UDK. I don't personally know as much about this one, but it's definitely capable of what you're after and again you can start development for free; it's then just $99 when you want to publish, and royalty payments if you make over $50,000. You should be able to find out more on the website and in the faq.
Leadwerks. Again, well supported and capable of what you want to do. Programming in C++ or Lua, and again there's some form of visual editor offered.
I remember someone suggesting a pretty capable looking 3d engine that didn't require any programming recently, but unfortunately I can't for the life of me remember what it was or where the recommendation was. Hopefully someone else might know what it was!
All of these can be tried for free, so I'd suggest checking out any of them that seem interesting. I'm sure others will have some good alternative suggestions and further feedback on these ones as well.
For reference, you can achieve the "less real" look you're after with the right artwork even if the engine is capable of "better".
Hope that helps give you a starting point. :)
