//EDIT: Damn, took too long typing and had a couple of additional replies sneak in before I posted...
There's a real difference between a game designer and an "idea guy".
A game designer has real value; they provide vision and guidance, and are often the driving force behind indie projects. A real designer is able to work within constraints, is able to "find the fun" and build a complementary experience around that, is able to measure and adjust for the impact of various elements within a design, and can often be the difference between an interesting tech-demo and a masterpiece of game design that can become a smash hit. There are very few people who question this, but unfortunately most people who think they're designers are actually just "idea guys".
An "idea guy" is exactly as useless as common wisdom tells you it is, and isn't the person who fills the role you're describing. They have an idea which may or may not be good, and offer little if any additional value to a project. Projects with an "idea guy" on board are completed despite the idea guy, and are only better than "programmer only" projects out of luck if at all.
Essentially -- if we put aside the subjective and realistically meaningless point about games as art -- you're correct about a real designer, but the thing you've missed is that "the idea guy" is a very poor substitute for the real thing. You'll find very few people who dispute the value of a real designer, but they're absolutely right to devalue the idea guy, and it's a real shame that particular scorn isn't universal enough to prevent hordes of projects that are doomed before they even begin.
Hodgman summed it up very well when he said:
design is a real skill set, which is why most games companies will have a few on their payroll. However, it's not their initial ideas that make them valuable, it's their skill at being a guiding hand throughout the entire development of the game that makes them valuable.
Your stereotypical idea guy does not possess this skill set, or any skill set really.
See also some of the discussion from the topic "what programmers want from a designer", in which several people touch on the value of real designers.