I can sympathize with the sentiment, but you have to ask yourself why the industry is in decay. Part of the reason D&D and other tabletops have declined in popularity is because other things have come along -- back in the day, if you wanted to play a fantasy-based role-playing game with deep plots, tabletops were your only real option. Today, MMOs like WoW provide those same themes, role-play, similar social interaction, and the accessibility of playing whenever you choose without having to herd 4+ friends together at the same place and time. Modern interactive games also provide more immediate gratification. As a video gamer who recently gave D&D a good shot, one of the things that struck me was the glacial pace with which the game moves -- with 5 other players, a single turn takes an hour, 50 minutes of which I'm essentially idle and disengaged. In an MMO or any other game I'm always actively doing something. All of this is to say nothing of all the other competing styles and genres of games available today, and other kinds of entertainment that have never been more accessible and immediately available.
I'm a veteren of D&D and GURPS. I have NEVER seen a turn take more than ten minutes, even with a dozen people playing. Not a single time. More importantly, there are advantages to actual tabletops that video games can never provide, such as the ability to do anything at any time, even if the creator of the game didn't think about it or write rules for it.
Excluding the turns we decided to take breaks during. Normally, we wait until after combat, but sometimes breaks have to happen suddenly. And even then, you'd have to be counting the break into the duration of the turn, and I personally don't.
Further, WoW is considerably less engaging. As the party tank, you can just push the autoattack button and walk away, and it won't hurt your effectiveness any. The other classes don't take much extra. The game requires barely any input, and it bores the shit out of me. Add on how insanely repetitive it is, the lack of customization and the lack of incomparable removing all options from advancement and it's the sorriest excuse for an RPG I have EVER seen. It makes Skyrim look deep.
I think the more classic experience does still appeal to some people -- the industry may be a smaller part of the overall entertainment pie today, but I'd hazard a guess that its probably as large or larger than its ever been in total numbers today. Large publishers, indeed large companies of any kind, tend to ignore the small slivers, which makes them a sort of "cottage industry" as far as the wider gaming industry is concerned, even though the sliver may be entirely viable in its own right.
What I'm getting at is that you may very well be onto something, but its naive to think that everything is horseshit just because its mainstream -- horseshit doesn't sell like mainstream products do, they may not appeal to you, you may be desperate for something different, but that's clearly not what the mainstream audience wants. Is that lamentable? Probably, on some level. Anyhow, if the impetus behind your business plan is that the mainstream is crap, and there's a conspiracy (or unwillingness) by the mainstream publishers to keep table-top-style games from returning to the mainstream, I think you'll be disappointed with that thesis.
THAT IS NOT WHAT I SAID AT ALL. I do NOT hate all AAA games for being mainstream. I hate most AAA games for a short list of specific reasons. I hate AAA shooters because there's a whopping three styles: CoD, Halo and GoW, the most popular of these is a completely mindless twitch-fest that requires nothing beyond hands and a dozen or so functioning braincells. The rest would be fine if they weren't being ripped off on a daily basis. (Seriously, people, stop cloning Halo. If people want to play Halo, they'll play Halo, not your clone of it.) I don't really hate AAA RPGs yet, but I am strongly disappointed in the direction they are taking. They are being watered down over and over again, becoming more and more casual with less and less choice and freedom. Give them five years, they'll all look like Fable III and Final Fantasy XIII: straight-ass fucking hallways with no challenge, no choice and no fun. I don't give a shit about racers or social games, so I'll skip them. I haven't played a fighting game since Soul Calibur V, and I haven't played an RTS since Command & Conquer 3, but that's not really out of dislike. The only games I really care about now are RPGs and shooters, and I have very specific reasons why I think the AAA industry is doing a shit job making those.
Personally, though I don't much like tabletop games, I think there's actually a market to be had in creating an online platform for these kinds of games -- that is, one which allows people across large distances to play together without being too difficult to use and to author content and rulesets for. That's of course different than what you seem to describe.
HOW is that different, exactly? Because that appears to be EXACTLY WHAT I JUST DESCRIBED. It's a 2d representation of a tabletop game, with multiplayer. It comes with a modding kit, which allows for custom content and rule adjustments.
But regarding whether you need AAA graphics, real-time 3D, or other chrome, I say no. What you need is art that looks professional for what it is, and which has style. People know production value when they see it, regardless of what form it takes. Its that property of the visuals (and audio) that says to people "this game is worth my time" -- shoddy production values say "not even the author thinks this game is worth his time". Plenty of successful games have "simple" graphics, be they in 2D or 3D, and nearly all of those have obviously high production value. Ultimately it comes down to time and resources, and simpler graphics with higher production value are more appealing than complex graphics with low production value.
Extra credits did an episode on that once, I believe it was called "Graphics vs. Aesthetics." Also, you don't know what "production value" means. "Production value" means "the amount of money put into production." It has nothing to do with quality. At all. What you are thinking of is "aesthetics."