For me, I'd say it's a mix of Olof and Mussi's responses.
When I begin a game, I consider the foundation of it - what I feel makes it unique in both atmosphere an gameplay. From there I more or less branch off in every direction and seek to explore every possibility using the atmosphere and gameplay, so that the player can learn on their own just what they are capable of. Typically it's me sketching new ideas and their functions into a day planner or something, but as I test it, and ask for advice from other, one of the key things I look for is proposed changes.
If an element in the game seems more like a matter of luck than anything in such a way that it seems impossible or frustrating, I remove it or look for ways to better it. Sometimes I have an initial idea of how I'd like something to look, and it ends up not fitting into the game as well as I had hoped.
When I feel that the game is polished, and appears stable, and runs efficiently, then I feel it is able to be released. The main game is complete in my eyes.
However, much like Olof said, I may still support it, design add-ons, extra levels, fix missed bugs, and tons of other things.
When I've exhausted my ideas, when I feel like it is time to move on, then the game is truly finished once and for all.