Some general goals you want from a game design:
* Promotes itself (attractive in words/pics/videos, social aspects so players get their friends in, user-generated content so people post it all over the internets for free publicity, etc)
* Provokes emotions (Details depend on game. If you sell your game as "makes you frustrated and hate the game", thats fine, and people will like it as long as it does what it claims to do)
* Does this effectively (quality)
* Does this for a reasonably long time (the game must either contain, or generate, interesting experiences over time, so its worth peoples money and to keep the community active)
* Does this for a reasonably large audience (familiarity to make it approachable, novelty to make it interesting, choice of platform)
* Does this for a reasonably low effort to implement (you want the design to be efficient. This is one reason why emergence is good, as its about patterns that you didnt have to manually build into the game.)
All of those overlap, and you always must make tradeoffs. Overall in game design, everything is interdependent, which is why iteration and other such 'dumb optimization' approaches are necessary (you cannot plan it upfront, because understanding it fully is too difficult). Of course, dumb optimization is dumb, which is why we should seek to understand as much as possible, so we can plan (efficient) more than we test (inefficient) to get the same result.
Now, those general goals would be how you evaluate the design at high level.
To evaluate the design, you apply your knowledge of human psychology (often relying on the assumption that "what I like = what others" like at first), as well as knowledge of the technical fields (ease of implemention) for the goals not related to player. So you can play it yourself, or imagine whether youd like to play it (or a similar game, to understand what effect the differences between the designs might have). But you can also get feedback from other people, do some proper playtesting, etc.
But you need something to evaluate first. Maybe you add something to existing designs and evaluate the result. Maybe you synthesise something original in your head and evaluate that (possibly after making a prototype to be more empirical about it). Then you can apply 'design fragments' that you think might contribute to enjoyable gameplay, to build up the complexity of your design (so you have larger possibility space to work with when trying to locate a particularly good design), which have something to do with following aspects of gameplay:
* Control (do you move a character? interact with things by shooting at them? give orders? tweak numbers? build something? activate things at right time? aim? what parts of the world can the player directly or indirectly interact with?)
* Structure (what primitive elements the game is built of, how theyre ordered. Structure is important for implementation efficiency, and player understanding. Pacing.)
* Physics (how do the elements interact, what rules they follow. How does the world change over time. How does it respond to player actions.)
* Valence (objectives, rewards, punishments, anything the player cares about, whether its formal like a 'points' system, or informal like having some cute animal people just care about)
* Communication (things that make the player learn, interpret, understand, feel)
And theres some common fragments, known good approaches, either because of momentum (its very easy to do what others do), or because player familiarity with those fragments (so theyre used BECAUSE theyre popular), for example:
-Being able to move a person (movement=control, what results from movement=physics, person=valence because people dont really care to act for inanimate objects)
-Getting points (valence, people like to get things)
-Click to shoot (control, physics for how the 'projectile' works)
-Zombie theme (valence, people care more about their survival when its against zombies, than against grey boxes, also communication, because people understand zombies are to be avoided)
-Map being divided into levels (structure)
-Having items you can pick up and do something with (structure, control, valence because players like to get things)
You find such fragments from everywhere, not just other games (any knowledge, pattern, system, behavior can be included in a game).
So, given a game, you could look at each aspect of gameplay separately, see what things affect that specific aspect, and then see how those things build toward the general goals of the design (to understand the design).
Going the other way, its a difficult problem. Its less like "finding the solution" and more like "finding any solution". So at first, a creative problem. Pick random things, see if you can think of a way to fit them together into something that works (Im not really going to think about 'how to be creative', doesnt really relate to games specifically).
Once you have found a starting point, you can get more systematic, start following rules/policies to extend your game following its structure and goals. Those rules/policies depend on what structure and goals you went with. For example, if you want every playable character to be equally good but also be different from other characters, thats a rule you can follow to produce extra characters.
Kind of like simulated annealing (at first relatively random sampling of possible solutions, then closer and closer examination/tweaking once you have a good general idea).
Any general rules will be directly related to the general goals (like adding nice coherent art/animation/effects because they create a more emotional experience, or introduce new things manually or through emergence at steady pace to keep the experience interesting over long time). There will also be more specific rules formed for any particular game or set of games/type of situations, that have been found to be relevant in the specific situation. You would think and explore those once you know the specific context (I guess a general rule would be to research what others did in a similar situation, whenever you have some design fragment you want to perfect, but thats just general design, not specific to games).
The general goals can be simplified as:
1. The game produces an experience that enough people want
2. The game does it efficiently and effectively enough