game engines tend to be genre specific. IE shooter engine, 2d scroller engine etc. An engine that makes angry birds type games would be useless when making an f-22 flight simulator.
so, first question: what type(s) of games do you want your engine to be able to do?
you should start by implementing one gametype to start. once you have that, you can add capabilities to do other game types.
and you should have a "target game" in mind. the same way you have a "target player" or "target user" in mind when you make a game. IE who's your customer?
recent postings on this topic have caused me to revisit the "game engine" idea. if there was a silver bullet engine that could be built, i sure could use it, as i have more work to do than i have time for (imagine being an indie with about half a dozen titles to maintain).
but recent contemplation of the question has again led to the conclusion that genre specific or one-off engines are the way to go.
an alternative to the engine approach is the library/template. here, instead of a program the user plugs data into, you have libraries and boilerplate code the user uses to build the program. This approach is much more flexible. One can pick and choose the components used: lets see, i'm doing an F22 sim, i'll use the skybox and ground systems from the library, but don't need the pathfinding. i'll use the library's physics engine as it already supports 3 degrees of rotational freedom (which most shooter engines don't). and so on... this way the code that belongs in the game is in the game, and the code that belongs in the library is in the library, with a minimum of callbacks.
so the first step is to come up with a target game. what kind of game your engine should do (first). this will define the genre or category of games it can do. it will also define the component list (i need ground drawing, skybox, movement & collision, etc.). then you decide whether to do an app or a lib. as i said, libs are more flexible. but with an app, you can sit a non-programmer down and they can build a game. that's the basic tradeoff. coders can do anything code related. non-coders con only do what can be done with tools made by coders for non-coders. So in that respect, you need to identify your target user, as well as the type of target game they'd be making. so the question becomes: who's my customer? a programmer, or a non-programmer? and what kind of game are they making? this will determine lib vs app, and game type supported.
once you've decided on lib vs app and gametype, then its very similar to building a game, but you spend time on the "engine" instead of on content and such.