It's a tough problem because imposing goals seems to imbue a game with meaning, yet imposing goals runs counter to the very freedom that draws people to sandbox games. Minecraft gets by with a massive amount of complex, emergent gameplay arising from how the blocks interact with the environment and each other, but still faces that "but what now?" sense (of existential angst?) over time. "Where is this all going? What's it all for?" seems difficult to avoid.

Ramping difficulty (ala Dwarf Fortress) could help but might seal late stage gameplay behind a competency wall.

Some ideas:

* Steal minecraft's blocks, but make them people generating effects you have to deal with: Merchants are great for taxes, but merchants selling too much ale convert laborers to drunkards; introducing priests lowers drunkenness by increasing virtue, but causes people to be less happy and productive; troubadours make people happier, but make priests angry; etc. etc.

The game could introduce a wide range of "people blocks" with different interactions, even maybe changing behavior based on time (priests become less effective over time as population grows), population density (troubadours form an art district, creating new trade goods) and proximity (labor district next to docks amplifies productivity, but increases smuggling crime)

* Player defined disruptive randomizers (within a range): Let players specify before game a range of events with percentage chances of happening. Similar in spirit to SimCity & spawning tornadoes or earthquakes, but more integrated into the game along the lines of difficulty. Better if you can throw a bunch in a big list, some delayed by time (so the barbarian invasions never happening in the first 30 min of playing).

* Rulebreakers: You understand the rules and are bored? What happens when someone opens the ancient tomb or pandora's box or invents a new technology which upends them? Houses built from wood from the mystical forest no longer burn, but attract a procedurally defined threat (orcs one game, bandits another, cultists the next); metal mined from a cursed mine procedurally affects gameplay, in one game allowing construction of boats that are twice as swift but mysteriously cause fish to die (disaster for an island nation), in another making it possible to build greater buildings which sometimes attract ghosts! (I'd suggest making several items have multiple, overlapping effects, by the way, so that effects like fish kills can't easily be just the result of cursed metal, but also farming practices or an evil, aquaphobic cult)

Rulebreakers & people blocks actually might work well together: Imagine gems which attract the best artisans, which make nobles happy, which are (initially unknown) from a cursed mine, which then turns nobles quarrellous and greedy.

A variant of all this could be more of a hybrid game, by the way, like one which funnels gameplay variation and rule changes through a "choose your own adventure" style overlay similar to King of Dragon Pass. The replay value would be phenomenal if the "story" were more loose and procedurally generated, maybe with one game pitting players against a rising threat while another accentuates settling conflict between fractious populations while still a third is more of a murder mystery involving a powerful traitor creating chaotic and disruptive effects. Much tougher to implement and a nightmare for balance, but experimental players would probably love it!

A big advantage of such a hybrid game would be that it has an ending, which drives that satisfying internal sense of completion and sidesteps "growth as if you're a cancer" ethos I think people find unappealing over time.

I think you are missing some games from your playing experience of this genre. You should try Artist Colony, a game from the Timebuilders series, a game from the Virtual Villagers series, All My Gods, and Rebuild 2. Each of these games takes a different approach to blending story/quests and building.

I am not even sure your problem is actually a problem. If people get bored of a game, because they know it inside out, and played it to death, then you basically achieved all you can ever ask. There is little that such people haven't seen or tried in your game.

They used all your content!

What more can you want?

I don't agree with this - as a player, I just need more structure to enjoy a game than some other players do. Games like The Sims or Viva Pinata, which are extremely interesting in concept to me, always lose me before I've fully explored them because the lack of goals and structure means I don't feel oriented or motivated within the game. Even something like Elder Scrolls Oblivion, which is objectively a high-quality game, I have a hard time staying invested in. And the same is true for most open-world MMOs; the only one I ever managed to play for along time is A Tale In The Desert because it has a specific set of tests (aka quests) to pursue in between more sandboxy sim play.

How was it handled in ceasar and such games?

it wasn't. i played all 3 versions of Caesar. in regular gameplay, once you completed your mission, you got a new mission (new city to build), with new and different challenges. There may have been an option to keep playing around with your current city before moving on to the next mission. In open play sandbox mode, it was just you, the map, and the buildings. The map size limited how much you could build. But there were many types of buildings with very in-depth economic modeling. This would lead to endless tinkering and optimization. It was also rather difficult to get and maintain the highest value housing units. So while they didn't add any extra stuff to try to give the player more content once they had used up the game, the game itself was in-depth enough that its was very difficult to use up in the first place. Also, Caesar was the franchise that used "Walkers" rather than distance to determine the influence radius of buildings. So road layouts were much more important, and something you almost never stopped fiddling with.

in simcity 1.0, the map size was the big limitation, and a population of 1 million (maybe you should try urban planning or perhaps VLSI chip design) was the ultimate prize. they added stuff like godzilla cause it got old after a while. it didn't have the depth of a Caesar type game necessary to keep you trying to master it. just surround the housing with parks, and when crime breaks out, tear down all the churches and put up police stations. guaranteed win, every time.

generally speaking, at all times, in any game, a player needs a goal, either provided by the game, or a self determined goal. and they need to be challenged, and they need a continuous supply of compelling new content. and they need for there to be some point to it all. this is what's required for long term play

no goal = nothing to do = stop playing

no challenge = boring = stop playing

nothing new, or nothing i'm interested in = boring = stop playing.

no point to it all = no motivation = stop playing. kind of a meta version of "no goal". but no overall goal, as opposed to no goal right now. whats the point of skyrim? buy all the houses and horses?

Well, Caesar is outlined just above and Pharaoh from same company was on the same boat as well but with an addition of pyramid building which was taking serious time and material.

Simcity up until Simcity 4 (best game in known universe) was single city challenge, SC4 added regions where you can manage several cities loosely in interaction with each other. For Simcity series, after certain point it wasn't about beating game, packing millions of population into region but creating something appealing to your sense of achievement and aesthetics. If you check Simtropolis City Journals, you can find amazing things.

But returning to what you actually asked,

I think Tropico is a good example showing that there's room for improvement. Tropico also has city building elements but also includes factions , foreign powers , balances etc. So it poses a challenge rather than building a banana republic.