I am hoping to get ideas for a very small indie project, so "massive" here does not mean the same thing as the "massive" in "MMO". What I am thinking of is perhaps 100-200 players fighting on a single battlefield. The setting could be medieval war, space ship combat, modern army etc. The graphics could be 3D, 2D or even text-based.

The dream scenario is to have 100-200 players (or possibly 1000+) fighting a war, some of them might rise up to be leaders, planning strategies and scoring minor victories

However, when trying to actually implement such a game, we run into many problems. For example:

1) As the number of players rises, the actions of individuals or even groups of individuals diminishes and no longer matters.

2) 1000+ players in an area leads to overcrowding and latency. E.g. if I recall correctly, eve online require massive battles to be scheduled in advance.

3) No one wants to be on the losing side. If one side starts to lose the war, people might all jump ship to the other side. Also, keep the number of players on both side balanced is challenging.

4) The side with the most players online tend to win the most. Some times, it boils down to one side having 100 players online while the other side has 50 and could not hold the fort.

I have played games that tries to implement a "massive battlefield" scenario, and they all resorted to very artificial mechanics.

E.g. one game made it so that while a faction can lose specific areas to another, the headquarters can never be taken. So, factions will go back and forth, each taking turn to win all the areas, except for the HQs, depending on who has the most players online.

E.g. another game pretended that there was a huge battle going on in the background, but players take part in skirmishes (FPS team death matches). No impact is actually made on the "battle" alluded to.

Is it truly possible to design a big battle game? Or is this a hopeless goal?

If you're OK with text-based battles, I played a browser game some years back which did it pretty effectively. I don't recall the name, but it was a gang war setup, where battles were fought between gangs of a hundred or so a side every night. You had no control over your actions during the battle, but there were UIs at various levels (clan, chapter, and individual) where you'd program strategies involving equipment, fighting style, etc., and the whole thing took ten minutes or so to crunch on a cheap web server. Maybe not ideal, but you got a nice detailed account of the battle, and you never missed a fight.

You could provide a little more player involvement using turn-based simultaneous actions without stretching server hardware requirements, especially if you don't mind making players wait a minute between turns. Place a restriction on turn length, of course.

1) May be down to clan leadership upholding their responsibility to their members, and making them feel valued. After all, if all the nobodies left, you'd have no-one to fight. Progression is important too though - the opportunity to look forward to having a greater impact. I think most players understand that when they're new they won't be as effective as when they've had some time to advance and specialize. Also, don't under-estimate the "I was there" factor. Most players will enjoy the spectacle even if they die in the first volley, provided you let them stay till the end as spectators.

2) 1000+ yes, but if you're happy with 100, that's a lot more manageable. And if you can get 100 players in your "very small indie project" world at one time, I think you're doing pretty well.

3) I don't think deserters are your problem. It's up to individual clans to produce loyal members and winning strategies. That's politics.

4) Limiting participation in some way may be the answer. If you have 50 men, join a 50-man battle scenario. That's pretty hectic already. 1000-man battles don't occur on a daily basis in EVE, and I can tell you that scheduling is a huge part of successful (not just victorious, but stable) engagements in EVE, and was even more so in the days before you could order up some extra beef for your cluster. Availability of players (on both sides) and system resources are both important considerations. It may seem like undesirable meta-gaming, but at that level of participation there's no point trying to fight it anyway, so have clan leaders schedule battles ahead for both their sake and yours.

What you're asking for (1000+) is an MMO. Yes it's possible.

50 is way more reasonable, and is possible with some good network occlusion.

Seriously though, don't even worry about the game mechanics. Your biggest issue would be the fact that 30% of your community would quit the game once their faction starts to lose. That's the problem with modern day persistant world PVP games.

If I was way younger with tons of time to burn, and could afford to be online all day, then sure, it would sound neat. However, as most people with jobs know, I can only afford to be online playing games for an hour or 2 per day, where my impact would be minimal.

Regarding the issue of player attrition, I think it makes sense to have mechanics that encourage loyalty and discourage disloyalty, but only to an extent. Keep in mind, some of the most disastrous losses and most spectacular victories in human history have been due to political rather than tactical maneuvering. Also, desertion in battle and uncoordinated retreats are primary factors in reducing the total number of casualties. Very rarely has a real battle resulted in 100% casualties, because people will naturally leave when they realize they have no hope for victory. This also introduces very important strategic options. Managing your own troops morale and the enemy's morale can be as important as managing their rations or ammunition. Rather than introduce an artificial mechanic for this, you could let human player's real emotions dictate the morale of their army.

Most MMOs circumvent this issue by spreading players across different locations, allowing to distribute the server load as a result.

However, clever and optimized engineering could allow for a lot of players to be present at the same place simultaneously. For example, the mere fact of using "deltas" instead of communicating the position of every character every tick is ideal. It reduces the amount of data spread across by sendin gonly what has changed since last tick (who moved, their new coords, etc.)

With the technical aspect out of the way, there are mechanics that can help make large scale battles feel more appealing.

The inclusion of static defenses granting a side on their defensive end with an advantage will help pacing. If one side grows stronger (500 vs 300), all the losing side has to do is move closer to their base and utilize these static defenses. Assuming these defenses make them 50% more efficient, they are effectively fighting a 500 vs 450, which is much more balanced.

Of course, no one would willingly go on the defensive end and try to win from there, so there still needs to be incentive to win the fight, enough so that each side risks an attack.

The interesting part here is that, though players may be outnumbering their opponent (500 vs 300 for example), they might not cause an all-out attack. They might choose to prepare their attack and coordinate it efficiently, knowing that if the resistance is good, they might lose that edge and find themselves on the defensive end soon.

Is it truly possible to design a big battle game?

Of course.
Anything is possible, except time travel to the past and the Star Trek holodeck.
Your "big battle game" would be easier to design than to implement, as you already experienced.

I disagree. Implementing the game would depend on how complex the game mechanics are, so they would range from very easy(web-based 2D) to extremely hard(shooter3D);

However the more difficult aspect would be gaining enough popularity and people to fuel the need for 100+ players to be online at any given time. Although it is surely possible, it's not easy, and that's because you have to think of the implications that that number of people will bring both to the individual's experience, and to the community's experience. Also, finding a role/meaning/weight to carry for 100+ people in one battle while maintaining the illusion that the individual player is somehow special and matters in that humongous mass of people is hard.

Looking at Planetside 2, you will see that, while there IS one common battlefield, in one continuous war, it's actually divided into many skirmishes that rarely exceed 90 people total. Also, the quick team-join and squad system limit the player to a somewhat limited group of people with which he can relate to and befriend. The game keeps the Massive persistent battlefield idea more as an "illusion of grandeur", making the player think that his actions have an impact on the world around ( and it does, actually), which keep the player invested. The gamer starts to develop feelings of "nationalism" for his "country" and a bond of camaraderie forms with those on the same side.

The only other kind of persistent war-like zone I can think of are strategy games akin to Strongholds or Travian and the like. The former being better in the aspect of making the player feel more involved, while the latter is focused purely on competition.

In Strongholds the player, sooner or later, will be needed to join a house, and later a faction. From then on, he and his faction will fight to conquer and control as much of the country as possible, in a contest with other houses and factions. From that point, the real game starts. The game was DESIGNED to be played by the factions, by communities. Also, by using the vote system for control of the parish, even new players will feel like they can contribute and will commit to the game.

In Travian however, the accent falls on competition. Travian is, by all means, made to be played as an individualistic game. Players don't join guilds or factions because they want to make friends and contribute to the guild as a whole, like in Strongholds, but more as a means to survive, as they know that, if they stay solo forever, they will get conquered by the many. As such, while the faction becomes more important than the individual, the player will still fight for his own small bunch of villages. To exemplify, in Stronghold, the player doesn't care if he/she loses a few hundred soldiers, a couple of towns or the personal control of the parish, as long as the faction gets an advantage, nor will he feel bad for falling down the rankings because of it. However, in Travian, if you feel that you are falling to low in the rankings, or that you are way better than the others in your faction, you will either quit the game, or betray the former faction.

Both games were designed to fit thousands of people, actually they were designed to DEPEND on a huge number of people playing them. They managed that by being free (PlanetSide2 is free too btw), and finding ways to get the player engaged. Now, designing a game to be dependent on a huge number of people may be easy, but making that same game to be able to draw the player in and keep a tight grasp on him is hard. Stupidly HARD.