What is a single unit versus a massive army? Strength in numbers.
I may have worded it badly. In Civ, the comparison is between (for example) a) one stack of 15 units, and b ) five stacks of three units each. We can also add c) 15 single units as a third condition. All these three conditions have the same "strength in numbers". In Civ4, due to its mechanics, option A is almost always the best one. In Civ5, option C is the only one left since the others have simply been removed. What I'd prefer is a rules system that doesnÄt simply forbid one or more of these options and doesn't clearly favor one either, but lets the player make a strategic decision how/if to split his stack, which can turn out to be good or bad, depending on the counters the opponent has.
What real-life examples are there to large army counters that don't work similarly or better versus smaller ones?
There are lots, I'd think. The most simple one is artillery. try to fire a cannon on a battlefield with 20 swordfighters on it vs. firing the same cannon on the same area filled with 200 people. In the second condition you have a much higher chance to hit and therefore will do more damage (on average) with each shot. This could be easily modeled by giving artillery a chance to hit (miss) that increases (decreases) with the size of the stack it fires upon, or (if you don't want the potential player frustration when many shots miss) by scaling the damage accordingly, or by letting artillery do a lot of collateral damage. Other examples would be gas (the same amount of gas will do much more damage when many opponents stand close together), or (to an extent) minefields, though the latter would be difficult to implement. But generally, any area of effect damage, would work well as a counter against stacks.
"More damage" isn't the only way to implement a plausible counter against huge stacks either. Another option is the choice of defender. In Civ4, the best defender will always defend against every attack, which often makes it difficult to "crack" a huge stack even if it has just a few good defenders. However, why do the mechanics assume that in a huge army, the few good defenders can as easily and reliably reach the point of attack as in a small group when there's possibly a whole army of hundreds of people between them, instead of free space? That's not realistic either. So, another possible counter against huge stacks is to implement a chance that a non-optimal defender will have to fight the battle, and have the chance increase with the number of units in the stack.
What I don't get is how "remove the stacking feature" has turned out to be a widely accepted "cure" against the power of stacks. Imho, if one wanted to remedy that, there are many options to
fix the mechanic by implementing appropriate counters, instead of removing the whole feature and calling that an improvement. The whole discussion about "where should we put the hard cap to prevent stacks of doom" is, imho, based on the flawed premise that hard caps are a good solution at all. Usually, implementing counters is a better option (and leads to richer games) than implementing hard caps.