What is interesting is all the talk about melee vs anti cavalry without much talk about how cavalry and other units play into this. Yes melee on paper don't have any weakness but now let say we remove the whole anti cavalry line and now melee will look terrible.
Indeed, I intended for this point to be implied in my previous post. Infantry are weaker against cavalry, AC beat cavalry. I didn't bother with AC v Cavalry because Cavalry only have minor bonuses to help them dampen damage from AC (+5 for LC from Caparison. HC on the other hand have promotions that could potentially be OP when combined in certain scenarios).
You're essentially saying that, putting aside variables, Spears lose to Warriors when defending, and AC lose to Melee once we get to the Modern Era. How are AC then good at Defending again?
Well, AC are supposed to lose against Melee as per the rock-paper-scissor rules. The point is that when defending, they don't lose by as much, and AC benefit from defending more than any other unit. Spearmen lose against warriors, as they should, but they win if they have a terrain advantage. Not unreasonbale.
But you still raise a good point, which is, despite all of what I said before, are Infantry still better than AC when defending overall? Obviously, they're an equal match to other infantry, but in most eras they get shredded by Cavalry (particularly HC). This isn't the case only when melee are "ahead of the curve" as
@Sostratus puts it.
Chariot > Warrior
Knight/Courser > Swordsman > Chariot; Swordsman = Horseman
Cuirassier/Cavalry > Musketman > Knight / Courser
Tank/Helicopter > Infantry > Cuirassier/Cavalry
Modern Armor > Mech. Infantry > Tank/Helicopter
They leap frog each other but it starts and ends with HC > Melee. Clearly LC excel in pillaging and targeting ranged/siege units (also based on their promotion tree) while can also act as support/utility (flanking bonus, escorting). The only promotion that Infantry have to boost their chances against cavalry is Urban warfare, which makes sense. Tanks can't maneuver too well in districts. So I guess you should try to place infantry on districts as much as possible. So without AC, infantry
should be highly vulnerable to Cav and HC in particular (their promotion tree makes them an even bigger threat).
So to answer your question, why are AC better at defense? Because they're the only class able to close the gap against the class that should have an advantage over it. AC, with the right tactics, can nullify melee attackers and beat Cavalry too.
After going through all of this, I'm more convinced that the game balancing is not that bad at all, it's just the AI that's bad. Walls and air units aside, then in theory, to defend you can rely mostly on AC and Ranged units, this doesn't require strategic resources which means you still have a chance even when lacking resources. Be prepared to use/sacrifice L-Cav to take down enemy siege units. Also on flat land, AC won't have the tactical advantages required to counter melee, which is fine, melee are supposed to have an advantage over AC, L-Cav can be helpful here too.
To attack you rely mostly on Melee and Heavy Cavalry with Siege to take down walls as needed. The promotion trees provide a surprising amount of tactical depth that is unfortunately irrelevant against a dumb AI. Firaxis should hire someone who helps design "tactics" games to help them sort out AI combat. Hello, Nintendo (Fire Emblem team)? It is really HC who can potentially run riot in the end game with 3-4 promotions leading to situations where they beat even AC.
One change I would make is to lower spearman cost from 65 to 50 so their cost-efficiency is in line with warriors, incentivizing players to prioritize offense/defense early on (in multiplayer).
@Sostratus' suggestion to bump them +5 str is reasonable on the surface but can lead to broken scenarios: they'd rule the Ancient Era, and if an opponent can't find Iron then they'd also be top dogs in the Classical era, until they're promoted to Pikeman who will continue to rule the Medieval Era against a foe without iron.
Bumping Pikeman +4 is more reasonable, although in that case a Pikeman with Thrust would beat a Swordsman with Battlecry which I guess is debatable? These changes would make AC stronger than melee overall in early game. Probably another cost reduction here is the more reasonable route though.
If you look at the cost-efficiency curve I
posted, AC start off far below the curve before following it in later eras. Not sure what the logic there is. Bumping both infantry and AT +5 might be an okay move, but they'd overpower ranged/siege units if changes are not made there as well, and imo would also make Infantry too level with Tanks. Tanks should be able to bulldoze infantry imo.
Edit: Modern AT +5 is maybe okay, though they'd beat infantry even when attacking, so probably not.
Tweaking maintenance costs might be reasonable as well. Just logically, defensive units shouldn't be expensive to maintain, but I haven't looked into those costs to have an opinion about their balancing.