I feel I should clarify which melee units I think need a lot of help, and which are fine.
The two units that need the most help are swordsmen and longswordsmen. Spearman and pikeman are fine where they are. Let's delve into the reasons for this.
1. Swordsman and Longswordsman require iron, Spearman and Pikemen do not.
2. Spearman and Pikemen require very little tech investment, while Swordsmen and Longswordsmen require ironworking and steel, respectively. Civil Service is a solid tech in its own right since it improves your freshwater farms. Perhaps even more importantly, you'll unlock CS anyway on the road to Education, one of the most important techs in the game. Unlocking Swordsman and Longswordsmen have a big opportunity cost as far as developing the infrastructure of your empire and accelerating your tech progression; Unlocking Spearman has little opportunity cost, and unlocking Pikemen has almost none.
3. Spearman and Pikemen are able to fill the same role that Swordsmen and Longswordsmen do. They both can fortify in a hill/forest next to a city, exerting a ZoC and waiting for the city to drop to almost 0 health so they can cap it. The extra combat strength of SM and LSM doesn't really come into play here. All 4 melee units generally have to fortify next to the city because if they choose to attack it, you risk having them sniped and then having no melee units left. If you build a lot of melee to counteract the risk of losing all your melee, the city siege either takes longer (if you don't attack with your melee) or becomes much costlier (since melee units lose hp each time they attack, while ranged do not).
4. Spearmen and Pikemen have the added utility of being able to devastate mounted units. Their effective combat strength against these units is 16.5 and 24, respectively.
5. Spearmen require 56 hammers to SM's 75. Pikemen require 90 hammers to LSM's 120.
Literally the only thing that the iron-requiring units have over their cheaper, conveniently-unlocked, horse-killing, role-stealing cousins is a few measly combat points.
Furthermore, Musketmen are in an okay spot. Not good, because they're melee, but not horrible. Why?
1. They require no strategic resources.
2. The 2-range equivalent to Musketmen are Cannons, which can only be unlocked after teching to Musketmen in the first place. Comparatively, CBs and XBs can be teched to with more economical tech paths than SM and LSM, respectively. In fact, CBs are 1-tier lower than SMs, while XBs's tech requires 3 less than LSM's (XB requires engineering, LSM requires engineering + Bronzeworking, Ironworking, Metal Casting). Further, and more obviously, cannon's require a set-up while CB and XB do not, meaning Cannons can not kite.
3. They are not outshined by the anti-cavalry of the era, because Lancer's simply can not fortify under a city to exert ZoC + wait for cap.
That said, it's still not advisable to mass musketmen as a strategy (much to the disappointment of America/France) simply because MM, while good for fortifying under a city and creating pressure around it, are not actually good for attacking the city itself, and you'll want as many units as possible attacking the city. Hence, you usually see mass cannons, or even XB + cannon, at this stage in the game to take cities.
After all's said, I think it's clear that SM and LSM at least need a buff, if not MM as well. In retrospect, my suggestion to change the mechanics of melee combat falls flat since you don't want to buff Spearmen/Pikemen, as they admirably fill the role of "defensive/utilitarian melee unit." Perhaps a simple combat boost to SM and LSM really is all that's needed. Alternately, perhaps SM and later units in that upgrade path (LSM, MM, Riflemen, etc) should start with a promotion that reads "+25% damage when attacking other units and cities." My reasoning for this is that Spearmen and Pikemen are largely defensive/utility units, so SM/LSM should be offensive units. Note this promotion wouldn't be available to warriors so as to prevent total cheese with jaguars/maoris/etc, but would indirectly be available to those units by premaking them and later upgrading them.:
By now, enemy city strength would be around 50 - able to, with a ranged unit, oneshot your lancers if you chose the path of pikemen, but only dealing 10-20 damage per turn vs. infantry (their combat strength is a whopping 70, about 3 times that of lancers). It's around this time that, after getting artillery, I begin to steamroll every single other civ on the planet. Now that you know which paths building the swordsman/LSM or the spearman/pikeman lead, which one do you choose? Which one is better not for your short term goals but for your long term ones?
