Thanks for the feedback so far, here's Take #2:
Horseman
Requires Riding
Requires Horses, Copper or Iron
6

, 2

, 50

Doesn't receive defensive bonuses
20% chance to withdraw from Combat
-50% city attack
+50% vs. Siege Units
Flank Attack vs. Siege Units
Upgrades to Heavy Horseman or Horse Archer
Horse Archer
Requires Nobility and Riding
Requires Horses
6

, 2

, 70

Immune to first strikes
Doesn't receive defensive bonuses
30% chance to withdraw from combat
+50% Attack vs. Axemen, Swordsmen, and Heavy Footmen
Targets Axemen, Swordsmen, or Heavy Footmen first in combat outside cities
Upgrades to Cuirassier
The biggest change is that I've shifted the Horse Archer into the mid to late Classical Era (though Nobility isn't an ideal tech, let me know if you have a better suggestion from the second or third column). Historically this makes better sense as proper Horse Archery didn't really emerge until well into the Iron Age but it also remained significant throughout Medieval times. This approach also spreads the mounted units out a bit better; it felt a bit excessive having 3 mounted units in the Ancient/Classical era and only 1 per Era thereafter.
Its cost is increased but rather than increasing their strength further I gave them an attack bonus against their same target units so that they can still be countered by Spearmen and Pikemen in cities.
Due to a lack of art I can't implement Light and Heavy versions of the Horse Archer, so my goal here is to define a unit that has enough of a niche to still be useful alongside Heavy Horseman, without being too strong early on. Thoughts?
Standard BTS Knights and Cuirassiers are immune to first strikes too, but Chariots and Cavalry aren't. I'd given it to Horsemen for consistency, but I'm not sure what Firaxis' reasoning is here. Opinions welcomed.
Yep, that's pretty much what I'm going for. Though in the case of the Horseman, I consider the city penalty to also be mostly about cavalry tactics being ineffective in the confines of an urban area. Hard to charge with walls about.
Having a heavy version of the Horse Archer would be good, but there isn't enough art. For many civs it's been a struggle finding/assembling even a regular Horse Archer. The situation for Cuirassier variants is even worse.
And that's exactly what its bonus is!
This will be fixed in 1.18.