There should be two "good" high-level units on the horsemen line. (Especially since there's a civilization that revolves around horses in the game whereas melee units are more or less clearly the best in all circumstances.) There's also a clear historical guideline to how this can work: heavy cavalry (shock cavalry, carried melee weapons like swords and lances, wore heavier former; canonical European knights fall into this category) and light cavalry (ranged cavalry with missile weapons like bows; the Mongols were an example).
Heavy cavalry should get a bonus against archery units and be able to use metal upgrades, light cavalry get a bonus against heavy cavalry and melee infantry and probably not use metal. Chariots should be an early-arriving horse unit that can upgrade into either of the two lines.
Chariots: 3/3/3, doesn't get defensive bonuses, can use bronze weapons, requires Animal Husbandry, Exploration, and horses.
Heavy Cavalry:
Lancers: 4/4/3, +40% against archery units, can withdraw from combat (10% chance), can use bronze, iron, and mithril weapons, don't get defensive bonuses, requires Horseback Riding, horses, and stables.
Cataphracts: 6/6/3, +40% against archery units, can withdraw from combat (10% chance), can use bronze, iron, and mithril weapons, don't get defensive bonuses, immune to first strikes, require Stirrups, horses, and stables.
Knights: 9/9/3, national units, +40% against archery units, can withdraw from combat (30% chance), can use bronze, iron, and mithril weapons, don't get defensive bonuses, immune to first strikes, require Warhorses, horses, and stables.
Light Cavalry:
Skirmishers: 4/4/3, +40% against heavy cavalry and melee units, can withdraw from combat (25% chance), don't get defensive bonuses, require Stirrups, horses, and stables.
Horse Archers: 7/7/3, +40% against heavy cavalry and melee units, 1 first strike, can withdraw from combat (35% chance), 20% chance to do 10% damage to attacking units, don't get defensive bonuses, require Stirrups, horses, and stables.
Elite Horse Archers: 11/11/3, national units, +40% against heavy cavalry and melee units, 1 first strike, can withdraw from combat (35% chance), 40% chance to do 20% damage to attacking units, don't get defensive bonuses, require Warhorses, horses, and stables.