Hmm, that's a tough choice. I don't typically build mounted units that much that often unless i'm looking for a power scout or a specific flanker against siege / ranged units.
I'd say my typical break down in invasion forces are,
40% - Melee
30% - Ranged
20% - Siege
10% - Mounted
Obviously the break down is subject my intended target, or if i'm in a defensive or offensive posture. As time goes on though and my army gains more promotions I start cranking out more siege units to speed taking cities.
Here are a few thoughts I had when reviewing UU mounted units in the Civilopedia.
African Forest Elephant - Stronger than Horseman (Which it replaces), Creates combat modifier in nearby units (-10% Combat Strength), and creates GG more frequently. So basically, its a horseman/maori warrior/samurai rolled into one. Also, doesn't require any strategic resources.
Companion Calvary - Pretty straight forward upgrade to the Horseman, moves faster, hits harder, and generates GG faster. Does require horses though.
Mandekalu Cavalry - Also a straight forward upgrade, but of the Knight. However, it can attack cities at full strength. Which means you could spam a fast moving army of city takers. Only downside with this strategy would be the lack of synergy with using siege units for those hard to take cities.
Sipahi - An interesting replacement to the Lancer, sees and travels further than a Lancer and pillages at no cost. Also, with the recent patch changes units heal when they pillage so this unit could make a great scout and harasser.
Cossack - Another interesting replacement, but of the standard Calvary unit. All it gets it is a combat bonus against damaged units, but when using it in a front line counter attacking role it can really make a difference.
Hussar - Another replacement of the Calvary, it is faster, and sees further, but most importantly it gets a bonus when flank attacking. This makes it idea in both a scouting role and in heavy combat role to help finish opponents off in a unit thick environment.
Feel free to look over your options in
http://www.dndjunkie.com/civilopedia/UNIT_HOME.aspx
-Dental