Don't underestimate mounted units. They're the best attack unit until you get tanks. They tend to have high attack values and good movment. The retreat ability helps negate the low defense disadvantage.
As soon as horsemen become available, you should start building them. By then you may still be in the rapid expansion phase, in which case the horsemen are good for tracking down barbarians before they can get to your settlers. Also, barbarians provide your AI opponents with military promotion opportunities, and you'd be best served taking those opportunities for yourself.
Once you get knights, you not only have a strong attack and good movement, but relatively good defense as well.
If you have a tech lead over your opponents, calvary are particularly deadly. You'll want to slowly build up your stockpile of mounted units throughout the game. The moment calvary become available, upgrade all your mounted units, then go on a campaign of conquest. The only things calvary need fear won't come until the next age. With their extra movement and increased fierpower, you can conquer a lot very quickly.
This works out perfectly because this expansion comes at just the right time for you to build up infrastructure to prepare for cranking out tons of tanks once they become available. Assuming you have any enemies left, tanks and the modern armor they upgrade to are the kings of modern combat.
Once you're dealing with waves of tanks, your stacks of calvary will still be useful. I use them to come up behind the tanks after cities have been captured. The calvary are used to suppress rebellion in newly conquered cities. Calvary can also be used for other secondary tasks like picking off enemy units within your newly conquered territory, or to protect workers near the front lines. All of this leaves your tanks free to do what tanks do best: conquer enemy cities.
Another upgrade path you may be missing out on is artillery. If you are playing at low levels of difficulty and can maintain a significant technological advantage throughout the game, then artillery might be useless to you. However any time you face an opponent of equal or better technology, you'd better have stacks of artillery units (protected by your best defenders, of course) ready to move during any invasion. Bombardment can weaken a superior enemy to the point where you can capture cities that would otherwise be beyond your reach. Bombardment units are the great equalizer.
Bombers and stealth bombers can be extremely useful in modern era wars between roughly equal opponents, but keep in mind they have range limits and generally don't do as much damage as artillery-type units. The more modern artillery units are great because their increased firing range means you don't have to get them as close to enemy cities, which in turn allows your invasions to go faster.
Speed is one of the main drawbacks of artillery. They can only move one unit per turn, so they can slow down your invasion. The other problem they have is a complete lack of defense. They're as helpless as a worker, so be sure to bring along defenders (like those spearman → mech. inf. guys you have so much of) with any artillery stack.
