If you have Nationalism researched, Riflemen are just as good as Guerillas for defence (both have 6 defence, and the extra 2 offence for the Guerilla doesn't matter much if you are using them defensively). If you have Replaceable Parts and Rubber, Infantry are undoutedbly better at defence and just as good at offence if you have to use them for that (for instance, to take out a 1-hitpoint Cavalry that failed to win a battle and retreated). I mainly use Guerillas in areas where rubber is not available.
If you have no Rubber (once you discover Replaceable Parts), securing some should be a top priority. Ideally you'll be able to strike quick with Cavalry. But if you have no Cavalry either, and were hoping to strike with Tanks (which require Rubber), then a targeted offensive to capture Rubber with Guerillas and lots of Artillery is probably about the only option left, and had best be done before the enemy gets Infantry (if possible).
My preferred units by era:
Ancient
Swordsman. Although not fast, it's by far the strongest unit of the era. The upgrade path isn't great, but if I'm fighting wars in the Ancient Age, this is the only unit that's both a challenge to kill and good on the offense. Horsemen have a better upgrade path and are fast, but cost no less and are considerably weaker. Iroquois Mounted Warriors and Egyptian War Chariots are worthwhile, however, and preferable to Swordsmen. (side note: building a large number of Horsemen in preparation for a mass-upgrade to Knights when you discover Chivalry [if you have a lot of gold] and then going on a rampage can be devastating and may be worth forgoing building many Swordsmen).
Medieval Era
Early on, Knights. Although expensive, they're both more durable and strike more quickly than Medeival Warriors, and thus worth it (ideally you'll be able to position them two squares from a city on a hill/mountain or in a forest - they can still strike, and unlike horsemen, they'll put up a good defence). Later on, Cavalry. Beelining for Cavalry is not a bad strategy, as long as you are playing militaristically.
Industrial Times
For most of the Industrial Times, Cavalry are still the best offence. Nevertheless, getting Infantry early on is often worthwhile if you are in heavy wars, oftentimes making it worth it to skip Nationalism entirely (although War-Time militarization may be enough to justify going for Nationalism). Artillery also proves to be very useful once the enemy gets a strong defence, and is generally a better long-term investment than Bombers - just be sure to protect them. Tanks eclipse Cavalry in my books, although there will be times even in the late Industrial/early Modern times where the speed of Cavalry (as reinforcements) will prove crucial to tipping the balance in a close battle. Thus, I'd advise keeping at least some Cavalry around (even if no new ones are trained) until Modern Armor are available. Cavalry can also be better than Tanks when facing weak opponents due to their speed.
Modern Era
Modern Armor are doubtless the best land unit, and going for them first is usually a sound military strategy (but remember they need aluminum and thus Rocketry). However, if you are suffering a successful invasion, a diversion to Mechanized Infantry first may be a better choice to stem the enemy tide. TOW aren't usually worth it unless you lack the raw materials for Mechanized Infantry, and Cruise Missiles, while cool, are a generally poor value. Keeping a couple around, on the railroad lines, to finish off ships that get too close to your shore (but aren't within the range of your air force) is okay, but using Cruise Missiles as your general-purpose artillery isn't cost-effective.
Of course, it's not a good idea to build
only these units - by all means, build Pikemen instead of Knights for defensive missions. But these are generally the best bang-for-the-buck offensive units.
Looking at your screenshots, your empire appears to be developing fairly well. The only red light is that you have many cities building Wealth. Generally Wealth doesn't generate enough gold to be worthwhile, especially before Economics. Cities generating wealth will also never ask what to build next, which may result in forgetting about them for many cities, leaving them underdeveloped (not that I've ever done that before

). There are times when it's a sensible choice - better than choking your economy with more Warriors if you don't plan to go to war, and it may be for the best in remote, entirely corrupt cities. But most of the time, building Libraries, Courthouses, or Marketplaces is a better investment.