Sometimes I wish that normal infantry could retreat. It would make gameplay so much more realistic.
Well, they technically are, if you believe that the HP of units isn't just the lives of the men, but morale and cohesion - so a unit that's redlined and recovers back to full HP isn't just bringing in replacements for men killed/taken prisoner, but resupplying and getting morale back to Regular/Veteran/Elite levels. For a lot of history, a force that routed/retreated would be run down by the opponents; only cavalry and similarly fast units could actually break away from an opponent quickly enough to prevent unit destruction.
What I mean is that opposing armies never stayed in cities and defended them like this, but sent their armies out to fight others. Like in WWI, the Eastern an Western fronts were not based on defending cities, but rather territory. If an army was defeated on the field, then there was (usually) not a secondary army defending the cities behind the front. The cities would be captured because the countrie's armies were defeated on the field.
Point taken - but Civ isn't a military sim, it's an empire sim. And if the enemy gets close enough to attack your cities, you're already in some kind of trouble in most cases (as you lose the productivity of the tiles the enemy units are on), so engaging away from the city is preferable. Also, at least in ancient times (my grasp of politics is rather fuzzy past the early MA), the destruction of the field army usually led to the capitulation of the losing side - Cannae, for example, would normally have led to the surrender of the Romans to the Carthaginians. The Romans, being jerks, however, decided to keep fighting. Up through the MA, you actually have the countries' major players on the field (the Harolds and William in England 1066, for example), so a crushing defeat would directly affect the losing side (and possibly the winning side as well), as their leaders would be dead/captured.
Essentially, the problem with your ideas is that they require things the Civ engine doesn't really take into consideration (for example, you can smash AI armies in the field the whole game, but if you don't take cities, you can't make them surrender).