You don't need as many units as the AI to defend your cities properly. Attackers always have a disadvantage, and you're smarter than the AI anyways. However, if you want to obtain peace without paying the enemy lux/GPT/gold, then you'll have to have more military power. Otherwise, they won't make peace with you - they'll constantly see you as the victim, yet to be crushed. So the best way to end a war is to go full-out on units, crush the enemy, and get a fair peace treaty (if you're lucky, you'll be able to actually sue them for peace!).
This information is especially relevant on Emperor/Immortal/Deity.