Regarding upgrades:
I don't know 100% when one can't built units and when one can't either.
But the important thing I know.
Can build maces when you are lacking
either rifling or mil/sci, when you have both you can't build maces. Thats why it's kind of important to build a bunch before you get both techs.
There are charts ingame in civopedia under the category "unit upgrades".
Unit upgrades cost are 20+ diff_in_hammer_cost*3, thats why a treb (which cost 80hammers) cost 80g to upgrade to a cannon (cost 100hammers).
And why it costs a fortune to upgrade chariots to cuirs, and why upgrading HA's to cuirs is a total ripoff and something you only consider because the great person system allows you to get a huuuuge stack of cash that you can't use productively any other way.
The treb->cannon upgrade is probably the most obvious example of a useful upgrade.
You reach engineering aeons before you complete the key techs you need. (Be that steel alone, or steel + rifling or steel + milsci).
During the time period between engineering and that final tech, what can you do with your hammers?
Either you squander them on some non essential infrastructure, or you build research/wealth which could speed up the arrival of the tech, or you build trebs.
If you have a very hammer heavy empire, best choice might be to just build wealth to reach the tech faster, because you can after the tech produce units fast anyway.
If you have a very commerce heavy empire though.... You are well rewarded for building trebs, because after the final military tech, you can then focus entierly on building rifles/grenadiers with the few hammers your empire got, and the rest of the empire can continue to go commerce at 0% slider, "building" cannons of the trebs you had.
And the reflection:
Even if they just reach tech parity all of a sudden you need 3 catapults's to soften a longbow, 2-3 cannons to soften a rifle, etc. I always run out of siege units this way.Maybe I'm just not bringing enough siege, but if I bring more then I don't have enough defenders to leave behind and some HA/Knights/Cuir sneak in and snipe back the cities I've taken.
I too lose cities to sniping mounted units, or other things.
Experience and attention reduces the frequency, but importantly is to accept it and retake them. War is messy.
And by what you write, I would say that the problem is not building enough units.
It can help to think of units as the cost of cities.
"Now I'm paying 6 catapults per city, up from 2 catapults prior to longbows. A pity I didn't strike earlier as 100 hammers lost per city was nice.
300 hammers per juicy city is still lovely though, I will keep going."
And best way to build enough units is to just put yourself in the mindset of Shaka/Ragnar/GK.
I frequently find myself massively overbuilding units, but that is usually a very comfortable problem to manage. Having built to few units, and being forced to a early peace treaty after just a pair of cities because reinforcements don't reach the front is a worse problem.