Yes, that's true. It seems upgrading costs way more than just building a brand new unit. For exemple, when upgrading my whole defensive army of archers, lomgbowman and such to musketman, it cost me more than 10,000 gold. I often end up just upgrading those units with good promotions and building my new army from scratch...
