They didn't. They might share some similar strategic roles (policing, mobility) but I'm not sure if it's fair to compare tactics of pre-modern wars to those of modern wars as they are quite different. In fact, I would argue this backwards: All upgrade paths should end somewhere. Longswordsmen and Crossbowmen shouldn't upgrade to rifles and Infantry shouldn't upgrade to Mech Inf.
I think this change might be well worth investigating because it resolves, or helps resolve, a couple of problems in one go (production is expensive compared to upgrades, upgraded veteran units are way stronger than anything your enemy could muster, you never build musketmen, etc.)
Of course, people will rage that they can't keep their level 25 units around anymore.