I think archers not obsoleting is due to their ability to upgrade to Crossbowmen. Therefore, archers don't become obsolete until you can build both Longbowmen and Crossbowmen.
As for weapons promotions, I believe that's for balance reasons. Archers are only T2 units, at the same level as Axemen and Hunters, for example. They're already 3/5 units with a decent bonus to city defense, allowing them weapons promotions (even just bronze) would make them extremely powerful for their cost. Consider that they're an equivalent ~6.25 already with no promotions, and ~7.5 with just CGI. That's far, far stronger than anything else available at its tier.
Here's my best attempt at a lore/flavor explanation for it though: Archers' bows lack the force to penetrate much more than cloth and flesh, and better arrowheads would have no effect on their effectiveness. Longbows have enough force that a stronger/harder arrowhead (bronze or iron) improves their ability to penetrate armor, therefore they have a use for it. Marksmen require Mithril to upgrade to, so they already have Mithril weapons, making weapons promotions redundant. Basically, archers' bows are only really effective against cloth and flesh, so arrowhead material doesn't really matter. Longbowmen can penetrate armor, and better arrowhead material improves penetration.
As for Horse Archers... hmm. Haven't had a good look at their bows, though I suspect they use composite recurve shortbows. In which case, they would likely have the power to penetrate armor with the right arrowhead. Still, balance reasons. Although, having a mounted lancer upgrade to an archer back to a lancer is rather odd. Not to mention what happens with UUs.