Part of this depends on civics. If you're in hereditary rule, you have a use for a lot of obsolete units. Even if you're not, you need at least one in every city.
Some, of course, should be upgraded. I generally make sure to upgrade the highly promoted ones - which for me usually means...