Have you ever thought about limiting army size by limiting the number of units that can be built? Say, 2 units for each city you own, an extra unit if a city has a barracks, and 2 or 3 extras come with the palace?
As for upgrades, I'm not a fan of taking them out but I see your reasons. Why not...