This is incorrect.
Horsemen do not get another move after they attack, even if they move only one square to attack. To get an extra move after attack, you need either the Blitz upgrade, or to overrun the unit you are attacking.
Keep horseme inside of a city is a waste of hammers, they are at their best when they are roaming the map, the extra movement that they get is why you they cost more than legions.
Catapults, Canons, Defensive units, and Artillary all stay inside the city if they attack a unit that is 1 square away from that city.
Catapults are really good for early counter attack, and they cost just as much as horsemen, so that's another reason not to use horsemen for counter attacking.
I like legions for counter attacking because they are so cheap, and you can spam a lot of them in such a short period of time. If they die, so what? It didn't cost much to make them. It hurts a lot more to lose a horseman army.