The AI needs formations.
It seems to be able to think to itself "Ok, I have two longswords and two crossbows, and I want that city, so I'll send the four of them over there."
What it needs to do is, having identified the four (plus or minus) units that will work towards an objective, assign the units into a formation. The units would then attempt to remain in this formation while moving. (They won't be able to maintain the formation perfectly because of terrain obstacles). For example, if the crossbows get left behind, then the longswords will stop advancing. And the crossbows won't ever advance past the "front" of the formation. And if too many of the units in the formation die, then the remainder retreat.
That is, essentially, what I do.