Well ... building units in Civ 4 is a lot like the Prisoner's Dillema, arms race, etc .... its part of the key Economy vs Military choices that are integral to Civ IV.
Personally I see all these AIs build an incredible amount of units ... seemingly beyond what a human is capable even with same bonuses ... yet hardly ever USING such units and, given parity with humans, relatively crashing their economy.
I guess it doesn't matter ... but it is kind of irritating that the AI hardly ever USES their units

... again, a lot of it has to do with the "prisoner's dillema" of building units.
Dillema -> do you not build units and have an awesome economy? or do you hoard units in prep for being attacked? If all the AI never built units, humans could walk over them all day with warrior rushes ... however if the AI always builds units and hardly ever uses them (like now) the human can just grow economically and build units later. At least relatively speaking.
I mean really, if you want to go for an economic victory you want simple Military parity. If you are going for a Military victory, you want at least Triple the enemy military ( in effectiveness, whether through numbers or promotions).
I "Suppose" you could set
some AI to be merely reactive, simply responding to military build up by trying to achieve (average) military Parity with all neighbors (average by median). Unless they had economical exess to support more soliders, and then build soldiers till that excess is expended, and if they have a greater temporary advantage, to utilize that advantage against weakest nearby opponent/ city.
And then I suppose other AIs (fewer, just the truly war-like ones), will look for Military Dominance (at least 50% more than the next guy) and use their soldiers more often in softening tactics. The more soldiers are on a square (than the other guy), the odds required to make the "attack now" decision decrease exponentially. At numbers parity, odds are as they are now. And if they have less units, odds required to "attack" increase exponentially.
EXAMPLE: Army A: 10 soldiers, Army D: 10 soldiers. Odds required to attack is ... 60%+
Army A: 12 soldiers, Army D: 10 soldiers. Odds required to attack is ... 56%+
Army A: 20 soldiers, Army D: 10 soldiers. Odds required to attack is ... 30%+
Army A: 30 soldiers, Army D: 10 soldiers. Odds required to attack is ... 10%+
and on the flip side ...
Army A: 8 soldiers, Army D: 10 soldiers. Odds required to attack is ... 70%+
Army A: 5 soldiers, Army D: 10 soldiers. Odds required to attack is ... 80%+
Army A: 2 soldiers, Army D: 10 soldiers. Odds required to attack is ... 95%+
Army A: 1 soldier, Army D: 10 soldiers. Odds required to attack is ... 99.5%+
as the soldiers attack ... the computer calculates after every 5 battles. If they sacrificed 5 soldiers for nothing (the RNG hated them) if they don't have enough excess soldiers ... then they may cease the attack. However, it would allow them to throw away extra soldiers in order to give them better odds with their prime soldiers (something they SHOULD have been doing the entire time!

) ... and it would allow them to use all those extra soldiers to MAKE VICTORIES instead of just waiting for good odds. :/
as for Heroes ... should always be accompanied by at least 1 similar unit. (1-5 unit bodyguard at all times). A hero should never attack alone. A hero never attacks below 90% odds ... and a hero (does NOT) count towards the unit-count in the previous equation. Also, regardless of odds, any Bodyguard units never attack!!
// that's how I would do it if I knew how to code the AI