Much of your post was either irrelevant to what I wrote or you seemed to misunderstand what I intended and went off on a different tangent or wrote something out of ignorance.
Dividing turns into phases makes the problem easier.
You misunderstood, in PG3, units had 2-3 stages of full move and they had the option to fire during those stages, though firing reduced their move or might cancel the option to move a 2nd or 3rd time, depending how many times they fired and how many times they could fire. Units in the PG series had a very wide range of stats regarding move and attack.
Fixing the maximum number of units on the map by a starting number of units makes it easier.
In the SSI games, there was no fixed starting number of units. They were set up so units could be added as reinforcements, both for AI and player. There a max number of units each side could have in total, not how many this total could be starting units.
Not having units that have 5 moves in a single step makes it easier.
In PG3 there were many units capable of 5 hex move per move stage and they could have 3 stages of movement.
Playing a defender rather than an attacker also makes it easier.
For an AI, that is true, since they don't have to move as much. But any AI designed for one of these games should be able to attack and defend. Other developers can do it and have done it.
Playing fixed scenarios with fixed objectives makes it easier.
Yes, but the base parameters used in setting up an AI can do this for games like Civ, also. If all of the PG (and the rest of the SSI games) were simple one off scenario games, that would be true, but they have campaigns of detailed scenarios, the ability to make random scenarios and have random campaigns made up of random scenarios. A Civ game is like a series of scenarios strung together, much like a PG game random campaign. In any given situation in Civ, there are viable options and useless ones. You set up the AI base to weed out the useless options first, then work on the viable options.
(i)If the AI expects to be fighting on one front in this game, and without any peek ahead forewarning or planning it gets attacked from another flank it will collapse horribly from poor decisions.
An AI can be set to be able to function on multiple fronts. Look at chess AI to see this.
(ii) The nature of the fixed unit composition and scenario objectives makes it easier to give the AI specific instructions that work. "You have these units, you aren't getting any new units, the enemy has these units, they aren't getting any new units, you need to accomplish X to win, they need to accomplish Y to win".
Irrelevant to the SSI games and my previous comments, see above.
Other issues like the level of bonuses for formations can make errors less costly as well, I don't think any formation errors in those games were as punitive as say a 15% flanking bonus and a 15% discipline bonus (admittedly these can be/have been lowered).
All the SSI games had vastly greater variances provided by bonuses from unit types, terrain, experience. One defensive bonus some units could have (and it was determined randomly when it would occur, by the game) tripled the defense stat of the unit. Units in the SSI games did not just have 1 or 2 combat stats, as in Civ, but had as many as 10. The naval games were more complicated, still.
I'm not suggesting its neigh impossible I'm suggesting that it is much harder to do well, and uses more resources like CPU time. And as for blaming 1UPT for Civ 5's problems it is nothing like blaming evolution for creationisms problems, evolution and creationism are not components of each other, 1UPT is a central component of Civ 5.
You misunderstood. The comparison with Evolution and Creationism was selected for another reason. Creationism is the obsolete, old theory, Evolution is the advance. Civ4 combat is the obsolete form, Civ5 combat is an advance over Civ4. Blaming 1upt for Civ5 failing is absurd because much of the changes to Civ5 were not because of 1upt, but because the game was changed for other unrelated reasons. In other words, apples and oranges, like Evolution and Creationism. An example.
The claim 1upt cause units to be costed so high.
The game was designed to be simpler and more casual than the previous one (Civ4). They also decided to limit map sizes, like in Civ4. This reduction also affected the numbers of units (and probably the number of unit types, too). If you want to limit unit numbers, one way is to raise the cost of building them. This would be taken irregardless of 1upt or stack of doom. Yes, 1upt means if you have too many for a given map size, your game becomes a traffic jam. But that doesn't have to be so if you have more tiles (either larger maps, or a greater number of tiles per map, as I wrote earlier, this would alleviate the traffic jam problem) or you allow units to be able to pass through friendly units. So as you can see, claiming 1upt forced the high cost of units is false reasoning. The reduction in size and simplification of the game is the main reason. And I didn't even get into the high costs of other things like buildings, units were also scaled to better match building costs. The reduction of map sizes, and number of cities one could use, in Civ4 is one of it's main failings in comparison to earlier versions. It is unfortunate that the company decided to keep this in Civ5, but that is a decision made by the suits. They wanted a simpler, shorter game that would appeal to more casual gamers. They started this with Civ4, Civ5 is a continuation of this.
As for 20 year old AI, that's a complete absurdity, the AI has been rewritten with every version of the game, ask any civ IV modder about the AI in civ IV, it's written in a different language from the original civ I for starters.
The core game engine is essentially the same and this means a lot of the base AI routines are also the same. The changes you talk about are additional code added to the core. This is why new ideas incorporated into later versions frequently don't work very well for the AI. One of the reasons I mentioned Civ's pathing failures in the first 4 versions is that it is a consistent AI failure of the game from Civ1 through Civ4. This means they used the same code, or almost the same code, on this. It was not a new code sequence. Most likely, pathing is in a core program that gets passed on to each new version of the game. The pathing has been improved in later versions, but in a way tweeking of the original code would produce, not by gutting the old and making a completely new code. Pathing is very simple AI code and would be simple to design a new one. The fact it was not designed new means that much of the original game code is still in the game. Other evidence for this are that the Civ series has consistent failures in other parts of the game that are carried from version to version. The inability to bombard with artillery. This was not in the 1st 2 versions, but added to Civ3. The AI failed to use bombardment with land units. This means the bombardment was not part of an all new core code, but something that was tacked onto the original code. The original didn't bombard, the add-on was an attempt to add this feature, but it failed because the core was not designed for it and could not be completely modified to use it unless it was rewritten. In Civ4, rather than fix the AI, they took out the bombardment instead. Why? Because the original core code couldn't do it (or the developers then couldn't find a way to do it under their budget) and that core was going to be also in Civ4. If they had programed a complete new AI code, they could have programed the bombardment in, but since they removed bombardment, it is obvious that older code was being used instead.