7th Circle said:
Is this within the context of a single game or over multiple games? Alternatively, are the strategies developed before the AI actually plays the game? In other words, how exactly does this work?
You can genetically alter probabilities to perform certain actions as an example.
This is the text idea behind it:
The AI decides to do A with the desired result of B. Once it is possible to measure whether or not the desired result occurred, the AI decides whether or not it was successful with achieving B.
From this point, the AI slightly modifies the probability to perform A based on whether it actually achieved B or not. (If successful, then A becomes slightly more likely to occur...modified by a SMALL RANDOM amount; if not successful, then A becomes slightly less likely to occur...modified by a SMALL RANDOM amount.)
An example would be this:
One game, the AI decides to be expansionist. It spreads as many cities as it can...but falls flat on its face due to money problems and the like that crop up, and loses the game--the ultimate unsuccessful "B". The next game, the AI is less likely to be expansionist, in order to attempt to win.
For a game as long as Civ4, though, small milestones should be used, in addition to whether or not the game is an AI win or AI loss. These milestones could be similar to:
1) If tech trading, does that AI reach the new age first?
2) If tech trading, does total production (or total science, or total food) of AI increase more rapidly, and compare more favorably to the production/science/food from other civs in 20 turns (or a random, but moderate-length, number of turns)?
3) If using great people for A, is there definite benefit after 20 turns? (Again, measures could be production, science, food, etc)
You get the idea, hopefully. Then there can also be game-wide probabilities...one that comes to mind is a balance between how much the AI should focus on food, science, and production (which are often a trade-off, though not always of course). These game-wide probabilities get modified based on whether or not there was a win or a loss.
Let me illustrate a bit more. Say there was a probability of 1/8 to focus on food, 3/8 to focus on science, and 1/2 to focus on production. The AI then chooses a short-term goal...say for instance #3 above, since a Great Person was just produced. In order to use that goal, it needs to decide whether to focus on food, science, or production, so it generates a random number (say, 0.451), and maps that to science (0 - 0.124 = food, 0.125 - 0.499 = science, and 0.500 - 0.999 = production, using the probabilities above). At this point, the AI notes what its current science rate is, relative to the others involved in the game. Say for instance, the AI's science rate is 0.924 of the leader's science rate.
The AI then decides what action to do (ie, how to use the Great Person), and performs that action (this could be done the same turn or after several turns, depending on the action to be performed). It keeps track of what result was used, and what probability(ies) were used to determine that result, and whether to modify them up or down if successful.
After the action is performed, the AI waits 20 turns, then measures the new science rate relative to the other players. Say for example the AI's new science rate is 0.921 of the leader's science rate in the game. This would be considered a minor failure of the action, so all probabilities would very slightly get altered for failure (ie, altered the opposite way as if they were successful). The amount of alteration SHOULD include a random component, but should also take into account the degree of failure (in this case, it is very slight). In other words, had the AI's rate fallen to 0.621 of the leader's, the probabilities would have been altered by a still minor, though more noticeable, amount.
Of course these are only primitive examples, but I want to illustrate the point clearly. The probabilities could be functions that vary based on how much is being produced and the map conditions, or based on turn number, instead of set numbers--then numerical methods for least squares could be used to find an approximate function for the new altered probabilities. There are also techniques for "mating" successful AIs in addition to this "mutating" idea of slightly modifying probabilities based on a measure of success or failure. In this way, the AI gets smarter than its programmer, and adapts to your strategies as well.
That's what happens when you ask a physics/computer science geek how something works...I hope I kept it understandable to the general public...and especially to Firaxis!
Sam
However, while certainly not perfect, I really feel that the civ 4 AI is not at all bad.
I too want an AI that is SMARTER and doesnt cheat. The argument that giving cheats to the AI is what makes it smarter is like giving the class idiot all the answers to a test and having him get a perfect grade and saying he is smarter. 
