New AI playing to win system

jdog5000

Revolutionary
Joined
Nov 25, 2003
Messages
2,601
Location
California
(**Mild spoilers**: This thread describes a new system for having the AI play to win the game and which AI personalities will be more likely to try for certain victory types. Any detailed info I post about specific triggers or strategies the AI uses will be wrapped in spoiler tags)


The new playing to win system is rounding into form and will be a part of releases 0.90 and higher. With this system, the AI will (finally) have two new fundamental abilities:

1) A strategy to aim for one or more of the available victory types.

2) An awareness of whether the other players around it are close to winning in a particular way.

There have been patchwork pieces of this before, but now it will be much more thorough and really influence the actions the AI takes from the mid-game on. Of course there will be XML options to turn this off, but in my mind it's one of the biggest possible steps forward for the AI in terms of making the game more exciting and strategic.

The AI will have strategies to go for Conquest, Domination, Culture, Space, and Diplomatic victories. Each of these victory strategies is modeled off of how Blake set up the Cultural victory strategy in the Warlords Better AI project (which was eventually folded in to BTS). For each victory strategy the AI will have 4 levels measuring how committed they are to the strategy.

- The first two levels of each strategy (ie, Conquest1 and Conquest2) are for somewhat randomized tendencies for the AI leader. Since this is new, there are going to be new variables in XML to control how likely an AI is to try the early levels of these strategies (and you can help pick them!).

- The upper two levels (ie, Conquest3 and Conquest4) measure when a player is closing in on a particular victory condition. They do not require the AI to have run a lower level strategy, but the lower levels are designed to put an AI in a position to adopt the higher level one. These levels also have no random component and no AI personality component, but instead are based on in past game actions and measurables like power or culture. An AI will only run one level 4 strategy at a time.


Here's how I'm planning to have the randomized tendencies work for the first level of each victory strategy:

iValue = Leader personality value for victory type from 0 to 100

iValue += Random dice roll from 0 to 99

if( iValue >= 100 )
. AI starts level 1 for this victory strategy

So, if the AI tendency value is 0 then it will never adopt it, if it's 100 then it will in 100% of games. The random dice roll is done in a very particular way. A random number between 1000 and 10,000 is drawn once for each player per game and saved. Then the five dice rolls are set up so that the AI will have a spectrum of random values:

Culture dice roll = (strategy rand + 10) % 100

Conquest dice roll = (strategy rand + 30) % 100

Space dice roll = (strategy rand + 50) % 100

Domination dice roll = (strategy rand + 70) % 100

Diplomacy dice roll = (strategy rand + 90) % 100

where % is the modulus, ie the remainder after division by in this case 100. So, an AI might get 90 from the random roll for domination, 10 for diplomacy, 30 for culture, 50 for conquest, and 70 for space, but whatever the specifics it will always have one strategy with a random 80+, one with <20, and the rest in between. The reasoning behind this is that each AI should start at least one of these first level victory strategies from the beginning of the game, and maybe two strategies.

There are a couple adjustments for special game options, like Aggressive AI means AIs are more likely to adopt militaristic strategies (+30), while always peace rules out militaristic ones and boosts culture and space (+40).


Picking values for a leader

So, for each leader we need to pick 5 numbers to set their natural affinity for each victory type. Here's how the distribution of tendencies should work:

- 0 is an extreme value, meaning the AI will never do a strategy in a normal game. These should be very rare.
- 30 is a good middle for the distribution and represents an average affinity for the victory type.
- 70 is also an extreme value for a normal game, they should be very rare.
- Each leaders 5 values should add up to 150, so that on average they start a game adopting 1.5 victory strategies at level 1.

As for what makes sense for a particular AI, Civ4 been around long enough for the community to know which leaders are good/decent/bad for different game styles. Having traits which play into a particular strategy certainly helps as do other AI characteristics like their odds for war and building units. Historical actions by the leaders may also be relevant.
 
This post sorts AIs by their propensity for different victories from High to Low. Only ~13 AIs will be listed for high (weight 70 down to 40), and ~13 for low (20 down to 0). The top/bottom three get 60 and up, so for culture Hatshepsut gets an 70, Suleiman 55, Pacal a 45, while Stalin gets a 20, Ragnar a 10, and Mao gets a 0. The 26 AIs not listed will get middle values between 40 and 20.

The high/low tendencies are intended to be loosely historical and to generally correlate to what the leader does well at in game. There's also by design a good spread so that different leaders of the same civ mostly have different tendencies and that all tendencies are well represented. Obviously for Space I've had to take some historical liberties, so it's mainly AIs with more science/production inclinations.

Culture

Hatshepsut
Cyrus
Louis XIV
Suleiman
Augustus
Gilgamesh
Kublai
Ghandi
Zara
Hammurabi
Pacal
Charlemagne
Mansa Musa
Washington

xxxxxxxxxxxxxxxxxx

Stalin
Bismark
Asoka
De Gaulle
Lincoln
Hannibal
Saladin
Ragnar
Peter
Joao
Genghis
Brennus
Mao


Space

Stalin
Frederick
Asoka
Darius
Hannibal
Wang Kon
Peter
Mao
Joao
Willem
Churchill
Pericles
FDR
Tokugawa

xxxxxxxxxxxxxxxxxx

Shaka
Ramesses
Mehmed
Charlemagne
Qin Shi
Elizabeth
Suleiman
Washington
Boudica
Isabella
Kublai
Montezuma
Cyrus


Conquest

Montezuma
Genghis
Shaka
Ramesses
Catherine
Boudica
Julius
Huyana
Ragnar
Mao
Elizabeth
Gilgamesh
Washington

xxxxxxxxxxxxxxxxxx

Joao
Zara
Charlemagne
Pericles
Louis XIV
Wang Kon
Suryavaram
Justinian
Frederick
FDR
Pacal
Victoria
Churchill
Ghandi


Domination

Victoria
Alexander
Napolean
Mehmed
Saladin
Bismark
Augustus
Kublai
Joao
Charlemagne
Suleiman
Tokugawa
Stalin

xxxxxxxxxxxxxxxxxx

Wang Kon
Zara
Loius XIV
Lincoln
Qin Shi
Elizabeth
De Gaulle
Sitting Bull
Darius
Hatshepsut
Ghandi
Hammurabi
Willem
Mansa Musa


Diplomacy

Ghandi
Sitting Bull
Isabella
Elizabeth
Brennus
De Gaulle
Suryavaram
FDR
Justinian
Mansa Musa
Qin Shi
Lincoln
Louis XIV

xxxxxxxxxxxxxxxxxx

Asoka
Ramesses
Shaka
Huyana
Mao
Julius
Victoria
Stalin
Catherine
Napolean
Augustus
Gilgamesh
Alexander
Tokugawa
 
I contemplated just basing them off of trait combos as Blake did for his version of the Cultural victory strategy, but there are a bunch of other possible factors like how "war like" they are or whether they're usually tops in tech, etc. So I decided new numbers for this new behavior would be a better, simpler system.
Please contemplate a bit more then. Being "war like" or tops in tech is merely a result of personality and traits, and I really think those numbers, even if you specify them individually for each leader, are mere derivates of traits & personality combinations and therefore should need to be treated as such, which means calculating them instead of changing an xml schema just for this.
A civilizations starting techs (from civ and difficulty) could be a factor too (ie starting with mysticism should add to diplo vic strat).
The only other constant factor I can think of would be UBs and UUs but in favor of not changing xml schemas I'd vote for ignoring those.

Unfortunately a personality is not just 2 items like the traits, but a couple of dozen that all can be factors, albeit small in most cases. I'd assume the more important ones would be flavor values, wonder / unit building probs and war starting threshold/rands/ratios.
 
I agree with Fuyu, the AI personality is the result of the interaction of a handful of XML paramethers with the game situation and trying to reduce that to a simpler system could bork some LH badly in some situations.

Say, for sake example, Asoka. Asoka is the archetypical Dr Jekill/Mr Hyde in terms of Civ IV LH, normally he is quite a nice guy because he builds not too much of units ... but in the rare ocasions he gets a decent power lead he pretty much becomes worse than Shaka or Monty in terms of agressiveness and he actually is quite good at warfare. How to model this situation ?

Or, let's say, Sitting Duc...er, Bull. He is not very warlike, but he builds a lot of units. That normally does not bode well for him ( boatloads of unused units ), but he also has a quite decent "courage" ( he will consider attacking someone with 135% of his power ) and does not care if the target is distant or not. That means he can either become a placid ruminant for the rest of the game or snap and start invading foreign and close lands with hordes of units if he manages to keep updated in tech and vassal people from left to right...

So i guess than using the paramethers that Fuyu stated would actually be more useful than a simple "Asoka is usually a peacenik, so let's make him to think it is highly unlikely for him to follow conquest strats", because that would hinder the Asoka AI when it actually starts being agressive.
 
one little thing (I guess it's a copy&paste glitch):

Code:
<!-- Better BTS AI: Allow AI to use Diplomacy victory strategy logic if space race victory is enabled

"if space race victory is enabled" is everywhere on BBAI_Game_Options_GlobalDefines.xml.
 
Surely using the parameters is itself subjective, because you then have to choose weights?

Exactly, picking which AI variables to base the computation off of and which ones have what importance is just as arbitrary and much more complex. Having new variables in much more transparent, it's immediately clear which leaders favor what and that the distribution of numbers is well formed.

For culture and conquest, putting together a reasonable set of traits, favorite civics, and other AI variables is straight forward. But what about space or diplomacy? Not so clear.

Fuyu:

The fact that a couple dozen AI factors would need to be included is why I think new variables are the way to go. It is so much simpler and easier to be sure the AIs are doing what is intended.

r_rolo:

The Asoka situation you mention is one of the reasons for the two tiered victory strategy approach. If we give Asoka low odds for conquest, then he won't often start with Conquest1 at the beginning of the game. But, if over the course of the game he does happen to open up a power lead and start attacking people, then he will start using Conquest3. The 3rd and 4th levels of the strategies are based just on in game actions and things like relative power, and an AI can adopt them without having started the game with the 1st level strategy.

Here's how the conquest strategy is currently set up:
Spoiler :


Conquest1
Trigger: Tendency + rand >= 10 OR running Conquest3 or Conquest4

Effect: Slight increase in willingness to spend gold on unit maintenance, slight inclination toward techs giving attack city military units

Conquest2
Trigger: Tendency + rand >= 10, four cities, top half of power graph OR running Conquest3 or Conquest4

Effect: A bit more likely to start wars

Conquest3
Trigger: Top 15% of powers, power is > 133% of average, has declared war at least three times, in medieval era or later (note: no dependence on tendency) OR running Conquest4

Effect: Similar to running Aggressive AI - willing to spend more on troops, favors military techs, more likely to consider wars with neighbors at Cautious and Pleased

Conquest4
Trigger: Top power, power is 200% of average, NOT running Space4 (final stages of space victory)

Effect: Small chance of considering war with friends if no other targets, high unit spending


The effect of Conquest1 is fairly subtle, but once Conquest3 is adopted the AI is very aggressive. While starting Conquest1 at the beginning of the game makes it more likely the AI will qualify for 3 later on, if other game circumstances make the AI take a more militaristic route then the AI may adopt Conquest3 without ever having run Conquest1 before.

(Some of these triggers don't yet scale well for small/large maps, I am working on that.)

So the system is setup handle this situation well I believe. Does that make sense?

Cybah:

Thanks for catching that.
 
I know this is a big request, but any chance of adding Religion victory in their too, since it was in BTS but disabled?

Anyway, I think this is a great idea. Some unified victory strategy would make the AI significantly better.
 
For culture and conquest, putting together a reasonable set of traits, favorite civics, and other AI variables is straight forward. But what about space or diplomacy? Not so clear.

Space race is usually only an option when conquest or domination becomes impossible, either due to huge maps with far flung continents or multiple teams have lots of nukes (MAD situation). So I don't think the AI should worry about the space race unless other people get nukes, diplomacy is out of the question, or the late game is draw. Basically, Space Race is the tie breaker victory condition, in my mind.

Diplomacy is usually a plan B for those heading to cultural victories. If, at 500AD, you lose your capital, your chance of a cultural victory approaches 0. But you might still be on good terms with everyone and not too far behind.

In both cases, those victory considerations should be secondary considerations, in the event that something unfortunate happens to plan A.
 
Can you not see how much more flexible it would be to create one formula than to define values for every single leader seperately?
Exactly, picking which AI variables to base the computation off of and which ones have what importance is just as arbitrary and much more complex. Having new variables in much more transparent, it's immediately clear which leaders favor what and that the distribution of numbers is well formed.

For culture and conquest, putting together a reasonable set of traits, favorite civics, and other AI variables is straight forward. But what about space or diplomacy? Not so clear.

Fuyu:

The fact that a couple dozen AI factors would need to be included is why I think new variables are the way to go. It is so much simpler and easier to be sure the AIs are doing what is intended.
Allow me to still disagree, what you do should not define the personity of a leader but merely reflect it by making him go for VCs that fit him best with higher propability, it's a slight change of behaviour, not personality, and thus belongs only to the SDK.
What you propose might work for the known leaders where you actually bother to define the values. I see 2 general cases where this will simply not be enough: random personalities, and when there are leaders in the game that were not in stock BTS.
Defining numbers looks more transparent, no doubt there. And I admit it's not equally clear and straight-forward for all victory conditions, but still you should try. If you say it's straight-forward for conquest and culture, I say it's not that bad for domination and space either: dom is similar to conquest, but it also profits from traits that decrease upkeep or increase income like FLAVOR_GOLD, space would benefit from economy bonuses like the financial and org trait and FLAVOR_SCIENCE and FLAVOR_GOLD, FLAVOR_RELIGION would add to diplo1 prob etc. and at the end of the day all values get normalised so total average chance is still 40% for a "well-formed distribution".

Finetuning can follow later, like how much each factor weights or if there are combinations that should count in a different way than what the sum of its parts because of synergies.
 
in the event that something unfortunate happens to plan A.

"Sir, Plan A was just eaten by a herd of predatory llamas." -- Commonly Overheard Report

Dood, he was looking for values between 0 and 8, not Low Medium High. I know this, cuz... it said so in his post. :p Seriously. Stop succumbing to llama mind-control.

p.s. -- The new patch/ladder for Diablo 2 happen tomorrow... which means I am probably disappearing forever at that point (since this will obviously carry a person up to the launch of Diablo 3, and it's a straight line from there to forever). I may also be overstating the matter slightly, we shall skiburd...
 
What you propose might work for the known leaders where you actually bother to define the values. I see 2 general cases where this will simply not be enough: random personalities, and when there are leaders in the game that were not in stock BTS.

Random personalities actually just means that the leader named Genghis Khan is actually say Asoka underneath. I was surprised when I learned that. The underlying AI personality for Genghis is not really randomized, it's just drawn randomly from the set of defined leaders.

People defining new leaders for the game would have to provide these 5 new values or their XML won't load properly. This is the one downside to the new variables approach.

Can you not see how much more flexible it would be to create one formula than to define values for every single leader seperately?

??? The single formula approach is inflexible, you then would never have a Creative/Philosophical leader who also likes to go for Conquest for example. New values can be assigned to anything, it's completely flexible.

I guess perhaps the reason we're disagreeing is that I intend the first level victory strategies to be personality-based, while the formula approach you propose is more utility-based (ie, the creative trait boosts culture output so going for culture victory is easier).

Say we design a formula that means that AIs with high probabilities to build units and high odds to declare war are more likely to adopt Conquest1. All we have done is create positive feedback: these AIs will now build even more units and more likely declare war. AIs with these settings are already more likely to be qualify for Conquest3 in the medieval era. The upper level strategies are inherently utility-based.

Now I certainly believe that personality and the perks each leader gets should be correlated. Absolutely. But with 53 leaders we can set up the variables so there's variety.

Firaxis has already taken this approach. For example, traits give leaders boosts for certain things but an AI with the Aggressive trait is not actually more aggressive. The variables which control how likely the AI is to declare war or attack with low odds are completely different. While most of the 10 AIs with Aggressive have MaxWarRands of 50 (high war likelihood), there's also Hammurabi with a MaxWarRand of 200 (middle of the road). That kind of variety is great and this is what picking new variables gets us: the ability to create correlation with some variance.

The other main benefit which I see as a big benefit of the variable system is that with variables we can immediately see what each leader likes and would not have any issues determining whether some leaders like too many victory conditions or too few. It self tests and there's no ambiguity.
 
People defining new leaders for the game would have to provide these 5 new values or their XML won't load properly. This is the one downside to the new variables approach.
And this is the main reason I'm against it. Not the only one though.

??? The single formula approach is inflexible, you then would never have a Creative/Philosophical leader who also likes to go for Conquest for example. New values can be assigned to anything, it's completely flexible.

I guess perhaps the reason we're disagreeing is that I intend the first level victory strategies to be personality-based, while the formula approach you propose is more utility-based (ie, the creative trait boosts culture output so going for culture victory is easier).
Why would you never have a Creative/Philosophical leader who also likes to go for Conquest? That's just his traits, the "inflexible" formula would look at his personality variables too to determine the odds. But true, even with a very military and war-prone personality, the odds for conquest would never be as high as for tokugawa - and honestly I see nothing wrong with that. I'd certainly want the AI to consider leader traits, instead of ignoring them completely. So yes, my approach is more utility-based but I'd factor in personality too, with higher weights even if necessary.

Random personalities actually just means that the leader named Genghis Khan is actually say Asoka underneath. I was surprised when I learned that. The underlying AI personality for Genghis is not really randomized, it's just drawn randomly from the set of defined leaders.
I learn something new every day. And still I see a problem with random personalities: VC strat odds are carved in stone/xml and get paired with traits/civs where they might make a lot less sense. And if you say traits should play no part in the VC decision then we simply disagree there.
Now I certainly believe that personality and the perks each leader gets should be correlated. Absolutely.
.. Ok we don't :p
But with 53 leaders we can set up the variables so there's variety.

Firaxis has already taken this approach. For example, traits give leaders boosts for certain things but an AI with the Aggressive trait is not actually more aggressive. The variables which control how likely the AI is to declare war or attack with low odds are completely different. While most of the 10 AIs with Aggressive have MaxWarRands of 50 (high war likelihood), there's also Hammurabi with a MaxWarRand of 200 (middle of the road). That kind of variety is great and this is what picking new variables gets us: the ability to create correlation with some variance.
Yes, this is now finally the point where we really disagree. You want to add another layer for AI personalities where I think we already have enough personality, and that the AI should focus more on trying to win than on roleplaying.
Variety could be achieved simply by being not simply dependant on the utility side like traits but also on personalities, so the result should not lack that much variety compared to your approach (assuming you are not using values that don't fit the leader at all) and there's the aspect of those numbers being mere odds of adopting the strat, it does not mean a leader has to, that would be different for every game which would already be variety enough in my eyes even with simple 40% standard chances everywhere.


The other main benefit which I see as a big benefit of the variable system is that with variables we can immediately see what each leader likes and would not have any issues determining whether some leaders like too many victory conditions or too few. It self tests and there's no ambiguity.
Yes, the transparency, and I'm still not buying it. It still looks just like the simple&easy way out, at the very high cost of changing not only an xml file but a whole schema.
A formula can self test just as well, if not better, and can be written to not make leaders like too many of too few strats (total sum must always be 20? Should be the simplest part of the formula).

:whew:
 
People defining new leaders for the game would have to provide these 5 new values or their XML won't load properly. This is the one downside to the new variables approach.
And this is the main reason I'm against it. Not the only one though.

Really? Then your concern is completely unwarranted. Adding new XML values is a piece of cake.

Why would you never have a Creative/Philosophical leader who also likes to go for Conquest? That's just his traits, the "inflexible" formula would look at his personality variables too to determine the odds. But true, even with a very military and war-prone personality, the odds for conquest would never be as high as for tokugawa - and honestly I see nothing wrong with that. I'd certainly want the AI to consider leader traits, instead of ignoring them completely. So yes, my approach is more utility-based but I'd factor in personality too, with higher weights even if necessary.

Jdog never said that these XML values would control the entire games victory plans. Read his post!
- The first two levels of each strategy (ie, Conquest1 and Conquest2) are for somewhat randomized tendencies for the AI leader. Since this is new, there are going to be new variables in XML to control how likely an AI is to try the early levels of these strategies (and you can help pick them!).

- The upper two levels (ie, Conquest3 and Conquest4) measure when a player is closing in on a particular victory condition. They do not require the AI to have run a lower level strategy, but the lower levels are designed to put an AI in a position to adopt the higher level one. These levels also have no random component and no AI personality component, but instead are based on in past game actions and measurables like power or culture. An AI will only run one level 4 strategy at a time.

Basically, the values will only control their initial disposition, not the ultimate choice! This isn't about adding another layer of personality, it's about setting values that will lead to the most optimal early game. A random formula might decide Monty needs to go cultural this time, when his abilities are much better suited towards conquest. The way Jdog has is planned makes sense, Monty will probably go for conquest first, but if he can't do that, he'll come up with a plan B, not dependent on the XML values.

There's no need to act like the sky is falling guys, this is a great idea; and I'm behind Jdog 100%.
 
Victoria having a certain likeness towards Domination, Stalin having a certain likeness towards Space etc. sounds like fun, providing, of course, that it will have a meaningful effect while not completely hamstringing their playstyles.

What happens when the values contradict each other, though, like a 10 unitprob 500 warrand 90/90/90/100/100 warattitude leader who is told to be geared for Conquest/Domination?
 
jodg while you do this please include a "Turtle" stratagy for AIs that are low on military and are attacked. Currently they just flounder around, and send off attack stacks to die in the agressor's territory while doing no damage. They really need to just mass troops, specifically collateral damage units, and defend inside their own territory; and they should drop all other tasks when this occurs. This is particularly important for AIs that are going for cultural victory and are then backstabbed.
 
Maybe I didn't phrase a few things as well as I thought I did ..

@Afforess: I imagine how much hassle adding 5 more xml values actually is will be directly proportional to the number of additional leaders you have in your BBAI-based mods :p
Doable of course and not really tragic but not necessary either.

No sky is falling, and all I ever disagreed about is how to determine VictoryWeight values that "make sense"."This isn't about adding another layer of personality, it's about setting values that will lead to the most optimal early game." <- We all agree there: jdog wants to do it by hand, I find it better to calculate them. For all BTS leaders the result should be really similar. For new leaders the values still need to be specified, for games with the random personality option (where the personality of one leader gets paired up with the traits and civ of a different leader), things get different because the values defined by hand would stay unchanged while the calculated ones would change a bit according to the new traits.
 
Top Bottom