1. We have added a Gift Upgrades feature that allows you to gift an account upgrade to another member, just in time for the holiday season. You can see the gift option when going to the Account Upgrades screen, or on any user profile screen.
    Dismiss Notice

New AI playing to win system

Discussion in 'Civ4 - Better AI' started by jdog5000, Mar 22, 2010.

  1. jdog5000

    jdog5000 Revolutionary

    Joined:
    Nov 25, 2003
    Messages:
    2,601
    Location:
    California
    (**Mild spoilers**: This thread describes a new system for having the AI play to win the game and which AI personalities will be more likely to try for certain victory types. Any detailed info I post about specific triggers or strategies the AI uses will be wrapped in spoiler tags)


    The new playing to win system is rounding into form and will be a part of releases 0.90 and higher. With this system, the AI will (finally) have two new fundamental abilities:

    1) A strategy to aim for one or more of the available victory types.

    2) An awareness of whether the other players around it are close to winning in a particular way.

    There have been patchwork pieces of this before, but now it will be much more thorough and really influence the actions the AI takes from the mid-game on. Of course there will be XML options to turn this off, but in my mind it's one of the biggest possible steps forward for the AI in terms of making the game more exciting and strategic.

    The AI will have strategies to go for Conquest, Domination, Culture, Space, and Diplomatic victories. Each of these victory strategies is modeled off of how Blake set up the Cultural victory strategy in the Warlords Better AI project (which was eventually folded in to BTS). For each victory strategy the AI will have 4 levels measuring how committed they are to the strategy.

    - The first two levels of each strategy (ie, Conquest1 and Conquest2) are for somewhat randomized tendencies for the AI leader. Since this is new, there are going to be new variables in XML to control how likely an AI is to try the early levels of these strategies (and you can help pick them!).

    - The upper two levels (ie, Conquest3 and Conquest4) measure when a player is closing in on a particular victory condition. They do not require the AI to have run a lower level strategy, but the lower levels are designed to put an AI in a position to adopt the higher level one. These levels also have no random component and no AI personality component, but instead are based on in past game actions and measurables like power or culture. An AI will only run one level 4 strategy at a time.


    Here's how I'm planning to have the randomized tendencies work for the first level of each victory strategy:

    iValue = Leader personality value for victory type from 0 to 100

    iValue += Random dice roll from 0 to 99

    if( iValue >= 100 )
    . AI starts level 1 for this victory strategy

    So, if the AI tendency value is 0 then it will never adopt it, if it's 100 then it will in 100% of games. The random dice roll is done in a very particular way. A random number between 1000 and 10,000 is drawn once for each player per game and saved. Then the five dice rolls are set up so that the AI will have a spectrum of random values:

    Culture dice roll = (strategy rand + 10) % 100

    Conquest dice roll = (strategy rand + 30) % 100

    Space dice roll = (strategy rand + 50) % 100

    Domination dice roll = (strategy rand + 70) % 100

    Diplomacy dice roll = (strategy rand + 90) % 100

    where % is the modulus, ie the remainder after division by in this case 100. So, an AI might get 90 from the random roll for domination, 10 for diplomacy, 30 for culture, 50 for conquest, and 70 for space, but whatever the specifics it will always have one strategy with a random 80+, one with <20, and the rest in between. The reasoning behind this is that each AI should start at least one of these first level victory strategies from the beginning of the game, and maybe two strategies.

    There are a couple adjustments for special game options, like Aggressive AI means AIs are more likely to adopt militaristic strategies (+30), while always peace rules out militaristic ones and boosts culture and space (+40).


    Picking values for a leader

    So, for each leader we need to pick 5 numbers to set their natural affinity for each victory type. Here's how the distribution of tendencies should work:

    - 0 is an extreme value, meaning the AI will never do a strategy in a normal game. These should be very rare.
    - 30 is a good middle for the distribution and represents an average affinity for the victory type.
    - 70 is also an extreme value for a normal game, they should be very rare.
    - Each leaders 5 values should add up to 150, so that on average they start a game adopting 1.5 victory strategies at level 1.

    As for what makes sense for a particular AI, Civ4 been around long enough for the community to know which leaders are good/decent/bad for different game styles. Having traits which play into a particular strategy certainly helps as do other AI characteristics like their odds for war and building units. Historical actions by the leaders may also be relevant.
     
  2. jdog5000

    jdog5000 Revolutionary

    Joined:
    Nov 25, 2003
    Messages:
    2,601
    Location:
    California
    This post sorts AIs by their propensity for different victories from High to Low. Only ~13 AIs will be listed for high (weight 70 down to 40), and ~13 for low (20 down to 0). The top/bottom three get 60 and up, so for culture Hatshepsut gets an 70, Suleiman 55, Pacal a 45, while Stalin gets a 20, Ragnar a 10, and Mao gets a 0. The 26 AIs not listed will get middle values between 40 and 20.

    The high/low tendencies are intended to be loosely historical and to generally correlate to what the leader does well at in game. There's also by design a good spread so that different leaders of the same civ mostly have different tendencies and that all tendencies are well represented. Obviously for Space I've had to take some historical liberties, so it's mainly AIs with more science/production inclinations.

    Culture

    Hatshepsut
    Cyrus
    Louis XIV
    Suleiman
    Augustus
    Gilgamesh
    Kublai
    Ghandi
    Zara
    Hammurabi
    Pacal
    Charlemagne
    Mansa Musa
    Washington

    xxxxxxxxxxxxxxxxxx

    Stalin
    Bismark
    Asoka
    De Gaulle
    Lincoln
    Hannibal
    Saladin
    Ragnar
    Peter
    Joao
    Genghis
    Brennus
    Mao


    Space

    Stalin
    Frederick
    Asoka
    Darius
    Hannibal
    Wang Kon
    Peter
    Mao
    Joao
    Willem
    Churchill
    Pericles
    FDR
    Tokugawa

    xxxxxxxxxxxxxxxxxx

    Shaka
    Ramesses
    Mehmed
    Charlemagne
    Qin Shi
    Elizabeth
    Suleiman
    Washington
    Boudica
    Isabella
    Kublai
    Montezuma
    Cyrus


    Conquest

    Montezuma
    Genghis
    Shaka
    Ramesses
    Catherine
    Boudica
    Julius
    Huyana
    Ragnar
    Mao
    Elizabeth
    Gilgamesh
    Washington

    xxxxxxxxxxxxxxxxxx

    Joao
    Zara
    Charlemagne
    Pericles
    Louis XIV
    Wang Kon
    Suryavaram
    Justinian
    Frederick
    FDR
    Pacal
    Victoria
    Churchill
    Ghandi


    Domination

    Victoria
    Alexander
    Napolean
    Mehmed
    Saladin
    Bismark
    Augustus
    Kublai
    Joao
    Charlemagne
    Suleiman
    Tokugawa
    Stalin

    xxxxxxxxxxxxxxxxxx

    Wang Kon
    Zara
    Loius XIV
    Lincoln
    Qin Shi
    Elizabeth
    De Gaulle
    Sitting Bull
    Darius
    Hatshepsut
    Ghandi
    Hammurabi
    Willem
    Mansa Musa


    Diplomacy

    Ghandi
    Sitting Bull
    Isabella
    Elizabeth
    Brennus
    De Gaulle
    Suryavaram
    FDR
    Justinian
    Mansa Musa
    Qin Shi
    Lincoln
    Louis XIV

    xxxxxxxxxxxxxxxxxx

    Asoka
    Ramesses
    Shaka
    Huyana
    Mao
    Julius
    Victoria
    Stalin
    Catherine
    Napolean
    Augustus
    Gilgamesh
    Alexander
    Tokugawa
     
  3. jdog5000

    jdog5000 Revolutionary

    Joined:
    Nov 25, 2003
    Messages:
    2,601
    Location:
    California
  4. Fuyu

    Fuyu Emperor

    Joined:
    Nov 5, 2009
    Messages:
    1,225
    Location:
    Austria
    Please contemplate a bit more then. Being "war like" or tops in tech is merely a result of personality and traits, and I really think those numbers, even if you specify them individually for each leader, are mere derivates of traits & personality combinations and therefore should need to be treated as such, which means calculating them instead of changing an xml schema just for this.
    A civilizations starting techs (from civ and difficulty) could be a factor too (ie starting with mysticism should add to diplo vic strat).
    The only other constant factor I can think of would be UBs and UUs but in favor of not changing xml schemas I'd vote for ignoring those.

    Unfortunately a personality is not just 2 items like the traits, but a couple of dozen that all can be factors, albeit small in most cases. I'd assume the more important ones would be flavor values, wonder / unit building probs and war starting threshold/rands/ratios.
     
  5. NotSoGood

    NotSoGood Emperor

    Joined:
    Jan 25, 2009
    Messages:
    1,077
    Location:
    Finland
    Sounds ok for me. Looking forward how this turns out.
    Finaly I can get beaten up by the AI with settler difficulty, yay! :D
     
  6. r_rolo1

    r_rolo1 King of myself

    Joined:
    May 19, 2006
    Messages:
    13,818
    Location:
    Lisbon, Portugal
    I agree with Fuyu, the AI personality is the result of the interaction of a handful of XML paramethers with the game situation and trying to reduce that to a simpler system could bork some LH badly in some situations.

    Say, for sake example, Asoka. Asoka is the archetypical Dr Jekill/Mr Hyde in terms of Civ IV LH, normally he is quite a nice guy because he builds not too much of units ... but in the rare ocasions he gets a decent power lead he pretty much becomes worse than Shaka or Monty in terms of agressiveness and he actually is quite good at warfare. How to model this situation ?

    Or, let's say, Sitting Duc...er, Bull. He is not very warlike, but he builds a lot of units. That normally does not bode well for him ( boatloads of unused units ), but he also has a quite decent "courage" ( he will consider attacking someone with 135% of his power ) and does not care if the target is distant or not. That means he can either become a placid ruminant for the rest of the game or snap and start invading foreign and close lands with hordes of units if he manages to keep updated in tech and vassal people from left to right...

    So i guess than using the paramethers that Fuyu stated would actually be more useful than a simple "Asoka is usually a peacenik, so let's make him to think it is highly unlikely for him to follow conquest strats", because that would hinder the Asoka AI when it actually starts being agressive.
     
  7. Cybah

    Cybah Emperor

    Joined:
    Jun 22, 2007
    Messages:
    1,480
    one little thing (I guess it's a copy&paste glitch):

    Code:
    <!-- Better BTS AI: Allow AI to use Diplomacy victory strategy logic if space race victory is enabled
    "if space race victory is enabled" is everywhere on BBAI_Game_Options_GlobalDefines.xml.
     
  8. Panopticon

    Panopticon Utilitarian

    Joined:
    Aug 4, 2007
    Messages:
    1,450
    Location:
    Ireland
    Surely using the parameters is itself subjective, because you then have to choose weights?
     
  9. jdog5000

    jdog5000 Revolutionary

    Joined:
    Nov 25, 2003
    Messages:
    2,601
    Location:
    California
    Exactly, picking which AI variables to base the computation off of and which ones have what importance is just as arbitrary and much more complex. Having new variables in much more transparent, it's immediately clear which leaders favor what and that the distribution of numbers is well formed.

    For culture and conquest, putting together a reasonable set of traits, favorite civics, and other AI variables is straight forward. But what about space or diplomacy? Not so clear.

    Fuyu:

    The fact that a couple dozen AI factors would need to be included is why I think new variables are the way to go. It is so much simpler and easier to be sure the AIs are doing what is intended.

    r_rolo:

    The Asoka situation you mention is one of the reasons for the two tiered victory strategy approach. If we give Asoka low odds for conquest, then he won't often start with Conquest1 at the beginning of the game. But, if over the course of the game he does happen to open up a power lead and start attacking people, then he will start using Conquest3. The 3rd and 4th levels of the strategies are based just on in game actions and things like relative power, and an AI can adopt them without having started the game with the 1st level strategy.

    Here's how the conquest strategy is currently set up:
    Spoiler :


    Conquest1
    Trigger: Tendency + rand >= 10 OR running Conquest3 or Conquest4

    Effect: Slight increase in willingness to spend gold on unit maintenance, slight inclination toward techs giving attack city military units

    Conquest2
    Trigger: Tendency + rand >= 10, four cities, top half of power graph OR running Conquest3 or Conquest4

    Effect: A bit more likely to start wars

    Conquest3
    Trigger: Top 15% of powers, power is > 133% of average, has declared war at least three times, in medieval era or later (note: no dependence on tendency) OR running Conquest4

    Effect: Similar to running Aggressive AI - willing to spend more on troops, favors military techs, more likely to consider wars with neighbors at Cautious and Pleased

    Conquest4
    Trigger: Top power, power is 200% of average, NOT running Space4 (final stages of space victory)

    Effect: Small chance of considering war with friends if no other targets, high unit spending


    The effect of Conquest1 is fairly subtle, but once Conquest3 is adopted the AI is very aggressive. While starting Conquest1 at the beginning of the game makes it more likely the AI will qualify for 3 later on, if other game circumstances make the AI take a more militaristic route then the AI may adopt Conquest3 without ever having run Conquest1 before.

    (Some of these triggers don't yet scale well for small/large maps, I am working on that.)

    So the system is setup handle this situation well I believe. Does that make sense?

    Cybah:

    Thanks for catching that.
     
  10. Afforess

    Afforess The White Wizard

    Joined:
    Jul 31, 2007
    Messages:
    12,239
    Location:
    Austin, Texas
    I know this is a big request, but any chance of adding Religion victory in their too, since it was in BTS but disabled?

    Anyway, I think this is a great idea. Some unified victory strategy would make the AI significantly better.
     
  11. Afforess

    Afforess The White Wizard

    Joined:
    Jul 31, 2007
    Messages:
    12,239
    Location:
    Austin, Texas
    Space race is usually only an option when conquest or domination becomes impossible, either due to huge maps with far flung continents or multiple teams have lots of nukes (MAD situation). So I don't think the AI should worry about the space race unless other people get nukes, diplomacy is out of the question, or the late game is draw. Basically, Space Race is the tie breaker victory condition, in my mind.

    Diplomacy is usually a plan B for those heading to cultural victories. If, at 500AD, you lose your capital, your chance of a cultural victory approaches 0. But you might still be on good terms with everyone and not too far behind.

    In both cases, those victory considerations should be secondary considerations, in the event that something unfortunate happens to plan A.
     
  12. Fuyu

    Fuyu Emperor

    Joined:
    Nov 5, 2009
    Messages:
    1,225
    Location:
    Austria
    Can you not see how much more flexible it would be to create one formula than to define values for every single leader seperately?
    Allow me to still disagree, what you do should not define the personity of a leader but merely reflect it by making him go for VCs that fit him best with higher propability, it's a slight change of behaviour, not personality, and thus belongs only to the SDK.
    What you propose might work for the known leaders where you actually bother to define the values. I see 2 general cases where this will simply not be enough: random personalities, and when there are leaders in the game that were not in stock BTS.
    Defining numbers looks more transparent, no doubt there. And I admit it's not equally clear and straight-forward for all victory conditions, but still you should try. If you say it's straight-forward for conquest and culture, I say it's not that bad for domination and space either: dom is similar to conquest, but it also profits from traits that decrease upkeep or increase income like FLAVOR_GOLD, space would benefit from economy bonuses like the financial and org trait and FLAVOR_SCIENCE and FLAVOR_GOLD, FLAVOR_RELIGION would add to diplo1 prob etc. and at the end of the day all values get normalised so total average chance is still 40% for a "well-formed distribution".

    Finetuning can follow later, like how much each factor weights or if there are combinations that should count in a different way than what the sum of its parts because of synergies.
     
  13. LunarMongoose

    LunarMongoose King

    Joined:
    Jan 29, 2006
    Messages:
    731
    Gender:
    Male
    Location:
    Boston, MA, USA
    "Sir, Plan A was just eaten by a herd of predatory llamas." -- Commonly Overheard Report

    Dood, he was looking for values between 0 and 8, not Low Medium High. I know this, cuz... it said so in his post. :p Seriously. Stop succumbing to llama mind-control.

    p.s. -- The new patch/ladder for Diablo 2 happen tomorrow... which means I am probably disappearing forever at that point (since this will obviously carry a person up to the launch of Diablo 3, and it's a straight line from there to forever). I may also be overstating the matter slightly, we shall skiburd...
     
  14. jdog5000

    jdog5000 Revolutionary

    Joined:
    Nov 25, 2003
    Messages:
    2,601
    Location:
    California
    Random personalities actually just means that the leader named Genghis Khan is actually say Asoka underneath. I was surprised when I learned that. The underlying AI personality for Genghis is not really randomized, it's just drawn randomly from the set of defined leaders.

    People defining new leaders for the game would have to provide these 5 new values or their XML won't load properly. This is the one downside to the new variables approach.

    ??? The single formula approach is inflexible, you then would never have a Creative/Philosophical leader who also likes to go for Conquest for example. New values can be assigned to anything, it's completely flexible.

    I guess perhaps the reason we're disagreeing is that I intend the first level victory strategies to be personality-based, while the formula approach you propose is more utility-based (ie, the creative trait boosts culture output so going for culture victory is easier).

    Say we design a formula that means that AIs with high probabilities to build units and high odds to declare war are more likely to adopt Conquest1. All we have done is create positive feedback: these AIs will now build even more units and more likely declare war. AIs with these settings are already more likely to be qualify for Conquest3 in the medieval era. The upper level strategies are inherently utility-based.

    Now I certainly believe that personality and the perks each leader gets should be correlated. Absolutely. But with 53 leaders we can set up the variables so there's variety.

    Firaxis has already taken this approach. For example, traits give leaders boosts for certain things but an AI with the Aggressive trait is not actually more aggressive. The variables which control how likely the AI is to declare war or attack with low odds are completely different. While most of the 10 AIs with Aggressive have MaxWarRands of 50 (high war likelihood), there's also Hammurabi with a MaxWarRand of 200 (middle of the road). That kind of variety is great and this is what picking new variables gets us: the ability to create correlation with some variance.

    The other main benefit which I see as a big benefit of the variable system is that with variables we can immediately see what each leader likes and would not have any issues determining whether some leaders like too many victory conditions or too few. It self tests and there's no ambiguity.
     
  15. jdog5000

    jdog5000 Revolutionary

    Joined:
    Nov 25, 2003
    Messages:
    2,601
    Location:
    California
    The second post has been updated with the intended spreads of weights over the various AIs for each victory strategy type. The highest and lowest 1/4 of each distribution are shown.

    Feedback is certainly welcome.
     
  16. Fuyu

    Fuyu Emperor

    Joined:
    Nov 5, 2009
    Messages:
    1,225
    Location:
    Austria
    And this is the main reason I'm against it. Not the only one though.

    Why would you never have a Creative/Philosophical leader who also likes to go for Conquest? That's just his traits, the "inflexible" formula would look at his personality variables too to determine the odds. But true, even with a very military and war-prone personality, the odds for conquest would never be as high as for tokugawa - and honestly I see nothing wrong with that. I'd certainly want the AI to consider leader traits, instead of ignoring them completely. So yes, my approach is more utility-based but I'd factor in personality too, with higher weights even if necessary.

    I learn something new every day. And still I see a problem with random personalities: VC strat odds are carved in stone/xml and get paired with traits/civs where they might make a lot less sense. And if you say traits should play no part in the VC decision then we simply disagree there.
    .. Ok we don't :p
    Yes, this is now finally the point where we really disagree. You want to add another layer for AI personalities where I think we already have enough personality, and that the AI should focus more on trying to win than on roleplaying.
    Variety could be achieved simply by being not simply dependant on the utility side like traits but also on personalities, so the result should not lack that much variety compared to your approach (assuming you are not using values that don't fit the leader at all) and there's the aspect of those numbers being mere odds of adopting the strat, it does not mean a leader has to, that would be different for every game which would already be variety enough in my eyes even with simple 40% standard chances everywhere.


    Yes, the transparency, and I'm still not buying it. It still looks just like the simple&easy way out, at the very high cost of changing not only an xml file but a whole schema.
    A formula can self test just as well, if not better, and can be written to not make leaders like too many of too few strats (total sum must always be 20? Should be the simplest part of the formula).

    :whew:
     
  17. Afforess

    Afforess The White Wizard

    Joined:
    Jul 31, 2007
    Messages:
    12,239
    Location:
    Austin, Texas
    Really? Then your concern is completely unwarranted. Adding new XML values is a piece of cake.

    Jdog never said that these XML values would control the entire games victory plans. Read his post!
    Basically, the values will only control their initial disposition, not the ultimate choice! This isn't about adding another layer of personality, it's about setting values that will lead to the most optimal early game. A random formula might decide Monty needs to go cultural this time, when his abilities are much better suited towards conquest. The way Jdog has is planned makes sense, Monty will probably go for conquest first, but if he can't do that, he'll come up with a plan B, not dependent on the XML values.

    There's no need to act like the sky is falling guys, this is a great idea; and I'm behind Jdog 100%.
     
  18. Lone Wolf

    Lone Wolf Deity

    Joined:
    Dec 4, 2006
    Messages:
    9,891
    Victoria having a certain likeness towards Domination, Stalin having a certain likeness towards Space etc. sounds like fun, providing, of course, that it will have a meaningful effect while not completely hamstringing their playstyles.

    What happens when the values contradict each other, though, like a 10 unitprob 500 warrand 90/90/90/100/100 warattitude leader who is told to be geared for Conquest/Domination?
     
  19. phungus420

    phungus420 Deity

    Joined:
    Mar 1, 2003
    Messages:
    6,296
    jodg while you do this please include a "Turtle" stratagy for AIs that are low on military and are attacked. Currently they just flounder around, and send off attack stacks to die in the agressor's territory while doing no damage. They really need to just mass troops, specifically collateral damage units, and defend inside their own territory; and they should drop all other tasks when this occurs. This is particularly important for AIs that are going for cultural victory and are then backstabbed.
     
  20. Fuyu

    Fuyu Emperor

    Joined:
    Nov 5, 2009
    Messages:
    1,225
    Location:
    Austria
    Maybe I didn't phrase a few things as well as I thought I did ..

    @Afforess: I imagine how much hassle adding 5 more xml values actually is will be directly proportional to the number of additional leaders you have in your BBAI-based mods :p
    Doable of course and not really tragic but not necessary either.

    No sky is falling, and all I ever disagreed about is how to determine VictoryWeight values that "make sense"."This isn't about adding another layer of personality, it's about setting values that will lead to the most optimal early game." <- We all agree there: jdog wants to do it by hand, I find it better to calculate them. For all BTS leaders the result should be really similar. For new leaders the values still need to be specified, for games with the random personality option (where the personality of one leader gets paired up with the traits and civ of a different leader), things get different because the values defined by hand would stay unchanged while the calculated ones would change a bit according to the new traits.
     

Share This Page