2016 Winter Patch - Combat AI...?

[Siesta Guru]

From what I've been reading since the game was released, the primary root causes of AI incompetency in combat are:

- AI currently overvalues unit self-preservation. e.g. even if they have a strong unit, they are reluctant to bring it near a walled city because god forbid their HP goes down from 100 to 96. If they do decide to bring it, as soon as it takes a hit, it wants to retreat and heal.
- AI units don't behave as a "pack", it's more each-for-his-own kind of mentality. So they don't "sacrifice" or "risk" any units because every life is precious
- AI is indecisive. e.g. one turn a unit gets orders to attack a city, next turn the orders might change to "chill out"

@Siesta Guru explains all this a lot more eloquently in the AI+ mod threads. https://forums.civfanatics.com/resources/ai.25439/

I understand some of this he is able to tweak but most of the behavior is in the SDK that only the devs have access to today

This is indeed mostly what's going on.
The self-preservation point is slightly off though. It doesn't do any individual unit self-preservation tactics like running away as long as it's above a certain threshold. If you see these units run away at high health, it's most likely that on an operational, team level, it decides that the situation is unwinnable. It could also actually not be running away, but could for example be wandering or trying to pillage. They do however sometimes choose not to attack units even if at full health if the combat strength of the target is significantly above their own. This is especially obvious around cities in later eras, as their combat strengths tend to be way above that of units.

The indecisiveness seems to partially be because individual units don't appear to be any memory. If there's several possible things they could be doing, and it can't decide on one of these being significantly better than the others. Then small changes in environmental factors will make them switch targets/types of behavior on every turn, often causing them to bounce back and forth.

I'd also add that the units appear to have no, or very little concept of terrain and distances. It doesn't seem to think about say trying to end a turn on a hill next to a river. And it's 'running away' movement just runs it away from what seems to be a nearby unit, without care for whether its new position will be reachable by enemy units.

In vanilla (fixed this in AI+), there's also some nasty issues where the AI fails to reach a condition and then gets stuck. The worst was requiring all units in its operation to be within 2 tiles of an enemy city/a gathering point. Which was often impossible due to too many units for the amount of tiles available to move to. Which was most commonly seen as units getting stuck on borders/just kind of standing around in your terrain and sometimes as them just staring cities down for hundreds of years.

The AI is especially in later eras also significantly held back by the game design and balance. The high city combat/health and healing in renaissance and up, require endured competent play to even stand a chance of taking a city. The AI is nowhere near the level required for that.
Also, In later eras the production costs of units ramp up so much, that it even takes the deity AI (whose economic bonuses are more like immortal in civ 5) 6-10 turns to build a single unit. On top of that, to make them actually competitive in culture/faith while production is necessary, they're forced to underbuild commercial districts, leaving them without money to sustain and upgrade their troops, disbanding is common even on deity. And even if they have the money to upgrade, they might not have the necessary resources. Leading to small underdeveloped armies after classical.
They're also really bad at using the correct policies to counter these issues, they'll get a military policy to build city walls faster over something to increase military unit production, even if it's building units over walls. And they will get the policy to reduce the cost to purchase tiles in a wildcard slot over the reduced maintenance cost, despite them sitting at 0 gold and -8 gpt.

Not that it even matters that much in later eras, since there appears to be a bug (?) that makes city assault operations be significantly less likely to even be considered.

 

[PhilBowles]

you might be interested to know that scouts can take settlers too

I tried doing exactly that and the game wouldn't let me - in two different sessions. If it's a bug it's an odd one. Scouts always could take settlers in earlier games but I haven't been able to this time.

Or perhaps I should have officially declared war first rather than the typical 'attack and get a declare war prompt'?

 

[juanpavo]

In vanilla (fixed this in AI+), there's also some nasty issues where the AI fails to reach a condition and then gets stuck. The worst was requiring all units in its operation to be within 2 tiles of an enemy city/a gathering point. ...

On top of that, to make them actually competitive in culture/faith while production is necessary, they're forced to underbuild commercial districts, leaving them without money to sustain and upgrade their troops, disbanding is common even on deity. And even if they have the money to upgrade, they might not have the necessary resources. Leading to small underdeveloped armies after classical.
They're also really bad at using the correct policies to counter these issues, they'll get a military policy to build city walls faster over something to increase military unit production, even if it's building units over walls. And they will get the policy to reduce the cost to purchase tiles in a wildcard slot over the reduced maintenance cost, despite them sitting at 0 gold and -8 gpt.

Excellent analysis (or hypothesis?). How are you able to gain such precise insight into AI decision algorithms?

 

[Siesta Guru]

Excellent analysis (or hypothesis?). How are you able to gain such precise insight into AI decision algorithms?

I'm the creator of the AI+ mod, so have spend quite some time looking at what's visible of the AI logic, plus about 200 hours of looking at AI-only games and some of the quite revealing logs.

 

[jollywonk]

It doesn't do any individual unit self-preservation tactics like running away as long as it's above a certain threshold. If you see these units run away at high health, it's most likely that on an operational, team level, it decides that the situation is unwinnable.

The indecisiveness seems to partially be because individual units don't appear to be any memory. If there's several possible things they could be doing, and it can't decide on one of these being significantly better than the others. Then small changes in environmental factors will make them switch targets/types of behavior on every turn, often causing them to bounce back and forth.

I'd also add that the units appear to have no, or very little concept of terrain and distances. It doesn't seem to think about say trying to end a turn on a hill next to a river. And it's 'running away' movement just runs it away from what seems to be a nearby unit, without care for whether its new position will be reachable by enemy units.

They're also really bad at using the correct policies to counter these issues, they'll get a military policy to build city walls faster over something to increase military unit production, even if it's building units over walls. And they will get the policy to reduce the cost to purchase tiles in a wildcard slot over the reduced maintenance cost, despite them sitting at 0 gold and -8 gpt.

Enjoyed this analysis in addition to oedali's earlier post. I am curious - for those who have a professional idea of how these things are done - are most of these problems a matter of spending more time and writing more lines of logic? Or do you observe (indirectly at this point) that the approach itself is too flawed?

It sounds to me from various articles or livestreams like they have all of one AI person (just checked credits, they mention 4 total designers/programmers if you do not count gameplay engineers). For a game that sold a million copies in the first few weeks - presumably $50M in early sales, easily a $200M+ revenue game over its lifecycle, you would really think they could afford to hire a couple more programmers to knock some of these issues out more quickly. I am not a programmer and don't know how much throwing more people at these things can help or not.

But I hear things like the AI allegedly not using the right military production cards to boost what they are currently making, and I don't get it - it seems like such a basic strategic requirement for AI. Or being indecisive and changing their plans every turn - seems straight-forward enough to commit them to a plan of attack for a few turns before reevaluating their chance to win and whether they should pull back. I don't want to bag on Firaxis because it seems from the patch notes that they are trying and they are improving the game - but I'm wondering if 2K or Take-Two (which I understand is publicly traded) are holding them hostage to a P&L (income) target and not giving them enough programming resources.

 

[Stringer1313]

I don't want to bag on Firaxis because it seems from the patch notes that they are trying and they are improving the game - but I'm wondering if 2K or Take-Two (which I understand is publicly traded) are holding them hostage to a P&L (income) target and not giving them enough programming resources.

That could well be the case. Folks on these forums have to be aware that it's all about the bottom line, and that Ed Beach et al. are all subservient to their publisher overlords. Just imagine Ed Beach trying to explain to Take-Two why they need more AI programmers: "Yes, our sales are skyrocketing, and yes whenever we release DLC we make even more money, but can you pony up an extra $500k+ so we can hire more AI programmers so that the .0001% of fans who are hardcore civfanatics can be satisfied? Oh and by the way, the hardcore civfanatics end up buying all our games and DLC regardless."

 

[Siesta Guru]

Enjoyed this analysis in addition to oedali's earlier post. I am curious - for those who have a professional idea of how these things are done - are most of these problems a matter of spending more time and writing more lines of logic? Or do you observe (indirectly at this point) that the approach itself is too flawed?

It depends a little on the type of AI you hope to make. If the goal was to make an AI that is as competitive as possible, as in, one that could give even the strongest players a run for their money on prince, then the approach itself is flawed and there's no way to make it work with most of the current combat and empire building systems.

But I think that was probably never their goal. What their goal likely is, is to have something that helps provide some sort of narrative. Have the AI act like people imagine 'real empires' would be like within this games environment. Have them interact with most game mechanics, let them do some science, spam some missonaries, settle cities, do some war, pose at least an illusory threat here or there, have some fun personality/diplomacy related stuff in there to make players feel like they are living breathing entities with their own distinct personalities.
Have players go through that world and come out with some memorable experiences, then post online how great it was that they formed a real friendship with Gandhi based on a love for peace and nukes, and how Harald kept raiding their coasts meanwhile. The player doesn't have to feel like engaging in some battle of geniuses all the time, just like they have meaningful interactions.

Now clearly, the AI misses that mark too in its current state. But it isn't necessarily as far off as they are of the competitive AI. I think for the most part, it could've been solved just by throwing more time, code and tweaks at the current system. Having a single additional AI programmer, or maybe even half of one, could've made it significantly more entertaining. The current system can't be truly competitive with good players, but it can with some work provide meaningful narrative. Successful surprise attacks, runaway AIs on immortal, surprise religious victories, etc aren't too far out. And of course agendas that feel more real, rather than the current 'Kongo denounces you for not converting him within 4 turns after starting a religion'. None of this is really out of reach in the current system. I think they just ended up short on time/manpower.


But yeah, if they wanted a competitive system, similar to say better chess machines, then the approach they took is not nearly adequate. It'd require much more 'fuzzy' situation dependent combat systems, which just isn't possible with the current behavior tree approach. For example, we now have a clear 'running away' hp threshold, while ideal techniques would require that to shift heavily based on the environment. Running away may not be ideal if this damaged unit can actually finish a more important unit off. You'd need completely different ways of approaching combat for that, likely involving good ways of evaluating the endresult of doing certain moves, and comparing them to other option (with good pruning, since the decision tree is huge).
The empire building part would require more integration with itself through soft overarching goals that can impact all parts of its performance in this area. Right now it's all still very disconnected from each other (although better than in civ5). It for example doesn't think about why it might need a certain great person, it just has weights applied to buildings, and then ends up stumbling into a great person. It doesn't say plan out a wonder to boost with the great engineers that can. And it wouldn't select a faster unit training policy to prepare for a war or because of tensions with a neighbor. A lot of this is pretty invisible to players, so not so important for the narrative, but it is required for them to be able to compete properly.

 

[Acken]

Like in many things it should be a bit of both. An AI that is capable while maintaining flavor and theme on a per civ basis.

The problem is that before theming it and adding different flavors it should first be capable regardless.

 

[PYITE]

This is indeed mostly what's going on.
The self-preservation point is slightly off though. It doesn't do any individual unit self-preservation tactics like running away as long as it's above a certain threshold. If you see these units run away at high health, it's most likely that on an operational, team level, it decides that the situation is unwinnable. It could also actually not be running away, but could for example be wandering or trying to pillage. They do however sometimes choose not to attack units even if at full health if the combat strength of the target is significantly above their own. This is especially obvious around cities in later eras, as their combat strengths tend to be way above that of units.

The indecisiveness seems to partially be because individual units don't appear to be any memory. If there's several possible things they could be doing, and it can't decide on one of these being significantly better than the others. Then small changes in environmental factors will make them switch targets/types of behavior on every turn, often causing them to bounce back and forth.

I'd also add that the units appear to have no, or very little concept of terrain and distances. It doesn't seem to think about say trying to end a turn on a hill next to a river. And it's 'running away' movement just runs it away from what seems to be a nearby unit, without care for whether its new position will be reachable by enemy units.

In vanilla (fixed this in AI+), there's also some nasty issues where the AI fails to reach a condition and then gets stuck. The worst was requiring all units in its operation to be within 2 tiles of an enemy city/a gathering point. Which was often impossible due to too many units for the amount of tiles available to move to. Which was most commonly seen as units getting stuck on borders/just kind of standing around in your terrain and sometimes as them just staring cities down for hundreds of years.

The AI is especially in later eras also significantly held back by the game design and balance. The high city combat/health and healing in renaissance and up, require endured competent play to even stand a chance of taking a city. The AI is nowhere near the level required for that.
Also, In later eras the production costs of units ramp up so much, that it even takes the deity AI (whose economic bonuses are more like immortal in civ 5) 6-10 turns to build a single unit. On top of that, to make them actually competitive in culture/faith while production is necessary, they're forced to underbuild commercial districts, leaving them without money to sustain and upgrade their troops, disbanding is common even on deity. And even if they have the money to upgrade, they might not have the necessary resources. Leading to small underdeveloped armies after classical.
They're also really bad at using the correct policies to counter these issues, they'll get a military policy to build city walls faster over something to increase military unit production, even if it's building units over walls. And they will get the policy to reduce the cost to purchase tiles in a wildcard slot over the reduced maintenance cost, despite them sitting at 0 gold and -8 gpt.

Not that it even matters that much in later eras, since there appears to be a bug (?) that makes city assault operations be significantly less likely to even be considered.

Seems spot on. It would be interesting to have all units be given the samurai ability and see how it effects their behavior, if at all. It seems like the reduction in attack strength from being injured has a significant effect.

 

[V. Soma]

Siesta, I like your "narrative AI" naming. I think narrative for this game is important, you know, immersion...

I still have hopes for this game, but not from the devs... they should give the tools (SDK?), the means to the serious fan modders...
khm, for example, to you, Siesta...

 

[Binkov]

So, without sdk, best one can do is make the rules easier for the AI? And biggest issues for AI are walls and warmongering? So if one mod removed most of warmongqring penalties and removed walls altogether (could one keep city's ranged attack for AI only?) - what sort of effect would it have on AI conquering cities?

 

[Siesta Guru]

Seems spot on. It would be interesting to have all units be given the samurai ability and see how it effects their behavior, if at all. It seems like the reduction in attack strength from being injured has a significant effect.

It'd have very little effect in behavior I think. The one thing that may happen is that they're less likely to perceive a fight as ineffective while wounded (if it even uses that samurai ability in its calculations for that), but that's not that big of a thing holding it back.

Siesta, I like your "narrative AI" naming. I think narrative for this game is important, you know, immersion...

I still have hopes for this game, but not from the devs... they should give the tools (SDK?), the means to the serious fan modders...
khm, for example, to you, Siesta...

Some more modding support would be awesome indeed, especially if the dll source is released. But I'm not anticipating that to happen anytime soon.

So, without sdk, best one can do is make the rules easier for the AI? And biggest issues for AI are walls and warmongering? So if one mod removed most of warmongqring penalties and removed walls altogether (could one keep city's ranged attack for AI only?) - what sort of effect would it have on AI conquering cities?

There's still quite a bit of room in improvement for the AI without dll access I feel, v9 of AI+ will be rather large at least (this is on immortal: http://puu.sh/sZKgQ/6ad6a33652.jpg). But yeah, late game city conquest is still way too hard with the current gamerules. It's not really the walls, more that the tech scaling of city combat strength puts them enough above the melee units of the same era that they become useless. And since the AI manages melee units way better than siege weapons, it ends up failing to take anything.

 

[Gort]

So, without sdk, best one can do is make the rules easier for the AI? And biggest issues for AI are walls and warmongering? So if one mod removed most of warmongqring penalties and removed walls altogether (could one keep city's ranged attack for AI only?) - what sort of effect would it have on AI conquering cities?

If we're going into the "strategic" thought of the AI, (warmongering and such) we need to decide what our actual goals for the AI are. Currently the AI is really only able to pursue two victory conditions:

1. Religious victory
2. Making the player ragequit

They don't pursue a domination victory - you won't see an AI go on a conquest spree, knocking over capital after capital, ignoring other cities. They will stumble into a culture, science or score victory, but they won't pursue it particularly.

But do we actually want an AI that tries to win? If we do, then things like diplomatic penalties for warmongering make no sense - but penalties for being the closest to winning do. If a "play-to-win" AI has decided to pursue a domination victory, it shouldn't matter how many gifts you've sent them or even if you've got an alliance - they might take your capital last if they like or fear you, but eventually those nukes are going to fly. Many players don't want that, they'd rather be able to make a game-long alliance with an AI player.

 

[leon_daking]

I tried doing exactly that and the game wouldn't let me - in two different sessions. If it's a bug it's an odd one. Scouts always could take settlers in earlier games but I haven't been able to this time.

Or perhaps I should have officially declared war first rather than the typical 'attack and get a declare war prompt'?

yes, declaring war first will let you steal settlers and workers with scouts

 

[PhilBowles]

If we're going into the "strategic" thought of the AI, (warmongering and such) we need to decide what our actual goals for the AI are. Currently the AI is really only able to pursue two victory conditions:

1. Religious victory
2. Making the player ragequit

They don't pursue a domination victory - you won't see an AI go on a conquest spree, knocking over capital after capital, ignoring other cities. They will stumble into a culture, science or score victory, but they won't pursue it particularly.

But do we actually want an AI that tries to win? If we do, then things like diplomatic penalties for warmongering make no sense - but penalties for being the closest to winning do. If a "play-to-win" AI has decided to pursue a domination victory, it shouldn't matter how many gifts you've sent them or even if you've got an alliance - they might take your capital last if they like or fear you, but eventually those nukes are going to fly. Many players don't want that, they'd rather be able to make a game-long alliance with an AI player.

This was an argument through much of Civ V, as the AIs were touted as trying to aim for victory. They could very rarely go for domination effectively and rarely tried, but they were more capable than some earlier versions at following reasonable science and culture victories, and could play for diplomatic victories. In contrast Civ IV could only really go for domination victory - but was very effective at it and it gave a sense of character to see Genghis run through the world conquering or vassaling everyone as he went.

The problem with the Civ V approach is that it relies on the player being willing to play along with the AI and get to the late game. I had some of my most rewarding games when that happened - such as a fight over CS influence with Genghis as we were both aiming for diplo victory in G&K, a literal fight over CSes in a game where my influence was unassailable and multiple AIs tried conquering my CSes to take them away, or a last-minute science win that the Ottomans tried to disrupt with a massive invasion. All of these have one thing in common: the AI has to be allowed to develop and become militarily powerful. Most players focus on just beating the AI in combat - even if not going for domination, setting the AI behind or crippling its military prevents it from ever competing with the player effectively.

 

[isau]

One particular game rule that is totally throwing the AI at present is that Encampments come later in the tech tree than Holy Sites and Campuses. So they build those relatively less useful districts simply because they are available, and run out of population for the Encampment. Then, the player ends up playing the whole game against units who never got the benefit of the Encampment.

To partially fix this, it turns out removing tech pre-reqs from the Encampment will goad the AI to build more of them. It's not a perfect fix, and it does mean AIs have even more trouble taking each other's capitals. But it puts a little more pressure on the player.

FWIW, one egregious thing the AI does not seem to understand is to put ranged units in its city and any encampments and shoot things with them. This makes the Encampment somewhat less useful.

In other news, I also suspect in the early game multiple AIs are all chasing Stonehenge. That is the only explanation for how someone always grabs it so early. The AIs that are losing the race basically lose all those turns doing nothing.

 

[Victoria]

I have lost a few high level games to science victories.
For culture they do not compete well but do try a little

 

[juanpavo]

I also suspect in the early game multiple AIs are all chasing Stonehenge. That is the only explanation for how someone always grabs it so early. The AIs that are losing the race basically lose all those turns doing nothing.

Interesting observation. That sounds likely to me.

 

[Disgustipated]

Anyone finding it more difficult to keep up in scientific (as well as cultural/civic) research? I find myself falling behind more than usual. It could be the characteristics of the Earth map, hard to say. I'm finding the need to build 2 campus districts (which is a good things as they should be used).

 

[PhilBowles]

Anyone finding it more difficult to keep up in scientific (as well as cultural/civic) research? I find myself falling behind more than usual. It could be the characteristics of the Earth map, hard to say. I'm finding the need to build 2 campus districts (which is a good things as they should be used).

In my newest Emperor game I'm slightly ahead on science by turn 100, rather earlier than usual - but that does come from having captured four of Brazil's five cities (Rio still has a science output of 19 by itself, so my science should accelerate when I finish capturing that).

Further combat AI observations: unless the AI planned to give me all its cities as a Christmas present it still defends very badly; the initial American attack was choreographed turns ahead, so I declared war and razed Philadelphia. Pedro then demanded to know why my units were massing at his border, prompting me to attack several turns early. I was able to go through three Brazilian cities without pausing to replenish my units' health - this may be an unfair example, as I rolled Sumer and so had War Carts at my disposal.

The AI is however building walls much earlier and most cities I attacked had garrisoned units ... unfortunately these are melee units, and didn't attack adjacent units even when they had the opportunity. The defence of Rio was better than average - a chariot sometimes left the city to attack only to return. The AI even built a catapult ... and parked it next to the city while the chariot remained inside. Still, it's something.

Every settler I've seen in this playthrough has been escorted - unfortunately for Brazil one of them was right next to the army about to take Manaus, so I got an extra free city into the bargain.

Overall, there's been an improvement, though it's not sufficient. AI units are now much more aggressive in multiples - but a single AI melee unit will usually flee and I've had multiple instances of AI units not pressing the attack on one of my badly-wounded units it might be able to kill.