I have a pretty good idea how to fix the combat AI

I've seen some of the programming "decision making" notes of the developer of PG forever, so that makes me as competent to suggest this as a badger is competent to talk about Hermann Hesse. But bear with me.

The following steps get refreshed every turn.

STEP ONE:
AI takes a look at it's own cities designates importance numbers for them. This is based on:
a. capital or not
b. total yield summed up in 2-hex radius of a city
c. number of resources available in 2-hex radius
d. whether city resources are available in other cities or not
e. population number of the city
f. number of buildings and wonders in the city
g. number of improved tiles around the city
h. proximity to competing civ (high bonus)

Example screenshot (I later decided on another city to be priority #1)



STEP TWO:
The AI marks tiles adjacent to its cities as "danger" or "safe" in the following fashion:
the AI draws a line in each of the 6 directions from every city.
a. if the line ends at hostile borders or passes a hex adjacent to enemy borders, the adjacent city tile is "hostile"
b. if the line ends in a friendly city or there's a friendly city adjacent to the line, the city tile is "safe"
c. if the line doesn't pass at least 5 friendly tiles, it's considered "hostile"

This is also the way AI determines "border cities". And here's the image:



STEP THREE:
The AI marks the cities and the safe zones as GREEN.



STEP FOUR:
The AI creates 3 lines of hexes, Orange (hostile), red (danger) and Green (safe) in the following fashion:
a. Any hostile hex is marked "red"
b. Any hex adjacent to a "red" hex AND would border a "green" hex is tagged "red"
c. Any hex adjacent to a "red" hex AND would NOT border a "green" hex is tagged "orange"
d. cities are automatically tagged "green"
e. any hex bordering two "green" hexes and a "red" one becomes "green"



STEP FIVE:
AI determines "good defense" tiles in the ORANGE LINE.
a. first it marks all hills and forests
b. then it marks all orange tiles that have a river crossing directly OPPOSITE of a RED hex (it does this for red hexes, but is not shown on images)



STEP SIX:
AI abandons all "good defense" ORANGE hexes that do not:
a: border a RED hill
b: do not border at least THREE red hexes
and marks them BLUE



STEP SEVEN:
AI designates melee and ranged units to hexes:
a. BLUE hexes gets tagged as MELEE
b. every RED hex gets tagged as MELEE
c. every RED hex that would border 3 or more MELEE gets converted to RANGED
d. every GREEN hex that borders at least two MELEE gets tagged as RANGED
e. every city gets tagged as RANGED
f. every GREEN that borders 2 or more RANGED gets tagged as REINFORCEMENTS




STEP EIGHT:
So far, we described several sets of tiles (melee, ranged, reinforcements) that can be easily calculated and understood by an AI. These tiles would help it defend itself IF it would have enough units to fill out all the tiles it used for the calculation. However, that is almost never the case. Now we have to implement a way to get the AI make use of the tile values it added itself to the hexes in question, considering the army it has at its disposal.

For us to help him do this, we are going to introduce a way of reducing the values of the stated hexes, from melee to "abandon", from ranged to melee and from reinforcements to ranged.

Lets say Montezuma has 6 units at his disposal. We also made the AI calculate distance of every red tile to each of his cities and Texcoco proved to be the shortest distance to all the red tiles. So, he has 6 units around Texcoco, 3 swordsmen, 1 catapult and 2 archers.

We invade Montezuma and he can see our first unit threathening him on the Cotton between Beijing and Tlatelolco.

By using the city priority numbers he decided upon this round he decides Tlatelolco is his highest priority and his units can reach it in 2 turns.


STEP NINE:
The Defense of Tlatelolco.

We tell it: AI you have 3 melee and 3 ranged units to defend yourself. We also code him so that AI can use only 1st and 2nd ring of a city to defend itself. The AI Determines that the following tiles match the criteria.



The AI places his units in the following fashion:
a. first it determines one unit MUST go into the city. the top priority for city units is, as we know, RANGED. and among ranged, ARTILLERY has top priority. So the catapult goes into the city. It has 3 melee and 2 ranged units left.

b. it has 10 vacant tiles and "less than 7" units. Thus, it cuts all the tiles not adjacent to the city first.


c. Since the ranged and reinf. tiles don't match the criteria anymore, they get reduced by one rank, to melee and ranged, respectively.



c. now he places the 1st swordsman on tile that was tagged as "good defense" (hills or river crossing opposed to green zone).
d. since no other optimal defense tiles are available, he proceeds to place the RANGED
e. and now he places the remaining melee so that they are adjacent to ranged (otherwise it would proceed to place melee on good spots and adjust RANGED accordingly).



STEP TEN:
The Issue of suicidal AI moves becomes fixed by simply coding the AI NOT TO MOVE units when on defense, except if one of the following criteria matches:
a. the hex changed state (from melee to abandon, from ranged to melee) or becomes vacant
b. peace broke out
c. enemy units are elsewhere



Damn, my brain hurts after these 5 hours.

That's pretty good. Of course, one other requirement is to get the AI to focus on defense over offense if you declare war on it. All of this is irrelevant if they move all their units to your city and get wiped out.

Louis XXIV said:

That's pretty good. Of course, one other requirement is to get the AI to focus on defense over offense if you declare war on it. All of this is irrelevant if they move all their units to your city and get wiped out.

Click to expand...

This is part one, the defense. I'll do the offense as well.

A lot of good work there although a bit more than I really can follow.

Here are some simple concepts I have to suggest (at KING level)

1) I go to my military Advisor and find out how much military the AI have over me. I am often shocked that I can build 2 warriors and all of a sudden the AI is laughable. I understand the Stacks of doom are gone, but the AI builds too few units.

2) The AI needs to agressively defend more. ONce attacked or at military disparity it needs to focus on defense.

3) The AI attacking City states too often leaves it's empire open to attack. The AI should break off attacks to send units back on defense.

4) The AI needs to be programmed to suicide units to break open a cities defenses. I have seen the AI surround a City State for a long time using ranged attacks but never sending a simple Musket until defenses are worn away.

5) Small empires mean a smaller military. The AI should settle more cities to increase it's overall production.

One big aspect is the continual economic costs which affects the AI more than humans. I think there should be a minimum free units in this game, maybe 1 or 2 per city. Or 1 Melee and 1 ranged per city. You can compromise this by increasing costs for additional units than what we have now.

This will require dll access, do you have the skills to implement this or are you throwing it out there for someone else to implement?

Some concerns:
- turn times. This is obviously doing a lot more than the game is now. Maybe instead of having every civ do this every turn, there could be some sort of trigger conditions that would cause the AI to re-evaluate on the next turn? Or perhaps the AI should only evaluate it's defense in this manner while it is in a defensive war? Peacetime evaluation could be done just for the number and different types of troops needed.

- should consider the effect of roads. For instance you would want to defend with melee/gunpowder behind a bridge, both for the increased defense of forcing your opponent to attack you across a river and also for the opportunity to counter attack across the bridge.

We also code him so that AI can use only 1st and 2nd ring of a city to defend itself.

Click to expand...

I wonder if this is the best idea? Yes, you can get the cities own ranged attack involved, which is nice. On the other hand your unit's can't retreat any further and you've also opened a route towards other cities.
If we were attacking this AI, I would feint towards Tlatelolco until the defensive forces deployed around the city and then I would march north and take Cologne instead. According to the algorithm, the most important city to defend.
If I were defending in this situation I would base my defense on the hills and try force him to either attack me on the hills or come through the gap between them. If he came through the gap I would attack those units with a reserve force. He should be smart enough not to do this though and should try to dislodge my units from the hills first.

I don't think any of these are reasons to abandon this kind of approach, but they are weaknesses in it.

Mannu said:

This will require dll access, do you have the skills to implement this or are you throwing it out there for someone else to implement?

Some concerns:
- turn times. This is obviously doing a lot more than the game is now. Maybe instead of having every civ do this every turn, there could be some sort of trigger conditions that would cause the AI to re-evaluate on the next turn? Or perhaps the AI should only evaluate it's defense in this manner while it is in a defensive war? Peacetime evaluation could be done just for the number and different types of troops needed.

- should consider the effect of roads. For instance you would want to defend with melee/gunpowder behind a bridge, both for the increased defense of forcing your opponent to attack you across a river and also for the opportunity to counter attack across the bridge.


I wonder if this is the best idea? Yes, you can get the cities own ranged attack involved, which is nice. On the other hand your unit's can't retreat any further and you've also opened a route towards other cities.
If we were attacking this AI, I would feint towards Tlatelolco until the defensive forces deployed around the city and then I would march north and take Cologne instead. According to the algorithm, the most important city to defend.
If I were defending in this situation I would base my defense on the hills and try force him to either attack me on the hills or come through the gap between them. If he came through the gap I would attack those units with a reserve force. He should be smart enough not to do this though and should try to dislodge my units from the hills first.

I don't think any of these are reasons to abandon this kind of approach, but they are weaknesses in it.

Click to expand...

Maybe not per turn yes, but definitely every time a new city is founded or an existing city changes an owner (or every 10 turns if both stated criteria are not met).

Yes, any tile designated as "good for defense" should be a signal for an AI worker to build a road access to it, although it could raise an issue of extra maintenance.

Yes, other cities will be vunerable to attack if this was a feint. But if it was a feint, those 1 or 2 units you sent will die quickly (thus triggering the "no enemy unit within sight range"), the stated 6 units will be alive and ready for a counterattack (which I will cover in my "Offense" AI suggestion post). Also, you wouldn't be able to defend 2 cities with 6 units either (well, maybe you would but that's not the point ). If Cologne wasn't a puppet it could rush-buy an artillery piece and Texcoco could rush-buy an infantry or cavalry unit (if finances allow it). Even if you pull off your feint and Cologne maybe falls, considering how MEAN I plan to design the (counter)attack AI you'd probably loose your whole attack force in the process, not just the feint troops.

Using the two hex-rings around a city that is important enough to defend is a good start. I don't mind expanding it once it has been implemented.

Sadly, I have no programming skills so yes, I'm hoping someone will pick this up.

Brilliant thread! I have been thinking about starting one myself, but now I'll just add my 2c here (I'm more from the chess field, but the base concepts still apply).

The AI should move every turn in order to maximize the "value" of its combat force versus the "value" of the opponent's combat force. There are two aspects to this: tactics and strategy. Tactics is concerned with which enemy units you weaken/kill, while strategy is concerned with positioning your units favorably. You have done excellent work on the value of positions for defensive units, and I suggest that when you make the example for the value of positions of offensive units, you use Tlateloco again - how the AI assesses the value of the enemy's offense is exactly how it would assess the value of its own offense.

The basic combat algorithm, IMHO, would work like this: The AI estimates the value of the enemy units, by where they are positioned and by how much they can damage the AI. The 'threat-value' is bigger if a unit can severely damage valuable AI units (value=strong, highly promoted, well-positioned), or if a unit can bring city defense into the red (or take the city). Then it attempts to eliminate as much 'threat-value' as possible, and to position its own units as favorably as possible. This might need multiple iterations, since the AI might realize that its attack leaves archers wide open, so that it needs to move the archer first, which changes which enemy units can be attacked (however, in a first version of "BetterAI" we might leave this out).

Since it is likely to be very time-consuming to test different outcomes of the battle, the AI needs to have very good value estimates of its own and the enemy's units.


Now about a few specifics about your algorithm:
Step 1: I'm not sure that the closest city should have such a high priority - it may be better to abandon a badly-placed city than to place the units at one unimportant end of the empire. Rather than closest, the border city with the most reachable neighbors should be the prime defense target in step h, since if that city falls, the empire breaks apart. Also, there should be a step i: The city with the most enemy units nearby gets quite a high value.
Step 3+: Your algorithm is great for peace-time, but if there are enemy units around, they need to be taken into account - if after the projected attack, the enemy can move in to kill your ranged unit, it may not be the best place to put it.

I think Age of Empires 2 had some nice AI combat programming.

They would attack you, but only after they had built up a force that could defend themselves.

Also, the AI would create stuff like divisions and battalions, lead by one unit, that could function as an autonomous attack or defensive force.

If the Civ ai could be programmed in the same way, it would mean that they would have large blocks of units, some defensive, some offensive, and it would be able to defend and attack more intelligently.

I think this is a good start for the issue... it is not by far a integral fix , though.

Some issues were already raised by other posters ... I'll just add something of my own:

- There needs to be a second algorythim that works in top of this for modifying the value of the tiles when in the presence of enemy units. A tile that is good for ranged units might easily cease to be in the presence of a mounted unit.

- We need to make the AI understand the value of the formation bonus ... IMHO that should be done by decreasing partly the rank if you have little or no formation bonus ( this is, we don't put archers alone or in the flanks )

-Probably this needs some more code to recognize range 3 ranged units

- The AI needs to understand that, even when attacking , it still needs to have a defensive posture ( this is a turn based game after all ... everyone passes most of the time defending by design ). A similar heuristic must be applied to attack.

- Reinforcements should not march alone outside of safe zones . This is more problematic , but i think it would not be a bad idea to make the AI to think that , besides areas out of reach of any possible unit, it can be ambushed everywhere ... and thus forcing the formation of a mini blob of units to march reinforcements to the combat area if it needs to pass non safe terrain ( this kind of situations happens a lot , having to combat in neutral areas just because it is easier to attack/ defend there )

Anyway, nice work. My hat is off

The general idea behind the attack routines I'll post as soon as I wrap my head around it are the following:

- grouping units into a "deathstar" formation; units never walk alone
- assigning each tile (and groups of tiles) a tactical value
- warpaths for the deathstars ("boring into enemy territory", assigning "threat levels" to both visible enemy units and terrain features)
- stripping individual units of their status and attaching them to either a Deathstar when on offense or turning them into tile attributes when on defense

I was originally going to post a reply bashing this as another bad idea, but you might have something here. I love your attention to detail. Hope firaxis reads this.

Thry said:

I was originally going to post a reply bashing this as another bad idea, but you might have something here. I love your attention to detail. Hope firaxis reads this.

Click to expand...

I hope either Firaxis or somebody with the right programming skills reads this.

Bibor said:

The general idea behind the attack routines I'll post as soon as I wrap my head around it are the following:

- grouping units into a "deathstar" formation; units never walk alone
- assigning each tile (and groups of tiles) a tactical value
- warpaths for the deathstars ("boring into enemy territory", assigning "threat levels" to both visible enemy units and terrain features)
- stripping individual units of their status and attaching them to either a Deathstar when on offense or turning them into tile attributes when on defense

Click to expand...

I would allow horsemen to walk alone ... their role is normally go ahead to scout or to be a separate group given the disparity of speed between mounted and the basic foot units/artilery. As long as you reassess the situation while moving it's ok to leave them outside ( but close enough for protection if needed ) of the death stars .

Other thing that needs work is the sieging aproach to cities: the AI needs to learn how to position ranged/ artilery units around cities and to pick the high ground + unobstructed view spots. maybe that would even be good for defense as well: if the AI has a map of where the enemy will position their units to siege, it is a good start for making countermeasures against the said siege

Bibor said:

I've seen some of the programming "decision making" notes of the developer of PG forever

Click to expand...

Where did you find this, I would be very interested in reading this myself?

r_rolo1 said:

I would allow horsemen to walk alone ... their role is normally go ahead to scout or to be a separate group given the disparity of speed between mounted and the basic foot units/artilery.

Click to expand...

This is essential. If you do not teach the AI to scout, the way to hammer it will simply be to observe the existence of a death star with your own scouts, determine its intended path, and set up a monster enclosure a few turns later that annihilates the death star. That's more or less what I do currently when on the defensive in the early game, since engaging the AI near your own cities is hard on your ability to develop terrain with Workers.

^ Completely agree. The AI often makes apparently suicidal moves that make no sense because it can't see my units and doesn't seem to have any inclination to discover where my units are, nor remember where they were last turn (and think that maybe they're still there).

@ OP: great thread. Can't wait for part 2! Will you post that in this thread or make a separate thread?

Good stuff so far. The best part of civ 5 is the addition of tactical combat. And its easy to win even if you make mistakes.

One pet peeve: why doesn't the ai attempt to eliminate units from the board? A city + ranged (from garrison) + melee will nearly always kill an attacking unit. But it seems the ai distributes attacks to cause maximum hp loss rather than kills units. I rarely lose units unless there is only one next to the city.

I love all the terrain discussion too. The ai should assign defendable entry points to its empire. Like buikding a fort or fort x 2 + ranged to defend a mountain pass. I spend a lot of time choosing chokepoints when expanding and it all goes to waste if the ai doesn't.

Some additional questions the ai should be asking

How do I kill the enemy's ranged/artillery support?

Touched on above: where are my optimal empire defense points?

And, why is my navy so bad?

Keep up the good work.

XuZidan said:

One pet peeve: why doesn't the ai attempt to eliminate units from the board? A city + ranged (from garrison) + melee will nearly always kill an attacking unit. But it seems the ai distributes attacks to cause maximum hp loss rather than kills units. I rarely lose units unless there is only one next to the city.

Click to expand...

If it does try to maximize HP loss instead of killing units, it's playing suboptimally in many if not most situations where this would come up. Living units cost no hammers to replace if they're allowed to heal- and they heal faster then they can be built. In addition, the last HP of damage is the best HP to remove in terms of an opponent's potential to deal damage to your forces.

10HP --> 9HP means a unit loses 5% of its combat strength.
9HP --> 8HP means a unit loses 5% of its combat strength.
...
...
2HP --> 1HP means a unit loses 5% of its combat strength.
1HP --> 0HP means a unit loses 55% of its combat strength.

This means that, given the option between taking a 10HP unit down to 1HP or finishing off a 1HP unit (of the same type), you can reduce your opponent's offensive power more by dealing 1HP to the nearly-dead unit. Sometimes it would be better to reduce the 10HP unit to 1HP due to other concerns, like tactical positioning, but I don't have any faith that the AI could be programmed to make the correct decision, so I'd err on the side of finishing off units in every situation (unless the AI can accurately figure out the better course of action in the majority of cases).

XuZidan said:

One pet peeve: why doesn't the ai attempt to eliminate units from the board? A city + ranged (from garrison) + melee will nearly always kill an attacking unit. But it seems the ai distributes attacks to cause maximum hp loss rather than kills units. I rarely lose units unless there is only one next to the city.

Click to expand...

That's another critical tweak; fortunately such a change would be incredibly simple to make compared to the problems Bibor's trying to address.

Tyrael said:

.1HP --> 0HP means a unit loses 55% of its combat strength.

Click to expand...

I assume you meant to imply this too, but I just wanted to make it explicit that as a unit that's at 8 HP is going to be easier to bring down to that critical 0 HP than one that's at full health, it should weigh how close the unit is to death even if it can't take it down in that attack or even that turn. Although then you'd have to give it some parameters for avoiding excessive overkill if it can.

Yes, thanks for clarifying. Good point about avoiding overkill- it seems like that would be an easy thing to teach the AI to do. Maybe it even does that already?