How fixable is the AI?

[JP1]

This isn't necessarily true. There's rarely a front line which spans 64 hexes with 16 pieces, some of those pieces able to move in all directions up to seven squares. A lot of basic AI failings occur with very few land units in a small area of land.

The main issue is massive numbers of 'moves' per turn. Even though there are fewer pieces in the standard civ battle, I'd be willing to bet that the branching factor in a Civ5 battle would be much worse because of each piece gets 2 (or more) moves per turn. Even a group of 3 or so units probably has many dozens of potential moves, even if you ignore moves which are identical aside from move order. (And because of things like flanking and the +15% bonus from the honor tree, moves which are different other than move order might actually not be.)

 

[slowjames]

The main issue is massive numbers of 'moves' per turn. Even though there are fewer pieces in the standard civ battle, I'd be willing to bet that the branching factor in a Civ5 battle would be much worse because of each piece gets 2 (or more) moves per turn. Even a group of 3 or so units probably has many dozens of potential moves, even if you ignore moves which are identical aside from move order. (And because of things like flanking and the +15% bonus from the honor tree, moves which are different other than move order might actually not be.)

Meh. If a good local evaluation function can be developed, I would bet that minimax could check up to a few moves in advance for the few parts of the board where it's necessary. Some things like A-B pruning can really cut away a lot of the trees. Although there will be no way to tell if they are doing something like this until we can see the AI they have.

People commenting about getting Chess AIs right are ignoring one minor issue: a "tolerable" chess AI can be developed with much more ease than the ones that fight grandmasters. Computer Science students do it in AI self-studies. No one plays Civ as well as a grandmaster of chess plays chess. We don't need a grand master AI for Civ, we only need a tolerable one (in my opinion). And that is a substantially different proposition.

The other thing is that although chess may seem very constrained, there are a couple of things about it that are more difficult than Civ. One of the biggest problems is that the evaluation function is much more difficult in chess, because a lot of it depends on interrelationships between the pieces. The other thing is that in chess pieces tend to affect other pieces all the way across the board, whereas in Civ pieces rarely affect units more than two steps away from where they are when they end their turn. Finally, in chess you need to "see" further. Moves often appear good even four turns in the future then turn disastrous. I'm sure there are some rare situations where that can happen in Civ, but in general that's not how things go.

I am sure making a human-level AI for a wargame or a game like Civ is difficult. But I suspect the problem is more economic than technical: simply put, wargames do not sell for their AI. After buying the game, people will gripe about it, but it's rarely a factor that enters into the buying decision.

Time will tell whether I'm right or wrong, but I suspect that a lot of the people commenting on these Civ AI threads have never actually programmed an AI themselves. I have, albiet one for an even simpler game (Othello), but that was in my first year programming. I'm confident that there is room for enormous improvements in the Civ V tactical AI, provided Firaxis wishes to pursue it, or we do.

 

[JP1]

Meh. If a good local evaluation function can be developed, I would bet that minimax could check up to a few moves in advance for the few parts of the board where it's necessary. Some things like A-B pruning can really cut away a lot of the trees. Although there will be no way to tell if they are doing something like this until we can see the AI they have.

People commenting about getting Chess AIs right are ignoring one minor issue: a "tolerable" chess AI can be developed with much more ease than the ones that fight grandmasters. Computer Science students do it in AI self-studies. No one plays Civ as well as a grandmaster of chess plays chess. We don't need a grand master AI for Civ, we only need a tolerable one (in my opinion). And that is a substantially different proposition.

The other thing is that although chess may seem very constrained, there are a couple of things about it that are more difficult than Civ. One of the biggest problems is that the evaluation function is much more difficult in chess, because a lot of it depends on interrelationships between the pieces. The other thing is that in chess pieces tend to affect other pieces all the way across the board, whereas in Civ pieces rarely affect units more than two steps away from where they are when they end their turn. Finally, in chess you need to "see" further. Moves often appear good even four turns in the future then turn disastrous. I'm sure there are some rare situations where that can happen in Civ, but in general that's not how things go.

I am sure making a human-level AI for a wargame or a game like Civ is difficult. But I suspect the problem is more economic than technical: simply put, wargames do not sell for their AI. After buying the game, people will gripe about it, but it's rarely a factor that enters into the buying decision.

Time will tell whether I'm right or wrong, but I suspect that a lot of the people commenting on these Civ AI threads have never actually programmed an AI themselves. I have, albiet one for an even simpler game (Othello), but that was in my first year programming. I'm confident that there is room for enormous improvements in the Civ V tactical AI, provided Firaxis wishes to pursue it, or we do.

As you said, with A-B pruning, an evaluation function which is at least passably good is needed so you don't prune away stuff you really shouldn't be, and we don't know if they're doing something like that already, or even what kind of an evaluation function they're using to figure this stuff out (if any at all). That's why I'm a huge fan of the Monte Carlo method for things with difficult evaluation functions, but going that route is hugely dependant on how fast the game can churn out thousands of random playouts of positions. I'm willing to bet it wouldn't work out well, and it's probably better to just create a huge, contrived evaluation function with experience over time and use that in standard minimax with A-B pruning.

But this is realistically going too deep, as you mentioned... People are just looking for a passable AI that doesn't do silly things... I don't expect the game to tactically destroy me, but I did kind of expect it to not plink a different unit every turn with arrow fire. :/

 

[BlackSpy]

I think that even if some of these basic improvements were made people would still be hugely unstaisfied.

"lolz, stupid AI didn't guard his flank and owned his catapults with my horsemen. Stupid AI"

It's not one step it needs to go up, it's about four.

The 1UPT limitation means that the advantage teh AI can be given (more tech to field somewhat better units and more units in total) don't have the full impact they might have against a competent human chosing the battlefield. There are only so many fights that can happen in every turn on 1UPT, like the Thermapolye example above.

 

[slowjames]

As you said, with A-B pruning, an evaluation function which is at least passably good is needed so you don't prune away stuff you really shouldn't be, and we don't know if they're doing something like that already, or even what kind of an evaluation function they're using to figure this stuff out (if any at all). That's why I'm a huge fan of the Monte Carlo method for things with difficult evaluation functions, but going that route is hugely dependant on how fast the game can churn out thousands of random playouts of positions. I'm willing to bet it wouldn't work out well, and it's probably better to just create a huge, contrived evaluation function with experience over time and use that in standard minimax with A-B pruning.

I don't think you should give up on that Monte Carlo idea quite yet. It's true that the game probably won't handle it well if you try to get it to process full weight moves in random fashion. But if I were writing it, I'd copy the game state into a smaller arena with simplified battle mechanics and unit structures. Probably all the data you need for a local evaluation of tactics can fit into a megabyte, which means that you can do some really intensive cache-friendly searches using the simplified mechanics. Hell, make N copies and assign one to each node in the case of a NUMA architecture and you're going to get massive gains from parallelism as well. You don't want to do the full-weight simulation if an approximation with much less data gives you a similar result.

 

[BobDole]

I think it's fixable enough to be good. I'm not expecting it to play like a genius, but it'd be nice if it at least lined up units correctly before it marched on your cities (that can't be too hard to code, right...maybe?). The AI has never been stellar in Civ, but it was able to make up for it through brute force in the past; it could put together a killer stack and send it your way, which made up for some of its weaknesses (baiting them to fight you in unfavorable terrain, attacks from multiple directions, etc.). If they can get it decent enough, they can cover up the rest of the problems with the usual bonuses that the AI always gets on higher difficulty levels and it'll be enough. That's all I'm hoping for. It doesn't need to be the Bobby Fischer of Civilization, it just needs to not be embarrassing like it can sometimes be now.

 

[JP1]

I don't think you should give up on that Monte Carlo idea quite yet. It's true that the game probably won't handle it well if you try to get it to process full weight moves in random fashion. But if I were writing it, I'd copy the game state into a smaller arena with simplified battle mechanics and unit structures. Probably all the data you need for a local evaluation of tactics can fit into a megabyte, which means that you can do some really intensive cache-friendly searches using the simplified mechanics. Hell, make N copies and assign one to each node in the case of a NUMA architecture and you're going to get massive gains from parallelism as well. You don't want to do the full-weight simulation if an approximation with much less data gives you a similar result.

I suppose if you could come up with some simplified mechanics for the simulation that would model it well enough, then the sky's the limit, and you could even do fairly large-scale battles pretty well. Then the only issues would be deciding when a simulation are is needed, figuring our what tiles/units should be used in the simulation, and of course, the simplified combat simulation itself which we're kind of assuming at this point.

 

[The_Reckoning]

Interesting discussion so far - am thinking just simple things like not throwing away an entire army, and like others have said, moving stuff in formation. Supposing Firaxis abandon working on the AI entirely and only make additional DLC civs, do you think we're capable of teaching the AI to, say, invade across water? Fix the horrible pathfinding?

 

[slowjames]

Interesting discussion so far - am thinking just simple things like not throwing away an entire army, and like others have said, moving stuff in formation. Supposing Firaxis abandon working on the AI entirely and only make additional DLC civs, do you think we're capable of teaching the AI to, say, invade across water? Fix the horrible pathfinding?

I'm not sure what you mean about the pathfinding, but that type of thing is relatively easy. Moving stuff in formation is a little harder, but if it can fight in formation the same logic should be able to make it move in formation. Throwing the entire army away is most of what I personally am really interested in fixing. Invasion across water is more difficult.

I have been thinking about this quite a bit, and talking about it with my coworkers, who are smarter and better educated than I am. We are fairly convinced that a lot can be done to improve the tactical AI (the part that computes the best move for situations that will develop in the next one to three moves). In many situations, especially the early game battles that go especially badly for the AI, an exhaustive search may even be possible. So, I'm about as confident as I can be that this part can be improved.

The strategic question, like invasion over water, is a little trickier. Searching the minimax tree is no longer feasible on the time horizons we're talking about there, so we are moving into more of a heuristic/strategic realm. That said, I've been thinking about some concepts that might help. Again, I can't say until I see what Firaxis has done to see how smart or not it is.

One idea is that of making the computer have plans and commit groups of units to those plans. The plan would have to be evaluated each turn to see if it was still feasible, if more units might be needed, etc. Then the committed units could move as a group with imaginary enemies informing the positioning of the units using the same minimax search that would be used for tactical warfare. One plan could be "invade across the ocean" and the number of units needed could be calculated by soldier score plus growth. Other plans could be used to get the AI to load up your border before attacking rather than trickling units in.

Another concept I've had is to teach the AI to be afraid of the shadows. Specifically, the AI while fighting you should calculate some sort of danger function on the borders of the fog of war, probably dependent on your soldier count minus what's visible and your other predicted commitments. Its unit positioning should be informed by that function. That would help with cases where it invades you and leaves ranged units open for attack from units it can't see behind the fog.

I've also been thinking of how to make the strategic AI a little meaner. Kicking the player (or other AIs) while they are down is one possibility. Balancing its armies against your weaknesses is another. If I ever really get my project to improve this off the ground, I'll probably want to solicit ideas from the community for devious settings to make the AI on higher levels meaner and deadlier. No promises yet though.

I think people have it backwards with respect to CiV and chess. Chess is the harder game for the reasons I outlined in my earlier post. The difference is the amount of effort that has gone into the AI. Chess has had tens or hundreds of researchers working on it over the years, plus lots and lots of money. CiV? One, maybe two harried AI programmers trying to balance the needs of casual gamers and hardcore warmongers, and probably busy wearing other hats as well. If the community puts in effort on a large scale, I expect we can see a lot of improvement.

 

[Nutteria]

Can I ask something?

Whouldn`t it be wayyy better if the AI has learning mode inherited in to it.As in the game AI files to expand after each game/battle and the next scenario to compare it self to the past.Couldn`t that fix quite fast the "noob" problems it has now?

Say you beat him via excesive use of range units with few meatshields infront for 9 battles.On the 10th the AI evaluates this past exp and goes naval surround on the choke point or mass cavalery?!

I know it sounds way more simple then it really is but if we have the possibility of "sharing" via files/file folders the past exp of the AI we can create quite a comprehansive data pack for a "DL THIS hard to beat AI pack/mod"

 

[dorn]

And think how long that took with all the limitations that exist in a chess game.

Frankly I don't think the tactical combat AI is whats messing up right now though. What's screwed up is the Civ on a global level can't understand that the 5 units invading the border are going to rampage across their entire nation.

It needs to learn to group up units far away then move them towards your army in formation and try and pick points of defense to stop you. It feels like right now the global AI is trying to do some sort of stupid attrition strategy. Instead it should just let you take a few cities then stalemate you and start building up an army to crush you while making allies to do the same.

 

[JP1]

Can I ask something?

Whouldn`t it be wayyy better if the AI has learning mode inherited in to it.As in the game AI files to expand after each game/battle and the next scenario to compare it self to the past.Couldn`t that fix quite fast the "noob" problems it has now?

Say you beat him via excesive use of range units with few meatshields infront for 9 battles.On the 10th the AI evaluates this past exp and goes naval surround on the choke point or mass cavalery?!

I know it sounds way more simple then it really is but if we have the possibility of "sharing" via files/file folders the past exp of the AI we can create quite a comprehansive data pack for a "DL THIS hard to beat AI pack/mod"

It would be nice, of course, but it's often harder than you would think to have to computer draw the correct conclusions from a game... They can often see outcomes, but can't always figure out the causes behind them, especially if you might be dealing with things like, "Oh, he just tactically out-maneuvered me this turn" versus, "Oh, I chose the wrong tech path 50 turns ago and don't have the counter-unit I need."

And if we're discussing combining files from many people, noise and conflicts might be a big issue.

 

[Stalker0]

Perhaps even some changes in prioritization could take the AI up a notch.

1) Prioritize pillaging. Right now the AI doesn't pillage very much, especially the barbs. Pillaging allows the AI to use its initial big army while its still got it. It may have problems taking the cities, but at least it does an initial hit to the economy that helps level the playing field when it loses its army.

2) Keep the workers safe. There is a priority value in teh xmls for protecting workers and its very low. In Civ IV the AI would send all workers back to the cities when war happened. It was exploitable, but better than picking up free workers. If Civ V did the same it would help limit the issue.

3) Up the priority of damaging cities compared to capturing them. Capturing cities has a high priority, but damaging them is very low. This explains why sometimes the AI just circles a city and doesn't attack it.

 

[The Great Apple]

I do find it very strange when the AI runs around in city bombard range not really doing anything. It needs to wait outside unless it's going to either attack or pillage.

I agree with all of Stalker0's points, and if they all can be done in XML this might a significant difference. My other "simple" change suggestion would be to tell the AI to avoid dangerous water while at war so that it stops throwing away embarked units, though this probably wouldn't be at the XML level.

RE: Cleverness in the AI. I imagine they've done it in a similar way to Civ IV's AI, whereby units have aims, are sorted by fitness, then moved fairly individually to acomplish those aims (often resulting in a big stack!). I guess there's probably some sort of formation thing in there, but it doesn't seem to be working. I find the idea of a "tactics engine" quite interesting and potentially something that could work.

 

[drachasor]

I do find it very strange when the AI runs around in city bombard range not really doing anything. It needs to wait outside unless it's going to either attack or pillage.

I agree with all of Stalker0's points, and if they all can be done in XML this might a significant difference. My other "simple" change suggestion would be to tell the AI to avoid dangerous water while at war so that it stops throwing away embarked units, though this probably wouldn't be at the XML level.

RE: Cleverness in the AI. I imagine they've done it in a similar way to Civ IV's AI, whereby units have aims, are sorted by fitness, then moved fairly individually to acomplish those aims (often resulting in a big stack!). I guess there's probably some sort of formation thing in there, but it doesn't seem to be working. I find the idea of a "tactics engine" quite interesting and potentially something that could work.

It's not like Civ IV's AI was really that good either. Civ AI has always been awful...it is just more awful than usual this time around. Hopefully they get it to be less incredibly stupid though.

Frankly, they do game mechanics well and the art is usually pretty good. The rest of the game usually isn't that great at all (the engine and AI always have problems and are overly resource-hungry).

 

[slowjames]

This video might be interesting for those of you wondering about the Civilization AI.

http://www.youtube.com/watch?v=IJcuQQ1eWWI

 

[Monsterzuma]

A major difficulty in creating a chess-like AI engine for combat situations in Civ5 is the fact that not all enemy units are ever in sight. The part of the battlefield that is every visible to the AI is a tiny portion of everything that is going on, even on a single battle basis.

There is also the fact that there is a major element of randomness even in the observed parts of the game.

It's not like Civ IV's AI was really that good either. Civ AI has always been awful...it is just more awful than usual this time around. Hopefully they get it to be less incredibly stupid though.

They've mostly just set the bar extremely high for themselves this time.

 

[MeatUnit2]

There are a few things I would consider when “revamping” the Civ V combat system.

One would be the addition of some sort of unit morale factor.

Units that are isolated, fighting against great odds, that have taken heavy damage, or whose home empire is deeply unhappy, would be subjected to a random number test against a table with possible negative results, the severity and possibility of which would depend on the severity of the conditions. Negative results such as refusing to advance/attack, refusing to move/mobilize, retreating, disbanding or becoming “broken” until rallied, surrendering. I don't believe this would be a difficult add on.

The complement to this would be to increase the number of generals or officer units who would be able to bolster the morale of the units in their immediate vicinity. Since Great Generals are already part of the game I don't think this would be a difficult add on (it's a related/similar idea). Officer units would have a “bonus” value assigned to them, some might even have negative values, but without the proximity of an officer, land units would not engage in offensive combat. A player would see a real return on investment from his military academy buildings if they were turning out better officers. The range of influence of an officer unit would increase at higher tech levels. [The AI could “cheat” by having more high quality officers with greater effective range than the human player.]

The Bolshevik factor. If your empire's unhappiness gets too bad, your units begin to test against a morale table whether or not they are being employed in combat. Results could range from units becoming slower, or immobilized until happiness conditions improve, to units killing their officer units and moving at full speed back toward your capitol. Farm and trade improvements in your territory burning, two headed goats being born. That sort of thing. [Human players would really have to watch unhappiness while the AI maybe not so much.]

Another thing I would consider would be the addition of wild card or random events associated with combat. This may not sound particularly fun (I know that in Civ IV having a couple of subs loaded with nuclear missiles disappear in the Bermuda Triangle wasn't fun). I am suggesting that there be the possibility (a small possibility) in combat for a random event to occur. And I would make a long list of small random events, such as the nearest forest tile catching on fire and cannot be occupied, or the city being attacked is abandoned and is set on fire (just as if it were being razed) or the nearest trading post is destroyed, units retreat rather than attack. That sort of thing. No battle plan survives contact with the enemy. [Again, this can be weighted in the AI's favor.]

Some other ideas:
While at war, land units in enemy territory incur a point of damage per turn. (Attrition due to the hostile natives, IUD's, lack of home cooking.)

In the industrial age you can build a landmine unit that is moved into position and expended/used up to lay down a mine field (oh how the AI could make invasions horrible with something like that). All land units would have to stop on the mine field when traversing it, and enemy units would take some damage, before moving on in the next turn. Combined with the ranged units in the game this could be a very strong defense. Conversely, a landmine unit could move onto an existing mine field and expend itself to remove the hazard. So a landmine unit could expend itself either laying down a minefield or removing an existing mine field. Perhaps there could be a similar water based unit that could mine or clear mines from coastal tiles. You would have to tinker with unit cost and the amount of turns it would take to lay mines/remove mines to find a good balance.

Most of these ideas would make aggression more difficult for a human player and give the mostly defensive AI more of a fighting chance against human player attacks. They may or may not have any merit, but I'm throwing them out there for your consideration.

 

[gingerbill]

This isn't necessarily true. There's rarely a front line which spans 64 hexes with 16 pieces, some of those pieces able to move in all directions up to seven squares. A lot of basic AI failings occur with very few land units in a small area of land.

yes but the CIV AI has a billion other things to think about.

 

[Jimbo30]

One of the biggest problems right now is the human player can use the game mechanics far too easily.

Great Generals are a free 25% bonus to the human player, simple as that. The -33% combat penalty in open terrain needs to be removed as well. Those two changes alone would help the AI immensely.