Technical commentary on Civ/game AI

[nzcamel]

Civ III was a fantastic game, and continues to be a good game. The graphics are dated, but the gameplay is OK, and the AI is good. I liked the good old times when an Empire was something more than 5 cities, and when you where afraid of your neighbours armies. And the amount of mods and scenarios was awesome.

Just don't try to build a city on a distant continent! Oh and enjoy all those farms n mines

 

[PhilBowles]

I'm thinking there should be a saying, "it wouldn't be a Civ forum if every other post wasn't bashing the AI". Warning, long post.

So what about Civ, connectionist sounds pretty good, why not do that here? Well let's see. We'd need about, oh say 100k to 500k full games as training data to start, let's just generate data from AI vs AI games. Then we'd need to define the network, what's the cost function? In other words how do you define the 'value' of a move? Well that's a great question, like what's the numerical value (from 0-100) of making a builder? It needs to be a formula, how would you express that? Be careful and don't turn it into a classical system by putting too much thought into it, if it's complicated it probably won't work. But what about the hundreds of other things you could do at any moment of the game? What's the cost function for that? Finally, what are the input neurons and output? Given the number of orthogonal actions at any point it kind of seems like the network would be bigger than what even our modern CPUs/GPUs could handle. Brrrrr .... I'd love to get some time to look into this, but at first glance it's not an obvious solution.

There was a university team that used Civ V to train a neural net-based AI, as the news articles at the time put it (presumably simplifying) by reading the rules and learning. If Firaxis had the tools to do that following the development of a Civ game I'm not clear why they couldn't use the code of the resulting trained AI in their eventual commercial product.

The simplest solution of all, however, is simply to reverse course and stop making the games too mechanically complex. If they're going to stick with classical AI it's not sufficient to throw their hands up and say "Well, what did you expect?" If they can't design an AI that can play the game as it is, approach the problem from the other direction and design the game in such a way that the AI can play it. Reduce the amount of decision-making but make individual decisions more relevant, and cut down on the number of peripheral systems and excess resources the AI needs to manage.

If one thinks that AI is too weak, first try to play without save/load cheat. Start a game and play it to the end without reloading when something does not work out as planned.

I always play that way. The sole exception was in a recent game when I lost to a religious victory I wasn't expecting (because I didn't think the AI was capable of it) ten turns before I would have won, and then I reloaded because I simply didn't want to play on the same map size (Tiny) again.

The Civ VI AI simply remains too easy. This was on Deity, as were my recent victories without any reloading at all, and there was no need to optimise play in the slightest to succeed, just expand a lot early.

 

[Siesta Guru]

LOL @ self-assessment. Firaxis probably believe their AI is doing quite well too.

Finding 10% (say) of the many tasks where the AI performs poorly is not the same
as getting the other 90% to perform at least satisfactorily and transparently.

With 'fairly well', I meant I'm consistently in the top 1-5% of teams. That's measurably 'fairly well', not some kind of hunch I woke up with lol. There's no indication whatsoever that firaxis tried to measure their AIs performance in some kind of objective fashion. Of course there are no alternative AIs to compete against to begin with, but the provided debugging tools don't even seem to allow playing multiple versions against each other, let alone batch game runs, both of which are really important when it comes to measuring AI performance. Looking at games and human play is important too, but considering how bloody long games take in civ, it's just not all that viable of a method.
Imagine AlphaGo having to determine its power only through playing long games of go against a mid-tier go player. You'd get nowhere.

Anyway, first testing shows that the tip from @Gedemon is basically the holy grail I've been looking for, so it's time for me to stop complaining and to start getting to work It may be possible to get close to modded Civ 5 AI with the help of these functions without even the need for a dll release. (It'll take some time though, I'm going to have to entirely redo some sections of the AI to unlock the full potential)

 

[LukaSlovenia29]

Some of the mods for civ games, such as the Vox Populi, are compelling proof that the Civ AI could be and could have been immensely better by Firaxis with relatively modest financial resources, which would repay themselves many times over due to increased sales.

And for those people/customers that wish to have a very easy game without being bothered by the AI, Firaxis could still have a very simple settler/chieftain difficulty setting with loads of bonuses to the human player and with purposefully strong inclination towards sub-optimal decisions by the AI.

 

[Gedemon]

@Siesta Guru That's great news

Speaking of comparaisons, do you think it would be possible for you to assign different (parts of) AIs to different players if you have to start from scratch?

Or would it be too much work?

That's really something I'd like to be able to do, as well as activating/deactivating some parts (thinking of total conversion mods)

 

[Siesta Guru]

@Siesta Guru That's great news

Still trying to work my way around some issues, but it looks like individual unit control may actually be partially possible using this too by just always applying unique operations to every unit, so there's a lot of potential. I've also already confirmed settling behavior can be entirely overruled this way too

Speaking of comparaisons, do you think it would be possible for you to assign different (parts of) AIs to different players if you have to start from scratch?

Kind of. I'll at least be able to have different players execute different parts of my mod based on lua logic, that'll be as easy as executing different methods based on civ id/name. However, I don't think I'll be able to have it play against the base AI, because I need to do some filthy XML changes to be able to turn the base AI off.
Not sure whether it'll be possible to make it easily adjustable for players though. Ideally players would be able to select an AI style for every opponent, but overall game setting options would be cool too. I'll definitely do that if possible to at least allow for a difference in a competitive and a roleplaying AI.

But yeah, that's for later, civ lua development/debugging is rather slow so even just the basics will take a while, do you happen to know some tricks to speed it up?

Still undecided whether I should allow my code to use vision cheats. I'm really tempted because vanilla uses such cheats extensively too, and it just delivers better results and is much easier to make. However it feels unfair and takes away some vision based strategic options... Hard choices

 

[LukaSlovenia29]

Siesta Guru, I think Vox Populi uses the system that throughout a given turn, the tiles revealed by a unit remain fully visible. For example a horseman starts a turn on top of a hill, seeing lots of tiles, and then makes a move into a forest. The player/the AI (to whomever the unit belongs) will still until the end of the turn see all the tiles that were revealed by units throughout the turn (not just revealed at the beginning of the turns, but throughout the turn). I think the feedback/results were that this helped the AI a lot to "have a memory" of the position of the units. You could perhaps extend this to the visibility of those tiles not just for that turn, but for the next turn or the next (xyz) turns (to represent sentries, reconnaissance missions by individual soldiers etc.).

 

[Kwami]

There's no indication whatsoever that firaxis tried to measure their AIs performance in some kind of objective fashion. Of course there are no alternative AIs to compete against to begin with, but the provided debugging tools don't even seem to allow playing multiple versions against each other, let alone batch game runs, both of which are really important when it comes to measuring AI performance.

I'd like to think that the developers have a large number of development tools that they haven't made available to us. They certainly run AI test games all the time and they've stated as much before.

Some of the mods for civ games, such as the Vox Populi, are compelling proof that the Civ AI could be and could have been immensely better by Firaxis with relatively modest financial resources, which would repay themselves many times over due to increased sales.

1. How are the turn times with VP vs. not?

2. Do you have any evidence that better AI would increase sales in any meaningful way? I suspect that it would not. For the vast majority of players, the lackluster AI is just fine.

 

[Siesta Guru]

I'd like to think that the developers have a large number of development tools that they haven't made available to us. They certainly run AI test games all the time and they've stated as much before.

They seem to use the firetuner for that, which they've made available for modders.

Do you have any evidence that better AI would increase sales in any meaningful way? I suspect that it would not. For the vast majority of players, the lackluster AI is just fine.

Finding hard evidence for this is impossible, but there's anecdotal evidence enough (I've not bought beyond earth for AI reasons). The game loses replayability so less people buy DLC. Then there's also the part where those who care most about the game are both the ones most likely to leave reviews and make recommendations, and they happen to be the ones affected most by the AI. Just have a look at any place that people would look at when determining whether to purchase the game. Steam reviews, these forums, reddit, all of them are filled with people regularly complaining about it.

 

[Xmonger]

Which is a challenge it fails at, clearly, or we wouldn't be here, nor would all the other AI discussion threads here.

Complaining posts aren't representative because most people won't go on a forum, and they certainly won't go on if they're happy. Especially 'fanatics, there are what, 500k+ copies of Civ VI sold (no idea, is it 1M+?) and a few hundred complaints on boards about the AI? Doesn't tell you much.

 

[Kwami]

Complaining posts aren't representative because most people won't go on a forum, and they certainly won't go on if they're happy. Especially 'fanatics, there are what, 500k+ copies of Civ VI sold (no idea, is it 1M+?) and a few hundred complaints on boards about the AI? Doesn't tell you much.

From Wikipedia:

"By May 2017, the game had sold more than 2 million copies, contributing significantly to publisher Take Two's 2017 financial year which had reported revenues of $576.1 million. Take Two stated that Civilization VI was on track to surpass Civilization V's lifetime sales of 8 million.[37]"

Finding hard evidence for this is impossible, but there's anecdotal evidence enough (I've not bought beyond earth for AI reasons). The game loses replayability so less people buy DLC. Then there's also the part where those who care most about the game are both the ones most likely to leave reviews and make recommendations, and they happen to be the ones affected most by the AI. Just have a look at any place that people would look at when determining whether to purchase the game. Steam reviews, these forums, reddit, all of them are filled with people regularly complaining about it.

The question isn't whether a better AI would generate any sales at all. Rather, would a better AI generate enough sales to compensate for the additional development costs? And, would those resources be better spent elsewhere?

We can be super conservative and pretend that you can hire a good AI programmer for $160k/year, inclusive of salary and all benefits. Let's say that he works on the game for two years, which costs the company about $320k. If the studio gets $40 per game (and they almost certainly do not!), then they would have to sell an additional 8.000 copies to make up for the cost of the additional AI developer. Are there 8,000 lost sales because of poor AI? Maybe. It's hard to say for sure.

But what else could $320k buy you? Maybe that's 10 more civilizations and leaders, or a couple of new major features in the next expansion. Those kinds of things are likely to generate far more sales than would better AI.

 

[Japper007]

Complaining posts aren't representative because most people won't go on a forum, and they certainly won't go on if they're happy. Especially 'fanatics, there are what, 500k+ copies of Civ VI sold (no idea, is it 1M+?) and a few hundred complaints on boards about the AI? Doesn't tell you much.

Sure but the game also sits on "Mixed" on Steam currently, with only about 60% positive reviews. I skimmed through the top 100 "marked helpfull" reviews and most of them (both the negative and positive, amusingly) come down to this in aggregate: good mechanics, bad AI. With the positives saying that they recommend it with the caveat of bad AI, and the negative ones often being flat out: can't recommend the Single player experience purely because of the AI, while mechanics are great... If half your user base is negative towards a title because of your AI you are doing something wrong. Even civ 5 at launch didn't have half the complaints on AI that this one has.

There a ~50.000 reviews on Steam currently, so about 10% of that salesnumber which you pulled from somewhere smelly.

Now I'm not negative towards civ 6, I have about 500 hours in it according to Steam and I'm not nearly done with the game, but that is mostly because I have rediscovered the joy of multiplayer. I used to be a fanatical AoE2 and Rise of Nations multiplayer guy back in the day but I lost interest and my competitive edge. I only played a little MP in previous civ-titles, and even that at a very casual level with a group of friends. Now I picked up multiplayer again solely because of the atrocious AI. Before that the game sat collecting dust in my Steam library, only breaking it out briefly after every DLC with the same result: win Deity like a cakewalk with the new civ, then quit because there was no challenge. In a multiplayer environment this game is positively epic though, well designed and with lots of different strategies to try and no rubber-banding bull like BNW's 5% tech penalty for people who dare to play the game like it's a 4X.

 

[Bad Wolf]

Are there 8,000 lost sales because of poor AI?

Out of a pool of 2 million? I'd bet at least 8000 chose to forgo at least the DLCs because they gave up on the game because of poor AI.

I also just don't understand a lot of the logic in this thread. The technical arguments are beyond me, but... if you could have a better AI, and it were totally possible and cost-effective (which it probably would be, considering modders seem able to do it in their spare time), why wouldn't you want that?

Also, just imagine if the enemies in a first person shooter ran around shooting at walls or something, even on the hardest difficulty - sure they take 10 shots to kill on the hardest difficulty instead of 1, but they're still just running around shooting at walls. You think there wouldn't be a bunch of people RIGHTLY complaining about this and demanding it be fixed in a patch, so that the game they spent $80 on is no longer broken? You think it wouldn't affect sales numbers?

 

[LukaSlovenia29]

@Kwami ,

I myself haven't bought Beyond Earth because of the overwhelming reviews about the bad AI. I myself would much more strongly recommend Civ V to my friends/other people now that the VP mod has transformed it into the best strategy game I've every played. I would include LOTS OF WARNINGS for someone considering just BNW Civ V, if there weren't so many great mods for it. It's the same with Civ VI -> as it stands now, I would be very wary in recommending it. Perhaps once they release the full material needed for the mods and once the mods have greatly improved the game, it will be much more worthy of a recommendation. I'd be much more inclined to ASAP buy (and preorder) all of Firaxis Civ content if I were confident in the AI performance.

I imagine all of this, multiplied many times across the world with other people critical of Firaxis' AI development, could cause a not-so-small change in the number of sales.

 

[Bad Wolf]

How many people base their decision about whether or not to buy a game on the reviews? Particularly the Steam reviews, if they're buying it on Steam? I'm sure it's not insubstantial. So yeah I have no doubt that the crappy AI has cost them sales.

 

[isau]

Again I'm not super experienced with the AI in Civ 6. I am a little bit with Civ 4, from modding the C++ files back in the day and adding AI support for some new features, but still far from an expert.

Basically the issue with Civ 6 AI from my perspective appears to be overthinking. Units are given weights and decision trees, leaders have random agendas etc etc. It's all very complex. Really I think this is an area that would benefit from a top-down approach. Stop thinking of it as "creating an AI player that reacts as a human would." That isn't even the real goal. IMO you have to think of the AI player as an obstacle in the player's path, like any enemy to avoid or projectile to dodge in an action game.

From that perspective, IMO you'd achieve a reasonable opponent by just teaching the AI to always try to build air units when it can, and to use them when something is in range. Don't think about it as nodes on a decision tree or anything complex. Just fill the level with opposition. It really does not need to be very complicated. I feel like a dedicated 40 hours of work could easily fix the major issues with these systems.

An AI that consists of complex node decision making but yet can't make simple decisions like "Bomb the enemy player" IMO widely misses the mark. There are some things about a game like Civ that might require complex AI but IMO this really isn't one of them, any more than it is a complex AI task to place enemies in a level of Super Mario. That's really all that most players want.

 

[Bad Wolf]

Yeah, I'm just getting tired of seeing the AI players build aerodromes and then not build planes. Like... why????

 

[ChocolateShake]

There are a few loose ends to the AI that need to be tied in order to make them more competitive. The most glaring issue for the combat AI in my opinion is the tendency to be vulnerable while on the move. What I'd like to see is an AI that encourages me to engage heavily in diplomacy in order to stay safe.

Right now I use @Siesta Guru's excellent AI+ mod, in my recent games I've seen the AI take cities and Nubia had actually taken out 2 civs. I know the vanilla AI has been improved and can be more of a threat now, but I'm sure AI+ is doing a good job enhancing it.

I think the main criticism of Civ 6 is the AI, and for that reason I think Firaxis has to make upgrading it more of a priority. The vast majority of players probably don't play frequently enough to notice, but I'm sure it's affecting sales and causing some people not to buy DLC.

I personally am enjoying the game because I play domination games very rarely, and because the AI is enough of a threat to me as my army is usually small. Firaxis can and has improved the AI, and I think by the time of the second expansion it'll be in much better shape. From a technical standpoint, having a better AI is definitely doable. In Civ 4 I remember the AI would sometimes split its army into two and feign attacks on one front while marching on another. It wasn't a tactical genius or anything, but that kind of behavior did a good job of hiding its mistakes.

 

[Kwami]

I also just don't understand a lot of the logic in this thread. The technical arguments are beyond me, but... if you could have a better AI, and it were totally possible and cost-effective (which it probably would be, considering modders seem able to do it in their spare time), why wouldn't you want that?

I don't think anyone wouldn't want a better AI if it was cost-effective and technically possible and didn't increase turn times much and... But that's a big if, isn't it?

I guess there are basically three possibilities here.

1. Firaxis is incompetent or mean or whatever and refuses to make better AI because they just don't want to.
2. Firaxis has crunched the numbers and come to the conclusion that other features are more important to future sales.
3. There are enough technical limitations to prevent an AI that people on this board won't complain about it.

Of them, I'd say that #2 and #3 are much more likely than #1.

 

[historix69]

I remember turn based strategy games in the early 90s taking 30 minutes for one AI turn on a 80486 PC. (Those games usually started with ca. 100 enemy units and once the player had destroyed the mass of initial units, AI turns became faster.)

AI in a Civ 6 game has to manage up to 62 civs.
AI has to deal with longterm strategies like expanding, preparing for war, winning the game.
AI has to deal with shorttime affairs like reacting on opportunities (e.g. capture a settler) and other players' actions (e.g. replace a destroyed unit, defend against an attack.)

Nobody complains when a human player in a single player game takes 30 min to rethink his strategy and optimize everything during his turn. If there are 20 civs on the map and each civ would need 30 min to optimize, a turn would take around 10 hours. The consensus is that AI turns shall not take forever and best be done in a few seconds, e.g. if there are 20 civs on the map and total AI turn time shall be around 1 min, this leaves ca. 3 sec per individual AI turn. One way to keep AI turns short is to use a lot of longterm orders and not rethink the global strategy every turn but only from time to time. Longterm orders might conflict with shorttime necessities like taking opportunities or reacting on emergencies. How much computing time would the average player allow for AI turns to not get bored (especially when maxing out number of Civs/AIs)?

When watching a full AI self-running game (auto game), how obvious is AI failure?