Technical commentary on Civ/game AI

[nzcamel]

Maybe. A developer or artist costs about $180k/year (depending on overhead), 10 is a very small team and is 2 million being burned year after year. I have a small game company (consisting of three people) and I doubt I'd throw $400k/year at at AI if I was the size of Firaxis - one dedicated developer is generous IMO.

I appreciate that Civ Fanatics is not representative of the average player; but given how much they've made off VI, I think they can afford to put a bit more into AI. It wasn't a surprise that it's done well, so they can budget for that. And reviews are a bit mixed at the mo...

I gotta say...is real time strategy really harder for AI? Or is it that the limitations of what the human can see and do at any time compared to the AI (which can "see" and move all it's units at once) which gives it a different kind of advantage?
Cos the issue with turn based games (esp SP) is that the human can take their time to optimise the best move or choice in every case. In real time strategy we don't have that luxury. I'd suggest that real time AI gets made to look better by the very nature of putting time pressure on humans.

Yup

And I think I worded what I said badly - Even if real time is harder for the AI (makes sense)...it's actually even harder again for humans. Limited size screen, limited ability to make good decisions quickly. So my point was that it's kind of unfair to compare turn based to real time; as I think that real time AI has a leveling advantage that turn based AI does not. They can get away with more mistakes that the human will never even notice.
Every game in Civ we all can (and some do) scrutinise every single move the AI makes that is visible. That just doesn't happen in real time games to the same degree.

 

[steveg700]

Oh, so there's nothing a programmer to prevent an AI civ from rolling a single artillery unit up to a well-defended city as a means of conducting war? It's just "weather" that can't be corrected through programming? It's just uncontrollable "weather" that has the AI not able to distinguish between land and water?

No doubt that's some of the "excellent" and "smart" AI behavior that so frequently impresses Xmonger.

Well then, so much for 1UPT.

 

[Zuizgond]

I work in AI and have been in the field for many decades. [...] Having said all of this in my tiny opinion the Civ AI is excellent, I'm continually surprised at the smart moves the AI's are making.

If you think Civ6 AI is excellent, I wouldn't hire you as AI developer.

 

[Deggial]

Hi Xmonger, I find it interesting that these deep-neuronal-network-AI threads keep popping up in the forum now. I started one myself not so long ago and you might find a read interesting. I certainly did, even if I didn't partipate much after my initial OP.

Anyway, I don't think that learning by analyzing human-played games will be the solution. Some hundreds will be not be enough anyway and all those humans mistakes while playing won't be helpful either. AlphaGoZERO was tought just the very basic Go rules, played millios of games against itself and finally easily defeated its precursor (AlphaGo, which learned by analyzing thousands of professional Go parties!)

 

[Siesta Guru]

I gotta say...is real time strategy really harder for AI? Or is it that the limitations of what the human can see and do at any time compared to the AI (which can "see" and move all it's units at once) which gives it a different kind of advantage?
Cos the issue with turn based games (esp SP) is that the human can take their time to optimise the best move or choice in every case. In real time strategy we don't have that luxury. I'd suggest that real time AI gets made to look better by the very nature of putting time pressure on humans.

It's an 'it depends' kind of thing, which is why I didn't name real time as one of the reasons AOE might be harder than Civ for bots.
Real time based AIs are harder to build and put harsher time limits on it, which makes real time bots less competent. But indeed, humans are also less competent in real time, so it's going to depend heavily on the type of game.
Regardless of the answer though, the competency of the AOE AI feels higher for reasons unrelated to the time controls. The AOE bot frequently manages to be surprising, makes relatively few blunders, doesn't do stuff like having half of its units stand around and engages with all aspects of the game. None of these are made easier because of the different time system.

Maybe. A developer or artist costs about $180k/year (depending on overhead), 10 is a very small team and is 2 million being burned year after year. I have a small game company (consisting of three people) and I doubt I'd throw $400k/year at at AI if I was the size of Firaxis - one dedicated developer is generous IMO.

Firaxis seems to have 100+ people working on these games. That includes a lot of other non-dev/artist stuff obviously, but to have essentially less than 1 person on AI (the ai dev seems to do some other things too) when it's such a core part of the gameplay experience, it's just rather minimal and makes it little of a surprise that there are so many complaints about this. It's a large game and even just hooking every functionality up to the AI decision process will take a lot of time

 

[Xmonger]

Non capturable settlers is just a nightmare, see the opposition settler walk around my city and settle on the other side? Its one thing to have a great musician wondering around but its quite another to have a unit as subtly devastating as a settler. You may not think it so but if the civ also has a few thousand gold then its another story.

What are the rules around capturing Settlers? From a reddit thread it appears that if you have a support unit with you (general, balloon etc) then the Settler is destroyed, not captured.

Anyway, I don't think that learning by analyzing human-played games will be the solution.

Yes I believe I said that, the easiest way to generate data is from existing AI-AI games. I didn't read the paper on AlphaGo Zero, but as far as I understand they seeded the initial weights differently for the two players, giving two different players of equal ability (at least this is how they must have done it). So the choices the AI's are making is initially random, then it learns from the cost function what works and what doesn't. This is different from Civ in that it has an existing AI which can be learned from. But it still gets into the issue (I haven't spent time thinking through) of the network size, need to think about the feature set size vis-a-vis Go.

Firaxis seems to have 100+ people working on these games. That includes a lot of other non-dev/artist stuff obviously, but to have essentially less than 1 person on AI (the ai dev seems to do some other things too) when it's such a core part of the gameplay experience, it's just rather minimal and makes it little of a surprise that there are so many complaints about this. It's a large game and even just hooking every functionality up to the AI decision process will take a lot of time

Firaxis also has other titles. I won't argue it any more but head count is always an issue at every software company in existence, deciding a company isn't dedicating enough people on X without knowing the details of their resource allocation is a waste of time.

 

[Victoria]

What are the rules around capturing Settlers? From a reddit thread it appears that if you have a support unit with you (general, balloon etc) then the Settler is destroyed, not captured.

Well I have never captured a settler while holding a loon or a Generals hand. both activities are beyond what i would consider normal play so I could not comment on them.
I do know that they are normally captured if you kill their escort. Thats the main thing. i can test the other if you really want, just seems bizarre to capture it wile having a loon and a general jut needs to be near, not on.

What has replaced capturing settlers for me is capturing CS. This is the most weird thing but it just works. Walk around a CS with your starting warrior/scout and eventually a civ will attack it. There is normally one turn where it is practically zero health and they do not bother to take it that turn... bam you steal it off them. Practically every game I try it and it has only failed once.

 

[nzcamel]

It's an 'it depends' kind of thing, which is why I didn't name real time as one of the reasons AOE might be harder than Civ for bots.
Real time based AIs are harder to build and put harsher time limits on it, which makes real time bots less competent. But indeed, humans are also less competent in real time, so it's going to depend heavily on the type of game.
Regardless of the answer though, the competency of the AOE AI feels higher for reasons unrelated to the time controls. The AOE bot frequently manages to be surprising, makes relatively few blunders, doesn't do stuff like having half of its units stand around and engages with all aspects of the game. None of these are made easier because of the different time system.

There must be a closer fairer comparison that can be made in turn based games. It says a lot if there isn't.

 

[Karpius]

We are probably ten years away from a gaming AI that people around here might consider adequate. Possibly longer.

In the meantime, I would have to ask (because I just don't know - Civ6 is the only game I play) is there another strategy game as complex as Civ6 that has a better AI? Does it involve as many units, as many players, as many conditions and moves per turn?

The example @Victoria mentioned above was based on logs I sent her. Just that one example opened my eyes to just how much the AI is doing behind the scenes (as it should) that the player is generally unaware of. Here's the scenario....
Spain is the last civ on my continent. I don't want to declare war, so I hit Spain with spies, park units at the border, convert cities to my religion and never agree to any promises. I have no issues with the other civs in the game and even have friendly (not declared) relations with Germany. Suddenly, Spain and Germany go for a joint war, so I asked Victoria to put her admirable skills to work combing the logs. In all the time I was poking at Spain, Spain was trying desperately to get someone to join a war against me. Now isn't that relatively smart AI? Oh sure, from the human perspective it seemed odd that Germany declared war, but since it was not a declared friendship, certainly within the bounds of the game. Also, from a realistic point of view, friends have turned on friends throughout history.

I know the AI sometimes makes non-optimal moves. Sometimes it does downright stupid things. But I would wager further study of the logs might reveal a linear progression to that action that makes it slightly more understanding....despite it ultimately being dumb. The point is, the AI has thousands of computations to make. It must review each rule for each civ and unit and other mechanic each turn. The results might not always be optimal to the human eye, but were the only way for the civ to go given everything else.

I had one game where Gitarja built quite the air force and used it quite correctly. Bomers made their runs before land units attacked and fighters were deployed to defend as they should. So that tells me that the AI somehow *knows* what to do with air units. In another game, however, Gitarja built an air force that she did little with beyond ferrying units back and forth. What was the difference? I don't know and now I wish I had the logs to compare, but something tells me the progression of the game itself drove Gitarja into that mode. I have seen similar situations in other games where sometimes the enemy was smarter than other times. But mostly, I see a middle-of-the-road level of tactics and strategy which means there will likely be extremes on either end of the spectrum.

I get it, we all want better AI. We want the AI to be smart each and every time. We don't want to see the dumbness that can happen. It doesn't hurt that we keep poking Firaxis to address this, but I don't think anyone should assume Firaxis doesn't want to. I don't believe they are being lazy or cheap or uncaring.

But going back to my question at the top..please point me to a more complex AI.

 

[Siesta Guru]

What are the rules around capturing Settlers? From a reddit thread it appears that if you have a support unit with you (general, balloon etc) then the Settler is destroyed, not captured.

This is unfortunately correct as I found out in my last game.

Yes I believe I said that, the easiest way to generate data is from existing AI-AI games. I didn't read the paper on AlphaGo Zero, but as far as I understand they seeded the initial weights differently for the two players, giving two different players of equal ability (at least this is how they must have done it). So the choices the AI's are making is initially random, then it learns from the cost function what works and what doesn't. This is different from Civ in that it has an existing AI which can be learned from. But it still gets into the issue (I haven't spent time thinking through) of the network size, need to think about the feature set size vis-a-vis Go.

Yes the easiest data generation methods are either based on AI vs AI games, and through analyzing human replays. In the latter case, you would reward/punish the AI based on how well it does at mimicking good human players in known game situations with known player actions.
Alphago seems to have started with two identical players, then used the learned data in these games to have the new version play against the old version. After achieving some amount of improved, they set this newly trained version as the new base enemy to play against.

 

[Victoria]

If you are wondering, when you next get a joint war look in the AI_diplomacy.csv file (you’ll need player-stats.csv for the civ number) and you can normally see one of the joint war civs trying to get a joint war deal for a while... if you are nice to your friends.

 

[Siesta Guru]

There must be a closer fairer comparison that can be made in turn based games. It says a lot if there isn't.

There probably is, but I don't play many games.

The closest I know of is the battlecode 2018 competition I just competed in, a small competition for amateur AI enthusiasts that lasted 3 weeks. The game there was a turn based game played on a tile based map, it has several unit types with a combat system somwhat similar to civ, a tech system, different maps and so on. My entry, as well as the entries of most competitors I played against, seemed significantly stronger than the civ AI does based on watching AI vs AI games. The game is much smaller than civ, and you can't play it as a human, so its not an entirely fair comparison either, but you're not going to find a perfect comparison anywhere to begin with. The reality was just that bots made significantly fewer obvious errors and were capable of both good macro and micro behavior that Civs AI doesn't even remotely get close to.

 

[historix69]

A lot of people here complain about AI.

If you do a multiplayer game with 20 opponents, 10 AI and 10 random human players, no communication, is it be possible to exactly identify all 10 AI players solely by how they play (play-style)? (It would be kind of Turing-Test for Game-AI.)

 

[Japper007]

There must be a closer fairer comparison that can be made in turn based games. It says a lot if there isn't.

We are probably ten years away from a gaming AI that people around here might consider adequate. Possibly longer.

In the meantime, I would have to ask (because I just don't know - Civ6 is the only game I play) is there another strategy game as complex as Civ6 that has a better AI? Does it involve as many units, as many players, as many conditions and moves per turn?
(...)
But going back to my question at the top..please point me to a more complex AI.

Pandora: First Contact. It has an extremely complex economy and unit system, far harder and more micro-intensive than civ 6's. It's semi-1UPT (army building style, like the Endless series games), turn-based 4X, I'd say it qualifies.

AI is decent enough that the developers confidently put the "Hard" (second from top difficulty tier) as the equal footing (i.e. no cheats for either Human or AI, "Moderate" actually has to give the human cheats! Imagine that for civ 6 Prince Level!) setting. I can barely beat it and I play most 4X on the highest difficulty level without much sweat.
The most galling fact about it is that it's AI was extremely poor on release, to the point of the title getting reviewbombed, but they actually made it much better by just assigning a full-time AI developer for a year solely for the purpose of fixing a broken game and as free patching, not as part of a DLC programme. And this was an indie dev, not a triple A studio like 2K/Firaxis! It shows you just need to put resources into AI development and they can really bear fruit (title is now overwhelmingly positive on Steam, last I checked).

People have also brought up both Vox Populi for civ 5 and AI+ for 6, both substantial improvements over civ 6 basegame AI. Both made by people as mods and not a paid for job like the AI engineer at Firaxis.

More generally to the topic, I think many flaws in the current AI comes from the developers just not knowing effective strategy for their own game. Let's look at some examples from their streams:

-They do not understand the strenght of chopping/improving tiles and overflow, result: AI practically ignores these systems and most of it's territory is garbage land that is also unimproved. (it also led to ridiculous design like Magnus, but that is another debate than AI)
-They think "Tall" is something that exists in civ 6, result: AI barely expands
-They do not understand how to attack with the 1UPT system, result: AI can be spectacularly outmanouevred even by a newbie.

Developers incompetent at their own game will code an incompetent AI, what a surprise!

 

[Kwami]

I don't think it's fair to say that the developers don't know how to play the game. Whatever they demonstrated on live streams is almost certainly not typical.

 

[Japper007]

A lot of people here complain about AI.

If you do a multiplayer game with 20 opponents, 10 AI and 10 random human players, no communication, is it be possible to exactly identify all 10 AI players solely by how they play (play-style)? (It would be kind of Turing-Test for Game-AI.)

Yes I can. I play a lot of Hotseat games through PYDT client, often the larger games have a few AI's in there to pad out the numbers (we still have way to few active players! Everyone interested in Civ Multiplayer games please install this client and join a few!). On Prince (so equal footing Human vs AI) AI is significantly behind by the 30 turn mark on Standard, hopelessly behind by 60. I don't even have to see the AI in action to notice it, the score is enough to tell the story!

 

[Victoria]

This AI is not designed to win, There are too many throttling techniques it uses.

 

[Kwami]

This AI is not designed to win, There are too many throttling techniques it uses.

Like most other games, Civ VI is designed to have the player win most of the time. The AI is just there to make winning more difficult.

 

[Japper007]

Like most other games, Civ VI is designed to have the player win most of the time. The AI is just there to make winning more difficult.

Which is a challenge it fails at, clearly, or we wouldn't be here, nor would all the other AI discussion threads here. I don't think anyone wants Deep Blue level AI mostly because it is just not fun to be guaranteed to lose. But can't we all agree that AI, A) can be better and, B) a better AI is more fun and challenging to play against?

 

[ShallowSeas]

Allow me to address the elephant in the room. Any improvements in the AI would have to be balanced against the performance costs, so even if it were a simple matter to improve the AI it might still not be worth it. A casual player who finds prince difficulty challenging isn't going to notice better AI, but will certainly notice longer turn times, especially if he or she plays on large maps with many computer players.

Saying that Firaxis can afford to spend more money developing AI is like saying that Bill Gates can afford to give me a million dollars. Both statements are technically true, but they are completely irrelevant.

I think improvements can and should be made to the Civ VI AI, but those improvements would probably limited to the tweaking of certain behaviors. The only way to make a huge improvement to the AI is to make the game simpler.