AI beats top human player. Yet another AI discussion

[EpicWestern]

You can always have different difficulty settings for the AI.

A tough AI should be a top priority for the civ franchise. Long term goal would be that the AI is tough enough that bonuses aren't necessary. Strategy discussion would deal simply with how to win the game as opposed to how to do it efficiently. Anyway, even with different difficulties, we don't really need to worry about the AI being too tough just since we're so far away from that happening, probably even more than 10 years away.

 

[TheMeInTeam]

I don't want the AI to crush me every time, but yes, I would appreciate an AI that simply "didn't make mistakes". It also sucks the level of bonuses they get - I mean, when they start with tons of units and multiple settlers on higher levels, it's not rare for an AI to potentially have 4 cities in place before I get my 2nd (2 from their initial settlers, 1 captured from a city-state, and then 1 settler on their own). I'd rather they get more bonuses through the game as opposed to so many bonuses early on, since it can be frustrating starting a new game and seeing them jump so far in front.

If the AI didn't make mistakes, you would need to get very lucky to beat it.

You do make mistakes in the game just like every human player, after all.

Any top shelf AI would certainly have randomizing elements (because some features of the game are random) and it would take hours upon hours of careful observation just to be certain you even knew how it played.

We don't have to operate like neural networks or something. You actually CAN look at a set of actions by a winning faction and make estimations of why each action contributed or did not contribute to victory. The better you are at the game, the more accurate these estimations will be consistent to future anticipated experience.

If the AI played perfectly you could just turn the difficulty down. It's a silly thing to worry about, until we're nearing singularity/sentient AI.

But come on. In this game? The AI is programmed to game-throw, BY DESIGN.

 

[UWHabs]

If the AI didn't make mistakes, you would need to get very lucky to beat it.

You do make mistakes in the game just like every human player, after all.



We don't have to operate like neural networks or something. You actually CAN look at a set of actions by a winning faction and make estimations of why each action contributed or did not contribute to victory. The better you are at the game, the more accurate these estimations will be consistent to future anticipated experience.

If the AI played perfectly you could just turn the difficulty down. It's a silly thing to worry about, until we're nearing singularity/sentient AI.

But come on. In this game? The AI is programmed to game-throw, BY DESIGN.

I'm not talking about "not making mistakes" down to minute details. At least they've fixed the bugs in 5 where they would keep rotating their siege units into the water where they couldn't fire. I'm talking more about just making sure that if I'm in the middle of invading the AI, maybe they don't continue to work on Huey Teocali on the only lake tile in their empire. Or building Petra on the only desert tile so that it literally has no use to them. Or build a settler, but don't settle a city with them because... I don't know why. Or even something simple like I had one game where I literally blocked their path with units, so they started moving away, and then I opened up the path, so they would move back. And I successfully did that enough for me to get my settler to the spot I wanted to build my city.

They are getting better, and just the nature of the game, humans will always have a certain degree of edge. But I would like the AI to get a little more to the point where they don't need ludicrously stupid advantages at the higher levels to be able to beat a human.

 

[Magma_Dragoon]

I would prefer an AI that made no mistakes on purpose by being lawful stupid in adhering to it's agenda.
I don't like that Qin builds wonders that don't benefit him just because, and will put Petra in a city with no desert but the tile it's built on, or build the Colosseum on the coast where it is marginally effective
I don't like that Victoria settles one tile islands with no seafood because it's another continent when there are plenty of better options available
I don't like that Montezuma will waste a district slot in every city on entertainment complexes because muh UB

 

[Karmah]

We don't have to operate like neural networks or something. You actually CAN look at a set of actions by a winning faction and make estimations of why each action contributed or did not contribute to victory. The better you are at the game, the more accurate these estimations will be consistent to future anticipated experience.

And arguably, you just describe a deep learning network (neural networks) in a nutshell
Optimizating estimation is the just the essence.

 

[TheMeInTeam]

And arguably, you just describe a deep learning network (neural networks) in a nutshell
Optimizating estimation is the just the essence.

These estimations usually take an enormous number of iterations to even kind-of approach the desired result. I've seen some interesting videos of someone doing it in SNES Mario World on YouTube.

The goal of that game and its interactions are much simpler than civ, and that's before such a thing would have to deal with someone who adjusts for it once it's pretty deep down a branched path towards its approach. Human beings have heuristics and can adjust for this more robustly, to put it mildly.

That's not to say such will be true in 30-40 years, it might not be any longer, but right now the goal is still more along the lines of "make AI quasi-functional".

 

[Acken]

Some of the AI tasks could easily use such models. It would make the AI always pick the best options which may not be fun in an unbalanced game but technically speaking I don't see an issue for say choosing production, placing cities, policies, techs etc. Fighting and moving units is a way harder task but you could begin by at least make it learn through its own parameters instead of having it hardcoded.

The thing though is that if your AI is not capable of making an attack or puts city on the other side of the map I'm not sure you have the team to begin that kind of forefront work or just the opportunity to take the risk.

 

[Ferocitus]

Some of the AI tasks could easily use such models. It would make the AI always pick the best options which may not be fun in an unbalanced game but technically speaking I don't see an issue for say choosing production, placing cities, policies, techs etc. Fighting and moving units is a way harder task but you could begin by at least make it learn through its own parameters instead of having it hardcoded.

The thing though is that if your AI is not capable of making an attack or puts city on the other side of the map I'm not sure you have the team to begin that kind of forefront work or just the opportunity to take the risk.

Of course the AI could be improved in many ways, but it's not just as simple
as wishing Firaxis would throw more money and effort at the problem now.

There are many simple fixes that Firaxis should do immediately, but there are
also many longer-term strategies that it would be foolish to attempt to
optimise now, given that there are more civs and enhancements planned.

The game in its current flawed state is enough to beat most casual players at
intermediate levels, and it gives most of them an enjoyable, challenging game.
Maybe Firaxis consider that that is good enough at this stage since the
initial release.

How do you decide on a set of "best options" in order to rank them in a
meaningful way? What might seem like a great option based on looking 2 moves
ahead could easily be a dreadful one when looking 3, or 30, or 300 moves ahead.

The specific example of China building Petra is interesting.

Yes, it is stupid to build Petra on the only tile capable of improvement.

But, by denying other Civs of that wonder, it might actually be a reasonable
long-term strategy, or it might also be awful in the long-term.

It is consistent with China's wonder-hungry trait. It can waste gold, time
or production that will later endanger the empire or send it broke. When they
get the right combination of wonders (or none!) to help themselves and hinder
other civs, the gamble could be enough for them to win.
It's not like there are no historical precedents for leaders spending money
and effort, or employing flawed strategies and tactics, on something that leads
to the ruin of their empires.

 

[Acken]

Of course the AI could be improved in many ways, but it's not just as simple
as wishing Firaxis would throw more money and effort at the problem now.

There are many simple fixes that Firaxis should do immediately, but there are
also many longer-term strategies that it would be foolish to attempt to
optimise now, given that there are more civs and enhancements planned.

The principles of machine learning exactly make it easier to adapt to new rules as long as you can capture the relevant information to make it adaptable.

The game in its current flawed state is enough to beat most casual players at
intermediate levels, and it gives most of them an enjoyable, challenging game.
Maybe Firaxis consider that that is good enough at this stage since the
initial release.

Its always pointless to bring that fact in a discussion on how the AI could be improved. We know this is how Firaxis does business these days. Its no excuse.
The thread is more about what could be done not what Firaxis will do. We all know what they'll do: the bare minimum.

How do you decide on a set of "best options" in order to rank them in a
meaningful way? What might seem like a great option based on looking 2 moves
ahead could easily be a dreadful one when looking 3, or 30, or 300 moves ahead.

I don<t even know many humans that can think that far forward. Nobody is asking for world champion level. If you have the choice between a University or a caravansary the choice is likely obvious though.You could argue though that it would make the game boring to have all AI do a 4 Citry NC into Universities. I agree and is something I mentioned earlier.

The specific example of China building Petra is interesting.

Yes, it is stupid to build Petra on the only tile capable of improvement.

But, by denying other Civs of that wonder, it might actually be a reasonable
long-term strategy, or it might also be awful in the long-term.

It is consistent with China's wonder-hungry trait. It can waste gold, time
or production that will later endanger the empire or send it broke. When they
get the right combination of wonders (or none!) to help themselves and hinder
other civs, the gamble could be enough for them to win.
It's not like there are no historical precedents for leaders spending money
and effort, or employing flawed strategies and tactics, on something that leads
to the ruin of their empires.

Thats not the way the decision was taken though. If I play random moves in chess sometimes I will make a good move. I still think an algorithm that does that is trash.

 

[Magma_Dragoon]

Some of the AI tasks could easily use such models. It would make the AI always pick the best options which may not be fun in an unbalanced game but technically speaking I don't see an issue for say choosing production, placing cities, policies, techs etc. Fighting and moving units is a way harder task but you could begin by at least make it learn through its own parameters instead of having it hardcoded.

The thing though is that if your AI is not capable of making an attack or puts city on the other side of the map I'm not sure you have the team to begin that kind of forefront work or just the opportunity to take the risk.

Firaxis doesn't need to resort to computationally intensive meme programming to do that, at least when it comes to civic/tech/build decisions. They need to attach relevant metadata (already done) to civics/techs/buildables and have the AI evaluate it in a meaningful way when deciding what to build.

The specific example of China building Petra is interesting.

Yes, it is stupid to build Petra on the only tile capable of improvement.

But, by denying other Civs of that wonder, it might actually be a reasonable
long-term strategy, or it might also be awful in the long-term.

Qin AI built it because he failed to accurately evaluate the effects of the wonder and how it would benefit him. Other AIs also fail to identify ideal locations for Petra and build it in cities which gain nearly nothing from it, Qin is just more likely to do it. This is an unresolved issue from 5.

It is consistent with China's wonder-hungry trait. It can waste gold, time
or production that will later endanger the empire or send it broke. When they
get the right combination of wonders (or none!) to help themselves and hinder
other civs, the gamble could be enough for them to win.
It's not like there are no historical precedents for leaders spending money
and effort, or employing flawed strategies and tactics, on something that leads
to the ruin of their empires.

Even a lawful good paladin shouldn't go blurting out the truth if it would get his party killed.

 

[Mr. Shadows]

The field in general in fascinating to me. I think Musk is right that AI will have a huge impact, and that it's difficult to say exactly what that will be. I have a feeling better video games aren't even the tip of the iceberg Having said that it does give me hope that we will see a decent AI one of these days. Right now I doubt my cpu would be able to run the game it had an AI capable of beating a good player, but it won't be long before machines like that are common-place.

 

[Bactrian]

In most strategy games there is a difference between avoiding misplays and being a good player. Good players not only avoid mistakes, they see innovative strategies, display creative problem solving, and engage in long-term planning. I think we all agree that an AI that plays Civ truly well is currently beyond the grasp of both Firaxis and modders alike. What I think is not out of reach is an AI that avoids misplays. There is no reason the AI should be sending out settlers unescorted, failing to finish off wounded units during warfare, failing to build follow-on units during wars, selling great works for paltry amounts of gold, etc.** CiVI's problem isn't that it can't beat a competent player and never will, it's that it can't avoid misplay after misplay in every single game.

** I'm not even going to count the complete inability to use air or naval forces for the purposes of this discussion.

 

[cazaderonus]

From an article i red maybe 2 years ago, i think the consensus regarding AI is that you can "easily" code an unbeatable AI for any game that has a simple set of rules like chess, or go, because there's only so few parameters in the game despite the billions of combinations. But CiV was actually taken as an example in that same article as the perfect example of what an AI cannot process.

One might think that CIV is actually a lot easier to master than chess, and i d tend to agree, but the dozens of parameters you find in the game, despite being a lot less complex and offering a lot less depth than the one parameter from chess, makes the AIs go all wonky. I'm no programmer, but chess AI are basically coded using a library of millions of reference games, and it uses it to adapt to the moves of its opponent. In CIV, you just cannot have that same approach because of the complexity added by all the layers of the game.

I still believe that coding an interesting AI for civ requires some more human input. Again i'm no programmer, but i dont see why they dont create a library of hardcoded guidelines for the AIs in the game, that will force their decision on specific gameplay elements. I thought agendas would be the perfect way to do this but it just didnt.

My point here is that if you were to play vs a human in CIV, you'll basically understand early-mid game where they are going if you pay enough attention to how they build their empire. play a few games with the same player, and you'll understand its playstyle and it will be very alike from game to game. So, maybe it would be more efficient for AIs to have less randomness and adaptability but a solid recognizable playstyle. That's where agendas fail in VI. instead of doing what i just said, they just add another layer on top of the basic AI behavior, and that just ends with non-sensical decisions.

 

[Acken]

Firaxis doesn't need to resort to computationally intensive meme programming to do that, at least when it comes to civic/tech/build decisions. They need to attach relevant metadata (already done) to civics/techs/buildables and have the AI evaluate it in a meaningful way when deciding what to build..

Thats already what everyone is saying I think. The question raised by the thread is exactly on the estimation part. Either its done by a logic based algorithm like in civ5 and civ6 and we only wish it was better made or we go an eentirely different route where the AI is trained to do this estimation. Computation of the weights is intensive but that would be done on Firaxis side not by the player. The player would have only access to the end result that is being updated each patch. For something like buildings the networks would not be very deep or have that many inputs and vectorized calculations may be not slower than the currentfor loops Firaxis uses anyway.

However all this raises quite a lot of questions when it comes to modability of the AI.

 

[crazyundead]

So Dota2 is in its core, a 5v5 game. Sure, this OpenAI bot beats Dendi in a 1v1 (and also some other top mid players like Sumail and RTZ) but we are still years away from having 5 bots beating 5 human pros in a real match. Additionally the very next day this bot was defeated 50 times by semi-high level Dota2 players because, surprise surprise, an AI being an AI can be easily exploited and that was what was used to beat this bot.

In short, this bot might be impressive but its strength is laser-focused on one thing and one thing only (beating an opponent 1v1 in lane) but if you extrapolate to an actual dota match it will still just behave like the regular stupid AI on the live Dota2 server. If I were to translate this bot's feats into Civ term it might be something like an AI that consistently militarily beat human players on a narrow stretch of standardized terrain, think a linear stretch where the bot starts one end, the human on the other, both locations with the exact same geographical layout and resources and no room to expand but to conquer the enemy city, oh and the human and AI play the exact same Civ (mirror matchup) and the only victory condition is domination.

 

[Abraxis]

To put it simply. First rule of any engagement is to first master the terrain and environment in which it's faught. I think that's even in the Art of War on this website.

This Dota AI is programmed specifically to master its environment. Civ AI must be programmed to adapt to any possible environment. This is a substantial jump in complexity.

It's the same as the Chess vs Go, except worse because Go still at least has a static board.

This is also what makes Civ unique among 4X. Space 4X, while having many great games, is really just a cop-out. It's a thematic way of avoiding the problem of having a complex environment with many errant variables to control. Space is static, and the simplest environment in the universe to master when all the practical part is simulated.

 

[Disgustipated]

I'm no programmer, but chess AI are basically coded using a library of millions of reference games

This got me thinking, how difficult would it be for the game to create some kind of after game file in your directory that could be referenced in future games to improve the AI? Would it be possible to create an AI that can learn from its mistakes and get better the more games you play? I'm no programmer, but this doesn't seem impossible to me, at least at a basic level. When you get to the end game screen, the game can evaluate how the AI's did, and create a file (that can't be altered by us humans of course) noting improvements it could do in future games. A lot of this computational power would be done at the end game screen, so I don't think it would slow down turn times since it's all done at the end. Some of those graphs at the end could be used to improve the AI possibly. A civ is low in gold per turn, next game they build more commercial districts. Just a thought, probably not possible this game, but I can dream about the future. If a civ is high in faith per turn, but they lose the game, next game they build less holy sites. Intelligent human players would still be able to game the system, but I still think it would be an improvement.

That said, I still like the flavors of Civ's this game. And I wouldn't want that to go away in the name of challenge. See, I have no problem with China building Petra in their 1 desert square in the example above. Do you not hear him when he gets on you? He is the preeminent builder of wonders. He doesn't do it solely for "numbers", he does it for the prestige of his Empire. And yes he is denying someone else (usually me ) that wonder. While it doesn't benefit him at all, it does hurt one of my cities. So I would want to balance any kind of learning AI I described above to still have Civs keep their flavors for what they favor. My biggest problem with these flavors is Norway still doesn't build enough ships. I often see Rome build more ships than Norway. Part of that is the Earth map I play, and Rome seems to see how many water squares they have, and builds a ton of ships accordingly.

 

[Bactrian]

This got me thinking, how difficult would it be for the game to create some kind of after game file in your directory that could be referenced in future games to improve the AI?

Or to send back to Firaxis for AI training. I would be willing to opt into a system that logs games and sends the results back to Firaxis. Their servers could then parse the files and they could then use all those games (potentially millions) for training. That might result in a monthly (or even weekly) AI update that incorporates all the latest knowledge that players have provided. The biggest problem I see is that the log might quickly become very large.

 

[Kurnn]

Civ are about strategy. On many levels. That game with "The best player in the world" is playing like its world of tanks.

 

[cll3]

The concept of AI has changed alot. What we grew up with is just a constant set of rules the computer uses to play against us. What we have now is learning AI, which trains itself statistically. It reduces the concept of AI we grew up with to just a program.

I've played Go and Chess since I was a kid. Deep Blue took a game off Kasparov, people lost their shlt. I was surprised too, but it was only one game. Plus I remember telling myself and others "At least Go will always be safe. So much more intuition required, so many multiples higher in branching factor."

Then 20 years later AlphaGo beats Lee Sedol 4 out of 5, but by this time, I was no longer surprised. In those 20 years my concept of AI has changed so much I went from thinking an AI would never beat a Go world champion, to knowing it was an inevitability.

We all use "AI" as a colloquialism for many things not really considered AI anymore, for example the computer opponent in Civ6, whose decisionmaking is purely driven by a static set of rules; it's not true AI. The only path to artificial intelligence is through machine learning.