Do you want an AI that can make reasoned choices?

[Victoria]

I have always struggled with the word AI beg bandied about in these forums. If you want an AI to play the game against you they have to either use reasoning or blindly look at statistics. Ideally we want an AI to be able to reason right?
An interesting test of the 'best' AI's for reasoning showed some interesting stuff, I directly add the best bit of this short read.
taken from here https://www.theregister.com/2022/03/16/scienceworld_ai_benchmark/

"Scoring of the 30 different tasks in ScienceWorld is based on a scale of 0.00, a total failure, to 1, indicating perfect performance. The highest score for any AI under test was 0.54, and that was on one of the simplest: identifying a non-living thing. For the ice, the best was 0.04.


In fact, a random-action generator stood out, with 0.63 for identifying a non-living thing. Building circuits was also abysmal. Virtually all the scores were low. :"

 

[vorlon_mi]

I have always *imagined* that the game-supplied opponents were calling an API to move a unit, build a terrain improvement, build something in a city; they were deciding what they should do with each unit and city based on some algorithm. In my head, I was never clear what those algorithms were, exactly. Some were simpler than others, such as move units to conduct a war. In earlier games in the franchise, nearly all AI worked towards a victory condition that could be "built", such as culture, space, or diplomatic.
In real life, from what I've read here, those decisions about what to build or whether to declare war are table-driven. If a certain value is high enough -- say, threat from a neighbor or diplomatic anger -- the logic would say, "begin plotting war." Does that deflate my imagination? A little. I want my computer opponent to try to win the game, using the same tools that I have: building, moving, and using units; settling and improving cities; improving terrain and changing governments. I want the opponent to pursue a goal consistently, even if it doesn't take the same steps toward that goal that a human player might.

 

[General_Sahib]

Ideally, we want Borg level opponents. Whatever we throw at them, we want some cunning modulation, I.e. evolution in tactics, to counter us. This game could be revolutionised if the ai warfare was strong enough to adapt and evolve to the human player's combined arms approach. At present, on Deity, the ai's best defence is rushing city walls, which humans still will ultimately and optimally counter with siege weapons (or possibly promoted range weapons). That is basically it... the rate limiting step, the central pivot, the whole domination game in a nutshell.

This is what actually makes the game so boring after some time. We all know the routine and we're already preparing for the ai walls rush from turn 10 because that is the only curve ball the ai ever has against us. If the programmers can arm the ai with just a few options, just a handful of alternative merit based strategies that it could RELIABLY use against us when the opportunity arose, new life would be breathed into the soul of this game. Civ 7 planners must genuinely consider arming the ai with a decent RANGE of strategies for use against us.

 

[Katzenkrimis]

Ideally we want an AI to be able to reason right?

.


AI in video games has always been bad. For the fifty years that video games have been around, which is quite young, it's always been an elementary-level science project.

Understandably.

100 years from now there will be massive improvements to AI in the video game industry.

The only thing we can do today is scale video games amongst themselves. As in, which games have, not necessarily 'great' AI, but which games have the best AI among a sea of garbage. Civilization VI had some of the worst AI I've ever seen in a strategy game when it was released. And I've been gaming since the late 70's, early 80's. Units weren't using air support at all. And land armies were traveling out to sea in transport boats to attack navies.

I remember when Supreme Commander Forged Alliance was released. The AI was terrible. A guy by the name of Sorian made an AI mod for it. His AI was much improved from the original game because it was adaptive. It adjusted itself according to your moves.

Despite the hatred that Mass Effect Andromeda received, I played that game on release and was surprised by the flanking that was taking place. I liked how the AI would try to flank me. Shooters aren't known for having good AI. So, subtle improvements like flanking are very noticeable. Years later, I noticed that games like Cyberpunk, and Far Cry 6 also had fleeting moments of a more intelligent AI. As more flanking was taking place.

In sports, Konami's Pro Evolution Soccer, in 2018, was the first game that I played that had, what they called, an 'Adaptive AI'. Countering your strategies as best it could.

None are perfect. None are amazing. But some developers invest in AI much more than others. When they do, it's very noticeable and a treat to play.

Civilization VI's AI is light years ahead of where it was in 2016. For the first time since release I played six games this week. I only won one. It was a domination victory, and it was draining. Not easy. Sure, the AI makes some silly moves here and there, but it's forgivable. Depends on your standards. For video games your AI standards should be low.

I don't know if you've ever played World of Tanks but there is a move that you do where you show the front of your tank's tracks to the opponent as you are hiding behind a structure. You bait the opponent into taking a shot. Once he takes that shot, you quickly drive forward and return fire, then back up again into cover. It's a genius move and it can be done in Civ, using the hexes. The AI did it to me this week and I was surprised. Clearly the AI engineers for today's Civ VI are different from the ones who originally worked on this game. That, or they are much better at their jobs. I suspect the former.

This game has come a long way from transport ships chasing enemy battleships across the sea.



.

 

[Quintillus]

I'll answer the question with another question - how do you define "reason" with an AI? And how does that differ from statistics?

Background: I'm a professional computer programmer with 10 years' experience, and worked with a professor to develop an AI system around logical reasoning in college. So, slightly more qualified to talk about AI than the typical software developer who doesn't have that AI background, but also not a fully-up-to-date expert myself.

---- What is AI? ----

Spoiler :

The word AI is bandied around a lot, and often without a lot of precision. Not just at CFC, but in life. You'll see ads for AI-powered thermostats, voice assistants, websites, probably toaster ovens by now too. Machine learning, AI... they're both buzzwords, but they also have a meaning. But it's often a bit of a black box what that meaning is. It's often presented as, "because it has AI, it's better than if it didn't", when as we all know from playing video games, the quality of the AI - what goes on behind the scenes - matters.

Fundamentally, a computer doesn't have a brain like a human does. It doesn't have high-level reasoning of the sort we'd employ when considering, "does it make sense to trade with the Inca, or is that just going to help them win the game?" A computer can only make simple logical comparisons, such as, "is 42 greater than 37?" These can be layered on top of each other, and consider various factors, but ultimately, when you decide to propose a deal to the Inca, the computer is going to run some algorithm, evaluate some comparisons, and return either "yes" or "no" based on an accumulation of low level decisions.

So, the question of can the AI reason, ultimately comes down to not whether it has the same capabilities we have - it doesn't - but whether the algorithms that were used in programming it can do a convincing job of simulating that state. IMO, that is desirable. Civ4 did this at times; sometimes the AI would decline a deal or refuse to trade a space-based tech because, "We want to win the game." Behind the scenes, there was some comparison such as "if this tech allows a spaceship part, or is the prerequisite to allowing a spaceship part, don't trade it." It successfully created the illusion of an AI that was trying to win by Space Race, and was smart enough not to trade away what you needed to catch up to it.

--- Making a Convincing and Effective AI ---

Spoiler :

I don't know enough about ScienceWorld to really evaluate the article, but I think the takeaway is that it is hard to make an AI that is convincing and/or effective at tasks that are somewhat complex. Which is true. Boston Dynamics has done some really impressive stuff with robots that can walk, bipedal, across all sorts of terrain. But they've spend years and years building it, surely building on the work of others as well, and probably hundreds of millions of dollars on it, if not billions. And those robots still couldn't wash your dishes or hang your clothes on a line. I remember back around 2017 when car companies were saying they'd have full self-driving cars by 2020; I was not at all surprised that didn't come to pass. To make an AI that is passable in relatively simple conditions is not too difficult; if a highway has reliably painted lines, and it's never rainy or snowy, and all the other drivers stay in their lanes, you could make a car that stayed in its lane pretty easily. But what if snow covers the edge lines, or another driver drifts out of their lane, or you want to exit the highway? What if there are multiple lanes painted because of a slightly rerouting due to construction, or if it's down to one lane and all that lane has is a concrete barrier on one side and orange barrels on another? A simple AI isn't going to be up to the task.

Civ has the same basic problem; you can make an AI that does a halfway decent job of choosing where to build cities, but how can it handle the sheer variety of strategies a human might throw at it? What if the human turtles up and goes for a culture victory? What if the human zerg rushes it? What if the human brings siege weapons? What if the human gets lucky with strategic resources and it doesn't? What if it attacks the human for that Iron, but another AI attacks it to take its Horses? How can it form alliances with other AIs to counter an aggressive human, without it seeming to be programmed to always gang up on a human that decides to fight an AI?

Ultimately, it's a question of how much effort is put into the AI, as well as how hard the problem actually is that you're trying to solve. In the 1980s, the Japanese government funded research with the goal, "to write programs and build machines that could carry on conversations, translate languages, interpret pictures, and reason like human beings". Forty years later - longer than digital computers had been around at the time the research was started - computers still can't carry on conversations of any depth, have only recently become fairly decent at translating languages (and that likely is only true for the most common ones), since about a decade ago are good at identifying objects in pictures (although not necessarily interpreting events between multiple pictures), and cannot reason like human beings. Some problems are really difficult, and in the realm of games, how difficult it is can depend on the mechanics that were chosen.

--- AI Approaches --

Spoiler :

It's also worth noting that there are a couple different approaches to AI. Machine learning is often a fancy way of saying, "pattern recognition". There are some algorithms, you give the algorithms a whole bunch of data, and afterwards it can identify patterns. This is how Google Reverse Image Search works - you upload a picture of the Eiffel Tower, it has already seen a bunch of similar pictures, and it can tell you that your picture is of the Eiffel Tower. Whereas with games, it's typically more handcrafted code considering all sorts of conditions around the state of the game and the mechanics. This is in part because if you wanted the AI to "learn", it would need a lot of data, more than it could get from any one human opponent. If you win against an AI, it doesn't know if that's because you researched Bronze Working first, or because you used the terrain features better, or because you build an Arts District while it built a Military District. And even if the developer did combine data from all the human players, and AI-vs-AI games, and the AI did learn patterns effectively, you might wind up with an AI that plays very optimally, but is not at all fun to play against because the human would always lose - or all but the best humans would always lose. But it's just as likely that the AI would still be hopelessly lost as to why it's losing, and might perform worse than it currently does when it plays with hand-coded guardrails around how it should behave.

--- Conclusion ---

The AI in strategy games is never going to be "perfect", and we don't want it to be, then it would be like playing Deep Blue in chess as an average chess player, and there's no fun in that. But we want a little bit of a challenge, we want it to appear to be trying to win. We want some insight into what its "motivations" are (really, the factors that go into its comparisons), but we want it to appear to be higher-level, too.

But General Sahib is right that in Civ and most strategy games, if you play enough, you pick up on the AI's patterns. Sometimes sooner, sometimes after more time. But once you do, that can make the game boring. I agree that a greater variety of AI strategies would make the game more interesting. Even if some of the strategies aren't very effective, the very fact that different AI opponents are trying different things could make the game overall more challenging - and perhaps even more so, interesting - because you'd have to consider that the AI might do something different.

Adapting and evolving in reaction to the human could be a tall ask in the general sense, but in broad strokes could likely be implemented somewhat. If the human takes a clear lead in the space race, the AI could decide to try to intervene militarily, for example (modded Civ4 AIs can be brutal in that regard). Perhaps if the AI is pursuing "military buildout A" and it fails miserably in one war, it can at least change to one of B, C, D, or E so you have something different to face next time.

IMO, AI is the biggest weakness of the recent iterations in the Civ series (V and VI), especially considering that IV was pretty good among strategy games in that regard. Can Firaxis improve the AI in Civ7, making it so that it both appears to have high level reasoning, and is competent enough to offer a challenge? I'm not personally optimistic, but maybe I'll be surprised. On the one hand, I suppose it's a good thing, what Katzenkrimis says about Civ VI's AI improving over time. On the other hand, it's still a lot less of a challenge than the Civ3 AI.

 

[Siddorm]

Disclaimer: I am not an AI expert.
I will be shocked if Civ 7 does not have a neural-net-based AI that is capable of a superhuman level of play.
I view the approach behind Maia Chess as having the best potential for AI in games like Civ 6.
A related point: I view Civ 6 as bloated with excessive and redundant content (such as lots of policy cards that I never use etc.); I believe that AI performance would increase if such content were removed.

 

[Oberinspektor Derrick]

I view the approach behind Maia Chess as having the best potential for AI in games like Civ 6.

I dont know anything about Maia Chess, but I did follow with interest how Google's Deepmind revolutionized chess.
Similarly, it was interesting how the engine only partially was able to play Starcraft on a high level, and showed significant weaknesses there when playing 2/3 races.

Chess is after all, quite an "easy" game to learn.
It has a grid of 8x8 squares, and few rules you need to know.
Starcraft on the other hand is a lot more complicated, and here the engine started to struggle.

I view civ as magnitudes more complicated than even Starcraft is, with a lot more dynamic variables that keep changing and react with one another.
For this reason I believe that we are still far from AI being able to play the game on a high level, like a human can do.

 

[Linklite]

Disclaimer: I am not an AI expert.
I will be shocked if Civ 7 does not have a neural-net-based AI that is capable of a superhuman level of play.
I view the approach behind Maia Chess as having the best potential for AI in games like Civ 6.

I would be very surprised. Chess and Civ are very, very different beasts when it comes to programming. Even getting human level abilities in an AI for Civ would be astoundingly expensive to develop. Doable, but not by Firaxis and not for a game like Civ. At least, not for a decade or so.

 

[bbbt]

Having an overall great AI is very daunting. Focusing on a few key features that are least impacted by the AI bonus handicaps would go a long way. In that regard I'd say tactical combat, diplomacy, and win conditions - though at least the first two are two of the hardest areas of course.

I agree that a more diverse AI in terms of civ behavior would also be ideal (and probably more challenging). I think an AI that was more varied in it's behavior - even if that meant some CIVs would underperform significantly at times - is a better option than them all being vaguely competent but samey.

 

[Siddorm]

I view civ as magnitudes more complicated than even Starcraft is

I believe that this is not the case because Deepmind says (about Alphastar) "There are up to 10^26 possible actions available to one of our agents at each time step, and the agent must make thousands of actions before learning if it has won or lost the game".
On reflection: I don't actually know what the maximum number of actions per turn in a Civ 6 game is, so my belief may be wrong.

 

[Benevolent Dictator]

I don't know if you've ever played World of Tanks but there is a move that you do where you show the front of your tank's tracks to the opponent as you are hiding behind a structure. You bait the opponent into taking a shot. Once he takes that shot, you quickly drive forward and return fire, then back up again into cover. It's a genius move and it can be done in Civ, using the hexes. The AI did it to me this week and I was surprised. Clearly the AI engineers for today's Civ VI are different from the ones who originally worked on this game. That, or they are much better at their jobs. I suspect the former.

I used to play quite a bit of World of Tanks on console until they made a major overhaul to the game around Christmas 2020 and I gave it up for good.

World of Tanks, an arcade tank simulator with third person battles PvP, has some abstract parallels to a 4x history game like Civ.

World of Tanks is a very old game for a PvP, dating back a decade or so.

It began as an arcade shooter but with real historic pre-Cold War tank designs and specs along with gameplay centered around real life ballistic physics with shell penetration and armour thickness and angle considerations.

The game was designed by a team from Belarus, popular in Russia and was designed to run on very low spec hardware.

Given hardware and graphic limitations, it was purposefully decided that tanks in-game couldn't fight or even see each other past 454 m apart.

So for a game originally designed for historic tank spec and ballistic physic realism, this design and hardware limitation created an environment where arcade play meant tanks fighting in closer quarters than their IRL counterparts.

This also lead to expert players exploiting in-game rules and physics in ways the developers did not necessarily design or intend.

Expert players soon began deploying strategies such as "sidescraping", where they would hide the front of their tank behind a corner of a building, but purposefully exposing the side of their tank at an extreme angle to make it almost impossible for their opponents to penetrate their tank's armour due to the angle of impact of incoming shells, while simultaneously keeping their own turrets free to damage enemy tanks in turn.

So entirely new metas developed in-game shaped by historic realistic specs, IRL ballistic physics systems employed and in-game rules and hardware limitations.

The small problem however was that a game originally designed as a historic and realistic tank simulator soon became dominated by successful strategies that were never used and would never be used IRL.

Soon afterwards, the World of Tanks developers ran out of actual IRL tanks they could port into the game and gave up on realism and historic simulation. Instead, they embraced the game's arcade elements by flooding the game with make believe and historic blueprint tanks that were never built, dropping any pretense of historic accuracy and embracing game elements officially such as "sidescraping" that were community created and promoted.

 

[kaspergm]

I'm probably in the minority, but a super-duper-state-of-the-art AI is not a priority for me. Or put in other words: I don't need an AI that plays like a human player. I realize that saying "just play multiplayer" to those who wants that is not really a solution because of all the practical obstacles related to multiplayer playing a game like Civ, but for me who's not really a competitive player, my wishes for an AI is something else.

That does not mean I don't have hopes for a more capable AI in Civ7. On a grand scale, I'm hoping for an AI who'll be able to actually follow through on a game strategy and build the required infrastructure to support this strategy. Now I'm not very keen on thinking the victory conditions explicitly into this - I don't want an AI to pursue a "domination victory" and just mindlessly DoW everyone it encounters, but I do think an AI that focuses on military should adopt governments and policies - in whatever form they will exist in future civ games - and be more aggressive than an AI that pursues a strategy based on, say, science or culture.

On a smaller scale, I'm hoping for an AI that will be less prone to do some of the mindless things we've seen Civ5 and Civ6 AI do. Here I'm thinking about units being mindlessly embarked or killed by throwing them into heavily defended cities, and how many years did we actually wait for the AI to learn to set up and use siege weapons? And to this day, Civ6 AI still does not know that the "defend" action exists, which would go a long way in actually helping city states survive with their smaller armies.

 

[Vandlys]

Ideally, we want Borg level opponents.

Who is this "we" you speak of? I don't want that level of AI in my game. I mean, if there is an option to turn it down a notch or 100, sure, but only AI like that? No way, I play games for entertainment, and Borg-level AI does not sound entertaining.

 

[General_Sahib]

Who is this "we" you speak of? I don't want that level of AI in my game. I mean, if there is an option to turn it down a notch or 100, sure, but only AI like that? No way, I play games for entertainment, and Borg-level AI does not sound entertaining.

"Borg" is a bit hyperbolic I suppose. I like your idea of a sliding scale. Borg at the top end....down to Tim Henman at the bottom.

 

[Grassland Farm]

I've only completed 1 game so far (old timey 4 purist here, who also has moved to 5 once Vox Populi did a good job with it), so I don't have much insight, but my first glances are:
- AI on lower difficulties is absolutely laughable. I'm a civ veteran so maybe I shouldn't have started with Prince, but come on now - the AI had maybe 2-3 units to defend their land across the game, even well into Industrial Era. Most of my warmongering was swooping empty cities.
- Once I had moved to higher difficulties it felt more right but the AI was extremely passive that time - as I had switched from Genghis Khan to Jayavaraman and played peaceful. I basically never got declared war on, even when the AI had an army that couldve butchered me - I spotted that army, in fact, because it was attempting to defend a city that they forward-settled into me and lost to Loyalty right away... to me.
- Same game, I was friends was 6 civs, enemy with 1. Maybe it was just a coincidence of AI personalities, but I found that a bit too comfortable.
- New game: Not much to say (I actually got declared war on and had to defend myself by rushing a few units) but I noticed my only neighbour on my continent, Kupe, struggles horribly and hasn't settled anything but his capital. I suspect it's a case of "civ is so unique AI can't play it", fairly symptomatic of how civs have gotten more unique since 5. I might consider disabling Maori for my next game.

Oh and last but not least: World Congress is made boring by the AI. They're not stupid at it, but they tend to choose the same option and to choose positive options over negative options, even when it could help them (e.g I think a peaceful builder civ should make it harder to build Encampments, rather than vote for double production in City Centers like everyone else)

 

[Heathcliff]

Main problem is that ai doesnt build enough units

 

[kaspergm]

Main problem is that ai doesnt build enough units

Well that is objectively a debatable statement, given that the player can beat the AI with much fewer units than the AI has. So one can argue that instead of building even more units - which would undoubtedly help the AI, but also take time away from building other things - making it more capable of managing its units would be a bigger help (but arguably also a much harder task to code).

 

[Victoria]

The main problem is the game engine does not take in all the valid data and make logical conclusions based on it, partially because there is too much data and too many possibilities but there is also a lack of memory by the game engine.
Reason is ideally when something can make up its own mind through reasoning, but neural networks are showing that 'AI's' ar not reasoning, they are learning optimum paths without adequate bias.

 

[Leyrann]

- Same game, I was friends was 6 civs, enemy with 1. Maybe it was just a coincidence of AI personalities, but I found that a bit too comfortable.

Maybe it's just because I got better at the game or play it differently these days, but I feel like this is a more recent thing. I don't remember at all from the past that I could befriend every AI I came across, yet nowadays that's exactly what I do in every game. Mind, I haven't played for two and a half years between shortly after GS release and a few months ago, so there were probably numerous patches in that time.

That said, this was more noticeable on Emperor. On Deity, some AIs remain hostile to me for a longer time, however I can usually still (almost) cap out my alliances when playing on a standard map (so 7 AIs total).

 

[universecreep]

For me an AI that was able to use the game systems correctly is a priority. I guess this could be thought of as making reasoned choices.

Civ IV was probably the pinnacle for me in this regard although Civ V was pretty good too. Civ VI suffered from too many gameplay elements that the AI couldn't use or used ineffectively.