Do you want an AI that can make reasoned choices?

[TheMeInTeam]

There is no chance that will happen because computers aren't fast enough and don't have anywhere
near the amount of resources to handle a large enough "neural net" for Civ. And if computers could
do that, you would not like the size of your electricity bill!

My understanding is that training such AI would be expensive and time consuming, but once a strong enough agent has been trained using it would not be nearly as hard on machines. That has a few obvious problems, though:

1. AI would have to be retrained any time significant changes are made to the game.
2. This is considerably more expensive than using what they do now, and requires expertise they are unlikely to have just sitting around in-house.
3. This is completely unnecessary for commercial success, as we've observed for the last 10+ years.
4. A sufficiently well-trained neural net AI would create an unwinnable game for humans, and thus you'd have to assign bonuses or deliberately nerf that AI back down. Even more cost/nuance for minimal benefit.

All this in a game that still has bad UI and performance? No way. I think computers could handle it in principle, but that doesn't make it a reasonable idea. Especially since even though Civ 6 deity is "easy" compared to Civ 4 deity, it is still a small % of players that bother with that difficulty level. The game can and should have other priorities, such as fixing the UI, some basic design choices with mechanics/tradeoffs, and performance.

ML in civ is fun to think about. And I'd love to see how such an AI would play. But it's not feasible for a commercial game to my knowledge. People generally don't want opponents they can't beat, and they cost a bunch extra to make.

 

[Leyrann]

Especially since even though Civ 6 deity is "easy" compared to Civ 4 deity, it is still a small % of players that bother with that difficulty level.

One criticism I do have that I hope will be addressed for Civ VII: I don't like the Deity AI starting off with 3 settlers, even if they only spawn on city settle. I want either the bonuses tuned or the AI strong enough that a proper Deity difficulty starts with no more than 2 settlers, and I'd prefer if it's also the only difficulty that spawns more than one.

Simply because otherwise the AI's early game advantage becomes very frustrating (it's my main gripe with Deity at this point), and in the late game you'll still eclipse them anyway. Basically, giving the AI such strong front-loaded bonuses turns the game into "can you survive an ancient or classical era rush so that you can then outsim the AI to victory?". If the answer is no, you scale down the difficulty, if the answer is yes, you can beat Deity every time again and everything after turn 100 is boring.

 

[TheMeInTeam]

^ While that was less true in Civ 4, it has always been somewhat true. Bonuses heavily front-load difficulty, and either you get past a certain threshold fast enough or you tend to be in a lost position.

 

[Ferocitus]

Chess neural-net-based-AI can self-play thousands of games per day at a superhuman level on an average modern PC.
I agree that Civ is more complex than chess, but this is also true for human players (I am not saying that such an AI would play perfectly).

Sure, but the number of hexes, their different types, the number of different units, and the number of
possible moves is far, far greater than chess. There are about 10^120 different possible chess games.
For each turn there are only a few hundred possible moves at the very most, and there are no unknown
squares or pieces. A neural net can work well with that.
The possible number of moves on each turn in Civ after a few turns is in the many thousands. and there
are lots of unknowns. After each turn when more hexes, pieces and other features are revealed, any
neural net will have to factor them in. Because they are so brittle, it will also have to abandon a lot of its
previous calculations and predictions in the light of that new information. Removing possible branches in
chess is also necessary, but there's no need to factor in new pieces, hexes etc.
Forward planning by neural nets is unlikely to end up playing a better game because it will also be
intolerably slow. Nor will it necessarily create a game that's more fun to play against.

I can, however, see a use for some form of AI/data mining to optimise the weightings for certain game
"ratios", e.g. culture, gold, production, and faith.

 

[Siddorm]

The reason I mentioned Maia Chess is because it can play chess as badly as humans without exhibiting "broken" behaviour. Quoting maiachess.com "Maia’s goal is to play the human move — not necessarily the best move. As a result, Maia has a more human-like style than previous engines, matching moves played by human players in online games over 50% of the time."

 

[Leyrann]

^ While that was less true in Civ 4, it has always been somewhat true. Bonuses heavily front-load difficulty, and either you get past a certain threshold fast enough or you tend to be in a lost position.

I don't think any game prior to Civ VI had the AI start with 3 settlers on Deity, right?

 

[Grassland Farm]

I don't think any game prior to Civ VI had the AI start with 3 settlers on Deity, right?

Civ4 is 2 settlers I think. The main block being that the AI starts with Archery (and free Archers) so they can't be rushed by anyone except Incas

 

[TheMeInTeam]

Civ 4 deity was two settlers, yeah. IIRC it has more extreme bonuses, however. Though BTS toned them down relative to Warlords.

What made it harder to beat was the sheer volume of units + sheer bonus tech forced you to tech very fast to match/surpass it, otherwise you'd just lose to culture or space while being unable to handle stacks in the hundreds to contest those. It put players on much more a timer wrt win pressure, 1800s space race wins or even earlier culture wins weren't that uncommon. Player can go faster with excellent building/planning, but a lot can go wrong. Having 50-100 units shoved in your face for a long time can crater tech rate etc.

When it was still the newest Civ game, there were only a few handfuls of players that could reliably beat deity that posted. I was not one of them, though I could manage it sometimes.

 

[Leyrann]

When it was still the newest Civ game, there were only a few handfuls of players that could reliably beat deity that posted. I was not one of them, though I could manage it sometimes.

Yeah, I heard about this. I never played Civ IV all that much myself, and when I did I usually played on... Noble, I think, but I got the impression that Civ IV Deity was significantly harder than Civ VI Deity. Even Civ V Deity, as far as I understand it, is harder.

What I do know for sure is that Civ VI is the only one I, personally, play on Diety (and can win relatively easily).

 

[civac]

Civ4 AI gets increasing bonuses with era progress. Deiy AI gets a huge head start and it is at its strongest in the early game similar to Civ6. But unlike Civ6, the Civ4 AI starts accelerating again in the modern eras. Late game tech speed and unit production is absolutely bonkers.

 

[shevvek]

I mostly agreed with the recent PotatoMcWhiskey video about the AI mainly mattering in terms of generating an interesting experience to play against. I don't really care if the AI is doing smart things, but I want the game to feel like an interesting and interactive challenge.

I can think of a few specific areas where I feel like the Civ6 deity AI is really lacking:

1. Really basic city planning. When I conquer or flip a city and I see that the AI never bothered to build a Monument, or it placed a Campus with 0 adjacency even though there was an obvious place for a +4 Campus, it feels really lame. Or settling a city on a tile with no access to fresh water or aqueducts literally one tile away from a better spot. The city and district placement is particularly irritating, as I can't fix it even after capturing the city and it just stays as a reminder for the rest of the game (unless I raze, which isn't even always possible).
2. Linear growth instead of exponential. The end game graphs always look the same: every AI is a straight line and my graph curves upward. I'm perfectly happy for the AI difficulty to be based on "cheating" bonuses, but the bonuses should either compound as the game progresses, or there should be rubber banding to keep the AI competitive with the human player. It's no fun to play solitaire for 50 turns just to close out a game after already making 5x the top AI's science and culture.
3. End game competition. Close end games can also be kind of unfun if it turns into a really drawn out grind/spamfest. I think it would be really cool to see Civ7 have an endgame crisis event system for each of the victory types, to bring the game to a satisfying narrative climax and conclusion.
4. Really obvious glitchy behavior. I don't mind at all if the AI is glitchy and illogical on the other side of the map where I'm rarely paying attention, but it's jarring when the AI behaves inconsistently toward the human player. For example, refusing to renew a friendship for a single turn, then requesting a friendship the next turn. Or refusing to join a war at any price, then agreeing as soon as you offer to pay 1 gold.
5. Overly consistent early game choices. If the AI has a large starting bonus, it needs to have more randomized early game behavior, as otherwise the human player effectively never has the opportunity to get certain early game wonders or religious beliefs. Don't get me wrong, IMO it's fine if certain things are really strong/desirable and require a strong focus to compete for them, but it's boring when every AI civ picks the same religious beliefs in the same order every game. You'd think different cultures would at least prefer different ones.

 

[TheMeInTeam]

It's no fun to play solitaire for 50 turns just to close out a game after already making 5x the top AI's science and culture.

The correct solution to this problem is to have a mechanic which ends games that are over, regardless of AI.

The Civ series has been reliably bad at that. It was least noticeable in Civ 4, where you could roll through turns really fast, but it was a problem even there and still worse than games > 10 years older than it.

Games have played around with other solutions, but they tend to be worse. If you rubber-band, you decrease how much previous choices matter. One last ditch dogpile could make sense, but making the AI evaluate the game state correctly to trigger this implies knowing how to make the AI good at the grand strategy aspects of the game, and that's back to square 1 lol.

 

[rocksinmypath]

There are some common and understandable misconceptions about artificial intelligence in this thread. There's a trend of equating AI with machine learning, which I think is unfortunate as it misleads people to think that somehow results from one application of ML are relevant to a completely different situation. We're probably many, many, many years away from an AI that can correctly tell us what will happen when you put an ice cube on a stove AND play Civ as well as expert human players. Successful ML applications today are much more limited in terms of what they can do. Also, the article mentions that "complex" AI models aren't smarter than fifth graders, but from my experience, complexity of a model has very little to do with how well it works, and if anything, simpler models generally work better. In fact, we have many problems in our world, for which ML is just an inappropriate solution, and you're much better off trying to solve them with less fancy methods. Sure, there are problems that can be solved well without ML but can be done better with it, and those that are infeasible to solve without ML, but AI or ML doesn't give you a one-size-fits-all solution to all problems. I think of ML as a tool, like a power drill, that can be very useful and sometimes clearly superior to a simpler tool like a screwdriver, but you wouldn't use it to eat a bowl of cereal, because a dumb spoon will do a much better job.

For a game as complex as Civ, a competent AI will probably have several different components, some of which may be implemented with ML. Maybe there are some lessons from successful ML-based methods from other strategy games like chess, go and Starcraft that can be applied to Civ as well, but Civ is different enough from other games that it will probably require a monumental effort from the developers to create a highly competent, ML-based method. Civ is played on a much larger map than chess and go. It's played by more than two players, and each player can make an unbounded number of moves per turn. You're not privy to all information on the map. Getting more information isn't free, so you need to balance information-seeking with other things you need to do in the game. Even if you were able to get all the information there is about a game, how much of it do you really care about? In some ways, you have to play the game as if you're the only one. Each game of Civ is meant to be unique. Maps are generated randomly. There are ~50 leaders in the game. You can even play with a different set of rules every time, and the next iteration of the game will have a complete different set of rules. There's no way Firaxis can collect enough game data that can be useful for training an AI before the game is even released. Many of the questions the devs will have to answer still more or less belong on the cutting edge of the field of AI.

Besides, a human-like AI won't necessarily make the game fun. Some people might be okay with it, but I think it'd be very controversial to let the AI declare surprise wars just to steal settlers and to move their units in a way to deliberately block other players' units. I think Firaxis has done a very good job of making the AI feel like AI, and I don't mean that in a sarcastic way. It's good that the each leader has their own set of agendas and that they play the game "honestly" without trying to be extremely exploitative like humans can be.

Having said that, I do think there's room for improvement, and a (not perfect) solution doesn't have to involve ML. The game could be designed to be more AI-friendly without being unfun for humans to play. They could remove some traps the AI is prone to fall into. Trading with human players is a good example of this. AI constantly engages in terrible deals, and it's widely accepted that if you want to play well on a higher difficulty, you need to learn how to exploit this weakness, giving away your citrus for 10 god per turn to an AI player or get three other luxury resources for may 5 gold per turn. Trying to figure out how to best fleece the AI has never been fun for me, but I do it because I might as well. Instead of allowing players to trade directly with each other, there can be a marketplace where trading occurs between each player and the "game master", who sets the price for everything, and there's no room for negotiation. Less room for AI to do something stupid, and less tedious for humans. Win win if you ask me.

Another way I think the game could be more accommodating to the AI is if it abandoned binary victory conditions in favour of a victory point system. This is just a hunch that I have, but I think the AI will tend to do better in a victory point system, where it just needs to figure out relatively short series of conversions (e.g. x food > y production > z culture > w victory points), each leading to a small number of points toward victory, rather than figure out how to send people to Mars through much longer-term planning. Besides, religious and diplomatic victories are just terrible in my opinion, and if the devs would just move on from VCs, they could get rid of features like religious combat and to "vote with the herd for victory points" mechanic.

 

[uhu]

Agree wholeheartedly with nearly all!

The game could be designed to be more AI-friendly without being unfun for humans to play. They could remove some traps the AI is prone to fall into. Trading with human players is a good example of this. AI constantly engages in terrible deals [...] Less room for AI to do something stupid, and less tedious for humans. Win win if you ask me.

Yeah, The game could be designed to be ... & Win win, indeed. From the dedicated player's point of view and if the producer's priority would be to make a good game.

Civ6 AI (in the sense of a smart automatic washing machine ) could be vastly improved, if implemented at all (in the beginning Aircraft units, Heroes, Secret societies, Pirates ...), adequately designed (eg Trading, World Congress) and in general working as designed (i.e. properly debugged). Would generate most bang for the buck. But just this is too expensive apparently -- as the current (and probably final) state of civ6 shows.
Because the producer's priority is to sell maximum number of copies. And this coexists happily with the casual player's point of view: he is no great strategist (expects to win on deity after a couple of games with exceptionall start and reloads etc., so existing, horribly exploitable Trade is OK) and he likes shallow sensations. Whow, death robots, vampires, zombies ... K-pop dolls and QA in bear pelt. And visions of endless roller-coasting ...
Civilization is entry level and I suppose, they'll emphasize that even more in the future. Low need for ideas like "first learning a game and then mastering it", extravagant AI or strange things like "Modding".

Probably those players, who are looking for a deeper experience, fine balanced & coherent mechanisms -- finally a higher level of quality in design, implementation, customer support etc. should look elsewhere. Maybe find in OldWorld: it is a diamond.

 

[kaspergm]

5. Overly consistent early game choices. If the AI has a large starting bonus, it needs to have more randomized early game behavior, as otherwise the human player effectively never has the opportunity to get certain early game wonders or religious beliefs. Don't get me wrong, IMO it's fine if certain things are really strong/desirable and require a strong focus to compete for them, but it's boring when every AI civ picks the same religious beliefs in the same order every game. You'd think different cultures would at least prefer different ones.

I think this is a really good point, that I have not seen stated in this way before. Of course, the downside may be that if the AI just picks randomly, it will appear, well, just random, which is not desirable either. But I agree Civ6 has way too many "fixed-order" elements - which at best can challenge the player for the really strong stuff, and at worst make the AI always make bad decisions (like the AI never picking Work Ethics, which, at least in the hand of the human player, can be completely game-breaking) and make the game boring and predictable, not to say abusable (World Congress, ugh).

Another way I think the game could be more accommodating to the AI is if it abandoned binary victory conditions in favour of a victory point system. This is just a hunch that I have, but I think the AI will tend to do better in a victory point system, where it just needs to figure out relatively short series of conversions (e.g. x food > y production > z culture > w victory points), each leading to a small number of points toward victory, rather than figure out how to send people to Mars through much longer-term planning. Besides, religious and diplomatic victories are just terrible in my opinion, and if the devs would just move on from VCs, they could get rid of features like religious combat and to "vote with the herd for victory points" mechanic.

I agree completely. The victory conditions are not even fun to begin with, so it's not like anybody would feel sad to see the umpteen turn wait for your spaceship to reach the destination or the spam of gazillion apostles or rock bands to go away.

The thing is, Civ6 already has a framework for this incorporated in Era Score. I'm not saying the Era score system was perfect (like Eurekas, there was a large element of doing non-sensical things just to earn the points in it), but still, the idea that doing certain actions or achieving certain milestones through an era could give you points is a good basis, and victory could be based on this - either through a simple cumulative score, or probably better, through a system where the players with most era score in each era earns points that will decide the victor at the end of the game. Btw. Humankind had some good ideas in this area, even if their system was far from perfect also.

 

[shevvek]

They could remove some traps the AI is prone to fall into. Trading with human players is a good example of this. AI constantly engages in terrible deals, and it's widely accepted that if you want to play well on a higher difficulty, you need to learn how to exploit this weakness, giving away your citrus for 10 god per turn to an AI player or get three other luxury resources for may 5 gold per turn. Trying to figure out how to best fleece the AI has never been fun for me, but I do it because I might as well. Instead of allowing players to trade directly with each other, there can be a marketplace where trading occurs between each player and the "game master", who sets the price for everything, and there's no room for negotiation. Less room for AI to do something stupid, and less tedious for humans. Win win if you ask me.

1000x this!

The victory conditions are not even fun to begin with, so it's not like anybody would feel sad to see the umpteen turn wait for your spaceship to reach the destination or the spam of gazillion apostles or rock bands to go away.

The thing is, Civ6 already has a framework for this incorporated in Era Score. I'm not saying the Era score system was perfect (like Eurekas, there was a large element of doing non-sensical things just to earn the points in it), but still, the idea that doing certain actions or achieving certain milestones through an era could give you points is a good basis, and victory could be based on this - either through a simple cumulative score, or probably better, through a system where the players with most era score in each era earns points that will decide the victor at the end of the game. Btw. Humankind had some good ideas in this area, even if their system was far from perfect also.

Agree completely about the spammy win conditions. (Personally I usually go way overkill with industrial zones and spaceports in my science games so I just turbo laser stations and win in a few turns – though I'm sure that's not the optimal build.)

I'm a bit skeptical of victory through Era Score, though. I have a fair number of games where by the late game I'm easily chaining together successive golden ages and loyalty flipping entire neighboring civs almost by accident. Some mods make this a bit better balanced like 6T Grand Eras it's a bit harder to hit a golden age every era. But even then I think this would end up feeling pretty grindy.

I don't think it's possible to create a compelling endgame just by making the AI more competitive. Either the AI is too weak to keep pace and gets steamrolled by human players, or the AI is strong enough to keep pace, and then the endgame is an endless grind. To make endgame fun, it has to be about more than just blobbing or waiting for a number to grow big enough. It should involve choices that reflect what your civilization values and what it is willing to sacrifice. The endgame should be a satisfying and memorable conclusion to a story of how each different civ navigated the challenges of history.

My preference would be for a more dramatic narrative/event driven endgame. Just making stuff up off the top of my head, but for example, maybe once you hit late game science the robots turn against you. There could be different paths to defeat the machine uprising. Maybe your society gets rid of technology and returns to the land, or maybe humans merge with the machines into cyborgs, or maybe you win a war to enslave the machines but leave your empire in ruins. There could be other similar scenarios depending on what type of empire you've built (e.g. culture, faith, gold, military), maybe even multiple different possible scenarios so not every science or culture game has the same one. Maybe there could be an endgame scenario like Dramatic Ages mode on steroids. (Too soon to have an endgame scenario about defeating a global pandemic?) There could even be civ specific interactions with endgame scenarios.

The challenge for designing this sort of the endgame system would be to avoid making it feel like a minigame where the early and midgame choices get erased. The endgame should still play out using the core civ mechanics. I haven't played too much of other 4x games, but I believe similar endgame mechanisms are in a few of them (I think Stellaris and Endless Space?).

 

[General_Sahib]

There are some common and understandable misconceptions about artificial intelligence in this thread. There's a trend of equating AI with machine learning, which I think is unfortunate as it misleads people to think that somehow results from one application of ML are relevant to a completely different situation. We're probably many, many, many years away from an AI that can correctly tell us what will happen when you put an ice cube on a stove AND play Civ as well as expert human players. Successful ML applications today are much more limited in terms of what they can do. Also, the article mentions that "complex" AI models aren't smarter than fifth graders, but from my experience, complexity of a model has very little to do with how well it works, and if anything, simpler models generally work better. In fact, we have many problems in our world, for which ML is just an inappropriate solution, and you're much better off trying to solve them with less fancy methods. Sure, there are problems that can be solved well without ML but can be done better with it, and those that are infeasible to solve without ML, but AI or ML doesn't give you a one-size-fits-all solution to all problems. I think of ML as a tool, like a power drill, that can be very useful and sometimes clearly superior to a simpler tool like a screwdriver, but you wouldn't use it to eat a bowl of cereal, because a dumb spoon will do a much better job.

For a game as complex as Civ, a competent AI will probably have several different components, some of which may be implemented with ML. Maybe there are some lessons from successful ML-based methods from other strategy games like chess, go and Starcraft that can be applied to Civ as well, but Civ is different enough from other games that it will probably require a monumental effort from the developers to create a highly competent, ML-based method. Civ is played on a much larger map than chess and go. It's played by more than two players, and each player can make an unbounded number of moves per turn. You're not privy to all information on the map. Getting more information isn't free, so you need to balance information-seeking with other things you need to do in the game. Even if you were able to get all the information there is about a game, how much of it do you really care about? In some ways, you have to play the game as if you're the only one. Each game of Civ is meant to be unique. Maps are generated randomly. There are ~50 leaders in the game. You can even play with a different set of rules every time, and the next iteration of the game will have a complete different set of rules. There's no way Firaxis can collect enough game data that can be useful for training an AI before the game is even released. Many of the questions the devs will have to answer still more or less belong on the cutting edge of the field of AI.

Besides, a human-like AI won't necessarily make the game fun. Some people might be okay with it, but I think it'd be very controversial to let the AI declare surprise wars just to steal settlers and to move their units in a way to deliberately block other players' units. I think Firaxis has done a very good job of making the AI feel like AI, and I don't mean that in a sarcastic way. It's good that the each leader has their own set of agendas and that they play the game "honestly" without trying to be extremely exploitative like humans can be.

Having said that, I do think there's room for improvement, and a (not perfect) solution doesn't have to involve ML. The game could be designed to be more AI-friendly without being unfun for humans to play. They could remove some traps the AI is prone to fall into. Trading with human players is a good example of this. AI constantly engages in terrible deals, and it's widely accepted that if you want to play well on a higher difficulty, you need to learn how to exploit this weakness, giving away your citrus for 10 god per turn to an AI player or get three other luxury resources for may 5 gold per turn. Trying to figure out how to best fleece the AI has never been fun for me, but I do it because I might as well. Instead of allowing players to trade directly with each other, there can be a marketplace where trading occurs between each player and the "game master", who sets the price for everything, and there's no room for negotiation. Less room for AI to do something stupid, and less tedious for humans. Win win if you ask me.

Another way I think the game could be more accommodating to the AI is if it abandoned binary victory conditions in favour of a victory point system. This is just a hunch that I have, but I think the AI will tend to do better in a victory point system, where it just needs to figure out relatively short series of conversions (e.g. x food > y production > z culture > w victory points), each leading to a small number of points toward victory, rather than figure out how to send people to Mars through much longer-term planning. Besides, religious and diplomatic victories are just terrible in my opinion, and if the devs would just move on from VCs, they could get rid of features like religious combat and to "vote with the herd for victory points" mechanic.

Great suggestions here. Your point about machine learning (vs "artificial intelligence") is very informative. Surely a simple machine learning algorithm is well within reach for the computer opponents, such that over the course of a single game, the machine would learn to not be suicidal? Surely the minimum possible/reasonable machine evolution over the course of a single game should entail an analysis of how aggressive the human player is, forcing the machine to divert its energies from planning a rush for spaceports commencing in the classical era into at least attempting to eradicate the one major threat in the game - the human player? If I have surprise dow'ed 2 out of 8 opponents and not stopped until eradicating them from the game, I would like the algorithm to take note and adjust battlefield tactics and objectives, not just the usual predictable standard "retaliation" of calling a summit and going on a doomed quest to liberate one capital that I took over, which only leads to me having to ransack another capital slightly earlier than previously planned.

I still know very little about the tech behind AI and ML but I would assume that ML at the very least entails spotting a pattern (i.e. that of the human being aggressive against civs, surprise dow'ing, ignoring requests for peace with a lightning quick rejection click) and then acting in a new way on the basis of that pattern.

 

[Solver]

Machine learning in its current shape is irrelevant to Civ games. ML is good at getting AIs to perform a certain task given many - millions and millions - of attempts. In a game like Civ, just coding a few rules into the AI does the same thing. As a silly example, Civ6 has cavalry units and anti-cavalry units. Yes, you could use ML to observe many battles and have it eventually figure out that Horsemen are very poor against Spearmen. But why bother with that when you have designers who already know this and can tell the AI to look at the strength modifiers before it attacks?

So many players still say they want a challenging, intelligent AI. When the people on this forum say it, I believe them! But that's not what most people want. People want an AI that's fun to play against and that poses a moderate challenge but doesn't win the game. People want an AI that does things to remind the player that it's playing the same game. For many players, it's more important that the AI show up with trade proposals than the AI be able to pull off a good attack.

I also bring up stats every time this topic comes up. In Civ6, the lower difficulty levels are by far the most popular. Prince is the default difficulty, with slight AI bonuses. The difficulties below Prince give the player bonuses. Well, Prince is over twice as common as King! Some 18% of those who play to completion don't go above Settler, the "sandbox beat up the AI" difficulty. That is over twice as popular as Deity then. I'm sure most people here have beaten Deity, and that many here don't even find that much of a challenge.

The entire machine learning idea is a red herring. To make an AI that's more likely to win - or make the player lose - there's plenty of other changes that could be implemented, and I guarantee that most players would hate them. An AI that plays to win will declare war to grab any unprotected Settlers or Workers. It would consistently declare war if you're fighting someone on the opposite side of your empire. It would never trade any strategic resource to you, etc etc. If a majority of players wanted an AI that challenges for victory, it wouldn't be accomplished through ML.

 

[aieeegrunt]

Machine learning in its current shape is irrelevant to Civ games. ML is good at getting AIs to perform a certain task given many - millions and millions - of attempts. In a game like Civ, just coding a few rules into the AI does the same thing. As a silly example, Civ6 has cavalry units and anti-cavalry units. Yes, you could use ML to observe many battles and have it eventually figure out that Horsemen are very poor against Spearmen. But why bother with that when you have designers who already know this and can tell the AI to look at the strength modifiers before it attacks?

So many players still say they want a challenging, intelligent AI. When the people on this forum say it, I believe them! But that's not what most people want. People want an AI that's fun to play against and that poses a moderate challenge but doesn't win the game. People want an AI that does things to remind the player that it's playing the same game. For many players, it's more important that the AI show up with trade proposals than the AI be able to pull off a good attack.

I also bring up stats every time this topic comes up. In Civ6, the lower difficulty levels are by far the most popular. Prince is the default difficulty, with slight AI bonuses. The difficulties below Prince give the player bonuses. Well, Prince is over twice as common as King! Some 18% of those who play to completion don't go above Settler, the "sandbox beat up the AI" difficulty. That is over twice as popular as Deity then. I'm sure most people here have beaten Deity, and that many here don't even find that much of a challenge.

The entire machine learning idea is a red herring. To make an AI that's more likely to win - or make the player lose - there's plenty of other changes that could be implemented, and I guarantee that most players would hate them. An AI that plays to win will declare war to grab any unprotected Settlers or Workers. It would consistently declare war if you're fighting someone on the opposite side of your empire. It would never trade any strategic resource to you, etc etc. If a majority of players wanted an AI that challenges for victory, it wouldn't be accomplished through ML.

I would be very happy with all of the above, and I don’t see why that would make people hate the game. AI in previous Civ titles would do some or all of the above, particularly the “grab settler” one; that was almost an exploit, as if you wanted an AI to attack you, you dangled a settler/worker as bait

 

[Solver]

Most players hate that kind of thing. On a hardcore player forum like here, sure, you'll have plenty of complains saying the AI is too poor. But overall, there's many more complaints about the AI not being "fair" and these complaints often come from the AI doing smart things. The classic example being Civ3/Civ4 tech trading, the AIs would effectively trade techs among themselves so people felt it was unfair as all AIs ended up at the same tech level. Back before Civ5, when cities required a unit to be undefended, people complained if the AI just walked into undefended cities.

This isn't a problem that has a solution. AI behavior in a civ-like game is a scale. On one end of the scale, you have a decorative AI that exists to provide some personality and the appearance of having opponents, even though the AI isn't trying to win and performs on the level of somebody just learning the basics. This is the Settler difficulty. On the other end of the scale, you have a ruthless AI that mainly tries to prevent you from winning. That means ganging up on you, that means using "dirty tricks" like grabbing unprotected units, that means launching an all-out desperate assault if you're getting close to victory. This provides a maximum challenge but sacrifices some Civ mechanics. Most obviously diplomacy - if the AIs always attack once you start the Apollo Program (or whatever threshold), then it's impossible to keep peace through diplomacy. Such a ruthless AI also has to sacrifice personality, there can be no range of personalities if they're supposed to do the same thing in the late game.

I believe the best we can do is to give the player some control of how the AI behaves. Civ4 and Old World go beyond traditional difficulty levels, they have explicit behavior switches to tweak where on this scale you want to play. You can tell the Civ4 AI to be ruthless, in which case it will, for instance, poison water supply when it can, a mission the AI normally doesn't do because it frustrates players. Or you can tell the Old World AI to be ruthless, in which case it will grab a city site you leave undefended. Both games also support a sandbox AI toggle where the AI stops providing a challenge and is guaranteed not to declare war.