Basic question re AI: RTS versus TBS

[Midgardsormr]

Firaxis does not really want the AI to win per se, they want the AI to play make believe, ie act like the personality of the civ leader. The purpose of the AI in civ6 is simply to give the player a "big personality" to interact with so that they are not playing by themselves.

The trained neural net wouldn't necessarily have to be the only AI system. The current AI could be used for policy decisions—when to go to war, which cities to attack, what VC to pursue, how to respond to player actions; and a neural net could be used for combat decisions, which aren't generally influenced by leader personality, anyway. And the combat system has far fewer variables than the overall game, so training the AI would be inexpensive in comparison to building one that would play the entire game. Maybe still more expensive than Firaxis might be open too, but not as costly as training on the entire game. It would probably be quite an overhaul of the game, though—getting a neural net to play a game is an entirely different thing from building it into the distributable code.

As previously mentioned, a neural net could be trained to a specified level of competence then halted. It might even be possible to train several on different games, resulting in multiple different play-styles that could be assigned to different civs, or perhaps randomized, so the player wouldn't immediately be able to guess what strategy the AI would use.

 

[Biz_]

RTS AI is usually not programmed to examine your scenario of endless amounts of possible moves every second. there's usually lots of planning and sequencing and scheduling of various things. it's not programmed like chess where it can just try every possible combination of commands and pick the best one

for example, deciding to reposition a damaged unit in a battle is an independent decision from deciding what to spend money on next which is an independent decision from micromanaging a probe to make the next supply pylon. you can make all of those decisions separately and still arrive at reasonable AI behavior

let's say there are 5 million possible decisions for each category. the AI has enough time to look at 5 million x 3 moves, but it's not going to look at 5 million ^ 3 moves

to answer the second part, this custom starcraft AI got trained with lots of machine learning. it looked at and played a lot of identical scenarios over and over again to find the best strategies. the way their AI gets better only works for perfectly symmetrical games. it can basically just say "pit 2 strategies against each other. pick the one that wins. repeat 1 billion times". their access to resources and units is perfectly symmetrical. the maps are all perfectly mirrored. it's literally a game of "anything you can do i can do better"

this technique doesn't work for deep strategy games that use random maps and require exploration and have asymmetrical access to resources and units. you can't just memorize patterns and build orders and expect the same level of success.

 

[Archon_Wing]

lol but despite many years into Starcraft 1 research, no bot even remotely touches any of the pros.

I've always thought Starcraft 2 was much more of a shallow game. (still fun) Technically speaking it's hard and has no margin of error, but strategy wasn't much. Bots would have the edge when they can have 10000 apm.

And funny enough Blizzard themselves make terrible AI, since their games are meant for multiplayer.

 

[CornPlanter]

I am confused about how an AI exists that is able to beat programmers in an RTS game involving an infinite number of possibilities and choices that have to be made every nanosecond, when the same AI is unable to play a TBS involving a smaller number of choices (granted, still a lot), on less of a compressed timetable.

What are you taking about, unable to play a TBS? The same team made AI (or probably its even the same AI with some adjustments) that absolutely destroyed best players in Chess and Go, turn based games.

 

[Disgustipated]

Bots would have the edge when they can have 10000 apm.

Although in the example in the OP, the bot APM was limited to that of top players.

 

[Archon_Wing]

Oh dang reading is hard. Well, AI doesn't have to deal with wrong presses of key.

 

[Victoria]

The larger the possibilities and complexity, the more chance Neural stuff will whip your ass.

With civ quite a lot of it is assessing what you cannot see and having memory.

 

[uppi]

RTS AI is usually not programmed to examine your scenario of endless amounts of possible moves every second. there's usually lots of planning and sequencing and scheduling of various things. it's not programmed like chess where it can just try every possible combination of commands and pick the best one

for example, deciding to reposition a damaged unit in a battle is an independent decision from deciding what to spend money on next which is an independent decision from micromanaging a probe to make the next supply pylon. you can make all of those decisions separately and still arrive at reasonable AI behavior

let's say there are 5 million possible decisions for each category. the AI has enough time to look at 5 million x 3 moves, but it's not going to look at 5 million ^ 3 moves

to answer the second part, this custom starcraft AI got trained with lots of machine learning. it looked at and played a lot of identical scenarios over and over again to find the best strategies. the way their AI gets better only works for perfectly symmetrical games. it can basically just say "pit 2 strategies against each other. pick the one that wins. repeat 1 billion times". their access to resources and units is perfectly symmetrical. the maps are all perfectly mirrored. it's literally a game of "anything you can do i can do better"

this technique doesn't work for deep strategy games that use random maps and require exploration and have asymmetrical access to resources and units. you can't just memorize patterns and build orders and expect the same level of success.

The limitations you mention are true of the AlphaStar AI for now, but there is no fundamental reason that it cannot overcome these limitations. If development on this continues at the pace it has, it will be able to handle any map and any matchup very soon. There are neural networks that are very good at image recognition and interpreting a random map is pretty much the same task. There is no reason that you couldn't teach an AI exploring and exploiting a random map, except for the required time, resources, and data.

 

[_hero_]

The fact that it is an RTS is a major reason why the AI is better. If you think about what machines can do better than people, processing large amounts of information in a short time and responsiveness are probably the top 2.

Compare combat in an rts vs tbs as a player. In an rts, it is essential that you act quickly. A high number of actions per minute is considered one of the key things that separates pro players from the rest. An AI can always top even the best player in actions per minute by a large margin.

In a tbs, you can take all the time you need to figure out what the best course of action would be. In Civ6 if you want to attack the enemy you can even get a preview of the expected outcome for each possible target. In other words, the player has the time to figure out the optimum strategy negating both the AIs responsiveness and it's ability to take in large sums of information.

 

[Kaan Boztepe]

for the deepmind AI they programmed it to be about half as fast as the human players in terms of mouse/keyboard clicks per minute. the main difference in my opinion is that the RTS have 1 singular victory condition with 1 clear path to achieve it ( domination/built army ) . a 4x game has a lot more different viable strategies that each need to be evaluated and defended against. Thus the AI is having a hard time adjusting.
I do see similar behaviour from all 4x games I played that the AI would not commit to an attack , i.e. keep recalling army for any number of reasons. ( seeing a barbarian scout / missionary incoming / a near city state vulnerable ). I believe the lost oppurtunity cost of moving the army around without any gain is ultimately the greatest failing of the AIs for the 4x genre.

 

[civplay]

DeepMind defeated Starcraft pros 10-1 (with some debatable cheating, but still impressive). So how long do you think until we get a DeepMind level AI for Civ? I would love a civ game where the hardest difficulty, the computer is unbeatable, but without any cheat.

 

[Pistol90]

I belive first step would be some kind of framework for AI in games for generic use. Then it would be started to be implemented by devs. But I dont belive we will see that kind of AI in Civ7 ... it is still 6 or 7 years earliest for serious neural networks in gaming

 

[bbbt]

When civ is played primarily online and unmodded?

They've had Hiro AI for FreeCiv for years.

Moderator Action: One topic, one thread. Threads merged. -- Browd

 

[Unconquered Sun]

DeepMind defeated Starcraft pros 10-1 (with some debatable cheating, but still impressive). So how long do you think until we get a DeepMind level AI for Civ? I would love a civ game where the hardest difficulty, the computer is unbeatable, but without any cheat.

From what I know about DeepMind's process of "getting good", never.

 

[Naeshar]

I agree with the posters above me.
To answer the OP's issues:

Why is it so hard to make a good Civ 6 AI?

It's probably not. 4x like Civ6 is a different type of game from Chess, Go and StarCraft2. Not really more complex, but has different goals. There is one in all the three: To beat your opponent to crumbles. War is the most powerful tool in Civ6 as well, so you ask for a combat AI unlike the one we have. Sure it might be possible and perhaps easy to beat human in 1v1 matchup... fun? Not so much. I don't think Civ6 is about player beating the AI or vice versa. Usually there are multiple AI players and multiple victory conditions. I can imagine if you set all the AIs to same level, they could beat a human player out of the game and continue fighting themselves until the one with best starting conditions prevails or an equilibrium establishes, where sides change whenever one civ starts to prevail... Options are endless, and the game would be eventually won by science.

Secondly, companies making games want to make profit. What is the sense in creating an unbeatable AI to a product. Is there any satisfaction in buying a game that has unbeatable AI? Why bother if you can make the AI look sensible enough and allow the player to win if he tries hard enough? Why bother with a precise AI if the majority (there are not so many Deity players) does not beat the current higher levels constantly? Imagine the AI company creating an AI that collectively stomps the player out of the game on Prince (even grounds). Is that such a great achievement? A 1v1 fight is not a Civ representative game. Alternatively, is it such an achievement to show an AI that can consistently keep the player at bay and win by collecting GWAMs after a long game? I think not... neither do the AI companies striving for achievement, popularity and funding.

tl;dr: I think it would not be hard, but would it be worth it?

 

[birbas12]

AlphaStar (the SC 2 AI in the news item) is an advanced self-learning network that has learned to play basically by playing against itself for I don't know how many thousands of hours. Firaxis uses an old-school manually coded system that consists of local rules, some global decision heuristics and adds some probability into the mix. In addition, its implementation is woefully inefficient (there is no reason why it should take some much times between turns) and - in my opinion of course — has been done very lazily.

If you want a more poetic comparison: AlphaStar is an autistic savant that relies on creativity in playing the game; Firaxis Civ 6 AI is a monkey (and not particularly smart one) that has been trained to respond to a bunch of lights and tries to grab a bunch of bananas by guessing the right order of button presses.

 

[iammaxhailme]

Humans are smarter than computers, but not as fast. That's why it's easy to make a challenging AI in a game like Supreme Commander, but it's hard in Civ; in Civ we have all the time we want to optimize.

 

[birbas12]

What is the sense in creating an unbeatable AI to a product. Is there any satisfaction in buying a game that has unbeatable AI?

You can make an unbeatable AI beatable by increasing its probability to make mistakes.

Why bother if you can make the AI look sensible enough and allow the player to win if he tries hard enough?

That is indeed the reason why games still rely on rule-based AI systems. However, Civ 6 AI doesn't look sensible. It's not about winning or losing either. For me, for instance, there is little fun in playing ion higher difficulty because the AI is not smarter or more challenging, it just outproduces you. You are not being outplayed by the AI, you are being outplayed solely by your own mistakes. With the current AI, single player games are boring and uninspiring on lower difficulty levels, but annoying and unfair at higher levels.

The AI doesn't have to be unbeatable, but it should be FUN to play against.

 

[SupremacyKing2]

Humans are smarter than computers, but not as fast. That's why it's easy to make a challenging AI in a game like Supreme Commander, but it's hard in Civ; in Civ we have all the time we want to optimize.

True. It is also worth noting that RTS games like Supreme Commander and Starcraft are entirely about destroying the other player's base. The only thing the AI really needs to be able to do is collect resources, build units and rush you. There is no empire building or culture victory or science victory like there is in a civ game. So in Supreme Commander, which I've played, the easy AI will have a small number of units and wait awhile before attacking you whereas the hard AI will build a lot of units and attack you right away in force. That's basically the only real difference you need in a RTS AI to have different difficulty levels. And in my experience, the AI in Supreme Commander is beatable if you are able to survive the early rush. But in a game like civ, the AI needs to be able to do much more than just rush you in the early game. It needs to do science, culture, expansion, build wonders etc...

 

[kryat]

The other thing I’ll add is that Starcraft is actually a much simpler game than Civ 6 because it has a simpler ruleset. Many RTS games boil down to a basic rock-paper-scissors mechanic in combat, and don’t have to think that hard about non-combat actions. Civ is more complex because there’s more things to do than advance troops on an opponent. Experienced humans are good at narrowing the list of possible actions to a smaller subset that makes most long-term strategic sense.