Deep Learning and Civilization

[sossos]

I have been both a civilization player and go player for years. I was also playing chess, Heros of Might and Magic, StarCraft, Warcraft and Age of Empire series.

The news that Alpha go beat the top human player has made me think about whether the techniques of deep learning can be applied to civilizations. When we are arguing which strategy is the best, can anybody prove that his strategy is the best? Probably not. This is similar to the situation in game go.

This is a start and I wonder whether there's anybody has the same interest. I'm afraid I have no enough time to realize machine learning in civilization by myself. However, if there're so many fanatics in this forum, can we do it together?

 

[sampsa]

Interesting subject, especially since with some chess background I would guess the level of the AI in this game is somewhere in the neighborhood of 1000 elo points. Yes, that bad, really. That being said, chess is probably a much easier game for an AI than go or civ4.

Can't really contribute anything meaningful for the actual subject, but I will follow the thread with interest.

 

[elitetroops]

I think there is a lot of areas in Civilization where the computer could outperform the human, mainly because lots of it is math. City micro, whip timing and so on is all math. A computer could go through all possibilities to find the fastest way to achieve it's goals. The further ahead these goals are, the better it would be. I'd assume modern chess and go computers are already looking quite a few moves ahead.

Fighting stack vs stacks is also a really complicated mathematical situation. The possible outcomes are so many that it's not practical for a human player to calculate the odds for stack A beating stack B, and how these odds would change if you use one more catapult for bombardment, or send in a C1 unit before a C2 unit at 1% lower odds. A computer could do these calculations and always choose the best time to attack and the optimal attacking order. Maybe the amount of calculations gets a bit out of hand with larger stacks though...

However, while a computer could calculate the best possible way to reach it's goals, it's a lot harder to define what those goals should be. In this area I think the human player will always have an advantage over even the most sophisticated AI. At least for as long as Civ 4 is still relevant.

 

[Metaslab]

I think the thing that makes Alphago's recent victories significant is the fact that contemporary machine-learning techniques are beginning to be able to define goals. When Deep Blue beat Kasparov at chess, this was much more to do with brute-force computing power, something like the stack calculations you mention. Go - and I imagine Civ - present vastly more possible game-states than chess, so whereas it was plausible for a chess AI to simply run through every potential permutation many moves ahead and arrive at a fairly definite best path, the branching possibilities in very complex games soon create more decision trees than even modern computers can efficiently process.

The nearest I have to any specialist knowledge on this topic is that a few years ago I worked on a project with a UK university where I teamed up with an AI researcher who gave me a basic introduction to neural networks that I then turned into a comic strip aimed at 7-11 year olds. Neural networks are much closer to a human process of trial and error: the programmer sets a few variables and conditions by which the AI can measure its success, and then leaves it to run simulations. The AI arrives at something that "works", but isn't expected to be able to account for every possible outcome. So regarding sossos' question about proving that solution X is the best, I don't think Alphago would be able to, at least not in the sense of a mathematical proof, nor is it intended to; but it can say, "this works".

 

[elitetroops]

There's also 2 different ways to look at this:

a) Can you make an AI that can beat the human player at equal terms, like in a multiplayer scenario where the AI has no special bonuses over the human player.

b) Can you make an AI that could beat this game from a single player perspective more efficiently than a human player could. As in give the same start to a strong human player and an AI controlled player, both go up against normal deity AI and have to play that start out to a Space Race victory as fast as possible.

I think the latter would be more easily achieved, as you can program the AI to perfectly exploit current AI behavior. It would know all the code and know exactly how the other AI in the game would react to it's actions. Making an AI capable of predicting human actions is a lot harder.

 

[elmurcis]

Just a tiny tip-->Copy Seraiel mind /Half joke, half serious thing... if computer will manage to collect info from actual civ player... oh my.... civilization will get in huge trouble... and I don't talk about game../

 

[elitetroops]

Just a tiny tip-->Copy Seraiel mind

Are you assuming that Seraiel is actually human..?

Spoiler :

 

[Metaslab]

I think it was misdirection to mask the fact that elmurcis is actually a malevolent AI out to destroy us meatbags

 

[elmurcis]

Are you assuming that Seraiel is actually human..?

Spoiler :

Spoiler :

Actually I have no proof that would exclude this possibility. Even if he has human body, he might have some mixed human-AI mind Or even me... or you... because we are somehow like AI , just from bio-material - we learn how to live longer (so we have more time to use things that decrease our lifetime ), have better houses (so it takes longer time to rebuild towns after they burn down), techs etc. and ofcourse, our main advantage over animals: how to kill better. For example itts hard to believe but thats what made Metal Casting necessary (that pretty expensive tech for early game ).. You don't need great metal for hunting animals (cheap tech) withs spears... or even for plow (need good but not excellent iron). But when its time to kill neighbours, you want to get that edge advantage. And here comes sharper swords, better catapults, faster horses and Stealth Fighters (Ninjas I mean)
So yeah, as soon as Ai will get it that limited (these guys and girls cost big money) but superb power military (build for specific enemy) is what wins game human is dead.. well, atleast keyboard or monitor

 

[Noble Zarkon]

I'm fascinated by this topic as I work for IBM and we are investing a lot of effort in this area. Will see if we have any tools we can bring to bear on this.

 

[Metaslab]

Very interesting! Will look forward to hearing any thoughts you have.

 

[pigswill]

I wonder how an AI would cope with RNG/uncertain outcomes which aren't present in stuff like chess and go.

 

[DrCron]

I think there is a lot of areas in Civilization where the computer could outperform the human, mainly because lots of it is math. City micro, whip timing and so on is all math. A computer could go through all possibilities to find the fastest way to achieve it's goals. The further ahead these goals are, the better it would be. I'd assume modern chess and go computers are already looking quite a few moves ahead.

Fighting stack vs stacks is also a really complicated mathematical situation. The possible outcomes are so many that it's not practical for a human player to calculate the odds for stack A beating stack B, and how these odds would change if you use one more catapult for bombardment, or send in a C1 unit before a C2 unit at 1% lower odds. A computer could do these calculations and always choose the best time to attack and the optimal attacking order. Maybe the amount of calculations gets a bit out of hand with larger stacks though...

However, while a computer could calculate the best possible way to reach it's goals, it's a lot harder to define what those goals should be. In this area I think the human player will always have an advantage over even the most sophisticated AI. At least for as long as Civ 4 is still relevant.

I agree with your first 2 paragraphs and disagree with the last one. Let's face it, Civ (even 4, which is the hardest) is definitely easier than both chess (even if CIV has more possible outcomes) and go.

Calculating what the goals should be in Civ 4 isn't so hard. In fact, the calculations we use to decide our preferred victory could be programmed into an AI. For example:

- If enough land for peaceful rex, UB with culture bonus, traits like industrious or philosophical, 2 rels spread to our land by rivals... we can go for cultural peaceful victory.

- If not enough land for decent rex, we can go for an early rush (easy for AI to calculate which one is most optimal due to attack values, tech situation, resources, etc).

- If good land for 7 cities, fin leader, horses available, peaceful neighbors... peaceful race to lib and then cuirs attack. Go for domination, or diplo with several vassals if UN can be reached before rivals.

These are just a few examples and I'm not a deity player so my reasoning should have room for improvement, but I just want to illustrate that it's really not so hard to decide which way to go based on your situation. And if you consider that Deep Learning is a computer that teaches itself, endlessly, it would rule these variable in no time.

 

[elitetroops]

These are just a few examples and I'm not a deity player so my reasoning should have room for improvement, but I just want to illustrate that it's really not so hard to decide which way to go based on your situation. And if you consider that Deep Learning is a computer that teaches itself, endlessly, it would rule these variable in no time.

Actually, it is a lot harder than that. The examples you are listing are basic strategies recommended to beginners, because they are safe bets against current AI. In terms of victory date they are far from optimal. If you add in strong human players as opponents, I'd assume they'd be straight out horrible (never played MP though). If you want to compete with the best of the best, it becomes a lot more complex than that.

As you can see in the ongoing G-Major thread in the HoF forums, even the most experienced players are still contemplating such basic issues as "is it worth getting an Academy in capital when aiming for fastest possible Sushi". The inexperienced players would probably say absolutely, but with players like WastinTime doubting this, it seems that it's not such a straight forward decision. (Note that the inexperienced players are most likely right, as it would help them. But if the rest of your game is maximally optimized, then it might not fit into the plan anymore.)

 

[DrCron]

If you add in strong human players as opponents, I'd assume they'd be straight out horrible (never played MP though).

(..)

even the most experienced players are still contemplating such basic issues as "is it worth getting an Academy in capital when aiming for fastest possible Sushi".

Regarding the 1st point, I was thinking single player only in my examples. MP I guess you should consider war much more, much earlier, and completely crazy diplo. Either way, it can still be calculated. And I think a really smart AI would always win wars against human players because of what you pointed out regarding stack optimization.

Regarding the 2nd point, that particular example I'm pretty sure can be calculated for each game, if you have unlimited calculation skill and the fastest AI speed. And regarding optimizing time in general, the AI could just decide a specific victory from turn 0 (as we humans do in many of our games anyway) and play accordingly.

I really see no reason to believe an AI like this one wouldn't be exponentially better than any human player could dream of. I mean, imagine the precision to which it can optimize every worker turn, every city settling, every turn micro-management, every fail-gold (based on each rival AI tendency to build each specific wonder, which it would immediately know), every.... everything. Seriously, with unlimited game variables knowledge and calculation capacity, plus perfect memory, it would be way too easy.

 

[Pangaea]

The AI could be greatly improved, and with the bonuses on the higher levels, be impossible to beat for the human. But there are things that are hard for an AI to get truly good at, like strategic thinking and planning. Put it this way, chess has 32 moving parts, and very quickly much fewer than that, and very clear rules and outcomes. Civ 4 isn't like that, not even outcomes are certain. Would a chess player attack a pawn with the queen if the queen could lose? Maybe not!

 

[Anysense]

One of the biggest problems will be the distance between a decision and its consequences. Then usually decision cant be good or bad on itself because it is a part of a plan. How the AI will determine what it did wrong? Most humans can't learn such things as planning and analysis on their own. They learn it from other wiser humans.

 

[Pangaea]

Most humans can't learn such things as planning and analysis on their own. They learn it from other wiser Seraiels.

That's a fair point

 

[elitetroops]

DrCron, creating an AI that beats single player should be no problem, as I said already earlier. Give it the complete game code and it can always predict what every other AI will do.

But I think the real question in this thread is about an AI that can beat human players at equal terms. You say Civ is easier than Chess. I doubt it, not from an AI perspective. The amount of possible game states in Civ are vastly greater than in chess. In addition, almost every decision is made based on incomplete information. You cannot see the entire board, nor can you know with complete accuracy the outcome of your actions. Some kind of brute force method of calculating all possibilities is well beyond the calculation capacity we have today. We are quite far from "unlimited calculation capacity".

The thing is, civ is not all about perfect micro and perfect decisions. There are no perfect decisions. And sometimes you should throw perfect micro out of the window if you have good reasons to do so. It's about seeing the big picture. In this area computers are still way behind humans. You can easily make a program that outperforms any human in math, but when it comes to separating cats from dogs, you will find it very hard to create a program that performs at the level of a 3-year old. I'm sure it's possible, probably done already, but it's just an example of how some things that are very easy and straight forward for the human mind become very complex problems to solve as a series of logical operators.

 

[slengtorm]

In addition, almost every decision is made based on incomplete information. You cannot see the entire board, nor can you know with complete accuracy the outcome of your actions.

I'm surprised it took so long for this point to be mentioned.

Really, two limitations are highlighted here. First, the lack of complete information in the present means that it is not easy to define goals, whereas in chess, Go, Arimaa this is rather simple. There are any number of situations in which, based on the information given, it would be perfectly rational and seemingly optimal for an AI to choose a particular goal, be it a victory condition or something more immediate, but unknown information would render that goal suboptimal.

Second, not knowing outcomes, whether influenced by unknown information or simply by the RNG, would seriously hamper an AI's calculations. Basically, what would need to happen would be for the AI to choose a course that, given the probabilities, would be most likely to be the optimal decision in the greatest number of situations.

In other words, I doubt even the assertion that it would be reasonably simple to develop an AI that would be vastly superior to a strong human player even in single player. (Measured in victory dates or some such metric.) We would see innumerable instances in which the AI, doing a single Deity run on a given map, would achieve an inferior victory date to the human player doing a single run, or even fail to win entirely. This would happen all the time because imperfect information would severely limit the AI's main weapon (number of calculations). Instead of doing what is necessarily best, the AI would do what, given hundreds of thousands of runs at this map, would produce the greatest number of early victory dates. Quite a different thing from succeeding in this one instance, really.