AI gets better at playing Civ2 by reading the manual

[adept42]

I don't usually post here, but this was just too good not to share:

Computers are great at treating words as data: Word-processing programs let you rearrange and format text however you like, and search engines can quickly find a word anywhere on the Web. But what would it mean for a computer to actually understand the meaning of a sentence written in ordinary English -- or French, or Urdu, or Mandarin?

One test might be whether the computer could analyze and follow a set of instructions for an unfamiliar task. And indeed, in the last few years, researchers at MIT's Computer Science and Artificial Intelligence Lab have begun designing machine-learning systems that do exactly that, with surprisingly good results.

In 2009, at the annual meeting of the Association for Computational Linguistics (ACL), researchers in the lab of Regina Barzilay, associate professor of computer science and electrical engineering, took the best-paper award for a system that generated scripts for installing a piece of software on a Windows computer by reviewing instructions posted on Microsoft's help site. At this year's ACL meeting, Barzilay, her graduate student S. R. K. Branavan and David Silver of University College London applied a similar approach to a more complicated problem: learning to play "Civilization," a computer game in which the player guides the development of a city into an empire across centuries of human history. When the researchers augmented a machine-learning system so that it could use a player's manual to guide the development of a game-playing strategy, its rate of victory jumped from 46 percent to 79 percent.

As revealed by the full academic paper, they taught their AI to play Civ2.

It sounds like this technique could help any game designer build a better AI for their game. The next step would probably be an AI that can "read" a much longer text for tips to improve it's game play... like a web forum. After all, if you can learn from the accumulated wisdom of other players, why shouldn't the AI get the same advantage?

 

[Misterboy]

Just saw this on Slashdot. Was going to post it, but you beat me to it

 

[Ali Ardavan]

Very interesting indeed. Thanks for posting it.

 

[starlifter]

Its interesting that some key files are "not available" from the MIT site, to examine exactly what they had done. Its a "step forward" in A.I., but the underlying mechanism is not much different that what I've seen/done in academic & gov't work in "A.I." since as early as 1987. What makes it interesting is to see exactly what they did to control the game, and how the rules were selected from the associations. Civ II can have great variations in the outcome of even the same start position; certain other things can enhance or reduce subsequent game effects.

Choosing the site of a village, for example, based on adding a new "rule" (e.g., build on Grassland, Plains, or river) would be interesting. Would it build on a Mountain River, as coded?

The improvement, would need many "runs" averaged out, to gain meaningful results. If one or two "runs" were done, and the first gave one result... then a subsequent gave a different result when the "manual" was used, this would not be enough to make a conclusion.

What's important is how the game's decision tree is positively altered, without creating a catastrophic mistake along the way.


One of the fundamental issues with Human or AI, is in flight. Imagine 2 pilots. One average, and one "hot shot". The Average pilot makes neither spectacular maneuvers (e.g., exact turns, perfect landings, precise altitudes +/- 1 foot, etc.), but the "hot shot" is God's Gift to Piloting... right up to the point he flew his aircraft into the mountain. A.I. and Human I. are not dissimilar. One may be 20% "better", but do you want to board an Airline when your pilot announces "I'm 50% better than the rest, and its only a 1 in 1000 chance I'll get you all killed", or "We may have a bumpy landing, but we're going to arrive alive 100.0%".

In Civ, what level were they playing? What kind of Map? Victory Conditions? Input & control mechanism (e.g., their A.I.'s "awareness" of the full game state space)?

So, the interesting part is not in the "claim", but in how the "claim" was determined & computed, and moreover (as science), if it can be repeated and validated.


If nothing else, I'd just like to see the code, constraints, full data sets & how they controlled The Game .