Real AI?

[Olleus]

i have an idea on how to make the ai a lot more intelligent. Right now the ai behaves in exactly the same way, how about making it learn from a game? say if it gets invaded and all its cities are captures, next game it will build a bigger better army, specialy more of the troops that captured its cities. If a civ looses to a cultural win then next game it will concentrate more on culture.
I don't know how to implement these ideas but i have a few ideas. It should be (technicaly) possible to make a script that looks into the game save files and termines which civs lost and how. Then using this information it should be able to alter the XML lines that say how likely it is to build some particular unit or building.

E.G.:
I play as the romans on a map against the aztecs and americans. Early on in the game my knights completely slaughter the aztec's macemen and towards the end of the game i beat the americans in a close fought comflict between tanks and infantry.

Once the game has finnished the 'inteligent AI' program looks at the review video (the thing at the end of the game that shows everything thats happened(forgoten the exact name)). It sees that it took me 10 turns to destroy the aztecs, from declaring war to capturing there last city, and sees that i built a lot of knights and them a lot of macemen. So it lowers the chance of ai building maceman and increases the chance of it building knights. It then looks at the war against America and sees it took 80 turns for them to die, judges this to be a long time and so alters nothing.


As i said i have no idea on how to make this work but i am (quite) sure that someone who knows a complex programming language could do it. This could be the start of AI that actualy learns?

 

[Kolyana]

Soren has actually been working on a personal project in which the AI learns from the player and adapts. I don't know the extent of this or what success he has had, but - yes - it would be nice to implement something like this, without a doubt.

AI in games has always fascinated me and way back in Civ 1 I was thinking of a method to enable to AI to learn from the player, adapt accordingly and develop new strategies.

It's a real shame that the gaming industry hasn't put more effort here.

 

[Psyringe]

"Learning AI" is an idea that circles around strategy game design for at least ten years now - unfortunately, no one has been able to put it to work yet, at least not in a game as complex as Civ4.

There are several problems involved, all of which are difficult to solve. Here are four:

- While it's relatively easy to determine whether a game was a success or a failure, it's quite hard to attribute this result to specific decision you made. For example, if your knights win you several battles, what's your conclusion? To build more knights next game? But they may have won only because your enemies were too backwards to field a strong army. Or your knights may have won because you used a culture bomb to strip your enemy of copper, so that he couldn't build spearmen to counter your knights. So what's your conclusion, generate more artists next game? Or build more knights? Or focus on stripping your enemy of his resources instead of fighting?

- An algorithm that has to do that much data processing will use lots of CPU power. Today's PCs might not even fast enough for its implementation in a game so complex as Civ4.

- Writing algorithms which can sensibly evaluate actions and there results in this way are so difficult to program that you need a lot more manpower o do so, which means you have more expenses.

- The algorithm can easily be thrown off by any player who wants to do so. For example, I start three games in which I never use spearmen to counter the AIs knights, and never use city defense promotions, and never attack knights at all. The AI will now conclude that knights are virtually invincible, and build lots of them. Now I start a "real" game on a high difficulty level, prepare for the knights with lots of spearmen, slaughter them, and afterwards boast in rthe forums that I won on a high difficulty level.

As you see, the idea itself isn't bad, but it's very, very hard to do it right.

 

[cleverhandle]

As i said i have no idea on how to make this work but i am (quite) sure that someone who knows a complex programming language could do it.

I would love my own little flying car, like in the Jetsons. I don't know anything about aeronautics or engineering, but I'm (quite) sure that someone can build me one, and furthermore that it will be cheap and fuel-efficient.

 

[Olleus]

point taken.
What if the player himself said what the AI had done wrong. Sure this could be abused but then players would be ruining there own game. Then when you said how you won this data could be sent to a central database where they would evaluate it and then send patches to improve the ai.

When you're evaluating how you won you would have a form where you would have to tick about 50 boxes saying what victory condition you won by and how you achieived it (build one unit in particular, strip civs of resources, monopolise on religion, ect...ect..) The central database would then see if their was one way with which a lot of people won (lets say beelining for gun powder) then it could give a higher priority for gun powder.
This would be semy automated learning but with all the complex things done by the player. Sure this would cost a bit but people might prefer buying a game where the AI is constantly progressing.

 

[Junuxx]

Player- or pc-evaluated is not the problem. The problem is the connection between the evaluation of a whole game and the mapping to crucial decisions. edit: Sorry, didn't read the whole thread. I see Psyringe remarked about exactly the same

The problem is not calculation time, If there's any game genre where learning techniques could be applied, it would be TBS.
The real problem is giving such a system enough training. What if training a bot decently takes thousands of example games, where every decision has to be rated somehow? That's a massive amount of work!

But, anyway, it's really not impossible. I am an AI student and I've seen learning techniques been applied succesfully in many fields. Just not in strategy gaming. Shooters like Farcry include learning opponents though (well, to some extent, but that's fine).

If there's a couple more AI people around here... I'd like to give it a try. An overhaul of Civ's AI system

 

[magerain]

I have a vague feeling that AI already learns, but it's learning is bound to one game. In several of my games, civilizations beaten up harshly by me, didn't retaliate when they had such chance, as though they were afraid .
That could be coincidence of course.

 

[arrummzen]

When you are programing a computer to play tick-tack-toe, where is the true challenge? Most people say that the challenge is for the computer to
chose its next move. This is wrong. Any simple search algorithm can find the best move out of the list of possible moves, the difficulty lies in
determining the value of the game sate after each move. ig. after putting a 'x' at [0,0] are we closer or farther from victory?

Why would you want an AI that "learns" the value of the knight? It already knows that (well it should, a table of unit powers and abilitys should
be hard coded into the AI, after all, if the player gets to know unit stats the AI should as well).

Just blindly selecting units based on passed experience will lead to a fall, because the next game will not be exactly the same as the last. You
must chose wich problems to solve with machine learning, and which ones to solve with a simple deterministic algorithm. See "Machine Learning" by
Tom. M. Mitchell.

"If a civ looses to a cultural win then next game it will concentrate more on culture."
Simply weighing move values is not very smart, I believe the game already knows that "If you chase two rabbits, you will lose them both". The AI
must have a clear path to victory, its moves must be made to reach that goal. If it thinks it can win a cultural victory then it should invest
more in culture, but it should not get a bunch of great artists if it thinks it can easily crush it's well cultured rich neighbors militarily. This
could be implemented using a HTN (Hierarchial Task Network). For example the root task would be "win", under it would be a set of nodes for each type of victory, "cultural", "domination", "diplomatic" etc. Under each of those there would be tasks such as "destroy other civilizations", "build units" "stop other players cultural victory" etc. Under "stop other players cultural victory" it would have steps to execute if it learns one player has two cities approaching legendary culture, such as "attempt to get other civilizations to attack cultural winning civ", "try to take a city with legendary culture" etc. See Neil Wallace's article on page 229 of "AI Game Programing Wisdom 2" for more information.

The AI you describe would be more like a child thinking "wow, these battle cruisers are powerful, I should build more of them next game" rather than a cunning and intelligent general.

Now we have an AI that thinks logically with a HTN, but it still doesn't have the ability to improve itself, its just a deterministic FSM (Finite State Machine). Next let the game the AI just played be a phenotype and the HTN it used to be its genotype. After each game, evolve the AIs using asexual (random mutations) or sexual (grab characteristics from two (or more) successfully AIs and "mix" them). You will now have a new AI with a new HTN to try in the next game (along with some successful AIs from previous games). See "Creative Evolutionary Systems" for more information.

Thank you for your time,
Arrummzen

 

[arrummzen]

“The problem is not calculation time, If there's any game genre where learning techniques could be applied, it would be TBS.”

A direct quote from Soren Johnson – Firaxis Games from “AI programing Wisdom II”.
“The most obvious challenge for a turn-based AI is that the user can have an infinite amount of time to conduct a turn. The player can conduct extensive micromanagement – and advantage not available in a real-time game. Unlike the human player, turn-based AI's game time is usually limited to a few seconds per tern. This time discrepancy encourages the most competitive players to spend as much time as possible micro-managing their empire to out-perform the AI.”.

It looks like you can Soren Johnson don't agree on this one. Note that he was talking about Civ III in this article, his opinion might have changed by now.

“- The algorithm can easily be thrown off by any player who wants to do so. For example, I start three games in which I never use spearmen to counter the AIs knights, and never use city defense promotions, and never attack knights at all. The AI will now conclude that knights are virtually invincible, and build lots of them. Now I start a "real" game on a high difficulty level, prepare for the knights with lots of spearmen, slaughter them, and afterwards boast in rthe forums that I won on a high difficulty level.”

Not as long as you play with more AI's than players. The AI will learn from other Artificial Intelligences. And improve accordingly. The game could also run “AI vs AI” matches while you are not playing (ig at work or while you are trying to decide what to do on your turn).

"What if the player himself said what the AI had done wrong. Sure this could be abused but then players would be ruining there own game. Then when you said how you won this data could be sent to a central database where they would evaluate it and then send patches to improve the ai."

This is done with some real world AIs. There are many examples of this being used in "Evolutionary Art" and "evolutionary music". An AI will write a some songs, and a human will provide feedback. For some tasks this has been shown to be more efficient than a human working alone, because the AI can quickly model many small details. The problem is, I don't know if you can get players to sit down and fill out the “how did I do” form accurately. It would be possible for a the AI to show a graphic of the HTN it used and then let the player rate each part, or even, build a new HTN from scratch.

The manual said they would be releasing an “AI SDK” in early 2006. I am looking forward to getting to play with it (or even matching my AI vs. someone else's AI).

Thank you for your time,
Arrummzen

 

[Eigenvector]

I presume you aren't speaking about a REAL AI, that is, one that doesn't require input from us because its capable of making its own decisions based on information that it learns either from itself or others. That is, the same way you and I do it.

Somehow I doubt Soren will be the man who ushers in the next great era of computer programming. As it stands the computer simulation routines in Civ 4 aren't all that advanced given the limited options it has to shoot for.

I like that quote from Sid on how the computer only has a few seconds to do its calculations. What a laugh, in electronic terms that close to a few hundred or thousand operations based on clock cycle frequency and how many ops per second your computer will support. Given that the leaders in the game all have specific goals they move toward based on their personality that means the computer really only has a very limited number of options in which to calculate success probability.

 

[Sorceresss]

Thank you for your time, Arrummzen

No, Sir : thank YOU for YOUR time.

Those two very informative & competent previous posts must have taken you some time & patient care to compose.

I am not a programmer, but I am very fascinated by A.I. design in computer games : from an epistemological perspective.

THXxx again : We want more!

P.S. I have copied your two posts in my WordPad to save them for future meditation.

 

[Fru]

Hi there, I've played all versions of CIV and love the series but the AI has always bemused me. I realise it is difficult and you mentioned man hours.

Would building a mod that strips away all but the essential elements of CIV make it easier to mess around with the AI? Something more along the lines of CIV 1. If more people were working on it comparing notes etc things could move forward at least a little at a time.

Even if someone made such a mod, is AI programing just too complex?

Also, it would be interesting to watch CIV in a kind of god mode where you could watch AI or bots duke it out.


I'm dreaming but I'm sure some of you guys will bring me back down to earth.

 

[GenericKen]

Arr> I think you speak too high-mindedly of civ's AI. The largest problem with it right now is that it only plays for score, and doesn't seem to be aware of the individual victory conditions at all.

An HTN would be an interesting approach (with Civ interpreted as a game of ordering), but I think you miss the point. If Civ implemented a perfect or near-perfect AI, the player would lose 7/8ths of the time. The real objective of game AIs is to avoid doing anything overtly stupid, and they key there is intelligent pruning.

The biggest problem with that, is that the gameplay of Civ4 is surely going to be changing with patches and the expansion, and it's thus difficult to determine what pruning is intelligent.

 

[Sorceresss]

If Civ implemented a perfect or near-perfect AI, the player would lose 7/8ths of the time.

If that "near-perfect AI" did not cheat and did not get special bonuses at the Noble level of difficulty (and below, of course), I would not mind losing 100% of the time...because I would have the satisfaction of being challenged (and somewhat educated) by an elite A.I., designed & coded by expert engineers.

 

[Fru]

Lets face it, we are playing a vastly complex game of chess here with Civilization. I can't remember who played the super computer at "Vanilla" Chess but I bet it was a close game.

Chess isn't broken but Civilization is. It's a case of too far too fast I'm afraid.

We are playing a game which is largely untested in human vs human terms against AI opponents! If thousands of people played this game multiplayer the rules/gameplay options would have to be changed extremely quickly to try to keep everyone happy. It's never going to happen so why do we torture ourselves like this?

Civilization is a great concept but its only fun until you take in all the variables and make more sense of it than the AI. Starting at a disadvantage may give some people kicks but nobody is really learning anything except overkill.

 

[Junuxx]

Good to see some AI expertise around! Are you interested in working on an AI mod, arrummzen?

When you are programing a computer to play tick-tack-toe, where is the true challenge? Most people say that the challenge is for the computer to
chose its next move. This is wrong. Any simple search algorithm can find the best move out of the list of possible moves, the difficulty lies in
determining the value of the game sate after each move. ig. after putting a 'x' at [0,0] are we closer or farther from victory? [/b]

Doesn't make much sense to me, really. Evaluating is an essential part of choosing. Choosing the next move is a challenge. True, once states are evaluated, choosing is easy. But that doesn't make "most people [that] say that the challenge is for the computer to ch[o]ose its next move" wrong...

Why would you want an AI that "learns" the value of the knight? It already knows that (well it should, a table of unit powers and abilitys should
be hard coded into the AI, after all, if the player gets to know unit stats the AI should as well).

The units strength and general purpose (Land domination and city attack, counter-melee) are probably hardcoded, but obviously not the best time to build them, the best units to combine them with, the optimal number to group them in, the optimal attack/heal/pillage cycle, et cetera. Tactics stuff.

See "Machine Learning" by
Tom. M. Mitchell.

Funny. I have a test on that book Tuesday . It's right here on my desk now, at Chapter 8, Instance-based learning

....

The AI you describe would be more like a child thinking "wow, these battle cruisers are powerful, I should build more of them next game" rather than a cunning and intelligent general.

I agree, mostly. But on the other hand, I think that even such an AI would completely crush the kind of players that use to stick to their old and tried strategies.

Now we have an AI that thinks logically with a HTN, but it still doesn't have the ability to improve itself, its just a deterministic FSM (Finite State Machine). Next let the game the AI just played be a phenotype and the HTN it used to be its genotype. After each game, evolve the AIs using asexual (random mutations) or sexual (grab characteristics from two (or more) successfully AIs and "mix" them). You will now have a new AI with a new HTN to try in the next game (along with some successful AIs from previous games). See "Creative Evolutionary Systems" for more information.

Thank you for your time,
Arrummzen

You mean essentially a weighted Finite State Automaton?
I'm not sure if that's enough to grasp the scope of Civ. I really think you need to do some feature and trend abstraction (of its own and the player's behavior) if you want the AI to show real planning.

 

[BlackMage]

I think that the flaws in the AI can be seen quite easily in how the difficulty levels pan out. Sure, the AI is better as the levels get harder, but their biggest advantage is how they start. More settlers, archers, things of that nature are a crutch so that the AI can get entrenched in a high score fast, so that the human has to race to catch up, rather than starting on the same level as the player and using superior strategy to win.

 

[Junuxx]

I have a vague feeling that AI already learns, but it's learning is bound to one game. In several of my games, civilizations beaten up harshly by me, didn't retaliate when they had such chance, as though they were afraid .
That could be coincidence of course.

That's the reputation system I guess. It's quite good. It has been in Civ (in some way) since Civ 2. In Civ 1, you could make war and peace on a rival civ as often as you wanted and they wouldn't remember. You could backstab them time and time again and they kept offering peace.

“The problem is not calculation time, If there's any game genre where learning techniques could be applied, it would be TBS.”

A direct quote from Soren Johnson – Firaxis Games from “AI programing Wisdom II”.
“The most obvious challenge for a turn-based AI is that the user can have an infinite amount of time to conduct a turn. The player can conduct extensive micromanagement – and advantage not available in a real-time game. Unlike the human player, turn-based AI's game time is usually limited to a few seconds per tern. This time discrepancy encourages the most competitive players to spend as much time as possible micro-managing their empire to out-perform the AI.”.

It looks like you can Soren Johnson don't agree on this one. Note that he was talking about Civ III in this article, his opinion might have changed by now.

He says that this is the most obvious challenge for TBS AI. He doesn't compare TBS AI to RTS AI. Consider the differences. Realize that many RTS games have (sort of) continuous time and space, in contrast to Civ's discrete turns and tiles. Discrete units are way, way easier to reason with. Also, Civ can afford a slowdown in the later game. A multiplayer shooter AI cannot!

Another important thing is than in non-tbs games, the player can study the map or level extensively, too. In Counterstrike, players know the levels like their own backyard. In a RTS game like Rise of Nations or Age of Empires, single player, lots of players press Pause all the time to decide what to do next (I do).

The manual said they would be releasing an “AI SDK” in early 2006. I am looking forward to getting to play with it (or even matching my AI vs. someone else's AI).

Did you say some else's AI? *Waves*
I hope you don't interpret my extensive quotes as offensive, they are not intended that way, I just err, disagreed
I'm looking forward to the SDK too

 

[Puzzlinon]

I have a suspicion that the AI might be made to play a more challenging game if it had more "traditional-AI" problem-solving structure, but at a big cost: it would be less fun to play. The AI right now feels pattern-based... I'd guess it's based on recognizing situations and using them to look-up overall tactics plans in a playbook.

One advantage of that is the patterns can be recognizable behaviors, as if a clumsy person were playing a stereotyped "leader." If the AI was put together as more of an general problem solver, it might do better, but the AI probably wouldn't "feel" like a natural opponent; it would be too unpredictable. It would probably be very irritating to play, like playing someone who "doesn't get it" but still manages to get in your face.

Part of the problem with making the AI adaptive comes from that pattern/playbook approach. Basically the AI is playing strategies the way you might try out someone's new "win by culture" plan in Strat-Articles. So the intelligence comes pre-packaged, in (in computer terms) quite sophisticated play plans. Trying to combine that with a "true AI" self-training engine would be nearly impossible... for arcane reasons. (I can only explain by analogy; it would be a bit like trying to have a laser printer learn to write by improving on the pages it's printing... there's no understanding of the writing inside a laser printer, it's all in the original author. Something vaguely similar happening here.)

I think the best way to improve on the current CIV AI would be to crack open its playbook and fill it with the best ideas from a lot of human play. I don't know how much of that will be exposed with the SDK when it's released; it'll be interesting to see.

 

[calyth]

Trying to combine that with a "true AI" self-training engine would be nearly impossible... for arcane reasons.

I think there are learning AIs for games like Go (which BTW is actually much harder to write an AI for Go than for Chess, as someone earlier pointed out it's just searching for the best next move). It isn't impossible to have a self-learning engine. But it might be the case that before release, they'll have to make it play against something to learn enough skills to keep us gamers interested. Then it would learn as we play.
This is actually really interesting. But I doubt we would be able to simply mod the AI through a couple of python files.