dh_epic's AI Solution

Analysis: the AI Dilemma

In the ideal world, we'd like to have AI's that are as smart as a player, convincing us that Hammurabi or Lincoln is actually a human being. But that ignores a very important question: how do humans play Civilization? The answer depend on who you ask.

Some people say humans play to win. That's why they break Right of Passage agreements, favor conquest over any other strategy, and pick on the weak.

Others disagree. The argue that Civ is in some ways like a simulation. That's why some humans forego the highest profits, to give them a sense that they have real interactions with their allies, enemies, and population.

My point has nothing to do with designing the rules of the game. It has everything to do with Artificial Intelligence. If the AI is supposed to be as smart as the smartest human, which kind of human are we talking about? The kind who plays ruthlessly, or the kind who plays immersively?

The Solution: Two Types of AI

Type A: Competitive AI or Game AI

These AIs would be rare. When a civilization is controlled by a Type A (competitive) AI, they play to win. They don't keep friends, they betray them when it's convenient. They don't have hard feelings towards Napoleon or Hitler for trying to conquer the world, they COPY them. In other words, they play like the most ruthless game player, and race to the finish.

Type B: Reactionary AI or Sim AI

These AIs would be more common to give the constructed world an immersive experience. 7 out of 10 Civs would play in a way that builds at their own pace. They want to be on the side of "good", and thus hold grudges against those who have wronged them.

They do start wars. Not war for profit, but because they somehow calculate them to be morally right. This Civ is Fascist, I am Democratic -- so I will fight him. This Civ is from the Middle East and they are attacking a fellow European Civ, so I will join the crusade against the Middle East. It might join wars it knows it cannot win. But otherwise this AI is quite receptive to peace, since they have no desire to win the game -- they are merciful to the extent that nations in real life can be merciful.

Interaction:

If your Civilization and the Type A Civilizations are the main characters, the Type B Civilizations are the supporting roles. If your Civilization and the Type A Civilizations are the masters, the Type B Civilizations are the pawns. If your Civilization and the Type B Civilizations are the monkeys, the Type B Civilizations are the trees.

In other words, the player and Type A Civilizations *use* the Type B Civilizations. When you start a crusade, you get them to join you. When you start your quest to world domination, they're the first one to get conquered. They are your long-term trading partners. And if America and USSR are Type A Civilizations in a Cold War, then you win the Cold War by converting the Type B Civilizations to your side. It is compatible with many gameplay ideas, new and old.

Simple Implementation: How to Cut Corners with AI

Type B: Don't Need to Win

Type B Civilizations are incredibly easy to implement. Why? Because they don't need to win. The only balancing and speed-adjusting you need to do is to make sure that they stay relevent later in the game. Otherwise, it's a question of giving them "realistic" reactions. You know you have a good Type B Civilization when it's possible for a Human Player to be allies with them for an entire game. They generally play honest and steady.

Type A: Don't Need to be Real

Type A Civilizations are something trickier. But all that realism stuff doesn't matter. The only thing that matter is if it can maximize the benefits it acquires in conquest, production, growth, gold, and science. They don't need to follow reptutation, or who's furious. They need to stay neck-at-neck with a highly intelligent player.

Which is why I suggest the Type A Civilizations cheat.

Some would argue that a Type A Civilization that cheats would feel fake. But they don't need to be realistic, the realism comes from the Type B Civilizations. The world would still feel real.

Some would argue that a Type A Civilization shouldn't cheat, it should genuinely outwit the player. Granted, that would be great. But I ask you to hear me out. To outwit the player, the AI would need to master multiple aspects of the game AND avoid being exploited by some flaw in the programming -- a level of complexity that might not be feasible for the programmers, nor your Pentium 4.

Cheating by Catching Up

If you've played a racing game, you'll know that it's easy to smash your second-place opponent and make it impossible for them to catch up. Which is why so many racing games have "catch-up" cheats, where the nearest competitive car will get a little speed boost to stay within striking distance of first place.

Implementing a "catch-up" cheat would be much easier than implementing an AI that outwits the player. Not to mention that catching up will never cream the player so bad that the player quits half-way through, unlike a botched game at a high difficulty level. And catching up prevents the game from being won in the middle ages, such that the rest of the game is a waste of time -- unlike 99% of the Civ games I play.

If the player were fighting a Type A Civ, that Type A would not try to catch-up. It would fight vigilantly, but gradually drop off, as expected. But as that Civ is conquered, another Type A Civ tries to keep pace with the player by conquering other Type B Civs -- and cheats to do so. By the time the player has taken out one of its top competitors, a new competitor has emerged that rivals the player's strength.

Wrap-Up

This simple solution gives the game both realism AND competition, sidestepping the difficult question of what kind of human the AI should imitate. And it doesn't need the processing power of deep blue to do it.

I love the idea of more then one kind of civilization. Maybe if there weren't any type A civilizations left a type B could become one. Also, maybe type B civs could emerge all the time, through barbarians learning about civilization, civil war, or rebellion.

I think if all the Type A civs die out, then you deserve the win.

You can play against "Type A" any time you want. 'Tis called "multiplayer". Why turn single player into a pale imitation of that?


Besides, there's an inherent assumption in DH's thesis that says that "Type B" cannot be competitive. However, I suggest that far more important than any single civ is what the civs do in relationship. Several Type B's combining can easily open a can of wh00p@$$ on a Type A, and give the bully a good working over in some dark alley into which they drag his lame self.


All civs ganging up on player (or strongest civ) like Civ1 and Civ2 grows stale.

All civs being unable to form coherent alliances like Civ3 means player has won as soon he's the strongest civ.

We need something in the middle, and I don't think DH's solution gets there. What if all the Type A's draw bad starts? They're not going anywhere. What if all the Type A's end up on one continent and stalemate one another? What if player is playing a small map with three opponents? How do you divvy up the Type A and B personalities in any reliable way?

Civ3 games often break on pure luck anyway. Anybody who has watched the Epics tournament knows how often the Civ3 AIs split on performance. One player's game, Blue trounces, the next player's game, Blue is wiped out in BC times. Some maps, you can see that terrain dictated who would be strongest. Others, it breaks on a wonder, or a leader pop, or the random rolls of dice to determine timing on who attacks whom and when. Game A, the big dog takes on targets one at a time and gobbles them. Game B, the big dog gets two-timed in the ancient era and ripped a new one. Game C, the big dog eats one neighbor, then the rest gang on him and they play to stalemate. Game D, little dog pops a leader and forms an army, pops a sci leader and gets to Pyramids first, and becomes the big dog.

Either terrain dictates, or luck dictates, because all the AIs are playing the same strategy. Throw player into the mix, and things get murkier, but we don't know how good a "Type B" AI in Civ4 might be.


I know DH believes strongly in the split AI concept, some playing clean, others playing dirty. His aim is to please both sets of humans: those wanting MP type competitive action, and those wanting more immersion. I wonder if neither side would be pleased with this fusion. Since the way that the alliances break down would override this distinction anyway, I don't see it as a deciding factor. Maybe the Firaxians will agree with DH, but I hope they focus more on the diplomacy. I think diplomacy is the overarching issue of single player. That, too, involves the AI, but not along the lines drawn here.


- Sirian

I'm glad you guys like the essence of the idea, even if it does raise new questions. I'm definitely open to debate about those add-ons, but by no means does the rejection of one necessitate the rejection of the essence of this idea.

If killing off the Type A civs guarantees victory, then you're essentially changing the way domination victory is faught. Then again, maybe killing off the top 4 superpowers in a game with a dozen Civs would be pretty damn close to hitting that 50% mark required for domination. Either way, it couldn't hurt to make domination happen faster.

Then again, assuming that you kill off the Type A Civs without acquiring the traditional domination victory, some people like to keep playing. Type B Civs only would be kind of a drag. I could see a situation where after the USA takes out superpowers like Britain, Germany, and Russia, suddenly Poland steps up. (Just an arbitrary example, seeing as Poland is right in there behind Britain, and should not be forgotten.)

I think the idea of new Type B Civs added as the game goes on is a thought that could be explored. Barbarians for one. But this gives me another idea.

The Type B Civs, not too concerned with victory, could be the stage for world events that no competitive player would ever want to encounter themselves but would like to see in the game for realism or even strategic sake. Sudan is a Type B Civilization, for example, that is dealing with genocide. The Type A Civilizations could use these as opportunities to leverage their power on a world stage, or they could ignore it as a part of the crazy complex world that we live in. Just throwing that out there as a platform for other people's ideas.

Thanks for the thought, Sirian. Sorry I didn't get to your topic, didn't see your post until after I finished mine.

You're right about the "hidden assumption". That Type B AI would have to be non-competitive to be realistic. But I think this assumption is well grounded in the way that Civ 3 was won.

In Civ 3, the winner is the person who pulls a "Right of Passage Rape" as someone once called it. The winner isn't the person who competes with the biggest dog in the yard, but who picks off the smaller threats one by one until their empire is bigger than the biggest dog in the yard. Discriminating against people is a bad strategy too -- in real life, racism and bigotry guided a lot of diplomacy, but in Civ it's smarter to be fair and hate everyone equally. There is no incentive to cooperate. Loyalties don't last, and neither do grudges, because players have no incentive to hold a grudge or a loyalty. It is much more prosperous to turn around and backstab an ally at their weakest, or to cooperate with a former enemy so you can focus your conquest on easier pickings. In other words, playing with realism (aka sanity) is a surefire way to lose. Much more profitable to be an untrustworthy psychopath.

This assumption is totally thrown into question, however, if they manage to make a game where cooperation is a necessity. Then breaking and burning all your bridges would make you pretty damn vulnerable (as it does in real life). In which case, I think you're onto something, Sirian. Putting a greater emphasis on diplomacy might be the answer to having an AI that plays realistically *and* competitively. But that would also require players to play realistically, and breaking your deals would be a much riskier strategy.

Giving this a bump because I'm curious how many people support it if they all voted.

DH, I doubt the poll will make any difference. If nobody but you supported the idea, yet you and your case persuaded any Firaxians looking on that you've got the right answer, then your idea might get used. If the poll has 100% support but the Firaxians disagree with your concept, it won't get used. Thus the poll would seem to be irrelevant. Your analysis is likely the most important thing. That seems more likely than polls to change minds.

In Soren's slideshow presentation, he mentioned something about all the tons of suggestions forwarded by fans. By now, your concept may have gotten a look. Either way, they've said nothing at all about anybody's suggestions as yet, so what more can you do? For me, at least, I see no need to beat a drum. Say it once, say it well, then move on, that's my motto. If I come up with good ideas and they don't get used... ::shrug:: Out of my hands. If I keep it pithy and everything I write is meaty, maybe my name will draw more attention when they see it. We can only do what we can do, then the rest is up to them.


- Sirian

Here are a few paradigms that would lend themselves to competative and somewhat realistic gameplay.

1) Cooperation with others over time lends itself to greater rewards then unilateralism. Currently in Civ trade is weak, since the only benefit of trade are things you can aquire by conquest.
2) Relative power is more important then absolute power. Civs should try to have 'power' over his environment, since controlling a thing is cheaper then owning it. Currently in Civ the only real measure of power is territory.

That leads to the next point....
3) Followers will tolerate a leader until they become too powerful to stop. This means whenever someone is becoming so powerful they cannot be stopped, civs will try to put them down. This includes AI civs, something Civ 2 did not address.
4) Civs will do what is necessary to protect their interests, this includes protecting the security of profitable foreign territory.
5) Civs will try to do whatever it takes to control their destiny as long as it improves their situation.

If the AI could learn to think like this, it would simulate real world interaction.

Hey Sirian, my intent is not just to get a vote but to get a discussion I figure getting people to run with the idea can also infect the way people talk about other aspects of the game. If "Type A vs Type B" becomes a distinction people use, it might help them to discuss other issues such as intelligence, religion, diplomacy, culture, or international economy.

Always love a meaningful discussion.

Sir Schwick, I think you're onto something with the difference between owning and controlling something. In Civ 3, it is easily more profitable to own it yourself. In Civ 4, it ought to be profitable to control and leverage something without having to pay the costs and maintainances involved in owning it. Otherwise more nations would try to conquer the world.

But with that said, does that mean you're not fond of the AI proposal here?

*cue hallelujah chorus in background* wow, this is it. this is what we've been looking for two types of AI the only question i have is why didn't anyone think of this before? although i would possibly include a third type, i'd call it a loyal AI. this AI holds grudges, and keepes allies, but they play to win too, but the two emotions are more or less equal

I think A vs. B fixes the symptoms rather than the core problems. The problems are that cooperation is not rewarded, manipulation is not rewarded, and diplomacy is not rewarded. Also, the current AI does not think ahead, but thinks one turn at a time. An A vs. B system would certainly not have the same symptoms, but requires adjustment each time you change part of the game. By fixing the causes, the AI plays correctly without having to tell them how.

Glad you like it, Yybor. I figure it might not be the most thorough solution to AI issues, but it has an elegance in its ability to resolve the realism AND challenge issues without having to deeply overhaul the game *and* program a deep blue.

Like Sir Schwick says, it does kind of solve the symptoms rather than the problems. While you end up with a stable world with a lot of order, being shaken up by a few superpowers... the player still only really has the option of playing like a megalomaniac. Even allying with Type B civs (or a third type that is also loyal), the player has no incentive to keep these alliances but their own personal sense of "honor". There are few penalties for breaking the alliances, and they're penalties that are easy to live with. It's easy to live as an isolationist.

I'd like to think, though, that this AI idea is very compatible with adding a new focus to the game.

Still, feel free to vote for agreeing with the problem but disagreeing with the solution, Sir Schwick. I figure it's fair to represent multiple viewpoints, and maybe someone will notice that fixing the underlying problem is still an important issue. I definitely agree with you there.

I'm also asking, does anybody think that this kind of balancing act is just unnecesssary?

Does anybody think that Civ 4 playing like a board game is a good idea? Where realism is out the window, and it's a question of who can conquer the world -- like a game like "Risk" with a bit more of a historical timeline? (Only Type A Civs.)

Does anybody think that Civ 4 playing like a sim is a good idea? Where building is the real focus, and war is just something you do, but it's less focused on traditional winning conditions? (Only Type B Civs.)

I'm actually a bit surprised nobody has taken these stances.

dh_epic said:

Does anybody think that Civ 4 playing like a board game is a good idea? Where realism is out the window, and it's a question of who can conquer the world -- like a game like "Risk" with a bit more of a historical timeline? (Only Type A Civs.)

Does anybody think that Civ 4 playing like a sim is a good idea? Where building is the real focus, and war is just something you do, but it's less focused on traditional winning conditions? (Only Type B Civs.)

I'm actually a bit surprised nobody has taken these stances.

Click to expand...

Holds up hand, waves.

Ahh, Sirian, you should have picked "Good Analysis, Easy Dilemma: I solved the dilemma by picking one over the other. (Post below)" ... oh well, no biggie.

I'm curious, I forgot exactly. Did you want to see Civ become more of a sim? Or did you want it to be a board game like Risk, where everyone plays ruthlessly? (and I think Civ 2 and 3 are already pretty close to Risk.)

I think Civilization IV should play like Civilization IV. It should be related to Civilization III, but I'd like to see it create a whole new standard as seen in the likes of Grand Theft Auto III. Lofty expectations? Maybe, only I think it could be done.

In the poll I picked hooray for everything, but I've changed my mind. I now want good analysis, bad solution, yes to shortcuts. While the idea of having an A1, B1, C1 etc has a good intent, I don't like the principle behind it. Every civilization should have an equal chance at being great. With an A1 programmed to go for the win, the B1 programmed to be mediocre, and the C1 programmed to be a good ally, not every civ is getting a fair chance to win. Now I know that a random start location, and different civ traits are not perfectly equal, but the principle behind these ideas is to make civs different, yet still give them each a fair chance.

I have an idea of my own (surprise surprise) but I basically got there from reading this thread. My theories are always as directly applicable to the game as I can make them, so here goes:

• Reputation is the single most important factor in determining AI behavior towards other civs.
• Catch up events are used to balance out the game when civs pull too far ahead or fall to far behind.

Right now reputation is basically a record of all the good and bad things you have done in relation to other civilizations. Reputation would be made ten times more important than it currently is, influencing every interaction between you and the AI. For example if both the player and the AI civ have had friendly relations all game, then the AI will be more likely trade techs, offer fail deals, give gifts, loan gold, and will generally be a good ally. When the player has a bad reputation, the AI will be very unlikely to trade at all, and will never enter into any treaties with the player as they can no longer be trusted.

The balancing power of a strong reputation rating is isolation. Think of getting a bad reputation like being stuck on a starting island in the middle of the ocean, your by yourself with no one to trade with. The added disadvantage is, that as well as not trading with the evil reputation civ, good reputation AI would often join together to attack the "evil" civ. The player would be encouraged to follow the same guidelines when dealing with different reputations, but they are only guidelines and not carved in marble (which would make a good luxury resource btw ).

So take two AI civs starting on an adequately sized island. If both civs get a fairly equal start, then most times the reputation system would keep their aggression in check, and they would trade with each other rather than try to wipe each other out. As the game went on, if they both remained in good standing, their alliance would grow to the point that one civ was attacked, the attacking civ would lose reputation with both civs as the second would enter war to defend their trading ally.

I'm sure some people would find keeping a clean reputation and playing by the rules a boring game, so there are a few exceptions. So long as a civ stops doing activities that generate a bad rep, its rep will make its way back to neutral over time. Therefore razing cities and knocking civs out of the game in the ancient age is tolerated much more than in later ages because time passes quicker. Breaking a ROP is still breaking a ROP, but it will take much less time to recover if the act is committed in the ancient age. This simulates the fact that almost all civs were barbarous in their primitive states.

Catch up events happen at about the same time for every team. The way to think of them is not that well played civs are getting punished, but that they are getting a speedbump thrown their way. It is essentially dh's system to prevent early victories but with an emphasis on averaging out the civs.

So for example lets take the late ancient events. If there is a civ that is dominating in science they would go into the dark ages where their research would slow to a crawl and they might lose a few techs. An large civ that had killed their neighbor early would probably have a civil war against some of their conquered cities. A medium powered civ might just have a barbarian horde invade and pillage their territory until defeated. A weak civ would get a golden age boost to catch up to the other civs.

Obviously that is just a summary of the events, but you can see how they'd work. There would be two more timed events. One in the middle ages that would feature the black death spreading between the powerful teams, and a renaissance for the lower powered teams. The great depression would follow after WWI. As far as the game is concerned, knowing when to trigger these events would be based on how far along a civ is on the tech tree, and knowing how hard to hit them would be based on demographics like % population and the number of techs it has in lead of other civs.

There are also tie-ins between this repuation system, the caravan system and multiple civ agreements, governments and some reorginization of the tech tree but I already wrote way more than I planned so if anyone wants to hear those ideas I could post em later. The basic idea is pretty simple, a good reputation becomes much more important and the events try to keep the game competitive.

Civ4 should be able to offer both types of AI. Why both? Type A AI is needed for multiplayer to shine: a competent AI if someone drops so that large games are not automatically "over" when the first players quits or has to leave. Likewise, a good Type A AI can be used as a placeholder for late arrivals, and they could hold their own in a Free For All, opening the chance to INCLUDE AIs BY CHOICE in a large game, without them being walking suckers to be exploited to death.

Yet that kind of AI is no fun in an epic single player environment, in my view, so we also need Type B AI. Yet it needs to be stronger, and I already spelled out my thoughts about how to do that: via diplomacy. Being stronger military and economic performers wouldn't hurt, either.


I chose correctly for how I feel: good analysis, bad solution. Yet even good analysis may not completely cover a subject. I disagree that the choice is between "strong" Type A and "weak" Type B. Type A can end up being weak, too, and Type B might be strong. Plus, if the rules themselves are better, there won't be as many loopholes for players or Type A's to exploit, so the Type B's might not be as disadvantaged as you indicate.

Do we know enough to make detailed suggestions? I question it. Looking around the forum, there are a lot of suggestions here. I can't imagine they'd all fit in one game. That's part of why you don't see me offering a lot of details, but instead focusing my comments on concepts. A good concept... they might pick up the ball and run with it. If they like a concept, they'll have to adapt it to fit with the new game anyway, right? And if they don't like a concept, there's no chance they'll use a proposal on how to implement it.


- Sirian

Sirian's a very conceptual guy. To a fault, I believe, at times, though I'm sure he'd argue that.

Nope, that's fair, Trip. Idealistic to a fault, at times, too.

Nevertheless, I can show a surprising capacity to get things done every now and then. Between all the bloviating, that is.

Email to Sirian said:

Hi Sirian

Tha map I enjoyed most in Railroad Tycoon 2 was your Grand Prix Map. Do you plan to bring it to RRT3 as well? MAny fans would be thankful...

Greetings

Mat

Click to expand...

Sirian's reply said:

Hi Mat.

I haven't gotten RT3 yet. Not sure if I will. I've been busy with many things. My RT2 maps... I'm always glad to hear when a player is enjoying them. I appreciate the call for a new version, but I cannot make any promises at this time, and no plan is in place for the short term.

Not the answer you were hoping for, but all I can do is reply sincerely.

Perhaps others in the community will produce good maps. Ought to be some good ones out there.


Best wishes.

- Sirian

Click to expand...

Mat said:

Hi Sirian

Ah, well. Too bad, but still I want you to know how much I (and many others) enjoyed your maps. That's enough for me, an we don't expect you to jump and design a new one
And yes, there are already a bunch of maps around. I try to do my own, but it's not that easy...

Thanks for your RRT2 maps

Mat

Click to expand...

My RT2 maps are included with the Platinum version of the game. And of course, I also loved the original Railroad Tycoon. I'm looking forward to the new version of Pirates. Should be fun.

One of the best AI's I ever encountered was Sid Meier's Covert Action. The game balance in that one was nothing short of BRILLIANT. Playing on the highest level with no reloads... one of my most challenging yet playable gaming experiences. Took everything I had to win half the time. Yet the AI was not designed to be omniscient (Type A) but rather to behave with some realism (Type B) and it left just enough openings for a good strategist to get by, if you also had the twitch reflexes to back it up when things went wrong.


If Sid Meier's Civilization Four can produce a Type B AI of the caliber seen in Covert Action, it should entertain me for many hours.


- Sirian

Sirian said:

Nope, that's fair, Trip. Idealistic to a fault, at times, too.

Nevertheless, I can show a surprising capacity to get things done every now and then. Between all the bloviating, that is.

Click to expand...

Well, putting the two of us together is definitely not always a good thing, at least for bystanders. Though at times I wish I had as much free time as you.

Back to reading about WWII in the Pacific I go...