AI tournament: Season 3

Previous Seasons
Season 1: winner Gandhi
Season 2: winner Gandhi


Things learned from previous seasons:

• Peaceful victories are much more likely
• Religious Victory can only happen if AP religion is also spread to spectator
• Some AIs make very poor decisions and therefore hand over a free win to others
• A bigger sample should yield more reliable standings

I propose to use Normal speed and run the tourney on a season-playoffs-finals format. While Epic/Marathon could provide more diverse victory conditions, the simulation time is greatly increased - the same can be said of the generated map. The map type will be Fractal and sea level will vary depending on the number of AIs on the map.

I think Fractal is rather balanced. Sometimes it'll favour warmongers, sometimes peacemongers. However, the big plus to me is that it's easy to generate an isolated start for the spectator. This way, removing the extra land is simple and doesn't affect the AIs too much.

The process for this season will be as follows:

Season games

Seeds were determined based on seasons 1 and 2. Many games had very limited information therefore some input was required for a few of them - therefore it might be very possible that some of the "weaker seeds" end up doing very well - the Chinese leaders come to mind. The top seed went into Group A, second in B, third in C, fourth in D, fifth in A, and so on...

Detailed scoring will be kept for the season in case there is further development to the series!

Games 1 and 2 will pit some of the strong seeds vs. weaker seeds.
Game 3 will face off the better seeds.
Game 4 allows the weaker seeds to shine a bit.... hopefully.

16 games scheduled

Playoffs games

More points are given to season games than playoffs. This is so that leaders that do go on don't gain an insurmountable advantage in the next seasons.

4 games scheduled

Finals games

The finals are slightly more valued - these guys are the best performers around and they deserve to be called upon again.

3 games scheduled

Tie-breaker
In the case of ties, the deciding factors of final placement will be as follows, in order of priority:

1. Most high-scoring finishes (e.g. 2 runner ups vs 1 runner up)
2. Head-to-head results in games where all parties were present
3. Deathmatch

Simulation
I suggest using the method quoted here. The spectator will be isolated by ice away from all civilization (removing interaction as much as possible). The cheatmode removes the necessity for espionage and thus will not affect AI slider.

A few trials have shown that the first 200 turns can be done in 10 minutes and most of the time is spent on the last 150~250 turns. The good news however, are that the process is automated and only requires the spectator to dig through autosaves at the end of the game to get an idea of the important events.

The AP religion will be added to the spectator at turn 150. In the unlikely event that the AP has not been built, all founded religions will be added.

Other
Have suggestions?
Want to contribute?
Anything else?

Feel free to post or drop me a line (or the person contributing).

Links
Demo Game

Season
Group A Game 1

Group A Game 2

Group A Game 3

Group A Game 4

Group B Game 1

Group B Game 2

Group B Game 3

Group B Game 4

Group C Game 1 <<< Current game!

Group C Game 2

Group C Game 3

Group C Game 4

Group D Game 1

Group D Game 2

Group D Game 3

Group D Game 4

Kos - doin' it with style.....and spreadsheets

Fractal threw me off at first when mentioned, as it would actually not generally be balanced at all, but if you are modifying the map for isolations then it should work fine.

How do you automate the games?

In my experience, Fractal seems worse than Pangaea or Continents for generating some civs with large areas to expand into while others are very boxed in.

Enable cheatmode in the .ini, then press ~ in game, type game.aiplay ###, where ### is the number of turns you want.


Plus it lets you see what the AI sees, for instance a test game I ran:

Culture Bomb said:

In my experience, Fractal seems worse than Pangaea or Continents for generating some civs with large areas to expand into while others are very boxed in.

Click to expand...

The same can be said of Continents and Pangea or any other map type. The only real balanced map could be Hub but it has its own problems with great distances to reach other AIs and is not a proper reflection of most maps played.

My main concern is mainly to reduce the impact of the human spectator. Editing the land out while keeping a fair ratio to other AIs is problematic - but it also might remove access to unique regional resources. Fractal gives isolated starts every now and then and good chunks of land.

By increasing the number of games for each AI, I'm hoping to reduce these effects a bit.

If any map type which addresses these issues exist, that would be best but I am unaware of those...

I feel kinda silly now, but also glad to be inspiring something like this!

Very nice, Kossin! Just great!

I have only one note that I forgot to mention in Season 2. Should we consider to put leaders of the same nation in the different groups? I mean if one nation appears twice then lets it will be playoff game at least. Or is it done by purpose? If there weren't games than it's possible just substitute some of them with each other. Something like Stalin <-> Augustus Ceaser, Mao Zedong <-> Mehmed, Alexadner <-> Gandhi and so on...

RedKi-rr said:

Very nice, Kossin! Just great!

I have only one note that I forgot to mention in Season 2. Should we consider to put leaders of the same nation in the different groups? I mean if one nation appears twice then lets it will be playoff game at least. Or is it done by purpose? If there weren't games than it's possible just substitute some of them with each other. Something like Stalin <-> Augustus Ceaser, Mao Zedong <-> Mehmed, Alexadner <-> Gandhi and so on...

Click to expand...

How about this? I kept the seeds at the same level but swapped them around to avoid repeated civilizations. It can also be arranged for playoffs but for the finals there's no way to go around it.

This will be interesting!

I also would prefer not having leaders of the same nation in one group. If nothing else, it'll be less confusing to watch.

To me, the most fair format would be simply a league, without playoffs or finals. In my eyes, these simply create games that are somehow more valuable and favor civs who get lucky in them.
If we can group everyone into 7 seeding groups, we could generate a number of games so that every civ faces every other (from the other seeds) at least once and none more than n times. I would prefer this for future seasons, to ensure all games are equally valuable, since everyone will run into good or bad luck at some point.

For scoring, I would suggest:

Winner - 12
Runner up - 8
Wild card - 5
Rest in scoreboard order - 4, 3, 2, 1

Runner up being the civ who came closest to winning, regardless of their score. If it's really close, there may be two runners up.
Wild card is for the civ who did exceptionally under the circumstances. If there's no such civ, everyone remaining gets points according to scoreboard (5, 4, 3, 2, 1).
For 6 player games, the last player gets 2 points.
If civs are eliminated, they are ranked in reverse elimination order, i.e. first to get killed comes last.

I think this would help us to get more detailed information for future seeding and reduce the chance of civs being tied.

Thrar said:

This will be interesting!

I also would prefer not having leaders of the same nation in one group. If nothing else, it'll be less confusing to watch.

To me, the most fair format would be simply a league, without playoffs or finals. In my eyes, these simply create games that are somehow more valuable and favor civs who get lucky in them.
If we can group everyone into 7 seeding groups, we could generate a number of games so that every civ faces every other (from the other seeds) at least once and none more than n times. I would prefer this for future seasons, to ensure all games are equally valuable, since everyone will run into good or bad luck at some point.

Click to expand...

Any format is possible, we only have to develop one that's convenient. Running more seasons is certainly a way to enhance future standings.

The format I proposed is still 23 games, so that's a lot of games already. The lower value of playoff-finals games will, over time, give a more accurate result. By no means would 1 season be sufficient - but after 10 seasons, a trend might start to develop. Did Gandhi get lucky twice? Would Justinian keep performing so well?

Having every AI play x times in groups of 6~7 will lead to 8x games. Or, in order to have every AI meet each other at least once, you run into huge numbers of games. Even though user input is quite low using autoplay, it would require a lot of work to get to a low importance result in the end.

Thrar said:

For scoring, I would suggest:

Winner - 12
Runner up - 8
Wild card - 5
Rest in scoreboard order - 4, 3, 2, 1

Runner up being the civ who came closest to winning, regardless of their score. If it's really close, there may be two runners up.
Wild card is for the civ who did exceptionally under the circumstances. If there's no such civ, everyone remaining gets points according to scoreboard (5, 4, 3, 2, 1).
For 6 player games, the last player gets 2 points.
If civs are eliminated, they are ranked in reverse elimination order, i.e. first to get killed comes last.

I think this would help us to get more detailed information for future seeding and reduce the chance of civs being tied.

Click to expand...

I disagree; score means next to nothing. It's already difficult to grant a civilization the "runner-up" or "wildcard" status when it's possible for a vassal to win with the lowest score possible.

For instance: say you have Shaka, Gandhi and Justinian in a game with a few others. Shaka makes Gandhi capitulate early. Justinian is the score leader the whole game and capitulates everyone else - feeding them technologies and boosting their scores ahead of Gandhi and Shaka.

One of Justinian's vassals goes on to win an unlikely diplomatic victory because he built the UN. However, Gandhi was 40 turns away from a cultural victory at the same time, closer than anyone else to a victory.

Who gets what?
The vassal is the winner.
Gandhi is the runner-up? Justinian is the runner-up?
What about Shaka? He's now dead last in score but his overall performance was better than everyone else.

There's no way to tell how the game will develop after the first AI wins other than playing on until each AI achieves a victory himself - the situation can completely reverse itself 20 turns after the first AI achieves a victory. In season 2, the wildcard AIs were much disagreed upon.

Now I'm not saying the point system I propose is perfect (far from it) but with enough season games it will clear up (see my previous post) eventually. If after 10 appearances Sitting Bull is still at 0 points while the next AI is at 8, that becomes clear enough.

tl;dr; It's difficult to assess the true ranking of AIs outside the first few - things get lost in the middle pack. More games needed, one season with points not enough.

Great!

Just to mention that I read posts thoroughly (sometimes)

Isn't the AI pretty terrible at naval invasions? I would think Pangaea would be more suitable. Though I suppose Pangaea is more suitable for warmongers, though a double Gandhi win goes against the grain there.

This should be fun to follow in any case. Will there be writeups of each match, like Ray did? I assume he followed them more closely than is possible with autoplay, but decent writeups would still be great.

Pangaea said:

Isn't the AI pretty terrible at naval invasions? I would think Pangaea would be more suitable. Though I suppose Pangaea is more suitable for warmongers, though a double Gandhi win goes against the grain there.

Click to expand...

I think Fractal basically eliminates Conquest/Dom, but that doesn't mean warmongers are at a disadvantage. A warmonger who has conquered his continent might very well win a "peaceful" victory condition

Demo Game - Turn 1

Here are the starting positions for each AI. I have done very small edits to the map (remove BFC riverside copper - not supposed to happen; add a bit of food when it's low comparatively) to equalize starts a bit. The amount of land available to each AI is not perfectly equal, but that's never going to happen.


A look at the land overview. While Ragnar and Tokugawa are a bit more cramped for space compared to others, the quality of land is quite good compared to Gandhi's tundra and deserts for example. Mao is stuck in the jungle. Early favourite: Mehmed with a considerable amount of good land at his reach.


The save is joined if others want to run their own RNG !

pomthom said:

Great!

Just to mention that I read posts thoroughly (sometimes)

Spoiler :

The Pool 2 in the Playoffs has 2* firsts for groups B and D...

Ok, ok I'm out

Click to expand...

Thanks, fixed.

Pangaea said:

Isn't the AI pretty terrible at naval invasions? I would think Pangaea would be more suitable. Though I suppose Pangaea is more suitable for warmongers, though a double Gandhi win goes against the grain there.

This should be fun to follow in any case. Will there be writeups of each match, like Ray did? I assume he followed them more closely than is possible with autoplay, but decent writeups would still be great.

Click to expand...

Very terribad and that's why I decided against Shuffle in the end. Fractal tends to give 2~3 continents or one big pangea. I've planned some stop points (T50, T115 = 1AD, T150) but some time should be spent on wars. Don't expect turn-by-turn reports however

pomthom said:

I think Fractal basically eliminates Conquest/Dom, but that doesn't mean warmongers are at a disadvantage. A warmonger who has conquered his continent might very well win a "peaceful" victory condition

Click to expand...

To be fair, Epic speed or Marathon would result in more military victories, but it consumes much more time. In any case, it's representative of human games that the more likely defeats come from peaceful VCs (barring early wars).

Thanks for doing this 3rd season kossin, i'll definitely follow it

One suggestion: could you maybe reduce the screen shots size a bit?

About the game:
I just completly agree with kossin and would put my bet on Mehmed

Demo Game - Turn 50

How is each AI faring by Turn 50?
Gandhi

Ragnar

Mansa Musa

Tokugawa

Cyrus

Mao Zedong

Mehmed II


A look at the land overview. Still a lot of expansion possible.


Gandhi's religion will have problems due to Buddhism's close network in the south. How about wonders? 1 is very low for Turn 50 on Deity...

Timo33 said:

One suggestion: could you maybe reduce the screen shots size a bit?

Click to expand...

The last time I asked about my screenshot sizes, the majority wanted larger screenshots

If the majority however changes their mind, I'll simply resize the civ4 window from now on.

~~~

That's all for today - I'll post more tomorrow. The game simulation is finished of course but writing the posts takes some time