[5.0.0-alpha3] Balance report from 925 self-played games

[johnchen]

I am pleased to present the results from 925 self-played games on VP 5.0.0-alpha3. Setup:

• Communitas_79a
• Tiny map, 4 players
• Prince difficulty
• All victory types enabled

This is a side product of our upcoming study on LLMs playing VP, where we ran pure-VPAI games as a baseline condition.

Civilization-wide Effects on Win Rate
Winning probability is intuitive; for a 4-player game, the baseline winning probability should be around 25%.

Civilization-wide Effects on Score Ratio
Score ratio is defined as a player's best score throughout the game vs. the best score of all players. It provides an alternative lens vs. simply looking at win rates.

 

[johnchen]

If someone is interested in the LLM version.
Score Ratio (GPT-OSS-120B as the macro-strategy player):

Score Ratio (GLM-4.6 as the macro-strategy player):

Since we only measured Player 0 (LLM player) in those two plots, here is one for Player 0 (VPAI):

 

[Bruhmoment]

Quite surprised to see Babylon and Ethiopia at the bottom of the win rates, considering they're usually strong civs that are technologically ahead

 

[epicivfreak]

Looks like some buffs and nerfs are in order here.

 

[FinMc]

I do wonder if Aztec being so strong is due to the small size/small player count

 

[hokath]

Tiny map with 4 players will naturally favor warmongers over scientists, no doubt.