3.10.14 AI-Only Game Statistics - Now With Some Religion/Instant Yield Insights

[L. Vern]

I have been requested to publish some stats for this congress session, so with this upcoming weekend being the last one before proposal phase ends hopefully there will be enough time to make use of them.

Settings used:
Map: Communitu_79a with all default settings
Difficulty: Emperor
Size: Standard
Game Settings: All default except no events, no ancient ruins, and no tech trading (note: same as every other run, just making it explicit in the description as many people were asking about it last time)

Spoiler Victory Types :

Spoiler Civilization Performance :

Rank	Civ	Games	Victories	Winrate	Culture Victories	Diplomatic Victories	Science Victories	Time Victories	Domination Victories	Avg Pantheon Turn	Avg Religion Founded Turn	Religion Founded Prop	Avg Religion Enhanced Turn	Religion Enhanced Prop	Avg Religion Reformation Turn	Religion Reformation Prop
1	The Maya	30	8	0.27	1	2	5	0	0	47.17	103.79	0.80	112.33	0.80	168.62	0.53
2	Austria	34	9	0.26	1	7	0	1	0	39.62	103.13	0.68	172.15	0.59	165.71	0.41
3	Korea	41	10	0.24	5	0	5	0	0	45.98	109.78	0.66	190.75	0.59	168.53	0.37
4	Arabia	47	11	0.23	10	0	1	0	0	40.11	98.30	0.94	170.66	0.87	160.49	0.79
5	Brazil	40	8	0.20	3	2	3	0	0	48.60	102.42	0.48	174.75	0.40	165.38	0.32
6	Germany	50	10	0.20	1	7	2	0	0	42.86	112.81	0.54	179.52	0.46	167.62	0.32
7	Japan	51	10	0.20	1	1	7	0	1	48.20	106.34	0.63	166.35	0.61	178.25	0.47
8	Babylon	43	8	0.19	0	0	8	0	0	45.72	107.18	0.77	169.86	0.67	168.55	0.47
9	Russia	40	7	0.18	1	1	5	0	0	58.42	111.53	0.38	170.00	0.35	188.14	0.35
10	Greece	55	9	0.16	1	6	2	0	0	40.95	103.81	0.65	164.18	0.62	180.46	0.44
11	Byzantium	44	7	0.16	1	2	4	0	0	43.18	99.70	1.00	151.50	0.91	174.66	0.80
12	Portugal	26	4	0.15	1	2	1	0	0	43.65	104.09	0.42	196.08	0.46	201.33	0.35
13	The Netherlands	39	6	0.15	2	4	0	0	0	38.92	105.97	0.74	168.48	0.64	170.11	0.49
14	Siam	39	6	0.15	2	4	0	0	0	35.79	94.10	0.79	173.93	0.74	156.39	0.46
15	The Celts	27	4	0.15	1	1	2	0	0	40.56	96.44	1.00	156.26	1.00	198.75	0.15
16	Carthage	34	5	0.15	0	0	5	0	0	47.29	102.74	0.79	174.88	0.74	169.80	0.59
17	India	48	7	0.15	3	0	4	0	0	0.27	79.55	0.98	117.64	0.94	183.29	0.65
18	Assyria	42	6	0.14	2	0	4	0	0	53.17	106.78	0.43	155.75	0.38	178.13	0.36
19	China	43	6	0.14	5	0	1	0	0	45.23	102.53	0.70	165.10	0.67	167.86	0.65
20	Songhai	46	6	0.13	0	0	1	1	4	59.09	111.52	0.46	174.70	0.59	198.86	0.46
21	Venice	46	6	0.13	1	2	3	0	0	35.87	100.21	0.91	165.46	0.80	152.62	0.74
22	The Iroquois	31	4	0.13	0	1	2	0	1	59.81	109.25	0.52	171.83	0.58	181.18	0.55
23	Egypt	26	3	0.12	2	0	1	0	0	41.50	109.12	0.65	162.29	0.65	172.00	0.50
24	The Inca	35	4	0.11	1	0	3	0	0	60.20	111.19	0.46	169.00	0.43	166.09	0.31
25	Persia	45	5	0.11	3	0	0	2	0	48.38	111.53	0.42	180.58	0.42	200.00	0.29
26	Morocco	38	4	0.11	1	3	0	0	0	41.03	103.27	0.68	178.59	0.58	162.36	0.37
27	France	44	4	0.09	0	1	0	1	2	55.77	111.56	0.41	183.44	0.41	241.53	0.34
28	Poland	34	3	0.09	1	0	2	0	0	42.44	108.62	0.62	199.44	0.53	171.12	0.24
29	Rome	23	2	0.09	0	0	2	0	0	54.13	109.33	0.52	161.08	0.52	179.08	0.52
30	England	41	3	0.07	0	3	0	0	0	54.05	109.57	0.34	178.43	0.34	185.60	0.24
31	The Ottomans	41	3	0.07	0	0	2	0	1	47.05	107.04	0.61	173.91	0.54	177.25	0.39
32	Sweden	42	3	0.07	1	0	0	1	1	59.64	115.29	0.33	175.94	0.38	186.85	0.31
33	The Aztecs	30	2	0.07	1	1	0	0	0	45.57	102.96	0.80	155.91	0.77	183.24	0.57
34	Polynesia	32	2	0.06	0	1	1	0	0	60.88	108.71	0.44	172.43	0.44	190.38	0.41
35	The Zulus	37	2	0.05	0	0	0	1	1	50.30	111.80	0.41	172.75	0.43	206.67	0.32
36	Denmark	38	2	0.05	0	0	0	1	1	55.97	115.43	0.37	190.12	0.42	180.18	0.29
37	Ethiopia	38	2	0.05	1	0	1	0	0	30.97	89.18	1.00	152.43	0.92	157.23	0.68
38	Spain	42	2	0.05	0	1	0	1	0	31.31	99.26	0.90	179.75	0.86	182.00	0.57
39	Indonesia	31	1	0.03	0	0	1	0	0	52.45	107.55	0.71	155.50	0.71	168.44	0.52
40	The Huns	34	1	0.03	1	0	0	0	0	60.15	116.42	0.35	179.08	0.35	205.00	0.21
41	Mongolia	34	1	0.03	0	0	0	1	0	48.32	109.55	0.59	174.20	0.59	175.36	0.32
42	America	39	0	0.00	0	0	0	0	0	51.05	108.11	0.46	178.78	0.46	165.13	0.38
43	The Shoshone	28	0	0.00	0	0	0	0	0	53.57	111.71	0.50	173.33	0.54	181.09	0.39

Spoiler Religion :

Event	Median Turn	Winrate When Achieved	Avg Win Turn When Achieved
Pantheon Founded	43	-	-
Religion Founded	105	0.16	407.52
Religion Enhanced	162	0.17	407.58
Religion Reformation	165	0.22	407.99

A more detailed breakdown into attainment times:

Per-Belief Breakdown:

Note that for the following section an aggregation interval of 100 turns was arbitrarily chosen, ideally there will also be versions grouped by era in the future but that kind of joining will require a lot of work so probably not soon lol

Spoiler Handicap Bonuses :

This section summarizes the lvl 6 instant yields (A/B/C Bonuses) the AI receives on average, and from what event types

Spoiler Handicap Bonuses vs Instant Yields :

Sadly all yields were not able to be determined from current logs (will have to implement this first), but here is a comparison of all instant yields the AI receives on average (does not include handicap bonuses) vs handicap yields. There is no breakdown by instant yield type as there's 42 unique sources and I haven't figured out a way to visualize them in a digestible way.

Spoiler Technology Times :

Spoiler Policies :

Huge thanks to @Stalker0 , @azum4roll , @rkkn , @ma_kuh for feedback and suggestions on discord

Questions/Observations for discussion:

• Thoughts on science being the new most common win condition? Did science get more tools or was cultural and diplomatic power just get sufficiently toned down?
• While the number of games is fairly low still looks like some significant swings in civ performance may have happened. Big winners: Russia, Maya. Big losers: Rome, Ottomans.
• How do we like the power level of the civs in human games subjectively? Is it noticable and detrimental that some civs always do quite well and some are vassals-to-be, noticable but you like the consistency and predictability? I think most of us agree that in a game there should be one or two "pop off" civs to create a challenge for the player but subjectively is it commonly the same civs that do so?
• Looks like a lot of the handicap yield power is in the killing units bonuses - these events happen a LOT, might have been an unintended side effect of the ongoing tactical AI improvements? Any other thoughts on this particular outlier?
• Is it bugged that AI almost never goes for god of the open sky/goddess of beauty/transcendent thoughts/revelation/ceremonial burial/holy laws/hero worship/council of elders?
• How much does AI difficulty bonus influence instant yields? Are they higher or does the AI instant yield line represent roughly how much a human would be getting in the same situation?

 

[L. Vern]

I ran out of attachment limit, so might as well make a post devoted to raw data (I was going to upload them as attachment to this post but was too large )

The process for these data analyses goes something like:
run games, gather logs -> process logs into clean (raw) tabular data -> analyze clean data, turn into tables and graphs

The first two steps require a lot of effort and some specialized knowledge, whereas the third step is much more accessible (ex. can open these files in excel and do user-friendly data exploration). As such, I've attached all of the raw datatables as a bundle of csvs in the hopes that someone will do something cool with it


Warning: the handicap bonuses and instant yield files contain a record of every such instance occuring, are 300mb and may hang your computer if you try to open them in excel

3.10.14_csvs.zip

drive.google.com

 

[rkkn]

- People have said that the AI plays the Mayans wrong, and yet they're still doing very well. Are they just OP?

- I think the AI needs to be changed to increase its evaluation of the situational bonuses on beliefs. it never picks pantheons with situational bonuses
and apparently it doesn't look to the future very much either? I saw a screenshot of it recommending sun god for a city with zero farm resources, and then someone else saying that it always recommends sun if there's a granary built. so it's just picking whatever gives the most bonus at the time?

 

[hokath]

For pantheons, it seems only around 8 of them really get picked. Is this what we expect?

 

[andersw]

- People have said that the AI plays the Mayans wrong, and yet they're still doing very well. Are they just OP?

Yes yields from Kunas are op you also get a free scientist very early.
Also not a single god of war?
Not a single holy law founder?
Two very good options.
edit: I assume this is the new tribute, I suspect the authority AI is not tributing CS enough, surprised to see mongolia down there.

 

[L. Vern]

Maya also have an exceptionally low average religion enhancement turn, I assume that's from getting free early great prophet from UI? Wonder how much that has to do with the strength of it

 

[Stalker0]

@vern I know a while back you thought that religion wasn't that big a win factor, and so far your analysis seems to agree with that, at least for the AI. Founding a religion isn't a dominant factor in the win rate as I would have expected.

 

[andersw]

Maya also have an exceptionally low average religion enhancement turn, I assume that's from getting free early great prophet from UI? Wonder how much that has to do with the strength of it

No you cant pick prophet until you have founded, again its kuna providing extra faith on top of belief.
edit: ahh yes enchance, yes in extreme case with a bit of luck in timing I've been able to found and enhance the same turn.

 

[Phosphoraptor]

The AI still struggles with domination. It seems the best strategy against warmongers is to simply hope they dont eat you first. They can balloon to scary sizes but will often fumble at the finish line.

 

[John.B.C]

The AI still struggles with domination. It seems the best strategy against warmongers is to simply hope they dont eat you first. They can balloon to scary sizes but will often fumble at the finish line.

Mainly because of the phenomena of them no longer using meele units to attack cities - even though the units carry the reduced city damage promotion - and even if the city is either heavily swarmed or completely surrunded by meele units, together with being without any ability to defend it self. In any scenario, the AI meele units will just 'hang around' without causing any damage to the city.

Unfortunatley, this still hasn't been fixed, since I posted about this almost 2 years ago! I don't even see anyone recognize this big flaw in the strategic AI as being a problem at all (why are diety players silent about this?). I personally remember the days when I was totally raped if a city was swarmed by meele units (it was especially very hard to defend costal cities against meele AI ships!). Today one just need walls and be careful about taking out whatever ranged unit that comes within range of the city.

The AI was perhaps not better at domination victories a few years ago - it was at least better at capturing cities - that's for sure!

 

[Rosete]

I don't even see anyone recognize this big flaw in the strategic AI as being a problem at all

Because it is not. Years ago, AI would just bash their units without regard. You set up a semi choke and the AI will feed their entire army into it. AI could take cities sure, but then they would have no units to actually keep it. The tactical AI we have now is leaps and bounds better than it was before. Now AI can preserve their units, divide their force well, and even use bait manuvers.

the AI meele units will just 'hang around' without causing any damage to the city.

Maybe a bug, I have not seen this in my own game.

doesn't look to the future very much either

I may be wrong, but this may just be a quirk of AI development? That AI can't think like humans do and so they aren't capable of long term planning.

 

[ma_kuh]

@vern I know a while back you thought that religion wasn't that big a win factor, and so far your analysis seems to agree with that, at least for the AI. Founding a religion isn't a dominant factor in the win rate as I would have expected.

I'm not sure about that, if I'm understanding right the "winrate when achieved" stats suggest about 1/6 winrate after founding, which would be compared to an expected average of 1/8. It does seem like after founding, enhancing isn't a significant boost to winrate, but reformation is, jumping to a little over a 1/5 chance.

If I'm interpreting that wrong please correct me.


This is post-GAP culling, and removing it from handicap bonuses, so it may be having an impact on CV wins, if I had to guess. But 26% is a fair portion of the pie for CV.

 

[Stalker0]

I'm not sure about that, if I'm understanding right the "winrate when achieved" stats suggest about 1/6 winrate after founding, which would be compared to an expected average of 1/8. It does seem like after founding, enhancing isn't a significant boost to winrate, but reformation is, jumping to a little over a 1/5 chance.

If I'm interpreting that wrong please correct me.


This is post-GAP culling, and removing it from handicap bonuses, so it may be having an impact on CV wins, if I had to guess. But 26% is a fair portion of the pie for CV.

oh its a factor for sure, but candidly I was expecting more like 80% winrates with religion.

These numbers honestly look about the same as a nice start vs a bad one in terms of impacting wins.

 

[Diomedes_]

1. I think it makes sense that in ai-only games science would be the most common win condition. Culture and diplomatic victories are just inherently more complicated that science ones and it stands to reason that the ai wouldn't be as good as humans in putting all of the pieces together.

2. you're right, it seems like the Russia UA fix worked as intended.

3. At least in my games, it feels like I can predict with maybe ~70% certainty how powerful a civ will be. There are always outliers in my games, like almost losing to Shoshone a few games ago or America being crazy powerful in my last game, but in general I think its okay that you have a rough idea of how powerful a civ will be going in. I certainly feel like the gaps in civ power are lower when being played by a human.

4. Yeah the tactical ai has definitely gotten much better since I first played VP ~2 years ago. I think ais getting hefty bonuses from unit kills makes the game more interesting as it allows for civs to snowball a bit harder. I usually play on small maps with 6-8 civs, and the best games are the ones where we're all in quite close competition through early Industrial but then 2-3 of the ais snowball and the game is reduced to a 3 or 4 way competition in post-Atomic.

5. I do think that the ai pantheon logic is pretty weak. I've noticed this too with the belief recommendations. Generally the recommendations are like pretty accurate, but they're not very creative and often most highly recommend the beliefs that are the most general purpose, i.e. the beliefs with the lowest performance floors but not the beliefs with the highest performance ceilings. Asking the ai to be really creative with their pantheon choices and tightly tailor their early and mid-game decisions around their pantheons is asking a lot of the ai code (as far as I understand how it functions), so I don't think its really that much of a problem. I am surprised that Goddess of Beauty, God-King, and God of All Creation are not picked more. I have seen Goddess of the Home in literally every game I've played in the last month and it seems quite strong in ai hands given that they aren't as hampered by unhappiness.

 

[John.B.C]

Because it is not. Years ago, AI would just bash their units without regard. You set up a semi choke and the AI will feed their entire army into it. AI could take cities sure, but then they would have no units to actually keep it. The tactical AI we have now is leaps and bounds better than it was before. Now AI can preserve their units, divide their force well, and even use bait manuvers.

Maybe a bug, I have not seen this in my own game.

I may be wrong, but this may just be a quirk of AI development? That AI can't think like humans do and so they aren't capable of long term planning.

I completely disagree ... I wonder if you even played back then .. they would take damage YES, but they'd also RETREAT if very damaged - which means that a healthy unit would replace it's damaged counterpart - just to continue the process! Sometimes a city fell in just a few turns, if one had failed to build walls. One was never safe!

I mean back then, a coastal city was truly lost if the AI showed up with an armada and if you didn't have a network of roads running along the coast and tons of ranged units on land - and even if that was the case the city was sometimes captured anyways!!! So many games I just raged quit ... cause of being 'hammered' by melee naval, which did so much damage even to cities with arsenals!


Today you don't even have to care ... defending is easy due to AI meele units doing nothing ... Even if a city lacks wall and you have but one lousy unit in it - against 10 AI melee units - the AI won't do anything with it's meele units.

I acctually remember how much I feared the city promotion ability on melee units (it was fairly new then in 2020-21 I think). Back then, that promotion acctually meant something in the hands of the AI. Funny thing is that it was acctually the AI teaching me about the danger of this promotion!

Check this post for my first entry on this issue (one year ago)

New Version - 2.7.3 (October 7, 2022)

Why doesn't the AI attack by melee units from inside the city? It looks like the AI thinks that the unit after the attack will be in the middle of the enemies, and not remain inside the city. In this case, the city fired on the weakest spearman and the AI could well kill him with an additional...

forums.civfanatics.com

 

[ma_kuh]

AI used to win CV with 37%, and Science was less than 20%. Here's a link for those interested in past results. I don't think it's a matter of complexity.

I was expecting more like 80% winrates with religion

I know you're speaking hyperbolically, but it's difficult to really say if Reformation is a game-winner or not. In any given game, 5/8 of players will have a religion, so if all 5 had equal chances of winning and 100% of wins involved a player with a religion, they'd be reported at 20% win rate. Reformation may be a more meaningful comparison than Enhancement, since Enhancing only requires a certain amount of accumulated faith -- and perhaps we see that, as the 16% figure and 17% are fairly close.

What is the win rate for Pantheon as the highest religious presence, I wonder. Can someone with just a Pantheon compete with the 22% win rate of Reformation players?

 

[John.B.C]

New Version - 2.7.3 (October 7, 2022)

AI also retreats when a unit is damaged, even by just 1 point.

forums.civfanatics.com

This has gone on for over 2 years now, with idling melee units NOT attacking cities and people ask why there are no domination victories?

 

[andersw]

Mainly because of the phenomena of them no longer using meele units to attack cities - even though the units carry the reduced city damage promotion - and even if the city is either heavily swarmed or completely surrunded by meele units, together with being without any ability to defend it self. In any scenario, the AI meele units will just 'hang around' without causing any damage to the city.

Unfortunatley, this still hasn't been fixed, since I posted about this almost 2 years ago! I don't even see anyone recognize this big flaw in the strategic AI as being a problem at all (why are diety players silent about this?). I personally remember the days when I was totally raped if a city was swarmed by meele units (it was especially very hard to defend costal cities against meele AI ships!). Today one just need walls and be careful about taking out whatever ranged unit that comes within range of the city.

The AI was perhaps better at domination victories a few years ago - it was at least better at capturing cities - that's for sure.

I agree that AIs dont capture cities correct, I shouldnt be able to snipe cities at 0 health with scouts, which happens too often.

 

[Stalker0]

I think its useful to do a little side by side comparison.

Here is 3.6

Spoiler :

And now 3.10.14

Spoiler :

It should be noted that the 3.6 version had a third more games played than 3.10.....so we may some variance in 3.10, aka the numbers are quite as tight.


Its actually wild how much worse the warmongers are doing (or if they are doing well, its no longer by domination). Look at the zulus, dramatically weaker, and now a lot of their wins are by time....they just aren't able to close out the win and take those capitals.

A decent uptick for France with their recent buff, looks like Huns are still weak (maybe teh AI is having trouble with the huns ability). The biggest one for me is Mongolia, I mean when I played them I felt unstoppable with their current bonus, yet the AI is having trouble getting the W with them.

 

[ma_kuh]

I wonder how much of that is their political game, and their other flavors more generally. Like do they just peak early and then have the world united against them?

I sort of wonder how a few hundred games of a single generalist civ played with random personalities would go, if you could assign "wins" based on just their flavor profile.

But then you'd also need to control for starts and such.