3.6 AI-only Game Statistics

[Enginseer]

Venice - Venice is gonna venice; he was top 5 back in v3.2, and now that trade route gold has been reduced, he's down at the bottom. They have changed so much so many times it makes my head spin to suggest changing them again. But, I think one of the biggest problems with Venice right now is that it only has 1 city that can produce GPPs. This either needs to be changed for Venice, or they need some way to compensate for it, because right now they aren't competitive as a CV civ, even though they have an entire wonder dedicated to going that route.

• proposal: Allow Venetian puppet cities to generate GPPs

But...they do generate GPP, their default focus just prevents them from prioritizing specialists.

 

[pineappledan]

But...they do generate GPP, their default focus just prevents them from prioritizing specialists.

View attachment 664922

If true then that’s a stealth buff that I wasn’t aware of. You’re just floating over the merchant specialist tooltip; Is there a progress bar for the next GMerchant in that city? are you sure those merchant points in that puppet are going anywhere?

Edit: oh wait that IS the progress bar. Lol. I don’t recognize it with that city view UI

 

[zeofig]

This is an unfounded claim. I would need to see some actual analysis comparing game results and correlating them with certain approach or win flavours before I take it seriously.
It will be very hard to make a convincing argument that it is the flavours that are hurting AI and not their victory emphases, because flavours for civs oriented towards certain victory types (eg. a domination civ) tend to be very similar.

The AI flavours are suited to the civs and their kits. Civs oriented towards domination, for instance, have high war and offense flavours. You can be reasonably sure that all AI will perform substantially worse with random flavours.
I am not, however, saying that all the flavours are perfect. I think Morocco, for instance, is too passive for his own good. But if there is an effect on win rate, it is not large.

Aside from that, @Recursive put in a monumental effort to adjust flavours to make them more suitable, and even created a third set of primary and secondary VictoryPursuit attributes that guide AI diplomacy and win biases. These VictoryPursuit bonuses are set individually for each civ and aim them toward 1 or 2 most likely victories, based on their overall kit, and the biases for each civ seem quite reasonable to me.

No, the far more likely case is that the AI is either not particularly good at using a component, irrespective of their flavours, or the component is weak, plain and simple. I think it does an incredible disservice to the years of work that people like Recursive have done on the diplomatic and AI flavors to cast aspersions at them here.

I didn't run the games, but if I had I would have played them on Warlord, where the AI has no bonuses, so the bonuses cannot bias results.

That being said, there is merit to running the tests on Emperor, which is the difficulty that the devs have publicly made statements about trying to balance around.

perfect balance would mean removing all differences between civs; we don't want balance if it comes at the cost of unique abilities etc.

This can't possibly be right.

The flavours may not be as high as you would like, but they are higher than any other building babylon is going to aim for in that era. As I understand it, the AI also doesn't consider building flavors all that much anyways

An unfounded claim? My "claim" is that the effect of flavours on winrate is currently unknown, relative to UAs etc. Your claims, that the effects of flavours are not large etc etc, are what is unfounded, since there is essentially no data that could currently support them. I certainly appreciate the effort of making this data and previous efforts to balance the flavours (and everything else). I'm just saying, why not simulate the flavours separately?

It could be dominated and I image the balancing would include it, since the default way of playing is with those personalities.

If civs are balanced around AI personalities, how does that affect the player?

 

[pineappledan]

So you’re proposing that AI games should be run without any attempt to get the AI to play to a civ’s strengths? What would that do? The flavours will still all be turned on in a regular game, so that won’t simulate anything close to a normal game. The AI games reflect a real game environment at the default difficulty, size, speed, and map settings that are the main target to be balanced for. AI players with no flavours is not a default setting.

We have also discussed if the tests could have been run without AI difficulty bonuses. There are a myriad of things that aren’t being controlled for, but calling it “silly”, for someone to spend weeks of their time to run hundreds of games in a way that wasn’t precisely how you would have done it is not cool.

Regardless, it is not the data we have, so there isn’t much sense in discussing it. I’ll defer to recursive, who knows more about the AI personalities, and to Vern, who knows much more about his test environment. If you want to run your own test games with wiped AI personalities I think he could help you get started.

 

[zeofig]

What I'm suggesting is that AI flavours could and should be simulated without civ abilities, and civ abilities without flavours (or with random personalities), if people are serious about balancing using AI games. What would that do? I think it's obvious and I've explained it, but it would let us see how strong the effects of flavours are compared to civ abilities. At a minimum I would also want to see both Warlord and Emperor data to take balancing conclusions seriously. Personally I think balancing using AI games has a lot of other issues, but you seem extremely keen to do it, and I have no doubt you possess the energy to keep on posting until whatever you have in mind comes into being. So really pineapple, could you not yourself put some of that energy towards generating some more meaningful data? After all you're the one who wants to implement a whole list of changes based on this data, but I think I've given some valid criticisms that a number of people here agree with.

 

[pineappledan]

If you’re unhappy with the data presented thus far, that doesn’t make it my, or Vern’s or anyone else’s responsibility to satisfy you. You’re the one who seems to have energy to burn and a mouth to run.

Many changes have already been implemented without a dataset this robust, and I would prefer this approach to what we have been doing up to this point. This particular dataset took weeks for Vern to generate and I did, in fact, spend some of my own time giving input on it as it was being made.

So yes, I will continue to advocate for balance changes based on test games. I have very little patience for people saying we have too little, or the wrong kind of data to be making changes. Gazebo used to run AI test games as his main way of balancing, and We have gone years since his departure moving the mod forward without anything but user feedback. We’ve never had this level of transparency or input to interpret and discuss test game results until now, so I’m aghast that this is suddenly not good enough for some people.

 

[zeofig]

If you’re unhappy with the data presented thus far, that doesn’t make it my, or Vern’s or anyone else’s responsibility to satisfy you. You’re the one who seems to have energy to burn and a mouth to run.

As I've said, I am not unhappy with the data, I just think it's insufficient to justify balancing decisions. You say it's robust, but I say it isn't. At least including separate data for flavours would make it so much more meaningful and interesting. I appreciate the effort it takes, I would just really like to see it improved further if it's supposed to be for balancing.

As for "energy to burn and a mouth to run", I really can't think of a better way to describe you. I've seen you doggedly pursue changes to this mod time and time again which are based on your own speculation and superficial aesthetic concerns, rather than actual experience and enjoyment playing the mod. These changes are often implemented because you just keep on going, while ignoring most criticism. I know from experience you can, and will, do this all day and all week. Frankly there's not much I can do other than watch you slowly make VP worse, since unlike you I do not have all day. I can't be bothered arguing with you any more. As usual you haven't actually responded to my points which were presented quite politely and reasonably, and have instead deferred responsibility for your own suggestions and dived into name-calling.

 

[azum4roll]

@L. Vern , do you still have logs for at least one Babylon game? There should be two building scores for each turn in one of those (CityStrategyAI?), first one purely from flavours, and the second after factoring in everything else. We can see whether flavours play a large factor here.

 

[CppMaster]

If civs are balanced around AI personalities, how does that affect the player?

Civs are balanced for both AI and humans. In case of AI, their personalities are included, because that's the default way of playing.

 

[L. Vern]

OK, but how do you know this data isn't dominated by the AI personalities/preferences (the main point of my post)? I think some of them might be extremely disadvantageous

I am quite sure a lot of the results are impacted significantly by flavors and personality, but there are 2 points I think are important to keep in mind as well:
1. The purpose of this exercise is to get an idea of how civs will behave and perform in something close to what people are actually playing on, hence using default size, VP map, settings, most common difficulty

Spoiler :

as its under these conditions that most players will be seeing the AI civs and under these conditions I think we should focus on making sure none of them stand out in a not fun or not interesting way (note that in a standard size game there will still be 7 AIs and their interactions with one another will still be better modelled by this approach than with equalizing all of their flavor and personality numbers)
2.
While in general I totally agree that some flavors are disadvantageous and may be a good place to increase performance, having a wide variety of personalities makes the game more interesting and keeping those is preferable in many cases

 

[L. Vern]

@L. Vern , do you still have logs for at least one Babylon game? There should be two building scores for each turn in one of those (CityStrategyAI?), first one purely from flavours, and the second after factoring in everything else. We can see whether flavours play a large factor here.

My script zips up and saves the entire logs folder for every game, they are quite large though so I've just uploaded the most recent 3 games with Babylon in it:

logs-babylon-1.zip

drive.google.com

logs-babylon-2.zip

drive.google.com

logs-babylon-3.zip

drive.google.com

 

[L. Vern]

I wanted to also make a few miscellaneous comments as a response to a few things I've seen in this thread, particularly those where I agree with the premise but draw a slightly different conclusion:

At a minimum I would also want to see both Warlord and Emperor data to take balancing conclusions seriously

Why not on Chieftain (or Warlord?), so it's bonus neutral?

Even then I would be cautious about balancing based on AI games. The AI gets numerical bonuses that interact with other bonuses in ways that a human won't experience (although I couldn't find what difficult these games run at)

This is a very good point, a similar one which has been made by Recursive a couple of weeks ago after which I did start running Warlord difficulty games Will get those out when it reaches parity with number of Emperor games or congress voting, whichever is sooner

I am not unhappy with the data, I just think it's insufficient to justify balancing decisions

Also agreed, should really only be a place to start off discussions, after that it's up to the community and their combined experiences to decide the tradeoff of a civ being very fun to play as vs being oppressive/snowbally/unfun to play against. I'm in general against nerfing things in a pve game but sometimes it can be healthy, and that decision can certainly not be done on the basis of these numbers

the effect of flavours on winrate is currently unknown, relative to UAs etc

Completely agree, but in a case like this it's not really feasible nor useful to isolate the effects of each variable on the outcome - we can still observe the combined end result which is what's important to peoples experiences in-game

So regarding balancing with and without referencing data:

Many changes have already been implemented without a dataset

Personally I think balancing using AI games has a lot of other issues

and a myriad of others:

What I've personally seen to be really useful with these datasets is helping to drive positive change when combined with player feedback and general community sentiment.

For example, in 2021-2022 I've noticed a lot of people on the forums and the discord talking about how the later eras are jarringly short and how culture victories are too easy/common, and I think seeing the numbers from AI games helped convince the rest that it was more than just anecdotal and overall the game did get better from those changes.

It's a useful addition to, but certainly not a replacement for player discussion for the most important factor of what should drive change, i.e. if it makes the game more fun or not

 

[zeofig]

Civs are balanced for both AI and humans. In case of AI, their personalities are included, because that's the default way of playing.

I do agree they should be balanced taking into account their personalities to an extent, yeah.

Vern, your automation repo looks very nice. I'm not sure it will be easy for me to get running as I currently play civ5 through proton+flatpak, but I will eventually have a go.

 

[pineappledan]

I haven’t played a game with Venice since 4 years ago. It’s not my jam. But @Enginseer is a venice main and we’ve been talking about them on the discord, going through each of their 6(!) components piece-by-piece. It’s pretty clear Venice has been neglected for a while.
- his St Mark’s Basilica is missing 2 abilities that the base national monument has (doesn’t give gap for adopting policies or when the city grows)
- at some point, puppets were blocked from building anything with a max player or max on the globe cap, meaning that puppets cannot build national wonders or world wonders. No exception was added for Venice, so they also can’t build guilds or refineries etc in any city other than the capital.
- An exception was added for Venice so that his puppets can generate GPPs by working specialists, but Venice still has no way to assign specialists outside his capital, and the governors in puppets only work merchants (Governor gold focus). So even if he could build guilds outside his capital, he would never fill the artist/writer/musician slots
- Aside from this hidden ability to generate Gmerchant points outside his capital — something every other civ can also do — venice has no other bonuses to help him generate his unique Merchant of Venice. He used to have +20% GMerchant rate for every puppet on empire, but now he gets a free 1 at trade and that’s his only help.

There’s more, but those are the big ones that jump out. Venice is overdue for some love, or at least some bug fixes

 

[gozar]

This is great data. While it is good that the distribution of victory types is still balanced, it strikes me that there is a huge difference in the timing. All of the early victories are cultural or domination.

An early domination is more understandable. Sometimes there is simply a runaway warmonger.

The median cultural victory in terms of time occured before 370. Some of these early cultural victories were by Civs that are more geared toward other victory types (Germany, England, Korea). There were only 3 diplo victories before turn 370. The earliest scientific victory was at 379.

To me this indicates that the most important consideration to winning a diplo or science is preventing an early cultural victory. This also confirms a suspicion that I think many of us have had that sometimes you win a cultural victory by accident. It is hard to believe that those victories by Germany and Korea were not simply runaways that had multiple paths to victory but simply went for the fastest and easiest.

I think this a bigger issue than balancing Civs. In my opinion, this indicates that there is too much tourism in the midgame. I suspect that late game tourism needs a buff to compensate.

 

[CppMaster]

It's ok that science victory is the latest (expect time), because it's a sim city.

 

[pineappledan]

Domination and Cultural victory are both indefinitely preventable, and you can even reverse a civ's progress towards one.

You can stop a domination victory by not letting yourself get killed
You can stop a cultural victory by having so much generation that you stonewall someone else's generation forever

Diplomatic victory is slower, but hypothetically unstoppable. The UN will unlocks very late and sessions shorten to 1 every 5 turns in information era, at which point the player who already has the most WC votes starts getting +1 WC vote per session. They will get the votes they need eventually unless another victory happens sooner.

Science victory is even more unstoppable, because tech progression is irreversible. All you can do is slow it down, but you can't truly set someone back. SV is effectively time victory that players can make faster by generating lots of . It has to be the slowest victory, because it is functionally a game clock that sets a time limit for all the other victories to happen inside of.

So no, the speed at which the victories happen is correct. The more preventable ones have to happen faster if they are to happen at all.

 

[N.Core]

While it is good that the distribution of victory types is still balanced, it strikes me that there is a huge difference in the timing. All of the early victories are cultural or domination.

Yeah, timing is definitely something that should be balanced IMO.

If you're a Science civ, you're hoping that someone doesn't win other victory types quickly enough so that can allow you to win SV.
Sadly, some disagree with this and prefer the current SV as it is.

Data already shows that you can only win SV if other civs are messed up and can't win other victory types, essentially turning SV into a "pity victory" that is only attainable if other civs allow you.

 

[axatin]

If you're a Science civ, you're hoping that someone doesn't win other victory types quickly enough so that can allow you to win SV.

SV is the only victory type without any interaction with other civs, so it has to be like that to some extent, otherwise it would always be the strongest victory type.

Some of these early cultural victories were by Civs that are more geared toward other victory types (Germany, England, Korea).

A part of the problem is that it's not necessary to actively do much for a culture victory. A diplo victory requires constant investments, and for a SV it's at least necessary to use processes in the late game, but tourism mostly just accumulates passively. The two or three buildings you have to build don't make much of a difference. A solution would be adding a tourism process and nerfing passive tourism sources.

 

[pineappledan]

The difference in the mix of victories also reflects a difference in the mix of civs in the game. There are different numbers of civs biased towards a CV than there are ones geared for SV, for instance

Ideally, victory type frequencies would match up to how many civs overall are focused on each VC. Another reason why I think domination needs to take more of the pie in particular, as it is by far the most common type of civ bonus.