(1-VT) Remove Difficulty Randomization of AI Choices

[InkAxis]

That's interesting, I have a couple questions then.
So in your opinion would it be better if all difficulties had the same randomization, as currently I believe diety/emperor has less options?
What if the randomization was weighted based on how much the AI valued the top three options?
Do we know how bad the scoring algorithm is?

 

[ma_kuh]

From the original discussion thread:

not quite true, in lower difficulty levels the search depth for the best moves is reduced. but i don't know if it makes much difference in practice

anyway the effect (and maybe even intention) of the randomization was to paper over AI stupidity. and that may still be a good thing.

imagine this choice:

* option A -> score 500
* option B -> score 495

should AI always pick A? probably not.

now you can argue that we need to adjust the scoring to take into account that the player already has a lot of A and thus the score for A should be successively reduced but that is much more complex than some randomization.

and it's important to mention that the AI makes weighted choices! so if the scores are 500 and 100 they will be picked in a 5:1 ratio (and not 1:1) as you might naively expect.

 

[CppMaster]

when the scoring algorithm is not perfect but you have many repetitions

Ok, so the scoring algorithm need fixing if it provides wrong values. Making AI consistent with choosing the option with the highest score would make flawes of it more noticable, so there is more chance to fix it.

 

[InkAxis]

The fact that it makes weighted choices is very important. That completely changes my vote, honestly should be mentioned in the OP because it isn't really "random" then.

 

[CppMaster]

The fact that it makes weighted choices is very important. That completely changes my vote, honestly should be mentioned in the OP because it isn't really "random" then.

Random with some probability distribution is still random, as opposed to deterministic.

 

[InkAxis]

Of course, but it's very different and not mentioned at all.

 

[Agiwan]

Maybe it can help solve (a little) the desyncs problem in MP games ? I know it is not the subject at all, but it can have a good side effect.

 

[josh4]

always picking the "top" choice even when there are multiple option with a similar score makes the AI worse. not better.

Does it mean if we reserve the proposal: random including deity, then we get china, india as progress, authority japan, etc? Might be very interesting

 

[ma_kuh]

What would the process for making this a menu choice look like? (Not asking for now, but maybe during ratification.) The benefits --evaluating AI behavior, running analytics tests-- are great, but randomness is how hidden synergies are found, and letting the AI "participate" in that is healthy overall, for those that want to see it.

 

[Rekk]

dang, should have vetoed this. i kind of assumed you had given up on it

always picking the "top" choice even when there are multiple option with a similar score makes the AI worse. not better.

Should we be instead adding randomized choices to deity AI?

 

[nekokon]

Surprisingly seeing this many yes votes. Feel like all the discussion in proposal thread got ignored or something.
Ilteroi already clarified the current "random" choice is weighted thus would provide better choice overall than picking only top score. Did ppl miss that part ?

 

[josh4]

Did ppl miss that part ?

AFAIK many pplplay difficulties where there's no randomization at all. But maybe we should do the opposite, try one version all random?

 

[CppMaster]

Surprisingly seeing this many yes votes. Feel like all the discussion in proposal thread got ignored or something.
Ilteroi already clarified the current "random" choice is weighted thus would provide better choice overall than picking only top score. Did ppl miss that part ?

No. It's better to get $10 instead of $1 100% of times, than 90% of times.

 

[CppMaster]

but randomness is how hidden synergies are found

Both synergies and anti-synergies, so choices that don't make any sense.

 

[ma_kuh]

Throwing a game by trying something new isn't uncommon though, in this game or others. Exploration of what can work is sometimes half the fun. And as I understand, it sounds like deity (what you might consider "everyone doing their best") already does that. It's organic to try things that don't seem best every time, sometimes just because you're risking something paying off that usually doesn't.

I won't pretend like the AI is doing this super intentionally. I think this proposal is good, and I think leaving it as-is would be fine. I think what would be even better is making it an easy-to-toggle option, so that those who want "quick match" style games can play with random Progress Attilas, and those who want "try hard" mode can still play at their level, but against [allegedly] optimal play.

 

[CppMaster]

Throwing a game by trying something new isn't uncommon though, in this game or others. Exploration of what can work is sometimes half the fun.

It does make sense for players that can learn from this, but not for AI that doesn't. AI could learn from exploration if it was trained using a reinforced learning algorithm, but it's not the case for VP.

 

[ma_kuh]

Fair enough. Like I said, for the recent analytics projects we've been running on AI performance, having a mode where they aren't accidentally ruining themselves is ideal.

 

[nekokon]

No. It's better to get $10 instead of $1 100% of times, than 90% of times.

Gameplay variety aside, that's only true if you assume the scoring system is perfect. In case of top choice at 100 and 2nd choice at 99 due to a slight mis-scoring, weighted random choice would still have 49.5% chance of taking the actual better option, rather than a 100% chance of picking the worse option.
Secondly it makes AI very predictable/railroaded into the same pattern, thus it's actually harder to judge if the scoring is good enough (since you can only see 1 out come every time in the same situation). If there're multiple outcomes you can at least deduce how the AI scored specific options based on their weight, and you can match it with your own experience to know if that scoring is good enough or not to adjust accordingly. Having AI only pick the top options you will never know if the 2nd choice is 99 vs 100 or 10 vs 100.
And lastly, gameplay variety.

 

[josh4]

No. It's better to get $10 instead of $1 100% of times, than 90% of times.

there are plenty of subpar policy choice cause of no randomness. imagine siam progress, or authority carthage with their boats.

 

[CrazyG]

Should we be instead adding randomized choices to deity AI?

I think this would fun to try, at least briefly. Partially because the scoring system isn't perfect, but also because it would cause variety.

A good example is that Oracle almost always gets built before Parthenon on Deity. This makes the parthenon a very safe choice for the player.

It also means there are AI who waste hammers not getting the Oracle. There is easily a situation where a random fluctuation to Drama before Philosophy not only hurts the player, but could help the AI, because now they get a wonder, where as before they would have missed it.