Single Player bugs and crashes v36 plus (SVN) - After the 24th of October 2015

[Thunderbrd]

I have just finished updating all the units but if you look in Thebes you will see a number of guard units that have used 16 or more of the 5 exp they have. With the latest upgrade of unit they did not even increase the 16 it just stayed the same and allowed me to give the new promotion. I think you can give one of the investigators 4 crime fighting promotions right now to see what is going on. It should not be able to get any more but it is still glowing blue. It may be the crime fighter can get investigation promo.

Ok, will look at this as soon as I can. Probably later today.

That issue with quality promos doing nothing still stands. Could it have anything to do with strength promos being removed with a quality promo, so you are essentially in retraining mode?

When you select a quality promo, you do lose your strength promos and you should then be able to reselect them. As for 'quality promotions are doing nothing' what do you mean exactly?

What needs work to resolve this are all the buildings, wonders and other things which increase production or military unit production. The AI builds all those buildings and there are even slowdowns then AI players build lots of units in the same turn.

A building review is needed, yes, but what also needs done is to adjust costs of units across the board according to the new cost assignment methods. However, this takes deeper evaluation. As do the buildings I suppose.

But this particular case wasn't so much caused by it being too easy to build units so much as the flaw that was making the AI build them like mad, thinking they could never get as much as they wanted.

Please stop pretending that the outcome of a large stack vs a large stack battle depends on good luck or bad luck. According to the law of large numbers, the bigger the stacks involved, the closer the result will be to the average. i.e. the influence of luck becomes smaller and smaller as a relevant factor, until you can completely ignore it.

If you only count up the amount of times the first round of battle takes place, yes. But the likelihood of a success in battle is only pure on that first round. It used to be that every round was equal. It no longer is. Thus the reliability of the law of large numbers only now applies to those first rounds and subsequent rounds get more and more difficult and less likely to make a difference and less damaging when they do. So its a world of difference from the original vanilla combats which the AI is still evaluating based on.

I repeated the attack in the save that I posted two times more (with new random seed on reload of course) and the result was the same: AI lost 145+ units and I lost 1 unit. I'm confident that repeating the battle several more times will not alter its outcome in any significant way.

Probably true.

And if you think that an attack that sacrifices 145+ units to kill one (average) unit of mine is in any way or situation a good decision, then I think you are violating common sense.

I'm not saying the AI is smart here. Just not... 'bugged' in the classical sense of the term.

However you are right that smarter AI requires more calculations which leads potentially to larger turn times. But I think that using smarter maths could improve this greatly. I might have an idea....

I do as well. I'll take some time here and respond to your comments and suggestions but rest assured I have been looking into this section of code as I see it process during debug efforts lately. And I think I've figured out how to fairly simply adjust the way the AI perceives power and stack differentials so that its more accurate to the way battle recalculates every round now.

Currently there are 2 flaws.
1) generally speaking, stacks add up the value of all its units and compares that to the sum value of all the units in the opposing stacks. What this means is that there is no consideration for how the ratio differences between the individual units in those stacks will dramatically adjust the value of a point of strength based on how much stronger one unit is over another.
Solution) Adjust this comparison so that its based on the AVERAGE strength of a stack vs the AVERAGE strength of an opposing stack, thus showing a bit more accurately the strength variation differential between the stacks, not just in overall volume. Adjust the str totals by the ratio difference in the averages. That should give a much better picture.

2) when the strength totals are derived, each unit's contribution is modified by some of the unit's tags. These modifiers are questionable because most of them are considering some strange tags to be including in it and not being modified very rationally for this particular purpose, which is one reason for slowdown as well. Koshling asked me to include these modifiers to the unit value but his answers to why we were doing this did not quite make the sense it should've made to me and I was left to think this was just a generic unit value tally. But this is supposed to be a quick, combat value tally and imo very few tags should make much of a difference to that. So some review and improvement to that side of things is in order.

Hmm if you say that the minimum damage is gone, then that opens up mathematical possibilities to speed up combat calculations.

Mostly, yes. I think there is still a small amount of min damage per hit, like 1 pt.

1) assuming equal hitpoints, the outcome of the battle depends solely on the ratio of strenghts. In other words: a strength 4 unit fighting a strength 2 unit (ratio 2:1) would have the same average outcome as a strength 6 unit fighting a strength 3 unit (also ratio 2:1).

There are other factors but assuming those factors, such as accuracy, dodge, armor, puncture, etc... are not in place, yes. Although, you must consider too that combat modifiers play into determining final strengths for each unit in a given battle.

2) assuming the same ratio in strength, the battle depends solely on the ratio of hitpoints. In other words: a 100 hp unit attacking a 50 hp unit has the same AVERAGE chance of winning as a 50 hp unit attacking a 25 hp unit. N.B. smaller amounts of hitpoints would cause a higher average "overkill" (damage wasted because the target unit is already dead) but let's neglect that.

hmm... final strength is adjusted by % of HP left. So then we are back to #1 ALSO.
Let's say a str 6 fighting a str 3 unit. But the str 6 unit has 50% of its Max HP remaining. The str 3 unit actually has the upper hand here. Because both end up at str 3 when they start fighting but the str 6, 50% damaged unit is also 50% damaged and has (lets assume) roughly 50hp left while the str 3 unit has 100hp left. So if they trade blows every round (both hit every round) then you will have the str 6 unit having lost all its hp before the str 3 unit does (and it will be losing more str each round than the str 3 unit because its ratio of str per hp is higher still.)

Additionally:
Old way: Every round of battle the combat odds stay according to the first round's odds evaluation. Thus as a unit got damaged once it entered battle, it would not weaken until after battle was fully resolved.

New way: Every round of battle the combat odds recalculate according to the new HP. Injured units thus lose actual combat strength according to those injuries with each round. Thus if I damage my opponent the first round but I manage to avoid damage, my odds dramatically improve for the next round.

Always: Odds form the basis of likelihood to score a hit on your opponent in a round and also forms the basis of the amount of damage you do when you do score a hit.

3) the AVERAGE outcome is one of these: a) defender dies, attacker gets x amount of damage. b) attacker dies, defender gets x amount of damage, or c) both die. If you depict (a) as a positive number (i.e. +x) and (b) as a negative number (i.e. -x) and zero if both die, then the average outcome can be rendered in a single number (which can be either positive or negative).

This holds true on a round by round basis but since we re-calculate all odds each round of battle, a total full battle odds is hard to pin down as it depends dramatically on the outcome of each round that takes place. It tends to make the strength/health differential between units mean exponentially more than it did in vanilla.

..then the outcome of all battles can be plotted in the following formula:

Average combat outcome = function of (input ratio of strengths; input ratio of hitpoints)

It is not so simple because strength ratios change as hit points are lost.

This is a 3 dimensional plot: on the X axis you plot the ratio of strengths, on the Y axis you plot the ratio of hitpoints, and on the Z axis you plot the average battle outcome.

The result will be a curved 2 dimensional plane in this 3 dimensional plot.

Now you can run a lot of simulated battles to construct this 2 dimensional plane. Make sure that you space out the possibilities: 1-100 hitpoints vs 1-100 hitpoints gives 10,000 possibilities, but if you go in increments of 5 hitpoints, then it is only 20x20 = 400 possibilities, less if you remove duplicates, like 100 hp vs 50 hp is the same as 50 hp vs 25 hp. Size Matters increases the range of hitpoints, and if I remember correctly, every level of the unit gives another 1 hp, but the principle is the same.

Since you only have to construct this plot once, calculation time is less critical (just let it run overnight) so you can e.g. take into account that the weaker unit drops strength faster than the stronger unit so the stronger unit has an extra advantage.

Once you have this 2 dimensional plane sufficiently mapped out (in the form of many, many calculated values in a table: input x, input y, and calculated result z) then you enter this table into a statistics program, and let it calculate a formula that is "best fit" in the form

Average combat outcome (z axis) = formula (input x axis which is ratio of strengths; input y axis which is ratio of hitpoints)

It is a lot of work and computer calculation time, but once it is done, calculating average battle odds would be much faster for the AI; it only has to put ratio of strengths and ratio of hitpoints into a single formula and out rolls the average combat outcome.

Three things complicate this: 1) ranged attacks 2) first strikes 3) withdrawal chance.
The first two are straight damage dealers that can be calculated before calculating the ratio of hitpoints. The third one, withdrawal, happens not that often and if it happens, it happens when most of the battle is already done. So for stack vs stack attack, it won't be much difference if you leave out withdrawal chance.

There are methods for odds calculation on a unit vs unit basis - and its got a fair degree of accuracy and takes into account first strikes, withdrawal and more. But running that for each unit against each unit in each stack is what starts taking a LOT of time on these longest turns (that must be done for the actual attack determination.) For a faster stack vs stack evaluation that is in place to determine if the stack should try to attack or not, a more simplistic evaluation becomes necessary, and it's not THAT bad... just needs some adjustments to get it to be a little smarter is all.

 

[tmv]

When you select a quality promo, you do lose your strength promos and you should then be able to reselect them. As for 'quality promotions are doing nothing' what do you mean exactly?

I have a unit with several strength promos (usually at least 2) when I get to a quality promo, I select the quality promo, the strength promos vanish (and leave me with a few retraining options), the quality promo is not incorporated (is that the correct word?) and vanishes as well, and the exp counter is reset.

That's why I was asking if the retraining option from the strength promos was affecting the quality promo, because this problem did not exist before your (badly needed) tweak to switch off quality promos during (regular) retraining.

It's also the reason I was asking if it was better to keep the number of quality promos in mind when upgrading, so the right number could be applied before any retraining - this might also solve the issue I mentioned above.

Edit: This is an important issue, because the quality promos right now are broken. I have not yet tried to avoid strength promos (I am going to do that next), but this is definitely hurting your best units.

 

[Dancing Hoskuld]

... and would like to see some reason beyond early stages of the game where you can or need to tax for gold. In the early game there is some gold pressure, but by somewhere in Classic there is very little pressure and the pace of acquiring more gold outgrows your expenses.

Playing Noble/Snail/Huge I am finding the gold pressure stays until late Classical. Unit upgrade costs help suck up a lot of money but only once was I not able to afford to upgrade all my units as they became available after Classical Lifestyle.

One thing I have noticed is that buildings defined in the XML with a commerce of 1 are providing 2 in game. I assume that this increase is due to scaling for size or speed.

A lot of buildings provide multiple benefits. This was done to get the AI to build them. I am not sure that is needed any more as the AI has come a long way. We should probably look at all the buildings that provide manufactured resources and reduce the other benefits. For example the Apiary provides two bonuses honey and wax plus , and , would not the bonuses themselves be good enough for the AI to choose it?

edit I find gold pressure to be hard to very hard up until somewhere after Tribalism where for awhile I can get out of trouble by building a new city to get rid of all the extra animal units I have sitting around.

 

[strategyonly]

. For example the Apiary provides two bonuses honey and wax plus , and , would not the bonuses themselves be good enough for the AI to choose it?

edit I find gold pressure to be hard to very hard up until somewhere after Tribalism where for awhile I can get out of trouble by building a new city to get rid of all the extra animal units I have sitting around.

The thing i an finding out lately is that u need ALOT of , especially after Classical, and right now there isnt enough IF u need to "upgrade" ur units to the next higher unit level, ie Guard to City Guard to Police etc etc, i am finding out in Modern Era alone, guards to police cost about 100,000 per city and by then ur lucky to have enough ONLY for 1 city . . at least for me that is . . then if u need the other units u'd have to go about 50 more turns or more to get them upgraded . . . again at least for me . . /. /

 

[Toffer90]

@TB:
I'll throw in an idea on combat evaluation for AI, it might not be helpful as I have no insight in the existing code used there.

The assumption my idea is based on:
The AI runs a risk analysis before engaging an opponent unit/stack in combat and figures out the amount of units it's willing to sacrifice in said offence before running a new risk analysis. The risk analysis compares all attacker against all defenders in one way or another
_____________________________________
My idea is that there should be a simple check after each unit vs unit battle that goes like this:

if defender HP > X (if the battle went much worse than expected)
----run emergency risk analysis (check total stack strength versus opponent again)
else
----attack with next unit

The HP value X would be stored/set during combat odds evaluation and somehow be based on the 10% worst outcomes for the attacker (AI) in said battle.

I might be off in my assumption, at any rate, if you need anyone to discuss code possibilities with you can just PM me, I might be able to trigger you to see new soloutions by me not being constrained by programming conventions.

 

[Dancing Hoskuld]

The thing i an finding out lately is that u need ALOT of , especially after Classical, and right now there isnt enough IF u need to "upgrade" ur units to the next higher unit level, ie Guard to City Guard to Police etc etc, i am finding out in Modern Era alone, guards to police cost about 100,000 per city and by then ur lucky to have enough ONLY for 1 city . . at least for me that is . . then if u need the other units u'd have to go about 50 more turns or more to get them upgraded . . . again at least for me . . /. /

Yes that is in the modern era which we have not even started to balance. In the late Classical early Medieval, where we are finalizing the balancing act, there is too much money, even for me

edit I am having one of those games where every time you build a fire pit it burns down in the same turn.

 

[JosEPh_II]

@DH and SO,

Have you updated to the latest SVN?

@TB,
Just had a very weird End of game. Was defeated in 13260BC by Japan in a Deity Epic game Standard map 5 AI. When Toku attacked, after taking city defenses down to 0 and killing my Tribal Guardian and Slingers only my Great General Tracker and Story Teller was left. The next turn he attacked again. This time the screen froze. The spearpoint cursor would change to runing man and then back this continued for almost 10 minutes before I did a Ctrl/Alt/Del and Restart on computer.

When I went into the autosaves the last one was at 5911BC, well after the defeat date of 13260BC. Loading that 5911 save the game once loaded finally showed the You Are Defeated screen. The game was running itself until I Ctrl/Alt/Del out of it.

I think this is a perfect example of what Alberts2 has been trying to tell you in your posting with him yesterday here in this thread. I hope you utilize what he's recommending.

JosEPh

 

[Dancing Hoskuld]

@Joe, I have give up on my old game due to the infinite promotions problem. I have updated to the SVN and have started a new game.

 

[Taxman66]

Playing Noble/Snail/Huge I am finding the gold pressure stays until late Classical. Unit upgrade costs help suck up a lot of money but only once was I not able to afford to upgrade all my units as they became available after Classical Lifestyle.
.
.
.
edit I find gold pressure to be hard to very hard up until somewhere after Tribalism where for awhile I can get out of trouble by building a new city to get rid of all the extra animal units I have sitting around.

DH, that pressure in Classical is minimal at best, how often or much are you taxing ?

 

[Dancing Hoskuld]

DH, that pressure in Classical is minimal at best, how often or much are you taxing ?

100% naturally otherwise I would not have anything to put into science. Oh you mean how much of my tax is going into the treasury! I have espionage at 10% and science varies but is usually 60-70%. It goes up to 90% and stays there towards the end of the era.

 

[Thunderbrd]

I have a unit with several strength promos (usually at least 2) when I get to a quality promo, I select the quality promo, the strength promos vanish (and leave me with a few retraining options), the quality promo is not incorporated (is that the correct word?) and vanishes as well, and the exp counter is reset.

That's why I was asking if the retraining option from the strength promos was affecting the quality promo, because this problem did not exist before your (badly needed) tweak to switch off quality promos during (regular) retraining.

It's also the reason I was asking if it was better to keep the number of quality promos in mind when upgrading, so the right number could be applied before any retraining - this might also solve the issue I mentioned above.

Edit: This is an important issue, because the quality promos right now are broken. I have not yet tried to avoid strength promos (I am going to do that next), but this is definitely hurting your best units.

Ok. I think I see what is happening then. It's being assigned but being removed as quickly because by assigning it it has become invalid to the unit. Oops. Ok. That's fixable. I should have a dll posted later tonight that repairs that then.

@TB:
I'll throw in an idea on combat evaluation for AI, it might not be helpful as I have no insight in the existing code used there.

The assumption my idea is based on:
The AI runs a risk analysis before engaging an opponent unit/stack in combat and figures out the amount of units it's willing to sacrifice in said offence before running a new risk analysis. The risk analysis compares all attacker against all defenders in one way or another
_____________________________________
My idea is that there should be a simple check after each unit vs unit battle that goes like this:

if defender HP > X (if the battle went much worse than expected)
----run emergency risk analysis (check total stack strength versus opponent again)
else
----attack with next unit

The HP value X would be stored/set during combat odds evaluation and somehow be based on the 10% worst outcomes for the attacker (AI) in said battle.

I might be off in my assumption, at any rate, if you need anyone to discuss code possibilities with you can just PM me, I might be able to trigger you to see new soloutions by me not being constrained by programming conventions.

The AI routine reruns from scratch every time a decision is made. If a stack attacks, it completely rechecks what it wants to do afterwards. It never comes up with a measurement of how many times its willing to attack. AKA, it would take some special programming to do something along the lines of your suggestion, which isn't a bad one and could help with speeding things up a bit if done right. But it would mean a restructuring break from normal AI programming convention.

@TB,
Just had a very weird End of game. Was defeated in 13260BC by Japan in a Deity Epic game Standard map 5 AI. When Toku attacked, after taking city defenses down to 0 and killing my Tribal Guardian and Slingers only my Great General Tracker and Story Teller was left. The next turn he attacked again. This time the screen froze. The spearpoint cursor would change to runing man and then back this continued for almost 10 minutes before I did a Ctrl/Alt/Del and Restart on computer.

When I went into the autosaves the last one was at 5911BC, well after the defeat date of 13260BC. Loading that 5911 save the game once loaded finally showed the You Are Defeated screen. The game was running itself until I Ctrl/Alt/Del out of it.

I think this is a perfect example of what Alberts2 has been trying to tell you in your posting with him yesterday here in this thread. I hope you utilize what he's recommending.

JosEPh

1) Alberts wasn't making any recommendations so you must not have understood the conversation. We were having a discussion regarding some coding theories to address a rare problem that MAY have something to do with what you found.

2) What you found is a bug that's been with us since our AND roots. This used to happen to me when playing AND actually. The game always enters an infinite loop when it freezes (but allows you to save during the delay). If you reload the save you go into the You Are Defeated screen. I have not been willing to waste time on this issue. The game is already done and not needing to proceed and if you load up the game again you'll see what you missed so it's about as low priority as it gets. SO has reported this one about 4 times since I started keeping a debug queue with this thread. I'll look at it again by the end of this cycle during the freeze period perhaps.

@Joe, I have give up on my old game due to the infinite promotions problem. I have updated to the SVN and have started a new game.

It's still an existing problem and I have a pretty good idea why its taking place (has to do with a free promotion assigned by the unit being invalidated as soon as its processed onto the unit during the upgrade and you then getting a retrain for it) and suspect I've already inadvertently fixed it before you brought it up. I just haven't posted a new dll yet. But there's a few things I need to get updated here so should be later tonight. You dropped a save for me on that so it should be easy enough to ensure it works fine now.

 

[JosEPh_II]

@TB,
Okay, thanks for the details.

JosEPh

 

[Thunderbrd]

I have just finished updating all the units but if you look in Thebes you will see a number of guard units that have used 16 or more of the 5 exp they have. With the latest upgrade of unit they did not even increase the 16 it just stayed the same and allowed me to give the new promotion. I think you can give one of the investigators 4 crime fighting promotions right now to see what is going on. It should not be able to get any more but it is still glowing blue. It may be the crime fighter can get investigation promo.

That issue with quality promos doing nothing still stands. Could it have anything to do with strength promos being removed with a quality promo, so you are essentially in retraining mode?

These should both be addressed now.

 

[olduser]

But this particular case wasn't so much caused by it being too easy to build units so much as the flaw that was making the AI build them like mad, thinking they could never get as much as they wanted.

They do it because they can build and afford them especially in the higher difficulty levels because of all the bonuses they get. Combined with all the other production modifiers they get alot production so at some points in the game they might just run out of better things to build.

Or they where using the C2C strategy 'More is Better'

 

[JosEPh_II]

T-brd wrote:
2) What you found is a bug that's been with us since our AND roots. This used to happen to me when playing AND actually. The game always enters an infinite loop when it freezes (but allows you to save during the delay). If you reload the save you go into the You Are Defeated screen. I have not been willing to waste time on this issue. The game is already done and not needing to proceed and if you load up the game again you'll see what you missed so it's about as low priority as it gets. SO has reported this one about 4 times since I started keeping a debug queue with this thread. I'll look at it again by the end of this cycle during the freeze period perhaps.

So have you ever PM'd 45* to see what they did to fix this in AND? I'm pretty sure they no longer have this problem.

JosEPh

 

[JosEPh_II]

They do it because they can build and afford them especially in the higher difficulty levels because of all the bonuses they get. Combined with all the other production modifiers they get alot production so at some points in the game they might just run out of better things to build.

Or they where using the C2C strategy 'More is Better'

I just tightened up the iTrainPercent and iConstructPercent in the GameSpeedInfo xml so this should slow it down some. More game play will be needed to see how much more it should be "restricted".

Also the iGoldModifier was increased so there will be less gold on all GS.

ls612: iGoldModifier = Global percentage modifier for ALL costs. Higher equals less gold, lower equals more gold. Default is 100.

Inflation was also upped on all GS.

In addition there there is also a BBAI Tech Modifier that we had earlier zeroed out "0" on all Era (CIV4EraInfo), those have now had new values put in. Preh Era kept the "0" value but the rest were heavily increased. Should cut down on the "I'm in TH Era at 500AD" reports.

More can be done if further gameplay reports warrant it.

Also Deity level modifiers for Player were also "upped" to increase level of difficulty in many areas. Making the past addition of giving the AI 2 Band of Homo Sapiens more deadly, it will now kill you off much faster.

JosEPh

 

[tmv]

SVN 9308: All Education Knowledge Base Buildings are enabled at Sedentary Lifestyle.

 

[Thunderbrd]

They do it because they can build and afford them especially in the higher difficulty levels because of all the bonuses they get. Combined with all the other production modifiers they get alot production so at some points in the game they might just run out of better things to build.

Or they where using the C2C strategy 'More is Better'

Building more troops are not the default. Unless they are barbarians. The default selection is building gold or research if there's nothing better to do.

But when they are trying to build to a certain point to enable their attack stacks to be considered ready to send into action and they can't get enough units into the stacks to do it, that will certainly cause them to overbuild like crazy.

So have you ever PM'd 45* to see what they did to fix this in AND? I'm pretty sure they no longer have this problem.

JosEPh

No. I've not been willing to spend time on a bug to fix the end of the game moving into the end of the game when it's already the end of the game, particularly when there's a player workaround.

SVN 9308: All Education Knowledge Base Buildings are enabled at Sedentary Lifestyle.

Oh... yeah. Oops. Fixing now.

 

[jatkinson]

These should both be addressed now.

I was using the SVN and encountered tmv's exact same bug regarding the Quality Up (Unbelievable Expertise) promotion failing and reseting the xp counter for my Tom Haskell Warlord Slinger.

I upgraded to the latest SVN just released and loaded up a save from just a turn before he got promoted. The Quality Up worked great and I was able to select the Elite Might promotion again, he's now a Strength 6.0 Super Slinger with Arctic III (Perfect Mongoose World gave me a polar start - had to spend 10 turns migrating south to find greener pastures), Combat V, Heroic I, Guerilla III, Woodsman II, Hunter I, To the Death IV and I forget what other promotions he has, he's level 14 . My only query is that his xp counter still reset, I'm playing with Infinite Promotions on. Is this a bug or is this intended behaviour.

Also on a slighty related note, are Musk Ox intended to be so suicidally aggressive? I am using Reckless Animals but they seem to activly hunt in packs on the polar ice and seem to attack even more often than wolves, lions or bears. Even with Reckless Animals on they are the only non predatory class animal to attack whlie neither Fox or Eagle species who are in the Predetor faction ever attack. I should also mention I'm using Size Matters and the other Size Matters option (can't quite remember what it's called). That Super Slinger has pretty much fed on a diet of Musk Ox since he was a lowly Stone Thrower at the very start of the game.

 

[tmv]

Is this a bug or is this intended behaviour.

Intended behavior. These promotions are so powerful that they are meant to reset the counter.