V36

[Thunderbrd]

Turn times are bad since the changes to the unit movement rules but they are even longer now.

The increased usage of the unit contracting and the healer production evaluation are slowing things down.

That would've had an effect earlier than a week or two ago when they are saying things really started slowing down.

I know that we need to figure out a better way to address the visibility aspect of the movement rules though.

So unit contracting is taking a lot of time to process is it? I didn't suspect it would have that much impact. Hmm... May have to figure out how to get the AI to evaluate once in a round and then never run the evaluation again that round. Somehow.

 

[Thunderbrd]

Cant WE just get rid of the "split" stuff and added promotions, i really dont like them anyways.

What's your problem with it when it's an option? Has no effect on the main game play. And if there are promos that you'd prefer to option off we can do that as well. Not sure which promos you have an issue with.

 

[Thunderbrd]

Need some help.

After SVN 8616 I updated to 8633, then 8645, 8650, and now 8657, somewhere between 8633 and current 8657 my turn times have increased tremendously. Had a turn to day where I just completed research for Matchlock and it took over 7 minutes for the EoT to process. Something changed in the processing of the turn data that has increased turn time nearly 10 fold from what I was experiencing at SVN 8616. I can say that I've never experienced this much lag in EoT processing for a long long time. (This is also the 1st game in 3 years that I've made it into the Ren Era.) It actually takes almost 2 minutes for the "running man" to appear from the normal game spearhead cursor too. Has some of the recent coding changes caused more looping processes for units? I have been involved in 2 heavy wars that saw my and several other AI build up Armies significantly. I have stack limit set at 50.

i7 2600k @3.4Ghz Cpu, 8GB DDR3 1233Mhz, Nvidia GTx 550 Ti vid card, recent Win 10 Pro x64 upgrade from Win 7 Pro x64.

9 player game on Normal, Emperor, Large (iirc) PM map and No Options that generate extra barbs or Civs. Only thing extra is my Crime modmod.

Is anyone else experiencing a EoT slowdown too? Can post a save game in Bug thread if Alberts2 or T-brd want to take a look see.

Any help would be greatly appreciated, Thanks.

JosEPh

Rereading this for hints as to what may be some things to take a deeper look at I noticed you indicated that you were playing with a stack limit of 50 units. This late in the game that MAY mean that you're getting some AIs trying to stack more units than 50 onto a given tile and when the unit that wants to do so is finding it can't for some unknown reason, it may be hitting what would've once been an infinite loop but is now instead cutoff after allowing some revolutions to retry. When this starts happening it can create a noticed slowdown and can mess with the AI intentions.

This is just one possible reason.

I realize that getting into this stage of the game we really start to suffer from the adjustments to the visibility/movement rules and I was hoping it wouldn't be THIS intense this early. Doesn't hurt the beginning of the game much at all so far as I can tell. But when you have THIS many units on the board it can really cause issues so it may need an extensive rework with some of the proposed solutions before we move forward here.

But if the stack limit is the issue then that's easily changed in the BUG options. See if it helps any to remove the limits. (And you said somewhere you thought Koshling had said that the limit should be around 50 but IIRC he said there should never BE a stack limit set or you could have some serious problems.)

 

[strategyonly]

What's your problem with it when it's an option? Has no effect on the main game play. And if there are promos that you'd prefer to option off we can do that as well. Not sure which promos you have an issue with.

Just seems like all the CTD's and bug error etc are from that area. and masybe causing the AI to slow up also besides turn times, i am just guesssing of course.

 

[Thunderbrd]

Just seems like all the CTD's and bug error etc are from that area. and masybe causing the AI to slow up also besides turn times, i am just guesssing of course.

It's a fair guess considering how long its taken to get things fully stabilized there. There was a CTD regarding the workers but that's been easily rectified.

It COULD still be an issue for memory IF (and only if) you play with the option. But I have a couple items on the task list to do that should make it much less taxing there. Additionally I've taken some very big strides forward in improving the AI usage of the merging and splitting functions so that not only will they not tax the memory so much with it, their usage of it makes the game MUCH harder because it's a lot smarter about it. (Been downright fun to play against since then actually.)

What's currently causing issues with the turn times is this infernal 'tunnel fix' I did a while back. To get the game to use movement rules properly for tunnels and other similar situations has apparently caused a lot of extra processing that the original design shortcut past with an EXTREMELY simplified movement/unit interaction system that I abhor due to the way it tied our hands in so many ways. But it did do well to keep processing to a minimum. What I'm going to need to do to really fix that situation is extremely in-depth and could take a month or more to solve. There could be some crashes taking place there in rare cases as well.

The other matter Alberts2 pointed to was that the healer AI is eating up a lot of processing as well. I can see how that would be quite possible. Again there, that's another super deep thing to fix and it sucks to even touch it now that its working so well. I've seen extremely appropriate behavior regarding healing and property control units.

But the main thing is to try to reduce how often the AI spends time evaluating. At the moment it evaluates if a city or growing stack NEEDS a healer or property control unit with each city and then evaluates where a healer or property control unit really needs to be each time that unit's turn comes up for the AI.

The evaluation is a little heavy itself so what I think needs to happen is a one-time evaluation on a player level that determines where it needs more of these units and where it doesn't need as many and sets up a single 'demand' count by plot and/or by 'stack' and then when the cities go to evaluate they just check that one reference and modify that number if they set themselves to do something about it.

There's truly a world of considerations that need to be made to start designing such a streamlined system and it would probably be best if such a system were setup to apply to many other unit needs as well so it would need to be pretty generically applicable. So again, months to design. But it would be highly valuable to opening up an ability to get the AI to be much smarter on many levels while at the same time being faster.

Furthermore, to do this would mean I would really need to start considering using some data structures I'm not very comfortable with. I still have a little trouble following Koshling's data structures as it is but I understand they have a lot of memory and speed optimization that caused him to opt to use them to begin with. And I suspect if I were to setup all I just mentioned that they would be the best way to apply a lot of that.

There's more to this that deals with how the AI considers its stacks as well. And again that's a lot to discuss with Alberts2 there. Some initial considerations have been put forth as it is.


BuildUps MAY still have some slowdown issues that have yet to be fully addressed but I really would've thought I'd streamlined it quite a bit after the recent rule fixes. I may need to do some profiling to really test that. One thing that really may need to be done is to again reduce the frequency that a unit evaluates it's 'best promotion'. I may be asking units to run this evaluation too often so as to always make sure they are using the best possible selections for them and that could be what's really detrimental to the turn times. I cut it down so that it's only checking each stationary unit 1 out of 5 turns rather than every turn. Could still be too much and I might have to just get them to figure out what the 'best' is for them and forget trying to stay updated to the greatest current needs.

 

[JosEPh_II]

@T-brd,
I'll reset the Stack limit to unlimited and see what happens.

I had to reduce my graphics options to Medium across the board And reduce my resolution down to 1600 x 1050(?), tried 1600x900 but 23" monitor it caused the build list to have bottom and side sliders and would only display 1 and 1/2 rows of the build list. These reductions took the turn time down from 12+ minutes to 7+ minutes. And at this point in time I can't afford to upgrade my Nvidia GTx 550 Ti vid card.

I hate to sound all harpy but I do believe the game is maxed out on promotion usage. I can't say any thing about your FoF or SM Options as I don't normally use them. So I've ruled them out as possible contributors to this slowdown. But the unit promo evaluations and withdrawal procedures and other base coding changes do seem to be involved. I'm unsure if we also have another Graphics overload problem. Plus as I posted in the Bug thread if you reduce a unit with withdrawal capabilities to less than str 1 but do not get a complete kill you can Not touch them by any means until next turn. Even a new attacker is Not allowed to attack them to finish them off. This Is a Problem.

Reverting some of the more recent changes may be necessary to get v36 out. Long term plans that leave these types of CTD/MAFs and/or hangs will be unacceptable to our users. We really really need some streamlining before any newer planned options get put in. I feel this even includes the Nomad start.

Final note:
And as you posted elsewhere Crime and Disease have much that needs done to them to address what you posted about them. I've seen the same thing and some of my adjustments After v36 release will start addressing that. But even that from me will be in stages. So getting back to a stable base even if that requires removing Sea tunnels and the associated coding changes may be vital And necessary so that V 36 can be released soon rather than later. Players will squawk I know if Sea tunnels are removed, but this seems to be a Necessary pruning so the Mod can advance.

JosEPh

 

[Taxman66]

If units were more expensive (in hammers) there would be fewer of them (and/or reduced unit building bonuses). I dont see why there needs to be huge stacks ever. Why do we need attack stacks of 50+? You can get the same variety with 15-20. Related: might want to reduce uu to 10 or so.

 

[Arakhor]

Personally, I couldn't care less about sea tunnels. I've never reached the Modern Era, even when the turn lengths weren't abominable.

 

[strategyonly]

Yeah i agree, i could care less about sea tunnels, i say "Take them out and the code" for now, can always LATER put it back in. But be careful not to affect the other coding "of course".

 

[Thunderbrd]

The coding changes weren't JUST about sea tunnels and removing them definitely won't be enough of a fix now. The full system would need to be reverted very carefully and then we can look at planning another approach to the problem. Would probably be easier that way anyhow but even that is going to take some weeks from now since its a bit of surgery.

Addressing the issue with units under 1 str not being able to be attacked should be very simple ... just a bit of an easter egg hunt is all.

But again, I'll be out for the weekend starting now.

 

[JosEPh_II]

If units were more expensive (in hammers) there would be fewer of them (and/or reduced unit building bonuses). I dont see why there needs to be huge stacks ever. Why do we need attack stacks of 50+? You can get the same variety with 15-20. Related: might want to reduce uu to 10 or so.

You do realize that You set the Stack limits in the BUG Options screen Caveman2Cosmos tab. There is a default setting enabled if you've made No adjustment.

The increments are 0 to 100, with 0 being unlimited. Choices given are 0, 1 thru 10, then in 5 increment steps to 40, then 10 increment steps to 100. You chose how big you want them.

Increasing unit costs will not change these settings and is an unrelated issue.

JosEPh

 

[alberts8]

You do realize that You set the Stack limits in the BUG Options screen Caveman2Cosmos tab. There is a default setting enabled if you've made No adjustment.

The increments are 0 to 100, with 0 being unlimited. Choices given are 0, 1 thru 10, then in 5 increment steps to 40, then 10 increment steps to 100. You chose how big you want them.

Increasing unit costs will not change these settings and is an unrelated issue.

JosEPh

What do you expect from using that option?

 

[Taxman66]

Its not the stack size itself, its the total volume of units. Fewer units = less processing resources, fewer crashes and faster turn times which will lead to an easier playing of later eras as well.

I was using 'attack' size as simply an example. IMO there is no significant gain of building 50/100 unit stacks that you cant get from a 15 or 20 size stack.

It would also mean fewer city defenders just sitting in cities doing nothing but eating processing resources.

If thjs approach is taken then other factors would need to be tweeked. For example fewer uu per culture, more city defense % dmamged per siege, more effect per police and disease units/promos.

In short I'd rather see 25 units as a massive death stack than 100.

 

[Sparth]

If units were more expensive (in hammers) there would be fewer of them (and/or reduced unit building bonuses). I dont see why there needs to be huge stacks ever. Why do we need attack stacks of 50+? You can get the same variety with 15-20. Related: might want to reduce uu to 10 or so.

I have similiar observation. Less units - shorter turn time and less micromanagment.

 

[alberts8]

I have similiar observation. Less units - shorter turn time and less micromanagment.

You could say the same about everything else in the mod like less techs shorter turn time, less buildings shorter turn time, remove the property system shorter turn time and less micromanagment and so on.......................................................

 

[Sparth]

You could say the same about everything else in the mod like less techs shorter turn time, less buildings shorter turn time, remove the property system shorter turn time and less micromanagment and so on.......................................................

I dont want to reduce quantity of possible units in mod but to reduce of them in-game.
And as for buildings I also want to combine some of them to new ones. For example instead ~20 factories for each crafted good I think we should have 5-6 types of them, one for each industry branch.

 

[alberts8]

It's a fair guess considering how long its taken to get things fully stabilized there. There was a CTD regarding the workers but that's been easily rectified.

It COULD still be an issue for memory IF (and only if) you play with the option. But I have a couple items on the task list to do that should make it much less taxing there. Additionally I've taken some very big strides forward in improving the AI usage of the merging and splitting functions so that not only will they not tax the memory so much with it, their usage of it makes the game MUCH harder because it's a lot smarter about it. (Been downright fun to play against since then actually.)

What's currently causing issues with the turn times is this infernal 'tunnel fix' I did a while back. To get the game to use movement rules properly for tunnels and other similar situations has apparently caused a lot of extra processing that the original design shortcut past with an EXTREMELY simplified movement/unit interaction system that I abhor due to the way it tied our hands in so many ways. But it did do well to keep processing to a minimum. What I'm going to need to do to really fix that situation is extremely in-depth and could take a month or more to solve. There could be some crashes taking place there in rare cases as well.

The other matter Alberts2 pointed to was that the healer AI is eating up a lot of processing as well. I can see how that would be quite possible. Again there, that's another super deep thing to fix and it sucks to even touch it now that its working so well. I've seen extremely appropriate behavior regarding healing and property control units.

But the main thing is to try to reduce how often the AI spends time evaluating. At the moment it evaluates if a city or growing stack NEEDS a healer or property control unit with each city and then evaluates where a healer or property control unit really needs to be each time that unit's turn comes up for the AI.

The evaluation is a little heavy itself so what I think needs to happen is a one-time evaluation on a player level that determines where it needs more of these units and where it doesn't need as many and sets up a single 'demand' count by plot and/or by 'stack' and then when the cities go to evaluate they just check that one reference and modify that number if they set themselves to do something about it.

There's truly a world of considerations that need to be made to start designing such a streamlined system and it would probably be best if such a system were setup to apply to many other unit needs as well so it would need to be pretty generically applicable. So again, months to design. But it would be highly valuable to opening up an ability to get the AI to be much smarter on many levels while at the same time being faster.

Furthermore, to do this would mean I would really need to start considering using some data structures I'm not very comfortable with. I still have a little trouble following Koshling's data structures as it is but I understand they have a lot of memory and speed optimization that caused him to opt to use them to begin with. And I suspect if I were to setup all I just mentioned that they would be the best way to apply a lot of that.

There's more to this that deals with how the AI considers its stacks as well. And again that's a lot to discuss with Alberts2 there. Some initial considerations have been put forth as it is.


BuildUps MAY still have some slowdown issues that have yet to be fully addressed but I really would've thought I'd streamlined it quite a bit after the recent rule fixes. I may need to do some profiling to really test that. One thing that really may need to be done is to again reduce the frequency that a unit evaluates it's 'best promotion'. I may be asking units to run this evaluation too often so as to always make sure they are using the best possible selections for them and that could be what's really detrimental to the turn times. I cut it down so that it's only checking each stationary unit 1 out of 5 turns rather than every turn. Could still be too much and I might have to just get them to figure out what the 'best' is for them and forget trying to stay updated to the greatest current needs.

The unit movement is still too slow even with the changes i made to speed it up, there must be a simpler faster solution.
The healer production code in CvCityAI::AI_chooseHealerUnit is very slow how could that be simplyfied?

I can't see why BuildUps should slow things down at he moment.

 

[Reisk@]

Personally, I couldn't care less about sea tunnels. I've never reached the Modern Era, even when the turn lengths weren't abominable.

Well then "maybe" you shouldn't judge them. i, personally, having played this, many times, unplayably broken, although a hugest promise of a moddy-thingy, i can and will verify that although broken in many other ways in previous versions counting back from preV36 to V33, the tunnels are great way to connect lands with islands that are one or two moves away from each other. Plus, they evidently are realistic and relevant way to connect close land masses in modern+more 'advanced' times and i definitely see no reason whatsoever to have them removed.

Please concentrate on fixing bugs rather than removing working and beneficial properties. Something like AI which is getting worse and worse by the version.

 

[Arakhor]

I'm sure that if the mod was realistically playable (i.e. without consuming the entire works of Tolstoy during turn waiting) up to the Modern Era and beyond, then, yes, the sea tunnels could be very useful. I dare say that there are years of work to go before reaching that point though.

 

[JosEPh_II]

What do you expect from using that option?

Sorry but I don't understand your question? The Option in BUG is always On and I'd have to start a new game to see what the default setting is. All I know is that setting it to 0 (unlimited) has a Hover over say it's not a recommend setting for "normal" game play. Now you tell me what is "normal" game play?

@Riesk,
There is code associated with Sea Tunnels that Is a problem (has bugs) whether you Love them or not. Until that is fixed then Sea Tunnels should be turned Off.

Please concentrate on fixing bugs rather than removing working and beneficial properties. Something like AI which is getting worse and worse by the version.

And as for the healer problem I'm hoping that the changes I want to make to Diseases after v36 release will abate the need for massive amounts of healers early game were that problem starts. Same for TW line and Crime and no I'm not looking for a bunch of new crime fighting units either. But rather a simple reduction of early Disease and crime and moving to later Eras the brunt of both. Right now it's mostly front loaded.

Also unit upgrade costs are all over the board. But in the game play itself it becomes easier to delete older units and just build new ones. So upgrades become unnecessarily burdensome.

Example: Do you think the AI as well as the player wants to spend 500+ to upgrade from Healer to Apothecary? Especially when the game makes you and the AI to build dozens of healers per city to combat disease by the time you can build Apothecaries. When it's just easier production wise to build several new Apothecaries and delete the old healers. Or keep all the promoted up healers even if it means by late med Era you have 30+ in your cities. And by Ren Era I've seen AI cities with 50+. And the same happens for the TW line too.

So does the AI need to be taught that it needs to delete old units to build new? Right now it's taught to keep the old because it has more promotions, that is were the current weight seems to be.

JosEPh