Single Player bugs and crashes - After the 23rd of September 2013

[demaron]

I'm still seeing the error Stormwind reported where all cities immediately reach magnate culture level. I deleted the cache in C:\Users\user\AppData\Local\My Games\Beyond the Sword\cache, the one in C2C/assets/cache and the one in C:\Users\Raeior\AppData\Roaming\My Games\Sid Meier's Civilization 4\cache but this error still remains. Do i have to delete any further files or is this bug still present? I'm using revision 6468.

 

[Thunderbrd]

The dll and the source code i posted don't have anything changed regarding the City founding issues. Because this problem needs a proper solution if you just remove the improvement you only reduce the chance of crashing.
The load issue is very ??? i changed to an older Sdk Version no i can at least sometimes start the debug dll without a crash. And one thing you can try is to change the xmlparser of the debug dll to rapidxml then it loads better.

1)Thanks for the source codes to get me caught up to you!

2)We can at least patch this discovered city founding issue then we can look into what else is taking place - I'll implement that before committing a new update. If it's only a fix for that issue then it means there are probably other issues that are related but somewhat independent. Still all probably due to some leftovers from the Super Forts merge which is to be expected from any merge from a vanilla mod into C2C. Even the most careful programmer would surely run into some unforseen issues there that wouldn't be revealed until a lot period of testing dredged them up.

3)Loading with rapidxml kinda mutes the whole point of running a debug dll to make sure we're checking the xml properly doesn't it? Then again, the xml checker does work pretty well... So for now I figure I'll just have to make sure the cache has been defined with a core dll load before attempting to use the debug dll to process through any bug reports.

A couple of things that i have noticed since Koshling has not been doing the SDK/dll stuff recently.

When playing it is very buggy, meaning it hesitates ALOT. The TURN TIMES have almost doubled. I just reached Classical Era, and the turn times are around 2 minutes each turn, i have a lot of cities on the map, but its only a Large map, and well thats not good for game play or keeping a persons interest what so ever.

What are the 2 most contributing factors for LOOOOONG turn times?? Do we need to get rid of stuff??

EDIT As i mentioned above, i am in Classical now and man, its so darn boring now, waiting is just too much to take anymore, sorry, everyone!!!
Now even the saves are taking 10 times longer

I'm not sure what would be causing such longer processing. He'd have to take a look into it. I haven't actually done much code tweaking since he began his hiatus.

However, I suppose my attempt to get the ai to plant its cities a little further apart from each other may have caused some of that. If you clock a test run on one of your turns on one dll, I can reverse those changes and provide you with a dll that has only that modification then you can clock a test run through the same turn with the new dll and we can see what kind of effect it's actually having. I can see a lot of potential delay and new processing stemming from trying to get them to plan for 3 plot width cities rather than 2. And I don't think it REALLY fixed the problem anyhow as the problem is rooted in the way the forts screw up the ai's city planning. Sadness.

I'll provide you with a dll without those changes later today after first updating us all with Albert's brilliant debugs.

I don't get to much feedback at home regarding turn times since my clever wife has discovered that if you play a multiplayer game with simultaneous turns on as a single player you can pretty much eliminate most of the wait between turns.

I'm still seeing the error Stormwind reported where all cities immediately reach magnate culture level. I deleted the cache in C:\Users\user\AppData\Local\My Games\Beyond the Sword\cache, the one in C2C/assets/cache and the one in C:\Users\Raeior\AppData\Roaming\My Games\Sid Meier's Civilization 4\cache but this error still remains. Do i have to delete any further files or is this bug still present? I'm using revision 6468.

This may be rooted in the INT limitation again. How much culture is required to reach Magnate level?

@MrAzure: You may need to recognize that there is a limit to the volume of a stored number in c++. I might have to convert all culture tags to utilize a greater volume than the standard integer allows to account for those changes (it's probably the issue we're having with some property values going out of whack too.)

 

[Toffer90]

@SO:
Denmark has unreadable city population number, red on red.
rev. 6493.

 

[strategyonly]

@SO:
Denmark has unreadable city population number, red on red.
rev. 6493.

Yeah i just saw that in the game i was playing and forgot, my bad.

Heck no wonder:

			<ColorTypePrimary>COLOR_PLAYER_[COLOR="Red"]RED[/COLOR]</ColorTypePrimary>
			<ColorTypeSecondary>COLOR_PLAYER_[COLOR="Red"]RED[/COLOR]</ColorTypeSecondary>
			<TextColorType>COLOR_PLAYER_[COLOR="Red"]RED[/COLOR]</TextColorType>

I'm not sure what would be causing such longer processing. He'd have to take a look into it. I haven't actually done much code tweaking since he began his hiatus.

However, I suppose my attempt to get the ai to plant its cities a little further apart from each other may have caused some of that. If you clock a test run on one of your turns on one dll, I can reverse those changes and provide you with a dll that has only that modification then you can clock a test run through the same turn with the new dll and we can see what kind of effect it's actually having. I can see a lot of potential delay and new processing stemming from trying to get them to plan for 3 plot width cities rather than 2. And I don't think it REALLY fixed the problem anyhow as the problem is rooted in the way the forts screw up the ai's city planning. Sadness.

I'll provide you with a dll without those changes later today after first updating us all with Albert's brilliant debugs.

I don't get to much feedback at home regarding turn times since my clever wife has discovered that if you play a multiplayer game with simultaneous turns on as a single player you can pretty much eliminate most of the wait between turns.

I believe it was bad before he left also, i just worded it that way. If it takes over 1 minute its just too long sorry. It should be that way at a minimum up to the TransHuman era at LEAST.

So what ever IS taking the MOST turn time limits, C2C just needs to sh_ t can some of it, weather it be units or something??

 

[Azurian]

Thunderbrd I think because of Subcombat classes, the info box on the very bottom left needs to be widened or Subcombat classes need to be as a list (one per line).

 

[Tzeentch]

You guys are doing a great job on C2C. C2C is the best mod for bts that I have ever played.

I have a bug that I thought was solve after reading thru the svn discussion but still exist for me. When I start a new game with svn 6493. I get super culture covering most of the continent I was on. I see messages that all the other civs are getting the same. Culture is at TXT_KEY_CULTURELEVEL_CULTUREMAGNATE. I did not change anything. Just started the game using custom game.

Not sure if anyone else is getting this bug. I have been playing with SVN 6272 without any problem.

Once again, thank you for creating this wonderful mod for us.

 

[Azurian]

You must have had save game that was at least a month long, I deleted the new culture ranks and restored the old ones like 5 or 6 weeks ago. Is the issue you Re having that the cultural level is showing up as TXT_Key?

 

[Stormwind]

Civ4CultureLevelInfo still has the new culturelevels referenced, maybe thats a problem?

 

[strategyonly]

Civ4CultureLevelInfo still has the new culturelevels referenced, maybe thats a problem?

Yeah i just looked and some are still listed there???

 

[alberts8]

1)Thanks for the source codes to get me caught up to you!

2)We can at least patch this discovered city founding issue then we can look into what else is taking place - I'll implement that before committing a new update. If it's only a fix for that issue then it means there are probably other issues that are related but somewhat independent. Still all probably due to some leftovers from the Super Forts merge which is to be expected from any merge from a vanilla mod into C2C. Even the most careful programmer would surely run into some unforseen issues there that wouldn't be revealed until a lot period of testing dredged them up.

3)Loading with rapidxml kinda mutes the whole point of running a debug dll to make sure we're checking the xml properly doesn't it? Then again, the xml checker does work pretty well... So for now I figure I'll just have to make sure the cache has been defined with a core dll load before attempting to use the debug dll to process through any bug reports.


I'm not sure what would be causing such longer processing. He'd have to take a look into it. I haven't actually done much code tweaking since he began his hiatus.

However, I suppose my attempt to get the ai to plant its cities a little further apart from each other may have caused some of that. If you clock a test run on one of your turns on one dll, I can reverse those changes and provide you with a dll that has only that modification then you can clock a test run through the same turn with the new dll and we can see what kind of effect it's actually having. I can see a lot of potential delay and new processing stemming from trying to get them to plan for 3 plot width cities rather than 2. And I don't think it REALLY fixed the problem anyhow as the problem is rooted in the way the forts screw up the ai's city planning. Sadness.

I'll provide you with a dll without those changes later today after first updating us all with Albert's brilliant debugs.

I don't get to much feedback at home regarding turn times since my clever wife has discovered that if you play a multiplayer game with simultaneous turns on as a single player you can pretty much eliminate most of the wait between turns.



This may be rooted in the INT limitation again. How much culture is required to reach Magnate level?

@MrAzure: You may need to recognize that there is a limit to the volume of a stored number in c++. I might have to convert all culture tags to utilize a greater volume than the standard integer allows to account for those changes (it's probably the issue we're having with some property values going out of whack too.)

3) I tried it because it is a differece to the release dll witch works fine and i see no other difference in the code where it crashes. I think we trigger something in the exe or the msxml3.dll but i might be wrong. The Schema validation using my tool or the debug dll somehow misses some errors we had that happening in the last days so it is not perfect. The same goes for the xml loading code sometimes it just reads the next value from the xml without checking the name of the element so if the schema validation misses an error and the xml loading does we have to go bug hunting again.

The next thing i do is going to work on the xml loading so that it detects more errors by itself. Then it might be possible that we can skip the validation when loading the game and have the xml modders check it before commiting to the svn using my tool or something else. Or we can switch to another xml parser...

A INT limitation with the culture levels is very possible Koshling fixed just one of them i saw i identical line somewhere that he did not change.

 

[Thunderbrd]

I believe it was bad before he left also, i just worded it that way. If it takes over 1 minute its just too long sorry. It should be that way at a minimum up to the TransHuman era at LEAST.

So what ever IS taking the MOST turn time limits, C2C just needs to sh_ t can some of it, weather it be units or something??

I'd have to really suspect that city planting evaluation coding change - yes it's improved ai but it's also going to be slower... was hoping it wouldn't have THIS much of an impact. Superforts might be adding significant processing time as well but that'd be Koshling's realm to sort out.

The # of Assets CAN cause some in-game delay but it's usually something that can be identified and improved in the code if it can be determined where we're generating loops that are too long. We can narrow the field of the loops with predefined vectors that would assist in the looping process so a lot of things can be done to reduce the impact of added assets. Those of us in the dll can work on those things as we see them or rather as we realize a given section could really be taking too long to work through.

Still, the addition of 100 new assets (like units) shouldn't greatly enhance turn delays unless the coding is hitting some really stupid looping spots. Koshling's usually really good at seeing where those 'bad processing' spots are. He's pointed out a number of them and shown me how to fix them in my coding to the point where I'm beginning to see it myself. Not that I'm a master of that task or anything...

Anyhow, point being it's better to recode to reduce processing than it is to remove elements to reduce processing imo.

Thunderbrd I think because of Subcombat classes, the info box on the very bottom left needs to be widened or Subcombat classes need to be as a list (one per line).

Agreed... I've been wanting to widen the help hover for the longest time and I saw where Platyping had stated how to do it but unless he was guiding us to the solution that exists in python somewhere I had not been able to find where it was stated I'd need to tweak in the dll. That would be a HUGE benefit for us to widen that hover!

Outside of that I do also agree that CCs will probably be needing to be removed there so that it can be purely expressed in the hotkey hover help page. However, that help page is going to need to be setup for in-game unit displays as opposed to base unit build helps.

So yeah, it's something I know needs to be addressed.

You must have had save game that was at least a month long, I deleted the new culture ranks and restored the old ones like 5 or 6 weeks ago. Is the issue you Re having that the cultural level is showing up as TXT_Key?

For reference, the web is telling me that Int limits us between the values of 2,147,483,647 (roughly 2 billion) and -2147483647. If you establish any numbers beyond that point you will find the game reads it to mean whatever number it randomly pulls out of a hat that stays within that range. C2C is having more and more problems with this and it's going to end up being a big issue for us to resolve over and over for differing values in the code... there are some solutions possible but it's a pain and usually will force the mod memory requirements through the roof.

3) I tried it because it is a differece to the release dll witch works fine and i see no other difference in the code where it crashes. I think we trigger something in the exe or the msxml3.dll but i might be wrong. The Schema validation using my tool or the debug dll somehow misses some errors we had that happening in the last days so it is not perfect. The same goes for the xml loading code sometimes it just reads the next value from the xml without checking the name of the element so if the schema validation misses an error and the xml loading does we have to go bug hunting again.

The next thing i do is going to work on the xml loading so that it detects more errors by itself. Then it might be possible that we can skip the validation when loading the game and have the xml modders check it before commiting to the svn using my tool or something else. Or we can switch to another xml parser...

A INT limitation with the culture levels is very possible Koshling fixed just one of them i saw i identical line somewhere that he did not change.

If you're as knowledgeable as you sound, perhaps you could take a look at the zipped files in the source codes (you have uploaded the modder's svn version right?) n47 did a lot of work to get us a highly improved loading mechanism BUT had to leave a day or two before she could get it fully debugged. She said she caught a LOT of xml errors that I don't think we've discovered yet since then either but she didn't have time to fix them all and commit. Perhaps you might have enough insight into what she was doing to fix whatever bugs she had left in those adjustments and we'd be 10 steps ahead of trying to start that project all over again no?

 

[strategyonly]

Got this python error when i clicked on Setting??

Traceback (most recent call last):

File "CvScreensInterface", line 1310, in handleInput

File "CvVictoryScreen", line 2056, in handleInput

File "CvVictoryScreen", line 849, in showGameSettingsScreen

AttributeError: type object 'CvPythonExtensions.GameOptionTypes' has no attribute 'NUM_GAMEOPTION_TYPES'
ERR: Python function handleInput failed, module CvScreensInterface

 

[alberts8]

If you're as knowledgeable as you sound, perhaps you could take a look at the zipped files in the source codes (you have uploaded the modder's svn version right?) n47 did a lot of work to get us a highly improved loading mechanism BUT had to leave a day or two before she could get it fully debugged. She said she caught a LOT of xml errors that I don't think we've discovered yet since then either but she didn't have time to fix them all and commit. Perhaps you might have enough insight into what she was doing to fix whatever bugs she had left in those adjustments and we'd be 10 steps ahead of trying to start that project all over again no?

Time is the issue but i see what i can do. First i work on the loading code in the info classes because where are things that could be done to prevent errors and to reduce the size of the code. After this i take a look at the xml parser because with the code reduction i'am planning to do it is easier to put another parser in.

With error prevention i mean for example read only Flavor items in the Flavors Tag because now it ignores the name of the items and if there is something wrong there you see that sometime later in the game.

 

[Tzeentch]

You must have had save game that was at least a month long, I deleted the new culture ranks and restored the old ones like 5 or 6 weeks ago. Is the issue you Re having that the cultural level is showing up as TXT_Key?

I ran a new game. Not a savegame. I have tried the savegame and same problem. It seems like its multiplying the small culture number with something.

I read that you have restored the old ones but it is still doing it. Not sure why though.

And yes,, the cultural level is showing up as TXT_Key.....

Thanks

 

[Azurian]

I ran a new game. Not a savegame. I have tried the savegame and same problem. It seems like its multiplying the small culture number with something.

I read that you have restored the old ones but it is still doing it. Not sure why though.

And yes,, the cultural level is showing up as TXT_Key.....

Thanks

My nightmare just appeared, I had no idea it didn't get restored property. That file is version2, there is a newer file where the last culture is Legendary (rank 16).

 

[Tzeentch]

lol no worries. At least you have an idea where the problem is. Great work, MrAzure.

To all modders of C2C, really appreciate the work you guys have put in. Salute.

 

[Snofru1]

With the newest SVN the game runs quick and smooth again. Thanks for the good work, Thunderbrd (I at least am quite sure that it were your changes) !

 

[Thunderbrd]

Interesting... I didn't revert the city placement AI yet so the speedup COULD be caused by some other contributing factor. hmm...

 

[Koshling]

I have his upload yes but my file sets are stuck in untested tweaks and evaluation runs - you wouldn't want me to merge during this process.

I've narrowed down the problem considerably and it runs right into the heart of complex Super Forts territory so it's taking me a lot of time to sort out the purpose of numerous data threads interacting with the path through the city founding process.

What's happening is in the initialization of a city, just before the city itself is defined, the plot the founding is taking place on is set to be owned by the player founding the city. Further on from there assumptions are made that the plot is owned by that player. This is where it's binding up.

It's binding up there because in the function that assigns the plot to the player, it's finding an improvement on the plot that gives culture and is assuming that improvement to be a fort (even though the plot is not owned until the city is being founded!) Upon THIS assumption, an alternative use of the plot claiming function is running that then loops BACK around through the whole plot claiming function itself, rewriting the plot ownership back to Null somehow. I haven't quite managed to evaluate that nested loop yet but apparently, it leaves the plot unowned or owned by NO_PLAYER. So when the city founding proceeds further it quickly binds up on the assumption that we're talking about the owner of the plot that's founding the city.

Now I'm finding I can't load the same dll since I deleted the cache and I'm trying to figure out how to get the initial load sequence to help me identify what improvement must exist at that site (improvement id 105 I think.) This COULD be simply an error in the way the xml was done for an improvement but the code shouldn't allow such an error to crash and should then be instead hedged to redefine the improvement if it's written in a way that's incompatible with the code or simply reprogrammed.

And yes, by all means Alberts should receive the team invite! Welcome aboard bud!

Finding a culture-giving improvement on an unowned plot sounds like an a bad state (that the code doesn't cope with). You will need to pin down what improvement (as you are doing) and then figure out how such an improvement can ever be on an unowned plot! Correcting that is likely to be a good idea even if you also correct the handling of that condition as well. I assume this wasn't an improvement that was placed using WB? That might cause issues for culture-giving improvements maybe? (placing them on unowned territory that is)

 

[Koshling]

@Thunderbrd

I did some additional changes to CvGameTextMgr::setBuildingHelpActual the changed file is attached.

@strategyonly

I uploaded the changed dll so you can try it.
http://sdrv.ms/1aHULq9

Svn write rights would be nice so i could commit my changes myself the sourceforge username is alberts2.

Added Alberts2 as a contributor...