Single Player bugs and crashes v37 plus (SVN) - After the 24th of December 2016

[Toffer90]

I think we need to determine if we want it to destroy the feature.

The process, if it's possible to map out all steps easily here, should be:
1)The old feature is recorded on the plot in a manner that would save if needbe - not sure if python can do that
2)The plot picks up the tornado as a feature and thus the existing feature is removed by default
3)The plot loses any improvements on the plot (but not necessarily routes)
4)The tornado lasts there for 1 round
5)Next round the tornado is removed and the old feature is re-assigned to the plot.

Tell me if this can all be done in the python.

Ok, so I got it to work nicely, but I just realized that if one quit the game during a turn where a tornado have removed a feature, the python won't remember what feature was there when you load the game up again... It wouldn't even be aware of any tornadoes that may still be on the map.
I guess I could store the list: [ ( CyPlot, feuturetypeBeforeTornado), ( CyPlot, feuturetypeBeforeTornado), etc. ]

to a text file in the userSettings folder whenever a tornado event makes a new tornado and thus make python read in the list from file whenever you load a save.
Max number of entries in the list equals the amount of civs present in the game. Each civ has a chance to get the tornado event on their turn.​

I really don't like this solution as it would have to store one list per turn in the same file in case you load an earlier save, it would also have to be one unique text file per unique game you have.
If you were to delete the user setting folder any save you have that had a tornado will be a bit broken as those tornadoes would never get removed.

Would it be easier for the dll to store this list in a game save file perhaps??
We could make it so that the event python gives the dll the plot which the event triggered on through a new dll function that is exposed to python. The dll could then keep track of which plots have a tornado and what feature that got removed and save that list variable in the save files. There can be as many tornadoes in the same turn as there are civs in the game.
At the beginning of player 0's turn the dll could call a python function that cleans up all the tornados that were made during the last turn.

Alternatively we could hardcode the event completely in the dll...

What are your thoughts?


Here's a baseline you could use in case you decide that it should be hardcoded.

Spoiler Relevant python code in CvRandomEventInterface.py :

######## TORNADO ###########
tornadoMap = {}
def canDoTornado(argsList):
    EventTriggeredData = argsList[0]
    x, y = EventTriggeredData.iPlotX, EventTriggeredData.iPlotY
    CyPlot = gc.getMap().plot(x, y)
    if tornadoMap.has_key(CyPlot):
        # There is already a tornado here.
        print "CvRandomEventInterface: Unexpected scenario in canDoTornado function"
        return 0
    iLatitude = CyPlot.getLatitude()
    if iLatitude < 60 and 20 < iLatitude:
        print "True"
        return 1
    print "False"
    return 0

def doTornado(argsList):
    EventTriggeredData = argsList[1]
    x, y = EventTriggeredData.iPlotX, EventTriggeredData.iPlotY
    CyPlot = gc.getMap().plot(x, y)
    CyPlot.setImprovementType(-1)
    iFeature = CyPlot.getFeatureType()
    tornadoMap[CyPlot] = iFeature
    CyPlot.setFeatureType(gc.getInfoTypeForString('FEATURE_TORNADO'), 0)

Spoiler Relevant python code in CvEventManager.py --- Removes tornado :

## Likely to cause OOS errors as it runs on the beginning of human players turn.
## Can probably be fixed by checking that it is player 0's turn.
    def onBeginPlayerTurn(self, argsList):
        'Called at the beginning of a players turn'
        iGameTurn, iPlayer = argsList
        aBox = CvRandomEventInterface.tornadoMap
        for key in aBox:
            entry = aBox.get(key)
            key.setFeatureType(entry, 0)
        CvRandomEventInterface.tornadoMap = {}

 

[Thunderbrd]

Ok, so I got it to work nicely, but I just realized that if one quit the game during a turn where a tornado have removed a feature, the python won't remember what feature was there when you load the game up again... It wouldn't even be aware of any tornadoes that may still be on the map.
I guess I could store the list: [ ( CyPlot, feuturetypeBeforeTornado), ( CyPlot, feuturetypeBeforeTornado), etc. ]

to a text file in the userSettings folder whenever a tornado event makes a new tornado and thus make python read in the list from file whenever you load a save.
Max number of entries in the list equals the amount of civs present in the game. Each civ has a chance to get the tornado event on their turn.​

I really don't like this solution as it would have to store one list per turn in the same file in case you load an earlier save, it would also have to be one unique text file per unique game you have.
If you were to delete the user setting folder any save you have that had a tornado will be a bit broken as those tornadoes would never get removed.

Would it be easier for the dll to store this list in a game save file perhaps??
We could make it so that the event python gives the dll the plot which the event triggered on through a new dll function that is exposed to python. The dll could then keep track of which plots have a tornado and what feature that got removed and save that list variable in the save files. There can be as many tornadoes in the same turn as there are civs in the game.
At the beginning of player 0's turn the dll could call a python function that cleans up all the tornados that were made during the last turn.

Alternatively we could hardcode the event completely in the dll...

What are your thoughts?


Here's a baseline you could use in case you decide that it should be hardcoded.

Spoiler Relevant python code in CvRandomEventInterface.py :

######## TORNADO ###########
tornadoMap = {}
def canDoTornado(argsList):
    EventTriggeredData = argsList[0]
    x, y = EventTriggeredData.iPlotX, EventTriggeredData.iPlotY
    CyPlot = gc.getMap().plot(x, y)
    if tornadoMap.has_key(CyPlot):
        # There is already a tornado here.
        print "CvRandomEventInterface: Unexpected scenario in canDoTornado function"
        return 0
    iLatitude = CyPlot.getLatitude()
    if iLatitude < 60 and 20 < iLatitude:
        print "True"
        return 1
    print "False"
    return 0

def doTornado(argsList):
    EventTriggeredData = argsList[1]
    x, y = EventTriggeredData.iPlotX, EventTriggeredData.iPlotY
    CyPlot = gc.getMap().plot(x, y)
    CyPlot.setImprovementType(-1)
    iFeature = CyPlot.getFeatureType()
    tornadoMap[CyPlot] = iFeature
    CyPlot.setFeatureType(gc.getInfoTypeForString('FEATURE_TORNADO'), 0)

Spoiler Relevant python code in CvEventManager.py --- Removes tornado :

## Likely to cause OOS errors as it runs on the beginning of human players turn.
## Can probably be fixed by checking that it is player 0's turn.
    def onBeginPlayerTurn(self, argsList):
        'Called at the beginning of a players turn'
        iGameTurn, iPlayer = argsList
        aBox = CvRandomEventInterface.tornadoMap
        for key in aBox:
            entry = aBox.get(key)
            key.setFeatureType(entry, 0)
        CvRandomEventInterface.tornadoMap = {}

Gotta think on this. A check of every plot for a tornado every round would be a big slowdown.

Ok... thinking. Lemme get back to you on this.

 

[Toffer90]

Hold on a minute, this just might work....
There exist two CyPlot methods that supposedly can store custom script data as strings directly on the plot.

VOID setScriptData (STRING szNewValue)
void (str) - Set stored custom data

STRING getScriptData ()
str () - Get stored custom data

Perhaps that data gets stored in save files?
If they do I will only require one new boolean tag for events to get this to work.

I'm gonna play around a bit more before giving up on this.

 

[Thunderbrd]

I'm thinking that it isn't worth new tags or excessive memory costing measures to fix one uncommon event. If it becomes a bigger issue with wider application then fine we move to another level to address this.

So if you simply run a check on all plots to see if there is a tornado there on game load and game load only, with the reaction to eliminate that tornado and let whatever feature was there simply be removed by this unusually strong storm result that happened because you saved and reloaded during that quirk of a turn, then we'd be fine. Maybe just make sure that the event can't happen on plots with natural wonders and we'd be good with that much resolution.

What do you think?

 

[Septimius]

The recosting hasn't been in effect long. I expect bad recosting spot reports to trickle in for a while. The human mind cannot do that much repetitive activity and be reliably accurate all the time. The slightest distraction can lead to an accidental visual shortcut or skip in the process and blam, you have an error. That the entire 2500+ list of buildings just got hand recalibrated in costs is... let us say it was the most unimaginably mind numbing effort I've ever undertaken. Thanks Mom for all those times I had to write 500 sentences as a punishment because they made it possible to tolerate this sort of thing enough to get it done!

And now I'm crazy enough to invite similar projects under discussion.

Well let me know if you need any help, I know nothing of code, but I can do repetitious digging through .ini files with Notepad +

Also, another minor bug - just hit the ancient tech line and got my first great prophet (!) - he can construct the CaoDai Holy See, which has to be a bug since it's a modern age religion. Obviously it goes without saying I don't have CaoDai as my religion... In fact I don't have any religion (what with no neighbours).

 

[Thunderbrd]

Also, another minor bug - just hit the ancient tech line and got my first great prophet (!) - he can construct the CaoDai Holy See, which has to be a bug since it's a modern age religion. Obviously it goes without saying I don't have CaoDai as my religion... In fact I don't have any religion (what with no neighbours).

Not a bug provided that you also have the game option Choose Religions on. But assuming it stands out as unusual then it probably is a bug and obviously that would mean there's a prereq needed to be defined somewhere still.

We may well be able to use some XML notepad+ labor here soon. Help would be desired but the core team is discussing the project plan first and it would take a bit of effort on the programming side before we're ready for help.

 

[Septimius]

Not a bug provided that you also have the game option Choose Religions on. But assuming it stands out as unusual then it probably is a bug and obviously that would mean there's a prereq needed to be defined somewhere still.

We may well be able to use some XML notepad+ labor here soon. Help would be desired but the core team is discussing the project plan first and it would take a bit of effort on the programming side before we're ready for help.

I don't have choose religions on. It's definitely a bug.

 

[Dancing Hoskuld]

Hold on a minute, this just might work....
There exist two CyPlot methods that supposedly can store custom script data as strings directly on the plot.

VOID setScriptData (STRING szNewValue)
void (str) - Set stored custom data

STRING getScriptData ()
str () - Get stored custom data

Perhaps that data gets stored in save files?
If they do I will only require one new boolean tag for events to get this to work.

I'm gonna play around a bit more before giving up on this.

Be careful when using Script Data. Don't forget we use BUG and it has the "new improved" methods for that in it. You need to use the same methods as are used elsewhere in the code or there can be problems.

Also it may just be simpler to have the Python loop through all the plots looking for a tornado at start up and if it finds one then get the script data rather than doing it the other way around.

You need to be careful using stuff on the dll-python boundary especially if it is rarely used. In many cases it was not checked for efficiency and the coders were paid by the line of code.

 

[Toffer90]

Be careful when using Script Data. Don't forget we use BUG and it has the "new improved" methods for that in it. You need to use the same methods as are used elsewhere in the code or there can be problems.

I'll have to investigate how BUG have interacted with plot script data, I'll also have to find out if any other python scripts or even the exe stores script data in plots for any reasons.
If someone could post an advanced game I could run some test on it with a small modification to print the script data for all plots each turn, that way I can find out if the data storage unit is in use already.

Also it may just be simpler to have the Python loop through all the plots looking for a tornado at start up and if it finds one then get the script data rather than doing it the other way around.

I don't know what you mean by "the other way around" in this case.
The way I was thinking about, was to set up a follow-up event that always triggers the turn after the first event.
The first event adds the tornado and stores in the plot any old feature with CyPlot.setScriptData( "FEATURE_X" )
The second event removes the tornado and reassigned the old feature if it is found in the plot when using CyPlot.getScriptData()
This is impossible at the moment because the amount of turns set in iEventTimes is scaled by gamespeed. I would need a new boolean that stops that scaling: <bNoScalingByGamespeed>.

Spoiler Event XML example of a follow-up event :

        <EventInfo>
            <Type>EVENT_TORNADO_START</Type>
            <Description>TXT_KEY_EVENT_TORNADO_1</Description>
            <PythonCallback>startTornado</PythonCallback>
            <AdditionalEvents>
                <EventChance>
                    <Event>EVENT_TORNADO_STOP</Event>
                    <iEventChance>0</iEventChance>
                </EventChance>
            </AdditionalEvents>
            <EventTimes>
                <EventTime>
                    <Event>EVENT_TORNADO_STOP</Event>
                    <iEventTime>1</iEventTime> <!-- 1 turn only on normal game speed; becomes 8 turns on eternity GS !! Not acceptable for the tornado event-->
                </EventTime>
            </EventTimes>
            <Button>,Art/Interface/Buttons/Process/Blank.dds,Art/Interface/Buttons/Beyond_the_Sword_Atlas.dds,8,5</Button>
        </EventInfo>
        <EventInfo>
            <Type>EVENT_TORNADO_STOP</Type>
            <Description>TXT_KEY_EVENT_TORNADO_1</Description>
            <PythonCallback>stopTornado</PythonCallback>
            <Button>,Art/Interface/Buttons/Process/Blank.dds,Art/Interface/Buttons/Beyond_the_Sword_Atlas.dds,8,5</Button>
        </EventInfo>

The <bNoScalingByGamespeed> should be nested inside the <EventTime> tag seen in the spoiler above.
I'm sure that these timed follow-up event will be stored in the save file.
The event time is only used if the <AdditionalEvents> does not fire right away, hence the <iEventChance>0</iEventChance>.

You need to be careful using stuff on the dll-python boundary especially if it is rarely used. In many cases it was not checked for efficiency and the coders were paid by the line of code.

Ok, noted; I'll make sure to do some python profiling to see how these functions perform.
They will only be used once each every time a tornado event fire so it shouldn't be a problem.

 

[Dancing Hoskuld]

Ok, noted; I'll make sure to do some python profiling to see how these functions perform.
They will only be used once each every time a tornado event fire so it shouldn't be a problem.

It is not the Python that is the problem in those cases. It is usually the C code in the dll where the inefficiency lies. It just gets blamed on Python When Koshling did that work on the PythonCallBacks he streamlined it by 1000's of times. Unfortunately we were not in a position to use it for more than the Great Farmer at the time and did not find the bugs until later.

 

[Toffer90]

It is not the Python that is the problem in those cases.

I know, but I can still find out the time the dll or the exe uses for these type of functions when it is python that requested them to process it.
Python needs to wait for an answer from the exe or dll before it can move on in the python script.

 

[Septimius]

Tiny bug, "herds" of Caribou don't give you any hunting food resources or subdued animals, all the other sizes of Caribou give food resources and/or subdued animals.

I've got a save game where you can see it for yourself, if useful.

 

[Thunderbrd]

Tiny bug, "herds" of Caribou don't give you any hunting food resources or subdued animals, all the other sizes of Caribou give food resources and/or subdued animals.

I've got a save game where you can see it for yourself, if useful.

Given that size doesn't adjust anything about food rewards yet, that's exceedingly strange. So sure... toss up a save on this so we can see what happens there.

 

[Septimius]

Given that size doesn't adjust anything about food rewards yet, that's exceedingly strange. So sure... toss up a save on this so we can see what happens there.

Sure thing. It's the only unit left that can move. You'll have to boost it's speed to three in order to get the caribou.

 

[Noriad2]

It has to do with a city attack and a lead unit in a looped through group being non existent when it shouldn't have been possible for it to have been somehow removed from the list before the check that finds it missing. I could read how it happens easier from a save in progress. It's possible that a unit is dying at an odd moment in time that some portion of the code isn't expecting. Probably an AI and the slightest differences in how you play the round could throw the random results into a different sequence thus how you get through.

Allright, I went back to an older save, made a few test saves where all units are fortified in different positions, so you only have to press enter to through the turns, and found a save that crashes often. But oddly, it doesn't always crash at the same turn. I started the game and then loaded this save, did this several times, each time just entered through the turns, and it almost always crashes on the second or third turn.

SVN9706 (with /usersettings/customdomadv directory removed, probably not relevant)
I have included the shortdump in the .zip for verification that it has the same crash cause as last time.

No hurry though, I bypassed this crash by regularly saving and loading, and after a while the crash did not happen anymore and I resumed my campaign.

 

[Thunderbrd]

Allright, I went back to an older save, made a few test saves where all units are fortified in different positions, so you only have to press enter to through the turns, and found a save that crashes often. But oddly, it doesn't always crash at the same turn. I started the game and then loaded this save, did this several times, each time just entered through the turns, and it almost always crashes on the second or third turn.

SVN9706 (with /usersettings/customdomadv directory removed, probably not relevant)
I have included the shortdump in the .zip for verification that it has the same crash cause as last time.

No hurry though, I bypassed this crash by regularly saving and loading, and after a while the crash did not happen anymore and I resumed my campaign.

I think I've figured out why this is happening in terms of the underlying disease rather than just the symptoms. I hope to try to tackle that this weekend as it's a major cause of instability. This save should help to show me whether or not the changes have created more stability or I've got a different problem or have simply failed to correct it. Thanks!

 

[Noriad2]

Suboptimal AI behaviour in the first few turns: AI sends its two bands of homo sapiens together towards a city location without escort, while his 8 stone throwers and his clubman escort a lone gatherer in the field. I managed to kill one band of homo sapiens with my clubman before he founded a city with his other band of homo sapiens. SVN9716, deity/nightmare.

 

[Thunderbrd]

Suboptimal AI behaviour in the first few turns: AI sends its two bands of homo sapiens together towards a city location without escort, while his 8 stone throwers and his clubman escort a lone gatherer in the field. I managed to kill one band of homo sapiens with my clubman before he founded a city with his other band of homo sapiens. SVN9716, deity/nightmare.

Added to the tasklist. Thanks for the report!

 

[CiverDan]

Getting a crash upon trying to load save with current SVN 9716. Bug seems to be with hyperlane route added in 9693 Says
" TAG ROUTE_HYPERLANE in module is incorrect. Correct module is pepper2000...civ4route infos...

 

[Thunderbrd]

Getting a crash upon trying to load save with current SVN 9716. Bug seems to be with hyperlane route added in 9693 Says
" TAG ROUTE_HYPERLANE in module is incorrect. Correct module is pepper2000...civ4route infos...

@pepper2000 Let me know if you cannot find any problems wth the xml.