Single Player bugs and crashes v38 plus (SVN) - After the 20th of February 2018

[Thunderbrd]

Does the iAsset value have any bearing on the choice of what units do? eg if a subdued animal has iAsset=1 and the Myth has iAsset=2 is the unit more likely to go and build the Myth?

I'd have to check but generally I'd assume no. iAsset is about point tally and AI evaluation of competitors total iAsset amount if I'm not mistaken. I could be wrong though. But generally speaking, the unit is more likely to go and build the myth if it comes up as something that can be built somewhere. It's pretty much an automatic priority over everything with the AI that's being used. And it barely looks to ensure the building is of any value - if it can build it, generally, it will. Not much is there to say not to. I'm not sure if there's anything to tell it not to.

 

[raxo2222]

Because each culture type (i.e. European) needed a different resource to get the different cultures. (So Belgium needs Chocolate (also NW)). There are more cultures than resources.

It was that way about 4 years ago - if it has changed since then - I do not know.

It may have, as lots of cultures were added as a separate mod-mod. But then they may just duplicate the required resource. Giving you the choice of two or more cultures with that resource.

So it has nothing with historic placement of plants/animals I guess.

 

[Thunderbrd]

So it has nothing with historic placement of plants/animals I guess.

There was some consideration given to that but when Sparth added a lot more cultures, the goal was also to not repeat previous combinations used but also to enable more cultures to be had within the 'what if' realm of a random map environment. Otherwise, if your 'african' culture in the northern tundra doesn't have any usual african resources around, you'll probably never get any cultures.

There's a lot of varying ideals in the design of culture access due to the numerous minds that worked on things. Someday, once the ideas project is underway, it'll get revisited.

 

[Dancing Hoskuld]

I'd have to check but generally I'd assume no. iAsset is about point tally and AI evaluation of competitors total iAsset amount if I'm not mistaken. I could be wrong though. But generally speaking, the unit is more likely to go and build the myth if it comes up as something that can be built somewhere. It's pretty much an automatic priority over everything with the AI that's being used. And it barely looks to ensure the building is of any value - if it can build it, generally, it will. Not much is there to say not to. I'm not sure if there's anything to tell it not to.

I am seeing stacks of subdued animals with a hunter just sitting just inside the borders. This was before the last dll update.

So it has nothing with historic placement of plants/animals I guess.

Animals are spawned by historic region on random maps, but we made Europe too big in our hurry. There is no way to spawn plants by historic placement on a random map.

 

[raxo2222]

I am seeing stacks of subdued animals with a hunter just sitting just inside the borders. This was before the last dll update.

Animals are spawned by historic region on random maps, but we made Europe too big in our hurry. There is no way to spawn plants by historic placement on a random map.

I meant animals as resources not units.

 

[olduser]

Despite fixing some of the main problems we've been seeing, this is still happening at times but not persistently. It seems they spend a round or two on wealth in between some builds and perhaps it is due to caching and buildings remaining not having value. I would think another unit might have been queued though and that's odd. It's worthy of more investigation but all in all I THINK things are working fairly well on this next dll... I think. Still observing. The gatherer decided to make use of itself right away but it's still a little odd that in the previous game it didn't.

One strange thing about the hunter escort - it wasn't calling for a unit with that AI type, just any unit with defensive or offensive capabilities.

EDIT: Also I'm thinking that the processes coming up may be an artifact of multithreading. I'm not sure about that yet but it's suspect.

This post is mostly for @alberts2. Although I can confirm the new dll has brokerage calls (both from cities and units) are functioning, I'm still a bit concerned about a few more minor points. I did learn this go around that units, when called through the brokerage, are supposed to immediately take priority over any building (though may not interrupt current builds and any queued other items.) This is not always ideal because it leads to slow responses at times. And I'm pretty sure that processes can be inappropriately queued and not overridden when they should be.

Anyhow, things should be working better at this point but some deeper study and improvement of that whole structure is in order at some point imo.

Good to hear that you made progress .

The brokerage system naturally can add some delay and some unit requests just have a low priority. The city needs to build other things even meager wealth too at times and can't focus on units only. This issue gets smaller with the number of cities a player has because other cities migh be able answer those requests faster.

 

[raxo2222]

Good to hear that you made progress .

The brokerage system naturally can add some delay and some unit requests just have a low priority. The city needs to build other things even meager wealth too at times and can't focus on units only. This issue gets smaller with the number of cities a player has because other cities migh be able answer those requests faster.

AI shouldn't waste time on gold/research/espionage converters, as some players struggle to keep up with buildings especially when playing with warlike AI on same continent.

Also it seems like AI really dislikes using one shot workers to improve land.

 

[olduser]

as some players struggle to keep up with buildings

That sounds more like a balance problem.

If you can't keep up with buildings in C2C that is somehow by design just compare the number of buildings C2C has with that of vanilla CivIV. The AI gets massive production bonuses on higher difficulties and that can make the AI run out of stuff to build while the human player can't keep up with production.

Also it seems like AI really dislikes using one shot workers to improve land.

It's programmed that way to not waste resources on those one shot workers. That was programmed at a time where the prehistoric era was much shorter and the time until other workers became available was shorter as well.

 

[raxo2222]

That sounds more like a balance problem.

If you can't keep up with buildings in C2C that is somehow by design just compare the number of buildings C2C has with that of vanilla CivIV. The AI gets massive production bonuses on higher difficulties and that can make the AI run out of stuff to build while the human player can't keep up with production.

I'm running autorun on Noble, so AI has same costs as player would have.

It's programmed that way to not waste resources on those one shot workers. That was programmed at a time where the prehistoric era was much shorter and the time until other workers became available was shorter as well.

I guess this should be changed as gatherers are really cheap by middle of prehistoric era.

 

[strategyonly]

2nd time this happened now, brand new game not even 1 turn , tried to go to 1st turn BUT poof CTD??? waited 8 minutes for map then this ,, GEEZ . . .
latest SVN

also looked at the python error:

Traceback (most recent call last):

File "C2C_World", line 4321, in generatePlotTypes

File "C2C_World", line 1237, in GenerateElevationMap

File "C2C_World", line 4792, in FindThresholdFromPercent

IndexError: list index out of range
ERR: Python function generatePlotTypes failed, module C2C_World
Traceback (most recent call last):

File "C2C_World", line 4414, in generateTerrainTypes

AttributeError: TerrainMap instance has no attribute 'terrData'
ERR: Python function generateTerrainTypes failed, module C2C_World
Traceback (most recent call last):

File "C2C_World", line 4491, in addRivers

File "C2C_World", line 4875, in placeRiversInPlot

AttributeError: RiverMap instance has no attribute 'riverMap'
ERR: Python function addRivers failed, module C2C_World
Traceback (most recent call last):

File "C2C_World", line 4543, in addFeatures

AttributeError: ClimateMap instance has no attribute 'TemperatureMap'
ERR: Python function addFeatures failed, module C2C_World

 

[raxo2222]

@Toffer90 something broke in World map script ^
Also game teds to crash after generating map multiple times in row.
It is as if game stores everything in memory when generating map.

 

[Thunderbrd]

I am seeing stacks of subdued animals with a hunter just sitting just inside the borders. This was before the last dll update.

They are joined to and commanded by the hunter. For whatever reason they aren't being released then. That's hard to track down in the code but at least knowing it's happening helps. They're probably stuck healing or some stupid thing like that. In short, the animals aren't in control of their own actions at that point. They're just following the Hunter, who is failing to release them to their own control. I can look into that. Eventually.

gold/research/espionage converters

Those are called 'processes'.

That sounds more like a balance problem.

If you can't keep up with buildings in C2C that is somehow by design just compare the number of buildings C2C has with that of vanilla CivIV. The AI gets massive production bonuses on higher difficulties and that can make the AI run out of stuff to build while the human player can't keep up with production.

I don't think it's just a balance problem. Something else is wrong because on autoplay you can clearly often see the AI select a Process when it has perfectly valid buildings in the list and unit needs and no cause for financial distress. It just switches to a process for a round or two between builds commonly and I have not been able to come up with any kind of theory as to why. A laborious patient debug watch on chooseProduction is in order to further diagnose this issue. The problem is that it then gets very confusing with multithreading so perhaps we'll wait until that's removed - as I said elsewhere, the going theory for me right now is that this is an artifact of multiple threads trying to queue what they find available and once a unit and building are queued, those extra threads only find a process valid so queue that and thus these processes are constantly being thrown into the mix and aren't being overridden later, perhaps due to some bad caching not allowing a clean recalculation of queue priorities. I have not yet tested to see if taking the number of city threads down to 1 in the globals has an effect on this. I suppose that would be a good immediate test of the theory at least. Maybe you can think of another theory?

Also it seems like AI really dislikes using one shot workers to improve land.

It does. I will be reviewing worker AI code to see if I can pull that thorn because it really is a problem for them. I've seen the coding for the reluctance in passing before.

It's programmed that way to not waste resources on those one shot workers. That was programmed at a time where the prehistoric era was much shorter and the time until other workers became available was shorter as well.

I admit that it's debate worthy as to whether they should just go with gusto on gatherers or not. If barbs come in and raze what the workers do too often and the AI is not effective at stopping those incursions then it does become a major setback for them to constantly feed the trolls. I'd like to see if I can reduce the unwillingness a little at least.

I guess this should be changed as gatherers are really cheap by middle of prehistoric era.

It's a valid point that units are allowed to be a bit cheaper now and it might not be as detrimental to care a bit more about the risks of gatherer training as much now.

@SO: thanks for the mini. I can actually use it at the moment. The python log looks pretty straight forward at offering a map script based diagnosis but this should confirm or lead to a way to bolster the code to avoid it.

 

[Thunderbrd]

10041

• Fixed an out of range error in World mapscripts that could happen on maps with so little ocean that only 1 or less plots are valid for the trench terrain.
  • Bug reported by @strategyonly

From the perspective of the code, the crash was a problem in the dll. But it was a problem because there was an assumption that there would always be a defined 'water' 'area'. On the initialization of an 'area', bWater is set with that call. I'm thinking this means that you don't technically have an ocean defined as an ocean if it's not big enough, but might be a little unrelated to the trench terrain in particular.

I'm not sure when or how the initialization of an area is made exactly... is that something the map script calls for specifically at some defined point or is that part of the dll side, reacting to what the script has created and assigning area determinations after the generation of the map details? (If the mapscript doesn't call it, I'd think it would have to be the latter.)

Anyhow, maps should be able to create all sizes of water masses, including none, safely. The code has been hedged against this. It was going to be a problem for any non-water map.

 

[Thunderbrd]

Update: I have figured out a lot of the problem with the workers having issues. It really helps to eliminate the inhibition for gatherers and I've managed to repair the escort AI so that workers are no longer waiting for help that never comes before going forth to do what they really need to do. This. Is. Huge.

There's a few other AI types I want to make sure are healthy and functioning but the breakthroughs are coming quick today. I'm not seeing them waste their time on processes either now that I figured out some things that were causing problems for some brokerage calls. This breakthrough may need to be applied elsewhere so be a little patient with me on getting these improvements on the SVN.

Just wanted to let you know that amazing things are happening here... we're soon to be able to truly fear the AI again I believe.

 

[KaTiON_PT]

Thank you for your help Thunderbrd!

 

[raxo2222]

Update: I have figured out a lot of the problem with the workers having issues. It really helps to eliminate the inhibition for gatherers and I've managed to repair the escort AI so that workers are no longer waiting for help that never comes before going forth to do what they really need to do. This. Is. Huge.

There's a few other AI types I want to make sure are healthy and functioning but the breakthroughs are coming quick today. I'm not seeing them waste their time on processes either now that I figured out some things that were causing problems for some brokerage calls. This breakthrough may need to be applied elsewhere so be a little patient with me on getting these improvements on the SVN.

Just wanted to let you know that amazing things are happening here... we're soon to be able to truly fear the AI again I believe.

Wonder if it gets good enough to make Noriad step back to Deity level
On my AI autoplay Egypt (11 cities and ~3000 points) could easily conquer Australia (3 cities and ~1000 points) but Egypt never conquered Australia.
I think it was because I had time victory enabled not total conquest.

I wonder what results I would get if I had space race or science victory enabled - basically one and same thing, as both of them would be decided in last era in hyperspace, just that in space race you build magic ship to travel to wonderland, and in science victory you build magic portals to wonderland.

 

[Dancing Hoskuld]

I still believe that the Space Race Victory should have been left much as it was in RoM as it lets people play to the mid-Transhuman if they want.

 

[Toffer90]

From the perspective of the code, the crash was a problem in the dll. But it was a problem because there was an assumption that there would always be a defined 'water' 'area'. On the initialization of an 'area', bWater is set with that call. I'm thinking this means that you don't technically have an ocean defined as an ocean if it's not big enough, but might be a little unrelated to the trench terrain in particular.

The python error and the CTD that SO reported were mostly unrelated, but I gather from what you are saying that both were caused by there being too little ocean (None in particular) on the map.
I fixed the python error that was directly tied to trench terrain placement code at least.

I'm not sure when or how the initialization of an area is made exactly... is that something the map script calls for specifically at some defined point or is that part of the dll side, reacting to what the script has created and assigning area determinations after the generation of the map details? (If the mapscript doesn't call it, I'd think it would have to be the latter.)

Spoiler Areas are define in, or some time after, this dll function is processed :

void CvMapGenerator::generatePlotTypes()
{
    int* paiPlotTypes = new int[GC.getMapINLINE().numPlotsINLINE()];

    int iNumPlots = GC.getMapINLINE().numPlotsINLINE();

    std::vector<int> plotTypesOut;
    if (PYTHON_CALL_FUNCTION4(__FUNCTION__, gDLL->getPythonIFace()->getMapScriptModule(), "generatePlotTypes", NULL, &plotTypesOut) && !gDLL->getPythonIFace()->pythonUsingDefaultImpl())
    {
        // Python override
        FAssertMsg((int)plotTypesOut.size() == iNumPlots, "python generatePlotTypes() should return list with length numPlotsINLINE");
        for (int iI = 0; iI < iNumPlots; iI++)
        {
            paiPlotTypes[iI] = plotTypesOut[iI];
        }
    }
    else
    {
        for (int iI = 0; iI < iNumPlots; iI++)
        {
            paiPlotTypes[iI] = PLOT_LAND;
        }
    }

    setPlotTypes(paiPlotTypes);

    SAFE_DELETE_ARRAY(paiPlotTypes);
}

That function pretty much asks the mapscript that is active which plot types (Ocean/Land/Hills/Peaks) are at what plot.

After this the mapscript starts asking the dll about the areas when it does the "addBonuses()" and "assignStartingPlots()"; two functions that the dll instructs the mapscript to run.

Spoiler Areas are define before this dll function is processed :

void CvMapGenerator::addBonuses()

{
    PROFILE("CvMapGenerator::addBonuses");
    gDLL->NiTextOut("Adding Bonuses...");

    if (PYTHON_CALL_FUNCTION(__FUNCTION__, gDLL->getPythonIFace()->getMapScriptModule(), "addBonuses", NULL))
    {
        if (!gDLL->getPythonIFace()->pythonUsingDefaultImpl())
        {
            return; // Python override
        }
    }

    for (int iOrder = 0; iOrder < GC.getNumBonusInfos(); iOrder++)
    {
        for (int iI = 0; iI < GC.getNumBonusInfos(); iI++)
        {
            gDLL->callUpdater();
            if (GC.getBonusInfo((BonusTypes)iI).getPlacementOrder() == iOrder)
            {
                PYTHON_ACCESS_LOCK_SCOPE

                CyArgsList argsList;
                argsList.add(iI);
                if (!PYTHON_CALL_FUNCTION(__FUNCTION__, gDLL->getPythonIFace()->getMapScriptModule(), "addBonusType", argsList.makeFunctionArgs()) || gDLL->getPythonIFace()->pythonUsingDefaultImpl())
                {
                    if (GC.getBonusInfo((BonusTypes)iI).isOneArea())
                    {
                        addUniqueBonusType((BonusTypes)iI);
                    }
                    else
                    {
                        addNonUniqueBonusType((BonusTypes)iI);
                    }
                }
            }
        }
    }
}

Edit:

Spoiler I guess this is the function that defines the Areas in the dll :

void CvMapGenerator::setPlotTypes(const int* paiPlotTypes)
{
    CvPlot* pLoopPlot;
    int iNumPlots;

    iNumPlots = GC.getMapINLINE().numPlotsINLINE();

    for (int iI = 0; iI < iNumPlots; iI++)
    {
        gDLL->callUpdater();
        GC.getMapINLINE().plotByIndexINLINE(iI)->setPlotType(((PlotTypes)(paiPlotTypes[iI])), false, false);
    }

    GC.getMapINLINE().recalculateAreas();

    for (int iI = 0; iI < iNumPlots; iI++)
    {
        gDLL->callUpdater();
        pLoopPlot = GC.getMapINLINE().plotByIndexINLINE(iI);

        if (pLoopPlot->isWater())
        {
            if (pLoopPlot->isAdjacentToLand())
            {
                pLoopPlot->setTerrainType(((TerrainTypes)(GC.getDefineINT("SHALLOW_WATER_TERRAIN"))), false, false);
            }
            else
            {
                pLoopPlot->setTerrainType(((TerrainTypes)(GC.getDefineINT("DEEP_WATER_TERRAIN"))), false, false);
            }
        }
    }
}

Specifically the line GC.getMapINLINE().recalculateAreas().

Spoiler That leads to this dll function :

void CvMap::recalculateAreas()
{
    PROFILE("CvMap::recalculateAreas");

    int iI;

    for (iI = 0; iI < numPlotsINLINE(); iI++)
    {
        plotByIndexINLINE(iI)->setArea(FFreeList::INVALID_INDEX);
    }

    m_areas.removeAll();

    calculateAreas();
}

 

[T C]

Since Thunderbird asked for some feedback, I ran two test games this afternoon with the latest SVN, although I forgot to turn logging on for the first one (D'oh!). The AI is building cities again. Improvements are so-so. Hunting is still non-existent I'm afraid. In the first, non-logged, game, I noticed that the AI was quite aggressive and militarily effective. So things are definitely going in the right direction.

Here are about 700 turns worth of AI logs from the second game:

 

[Thunderbrd]

Hunting is also fixed and improved on next dll already.