Single Player bugs and crashes v38 plus (SVN) - After the 20th of February 2018

olduser · Jun 30, 2019

Thunderbrd said:
@alberts2, @Anq, @AIAndy

Help!

This appears to get stuck at
void CvCity::completeOrderProcessing(void)

looping through creating the same unit over and over. I don't know enough about the storage method being employed to setup this loop:
for( std::vector<OrderData>::iterator itr = m_inProcessOrders.begin(); itr != m_inProcessOrders.end(); ++itr )
since m_inProcessOrders is not a basic integer loop, I'm not sure what ++itr is actually adding to. I know it's basically saying 'go to the next process item in the list now that we've completed the previous one' but I can't seem to figure out how this list is enumerated to begin with or how to find that information out. It almost seems like it keeps looping the same order over and over and maybe it's doing that but that's because the problem is further upstream and there was an inifinite number of See Invisible Dogs that were queued here - possibly to the point that the game hit the unit limit?

I'm not sure - this is really the deep end of the pool for me. I had hoped this would be an easier fix.

A few years ago it was quite common for the AI with it's huge bonuses to train a few hundred units in a single city i don't know if that has changed. Thats even more likely to happen if there's a production overflow from the previous turn.

There's also a ancient production bug in CivIV that is also present in other mods as well. Something gives a city an gigantic amount of producton this could be a integer overflow or a uninitiated variable somewhere. Because it's very rare it's nearly impossible to catch it happening and debug.

Toffer90 · Jun 30, 2019

Toffer90 said:
All those "cannot open source files errors" at the top are for files found only in

CivIV\Beyond the Sword\CvGameCoreDLL\Python24
CivIV\Beyond the Sword\CvGameCoreDLL\Boost-1.32.0

Maybe we should include those source files from BtS to our C2C SVN?

Is there a good reason why we don't have these source files on the SVN along with the rest of it?

olduser · Jun 30, 2019

Toffer90 said:
Is there a good reason why we don't have these source files on the SVN along with the rest of it?

Because they are included in every CivIV installation and you kinda need to have CivIV installed to do any kind of modding.

Yudishtira · Jun 30, 2019

The "Road" route type still gives way way too much move bonus. I counted at least 9 moves before they even cost 0.1 move points. That's more than 4 times the movement using maglev, up to maybe 30 times if your unit has 7 moves (gunship).

It was certainly working correctly back when it was the best route available. I can only think some later tech causes this effect.

Thunderbrd · Jun 30, 2019

Anq said:
You can picture the vector iterator as the pointer to its array. An array is a contiguous block of memory, and exact space of memory it takes is: Number of elements, times the Size of the individual element, such as char (1 byte), int (4 bytes), and so on. The pointer to an array references the first element, and as you increment the pointer, it then references the next element of that array, whose address offset will depend on the size of the individual element.
There is no builtin way (of C/++) to stop iterating at the end of array if we use pointer. Therefore the STL vector class provides the end pointer to help you iterate through its contents. The three for loop statements are:

Give me the begin-of-array pointer to work with this vector.

Until our pointer compares the same as the end-of-array pointer,

Increment it. (points to the next element on that array)

This is the standard way to iterate through vectors. Although the iterators are confusing at first sight, you'll praise this design as it adheres to the pointer arithmetic (this term describes the ability to increment, decrement, and compare two pointers as we work with arrays).

Secondly, when you dereference your pointer, written as *ptr, it lends out the value of the element at that address, whether as an L-value or R-value; that is, you can both assign the element some value if you use *ptr on the left side of an assignment operator (equal sign, =) and otherwise read the value if you use *ptr on the right side.

Thirdly, the dereferencing operator * takes lower precedence than member access operators, which are the dot (.) and the arrow (->). Thus you have to write (*ptr) when you intend to call some member function of that referenced element (can be a class object/instance, or even a pointer to them, if they are not stored in-line on the vector but in somewhere else), or else the member-access operators are first resolved, then the dereference asterisk at last, which only makes sense, of course, when the innermost member accessed is a pointer. Cf. https://en.cppreference.com/w/cpp/language/operator_precedence

PS. Writing (*ptr).something is the same as writing ptr->something. The arrow operator existed for this exact purpose. It is the member-access operator for a pointer variable. Thus the second line in the block:
(*itr).eOrderType = NO_ORDER;
can be easily rewritten as
itr->eOrderType = NO_ORDER;

All good info and although I don't know what dereference means exactly, I THINK I do actually understand most of this. What I find confusing and possibly could've been useful here, is if you have a size of say, 19 elements in the array, how would you get the element count you're actually on as you're looping through each element in the code? Would it just be necessary to setup an integer, such as int iI and add iI++; to the code in the loop block so you can see the count of which element you're actually on at a given moment during a debug?

MattCA said:
If you open up your .vcxproj file with a txt editor there are a couple paths to that stuff.
It's set wrong on my pc by default.
dunno if that'll fix.

Interesting - I thought this was automatically generated somehow. You're saying that I have to directly manipulate the project file?

alberts2 said:
A few years ago it was quite common for the AI with it's huge bonuses to train a few hundred units in a single city i don't know if that has changed. Thats even more likely to happen if there's a production overflow from the previous turn.

There's also a ancient production bug in CivIV that is also present in other mods as well. Something gives a city an gigantic amount of producton this could be a integer overflow or a uninitiated variable somewhere. Because it's very rare it's nearly impossible to catch it happening and debug.

Hmm... that's true. I remember that. Here's the really odd thing - I couldn't find a huge stack of units that had been generated the round after these hundreds of units had been made and the logging didn't actually show that they had been quite right, but as I was watching the code process at various breakpoints, clearly each iteration on this had a new unit ID being initiated. So what happened to all those units? No city looked to have the production capacity to have generated 900+ units in one round or even 300+, and 2 cities at least would have done that in the first round on this save and one on the second. That's far beyond the frequency we have ever seen before on the sudden production surge bug, which might happen once in a game. There's probably a lot to research and look into here and yesterday I did not have a lot of time and patience and probably shouldn't have been looking at it in the first place but was hoping this would be a quick and simple Unit AI infinite loop fix, hoping I could get it done and prep for release thereafter. Apparently, I'm very wrong and something truly messed up is taking place in the wiring here.

I can see from looking at the game itself that the AI isn't properly staffed in its cities with see invisible units, so they'd be wanting more, but not THAT many more. So the list of problems here includes:
1)How and why did it get queued up with so many to begin with?
2)What happened to these hundreds of units afterwards? I could not find them so I'll have to start looking at some advanced tracking into what city is actually getting stuck training all these units
3)How did the city get the production to build so many units?
and probably more as I start finding answers. Ugh... this could mean we are weeks away from release here since I have only a few hours a week to work on this right now. And priority 1 is trying to figure out what to do about the compiler errors.

Yudishtira said:
The "Road" route type still gives way way too much move bonus. I counted at least 9 moves before they even cost 0.1 move points. That's more than 4 times the movement using maglev, up to maybe 30 times if your unit has 7 moves (gunship).

It was certainly working correctly back when it was the best route available. I can only think some later tech causes this effect.

Ugh - then there's this. Can you post a save that shows this taking place please. Since you mentioned that the road works normally during the era its usually used, I don't think I'd have ever seen such a bug playing out.

Thunderbrd · Jun 30, 2019

Anq said:
@Thunderbrd Problems I see in VS2017 vsxproj file:

Remove redundant target Final_Release2

WindowsTargetPlatformVersion not necessary

So:

Code:

  <PropertyGroup Label="Globals">
    <ProjectName>C2C</ProjectName>
    <ProjectGuid>{37D4A3BE-6EDB-4A5F-A42A-B82ECF9D34FC}</ProjectGuid>
    <Keyword>MakeFileProj</Keyword>
    <WindowsTargetPlatformVersion>10.0</WindowsTargetPlatformVersion>
  </PropertyGroup>

Just remove this line? <WindowsTargetPlatformVersion>10.0</WindowsTargetPlatformVersion>

And remove all of this:

Code:

    <ProjectConfiguration Include="Final_Release2|Win32">
      <Configuration>Final_Release2</Configuration>
      <Platform>Win32</Platform>
    </ProjectConfiguration>

Looks like theres a few more blocks referencing Final_Release2 as well...

As for paths:
<NMakeIncludeSearchPath>E:\Civ4\CvGameCoreDLL\Boost-1.32.0\include;$(NMakeIncludeSearchPath)</NMakeIncludeSearchPath>
Doesn't look right at all since I have no E: drive. Is this part of the problem?

<IncludePath>D:\Program Files %28x86%29\Civilization IV Complete\Civ4\Beyond the Sword\CvGameCoreDLL\Python24\include;D:\Program Files %28x86%29\Civilization IV Complete\Civ4\Beyond the Sword\CvGameCoreDLL\Boost-1.32.0\include;.\xerces-c-3.1.1-x86-windows-vc-7.1\src;$(IncludePath)</IncludePath>

<IncludePath>D:\Program Files %28x86%29\Civilization IV Complete\Civ4\Beyond the Sword\CvGameCoreDLL\Python24\include;D:\Program Files %28x86%29\Civilization IV Complete\Civ4\Beyond the Sword\CvGameCoreDLL\Boost-1.32.0\include;.\xerces-c-3.1.1-x86-windows-vc-7.1\src;$(IncludePath)</IncludePath>

<IncludePath>D:\Program Files %28x86%29\Civilization IV Complete\Civ4\Beyond the Sword\CvGameCoreDLL\Python24\include;D:\Program Files %28x86%29\Civilization IV Complete\Civ4\Beyond the Sword\CvGameCoreDLL\Boost-1.32.0\include;.\xerces-c-3.1.1-x86-windows-vc-7.1\src;$(IncludePath)</IncludePath>

I have a D drive but nothing civ or programming related is on it... Do I need to re-target these?

olduser · Jun 30, 2019

Yudishtira said:
The "Road" route type still gives way way too much move bonus. I counted at least 9 moves before they even cost 0.1 move points. That's more than 4 times the movement using maglev, up to maybe 30 times if your unit has 7 moves (gunship).

It was certainly working correctly back when it was the best route available. I can only think some later tech causes this effect.

There's at least one random event that reduces route movement costs. Somehow those events reduce the movement costs far too much down to 1/120 :move:

.

Either there's some fundamental flaw in the event system that makes it possible for that kind of event to happen multiple times or they just happen multiple Times? The same was the case with vulcanos some versions ago.

Thunderbrd · Jun 30, 2019

<IncludePath>D:\Program Files %28x86%29\Civilization IV Complete\Civ4\Beyond the Sword\CvGameCoreDLL\Python24\include;D:\Program Files %28x86%29\Civilization IV Complete\Civ4\Beyond the Sword\CvGameCoreDLL\Boost-1.32.0\include;.\xerces-c-3.1.1-x86-windows-vc-7.1\src;$(IncludePath)</IncludePath>

Ok, so I tried to correct this path since clearly it is very very wrong - My file path is: C:\Program Files (x86)\Firaxis Games\Sid Meier's Civilization 4\Beyond the Sword\CvGameCoreDLL\Boost-1.32.0
and
C:\Program Files (x86)\Firaxis Games\Sid Meier's Civilization 4\Beyond the Sword\CvGameCoreDLL\Python24
But by changing it, I create far more error messages... oy vey.

Yes, I noticed that Program Files %28x86%29 should be the way to express (x86) in this statement, but trying it either way I still get the same 700+ errors when I try to fix the path.

raxo2222 · Jun 30, 2019

Yudishtira said:
The "Road" route type still gives way way too much move bonus. I counted at least 9 moves before they even cost 0.1 move points. That's more than 4 times the movement using maglev, up to maybe 30 times if your unit has 7 moves (gunship).

It was certainly working correctly back when it was the best route available. I can only think some later tech causes this effect.

Here are movement bonuses (routes sorted by tech level):
As flat movement and as movement reduction.
Trail: Movement cost of 1, all units can move by 1 tile.
Path: Movement cost of 0.83, all units can move by 1 tile.
Road: Movement cost of 0.50, all units can move by 2 tiles. Cost halved(?) if Motorized Transportation is known.
Paved Road: Movement cost of 0.33, all units can move by 3 tiles.
Railroad: Movement cost of 0.25, all units can move by 10 tiles.
Highway: Movement cost of 0.25, all units can move by 4 tiles. Cost is *0.25 twice(?): If Manufacturing and Skyroads is known.
Electric Railroad: Movement cost of 0.25, all units can move by 12 tiles.
Maglev: Movement cost of 0.17, all units can move by 20 tiles.
Vactrain: Movement cost of 0.15, all units can move by 20 tiles.
Gravity Train: Movement cost of 0.13, all units can move by 20 tiles.
Jumplane: Movement cost of 0.13, all units can move by 40 tiles.

Toffer90 · Jun 30, 2019

alberts2 said:
Because they are included in every CivIV installation and you kinda need to have CivIV installed to do any kind of modding.

Not entirely correct as the steam install of BtS does not provide the source files by default. One needs to activate the beta version of the game to get the source files with steam.

Thunderbrd · Jun 30, 2019

Toffer90 said:
Not entirely correct as the steam install of BtS does not provide the source files by default. One needs to activate the beta version of the game to get the source files with steam.

They aren't source files. They are in Beyond the Sword->CvGameCoreDLL-> Are you saying that this path doesn't exist in the non-beta steam version? I'm not sure how we COULD include them in the mod download since the file path is outside of Caveman2Cosmos. We could provide a separate download I guess.

Toffer90 · Jun 30, 2019

Thunderbrd said:
They aren't source files. They are in Beyond the Sword->CvGameCoreDLL-> Are you saying that this path doesn't exist in the non-beta steam version? I'm not sure how we COULD include them in the mod download since the file path is outside of Caveman2Cosmos. We could provide a separate download I guess.

The CvGameCoreDLL folder is the source files for the vanilla dll, and yes, that folder does not exist in steam BtS by default.

We have our own source files for our dll in the SVN, but when we compile our dll we have to point to those vanilla source files in CvGameCoreDLL because our source files depend on source files we haven't included in the SVN.

That's at least my limited understanding of it, could be wrong about it.
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
If we had them on the svn along with the rest of our source files then this line:
<IncludePath>

D:\Program Files %28x86%29\Civilization IV Complete\Civ4\Beyond the Sword\CvGameCoreDLL\Python24\include;
D:\Program Files %28x86%29\Civilization IV Complete\Civ4\Beyond the Sword\CvGameCoreDLL\Boost-1.32.0\include;
.\xerces-c-3.1.1-x86-windows-vc-7.1\src;
$(IncludePath)

</IncludePath>

could become:
<IncludePath>

.\Python24\include;
.\Boost-1.32.0\include;
.\xerces-c-3.1.1-x86-windows-vc-7.1\src;
$(IncludePath)

</IncludePath>

Which would be much easier for every dll modder who may have their BtS install in very different folder paths.
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬

Thunderbrd said:
<IncludePath>D:\Program Files %28x86%29\Civilization IV Complete\Civ4\Beyond the Sword\CvGameCoreDLL\Python24\include;D:\Program Files %28x86%29\Civilization IV Complete\Civ4\Beyond the Sword\CvGameCoreDLL\Boost-1.32.0\include;.\xerces-c-3.1.1-x86-windows-vc-7.1\src;$(IncludePath)</IncludePath>

Ok, so I tried to correct this path since clearly it is very very wrong - My file path is: C:\Program Files (x86)\Firaxis Games\Sid Meier's Civilization 4\Beyond the Sword\CvGameCoreDLL\Boost-1.32.0
and
C:\Program Files (x86)\Firaxis Games\Sid Meier's Civilization 4\Beyond the Sword\CvGameCoreDLL\Python24
But by changing it, I create far more error messages... oy vey.

Try changing it to:
<IncludePath>

..\..\..\CvGameCoreDLL\Python24\include;
..\..\..\CvGameCoreDLL\Boost-1.32.0\include;
.\xerces-c-3.1.1-x86-windows-vc-7.1\src;
$(IncludePath)

</IncludePath>

Yudishtira · Jun 30, 2019

Thunderbrd said:
Ugh - then there's this. Can you post a save that shows this taking place please. Since you mentioned that the road works normally during the era its usually used, I don't think I'd have ever seen such a bug playing out.

There are a few Road tiles in Ottoman territory, the green bit on the north coast of the main continent - between Maya and Inca. I have a gunship and a hunter somewhere near there.

raxo2222 · Jun 30, 2019

Yudishtira said:
There are a few Road tiles in Ottoman territory, the green bit on the north coast of the main continent - between Maya and Inca. I have a gunship and a hunter somewhere near there.

Route movement cost is negative... looks like some event or wonder broke it.

Thunderbrd · Jun 30, 2019

Toffer90 said:
The CvGameCoreDLL folder is the source files for the vanilla dll, and yes, that folder does not exist in steam BtS by default.

We have our own source files for our dll in the SVN, but when we compile our dll we have to point to those vanilla source files in CvGameCoreDLL because our source files depend on source files we haven't included in the SVN.

That's at least my limited understanding of it, could be wrong about it.
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
If we had them on the svn along with the rest of our source files then this line:
<IncludePath>

D:\Program Files %28x86%29\Civilization IV Complete\Civ4\Beyond the Sword\CvGameCoreDLL\Python24\include;
D:\Program Files %28x86%29\Civilization IV Complete\Civ4\Beyond the Sword\CvGameCoreDLL\Boost-1.32.0\include;
.\xerces-c-3.1.1-x86-windows-vc-7.1\src;
$(IncludePath)
</IncludePath>

could become:
<IncludePath>

.\Python24\include;
.\Boost-1.32.0\include;
.\xerces-c-3.1.1-x86-windows-vc-7.1\src;
$(IncludePath)
</IncludePath>

Which would be much easier for every dll modder who may have their BtS install in very different folder paths.
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬

Try changing it to:
<IncludePath>

..\..\..\CvGameCoreDLL\Python24\include;
..\..\..\CvGameCoreDLL\Boost-1.32.0\include;
.\xerces-c-3.1.1-x86-windows-vc-7.1\src;
$(IncludePath)
</IncludePath>

Thank you - I'll give this a try soon as I can.

Anq · Jun 30, 2019

Toffer90 said:
Try changing it to:
<IncludePath>

..\..\..\CvGameCoreDLL\Python24\include;
..\..\..\CvGameCoreDLL\Boost-1.32.0\include;
.\xerces-c-3.1.1-x86-windows-vc-7.1\src;
$(IncludePath)
</IncludePath>

I concur with using relative paths to the headers.

Thunderbrd said:
I do have to say that I don't feel comfortable packaging a version release when my compiler is showing all these errors, most of which are probably due to the recent compiler upgrade:

These are static analyzer / code assistant (aka IntelliSense) outputs. You'll build fine without seeing any of these errors. They are not really critical but to help you analyze code symbols.

Thunderbrd said:
All good info and although I don't know what dereference means exactly, I THINK I do actually understand most of this. What I find confusing and possibly could've been useful here, is if you have a size of say, 19 elements in the array, how would you get the element count you're actually on as you're looping through each element in the code? Would it just be necessary to setup an integer, such as int iI and add iI++; to the code in the loop block so you can see the count of which element you're actually on at a given moment during a debug?

Dereference comes as the opposite of "referencing" a value by its pointer. The other term for dereference is indirection, as in: you access the value of some variables directly and some indirectly, through pointers.

If you ever need the current element offset, you just subtract the begin pointer from the current pointer, like ( iter - vector.begin() ). This is the beauty of pointer arithmetic.
I suggest you read this lecture on pointer arithmetic; at the end, pointer subtraction is discussed: https://www.cs.swarthmore.edu/~richardw/classes/cs31/s18/offsite/pointer.html
Don't let yourself panic on how you would debug it after you switch to the pointer way. It naturally lends out a way to do debugging, once you've become familiar with this coding trick.

Thunderbrd · Jun 30, 2019

Toffer90 said:
Try changing it to:
<IncludePath>
..\..\..\CvGameCoreDLL\Python24\include;
..\..\..\CvGameCoreDLL\Boost-1.32.0\include;
.\xerces-c-3.1.1-x86-windows-vc-7.1\src;
$(IncludePath)</IncludePath>

Doing this I get some 2000 errors, these are the first, the same ones as when I have some 700+

Spoiler :

When I have errors like these, the intellisense seems to stop working, and I really rely on that thing...

Toffer90 said:
If we had them on the svn along with the rest of our source files then this line:
<IncludePath>
D:\Program Files %28x86%29\Civilization IV Complete\Civ4\Beyond the Sword\CvGameCoreDLL\Python24\include;
D:\Program Files %28x86%29\Civilization IV Complete\Civ4\Beyond the Sword\CvGameCoreDLL\Boost-1.32.0\include;
.\xerces-c-3.1.1-x86-windows-vc-7.1\src;
$(IncludePath)</IncludePath>

could become:
<IncludePath>
.\Python24\include;
.\Boost-1.32.0\include;
.\xerces-c-3.1.1-x86-windows-vc-7.1\src;
$(IncludePath)</IncludePath>

Which would be much easier for every dll modder who may have their BtS install in very different folder paths.

I may have to try this.

Anq said:
https://www.cs.swarthmore.edu/~richardw/classes/cs31/s18/offsite/pointer.html

Well, I tried to read this. I manage to understand a little of it, like maybe 10% of it or so. I didn't learn any programming past pointers through documentation and help files. Once it started trying to explain pointers, it just melts my brain and I can't seem to really get it. I feel like I'm trying to juggle too many thoughts at once trying to grasp it. But when I read the code, the processing of it makes sense usually. I get what you're saying about

Anq said:
If you ever need the current element offset, you just subtract the begin pointer from the current pointer, like ( iter - vector.begin() ). This is the beauty of pointer arithmetic.

This makes sense so thank you. I think what I find frustrating is that you can't see this unless it's added to the code - just looking at that loop, the intellisense help information doesn't just give the ID# of the iteration anywhere, which is really inconvenient when you're not really sure if it's an infinite loop or just way to large a number of iterations. Upon deeper investigation, finding the 'size' statement on this loop helped, and finding that at least the thing was being counted in hexadecimal helped because by doing that I was able to find that yes, with every iteration, the amount WAS increasing, indicating it wasn't just the same iteration over and over and since the count wasn't increasing with each iteration, it wasn't infinitely adding to itself. Still a lot of questions remain in this sticky point as stated above:

How and why did it get queued up with so many to begin with?
What happened to these hundreds of units afterwards? I could not find them so I'll have to start looking at some advanced tracking into what city is actually getting stuck training all these units
How did the city get the production to build so many units?
and probably more as I start finding answers to those questions.

It's good to know that these errors are not going to cause a problem in the dll itself once compiled, but there do seem to be some strange things a few of them are pointing at that may need correcting.

olduser · Jun 30, 2019

Toffer90 said:
Not entirely correct as the steam install of BtS does not provide the source files by default. One needs to activate the beta version of the game to get the source files with steam.

The Steam DRM doesn't allow it to run the game with a debugger attached. That makes that version useless for dll modding anyways.

This is also true for beta version which is the more compatible version and includes the sources as well.

olduser · Jun 30, 2019

Thunderbrd said:
Doing this I get some 2000 errors, these are the first, the same ones as when I have some 700+

Remove the path to the boost includes that should fix intellisense. The Intellisense doesn't like old Version of boost which we have to use.

Anq · Jun 30, 2019

Just reintalled Visual Studio again to check this.
Those errors pop out, same as you posted here.
It has failed digesting the boost library, therefore it is seeing every ordinary class as a boost:: python one, such as boost:: python:: CvPlot. *facepalm*
IntelliSense is a piece of --- when you add the boost library and this is why I dumped it in preference of another software.
I guess this is why nobody was able to answer me why boost::thread related classes are dllexport'ed even though BOOST_THREAD_BUILD_DLL is never defined in our project but BOOST_THREAD_USE_LIB is.

Single Player bugs and crashes v38 plus (SVN) - After the 20th of February 2018

Emperor

C2C Modder

Emperor

Spiritual/Creative

C2C War Dog

C2C War Dog

Emperor

C2C War Dog

Time Traveller

C2C Modder

C2C War Dog

C2C Modder

Spiritual/Creative

Attachments

Time Traveller

C2C War Dog

Prince

C2C War Dog

Emperor

Emperor

Prince

Similar threads